• Intermediate
  • Text, code and quiz

Neural Networks from Scratch

Build a neuron, the ReLU and sigmoid activations, a loss function and gradient descent in plain Python, and watch a network learn.

Level
Intermediate
Study time
1h 3m
Chapters
2
Lessons
5
Quizzes
1

Created by ReLU Learning Team
Courses by the ReLU Learning team · Last updated October 2026

About this course

Deep learning frameworks hide the mechanics. Here you write them yourself: a neuron as a weighted sum, activations, a loss, and the gradient descent loop that every neural network is trained with.

No libraries and no hand-waving. Every step is code you run in the lesson, small enough to understand line by line.

What you will be able to do

  • Compute the output of a neuron by hand and in code
  • Explain what ReLU and sigmoid do and why networks need them
  • Minimise a loss with gradient descent
  • Train a neuron to learn a logical function

What you need first

  • Comfortable Python
  • Machine Learning Foundations, or equivalent

Course content

2 chapters · 5 lessons · 1 quiz

1. One neuron

2 lessons · 22m
  • TextA neuron is a weighted sumFree preview10m
  • TextActivation functions, ReLU included12m

2. Learning

3 lessons · 41m
  • TextLoss and gradient descent15m
  • CodeTraining a neuron on AND20m
  • QuizCheck your understanding6m

About the instructor

ReLU Learning TeamCourses by the ReLU Learning team
  • 4 courses
  • 1 learner

Hands-on courses in Python, machine learning and AI, written so that every idea is something you can run.