I have written a lot about reasoning models here over the last two years on Ahead of AI, including a recent article on mechanisms to control reasoning effort levels in contemporary LLMs. Reasoning models are still as relevant as ever. In fact, almost all current-generation LLMs today are reasoning models or offer a reasoning mode.
For those who are interested in how reasoning models work, including inference scaling, reinforcement learning, and distillation, I wrote my recent Build Reasoning Model (From Scratch), which implements these from the ground up in PyTorch, which, in my experience, is still the best way to understand and learn how things really work.
Now, in the last few months, I also sat down and recorded accompanying lectures I wanted to share with you. So, below, you will find eight reasoning model-from-scratch coding videos (11 hours of content).
We start with a pre-trained LLM and implement methods to evaluate and improve it in Python and PyTorch. So, if you enjoyed my Coding LLMs from the Ground Up course last year, this is essentially the sequel (but you can also watch it without the prior course as a prerequisite).
I’m currently taking a short break and will be back soon with more exciting LLM research roundups next month. In the meantime, I hope you enjoy coding along!
How LLMs, reasoning models, and agents relate and fit together
As a small side note, and since I often get asked how reasoning models relate to today’s AI agents, here’s a short section on this before getting started with the videos.
A reasoning model is essentially an upgrade from a conventional LLM, that is, adding more post-training via reinforcement learning with verifiable rewards (and optionally inference scaling techniques) to make it more capable of solving more challenging problems. Classically, reasoning models are developed for math and code, but the capabilities can generalize to other problem domains as well.
A reasoning model then functions as the engine inside an agent harness, where the agent harness is a software or tooling framework around the LLM/reasoning model to tackle even more challenging problems (bigger coding tasks, travel booking, and all other sorts of personal assistant tasks). The task of the harness is sophisticated context management and giving the LLM access to tools.
Figure 1: An engine analogy for conventional LLMs, reasoning models, and agent harnesses. Figure from my Components of a Coding Agent article.
In this analogy depicted in the figure above, the model is the engine, and the agent harness adds the machinery that lets us use it for various things (as an upgrade from a simple chatbot interface). An agent can use either a conventional LLM or a reasoning model, but reasoning models are the more typical choice as they are more capable. (As mentioned above, most current-gen LLMs are reasoning models with variable reasoning effort settings.)
So, the bottom line is that reasoning models, as the “engine,” remain an important part of understanding modern LLM agents.
The resources accompanying these topics shown above are:
The Build a Large Language Model (From Scratch) book and videos
The Build A Reasoning Model (From Scratch) book (and videos below)
The Components of a Coding Agent article on Substack
The videos below focus on developing the reasoning model that can power such a system.
Scope and prerequisites
The reasoning-from-scratch videos focus on a small pre-trained Qwen3 model, where we implement the reasoning methods ourselves in Python and PyTorch. Starting with a pre-trained model lets us focus on the interesting part: evaluation, inference-time scaling, reinforcement learning, and distillation.
Regarding the prerequisites, I recommend being comfortable with Python and PyTorch. I designed the reasoning materials as standalone content, so the previous Build A Large Language Model (From Scratch) content is recommended but not strictly required.
Compute-resource-wise, you can work through the early text-generation and evaluation examples on a CPU. For the later experiments, especially training, a GPU makes the work much more practical. The first video covers the setup and some more recommendations; the repository also includes hardware and environment recommendations.
Figure 2: An overview of the topics being covered.
The companion book
These videos accompany my book, Build a Reasoning Model (From Scratch). If you like to read through the explanation in more detail and at your own pace, work through exercises, and so on, I recommend the book as a companion here.
For the most in-depth learning experience, my suggestion is to read a chapter, work through its code alongside the corresponding video, and then try the exercises.
The book is available from Manning and Amazon, and the print edition is in color!
Figure 3: A random excerpt from the Build a Reasoning Model (From Scratch) print book.
Thank you to everyone who has picked up a copy or recommended it to a friend. Your support helps me keep creating these materials.
The videos
As a last, quick side note before getting to the main videos below: For a shorter, more conceptual overview, I also recommend my AI Reasoning Models course on LinkedIn Learning (it should be free for 24h when accessing it via the link there, which shouldn’t then require a LinkedIn Premium subscription).
The course is about 90 minutes and explains how reasoning models work, covering the main ideas behind inference-time scaling, reinforcement learning, and distillation. It’s a good optional preview if you’d like the main ideas without working through the full implementations below.
To get to the main part of this article, here are the eight main lecture videos.
1. Motivation and code setup
The first video starts with an explanation of the relationship between conventional LLMs, reasoning models, and agents. Next, the main topic of the video is setting up the coding environment. This includes installing the Python and PyTorch dependencies (with uv), among others. I also discuss hardware choices and common setup issues.
Topics covered
The relationship between conventional LLMs, reasoning models, and agent harnesses
On the benefits of learning through code implementations
The scope of this video series
Installing Python dependencies
Choosing hardware and checking CUDA or Apple Silicon support
Using a remote GPU with VS Code and troubleshooting the environment
Watch video 1 · Setup instructions
2. Loading a base model, text generation, and KV caching
Before adding reasoning methods in the upcoming videos, we need a working base model. Instead of training it from scratch, since that would be a whole other book (see Build A Large Language Model (From Scratch)), we start with a pre-trained model.
Here, we load the pre-trained weights of a Qwen3 0.6B model. Then, we implement the text-generation process via a streaming text generator and add KV caching and model compilation to make text generation more efficient.
These implementations give us the baseline and base components we will evaluate and improve in the following videos.
Topics covered
Encoding text as tokens and decoding it back into text
Downloading Qwen3 weights and loading the model on the selected compute device
Going over tensor shapes, logits, and next-token predictions
Implementing greedy decoding and streaming text generation
Handling end-of-sequence tokens and measuring generation speed
Adding a KV cache to reuse attention keys and values
Combining KV caching with
torch.compileand comparing hardware performance
Watch video 2 · Chapter 2 code
3. Building a verifier for evaluation and reinforcement learning
In this lecture, we build a math-answer verifier step by step, from answer extraction to normalization and checking mathematical equivalence. We then use it to evaluate the model on MATH-500.
We use this verifier to evaluate the base model’s performance on the MATH-500 evaluation dataset.
However, the verifier also has a second job later in the series: providing the reward signal for reinforcement learning with verifiable rewards (RLVR).
Topics covered
Intro to verifiers and how they fit into LLM evaluation methods (other methods are covered in the book’s appendix)
Reusing answer verification as a reward signal for RLVR (more on that in chapter 6)
The verifier flow, including extracting final answers, normalizing answers and checking mathematical equivalence
Building an answer grader and an evaluation loop for MATH-500
Comparing base and reasoning models on MATH-500
Watch video 3 · Chapter 3 code
4. Inference-time scaling: sampling and self-consistency
Now we explore ways to improve answers while keeping the model weights fixed. But before implementing specific inference-scaling techniques, we first implement methods to generate diverse answers from the same prompt (this is standard practice in proprietary LLM chatbots, such as ChatGPT).
Concretely, we implement temperature scaling, multinomial sampling, and top-p filtering to generate diverse responses. Then we combine multiple responses through self-consistency, using majority voting over their final answers.
We also examine the tradeoff between answer accuracy and the extra computation needed to generate more candidates.
Topics covered
Comparing how additional compute benefits model performance during training and during inference
A gentle intro to inference-scaling through chain-of-thought prompts
Implementing temperature scaling and multinomial sampling to generate diverse answers
Adding top-p filtering to control the quality of the diverse answers
Combining candidate answers through self-consistency, one of the most popular inference scaling techniques
Evaluating MATH-500 accuracy against the cost of additional inference compute
Watch video 4 · Chapter 4 code
5. Inference-time scaling: scoring and self-refinement
Building on the previous video, we implement scoring functions to compare multiple answers generated from the same prompt. We can also use these scoring methods to break ties in self-consistency (covered in the previous lecture).
The scoring methods covered here include computing token and sequence log probabilities (this will also be relevant in the later reinforcement learning chapters). We then implement a self-refinement loop in which the model critiques an answer and attempts a revision.
The evaluation lets us check how useful those additional steps are for our model.
Topics covered
Using scoring functions for comparing multiple answers and breaking ties
Implementing a simple heuristic scorer
Computing token probabilities and sequence likelihoods (also relevant for the following lectures)
Using log probabilities for improved numerical stability
Building a self-refinement loop fo