Independent Paused
Learning with Verifiable Rewards
Does GRPO earn its cost over a properly built SFT baseline on GSM8K? The model and training foundations are built and tested; the study is paused before the first SFT run.
Vitor Sousa / Research journal
Notes on learning systems.
I’m Vitor. This is my notebook for working through machine-learning questions—from mathematical ideas to implementations and experiments.
Most of what I write is about transformers, evaluation, reinforcement learning, and recommendation systems. I try to be clear about what I’ve tested, what I think, and what I still don’t know.
Recent writing
All writing-
A Transformer from Raw Tensors: What 112 Tests Taught Me
What it took to verify a decoder-only transformer built from raw tensors, and the three properties that were easiest to get silently wrong.
-
After GDPO: A Map of the GRPO Family
Reading Dr. GRPO, DAPO, GSPO and GDPO as points along seven design axes of GRPO, with a guide to choosing between them.
Updated Oct 31, 2026
-
Building a Transformer: The Complete Forward Pass
Normalization, residual connections, feed-forward layers, and a decoder-only transformer assembled piece by piece with tested code.
Transformer Internals · Part 3
Selected investigations
All projectsA few projects where the code, method, and lessons are documented — with the status they actually have.
Published collaboration Completed
Tailor at Farfetch
Size recommendation as sequence classification: LSTMs and attention over a customer's history to model evolving fit preferences. A co-authored paper at FashionXRecSys 2023.
From the notebook
All notes-
Benchmark Your Benchmark
A number without variance isn't a result — and a benchmark without a correctness gate isn't a benchmark.
-
GELU / SwiGLU from Scratch
Why it matters: The gate is a content-dependent multiplier on each hidden unit — the FFN learns which features to let through, not just how to mix them.
-
Pre-Norm vs Post-Norm: Why It Matters
Why it matters: In pre-norm the residual stream is never normalized, so every block has a clean identity gradient path back to the input — deep stacks (≥12 layers) train without…