Blog

Vitor Sousa

Writing & Research

Articles & Engineering Notes

This is the hub for my long-form writing on data science, machine learning, and the engineering patterns that support them in production.

Each article aims to balance clarity with rigor: you will find walkthroughs of algorithms, postmortems from experiments, and the practical guardrails that emerge from shipping systems.

Deep dives and foundational pieces worth exploring first

Latest articles

Use the filters to surface the topics, stacks, and case studies that match your current problem.

GDPO: Multi-Reward RL Done Right Part 4 When GRPO meets multiple rewards, advantages collapse. GDPO fixes this by normalizing each reward independently before combining. Learn why this matters for tool calling, math reasoning, and any multi-objective LLM alignment. ~25 min
GRPO: Eliminating the Value Network Part 3 Group Relative Policy Optimization replaces PPO's learned value function with a simple insight: sample multiple outputs and use their relative rewards as advantages. 33% memory savings, simpler implementation, and the algorithm powering DeepSeek-R1. ~32 min
PPO for Language Models: The RLHF Workhorse Part 2 Deep dive into Proximal Policy Optimization—the algorithm behind most LLM alignment. Understand trust regions, the clipped objective, GAE, and why PPO's four-model architecture creates problems at scale. ~28 min
Reinforcement Learning Foundations for LLM Alignment Part 1 Master the RL fundamentals powering modern LLM training: from MDPs and policy gradients through value functions and actor-critic methods. The mathematical foundations you need before diving into PPO, GRPO, and beyond. ~35 min
Deploying Contextual Bandits: Production Guide and Offline Evaluation Part 5 Systems design, offline evaluation, and monitoring strategies for running contextual bandits safely in production. 24 min
Neural Contextual Bandits for High-Dimensional Data Part 4 When linear models fail, neural networks step in. Learn when to use neural bandits, how to quantify uncertainty with bootstrap ensembles, and handle high-dimensional action spaces with embeddings and two-stage selection. ~22 min
Implementing Contextual Bandits: Complete Algorithm Guide Part 3 Complete Python implementations of ε-greedy, UCB, LinUCB, and Thompson Sampling. Learn which algorithm to use for your problem with default hyperparameters and practical tuning guidance. ~25 min
Contextual Bandit Theory: Regret Bounds and Exploration Part 2 Understand the theory behind contextual bandits: regret bounds, the exploration-exploitation tradeoff, reward models, and why certain algorithms work. Math that directly informs practice. ~18 min
When to Use Contextual Bandits: The Decision Framework Part 1 Stop running month-long A/B tests that leave value on the table. Learn when contextual bandits are the right choice for adaptive, personalized optimization—and when to stick with simpler alternatives. ~20 min
Beyond the Vibe Check: A Systematic Approach to LLM Evaluation Stop relying on gut feelings to evaluate LLM outputs. Learn systematic approaches to build trustworthy evaluation pipelines with measurable metrics, proven methods, and production-ready practices. A practical guide covering faithfulness vs helpfulness, LLM-as-judge techniques, bias mitigation, and continuous monitoring. ~60 min
OpenELM Notes I wrote about OpenELM and how Apple approaches efficient language models. 8 min