Module 5: Prompt Engineering - How Decoding and Sampling Work

This episode explores the hidden layer between your prompt and the model’s response: decoding and sampling. We look at how the model moves from a field of possible next tokens to the one it actually chooses, why the same prompt can produce different outputs, and how that variation is shaped rather than random. We walk through the core strategies you will hear over and over in prompt engineering, from greedy decoding to temperature, top-k, and top-p, and the tradeoff each one creates between precision, consistency, creativity, and control. We also touch on why these settings matter differently depending on the task, and why newer reasoning models do not always play by the same rules.