A prompt is not just "a question you ask the AI." It is the entire input context that shapes the probability distribution over the model's next-token predictions.
Key Concepts
System Prompt: Sets the model's persona, constraints, and behavioral guidelines. Processed first, highest priority.User Prompt: The actual task or question. This is what most people think of as "the prompt."Assistant Prefix: Pre-filling part of the response to steer the model (e.g., starting with "Here is the JSON:")Instruction Following: Modern models are fine-tuned to follow instructions — but the quality of the instruction determines the quality of the output.Frameworks
Prompt = Context + Instruction + Format: Every effective prompt has these three componentsAttention Budget: Tokens at the beginning and end of the context window get more attention than those in the middle ("lost in the middle" problem)The "Lost in the Middle" Problem
Research shows that LLMs pay less attention to information placed in the middle of long contexts. Place critical information at the beginning or end of your prompt.
How Different Prompt Positions Affect Output
| Position | Attention Level | Use For |
|---|
| Start of prompt | Highest | System instructions, critical context |
| Middle of prompt | Lowest | Supporting details, examples |
| End of prompt | High | The specific question/task |
| Assistant prefix | Very High | Format control, response steering |
Common Mistakes
Burying the actual question deep inside a long promptNot using system prompts when availableIgnoring the "lost in the middle" effect in long-context tasksAssuming the model reads prompts like a human (linearly with equal attention)