Module 1 • Lesson 2

Tokens, Context Windows & Cost

Learn how tokenization works, what context windows mean, and how to manage costs effectively.

Tokens, Context Windows & Cost

Every interaction with an LLM is measured in tokens — the fundamental unit of processing. Understanding tokens is essential for cost management and effective prompting.

Key Concepts

  • Token: A subword unit, roughly ¾ of a word in English. "ChatGPT is great" ≈ 4 tokens.

    • Every interaction with an LLM is measured in tokens — the fundamental unit of processing. Understanding tokens is essential for cost management and effective prompting.

    Key Concepts

  • Token: A subword unit, roughly ¾ of a word in English. "ChatGPT is great" ≈ 4 tokens.
  • Context Window: The maximum number of tokens the model can process in a single request (input + output combined). GPT-4 Turbo: 128K tokens. Claude 3: 200K tokens.
  • Input vs Output Tokens: Input tokens (your prompt) are cheaper than output tokens (model response). Optimize accordingly.
  • Token Budgeting: Plan your prompt + expected response to fit within the context window.

    • Frameworks

  • Cost Estimation Formula: Total Cost = (Input Tokens × Input Price) + (Output Tokens × Output Price)
  • Context Window Management: System prompt + conversation history + user query + expected response ≤ context window

    • Practical Guidelines

  • Use a tokenizer tool (tiktoken for OpenAI, etc.) to estimate token counts before sending expensive prompts
  • For long documents, summarize or chunk rather than stuffing the entire context window
  • Output tokens are 2-4x more expensive than input tokens — ask for concise responses when possible
  • Different models have different tokenizers — the same text may use different token counts

    • Common Mistakes

  • Ignoring token limits and getting truncated responses
  • Paying for unnecessarily verbose outputs
  • Not accounting for system prompts in the token budget
  • Assuming all models tokenize text the same way

    • Read to the end of the lesson to mark it as read.