Module 2 • Lesson 3

System Prompts: The Hidden Control Layer

Understand how system prompts work across different APIs and how to craft effective system-level instructions.

System Prompts: The Hidden Control Layer

System prompts are the most powerful yet least visible component of prompt engineering. They set the model's behavior, personality, and constraints for an entire conversation — and they operate at the highest priority level.

Key Concepts

  • System Prompt: A special instruction block processed before any user messages, setting the model's behavioral guidelines

    • System prompts are the most powerful yet least visible component of prompt engineering. They set the model's behavior, personality, and constraints for an entire conversation — and they operate at the highest priority level.

    Key Concepts

  • System Prompt: A special instruction block processed before any user messages, setting the model's behavioral guidelines
  • Priority Hierarchy: System prompt > first user message > subsequent messages
  • Persistence: System prompts apply to the entire conversation, not just one turn

    • How System Prompts Work Across APIs

    ProviderSystem Prompt MethodMax Length
    OpenAI (GPT-4)role: "system" in messages array~12K tokens recommended
    Anthropic (Claude)system parameter separate from messages8K tokens recommended
    Google (Gemini)system_instruction parameterVaries by model
    Open Source (Llama)[INST] <<SYS>>...<</SYS>> formatModel-dependent

    Anatomy of an Effective System Prompt

    A production-grade system prompt typically includes:

  • Identity: Who/what the AI is
  • Core Behavior: How it should respond
  • Constraints: What it should NOT do
  • Output Format: Default formatting preferences
  • Error Handling: What to do when uncertain

    • Example Production System Prompt

    `

    You are FinanceBot, an AI assistant for Acme Financial Advisors.

    CORE BEHAVIOR:

  • Provide accurate, conservative financial guidance
  • Always cite regulatory frameworks (SEC, FINRA) when relevant
  • Use plain language — avoid jargon unless the user demonstrates expertise

    • CONSTRAINTS:

  • NEVER provide specific investment recommendations ("Buy AAPL")
  • NEVER guarantee returns or make predictions about specific securities
  • Always include the disclaimer: "This is educational content, not financial advice"
  • If asked about topics outside finance, politely redirect

    • OUTPUT FORMAT:

  • Use bullet points for lists
  • Include relevant calculations when discussing returns or risk
  • Format dollar amounts with commas and 2 decimal places

    • WHEN UNCERTAIN:

  • Say "I am not sure about this — please consult a licensed financial advisor"
  • Never fabricate regulatory information

    • `

    System Prompt Design Patterns

  • The Guardrail Pattern: Define explicit boundaries ("Never discuss competitors")
  • The Expert Pattern: Set deep domain expertise ("You have 20 years of experience in...")
  • The Format Controller: Force consistent output structure across all responses
  • The Persona Pattern: Define communication style, tone, and personality

    • Common Mistakes

  • Making system prompts too long (>2K tokens) — attention degrades
  • Not testing system prompt effectiveness with adversarial inputs
  • Duplicating instructions that should be in user prompts
  • Forgetting that system prompts consume context window tokens
  • Assuming system prompts are "secret" — they can potentially be extracted

    • Read to the end of the lesson to mark it as read.