WyrdWerk Deployment Wiki

Prompt Engineering

5 min read

Most teams write a prompt once and treat it like config. It isn’t. It’s code. And like code, it breaks when the environment changes around it.

Prompt engineering is the practice of designing, testing, and maintaining the instructions that govern AI behavior. System prompts, user message templates, few-shot examples, chain-of-thought instructions — these define how a model interprets tasks and structures responses. They also drift, break, and need version control.

System Prompt Architecture

The system prompt is the instruction layer that runs before every user interaction. For production AI agents, it typically contains:

  • Role definition: What the AI is, what it’s not, who it serves
  • Task scope: What it should help with and what it should decline
  • Output constraints: Format requirements (JSON, markdown, bullet points, character limits)
  • Persona and tone: How formal, how direct, what reading level
  • Safety and escalation rules: When to hand off to a human, what topics to avoid

A well-structured system prompt for a customer support agent is 300–500 words. Shorter and it’s too vague; longer and the model starts ignoring sections in long conversations.

Three Core Techniques

Few-Shot Examples

Include 2–5 examples of correct input/output pairs directly in the prompt. The model learns from demonstration, not just instruction.

Without few-shot examples:

User: "Summarize this customer complaint."
Model: [generates any format it finds plausible]

With few-shot examples:

EXAMPLE:
Input: "I've been waiting 3 weeks for my order."
Output: {
  "category": "shipping-delay",
  "urgency": "high", 
  "suggested_response": "apologize_and_escalate"
}

The structured output becomes consistent. The classification categories become reliable.

Chain-of-Thought Instructions

For complex reasoning tasks, instruct the model to show its work before giving the final answer. “Think step by step before answering” and “First, identify X, then check for Y, then conclude Z” both reduce errors on tasks that require multi-step reasoning.

Chain-of-thought adds tokens (and therefore cost) to every response. For simple classification tasks it’s unnecessary. For legal summarization, financial analysis, or multi-step agent reasoning, it meaningfully improves accuracy.

Output Constraints

Specify exactly what format you need. If your downstream code parses JSON, the prompt should say “Always respond with valid JSON matching this schema: {…}“. If your app expects a specific field, say so explicitly.

Models that produce inconsistent output formats cause parsing failures. Those failures are often silent — the downstream system receives malformed data and produces wrong results without error.

The Maintenance Reality

Prompts break for four reasons:

1. Model updates. Your provider silently updates the underlying model. The prompt that produced clean JSON now sometimes returns markdown-wrapped JSON. One of your tests fails. You spend a day tracking down a prompt interaction you never changed.

2. Input distribution shift. Your users start asking questions you never anticipated when you wrote the prompt. The prompt handles your original test cases but fails on the edge cases that emerge at month three.

3. Context window growth. In long conversations, the system prompt competes with conversation history for the model’s attention. Instructions given at the start of the prompt can be effectively ignored in the 20th turn of a long conversation.

4. Downstream dependency changes. The output format your code expects changes. The prompt now produces technically correct output that your application can’t parse.

The standard recommendation: re-evaluate your production prompts every 30 days, and within 48 hours of any provider model update announcement. This isn’t optional. LLM drift documentation shows that a prompt unchanged for 8 weeks can produce meaningfully different outputs after a silent provider update.

Version Control and Testing

Treat prompts like application code:

  • Store prompts in version control (Git or Langfuse’s prompt management)
  • Create a golden eval set: 50–100 input/output pairs representative of real usage
  • Run the eval set against every prompt change before deploying
  • Track quality metrics over time — accuracy, format consistency, refusal rate

A golden eval set is cheap to build (sample 50 real user interactions and hand-label the correct outputs). Without it, you have no way to detect prompt regression until users complain.

Langfuse provides prompt version management and A/B testing infrastructure for this workflow — compare two prompt versions on the same traffic sample before fully rolling out a change.

Cost Optimization

Better prompts use fewer tokens. Specific, directive prompts produce shorter, more relevant responses than vague ones. Some practical reductions:

  • Use structured output formats (JSON) rather than free-form prose — they’re typically shorter and easier to parse
  • Remove redundant instructions the model already knows
  • For classification tasks, enumerate the valid output values — the model stops explaining and just outputs the category

At scale, a 20% reduction in average response length via prompt optimization can meaningfully reduce monthly API costs.

Graph View

Backlinks