ysskrishna logoysskrishna

Chapter 4 of 8 · 3 min

Examples that teach (without over-fitting)

One or two well-chosen examples beat a paragraph of rules. How to pick them, label them and stop the model copying them too literally.

Describing a format in words is hard. Showing it is easy. A single example of the output you want often does more than ten lines of instructions. This is called few-shot prompting, and it works just as well for coding agents as for chat.

When examples help most

  • Formats with house style: commit messages, changelog lines, test names.
  • Transformations: "turn this log line into this JSON".
  • Tone: release notes that sound like your team, not like a press release.
  • Edge cases: showing what to do with an empty field or a weird input.

The basic pattern

Write a commit message for the staged diff.

Format: <area>: <imperative summary, max 60 chars>
blank line
<why, 1-2 lines>

Example
Input diff: adds retry with backoff to the payment client
Output:
payments: retry transient errors with backoff

Provider returns 503 during deploys; three retries with jitter
stop failed checkouts without hiding real outages.

The example teaches things that are tedious to describe: the lowercase area, the imperative verb, a "why" that mentions the user impact.

Choosing examples

  1. One typical, one edge case. The typical example sets the pattern; the edge case shows the boundary (for example, a diff that touches two areas).
  2. Realistic, not toy. Use real data from your project (scrubbed of secrets). Toy examples teach toy output.
  3. Different on the surface. If both examples are about payments, the model may assume every commit is about payments.

Labelling

Mark examples clearly so the model does not confuse them with the actual task:

<example>
Input: ...
Output: ...
</example>

Now the real input:
...

XML-style tags or clear headings both work. What matters is that the boundary is obvious.

Over-fitting and how to avoid it

Models copy examples closely: length, wording, even specific nouns. Signs of over-fitting:

  • Every output has exactly the same number of bullets as the example.
  • Words from the example appear in unrelated outputs.
  • Unusual inputs get forced into the example's shape.

Fixes:

  • Add a sentence saying which parts are fixed and which vary: "The area and length are fixed; the wording should fit the diff."
  • Use two examples that differ in length and content.
  • Keep the rule written down as well; the example illustrates it, it does not replace it.

Examples in skills

Skills (see Agent Building Blocks) are a good home for examples you reuse. Keep one short example in SKILL.md, and put longer worked examples in references/ so they load only when needed.

Step by step

  1. Take a prompt where outputs vary more than you want.
  2. Pick one real input and write the ideal output by hand.
  3. Add it as a labelled example. Run the prompt on three new inputs.
  4. If outputs copy the example too closely, add a second, different example and a "what varies" sentence.

Practice: one example versus three

Pick a task with a fixed output shape, such as turning a bug report into a ticket title. Write the prompt with one example and run it on five new bug reports; note how often the titles copy the example's wording or length. Then add two more examples that differ in length and subject, and run the same five again. Usually the outputs vary more sensibly with three varied examples, because the model can see which parts are pattern (the format) and which parts are content (the words). Keep the version that wins on your five inputs, and save those inputs; they are the start of the eval set in chapter 7.

ExamplesTypical result
NoneFormat drifts between runs
OneFormat holds, wording copies the example
Three, variedFormat holds, wording follows the input

Quick check

  1. What two examples make a good minimal set?One typical case and one edge case, different on the surface.

  2. Every output now has exactly three bullets like your example. What happened and what do you do?Over-fitting. Say what varies, and add a second example with a different number of bullets.

  3. Why label examples with tags or headings?So the model does not mistake the example for the real task.

Continue exploring

CourseCover image for Agent Building Blocks: Skills, Subagents, MCP and Hooks

Agent Building Blocks: Skills, Subagents, MCP and Hooks

A free, hands-on course on the four pieces behind Cursor and Claude Code agents: skills, subagents, MCP servers and hooks. Build each one, then combine them.

Agent Skills
Chat & AI
+1

9 chapters · 30 min

BlogCover image for 17 Thinking Skills for AI Agents: Pre-Mortem, Five Whys, Six Hats and More

17 Thinking Skills for AI Agents: Pre-Mortem, Five Whys, Six Hats and More

AI Apps & Assistants
Agent Skills
4 min read
ToolCover image for AI Token Counter

AI Token Counter

AI token counter with model pricing and context-window comparison in your browser.

Chat & AI
AI Apps & Assistants
+1
Productskills.sh+1Claude Code Subagents for Cursor

Claude Code Subagents for Cursor

Your Cursor subagents can finally run on the model you picked in each agent file.
Chat & AI
Agent Skills
AI Apps & Assistants
Bash
BlogCover image for How to Install Caveman in Cursor and Claude Code

How to Install Caveman in Cursor and Claude Code

AI Apps & Assistants
Agent Skills
+1
3 min read
BlogCover image for Publish AI Agent Skills to ClawHub: Stop Re-Pasting Your AI Workflow

Publish AI Agent Skills to ClawHub: Stop Re-Pasting Your AI Workflow

AI Apps & Assistants
Agent Skills
3 min read