Most developers learn a few prompting tricks, pick a favourite, and use it forever. But prompting techniques aren't interchangeable — zero-shot, few-shot, chain-of-thought, and system-prompt tuning each produce meaningfully different outputs depending on the task. I built Prompt Lab so you can see those differences in real time, side by side.
Why prompting technique matters more than people think
There's a persistent myth in AI-assisted development: that prompt quality only matters at the extremes — obviously bad prompts fail, obviously good prompts work, and everything in the middle is roughly equivalent. In practice, that's not true.
The same underlying request, expressed using four different prompting techniques, can produce outputs that differ not just in length and style but in factual accuracy, reasoning quality, and task completion. For engineers building AI-native products, understanding these differences isn't academic — it directly affects the reliability of your features.
The problem is that it's hard to develop intuitions about this without a way to compare directly. Reading about chain-of-thought prompting is one thing; watching a chain-of-thought response arrive alongside a zero-shot response to the same input makes the difference immediately obvious.
The four techniques Prompt Lab compares
Zero-shot: The baseline. Just the question, no examples, no special instructions. You give the model the task and let it respond with whatever its training suggests is appropriate. This is how most people use LLMs by default. It works well for tasks the model has been extensively trained on; it breaks down for unusual task formats or when you need specific output structure.
Few-shot: Include 2-3 examples of the expected input/output format before the actual question. The model learns the pattern from the examples and applies it. This is powerful for formatting, classification, and any task where "style" of output matters as much as content. A model that doesn't understand "answer in JSON" will understand it instantly when you show two examples of JSON answers.
Chain-of-thought: Instruct the model to think through the problem step by step before answering. The famous “let's think step by step” suffix isn't magic — it works because it forces the model to generate intermediate reasoning, which makes errors catchable and correctable before the final answer crystallises. For math, logic, and multi-step reasoning tasks, CoT can produce dramatically more reliable outputs.
System-prompt tuning: Set the model's identity, constraints, and response format in the system prompt rather than (or in addition to) the user turn. This is how production AI features are actually deployed — a carefully crafted system prompt defines the persona, boundaries, and output format, and the user turn just provides the task input. When done well, system-prompt tuning gives you the most consistent and controllable outputs.
The architecture: four simultaneous SSE streams
The core technical challenge was making all four technique comparisons stream simultaneously without one affecting the others. The naive approach — sequential requests — would serialise the outputs and introduce latency bias in both TTFT and total time.
Instead, the client fires all four fetch requests at the same moment using Promise.all, each targeting the same Anthropic endpoint but with different prompt construction:
// All four streams start simultaneously
await Promise.all([
runStream("zero-shot", buildZeroShot(prompt), setPanelA),
runStream("few-shot", buildFewShot(prompt, examples), setPanelB),
runStream("chain-of-thought", buildCoT(prompt), setPanelC),
runStream("system-prompt", buildSystemPrompt(prompt), setPanelD),
]);Each stream is managed independently with its own state: character count as text arrives, time-to-first-token (TTFT) recorded on the first text_delta event, and total tokens from the message_delta close event.
The result is four panels filling in simultaneously, making the timing differences visible immediately. Depending on the task, you'll often see CoT start at similar speed to zero-shot but keep generating significantly longer as it works through its reasoning. Few-shot can produce faster outputs because the examples help the model orient quickly. System-prompt tuning often produces the most consistent output length because the constraints are precisely defined.
What I actually observed
Building a tool that forces you to run the same prompt through four techniques reveals patterns that pure reading misses.
Few-shot beats zero-shot on format, not quality. When I tested reasoning tasks, few-shot and zero-shot often reached similar conclusions — but few-shot produced them in a cleaner, more consistent format. The examples teach the model how to present, not what to think. For tasks where output structure matters (structured data, classification labels, formatted reports), few-shot is worth the extra prompt tokens.
CoT is overkill for simple tasks, essential for hard ones. For factual retrieval or simple summarisation, CoT adds verbosity without adding accuracy — the model just narrates things it already knew. For multi-step reasoning, mathematical problems, or scenarios with multiple valid interpretations, CoT is the difference between a correct and incorrect answer. Knowing when to pay the token cost is the skill.
System-prompt tuning is underused for reliability. Most developers put everything in the user turn and then wonder why outputs are inconsistent. Moving persona, constraints, and format requirements into the system prompt dramatically stabilises outputs across different user inputs. It's how production AI features stay predictable — the user can say anything; the system prompt holds the behaviour.
Temperature is often the culprit for variance. The temperature slider in Prompt Lab makes this visceral: at 0.0, all four techniques produce nearly identical outputs across multiple runs. At 1.0, even zero-shot and few-shot diverge significantly on successive runs. Most production AI features that need reliability should be running at 0.0-0.3. Most demos run at 0.7+ because it feels more "creative."
Building it: a few interesting implementation decisions
Prompt construction is stateless: Each technique's prompt builder is a pure function that takes the user's input and returns the full prompt string. This makes it easy to test and ensures the comparison is fair — the only variable is the prompting technique, not any hidden state.
TTFT is measured client-side: Rather than measuring on the server (which would include network latency from the API), TTFT is measured in the React component from when the fetch request fires to when the first text character arrives in the browser. This is the latency that actually matters for user experience.
The model selector uses raw fetch: After the lesson learned with the Anthropic SDK's Vercel streaming issues, Prompt Lab uses direct fetch() calls with SSE parsing. This works reliably in any serverless environment and gives precise control over the stream format.
Why this matters for AI product development
If you're building a production AI feature, you're going to make a prompting decision. Most engineers make that decision once, based on intuition or whatever worked in a quick test. They don't revisit it when users report inconsistent outputs. They don't measure whether a different technique would be more reliable at scale.
The teams that build reliable AI products treat prompting as an engineering discipline, not a "make it work" heuristic. They test prompts against representative inputs, measure consistency, and deliberately choose techniques based on the task class.
Prompt Lab is the starting point for developing that discipline. Run your actual production prompt through it. Watch how the four techniques handle your edge cases. If they all produce good outputs, great — pick the cheapest one. If they diverge, you've learned something important about the reliability characteristics of your feature before your users do.
The live demo is at prompt-lab-delta.vercel.app. Source is at github.com/matua-agent/prompt-lab.