Chain-of-Draft Prompting: A More Efficient Alternative to Chain of Thought

Yusuf Ishola· March 5, 2025

With Chain-of-Thought (CoT) prompting, Large Language Models (LLMs) can now solve complex reasoning tasks with impressive accuracy, but often at high computational cost.

Chain-of-Draft (CoD) prompting, introduced by researchers at Zoom Communications, provides a solution by encouraging LLMs to generate minimal yet informative reasoning steps—similar to how humans jot down concise notes rather than elaborate explanations.

Let's take a look at this fascinating new method and see how you can start benefitting from it today.

What is Chain-of-Draft Prompting?

Chain-of-Draft is a new prompting technique that aims to reduce token usage while maintaining reasoning quality.

CoD takes inspiration from human problem-solving behavior where we capture only essential information in our notes. Unlike Chain of Thought (CoT) prompting that generates verbose step-by-step reasoning, CoD limits each reasoning step to the absolute minimum—typically 5 words or less.

This approach preserves the reasoning pathway while drastically cutting computational overhead.

Chain-of-Draft vs. Chain-of-Thought Outputs

Let's compare approaches using a simple math problem:

Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give to Denny?

Chain-of-Thought Process:

Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give to Denny?
A: Let's think through this step by step:
1. Initially, Jason had 20 lollipops.
2. After giving some to Denny, Jason now has 12 lollipops.
3. To find out how many lollipops Jason gave to Denny, we need to calculate the difference between the initial number of lollipops and the remaining number.
4. We can set up a simple subtraction problem: Initial number of lollipops - Remaining number of lollipops = Lollipops given to Denny
5. Putting in the numbers: 20 - 12 = Lollipops given to Denny
6. Solving the subtraction: 20 - 12 = 8
Therefore, Jason gave 8 lollipops to Denny.

[Response]: 8 lollipops

Chain-of-Draft Process:

Q: Jason had 20 lollipops. He gave Denny some lollipops. Now Jason has 12 lollipops. How many lollipops did Jason give to Denny?
A: 20 - x = 12; x = 20 - 12 = 8. 
 
[Response]: 8

CoD provides the key reasoning steps without unnecessary text.

How to Implement Chain-of-Draft

Implementing CoD requires only a simple prompt modification.

The beauty of CoD is its simplicity—no model changes, fine-tuning, or complex infrastructure required. You simply adjust your system prompt to instruct the model to provide concise reasoning steps.

Here's a basic implementation using the OpenAI API:

import openai

cod_prompt = """Think step by step, but only keep a minimum draft for each thinking step, with 5 words at most. Return the answer at the end of the response after a separator ####."""

response = openai.ChatCompletion.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": cod_prompt},
        {"role": "user", "content": "If a train travels at 60 miles per hour for 2.5 hours, how far does it go?"}
    ]
)

print(response.choices[0].message['content'])

The key is instructing the model to limit each thinking step to just a few words.

Performance Benchmarks: Chain-of-Draft vs Chain-of-Thought

CoD delivers remarkable efficiency gains without sacrificing accuracy.

When tested across multiple reasoning tasks, CoD consistently achieves similar accuracy to CoT while using a fraction of the tokens.

On GSM8k (a math reasoning benchmark), CoD achieved 91% accuracy compared to CoT's 95%, but used only 20% of the tokens and reduced latency by up to 76%.

For symbolic reasoning tasks like the coin flip problem, CoD matched CoT's perfect 100% accuracy while using only 14-32% of the tokens.

These results demonstrate CoD's practical value for production applications.

When to Use Chain-of-Draft

CoD excels in high-volume, cost-sensitive applications.

Consider using Chain-of-Draft when:

• ✅ You want to utilize reasoning capabilities but token costs are a significant concern
• ✅ Response latency is critical
• ✅ You need transparent reasoning but want to minimize overhead
• ✅ Processing high volumes of inference requests
• ✅ Working with reasoning-heavy applications like math problem-solving

CoD provides the best balance between reasoning quality and efficiency.

Limitations to Consider

CoD has specific constraints worth noting.

CoD performs best in few-shot settings (where you seed with examples), with accuracy dropping significantly in zero-shot (no examples) scenarios. This is likely due to the absence of CoD-style reasoning patterns in training data which makes it harder for models to 'understand' the approach.

Smaller models (under 3B parameters) also show a more pronounced performance gap between CoT and CoD. Additionally, tasks requiring extensive contextual understanding may benefit from more verbose reasoning.

Conclusion

Chain-of-Draft presents a practical, immediately applicable approach to more efficient LLM reasoning.

By using the minimalist reasoning approach of CoD, you can achieve comparable accuracy to Chain of Thought while significantly reducing costs and latency. This makes advanced reasoning capabilities more accessible for production applications with strict budgets and performance requirements.

Go ahead and play around with CoD and be sure to use Helicone to measure the improvements!

You might find these useful:

• How to Prompt Thinking Models like DeepSeek R1 and OpenAI o3
• Chain-of-Thought Prompting: Techniques, Tips, and Code Examples
• Tree-of-Thought Prompting: Key Techniques and Use Cases
• Prompt Engineering Tools & Techniques