Evaluating Prompt Engineering Strategies for Reasoning Token Efficiency in Large Language Models

Abstract

Large language models have become central to modern software development as 84% of developers now

use or plan to use AI tools in their development process (Stack Overflow, 2025). Despite the productivity

gains AI tools offer, reasoning models like GPT-5 generate extensive internal thinking traces before

producing any output, resulting in high token consumption and costs (OpenAI API Pricing, 2026;

CloudZero, 2026). To combat this, prompt engineering has emerged as a promising approach but most

existing research optimizes prompts for accuracy alone, leaving the tradeoff between token efficiency

and correctness largely unexplored (Shah et al., 2026; Wang et al., 2024). We evaluated 3 prompt types,

Structured Chain of Thought, Accuracy-Focused, and No Prompt as a control, using GPT 5 on AtCoder

competitive programming problems spanning difficulty ratings from 1000 to 1800, recording both the

AtCoder judge verdict and reasoning token count for each run. Across 450 submissions accuracy ranged

from 71.3% for No Prompt (avg 2,847 reasoning tokens) to 68.7% for SCoT (avg 2,778 reasoning tokens)

and 67.3% for Acc-Focused (avg 2,269 reasoning tokens). McNemar tests revealed no significant

improvement in accuracy across prompts, and token comparisons showed no meaningful reduction in

token usage. This suggests that for reasoning models like GPT-5, prompts have limited impact on

accuracy and token efficiency, motivating future work to isolate the effect of structured prompts on the

reasoning process itself, as the internal thinking of reasoning models may respond differently to prompt

design than the final output does.