Evaluating Prompt Engineering Strategies on GPT-5 for Accuracy and Token Efficiency on Competitive Programming Tasks

Abstract

Large language models have become central to modern software development as 84% of developers now use or plan to use AI tools in their development process (Stack Overflow, 2025). Despite the productivity gains AI tools offer, reasoning models like GPT-5 generate extensive internal thinking traces before producing any output, resulting in high token consumption and costs (OpenAI API Pricing, 2026; CloudZero, 2026). To combat this, prompt engineering has emerged as a promising approach but most existing research optimizes prompts for accuracy alone, and doesn’t consider token efficiency (Shah et al., 2026; Wang et al., 2024). We evaluated 3 prompt types, Structured Chain of Thought, Quick Solver, and No Prompt as a control, using GPT 5 on AtCoder competitive programming problems spanning difficulty ratings from 1000 to 1800, recording both the AtCoder judge verdict and the combined cost of all tokens for each run. Across 450 submissions accuracy ranged from 69.3% for No Prompt (avg $0.03905 per problem) to 66.0% for SCoT (avg $0.03927 per problem) and 63.3% for Quick Solver (avg $0.036710 per problem). McNemar tests revealed no significant improvement in accuracy across prompts, and paired t tests showed no meaningful reduction in token cost. This suggests that for reasoning models like GPT-5, prompts have limited impact on accuracy and token efficiency, motivating future work to isolate the effect of structured prompts on the reasoning process itself, as the internal thinking of reasoning models may respond differently to prompt design than the final output does.