Claude Sonnet 5 Demonstrates Greater Creative Output Quality Than Other Leading GenAI Tools on Introductory Programming Tasks

Abstract

Although recent studies suggest AI can boost creative performance and student confidence, most research has focused on domains like poetry, visual art, and graphic design. Creative programming, especially for beginner and intermediate students, has received far less attention, and few studies have directly compared the creative capabilities of today's GenAI models. This study investigates the question: which leading GenAI tool produces the strongest creative output on unconventional introductory Python programming tasks? We tested four widely used models, GPT-5.5, Claude Sonnet 5, Gemini 3.6 Flash, and Grok 4, using six open-ended introductory Python assignments. To keep conditions consistent, every model received the same prompts in clean browser sessions with no prior conversation history. Independent evaluators then scored the resulting programs using a shared five-category rubric, with each category scored from -2 to 2, covering originality, flexibility, functionality, complexity, and code quality. Inter-grader reliability was assessed using Cohen’s kappa; the value of 0.335 demonstrates a fair agreement. Scores were averaged and compared both overall and by individual criterion. Claude Sonnet 5 achieved the highest overall score (1.522), outperforming the remaining models (p<0.001). Gemini 3.6 Flash followed at 1.133, while Grok 4 (0.822) and GPT-5.5 (0.889) produced similar but lower overall scores. Claude's strongest performance was in functionality, complexity, and code quality. The study provides a framework for comparing creative programming output to help educators make more informed decisions about integration of AI tools in introductory programming courses.