1 publication

• Published Subscription Review Article 17 Oct 2025

  Methodology for Evaluating Code Synthesis in Large Language Models: ChatGPT and Copilot: A Review

  By Saurabh Sheoran, Dinesh Kumar

  Abstract: The authors introduce a comprehensive framework to assess the code-generation capabilities of large language models, focusing on ChatGPT and Copilot through a benchmark suite of 25 program synthesis tasks. Their main goal was to show why making proper comparisons is important, they did not focus on choosing the newest models, since they keep changing frequently. The critique examines how the methodology addresses both functional and non-functional aspects of code. In …

  Published in Recent Trends in Programming languages · Vol. 12, Issue 3, 2025 · pp. 01–07 Read article →