Hey quesohusker,
I ask myself when will these useless ChatGPT posts stop. You are right, it would not be particularly useful. Why would you need a helper tool to write g-code? 1. It is such a simple language, and 2. every CAD/CAM software spits out a nicer toolpath that really implements what you wanted to do.
By the way:
AI IN THE CROSSFIRE OF CRITICISM
Massive quality deficiencies in ChatGPT as a programming aid
A study reveals significant quality problems in ChatGPT’s answers to programming questions and warns against the use of AI chatbots in software development.
May 26, 2024, 11:54 a.m. , Andreas Donath
A study conducted by researchers at Purdue University has shed light on the accuracy and quality of ChatGPT [https://dl.acm.org/doi/pdf/10.1145/3613904.3642596] in answering programming questions typically found on Stack Overflow. The results, published in the proceedings of the CHI Conference on Human Factors in Computing Systems [https://chi2024.acm.org], raise significant concerns about the accuracy and conciseness of ChatGPT’s answers.
The research team led by Samia Kabir, David N. Udo-Imeh, Bonan Kou, and Tianyi Zhang conducted an in-depth analysis of 517 programming questions from Stack Overflow. They compared ChatGPT’s answers to these questions with the accepted answers given by human programmers on the platform.
The study used a mixed methods approach that combined manual analysis, linguistic analysis and user studies to gain comprehensive insights into ChatGPT’s performance.
ChatGPT’s struggle with accuracy
The manual analysis revealed that 52 percent of ChatGPT’s responses contained incorrect information. The researchers identified four main types of errors: conceptual (54 percent), factual (36 percent), code (28 percent), and terminology (12 percent). These results suggest that ChatGPT struggles to fully grasp the nuances within and context of programming questions, resulting in a high rate of inaccuracy in responses.
Interestingly, the study found that the frequency and age of programming questions influenced the accuracy of ChatGPT’s responses. Answers to common and older questions tended to have fewer errors than less common and newer questions. This suggests that the performance of ChatGPT depends to some extent on the amount of information available in the training data.
The researchers also found that while ChatGPT responses were often verbose, 77 percent contained redundant, irrelevant or unnecessary information. This verbosity was particularly evident in responses to conceptual questions and instructions. In contrast, debugging responses were more concise, even if they did not always match human responses.
Despite the high rate of inaccuracy, the user study found that participants preferred ChatGPT’s answers in 35% of cases. The reasons given for this were the comprehensive and well-formulated nature of ChatGPT’s responses and its polite language. However, participants also overlooked misinformation in ChatGPT’s responses 39 percent of the time, highlighting the potential risks associated with relying on AI-generated responses.
The results of the study emphasize the need for caution when using ChatGPT for programming-related tasks. The researchers emphasized the importance of verifying the correctness of ChatGPT responses before using them in real-world scenarios. They also called for further research into methods to detect and mitigate misinformation generated by AI chatbots. ■
© 1997-2024 Golem.de. All rights reserved.