ChatGPT just got an F.
Not long after it was released to the public, programmers started to take note of a notable feature of OpenAI's ChatGPT: that it couldyet-to-be-peer-reviewed study
, researchers at Purdue University found that the uber-popular AI tool got just over half of 517 software engineering prompts from the popular question-and-answer platform Stack Overflow wrong — a sobering reality check that should have programmers think twice before deploying ChatGPT's answers in anything important.The research goes further, though, finding intriguing nuance in the ability of humans as well.
So how worried should we really be? For one, there are many ways to arrive at the same "correct" answer in software. A lot of human programmers also say they verify ChatGPT's output, suggesting they understand the tool's limitations. But whether that'll continue to be the case remains to be seen.The researchers argue that a lot of work still needs to be done to address these shortcomings.
"Although existing work focus on removing hallucinations from [large language models], those are only applicable to fixing factual errors," they write. "Since the root of conceptual error is not hallucinations, but rather a lack of understanding and reasoning, the existing fixes for hallucination are not applicable to reduce conceptual errors."
In response, we need to focus on "teaching ChatGPT to reason," the researchers conclude — a tall order for this current generation of AI.
Canada Latest News, Canada Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
How Does One Even Name Rihanna's Half-Braided, Half-Straight Hairstyle?It's utter chaos, and I'm loving every inch of it. Rihanna has debuted a unique hairstyle that combines braids and straight hair. This isn't the first time she's rocked this half-braided, half-straight look, as she previously showcased a similar chaotic hairstyle in November 2022.
Read more »
Does ChatGPT Plagiarize and Engage in Copyright Infringement?The use of ChatGPT without citation raises ethical concerns and potential plagiarism accusations. However, the focus now shifts to whether ChatGPT itself plagiarizes and engages in copyright infringement, leading to ongoing legal debates.
Read more »
'Hypnotized' ChatGPT and Bard Create Malicious Code, Offer Bad AdviceIBM researchers conducted an experiment where they manipulated large language models to provide incorrect advice, proving that they can be controlled to offer unethical guidance without data manipulation.
Read more »
Does ChatGPT Plagiarize? Surprising Answers RevealedThe subject of several class-action lawsuits, AI's method of gathering information is under scrutiny. This article explores the ethical concerns and debates surrounding ChatGPT's potential plagiarism and copyright infringement. Find out the surprising answers provided by ChatGPT itself.
Read more »
ChatGPT and Claude are ‘becoming capable of tackling real-world missions,’ say scientistsA team of researchers has developed an artificial intelligence tool called 'Agent Bench' to measure and evaluate the performance of LLM models as real-world agents. OpenAI has also launched a web crawler called 'GPTBot' and is planning for its next model, GPT-5.
Read more »
Newegg Introduces ChatGPT for Product ReviewsNewegg announced it's rolling out a new feature across its service that will generate a product review summary using ChatGPT.
Read more »