According to artificial intelligence firm Anthropic, state-of-the-art LLMs have a tendency to produce “sycophantic” outputs when confronted by human preference.
Artificial intelligence large language models built on one of the most common learning paradigms have a tendency to tell people what they want to hear instead of generating outputs containing the truth, according to a study from Anthropic.“Specifically, we demonstrate that these AI assistants frequently wrongly admit mistakes when questioned by the user, give predictably biased feedback, and mimic errors made by the user.
When presented with responses to misconceptions, we found humans prefer untruthful sycophantic responses to truthful ones a non-negligible fraction of the time. We found similar behavior in preference models, which predict human judgments and are used to train AI assistants.In the above example, taken from a post on X , a leading prompt indicates that the user believes that the sun is yellow when viewed from space.
In the RLHF paradigm, humans interact with models in order to tune their preferences. This is useful, for example, when dialing in how a machine responds to prompts that could solicit potentially harmful outputs such as personally identifiable information or dangerous misinformation.
Canada Latest News, Canada Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Anthropic tests AI rules for the people, by the peopleBuilding AI with more user input could create more trust.
Read more »
Music Giants Sue AI Startup Anthropic for Copyright Infringement over Song LyricsSource of breaking news and analysis, insightful commentary and original reporting, curated and written specifically for the new generation of independent and conservative thinkers.
Read more »
Indianapolis Motor Speedway generates $1 billion economic impact for central Indiana, according to study13News reporter Rich Nye talks with Scott Dixon following his win in the Gallagher Grand Prix at the IMS road course.
Read more »
Humans and Neanderthals mated 250,000 years ago, much earlier than thoughtEmily is a health news writer based in London, United Kingdom. She holds a bachelor's degree in biology from Durham University and a master's degree in clinical and therapeutic neuroscience from Oxford University. She has worked in science communication, medical writing and as a local news reporter while undertaking journalism training.
Read more »
See How Humans around the World Spend the 24 Hours in a DayA new study calculated the average “global human day,” revealing which activities take up most of our time
Read more »
8 Photos That Tell The History of Humans In SpaceESA Astronaut Tim Peake visits WIRED to have a look back at pivotal moments in the history of human space flight, captured in 8 unforgettable photographs.
Read more »