Support NeoGAF

[Draugoth]

Source​

In a landmark study, OpenAI researchers reveal that large language models will always produce plausible but false outputs, even with perfect data, due to fundamental statistical and computational limits.

"Large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Such 'hallucinations' persist even in state-of-the-art systems."

"The study established that "the generative error rate is at least twice the IIV misclassification rate," where IIV referred to "Is-It-Valid" and demonstrated mathematical lower bounds that prove AI systems will always make a certain percentage of mistakes, no matter how much the technology improves."

"The OpenAI research also revealed that industry evaluation methods actively encouraged the problem. Analysis of popular benchmarks, including GPQA, MMLU-Pro, and SWE-bench, found nine out of 10 major evaluations used binary grading that penalized "I don't know" responses while rewarding incorrect but confident answers."

 

[Hookshot]

I love what AI is doing in the medical fields, they can check for blood cancers via eye tests now but you always need a person checking it to make sure it's not producing utter bollocks.

Sadly those pushing it hard seem to think it's magic and not easily derailed.

 

[bitbydeath]

"large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty,"

What could possibly go wrong.

 

[jason10mm]

I wonder if this is the solution....

Get multiple answers and they need a majority consensus to weed out the random oddball.

Damn, now I gotta watch it again.....

 

[Wildebeest]

For the first time in human history we will have to deal with plausible but false information. We may never recover.

 

[Futaleufu]

For the first time in human history we will have to deal with plausible but false information. We may never recover.

It has been an issue since the cavemen days

 

[ReBurn]

Humanity never needed to verify conclusions until AI came along.

 

[readonly]

Where one company has flaws, another will solve. Only ever a matter of time, so it doesn't even matter. It's just something we should know when using it for now.