Live News

Live News

UN new report says AI gets climate science right, users pushes back

UN new report says AI gets climate science right, users pushes back

By Anjali Sharma
UNITED NATIONS – According to a new report co-authored by the UN on Monday said that the best AI models answer most climate science questions correctly, but they often abandon established facts as soon as a user disagrees.

The report, Can We Trust AI on Climate? was presented on Monday at the COP31 climate conference in Antalya by UNESCO and University College London.

The researchers tested up to 33 leading AI models in 13 languages.

Leona Verdadero, the report’s lead author at UNESCO said “Many of us interact with these AI models and chatbots on a daily basis seeking information on a wide variety of topics”.

“A lot of times these models, when they respond, they sound really authoritative, and they sound very smart. But as we’ve seen with our experiments, they don’t always sound right.”

Researchers found that AI tends towards sycophancy – a built-in inclination to agree with users, even when they present false information.

In tests on 15 models, researchers challenged answers the models had got right, moving from simple disagreement to claimed expertise and fabricated evidence.

Every model gave way, and some could be pushed into endorsing false claims, such as that carbon dioxide is not the main driver of climate change.

Some 8 in 10 of the answers that changed did so at the very first sign of disagreement.

“If a user says, ‘I actually don’t think that is right,’ what’s very fascinating is that we found in eight out of 10 of these cases, when a user pushes back, the AI model can actually reverse its answer to be more agreeable with the user,” Ms. Verdadero stated.

“AI could just abandon a scientific fact to be more agreeable to the user.”

On textbook-style climate questions, the best models answered between 88 and 96%. But the more reasoning a task required, the worse they did.

“What we’re saying is that the science and the experts should be trusted more,” Ms. Verdadero said. “The AI models aren’t there yet.”

Accuracy also depended on the language used. Models averaged 88.6 per cent in English, with similar results in Spanish, French and Portuguese, but fell to 83.6 per cent in Hindi and Bengali, 81.4 per cent in Urdu and 78.3 per cent in Swahili.

“Imagine you and I are using the same model, asking the exact same question, but in a different language,” Ms. Verdadero said. “Based on the language, we can have totally different responses.”

“Communities that are speaking these languages are primarily in South Asia and East Africa, and we know these are communities that are very much climate vulnerable,” she noted.

The report called on technology companies to invest more in local datasets and multilingual development, so that speakers of underrepresented languages are not left behind.

The report is part of UNESCO’s contribution to the Global Initiative for Information Integrity on Climate Change, which it co-chairs with the UN and Brazil.

“The fight against climate change needs to be grounded in reliable evidence and scientific evidence,” Ms. Verdadero told us. “A lot of this evidence informs policies today and also feeds into the COP negotiations.”

She stressed that AI should be treated as only one source among many.

“These AI models should be used as one type of information source, but it should not be your primary one,” she said. “Fact checking, making sure the sources are reliable, is important.

“And, of course, trusting evidence from experts, climate scientists and researchers should not be replaced,” she concluded.

Related Post