TechAILatest.newsTrackers

AI Research News: Papers, Methods and Scientific Breakthroughs

Research that changes what AI can do: new training and inference methods, evaluation results, interpretability and safety work, and AI applied to science. Stories summarise the paper or lab report in plain language, name the authors and institution, and link to the original publication.

Ali Farhadi in a blue jacket and white T-shirt speaking into a microphone on stage in front of a TechCrunch Disrupt backdrop.Tech AI Latest

Ai2 study finds LLM novelty judges flip verdicts when prompts change, from 93.5% to 40.9%

AI Research

Ai2 reported on 27 October 2026 that LLMs judging research novelty are prompt-sensitive: GPT-5.4 fell from 93.5% to 40.9% accuracy after one wording change.

Summary

The Allen Institute for AI (Ai2) published a study on 27 October 2026 testing whether large language models can judge if a research idea is novel. Using ideas drawn from ICLR 2026 reviews, it found verdicts swing with small prompt changes: GPT-5.4's accuracy on human-versus-generated pairs fell from 93.5% to 40.9%. A simple Claude Opus 4.6 prompt beat purpose-built novelty checkers at a fraction of the cost. Data and code are open.

The Allen Institute for AI (Ai2) published a study on 27 October 2026 testing whether large language models can judge if a research idea is novel. Using ideas drawn from ICLR 2026 reviews, it found verdicts swing with small prompt changes: GPT-5.4's accuracy on human-versus-generated pairs fell from 93.5% to 40.9%. A simple Claude Opus 4.6 prompt beat purpose-built novelty checkers at a fraction of the cost. Data and code are open.

Read the full story

More ai research stories

RSS