Ai2 has released a study examining whether large language models can reliably judge if a research idea is new. The work, titled "Old Ideas, Novel Problems," comes with a benchmark, data, code, and an automated pipeline that researchers can use to build their own evaluation sets. The researchers call these systems "novelty judges" and note that there is no standard way to build one. Teams choose the model, prompt, evaluation format, reasoning budget, and sometimes a literature-retrieval system, and each choice can change the verdict. In testing six judges, GPT-5.1, GPT-5.2, GPT-5.4, Claude Sonnet 4.5, Claude Opus 4.5 and Claude Opus 4.6, the team found that small changes to the setup produced very different results. In one experiment, changing a few words in a prompt shifted a model's accuracy at identifying the more novel of two ideas by more than 50 percentage points. Adding retrieval or extra reasoning did not consistently improve results. Two purpose-built novelty evaluation systems performed worse than a simple prompted Claude Opus 4.6 baseline. The benchmark was built from ICLR 2026 peer reviews, using reviewer comments on originality rather than paper acceptance as the measure. It includes 154 high-novelty human ideas and 145 low-novelty ones, along with a second setting that pairs human ideas against ideas from a basic AI generator. In a 30-pair expert check, the expert preferred the human idea every time and judged none of the generated ideas novel. The researchers caveat that the generated ideas were assumed to be less novel, so those labels are weak rather than verified for every idea. For Australian businesses and professionals, this changes little today. The study is aimed at research teams building AI systems for scientific discovery, not at everyday users of AI tools. It does suggest a practical caution: if your organisation uses an LLM to screen ideas, rank proposals or judge originality, a single model and prompt may not give stable answers. Testing a few prompt wordings and models on your own examples before relying on the output would be a sensible low-cost step. No Privacy Act or regulatory obligations are raised by the research itself.