Act As a Real Researcher: A Suite of Benchmarks Evaluating Frontier LLMs and Agentic Harnesses in Research Lifecycle

AI systems are increasingly marketed as research assistants capable of literature review, hypothesis generation, and experiment design. But how honestly do existing benchmarks measure genuine research capability versus surface-level task completion? This work argues that current evaluations miss the subtle professional judgment that defines real scientific work — noticing a methodological flaw, flagging an ethical concern, catching an ambiguity that invalidates an experiment. Even top-performing configurations fall short of what a competent human intern would catch. For institutions considering AI in research pipelines, this benchmark offers a more honest stress-test and highlights exactly where human oversight remains indispensable. Authors: Jiayu Wang, Weijiang Lv, Bowen Fu, Jing Fu, Jiayi Song, Lingyu Zhang, Lanxuan Xue, Luodi Chen, Zepeng Xin, Kaiyu Li, Xiangyong Cao Paper: https://arxiv.org/abs/2606.07462v1