OpenAI has released GeneBench-Pro, a new benchmark that measures how well AI agents can make sound judgments in computational biology research[1]. Made up of 129 problems spanning genomics, quantitative biology, and translational medicine, it tests not the recall of facts or the execution of predefined workflows, but the kind of researcher-level judgment needed to choose an analytical path from ambiguous data. Even the latest model scores around 30 percent, making it a notable yardstick for whether AI can assist human experts in real scientific work.
Asking for the Path to the Answer, Not Knowledge
What GeneBench-Pro tries to measure is not the ability to answer facts. Scientific data does not arrive with instructions; researchers must judge whether a pattern reflects real biology or mere noise, whether the data can support the question being asked, and how each result should change what they do next[1]. OpenAI calls this chain of judgment "research taste" and makes its quality the target of evaluation.
In each problem, the model is given a realistic, messy dataset, a brief experimental context, and a target estimand tied to a downstream decision. The model must explore the data, choose an appropriate analytical approach, work through an iterative process, and supply a final answer. For the working environment, each problem provides an isolated workspace stocked with a standard bioinformatics stack including Python and PLINK 2.0[1].
129 Problems Built Synthetically and Vetted by Experts
Existing benchmarks often build multi-step questions around messy historical data, where there may be no single correct path and the evaluation ends up reflecting the arbitrary choices of the author. To avoid this, GeneBench-Pro builds each problem synthetically, simulating the entire causal structure of the data-generating process[1]. This lets the team tune each problem's difficulty, ensure that reasonable differences in analytical interpretation still land within accepted numerical ranges, and verify that plausible but incorrect analyses properly fail.
OpenAI also sent 82 of the 129 problems to external domain experts, including graduate students, postdoctoral researchers, industry scientists, and professors[1]. The experts assessed each problem's realism, whether the target answer was identifiable, and whether the methods and estimators were appropriate, and their feedback was used to improve the problems. Reviewers noted that the data contained technical and quality-control issues that demanded thoughtful, reflective analysis rather than the off-the-shelf application of a method. The problem set, along with 10 representative questions, is published on Hugging Face with an interactive web interface, and a 50-question subset will be provided to the third-party organization Artificial Analysis[1].
Even the Latest Model Scores About 30 Percent
On the actual results, OpenAI's strongest model, GPT-5.6 Sol, posted a pass rate of 28.7 percent at the highest reasoning level, rising to 31.5 percent with Pro mode enabled[1]. That is a sharp jump from when the original GeneBench was first being built, when the best frontier model at the time, GPT-5, scored below 5 percent. OpenAI suggests that progress is rapid even on hard-to-measure, systems-level scientific reasoning, and that the benchmark could be saturated by the end of the year.
The effect of test-time compute is also striking: at the lowest reasoning level GPT-5.6 Sol manages only a single-digit pass rate, while at the highest level it solves about six times as many problems as GPT-5.2 using roughly two-thirds the tokens[1]. Across model families, the gap with strong open-source models such as GLM 5.2 is larger than one would expect from coding benchmarks, suggesting open-source models are more specialized for coding. At the same time, the fact that even the latest models solve fewer than a third of the problems shows there is ample room for improvement.
Turning 20 to 40 Hours of Expert Work into a Few Dollars
GeneBench-Pro also raises a question about the economics of research. Reviewers estimated that a typical problem would take a human expert around 20 to 40 hours to complete[1]. Even at a conservative 200 USD (about 32,000 yen) per hour, the labor cost per problem reaches into the thousands of dollars (roughly several hundred thousand yen). By contrast, an AI agent's inference cost is only a few dollars per problem (about several hundred yen).
※1 USD = 162 JPY (as of June 30, 2026)
Current AI is still too unreliable to be handed an entire analysis from start to finish, but the cost gap is enormous, and even partial automation at today's capabilities could create meaningful economic and scientific value[1]. With sequencing costs having fallen dramatically and biobank-scale datasets now linking molecular, phenotypic, and health-record information, the bottleneck is shifting from data generation to turning that information into actionable insight[2]. If AI can reliably handle tasks such as hypothesis triage and follow-up on drug targets, it could become a real accelerant for the pace of research.
Summary
GeneBench-Pro is an attempt to measure how far AI has come on the core scientific skill of choosing a sound analytical path amid ambiguity, rather than on the volume of knowledge or routine processing. While even the latest GPT-5.6 Sol scores around 30 percent, the speed of progress and the sheer cost gap show that AI is edging toward practical use as an assistant in research settings. As model capabilities grow more sophisticated, benchmarks that probe these higher-level abilities look set to become ever more important.
出典:https://openai.com/index/introducing-genebench-pro
出典:https://www.biorxiv.org/content/10.64898/2026.06.29.735386v1
