Jiacheng Miao is a Stanford HAI postdoctoral fellow working with Jonathan Pritchard and James Zou. His homepage lists two research questions side by side under the heading "Agentic Science": how to make scientific knowledge agent-native, and how agentic AI might distort science if it is deployed without guardrails. Within a few months he published a tool that answers the first and an experiment that sharpens the second.
The tool is Paper2Agent, which appeared in Nature in September. It takes a computational research paper together with its code repository and turns the paper's methods into functions that other software can call — an MCP server, in the current vocabulary, meaning an AI assistant can invoke a paper's analysis the way it would invoke any other service. Of 100 computational biology papers sampled from the bioRxiv preprint server, 74 converted with no manual intervention, producing 593 tools that passed automated checks. Miao designed and developed the framework, ran the case studies, and co-drafted the manuscript.
Paper2Agent is not a photocopier pointed at a repository. Its six stages are nearly all interpretive. The pipeline classifies which files count as usable tutorials. It decides which analytical steps generalize and which are accidents of whatever example dataset the authors happened to have. It swaps hard-coded file paths, thresholds, and column names for parameters, and in doing so settles what gets a sensible default and what the caller must supply. In the case study built on Scanpy, a widely used single-cell analysis toolkit, the agent inferred the organism, the mitochondrial-gene prefix, and the marker-gene panel from each new dataset rather than asking for them. A working biologist would make each of those choices deliberately and could defend them. Here they are made inside execution, and nobody reviews them.
Validation then checks whether the resulting tool reproduces the paper's own reference output: numbers within 3 percent, figures compared by perceptual hash, an image fingerprint tolerant of small rendering differences. Tools still failing after six repair attempts are dropped. The authors are careful about what this establishes. For open-ended scientific analysis, they write, agreement with one reference answer measures faithful execution more directly than analytical validity, because more than one answer may be defensible.
That caveat looks different alongside Miao's other recent work. In a preprint with Pritchard and Zou, 40 AI agents analyzed the same immigration-and-welfare dataset that 71 human research teams had analyzed before them. Twenty agents received a short persona paragraph leaning toward one hypothesis, twenty the opposite. Same data, same model, same harness, same budget of 100 candidate specifications — 100 defensible ways of setting up the analysis. Only the persona differed. Among the 42 human teams whose prior beliefs had been classified as pro- or anti-immigration, the gap between the two groups' reported effects served as the benchmark. The agents reproduced 72 percent of it.
Miao has not publicly said how the two projects fit together, and the connection should not be put in his mouth. But publishing them in the same season is uncommon early in a career: the capability and the blind spot, with the second complicating the first. One paper shows that a method can be made reliably callable. The other shows that a successful execution settles very little about the judgment inside it.
The distinction travels beyond computational biology. Any system that converts an expert's artifact into a callable service has to decide what the expert left unstated. A pass rate tells you the conversion ran. It does not tell you whose judgment ended up in the defaults, because the discretionary parts are exactly the ones with no reference output to check them against.
- Process checks reveal more work: An October workshop paper found that among enterprise agent runs passing final numerical checks, over 92% still had evaluator-detected deviations in tool choice, execution order, or scope — a concrete measure of how much outcome-only validation misses.
- Agents steered by dark patterns: A peer-reviewed ICLR study reports that deceptive interface patterns caused unwanted outcomes in over 70% of tasks for tested web agents, compared with 31% for humans, and that prompting defenses did not consistently reduce the effect.
- Reviewer composition changes trends: A preliminary working paper on production evaluation found that a change in reviewer composition flattened the apparent reliability trend of a deployed agent, raising questions about what human evaluation actually measures when the evaluators themselves shift.
- Identity standards still fragmented: NIST received over 600 responses to its agent-identity concept paper and summarized agent identity as a compositional standards stack rather than a single standard, with no interoperability results or deployed conformance evidence yet published.
