# Shared knowledge for condensed matter physics: What one lab learns should help every other lab

David Sendor, v0.1 draft. A draft for people in the field to take apart. Every claim carries a source and a label: Human-checked or AI only.

> My goal is a condensed matter physics where what one lab learns helps every other lab. What was tried, what didn't work and which simple explanations were already ruled out should be easy to find, and AI should help researchers work through it. This is an open science effort, built on open-source tools made for data that labs want to keep private.
>
> None of this works unless the people who decide what counts buy in, above all journal editors and the senior, tenured academics who hire, promote and referee.

## The story in 6 parts

1. **The vision.** Where this could go: from today's pain points to a field that shares what it learns, and which phase gets each one there.
2. **Phase 1: an agentic research assistant.** Open-source tools that know condensed matter and run on any AI model: one for researchers before they submit, and a sibling for referees.
3. **Phase 2: a record of what's been tried.** A quick, citable home for the results that rarely become a paper: disappointing and unexciting nulls, modest positive results, and approaches tried and dropped. The field can see what has been tried and who tried it. Built on open repositories that already exist.
4. **Who has to act, and what would move them.** Journal editors and senior, tenured academics decide what counts. Here's why each group would say yes or wait, and what would move them.
5. **What am I missing?.** The questions I most want answered, from people who do the work.
6. **Appendix.** The detail behind the story: the evidence for each pain point, the long-term design, the systems underneath and how I'll test it.

## 1. The vision

Page: /where-this-goes/

Where this could go: from today's pain points to a field that shares what it learns, and which phase gets each one there.

Picture where this ends up. A student sees a new zero-bias peak. Within the hour, an agent shows her every shared device in the field that showed something similar, the ordinary effects known to produce it, the labs that ruled them out and how, and shared data from similar devices that shows whether the ordinary explanation is common, before she builds a new one.

#### From today to where this could go

| Pain point | Today | Where this could go | Starts in |
|---|---|---|---|
| New phenomena are hard to interpret | A lab, its collaborators and its referees judge a new signal against the simple explanations they happen to know. | A claimed signal comes with the lookalikes other labs have seen and the checks that rule them out. | Phase 1 |
| Nobody can know all the related work | You know your corner of the literature and what colleagues happen to mention. | An agent finds every similar experiment, including the ones that found nothing, and the people who ran them. | Phase 1 |
| What labs learn stays in the lab | Data, failed runs and replications stay on lab drives, and are often lost when a student leaves. | What's been tried is shared in minutes, with the data if the lab chooses, and labs that want to can keep every run. | Phase 2 |
| The exciting reading gets rewarded | The exciting interpretation is published, and the simple one is never fully tested. | Testing the simple explanation is a standard step, so careful work isn't at a disadvantage. | Phase 3 |
| Ruling out the simple explanation is expensive | Only your lab can test it, often with a whole new experiment. | Data other labs shared from similar devices shows whether the artifact is common, a first check before a new experiment. | Phase 3 |

The evidence for each pain point and the long-term pieces of phase 3 are in the appendix. [Phase 3 in detail](/appendix/phase-3/)

## 2. Phase 1: an agentic research assistant

Page: /phase-1/

Open-source tools that know condensed matter and run on any AI model: one for researchers before they submit, and a sibling for referees.

Phase 1 is a set of open-source agent tools that any lab can run on Claude, ChatGPT, Gemini or another model. The lab chooses where the model runs: a commercial model under its institution's data agreement, or an open model run locally for data that can't leave the lab. Nothing is hosted centrally; a shared, hosted record comes only with the optional run log in phase 2.

Much of this exists already. Frontier models read papers, code and modest datasets, and literature tools such as scite, Consensus and PaperQA2 already search and weigh the literature. What's new, and not built yet, is the condensed matter knowledge behind the tools and one workflow a researcher runs before submitting. Building that is phase 1. [c-scite][c-consensus][c-paperqa]

#### The research assistant

- Reads what the researcher gives it: the paper, the figure data and the analysis code.
- Lists the known lookalikes for the claimed signal (the trivial explanations and artifacts that can produce the same signature), which ones the paper doesn't rule out, and which control would test each, with related work.
- Asks how many devices were measured, how many showed the effect, and why the others were set aside.
- Packages and tags the data behind the paper, so sharing it later takes one step.
- Writes a report for the researcher. It stays theirs: it goes to a journal or referee only if they send it, or, later, if a journal runs the assistant itself.

#### The referee assistant

A sibling tool for referees, hosted by the journal in its own approved deployment. Many journals prohibit referees from uploading manuscripts to AI tools, to protect confidentiality, so the referee assistant depends on journals offering it. I expect that to change.

#### How they know the field

Through an open library of known lookalikes and curated literature that anyone can extend. It starts in whichever subfield is most open to trying it, seeded from that subfield's public sources. I use topological transport as the example on this site because its lookalikes are well documented: there, the seed would be Frolov and co-authors' 2026 Science paper on four 'smoking gun' signatures and their non-topological explanations, the Pittsburgh report, and a 2020 review of Andreev and Majorana bound states. The library grows with each subfield that uses it. [c-frolov-science][c-report-recs][c-prada]

Who curates that library is a question I want to raise. Beyond review articles, the field has no curated list of lookalikes, and curating data, not just papers, is work nobody is clearly paid or credited for. One path is a review of known lookalikes in a major journal, which gives its authors credit, kept current as a living review, the way Living Reviews in Relativity has authors update their articles. Longer term, journals could curate data as well as articles, as the American Economic Association's Data Editor already checks the data and code behind its papers. [c-living-reviews][c-aea]

#### Why start here

My hypothesis is that the software is cheap to build and asks nothing of a lab beyond the paper it already has. The library is the real work, which is why who curates it matters. Every use adds to it, and to the tagged data that phases 2 and 3 need.

#### What it costs

Philanthropy pays to build the tools and curate the library, researchers pay for their own model use, and a journal that offers the referee assistant pays for its own deployment. Since many of the pieces exist, the open question isn't whether this can be built. It's whether researchers want it and whether it can be kept going. They already use AI widely: in a 2025 survey of 2,430 researchers across fields, 84% used AI tools in their work, up from 57% a year earlier. [c-wiley-ai]

My hypothesis is that running the assistant costs little next to current journal fees; a nonprofit certifier in France already reruns a paper's code and data for 500 euros. The larger cost is people's time, mostly answering the assistant's questions, so the goal is tools automated enough that this takes minutes. [c-cascad]

## 3. Phase 2: a record of what's been tried

Page: /phase-2/

A quick, citable home for the results that rarely become a paper: disappointing and unexciting nulls, modest positive results, and approaches tried and dropped. The field can see what has been tried and who tried it. Built on open repositories that already exist.

High-profile journals mostly publish the exciting results: positive ones, and nulls that refute a prominent claim. Some of the middle reaches specialist journals, and much of it never becomes a paper: the disappointing null, the unexciting null, the modest positive result, the approach tried and dropped. Some of it gets said at conferences or over coffee, but it rarely becomes something anyone can find later.

How much is lost hasn't been measured in condensed matter. In one set of 221 social science experiments, strong results were 60 percentage points more likely than null results to be written up at all. Phase 2 is a quick way to record the rest, so the field can see what has been tried, what has been explored and who tried it. [c-franco]

#### How it works

- A short note, written with the assistant in minutes: what was tried, under what conditions, and what happened.
- An effort level, from a quick try to months or years of work, with how many samples or devices and their quality.
- Tags the assistant fills in: material, device, technique, conditions and outcome.
- Data attached if the lab wants, often already packaged in phase 1.
- The lab chooses who sees it: private, named collaborators, public, or public after an embargo.
- Each note names who did the work and gets a DOI, so it can be found and cited.

The effort level matters because a note could wrongly bury an idea: a quick try on a poor sample isn't evidence that an effect isn't there. Often the write-up shows it, since a few lines read differently from a detailed report, but a tag makes it explicit and searchable.

#### Built on what exists

Rather than a new database, phase 2 adds a condensed matter layer for tagging and search on top of open repositories that already issue DOIs and hold data. Zenodo registers a DOI for every upload, and NOMAD, built for materials data, already lets uploads be shared with reviewers, embargoed and frozen once published. Everything stays open source. [c-zenodo-doi][c-nomad-review]

#### Why it matters

Much of the barrier is effort, not competition: the leading reasons scientists give for not sharing data are lack of time and funding. And failed results have value: in chemistry, failed reactions kept in lab notebooks trained a model that predicted successful syntheses better than the chemists who ran them. [c-tenopir][c-raccuglia]

#### Optional: a log of every run

Labs that want to can go further and log every run as it happens, failed ones included, to a shared record. Why a lab would want it: nothing is lost when a student leaves, methods and figures are easier to write, the assistant can count the devices measured instead of asking, and the count of devices measured rests on a time-stamped record rather than memory.

It starts optional. The long-term goal is for it to become the field default, as depositing structures did in structural biology and registering trials did in medicine, once labs find it worth the effort. [c-pdb][c-ctgov] [The run log in detail](/appendix/the-run-log/)

## 4. Who has to act, and what would move them

Page: /decision-makers/

Journal editors and senior, tenured academics decide what counts. Here's why each group would say yes or wait, and what would move them.

#### The path

- Researchers use the assistant before submitting, if it catches problems before a referee does.
- Journals run it on submissions and give the report to referees, wherever their rules allow.
- Journals offer it to every author before submission.
- Labs record what they've tried, and those that want it start logging every run.
- Once the field values them, the report and the run log become the default, as deposit and registration did in other fields.

- **Physics journals.** Why they would: Claims that are never confirmed and quietly fade cost journals trust, and retractions most of all. Physical Review journals already require a data availability statement, so this is the next step on a path they're on. Why they wouldn't: Checks cost money and can slow review, and authors may take papers to journals without the requirement. What moves them in phase 1: Offering the assistant to authors, and to referees through the journal's own deployment, piloted in one journal first. Long term: Making the report, and later the run log, the default once the field values it, with a place for notes on what didn't work. [c-aps-policy]
- **High-profile multidisciplinary journals.** Why they would: Their names are on the most visible retractions. They already publish referee reports and offer reviewers private access to data and code. Why they wouldn't: Novelty and speed are what they sell. What moves them in phase 1: Offering the referee assistant through their own approved deployment. Long term: Visible cases, and a competitor moving first. [c-ncomms-files]
- **Private foundations.** Why they would: They can move fast, care about the health of a field and not only single results, and have launched shared infrastructure before. Why they wouldn't: Shared infrastructure is slow to pay off, and grantees may see new tools as extra work. What moves them in phase 1: Paying to build the assistant and curate the library, as a pilot with willing grantees. Long term: Paying for the shared record, and a view of the field that shows where money is going and where it isn't. [c-cos-funding]
- **Government funders.** Why they would: They already require data management plans, and they pay for the false leads. Why they wouldn't: Enforcing plans costs money, and attention is elsewhere. What moves them in phase 1: Tools that help grantees meet the data plans they already require. Long term: A view of what the field is working on and where nobody is investing, from records that cost them nothing to collect.
- **Lab heads and senior academics.** Why they would: Nothing is lost when a student leaves, methods and figures are easier to write, revisions are faster, and a collaborator's mistake gets caught before publication. Why they wouldn't: Time, fear of being scooped, and exposing mistakes. What moves them in phase 1: An assistant that catches problems before a referee does. Long term: Logging that runs in the background with tools they already use, and credit that counts for grants and hiring.
- **Students and postdocs.** Why they would: Their careers absorb the cost when a claim falls apart. The Pittsburgh report counts diverted careers, often of young researchers, among the costs of every disproven claim. Why they wouldn't: They have the least power and the most to lose. What moves them in phase 1: Senior people going first, and tools that package their data and analysis as they go. Long term: Reproducible work showing up in the papers and letters that hiring committees read. [c-report-careers]
- **Universities.** Why they would: Integrity cases are costly, and the Pittsburgh report asks them to count datasets and code as research outputs. Why they wouldn't: Hiring and promotion run on publication metrics. What moves them in phase 1: Approving the tools under their existing AI and data agreements. Long term: Peer institutions and funders moving first, and counting datasets and code in hiring and promotion. [c-report-institutions]
- **Quantum computing and AI materials companies.** Why they would: Their roadmaps depend on the physics being right, and AI models need measured data they can trust. Why they wouldn't: Their own data is proprietary. What moves them in phase 1: An open lookalike library they can use and add to. Long term: Shared, checked measured data they can train and test against.

One adopter showing results is the example others copy. A single journal offering the assistant to its authors and referees, or a single funder paying for it for its grantees, would be enough to start.

## 5. What am I missing?

Page: /what-am-i-missing/

The questions I most want answered, from people who do the work.

I'm testing whether this fits the field, and the people who do the work are the ones who can tell me.

**For lab heads and students**
- Which of these pain points costs your lab the most time? Which did I miss?
- Which subfield would be most open to trying this first?
- Where does your data live today, and which tools write it?
- If logging were nearly automatic, would it still be a burden?
- Would you run an assistant like this on your paper before submitting? What would it need to catch to be worth it?
- Would you trust a third-party service with your raw data, if only the assistant reads it? What would it need to promise?

**For journals and funders**
- Would you offer the assistant to your authors and referees? What would it need to cost per paper?
- Would a view of what the field is working on, and where nobody is investing, help you decide what to fund?
- As a referee, would you trust the assistant's report and the researcher's answers?
- Who should host a shared system of record for this field?
- Who should curate the field's data, such as a library of known lookalikes? Could journals take that on?

**For everyone**
- Which step would you adopt tomorrow, and which would you fight?
- What result would make your journal, funder or department pay attention?
- What's wrong or missing here?

How I plan to test each guess is in the appendix. [How I'll test it](/appendix/how-ill-test-it/)

Write to me at david@sendorai.com.

## 6. Appendix

Page: /appendix/

The detail behind the story: the evidence for each pain point, the long-term design, the systems underneath and how I'll test it.

Everything here supports the main story. None of it is needed to follow it.

### The pain points in detail

Page: /appendix/the-pain-points/

Five pain points, each labeled with how well it's documented, and how anyone could measure how big the problem is.

These are the pain points as I see them, from the literature and from conversations with physicists.

- **New phenomena are hard to interpret** (cases). A new signature often has ordinary lookalikes. In hybrid nanowires, zero-energy modes that resemble the Majorana signal can emerge without any topological transition. Four more 'smoking gun' signatures turned out most likely not topological once more of the data was examined (the Majorana example in this appendix). Documented for topological signatures. How often it happens across the field hasn't been measured. [c-prada][c-frolov-science]
- **Nobody can know all the related work** (opinion). arXiv listed about 17,000 new condensed matter papers in 2024, more than 45 a day, and what didn't work mostly never becomes a paper. A lab head put it to me plainly: we usually don't know all the literature related to our experiment, and that is a common root of an honest misreading. The volume is measured. How often missed work leads to a misreading is a working physicist's judgment. [c-arxiv-2024][c-notes-misreading]
- **What labs learn stays in the lab** (sourced). In a sample of 2014 articles from top journals in four fields, physics shared original data at the lowest rate, and asking authors directly works only part of the time: 39% of requests to Nature and Science authors succeeded. Physical Review journals require a statement saying where the data can be found, but public sharing is optional. Failed runs are seldom written up, and replications struggle to get published: journals turned down one group's replications as not novel. Sharing rates and reasons are measured in physics overall and in other fields. How often condensed matter data is lost hasn't been measured. [c-womack][c-data-requests][c-aps-policy][c-pittwire-rejected]
- **The exciting reading gets rewarded** (opinion). Publishing high matters for funding and reputation, so it's natural to prefer the more exciting interpretation and to spend less time on the ordinary effects that could produce the same observation. It sits on a spectrum, and mostly comes down to where the time goes. The Pittsburgh report adds that reputation and authority often stand in for checking when readers decide which results to trust. The judgment of physicists in the field: a lab head I've talked with, and the report's authors. [c-notes-misreading][c-report-trust]
- **Ruling out the simple explanation is expensive** (opinion). Sometimes the only test of a trivial explanation is another experiment, on a modified device where the exciting effect shouldn't appear but an artifact would. That can take months. Samples also vary: maps of magic-angle graphene show the local twist angle varying by close to 0.1 degree within a single device, so one device seldom settles a question on its own, and confirmation from other devices often comes later. The variation is measured. The cost of control experiments is a working physicist's judgment. [c-notes-misreading][c-twist-angle]

#### How big is the problem?

Nobody knows yet. I found no base rate for how often headline claims in condensed matter fail, and no systematic replication study of the field. The evidence is a run of costly corrections in the hottest subfields, plus structural signs: little data shared and few replications published.

Other fields have measured their problem. A mass reproduction of 110 economics and political science papers found coding errors in about a quarter of them, even though most claims still reproduced. [c-i4r-errors][c-mass-repro]

Physical sciences publish fewer positive results than social sciences, so publication bias may be smaller here than elsewhere. That count can't include null results that were never written up: 'we looked and saw nothing' leaves no paper. [c-fanelli]

Three cheap ways to measure it:

First, read published referee reports. Nature Communications publishes them for papers submitted since November 2022, so you can count how often referees asked for data and whether they got it. [c-ncomms-files]

Second, the test of the phase 1 assistant doubles as a reproduction study of the field, like the Institute for Replication's in economics. [How I'll test it](/appendix/how-ill-test-it/)

Third, pick one subfield, list its headline claims since 2010, and record which still stand.

### One example: the Majorana signals

Page: /appendix/one-example-majorana/

A new signal, ordinary lookalikes, and a selection of data that favored the exciting reading. It took years to sort out.

In 2018 a Nature paper reported quantized Majorana conductance, a signature that topological quantum computing was meant to build on. In 2021 the authors retracted it, after Sergey Frolov and Vincent Mourik pointed out inconsistencies between the raw data shared with them and the published figures. [c-majorana-retraction]

The lab behind it then shared the data of its retracted papers in public repositories and adopted a policy of publishing all the raw data behind its figures, with the processing scripts. [c-qutech]

In January 2026 Frolov and eleven co-authors looked at four more 'smoking gun' signatures of topological effects in Science. Limited data matched each one. With more of the data, the most likely explanations were trivial, meaning not topological. They argue that releasing complete datasets, discussing other explanations and saying how much was measured in total would make such claims more reliable. [c-frolov-science]

Part of this is structural. Samples vary from device to device, and showing the clearest device is often legitimate: disorder, bad contacts and gate leakage hide the physics in many devices. But it can also work like testing many hypotheses and reporting the one that fit, and from the paper alone a reader can't tell which: how many devices were measured, how many showed the effect, or why the others were set aside.

The assistant asks for that count and lists the lookalikes to rule out. With a run log tagged by device, the count comes from the record. [Phase 1](/phase-1/)

### Phase 3 in detail

Page: /appendix/phase-3/

The long-term pieces, each my hypothesis, with the pain point it addresses and what it would take.

- **A knowledge sharing platform for the field** (my hypothesis; addresses: What labs learn stays in the lab, Nobody can know all the related work, Ruling out the simple explanation is expensive). The long-term goal: a shared record of measured data, failed runs, controls and analyses that labs contribute to and draw from, with each lab in control of what's shared and when. Failed runs would count too, as phase 2 starts to show. Sharing has to be easy: an agent that packages the data and code behind a paper could remove most of that work, starting with papers already published, while the data still exists: in biology, the odds that a paper's data could still be found fell by 17% for every year after publication. Physics already has pieces of this: HEPData holds the numbers behind the plots of several thousand particle physics papers, and NOMAD takes condensed matter data in a standard format. Where samples vary, pooling many groups' devices shows which features are common and whether an artifact appears on similar devices elsewhere, without each lab building the control experiment itself, though it can't rule out an artifact in one particular device. What it would take: A neutral host, funding to build and run it, data standards for the field's instrument formats, access rules labs trust, and enough early contributors that drawing from it is worth contributing, with enough sample detail to compare devices across labs. [c-vines][c-hepdata][c-nomad][c-twist-angle]
- **Reproducible science as the norm** (my hypothesis; addresses: New phenomena are hard to interpret, What labs learn stays in the lab). Every published result can be rerun from its data and code, and the field moves there as a whole rather than lab by lab. Economics shows it can be done: its main society checks every computational article before publication. What it would take: Journals and funders that ask for it, tools that make it cheap, and a check before publication. Analyses with interactive steps done by hand are the hard part. [c-aea]
- **An AlphaFold for materials, as far as that's possible** (my hypothesis; addresses: New phenomena are hard to interpret). AlphaFold was trained on the Protein Data Bank's measured structures, an archive that journals have required deposits into since 1989 and that passed 200,000 entries in 2023, all in one standard form. Materials have a partial equivalent for structure and much less for measured properties, and values mined from papers by software are imprecise: one text-mined set of magnetic transition temperatures was about 72% precise. A shared record of measured properties, failures included, is what such a model would need, and it would be harder than for proteins: transport and STM data describe a device with its contacts, not only a material. What it would take: Measured properties such as transport, spectroscopy and phase diagrams, with the sample details and conditions behind them, in a standard form, from many labs, including the runs that showed nothing. [c-alphafold][c-pdb][c-pdb-size][c-kelly]
- **A map of where the field invests** (my hypothesis; addresses: The exciting reading gets rewarded). A map of where people research and publish, what's being funded, and where nobody is working, so funders can see where money would do the most good. In biomedicine, scientists follow conservative strategies near well-studied molecules, which slows collective discovery, and in materials science, research concentrates on fewer concepts over time. Nobody has measured this for condensed matter. A solar physicist pointed out a catch to me: a funding gap isn't always a scientific gap, because some areas get little money because they're largely solved. What it would take: Publication and funding data, plus a way to tell open questions from solved ones, which probably needs physicists' judgment as well as data. The map has to find real scientific gaps, not just quiet areas. [c-rzhetsky][c-marwitz][c-notes-solar]
- **Credit for less exciting work** (my hypothesis; addresses: The exciting reading gets rewarded, Ruling out the simple explanation is expensive). Controls, replications, null results and careful ruling-out of simple explanations count for funding, hiring and publication. Today they often don't: journals turned down one group's replications of condensed matter claims as not novel or no longer relevant, and the paper that finally appeared in Science spent what the university called a record two years under review. The Pittsburgh report asks funders, publishers and universities to count this work. Other fields have one working model in Registered Reports, a format in which the plan is peer reviewed before the data is collected and the paper is published regardless of outcome. What it would take: Journals with a place for this work, funders and hiring committees that count it, and a record that makes it visible. Registered Reports fit confirmation experiments better than exploratory ones. [c-pittwire-rejected][c-review-two-years][c-report-institutions][c-registered-reports]
- **Agents across the research workflow, building on phase 1** (my hypothesis; addresses: Nobody can know all the related work, Ruling out the simple explanation is expensive, The exciting reading gets rewarded). The phase 1 assistant is the start. Built out, the same agents would walk a researcher from the literature review to publication, and do the less exciting work that often gets skipped: packaging data and code, testing simple explanations against shared data, rerunning analyses, and reanalyzing old data with new methods. General models still struggle with condensed matter reasoning: on 50 expert-written theory problems, the best model in 2025 solved 30%. Old data can pay off: particle physicists revived data from a collider that shut down in 1986 and reanalyzed it with modern theory and simulation, and kept raw data makes the same possible for old transport, STM or ARPES measurements. What it would take: The library phase 1 starts to build, extended to instruments, data formats and known artifacts; access to the full text of the literature; and data and code in a form agents can run. Condensed matter analyses are larger and less standard than the cases tested so far. [c-cmt][c-jade]

Nearly all of this needs two things phases 1 and 2 start to build: a library that knows the field, and shared data, failures included.

### Why now: what AI makes cheap

Page: /appendix/why-now/

My hypothesis is that AI now makes the work behind shared, reproducible research cheap. The evidence so far comes from social science and machine learning, not lab data, so testing the assistant on condensed matter papers is the clear next step.

Reproducible research has always cost time: documenting each step, packaging data, rerunning someone else's analysis, checking the literature. Those are the tasks AI is getting good at.

- **Repairing broken replication code.** Agent workflows fixed deliberately broken analyses in 69 to 96% of cases, across five social science studies. [c-shah]
- **Reproducing published results, with people.** AI-assisted teams reproduced 91% of results and human-only teams 94%, across 103 teams, and human-only teams found more major coding errors. AI didn't help here. [c-ai-assisted]
- **Reproducing a machine learning experiment.** About $1.87 per experiment, on a benchmark of 13 AI papers. [c-autoreproduce]

Some lab software is ready for it. Many quantum device labs run measurements through open-source software that saves each measurement with its settings, though tuning done live often isn't saved, and AI models can now work with those instruments through the same software. [c-qcodes][c-instrmcp]

In one quantum device lab I've talked with, AI already writes instrument drivers and runs measurement sweeps. An experiment run this way logs every setting and every failed run as a side effect. [c-notes-lab]

#### The limits

- The evidence is from social science and machine learning code. Nobody has tested an assistant like this on condensed matter papers and lab data, which are large and often in instrument-specific formats.
- AI makes checking an analysis, capturing data and packaging it cheap. It doesn't make building a device or growing a sample cheap.
- Open-source measurement software is common in quantum device labs. Many other labs run LabVIEW, MATLAB or vendor software for STM, PPMS and ARPES, and each would need its own connector for the run log.
- It can't tell whether the instruments and samples are right, and control experiments stay expensive.
- On its own, the phase 1 assistant can only ask about the devices that weren't shown; it can't see them. That's what the optional run log adds.

How I plan to test the assistant on condensed matter papers is in How I'll test it. [How I'll test it](/appendix/how-ill-test-it/)

### What physicists propose and other fields already do

Page: /appendix/what-already-works/

Physicists have asked for most of this. Economics, crystallography, structural biology and particle physics already run versions of it.

#### Inside the field

The Pittsburgh report, published in Physical Review B in March 2026, gives recommendations for researchers, publishers, universities and funders. It asks for primary data with every paper, full parameter ranges, the number of samples tested, and data access for every co-author. [c-report-prb][c-report-recs]

It also asks funders to consider paying for replications, publishers to set acceptance criteria for replication papers, and universities to count datasets and code as research outputs. I see the assistant as a smaller, cheaper first step toward the report's goals, meant to sit alongside it. [c-report-institutions]

Frolov's group practices what it recommends. He found and sent a colleague 15-year-old raw data in under 20 minutes, and tells his students to be ready to do the same. He points to astrophysics, where sharing data is normal. [c-pittwire-practice]

#### In other fields

Each of these already does part of what this proposal builds toward: a neutral home for data, a check before publication, or a norm the whole field came to follow.

- **AEA Data and Code Repository** (Economics). Authors of empirical papers accepted by AEA journals deposit their data and code in a shared repository run by a university data archive, and the Data Editor reruns every one with a computational component before publication, about 400 articles a year. A society can require deposit with a neutral host and check before publishing. [c-openicpsr][c-aea]
- **ClinicalTrials.gov** (Medicine). Since 2005 the major medical journals consider a trial only if it was registered in a public registry, such as the one run by the US National Library of Medicine, before the first patient enrolled. Journals can make a time-stamped plan a condition of publication, and a field can move to it quickly. [c-ctgov]
- **Crystal structure deposition** (Chemistry and crystallography). Authors deposit structure files before publication, where they stay confidential but open to referees, and an automated check flags inconsistencies for reviewers. Reviewer-only access and automated checks already work for physical-science data. [c-ccdc]
- **Protein Data Bank** (Structural biology). Since 1989, no deposit, no structure paper: reviewers get an automated validation report while the data stays unreleased, and deposit is free to authors. Decades later the archive trained AlphaFold. A field-wide archive, publicly funded, can become a condition of publication, and pays off in ways nobody planned. [c-pdb][c-alphafold]
- **NOMAD** (Materials and condensed matter). Germany's materials data platform lets uploads be shared read-only with reviewers, embargoed for up to three years, and frozen once published. The closest existing home in this field, built for its data formats. [c-nomad-review]
- **HEPData** (Particle physics). A university-hosted, publicly funded archive holds the numbers behind the plots of several thousand papers, each checked by a reviewer before release. Physics already has a third-party home for a whole subfield's published data. [c-hepdata]
- **Institute for Replication** (Economics and political science). A mass reproduction of 110 papers found more than 85% of claims computationally reproducible, and 72% of significant results held up under robustness checks. A field can measure its own reproducibility, cheaply and at scale. [c-mass-repro]

### The run log and a shared record

Page: /appendix/the-run-log/

Optional at first, from phase 2, with the long-term goal of becoming the field default in phase 3: labs log every run to a shared record, so the assistant can show how many devices showed an effect instead of asking. Here is one paper followed through the steps, ending with how the field would make it the default.

With a run log tagged by device, the assistant can compare the final data with everything measured, for example 3 of 40 devices shown, instead of asking. Logging is optional at first. In phase 1 the assistant runs wherever the lab runs it. For a lab that logs, the same assistant can also run inside the record, so it reads the log without the data leaving it.

There's no acceptable fraction. Devices differ by design, many die on cooldown or are used only for tuning, and whether one shows the effect depends on the range measured. So the count comes by category, as the lab defines it, and devices that die before measurement never produce a run. The record tracks information; it doesn't judge value. The count and the reason are there for readers to weigh.

#### One paper, step by step

Here's one paper from a quantum device lab, followed through the five steps. Switch between today and the run log to compare, and select a step to see what it does and what each person can see.

1. **Log every run.** For a lab that opts in, its measurement software sends every run and its settings to a shared system of record, a tracked archive run by a third party. A run here is one measurement the software saves, such as a gate sweep, not a whole cooldown. The record is encrypted: the lab can see its own runs, and otherwise only the assistant, run on the record operator's systems, reads them. Nothing depends on someone remembering to upload, so failed runs are kept too. Someone still records which device each run came from, and devices that fail before measurement never produce a run. Proves: A full, time-stamped record of what was done. AI and software: Connects to the lab's software, documents each step and names files consistently. Can see it: lab: Every run; reviewer: nothing; public: nothing. Today: Data sits on lab computers and students' drives. Failed runs are seldom written up, and data is often lost when people leave.
2. **Mark the final data.** The researcher tags the runs that go into the paper as final. Everything else stays in the log. Proves: Which data was selected, and out of how much. AI and software: Tags and packages the final data with the analysis that turns it into figures. Can see it: lab: Every run, final data marked; reviewer: nothing; public: nothing. Today: The team picks the data for the figures, often the clearest device and parameter range. The paper rarely says how many were measured.
3. **Submit.** The paper goes to the journal with the final data and the analysis code, and the assistant runs on it, started by the researcher before or at submission, so the same results can go to every journal. The full log stays sealed in the record. Proves: Exactly which data the paper rests on. AI and software: Builds the submission package. Can see it: lab: Everything; reviewer: Paper and final data; public: nothing. Today: The paper goes to the journal with its figures. A referee who wants the data has to ask through the editors, which is unusual.
4. **Assistant checks, researcher answers.** The agentic research assistant, running as a fixed workflow inside the record, reviews the research against all the data. It reruns the figures from the final data and leaves code another lab can rerun, compares the final data with the full log (for example, 3 of 40 devices shown), and asks which known lookalikes were ruled out. It sends any issues to the researcher, who corrects each one or explains it. The reviewer gets the paper, the final data, the report and that conversation, but not the full log and not an open-ended agent. If something looks off, the reviewer sends questions back. The researcher answers each one or lets the AI answer from the data, and can decline a question only by giving a reason the reviewer sees; the editor settles disagreements. This raises the floor without setting a limit: a reviewer can still ask for more data, as today, and the researcher decides what to share. The check doesn't reject a paper on its own. Proves: The figures follow from the data, and the selection is explained. AI and software: Reviews the research against a log too large for a person to read, checks the literature for known lookalikes, and holds the conversation with the researcher. Can see it: lab: Everything, plus the AI's questions; reviewer: Adds the AI report and conversation; public: nothing. Today: The referee judges the paper as written. Unpaid and short of time, they rarely rerun anything, and they can't see what wasn't shown.
5. **Make it the default.** Adoption starts optional: labs choose whether to log, researchers run the assistant before submitting, and journals offer it to authors and run it for referees. Once the field values it, as other fields came to value trial registration and structure deposition, journals and funders make the report and the log the default, so it's a favor to no one. The paper is published with a badge that says exactly what was checked: the figures were rerun from the final data, and every issue the check raised, including how the data was selected, was corrected or explained. It doesn't say the result is right. The lab decides whether to share the final data publicly. Proves: A floor the whole field shares. AI and software: Keeps it cheap enough that making it the default is acceptable. Can see it: lab: Everything; reviewer: Paper, AI report and conversation; public: Paper, check badge, data if shared. Today: The paper is published with a statement that data is available on request. Across fields, those requests succeed only part of the time.

#### A plan, only where it helps

The Pittsburgh report cautions that pre-registration may not help exploratory research, and much of condensed matter is exploratory. So a plan isn't one of the steps: the full log and the selection check already show what was tried. For a confirmation experiment, such as testing a claimed smoking gun, it's worth a few minutes to write down in advance what would count as the signature (which plateau value, over what gate range, on how many devices), and the record time-stamps it. [c-report-prereg]

Most of the pieces already exist in some form. The new ones are the shared record, the assistant's comparison of the final data with the full log, and a view of the field for funders: what's being worked on and what nobody is investing in. [The systems underneath](/appendix/the-systems-underneath/)

### The systems underneath, and who holds them

Page: /appendix/the-systems-underneath/

The systems behind the optional run log: ten pieces, most of which exist in some form today, held by one shared service run by a neutral third party so the time stamps mean something. The data stays encrypted and only the assistant reads it. Select any piece to see what it does, what already exists and who can see it.

- **Lab tools** (In the lab; exists; seen by The lab): The instruments, measurement software and analysis notebooks the lab already uses. Nothing here has to change. Exists today: [QCoDeS](https://microsoft.github.io/Qcodes/dataset/introduction.html), [Labber](https://www.keysight.com/us/en/products/software/application-sw/labber-software.html), [PyMoDAQ](https://github.com/pymodaq/pymodaq), [eLabFTW](https://doc.elabftw.net/).
- **Connector** (In the lab; partly; seen by The lab): Open-source software that sends each run and its settings to the system of record as it happens, through the tools' own interfaces. No manual upload, so nothing depends on someone remembering. Exists today: [NOMAD API](https://nomad-lab.eu/prod/v1/docs/howto/programmatic/publish_python.html), [Zenodo API](https://developers.zenodo.org/), [Globus Flows](https://docs.globus.org/guides/recipes/instrument-transfer-and-share/), [QCoDeS export](http://microsoft.github.io/Qcodes/examples/DataSet/Exporting-data-to-other-file-formats.html).
- **Researcher replies** (In the lab; new; seen by The reviewer): The researcher corrects each issue the assistant raises or explains it, in a conversation the reviewer reads.
- **Final data tag** (Held by a third party; new; seen by The lab): Marks which runs the paper uses. It points into the full log, so the assistant can compare the two.
- **System of record** (Held by a third party; new; seen by The lab): One shared service, run by a neutral third party. Every run is stored encrypted, with a time stamp and a fingerprint, so nothing can be changed quietly later. The lab sees its own runs; otherwise only the assistant, run on the record operator's systems, reads them, and nobody can run ad hoc queries. Exists today: [Zenodo](https://help.zenodo.org/docs/share/access-requests/), [NOMAD](https://nomad-lab.eu/prod/v1/docs/tutorial/upload_publish.html), [trusted time stamps (RFC 3161)](https://datatracker.ietf.org/doc/html/rfc3161).
- **Submission package** (At the journal; partly; seen by The reviewer): The paper, final data and analysis code, built automatically. The assistant runs from it, before or at submission. Exists today: [Figshare at Nature Physics](https://www.nature.com/articles/s41567-023-02040-4), [Code Ocean](https://www.nature.com/articles/s41567-025-02808-w), [Dryad](https://datadryad.org/help/submission_steps/peer_review).
- **Research assistant** (Held by a third party; new; seen by The reviewer): For a lab that logs, the assistant also runs as a fixed workflow inside the record, hosted by the third party; the researcher can start it before submitting and send the results to any journal. Reviews the research against all the data: reruns the figures from the final data, compares the final data with the full log (for example, 3 of 40 devices shown), asks which known lookalikes were ruled out, and sends any issues to the researcher. Its report and the conversation go to the reviewer. Exists today: [AEA Data Editor (people)](https://www.aeaweb.org/journals/data/faq), [cascad (people)](https://www.cascad.tech/terms), [Living Science (AI)](https://livingscience.ai/blog/introducing-living-science).
- **Reviewer view** (At the journal; partly; seen by The reviewer): Where the reviewer reads the paper, the assistant's report on issues and the conversation between the assistant and the researcher. The raw log stays sealed and there is no open-ended agent: the reviewer can send questions, which the researcher answers or lets the AI answer, and can decline only with a reason the reviewer sees. Exists today: [Dryad private review links](https://datadryad.org/help/submission_steps/peer_review), [NOMAD reviewer access](https://nomad-lab.eu/prod/v1/docs/tutorial/upload_publish.html).
- **Public record** (In public; partly; seen by The public): The published paper with its check badge, which says what was checked rather than that the result is right, and the final data if the lab chooses to share it. Exists today: [ACM badges](https://www.acm.org/publications/policies/artifact-review-and-badging-current), [HEPData](https://www.hepdata.net/submission).
- **Field view** (In public; new; seen by The public): A view of the whole field, built from the tags on published papers and public notes, never from the encrypted record: what's being worked on, what's been tried, and where nobody is investing. It helps funders see where money would do the most good. It shows topics, not a lab's data or check results.

The record has to be complete and impossible to change quietly. Data flows straight from lab tools, with no step where someone uploads by hand. Each run gets a fingerprint when it arrives, a short code computed from the file that changes if a single byte does, so anyone can later confirm nothing was edited.

Size is a real constraint. The Pittsburgh report says free repositories handle gigabyte-scale data well and that sharing tools exist for everything except primary data above about a terabyte. [c-repositories]

The lab tools are the strongest piece. In many quantum device labs, open-source measurement software already saves each measurement with its settings, and NOMAD, a data platform built for condensed matter and best known for computed data, also takes experimental data through a standard format. [c-qcodes][c-nomad]

#### Who holds the record

The time stamps show that what was logged wasn't changed later. They can't prove that nothing went unlogged: a lab could leave an instrument unconnected. If a lab hosts its own log, they prove little, because the lab can change them. So the system of record should be one shared service run by a third party that labs and journals both trust.

#### Keeping raw data safe

Handing raw data to a third party raises real issues: export control, national-lab rules, industrial partners who supply samples under nondisclosure agreements, and university IT policy. So the protection can come in layers, depending on how sensitive the data is.

By default, the data sits encrypted with the third party, and only the assistant reads it. The assistant runs there as a fixed workflow that nobody else controls, so nobody can run open-ended queries against the data to learn what a lab is working on. The only questions it answers beyond its own checks are a reviewer's, when the researcher lets it. For export-controlled or similarly restricted data, a country or institution can host its own copy of the system, built from the same open-source code with the same data controls, so the data never leaves and only the report does. Which kinds of data need which layer is something to work out with labs. Encryption protects the data at rest; to run the assistant, the operator's systems decrypt it, so "only the assistant reads it" is a rule, enforced by contract and audit, not by encryption alone. Who operates the model, and whether a model vendor ever sees the data, are rules to set and audit.

In every layer the data belongs to the university, or the institution behind a private lab, as it does today; the third party only holds it. Nobody opens it without the owner's consent, as in the university's own misconduct investigation, or a court order.

Other fields already trust a neutral host with records like these, from economics to structural biology to particle physics. [What other fields already do](/appendix/what-already-works/)

Whether condensed matter physicists would trust a third party with their raw data is one of my biggest open questions. Encryption, a check that only an AI runs, and the lab deciding what goes public are meant to make the answer yes.

### How I'll test it

Page: /appendix/how-ill-test-it/

Each guess I'm making, by phase, and how I plan to check it.

Grouped by phase. For phase 1 the order is conversations with lab heads I know first, then the assistant test, then conversations with journals and funders with both results in hand. Phases 2 and 3 are tested as the earlier phases come into use.

| Phase | Question | Guess | Test | Counts as yes |
|---|---|---|---|---|
| Phase 1 | Are these the pain points that matter most? | Yes, with hard-to-read new signals and missed related work at the top. | Ask lab heads to rank them and say what's missing. | They rank them without rewriting the list. |
| Phase 1 | What would make their lives easier? | Methods and figures written for them, nothing lost when students leave, faster revisions. | Ask where they lose time today. | A benefit they name without prompting. |
| Phase 1 | Does the assistant raise the lookalikes that turned out to be right? | Yes where the library is seeded, such as topological transport; unknown elsewhere. | Run it on Frolov's four 2026 cases, and on cases published after the model's training cutoff so it can't have memorized the answers. | It raises the explanations the field settled on, before being told. |
| Phase 1 | Can the assistant rerun condensed matter analyses from figure data to figures? | Yes for clean code and data, harder for lab formats. | Run it on 10 to 20 condensed matter papers that already share data and code. | Most reproduce, and failures have clear causes. |
| Phase 1 | Will referees trust the assistant's report and the researcher's answers? | Yes, if the researcher must answer their questions or decline with a reason they see, and the editor settles disagreements. | Show referees a mock report and conversation for a real paper. | They can sign off, or know exactly what to ask. |
| Phase 1 | What does it cost per paper, in money and people's time? | Compute is cheap. The real cost is people's time, answering the assistant's questions and packaging the data; the goal is minutes. | Time it on a few real papers before submission, note every step that still needs a person, and compare the cost with what economics journals spend on checking. | Researchers call the time acceptable, and the cost is small next to current fees. |
| Phase 1 | Who adopts first: would a journal or funder offer this? | One journal or one funder, in whichever subfield is most open to it, which others then copy. Journals that already require data statements are closest. | Conversations with a journal editor and a foundation program officer, ideally through introductions. | A first mover willing to discuss a pilot. |
| Phase 2 | Would labs write short notes on what they tried? | Yes, if a note takes minutes and can be cited. | Ask 5 labs for one note each on something they tried and dropped, written with the assistant. | Most write one, in under an hour. |
| Phase 2 | Do the notes help other labs? | Yes, when they're tagged well enough to find and say how much effort went in. | Show researchers notes from their subfield and ask whether any would have changed what they tried. | Some would have. |
| Phase 2 | Do labs already keep their data in a central or cloud system? | Many do in some form, which would make the optional run log easier. | Ask two lab heads I know, then 3 to 5 more labs: which tools, where data lives, who can see it. | Most store data centrally already. |
| Phase 2 | Is automatic logging light enough? | Yes, if it adds minutes, not hours. | Show a one-page mockup of the workflow and ask what they'd drop or fight. | They'd try it on one real project. |
| Phase 2 | Can logging plug into the tools labs already use? | Yes, through their existing interfaces and open-source connectors. | List two labs' measurement and analysis tools and prototype one connector. | One lab's data flows in with no manual step. |
| Phase 2 | Will labs trust a third party with their raw data? | Worries about being scooped and about security; encryption and an assistant-only check ease them. | Ask labs directly. | Access rules answer the worries. |
| Phase 3 | Can AI flag selection by comparing final data with the full log? | Partly. It can show what share was used; judging whether that share is typical is harder. | Try it on a case with full public data, such as one of the published topological replications. | It flags the known problem. |
| Phase 3 | Does pooled data show whether an artifact is common? | Yes, for well-documented device types with enough shared runs. | Pool shared data from one device type across a few labs and check whether a known artifact shows up across them. | The known artifact shows up where expected. |
| Phase 3 | Would the field make the run log the default? | Only after labs that opt in find it worth the effort. | Track how many labs that try the optional log still use it a year later. | Most keep it. |

## Claims

Every factual claim on the site, with its source and whether a person has checked it.

- **c-report-prb** (AI only, confident): The Pittsburgh report was published in Physical Review B (113, 119601) on 20 March 2026 and gives recommendations for researchers, publishers, universities and research laboratories, and government and funding organizations, drawn from a conference at the University of Pittsburgh on 9 to 11 May 2024. Source: [Akrap et al. 2026, Physical Review B 113, 119601](https://doi.org/10.1103/27h6-yghn)
- **c-report-recs** (AI only, confident): The report recommends that authors release as much primary data as possible and practical, sharing materials in the form they have them when full FAIR standards aren't possible; include the full ranges of experimental parameters studied, all reasonable presentations of the data, all plausible interpretations and the number of samples tested; and give all authors access to all primary data and analysis files before they consent to publication. Source: [Akrap, Frolov et al. 2026, Report on reproducibility in condensed matter physics, sections II and V.1](https://arxiv.org/html/2501.18631v2)
- **c-report-institutions** (AI only, confident): The report asks funders to consider funding work that explicitly aims to replicate prior results, publishers to formulate clear acceptance criteria for replication efforts, and universities to consider datasets and code as research outputs alongside publications. Source: [Akrap, Frolov et al. 2026, Report on reproducibility in condensed matter physics, sections V.2 to V.4](https://arxiv.org/html/2501.18631v2)
- **c-report-trust** (AI only, confident): The report says that reputation and authority serve as substitutes for which results to consider trustworthy, and that in more than a few cases the presentation of results lacks sufficient detail to facilitate replication and verification. Source: [Akrap, Frolov et al. 2026, Report on reproducibility in condensed matter physics, section I](https://arxiv.org/html/2501.18631v2)
- **c-report-careers** (AI only, confident): The report says that behind every disproven claim are months of work, funding not dedicated to replication, and diverted careers, often those of young researchers. Source: [Akrap, Frolov et al. 2026, Report on reproducibility in condensed matter physics, section I](https://arxiv.org/html/2501.18631v2)
- **c-report-prereg** (AI only, confident): The report notes that some fields responded to the replication crisis with preregistration of hypotheses and study design, and says such preregistration may not be helpful in the context of exploratory research. Source: [Akrap, Frolov et al. 2026, Report on reproducibility in condensed matter physics, section I](https://arxiv.org/html/2501.18631v2)
- **c-repositories** (AI only, confident): The report says solutions for data sharing are already available except for primary data above about 1 TB, and that for gigabyte-scale datasets there are robust repositories such as Zenodo, Harvard Dataverse, OSF, Figshare, HEPData and Dryad. Source: [Akrap, Frolov et al. 2026, Report on reproducibility in condensed matter physics, section IV](https://arxiv.org/html/2501.18631v2)
- **c-frolov-science** (AI only, confident; Four case studies of topological effects in nanoscale superconducting and semiconducting devices.): In a Science paper published on 8 January 2026, Frolov and 11 co-authors showed limited data consistent with four topological phenomena and, by considering additional data, found the most likely origins of the observed patterns to be trivial. They argue that releasing comprehensive datasets, discussing alternative scenarios and disclosing the total volume of study would make smoking gun claims more reliable. Source: [Frolov et al. 2026, Science 391, 137-142](https://doi.org/10.1126/science.adk9181)
- **c-majorana-retraction** (AI only, confident): On 8 March 2021 the authors retracted their 2018 Nature paper claiming quantized Majorana conductance, after Sergey Frolov and Vincent Mourik pointed out inconsistencies between the raw data made available to them and the published figures. Source: [Zhang et al. 2021, Retraction Note, Nature 591, E30](https://doi.org/10.1038/s41586-021-03373-x)
- **c-qutech** (AI only, confident): QuTech says two of its Nature papers were withdrawn over irregularities in data processing, selection and publication, that the authors shared the data of both in public repositories, and that from April 2022 it requires publishing at least all raw data behind published figures plus processing scripts. Source: [QuTech, Timeline of the retracted Majorana papers](https://qutech.nl/research-engineering/qubit-research/retracted-majorana-papers/)
- **c-pittwire-rejected** (AI only, confident): Pittwire reports that when Frolov's group submitted replication findings to the journals that had published the original papers, editors rejected them as not novel or no longer relevant. Source: [Pittwire, University of Pittsburgh, 29 January 2026](https://www.pittwire.pitt.edu/features-articles/2026/01/29/sergey-frolov-replication-crisis-research)
- **c-review-two-years** (AI only, confident): The University of Pittsburgh's news release says the 2026 Science paper was submitted in September 2023 and spent a record two years under peer and editorial review. Source: [University of Pittsburgh news release, EurekAlert, January 2026](https://www.eurekalert.org/news-releases/1111705)
- **c-pittwire-practice** (AI only, confident): Pittwire reports that when a collaborator asked for 15-year-old data, Frolov found and sent the raw data in less than 20 minutes, that he tells his students to be ready to do the same, and that he says colleagues in astrophysics treat sharing data as a matter of course. Source: [Pittwire, University of Pittsburgh, 29 January 2026](https://www.pittwire.pitt.edu/features-articles/2026/01/29/sergey-frolov-replication-crisis-research)
- **c-aea** (AI only, confident; Economics; American Economic Association journals only.): The AEA Data Editor's lab checks the replication package of every article with a computational component in AEA journals before publication (Papers and Proceedings get a cursory review) and handles about 400 articles a year; from December 2022 to November 2023 it received 646 verification requests covering 527 manuscripts. Source: [Vilhuber 2024, Report of the AEA Data Editor, AEA Papers and Proceedings 114](https://doi.org/10.1257/pandp.114.878)
- **c-mass-repro** (AI only, confident; 110 articles in leading economics and political science journals with data and code sharing policies.): Reproducing and robustness-checking 110 articles from leading economics and political science journals with mandatory data and code sharing, the authors found more than 85% of claims computationally reproducible and 72% of statistically significant estimates still significant and in the same direction under robustness checks. Source: [Brodeur et al. 2026, Nature 652, 151-156 (Institute for Replication)](https://doi.org/10.1038/s41586-026-10251-x)
- **c-i4r-errors** (AI only, confident; 110 papers in leading economics and political science journals.): In 110 papers from leading economics and political science journals, the authors found coding errors in about 25% of studies after excluding minor issues such as missing packages or broken file paths. Source: [Brodeur et al. 2024, IZA Discussion Paper 16912](https://www.iza.org/publications/dp/16912/mass-reproducibility-and-replicability-a-new-hope)
- **c-ai-assisted** (AI only, confident; Quantitative social science; ChatGPT models of the GPT-4 era.): In an experiment that randomly assigned 288 researchers to 103 teams reproducing results from leading social science journals, human-only teams reproduced 94% of results and AI-assisted teams 91%, though human-only teams found significantly more major coding errors. Source: [Brodeur et al. 2026, PNAS 123, e2524747123](https://doi.org/10.1073/pnas.2524747123)
- **c-shah** (AI only, confident; Five R-based social science studies; a preprint.): On five fully reproducible R-based social science studies with deliberately injected failures, agent-based repair workflows succeeded in 69 to 96 percent of cases, against 31 to 79 percent for prompt-based workflows. Source: [Shah, Hopfgartner and Bleier 2026, arXiv 2602.08561](https://arxiv.org/abs/2602.08561)
- **c-autoreproduce** (AI only, confident; 13 machine learning papers; a preprint.): On a benchmark of 13 AI research papers, the average cost for AutoReproduce to reproduce one experiment was $1.87 with o3-mini as the model. Source: [Zhao et al. 2025, AutoReproduce, arXiv 2505.20662](https://arxiv.org/abs/2505.20662)
- **c-cascad** (AI only, confident): cascad, a non-profit certification agency of CNRS, HEC Paris and the University of Orleans, states a certification fee of 500 euros, with possible extra costs, and through a partnership with the French secure data center CASD can certify results that rely on confidential data. Source: [cascad, terms of service](https://www.cascad.tech/terms)
- **c-aps-policy** (AI only, confident; Physical Review journals.): APS announced on 1 August 2024 that every Physical Review article must carry a statement saying where the data needed to support or replicate its claims can be accessed, from late 2024; public sharing is encouraged but not required, and authors who don't share publicly must share with referees and editors on request. Source: [American Physical Society, Refreshed Data Availability Policy for the Physical Review journals, 2024](https://journals.aps.org/edannounce/APS-Releases-Data-Availability-Policy-for-the-Journals)
- **c-raccuglia** (AI only, confident; One family of chemical syntheses.): A machine learning model trained on about 4,000 recorded syntheses of templated vanadium selenites, including failed 'dark' reactions kept in lab notebooks, predicted conditions for new products with an 89% success rate, beating the strategies of the chemists who ran the experiments. Source: [Raccuglia et al. 2016, Nature 533, 73-76](https://doi.org/10.1038/nature17439)
- **c-data-requests** (AI only, confident; Nature and Science papers across nine disciplines.): Requesting data from corresponding authors of Nature and Science papers in nine disciplines, the authors obtained it for 39.4% of requests on average, from 27.9% to 56.1% across fields. Source: [Tedersoo et al. 2021, Scientific Data 8, 192](https://doi.org/10.1038/s41597-021-00981-0)
- **c-womack** (AI only, confident; A small sample: articles from 2014 top journals.): In a sample of 2014 articles from the top 10 journals in biology, chemistry, mathematics and physics, physics shared original data at the lowest rate: none of the sampled physics articles with original data made it available, and 8% of all sampled physics articles made any data available. Source: [Womack 2015, PLOS ONE 10(12): e0143460](https://doi.org/10.1371/journal.pone.0143460)
- **c-twist-angle** (AI only, confident): Nanoscale maps of magic-angle twisted bilayer graphene showed that even state-of-the-art superconducting devices have local twist angles varying by close to 0.1 degree, with gradients, jumps and some areas showing no magic-angle behavior at all. Source: [Uri et al. 2020, Nature 581, 47-52](https://doi.org/10.1038/s41586-020-2255-3)
- **c-rzhetsky** (AI only, confident; Biomedicine.): Modeling millions of biomedical papers and patents over 30 years, the authors found that scientists follow a conservative strategy focused on well-studied molecules, which supports careers but is inefficient for science as a whole; more risk-taking and publishing failures would substantially speed discovery. Source: [Rzhetsky, Foster, Foster and Evans 2015, PNAS 112(47)](https://doi.org/10.1073/pnas.1509757112)
- **c-marwitz** (AI only, guess; Materials science abstracts.): Building a concept graph from about 221,000 materials science abstracts (1955 to 2022), the authors observe that concept centralization increases over time, with fewer concepts accounting for a larger share of connections. Source: [Marwitz et al. 2026, Nature Machine Intelligence 8, 535-544](https://doi.org/10.1038/s42256-026-01206-y)
- **c-fanelli** (AI only, confident): Across 2,434 papers that tested a hypothesis, the share reporting positive support ranged from 70.2% in space science to 91.5% in psychology and psychiatry, and the odds of a positive result were 2.3 times higher in the social sciences than in the physical sciences. Source: [Fanelli 2010, PLOS ONE 5(4): e10068](https://doi.org/10.1371/journal.pone.0010068)
- **c-kelly** (AI only, confident; Magnetic transition temperatures only, mined with a rule-based parser.): An automatically text-mined dataset of 56,037 Curie and Neel temperature records from 108,181 papers had an estimated precision of 72% and recall of 61%. Source: [Kelly and Cole 2025, Scientific Data 12, 2000](https://doi.org/10.1038/s41597-025-06244-6)
- **c-ncomms-files** (AI only, confident): Nature Communications publishes reviewer comments and author rebuttal letters for all original research articles submitted from 1 November 2022, and Nature made publishing referee reports and author responses standard for new research submissions from 16 June 2025. Source: [Nature Communications, editorial process; Nature, 16 June 2025](https://www.nature.com/ncomms/submit/editorial-process)
- **c-pdb** (AI only, confident; Structural biology.): Mandatory Protein Data Bank deposit before publication was guided by the International Union of Crystallography in 1989 ('no PDB deposition, no structural publication'); most journals now also ask for an official validation report at submission, and deposit, validation and curation are free to depositors. Source: [Kurisu 2022, Journal of Biochemistry 171(1); wwPDB](https://doi.org/10.1093/jb/mvab133)
- **c-alphafold** (AI only, confident): AlphaFold was trained on experimentally determined structures from the Protein Data Bank released up to 30 April 2018, plus a set of its own high-confidence predictions. Source: [Jumper et al. 2021, Nature 596, 583-589](https://doi.org/10.1038/s41586-021-03819-2)
- **c-jade** (AI only, confident): Data and software from the JADE experiment, which ran at the PETRA collider from 1979 to 1986, were resurrected and reanalyzed with modern simulation and state-of-the-art QCD calculations to measure the strong coupling. Source: [Bethke and Wagner 2022, European Physical Journal H 47, 16](https://doi.org/10.1140/epjh/s13129-022-00047-8)
- **c-qcodes** (AI only, confident): QCoDeS is an open-source Python data acquisition framework from the Copenhagen, Delft, Sydney and Microsoft quantum computing consortium; each measurement run is stored in an SQLite database with metadata and a snapshot of the instrument settings. Source: [QCoDeS documentation](https://github.com/microsoft/Qcodes)
- **c-instrmcp** (AI only, confident): instrMCP is an open-source server suite that lets large language models interact with quantum device lab instruments through QCoDeS and JupyterLab, read-only by default with an optional mode that executes commands. Source: [instrMCP documentation](https://github.com/caidish/instrMCP)
- **c-nomad** (AI only, confident): FAIRmat, a consortium of Germany's National Research Data Infrastructure, is building data infrastructure for condensed matter physics; its software NOMAD supports experimental data through the NeXus standard, instrument integration and electronic lab notebooks. Source: [FAIRmat and NOMAD](https://www.fairmat-nfdi.eu/fairmat/)
- **c-nomad-review** (AI only, confident): NOMAD uploads can be shared read-only with reviewers, embargoed for up to 36 months, and become unchangeable once published. Source: [NOMAD documentation, uploading and publishing](https://nomad-lab.eu/prod/v1/docs/tutorial/upload_publish.html)
- **c-openicpsr** (AI only, confident; Economics.): Authors of empirical papers accepted by AEA journals deposit their data and code in the AEA Data and Code Repository at openICPSR, where the AEA Data Editor reruns them before acceptance. Source: [AEA Data and Code Repository, openICPSR](https://www.openicpsr.org/openicpsr/aea)
- **c-ctgov** (AI only, confident; Clinical trials.): Since 2005 the major medical journals have refused to consider a trial unless it was registered in a public registry such as ClinicalTrials.gov, run by the US National Library of Medicine, before the first patient enrolled. Source: [ClinicalTrials.gov; ICMJE trial registration policy](https://clinicaltrials.gov/policy/fdaaa-801-final-rule)
- **c-cos-funding** (AI only, confident): The Center for Open Science says funding from the Laura and John Arnold Foundation provided the initial funding to launch it in 2013; WIRED reports the initial grant was $5.25 million. Source: [Center for Open Science, Our sponsors; WIRED 2017](https://www.cos.io/about/our-sponsors)
- **c-ccdc** (AI only, confident; Chemistry and crystallography.): Authors deposit crystal structure files with the Cambridge Crystallographic Data Centre before publication, where they stay confidential but available to referees, and the IUCr checkCIF service produces an automated report that flags inconsistencies. Source: [CCDC deposition; IUCr checkCIF](https://www.ccdc.cam.ac.uk/deposit/upload)
- **c-hepdata** (AI only, confident; Particle physics.): HEPData at Durham University, funded by the UK's Science and Technology Facilities Council, holds the numbers behind the plots of several thousand particle physics papers, and each submission passes a designated reviewer before release. Source: [HEPData, submission documentation](https://www.hepdata.net/submission)
- **c-notes-lab** (AI only, confident): The head of a quantum device lab told David that AI already writes instrument drivers and runs measurement sweeps in their lab. Source: David's notes, from a conversation with a lab head (private; not yet cleared to name)
- **c-prada** (AI only, confident): A 2020 review of hybrid superconductor-semiconductor nanowires covers Andreev and Majorana bound states and the techniques for detecting them, including different types of robust zero modes that may emerge without a band-topological transition. Source: [Prada et al. 2020, Nature Reviews Physics 2, 575-594](https://www.nature.com/articles/s42254-020-0228-y)
- **c-arxiv-2024** (AI only, confident): arXiv lists 17,022 condensed matter articles for 2024, plus 4,938 cross-lists. Source: [arXiv, Article statistics for 2024, Condensed Matter](https://arxiv.org/year/cond-mat/2024)
- **c-notes-misreading** (AI only, confident): A condensed matter lab head told David that honest misinterpretation is common because data on new phenomena is hard to interpret, researchers usually don't know all the literature related to their experiment, and the pull to publish high for funding and reputation favors the fancier interpretation over the simpler one, on a spectrum of intent; testing the trivial explanation can take another experiment on a modified platform. Source: David's notes, from correspondence with a lab head (private; October 2026)
- **c-tenopir** (AI only, confident; Respondents mostly from environmental sciences and ecology; 2011.): In a survey of 1,329 scientists, the leading reasons given for not making data electronically available to others were insufficient time (54%) and lack of funding (40%). Source: [Tenopir et al. 2011, PLOS ONE 6(6): e21101](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0021101)
- **c-vines** (AI only, confident; Biology papers.): Requesting the data behind biology papers 2 to 22 years old, the authors found that, for papers where the status of the data was given, the odds of a data set being extant fell by 17% per year. Source: [Vines et al. 2014, Current Biology 24, 94-97](https://pubmed.ncbi.nlm.nih.gov/24361065/)
- **c-scite** (AI only, confident): scite is a citation index that shows the context of each citation and uses a deep learning model to classify whether the citing statement provides supporting or contrasting evidence for the cited work, or simply mentions it. Source: [scite, Quantitative Science Studies 2(3), 882 (2021)](https://direct.mit.edu/qss/article/2/3/882/102990/scite-A-smart-citation-index-that-displays-the)
- **c-consensus** (AI only, confident): Consensus is an AI academic search engine for peer-reviewed literature, searching over 200 million research papers. Source: [Consensus; Bentley University library guide](https://consensus.app/)
- **c-paperqa** (AI only, confident; Biology literature; 2024 models.): PaperQA2, a language model agent, matched or exceeded subject matter experts on three realistic literature research tasks, and identified about 2.3 contradictions per paper in a random subset of biology papers, 70% of which human experts validated. Source: [FutureHouse 2024, Language agents achieve superhuman synthesis of scientific knowledge, arXiv 2409.13740](https://arxiv.org/abs/2409.13740)
- **c-cmt** (AI only, confident; Condensed matter theory problems; models evaluated in 2025.): On CMT-Benchmark, 50 condensed matter theory problems designed and verified by expert researchers, frontier models struggled with all of the problems, and the best model, GPT-5, solved 30%. Source: [CMT-Benchmark 2025, arXiv 2510.05228](https://arxiv.org/abs/2510.05228)
- **c-pdb-size** (AI only, confident): The worldwide Protein Data Bank passed 200,000 entries in 2023. Source: [RCSB PDB news, PDB Reaches a New Milestone: 200,000+ Entries](https://www.rcsb.org/news/639b9e337f8444f313d20414)
- **c-notes-solar** (AI only, confident): A solar physicist told David that a funding gap isn't always a scientific gap: some areas get little funding because they are largely solved. Source: David's notes, from a conversation with a solar physicist (private)
- **c-registered-reports** (AI only, confident): Registered Reports are a publishing format in which protocols and analysis plans are peer reviewed and registered before data collection, then published regardless of outcome. Source: [Center for Open Science, Registered Reports](https://www.cos.io/initiatives/registered-reports)
- **c-zenodo-doi** (AI only, confident): By default, Zenodo registers DOIs for all uploads when they are published on Zenodo. Source: [Zenodo help, Digital Object Identifier (DOI)](https://help.zenodo.org/docs/deposit/describe-records/reserve-doi/)
- **c-wiley-ai** (AI only, confident; Researchers across fields worldwide; a publisher's survey.): In Wiley's ExplanAItions survey of 2,430 researchers in August 2025, 84% were using AI tools for some aspect of their work, up from 57% in 2024, and 62% were using AI for research or publication tasks, up from 45%. Source: [Wiley, ExplanAItions 2025: key findings](https://www.wiley.com/en-us/about-us/ai/study/key-findings)
- **c-living-reviews** (AI only, confident): Living Reviews in Relativity is a diamond open-access journal that publishes invited reviews in all areas of relativity, and is unique in maintaining a suite of reviews that are regularly updated by their authors. Source: [Living Reviews in Relativity, journal home page (Springer Nature)](https://link.springer.com/journal/41114)
- **c-franco** (AI only, confident; Social science survey experiments, not physics.): In 221 social science experiments funded through the NSF's Time-sharing Experiments for the Social Sciences, with a full accounting of what was and wasn't published, strong results were 40 percentage points more likely to be published than null results and 60 percentage points more likely to be written up. Source: [Franco, Malhotra and Simonovits 2014, Science 345, 1502-1505](https://doi.org/10.1126/science.1255484)