The case / Appendix

The systems underneath, and who holds them

The systems behind the optional run log: ten pieces, most of which exist in some form today, held by one shared service run by a neutral third party so the time stamps mean something. The data stays encrypted and only the assistant reads it. Select any piece to see what it does, what already exists and who can see it.

A proposal, not a finished system
  • Exists today
  • Partly exists
  • New
In the labSeen by the labHeld by a third partyEncrypted; only the assistant reads itAt the journalLab and reviewerIn publicEveryoneevery runfinal datastartsfull logissuesanswerspaperreport and answerspublishedLab toolsExists todayConnectorPartly existsResearcher repliesNewFinal data tagNewSystem of recordNewencrypted, time-stampedSubmission packagePartly existsResearch assistantNewReviewer viewPartly existsPublic recordPartly existsField viewNew
In the lab Seen by the lab
Held by a third party Encrypted; only the assistant reads it
At the journal Lab and reviewer
In public Everyone
Held by a third party

Research assistant

NewSeen by the reviewer

For a lab that logs, the assistant also runs as a fixed workflow inside the record, hosted by the third party; the researcher can start it before submitting and send the results to any journal. Reviews the research against all the data: reruns the figures from the final data, compares the final data with the full log (for example, 3 of 40 devices shown), asks which known lookalikes were ruled out, and sends any issues to the researcher. Its report and the conversation go to the reviewer.

People rerun papers before publication in economics. AI reruns exist for social science and machine learning. Nothing yet compares final data with a full log.

What exists today: AEA Data Editor (people), cascad (people), Living Science (AI)

Select any box to see what it does, what already exists, and who can see it. Solid boxes exist today, dashed ones would be new. The raw log never leaves the third party: only the assistant reads it, and the reviewer gets its report and the researcher's answers.

The record has to be complete and impossible to change quietly. Data flows straight from lab tools, with no step where someone uploads by hand. Each run gets a fingerprint when it arrives, a short code computed from the file that changes if a single byte does, so anyone can later confirm nothing was edited.

Size is a real constraint. The Pittsburgh report says free repositories handle gigabyte-scale data well and that sharing tools exist for everything except primary data above about a terabyte.1

The lab tools are the strongest piece. In many quantum device labs, open-source measurement software already saves each measurement with its settings, and NOMAD, a data platform built for condensed matter and best known for computed data, also takes experimental data through a standard format.2,3

Who holds the record

The time stamps show that what was logged wasn't changed later. They can't prove that nothing went unlogged: a lab could leave an instrument unconnected. If a lab hosts its own log, they prove little, because the lab can change them. So the system of record should be one shared service run by a third party that labs and journals both trust.

Keeping raw data safe

Handing raw data to a third party raises real issues: export control, national-lab rules, industrial partners who supply samples under nondisclosure agreements, and university IT policy. So the protection can come in layers, depending on how sensitive the data is.

By default, the data sits encrypted with the third party, and only the assistant reads it. The assistant runs there as a fixed workflow that nobody else controls, so nobody can run open-ended queries against the data to learn what a lab is working on. The only questions it answers beyond its own checks are a reviewer's, when the researcher lets it. For export-controlled or similarly restricted data, a country or institution can host its own copy of the system, built from the same open-source code with the same data controls, so the data never leaves and only the report does. Which kinds of data need which layer is something to work out with labs. Encryption protects the data at rest; to run the assistant, the operator's systems decrypt it, so "only the assistant reads it" is a rule, enforced by contract and audit, not by encryption alone. Who operates the model, and whether a model vendor ever sees the data, are rules to set and audit.

In every layer the data belongs to the university, or the institution behind a private lab, as it does today; the third party only holds it. Nobody opens it without the owner's consent, as in the university's own misconduct investigation, or a court order.

Other fields already trust a neutral host with records like these, from economics to structural biology to particle physics.

Whether condensed matter physicists would trust a third party with their raw data is one of my biggest open questions. Encryption, a check that only an AI runs, and the lab deciding what goes public are meant to make the answer yes.

More in the appendix

  1. The pain points in detail
  2. One example: the Majorana signals
  3. Phase 3 in detail
  4. Why now: what AI makes cheap
  5. What physicists propose and other fields already do
  6. The run log and a shared record
  7. The systems underneath, and who holds them
  8. How I'll test it