The vision
Where this could go: from today's pain points to a field that shares what it learns, and which phase gets each one there.
Picture where this ends up. A student sees a new zero-bias peak. Within the hour, an agent shows her every shared device in the field that showed something similar, the ordinary effects known to produce it, the labs that ruled them out and how, and shared data from similar devices that shows whether the ordinary explanation is common, before she builds a new one.
From today to where this could go
TodayA lab, its collaborators and its referees judge a new signal against the simple explanations they happen to know.
Where this could goA claimed signal comes with the lookalikes other labs have seen and the checks that rule them out.
TodayYou know your corner of the literature and what colleagues happen to mention.
Where this could goAn agent finds every similar experiment, including the ones that found nothing, and the people who ran them.
TodayData, failed runs and replications stay on lab drives, and are often lost when a student leaves.
Where this could goWhat's been tried is shared in minutes, with the data if the lab chooses, and labs that want to can keep every run.
TodayThe exciting interpretation is published, and the simple one is never fully tested.
Where this could goTesting the simple explanation is a standard step, so careful work isn't at a disadvantage.
TodayOnly your lab can test it, often with a whole new experiment.
Where this could goData other labs shared from similar devices shows whether the artifact is common, a first check before a new experiment.
The evidence for each pain point and the long-term pieces of phase 3 are in the appendix.