Canary BuildsResearch on compact models

Practice

House rules for how experiments are planned, scored, and filed

  1. 1Filed before it runs. Hypotheses and decision rules go into the research log ahead of any compute.
  2. 2Separate scorer. The model behind a headline figure never chooses outputs or trains anything.
  3. 3Stop, don't guess. Analyzers halt on missing data; corrections are new log entries.
  4. 4Bounds on every figure. Paired differences carry bootstrap intervals and exact tests.
  5. 5A complete record. Code heads, data hashes, environments, seeds, and GPU-hours are kept for each phase.