# Lockbox and measurement-reliability protocol

## What is available now

The current audit estimates target-level agreement under three replicate definitions. Cells are used to estimate a pseudobulk perturbation delta; cells are not treated as independent inferential replicates.

| Replicate definition | Targets | Median RMSE | Median cosine | Median sign concordance | Interpretation |
|---|---:|---:|---:|---:|---|
| Random cell split-half, five repeats | 2,160 | 0.1359 | 0.3314 | 0.5690 | Technical resampling reliability |
| Batch split | 2,160 | 0.1373 | 0.3333 | 0.5685 | Batch-separated aggregate reliability |
| `sgID_AB` guide-pair/construct split | 133 | 0.1435 | 0.0060 | 0.5025 | Limited exploratory construct-level reliability |

The low guide-pair cosine materially limits the biological interpretation of small model gains. The released target reliability field is descriptive and was not used as a training weight.

## What cannot be created retrospectively

All 2,160 eligible genes and the five outer split definitions have already entered model development, comparison or review. Selecting a new subset now would not create an untouched lockbox. The present splits are therefore labelled retrospective stability analyses.

## Prospective minimum design

1. Acquire a new dataset or reserve an untouched accession/batch before any model or triage revision.
2. Freeze target eligibility, biological-replicate minimums and guide-consistency thresholds.
3. Keep the lockbox inaccessible to feature design, hyperparameter selection, candidate thresholds and disease-prior construction.
4. Use target-level raw direction and sign concordance as primary endpoints.
5. Report biological-replicate reliability as the attainable ceiling and normalize model claims against it only with a prespecified formula.
6. Use grouped gene-family/pathway splits when the representation could transfer near-duplicate functional annotations.
7. Evaluate common and native coverage; missing predictions count against the declared task denominator.
8. Publish one confirmatory analysis; all alternate endpoints and threshold changes are exploratory.