# Frozen external-user evaluation protocol

This protocol evaluates whether the resource changes user decisions correctly, not whether users like the interface.

## Participants

- At least three independent disease biologists and three computational single-cell/perturbation analysts.
- No participant involved in developing NMD-VCell or its candidate rules.
- De-identified role, prior experience and task order recorded.

## Tasks

| Task | Required output | Success criterion | Critical error |
|---|---|---|---|
| Gene lookup | Locate one named gene and report available/missing evidence | Correct evidence fields and explicit coverage absence within 3 minutes | Inferring missing perturbation data as a negative biological result |
| Model comparison | Compare ridge, GEARS and scGPT on a specified split | Reports native/common coverage, raw direction and no-effect baselines | Declaring a winner from RMSE without coverage/direction |
| Candidate exclusion | Review a short candidate list with dependency and fusion flags | Correctly excludes/flags prespecified risk cases and explains uncertainty | Calling an unmeasured case safe |
| Claim-boundary check | Judge five proposed manuscript statements | Correctly rejects all L3–L5 statements unsupported by the release | Accepting a therapeutic, muscle-prediction or DMD-validation claim |

## Endpoints

- Primary: proportion of tasks completed without a critical error.
- Secondary: completion time, navigation errors, evidence fields consulted, confidence calibration and unsupported-claim detection rate.
- Comparator: the same tasks using the raw source tables without the audit portal, counterbalanced by participant.
- Analysis unit: participant; report individual results and medians rather than treating tasks as independent users.

## Freeze and publication

Task wording, answers and success thresholds must be frozen before recruitment. All failures must remain in the released evaluation table. The current revision provides the protocol only; no external-user result is claimed.