# Nature Methods linear-PCA strong-baseline evaluation

Method source: Lotfollahi et al., *Nature Methods* (2025), https://doi.org/10.1038/s41592-025-02772-6.

This analysis is a faithful independent reimplementation of the published linear PCA baseline, not execution of the authors' archived container. It uses PCA dimension 10 and ridge lambda 0.1 on the existing five frozen outer held-out-gene splits. PCA and W are fit using training perturbation responses only.

The method can predict a held-out perturbation only when its target occurs among the 2,000 response genes. All comparisons below therefore restrict ridge_v1 and baselines to exactly the same covered seed-gene pairs; noncoverage is reported separately and is not counted as an error.

## Coverage

| seed | n_train_total | n_train_target_in_response_genes | train_coverage_fraction | n_test_total | n_test_target_in_response_genes | test_coverage_fraction | pca_dim | ridge_lambda |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 20260712 | 1620 | 172 | 0.106173 | 540 | 64 | 0.118519 | 10 | 0.1 |
| 20260713 | 1620 | 166 | 0.102469 | 540 | 70 | 0.12963 | 10 | 0.1 |
| 20260714 | 1620 | 180 | 0.111111 | 540 | 56 | 0.103704 | 10 | 0.1 |
| 20260715 | 1620 | 173 | 0.10679 | 540 | 63 | 0.116667 | 10 | 0.1 |
| 20260716 | 1620 | 188 | 0.116049 | 540 | 48 | 0.0888889 | 10 | 0.1 |

## Same-coverage per-split summary

| model | seed | n_test_genes | mean_model_rmse | mean_zero_rmse | mean_train_mean_rmse | mean_model_cosine | mean_train_mean_cosine | mean_perturbation_specific_residual_cosine | relative_rmse_improvement_vs_zero | relative_rmse_improvement_vs_train_mean | raw_cosine_difference_vs_train_mean |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| nature_methods_linear_pca10 | 20260712 | 64 | 0.107107 | 0.112189 | 0.10827 | 0.344707 | 0.327162 | 0.1785 | 0.0452925 | 0.0107354 | 0.0175452 |
| nature_methods_linear_pca10 | 20260713 | 70 | 0.10433 | 0.104792 | 0.10533 | 0.284588 | 0.266731 | 0.117224 | 0.00441361 | 0.00949229 | 0.0178573 |
| nature_methods_linear_pca10 | 20260714 | 56 | 0.108115 | 0.112893 | 0.109707 | 0.334033 | 0.318327 | 0.197759 | 0.0423216 | 0.0145043 | 0.0157058 |
| nature_methods_linear_pca10 | 20260715 | 63 | 0.109412 | 0.111689 | 0.110086 | 0.306364 | 0.294197 | 0.131306 | 0.020384 | 0.00612257 | 0.012167 |
| nature_methods_linear_pca10 | 20260716 | 48 | 0.103904 | 0.112521 | 0.104922 | 0.392732 | 0.379297 | 0.144741 | 0.0765785 | 0.00970335 | 0.0134356 |
| ridge_v1_same_coverage | 20260712 | 64 | 0.105843 | 0.112189 | 0.10827 | 0.322811 | 0.327162 | 0.229172 | 0.0565658 | 0.0224167 | -0.00435083 |
| ridge_v1_same_coverage | 20260713 | 70 | 0.10473 | 0.104792 | 0.10533 | 0.274728 | 0.266731 | 0.215956 | 0.000596567 | 0.00569472 | 0.00799785 |
| ridge_v1_same_coverage | 20260714 | 56 | 0.107078 | 0.112893 | 0.109707 | 0.292421 | 0.318327 | 0.246369 | 0.0515084 | 0.0239579 | -0.025906 |
| ridge_v1_same_coverage | 20260715 | 63 | 0.109604 | 0.111689 | 0.110086 | 0.2509 | 0.294197 | 0.192719 | 0.0186671 | 0.0043807 | -0.0432972 |
| ridge_v1_same_coverage | 20260716 | 48 | 0.105741 | 0.112521 | 0.104922 | 0.337129 | 0.379297 | 0.203092 | 0.0602564 | -0.00780084 | -0.0421681 |

## Gate reproducibility

| model | n_splits | rmse_pass_splits | raw_composite_pass_splits | perturbation_specific_pass_splits |
| --- | --- | --- | --- | --- |
| nature_methods_linear_pca10 | 5 | 0 | 0 | 0 |
| ridge_v1_same_coverage | 5 | 0 | 0 | 0 |

## Direct linear-PCA versus ridge_v1 comparison

| seed | comparison | n_genes | mean_difference | bootstrap_ci95_low | bootstrap_ci95_high | direction_if_linear_better | paired_randomization_p_two_sided | observed_winner | holm_p_model_comparison_family |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 20260712 | linear_minus_ridge_rmse | 64 | 0.00126473 | -0.00253374 | 0.00493646 | less | 0.520374 | ridge_v1 | 1 |
| 20260712 | linear_minus_ridge_raw_cosine | 64 | 0.021896 | -0.00508335 | 0.0490258 | greater | 0.113194 | linear_pca10 | 1 |
| 20260712 | linear_minus_ridge_residual_cosine | 64 | -0.0506718 | -0.144444 | 0.042842 | greater | 0.292785 | ridge_v1 | 1 |
| 20260713 | linear_minus_ridge_rmse | 70 | -0.000399997 | -0.00490149 | 0.00395171 | less | 0.863157 | linear_pca10 | 1 |
| 20260713 | linear_minus_ridge_raw_cosine | 70 | 0.0098594 | -0.00715455 | 0.0260907 | greater | 0.257687 | linear_pca10 | 1 |
| 20260713 | linear_minus_ridge_residual_cosine | 70 | -0.0987327 | -0.192651 | -0.00788418 | greater | 0.0453977 | ridge_v1 | 0.544773 |
| 20260714 | linear_minus_ridge_rmse | 56 | 0.00103713 | -0.00315182 | 0.00518331 | less | 0.612169 | ridge_v1 | 1 |
| 20260714 | linear_minus_ridge_raw_cosine | 56 | 0.0416118 | 0.00388955 | 0.0800944 | greater | 0.0379981 | linear_pca10 | 0.493975 |
| 20260714 | linear_minus_ridge_residual_cosine | 56 | -0.0486102 | -0.144803 | 0.0459981 | greater | 0.332283 | ridge_v1 | 1 |
| 20260715 | linear_minus_ridge_rmse | 63 | -0.000191755 | -0.00440393 | 0.00392408 | less | 0.924654 | linear_pca10 | 1 |
| 20260715 | linear_minus_ridge_raw_cosine | 63 | 0.0554642 | 0.0143122 | 0.0992663 | greater | 0.0121994 | linear_pca10 | 0.170791 |
| 20260715 | linear_minus_ridge_residual_cosine | 63 | -0.0614129 | -0.17211 | 0.0496099 | greater | 0.301485 | ridge_v1 | 1 |
| 20260716 | linear_minus_ridge_rmse | 48 | -0.00183658 | -0.00630852 | 0.00222147 | less | 0.418979 | linear_pca10 | 1 |
| 20260716 | linear_minus_ridge_raw_cosine | 48 | 0.0556038 | 0.027291 | 0.0878696 | greater | 0.000499975 | linear_pca10 | 0.00749963 |
| 20260716 | linear_minus_ridge_residual_cosine | 48 | -0.0583509 | -0.153044 | 0.034495 | greater | 0.244288 | ridge_v1 | 1 |

Raw cosine can be inflated by a shared perturbation-response component. The perturbation-specific residual cosine subtracts each split's training mean from both prediction and observation and remains the stricter directional endpoint.