# Methods

This document describes each analysis behind the files in `data/`, in the order they build on one another. Each section names the script in `code/`, its inputs, what it does, the file it wrote and the checks recorded in that file. Field names in backticks refer to the result files.

| Section | Script | Output in `data/` | Reads |
| --- | --- | --- | --- |
| 1 | `measure_portfolio_dollars.py` | `2026-09-24-nci-unseen-portfolio-dollars-v1.0.0.json`, `2026-09-24-nci-fy2026-award-links-v1.0.0.json` | The public funding export |
| 2 | `measure_ins_coverage.py` | `2026-09-26-ins-coverage-v1.0.0.json` | Section 1 files; the Index of NCI Studies release |
| 3 | `audit_paper_route_links.py` | `2026-09-26-paper-route-link-accuracy-v1.0.0.json` | Sections 1 and 2; the section 1 API cache |
| 4 | `build_reuse_cohort.py`, `correct_reuse_cohort.py` | `2026-09-26-nci-reuse-cohort-v1.0.2.json` | Europe PMC; section 5 for errata 1.0.2 |
| 5 | `review_unchecked_databank_production.py` | `2026-09-26-unchecked-databank-production-review-v1.0.0.json` | Cohort version 1.0.1 |
| 6 | `measure_reuse_clock.py` | `2026-09-26-nci-reuse-clock-v1.1.0.json` | The cohort |
| 7 | `measure_reuse_predictors.py` | `2026-09-26-nci-reuse-early-signals-v1.1.0.json` | The cohort |
| 8 | `measure_reuse_reach.py` | `2026-09-26-nci-reuse-reach-v1.1.0.json` | The cohort |
| 9 | `classify_reuse_outcomes.py` | `2026-09-26-nci-reuse-outcomes-v1.1.0.json` | The cohort |
| 10 | `estimate_replacement_value.py` | `2026-09-26-nci-reuse-replacement-value-v1.1.0.json` | The cohort |
| 11 | `measure_reuse_per_dollar.py` | `2026-09-26-nci-reuse-per-dollar-v1.1.0.json` | The cohort |
| 12 | `measure_payroll_vs_wins.py` | `2026-09-26-payroll-vs-wins-v1.1.0.json` | The cohort; section 11 |
| 13 | `measure_draft_board_and_release_speed.py` | `2026-09-26-draft-board-and-release-speed-v1.1.0.json` | The cohort; section 7 |
| 14 | `measure_paper_fame_and_clinical_reach.py` | `2026-09-26-paper-fame-and-clinical-reach-v1.1.0.json` | The cohort; sections 7, 8 and 9 |
| 15 | `measure_reuse_beyond_nci_funding.py` | `2026-09-26-reuse-beyond-nci-funding-v1.1.0.json` | The cohort; section 8 |
| 16 | `measure_share_score_predicts_nci_reuse.py` | `2026-09-26-share-score-nci-reuse-v1.1.0.json` | The cohort; section 7 |

"The cohort" is `2026-09-26-nci-reuse-cohort-v1.0.2.json`. Rerun commands below assume the result files have been copied to `docs/evidence/`, as the README explains.

## Definitions used throughout

- **Core project.** An NIH award number without its application type, support year and suffix: "5 R01 CA123456-03" and "R01-CA-123456" are both core project R01CA123456 (`core_project` in `code/reuse_common.py`).
- **FY2026 dollars.** NCI's fiscal year 2026 funding of each core project (`funding_total` in the public ScienceDex FY2026 NCI funding export).
- **Dataset.** A study-level deposit: a GEO series or DataSet, PDB entry, SRA study, BioProject, ArrayExpress experiment, PRIDE project, EGA study or dataset, or dbGaP study. SRA runs, experiments and samples, BioSamples and EGA access-committee records are parts of a deposit and are not counted.
- **Produced, not cited.** A dataset enters the reuse sample only when the sampled NCI-funded paper produced it. The paper must be the earliest paper Europe PMC records naming the accession, and for GEO series and PDB entries the repository's own record must agree (section 4). The other databanks were checked against their repositories after the harvest (section 5).
- **Outside paper.** Any paper other than the producing paper that names the accession, after the corrections in section 4 remove co-producing papers, false matches and duplicate records.
- **Independent use, or use by an outside team.** An outside paper is independent when it shares no author surname with the producing paper. Author names are reduced to a surname key: the family name in lower case, with degrees and initials removed (`surname_key` in `code/measure_broad_reuse.py`). An outside team that happens to share a surname is not counted, so independent rates lean low.
- **Floors.** Europe PMC finds a paper only where its indexed text names the accession, and a paper can use data without naming it, so every reuse rate is a floor.
- **Associations.** Every comparison between groups is an association in a sample, not a cause.

## 1. The FY2026 portfolio in dollars

Script `measure_portfolio_dollars.py`; output `2026-09-24-nci-unseen-portfolio-dollars-v1.0.0.json` and, through `--projects-out`, the award links table `2026-09-24-nci-fy2026-award-links-v1.0.0.json`. Measured September 23, 2026.

**Input.** The public ScienceDex FY2026 NCI funding export (its URL is in the result's `export` block): 6,514 core projects and $4,132,427,741.

**Method.** Each core project goes into exactly one group.

- `official`: a deposit record names the award. DataCite was searched for every core project; a record counts only when the funding reference naming the project also names an NIH funder (1,047 records read, 80 set aside because the award number sat under a non-NIH funder). The CCDI Data Catalog was read in full; a record counts when its Grant ID names the project.
- `bridge`: no such record, but a paper NIH RePORTER links to the award names a deposit, in its MEDLINE databank field or through NCBI's curated PubMed-to-GEO link. Only deposit databanks with well-formed accession numbers count; trial registrations and reference databases do not. RePORTER linked 235,754 distinct papers to the portfolio. PubMed-to-GEO links were requested for the 14,519 linked papers inside PubMed's own "pubmed gds" filter, and a seeded sample of 2,000 papers outside the filter returned no links.
- `none`: neither route finds a dataset.

**Result.** `official` 232 projects ($440,133,517; 10.7% of FY2026 dollars); `bridge` 1,832 ($1,310,331,611; 31.7%); `none` 4,450 ($2,381,962,613; 57.6%). Counting only DataCite records typed Dataset, plus CCDI records, `official` would hold 150 projects ($325,463,278). `byActivityGroup` splits each group by award type, and `reconciliation` compares the rebuilt counts with the published ScienceDex releases.

**Checks.** All 22 checks in `checks` pass. Among them: the export sums to $4,132,427,741 over 6,514 distinct projects; every RePORTER batch equals the total RePORTER reports; PubMed served 235,574 of the 235,754 papers (the other 180 have no record); the parsed databank fields equal PubMed's own databank search on the same papers; and 40 of 40 sampled GEO series list the linked paper as their publication.

**The award links table.** One row per core project: `projectNumber`, `coreProject`, `activityCode`, `activityGroup`, `funding` (FY2026 NCI dollars), `status` (`official`, `bridge` or `none`), the counts behind the status (`officialDataCite`, `officialCcdi`, `publications`, `bridgeDatasets`, `bridgeSi`, `bridgeGeo`), the databanks found (`bridgeDatabanks`) and three flags (`officialDataCiteDatasetType`, `publicationListComplete`, `bridgeOnlyFromGeoPublicAfterImport`). It holds no person names. The dollar result records the SHA-256 of the table's rows as `perProjectTableSha256`. To check the table against it:

```python
import hashlib, json
rows = json.load(open("data/2026-09-24-nci-fy2026-award-links-v1.0.0.json", encoding="utf-8"))["projects"]
rows.sort(key=lambda r: (r["status"], -r["funding"], r["projectNumber"]))
text = json.dumps(rows, sort_keys=True, separators=(",", ":"))
print(hashlib.sha256(text.encode("utf-8")).hexdigest())
```

This prints `8b8d9c188384598208986dca8c4feab5947acf76045fa7fbd0c3914f26c3364b`, the value of `perProjectTableSha256`; the first check of the Index comparison (section 2) makes the same test. The rows count 232 `official`, 1,832 `bridge` and 4,450 `none`, and `funding` sums to $4,132,427,741. The digest covers the sorted rows, not the file's bytes, so it differs from the file's SHA-256 in `MANIFEST.json`.

**Rerun.** Download the export URL to `export-award.csv`, then `python code/measure_portfolio_dollars.py --export export-award.csv --work <scratch> --out <result.json> --projects-out <table.json>`. Every stage caches to `--work` and resumes.

## 2. The Index of NCI Studies comparison

Script `measure_ins_coverage.py`; output `2026-09-26-ins-coverage-v1.0.0.json`. Measured September 26, 2026.

**Inputs.** The award links table (`--projects`), the section 1 result, and release 2.4.1 of the Index of NCI Studies' public data (CBIIT/INS-Data on GitHub, downloaded to `--work`; its SHA-256 is pinned in `inputs.insRelease`). The release is the set of files loaded into the Index's site on July 9, 2026: 195 programs, 3,036 projects, 15,790 grants, 64,174 publications and 14,035 datasets.

**Method.** The Index's data model has no link from a dataset to a project or grant. The tie is two text fields on each dataset record, and both are counted. "Funding Source(s)" names, for GEO and SRA, the Index project whose paper NCBI links to the dataset, and for dbGaP it copies the study record's own funding attribution. "Cited Publication PMID(s)" is joined to every project the Index's publication table links to that paper. Grant numbers are read as an activity code, an institute code and a serial, with five-digit serials zero-padded. Each FY2026 project keeps its section 1 group, and the portfolio is regrouped three ways (`regrouped`): DataCite and CCDI records only; adding dbGaP study records that name the award; and adding every Index tie.

**Result.** The Index ties datasets to 222 of the 6,514 FY2026 core projects ($464,162,951; 11.2% of dollars). With every Index tie counted as part of the record, records show data for 389 projects ($665,829,504; 16.1%); the awards' papers alone find data for another 1,679 ($1,086,713,111; 26.3%); and 4,446 projects ($2,379,885,126; 57.6%) have no traceable data. Counting only records in which the deposit itself names the award (DataCite, CCDI and dbGaP study records), the record figure is 296 projects ($551,124,881; 13.3%). The publication route finds data for 217 of the 222 projects the Index ties to data (`publicationRouteAmongInsLinked`) and reproduces 2,449 of the Index's 2,454 GEO project-dataset ties (`depositLevel`). Of the Index's 1,068 dbGaP studies, 674 carry a funding attribution.

**Checks.** All 19 checks pass. Among them: the award links table is the one section 1 recorded (by SHA-256); the release archive matches its pinned digest; the Index's live site reported the release's counts, repository by repository; and a spot check of 20 randomly drawn Index ties (seed 20260926; 10 GEO and 10 dbGaP) found all 20 confirmed by the GEO or dbGaP record (`secondSourceSpotCheck`).

**Rerun.** `python code/measure_ins_coverage.py --projects data/2026-09-24-nci-fy2026-award-links-v1.0.0.json --work <scratch> --out <result.json>`. The optional `--portfolio-work` (the section 1 API cache, not included) adds the deposit-by-deposit comparison; without it that part is skipped and the group and tie tables are unchanged.

## 3. Audit of the paper route's links

Script `audit_paper_route_links.py`; output `2026-09-26-paper-route-link-accuracy-v1.0.0.json`. Measured September 26, 2026.

**Population.** The 1,679 core projects ($1,086,713,111) in which only the awards' papers show data (section 2), with their 18,388 project-dataset links rebuilt from the section 1 cache of RePORTER, MEDLINE and NCBI results. GEO holds 17,116 of the links.

**Sample.** Drawn once with seed 20260926 and frozen in the script: 50 links drawn at random from all 18,388; 50 projects drawn at random, with one link drawn within each; and, for the by-databank table only, 15 more links from each of PDB, BioProject, SRA and dbGaP.

**Method.** Each link faces three checks. (a) NIH RePORTER links the paper to the core project. (b) The paper's MEDLINE databank field, or NCBI's PubMed-to-GEO link, lists the accession. (c) The deposit's own record: a GEO series must list the paper as its publication; for other databanks the accession must resolve to a dataset record, and a record that credits only other papers counts against the link unless RePORTER links one of those papers to the same award. A link is correct when one of its supporting papers passes all three checks, and "cannot tell" never counts as correct. Every link the deposit's record did not confirm, and a seeded random 20 of the main 100, were read in full, paper and deposit record, by the analysis agent (Claude Opus 5.5): 73 links in all. Those readings are final and are recorded with their URLs and quotations (`handVerification`, and `HAND_CHECKS` in the script). Intervals are 95% Wilson intervals.

**Result.** 95 of the 100 main-sample links are correct (95.0%; 88.8% to 97.8%). The rules alone, without any reading, give 94 correct, 5 wrong and 1 undetermined. Every one of the 50 awards drawn by project keeps at least one correct link (50 of 50; 92.9% to 100%). Links that come from NCBI's PubMed-to-GEO link, alone or with the databank field, were correct 91 times in 91 (95.9% to 100%); links resting on the databank field alone, 4 times in 9. All five wrong links in the main sample name data the paper used from another study's deposit. Outside GEO the supplement shows lower accuracy, for example 11 of 19 PDB links. Without the 30 cancer center support grants (P30) the group holds $1,018,709,689, and without all 86 center and program-project grants, $898,809,413 (`dollars`).

**Checks.** All 13 checks pass. Among them: the population reproduces the published 1,679 projects and $1,086,713,111 exactly; every project's link count equals its recorded deposit count; the drawn sample equals the frozen one; every read link has a recorded reading; the paper's own grant list in PubMed names the award for all 100 main-sample links; and the reading agrees with the automated verdict on all 20 randomly chosen links.

**Rerun.** `python code/audit_paper_route_links.py --projects <award links table> --portfolio-work <section 1 cache> --ins-work <folder for the Index release> --out <result.json>`. The section 1 cache is not included.

## 4. The reuse sample (cohort)

Scripts `build_reuse_cohort.py` (harvest and merge) and `correct_reuse_cohort.py` (errata); output `2026-09-26-nci-reuse-cohort-v1.0.2.json`, cohort version 1.0.2, as of September 23, 2026.

**Frame.** PubMed records that Europe PMC links to NCI funding, through NIH's grant data (agency "NCI NIH HHS") or publishers' funder metadata (agency "National Cancer Institute"), by year of first publication from 2014 to 2025 (`frame`).

**Sample.** Each year is cut into 52 windows by first publication date. Each window's paper count is read and the window is listed in full. 3,000 papers a year are allocated to windows in proportion to their counts (largest remainder) and drawn at random within each window (seed 20260923); 36,000 papers in all (`funnel`, `papersSampled` for each year). The sample never takes the first papers of a listing, because Europe PMC lists the newest first.

**Datasets.** Accession numbers come from Europe PMC's text-mined annotations of each paper, classified with an allow-list of deposit databanks (`classify_accession` in `code/measure_route_coverage.py`). The distinct accessions are measured in a seeded random order until 125 datasets a year pass every screen, so the kept datasets are a random sample of those the sampled papers produced.

**Produced, not cited.** The sampled paper must be the earliest paper Europe PMC records naming the accession, reading every naming record; an empty or short answer is retried, never read as "no papers". For GEO series and PDB entries the repository then decides: a record naming the NCI paper confirms production, a record naming a different, earlier paper sets the dataset aside, and otherwise the release must fall between 12 months before and 30 months after the paper. Accessions the repository does not hold are dropped. This check set aside 301 datasets (`excludedByReleaseCheck`). In the final sample the repository's record names the NCI paper for 682 of the 780 GEO series and 321 of the 331 PDB entries; the rest passed on their release date.

**Contents.** 1,419 datasets, 107 to 124 per year: 780 GEO, 331 PDB, 119 BioProject, 97 PRIDE, 36 SRA, 22 dbGaP, 21 EGA and 13 ArrayExpress. Each row carries its producing paper, the paper's NCI core projects, the producing authors' surname keys, the repository check and every outside paper with its date and independence; `fieldNotes` explains the fields, and `funnel` records every step of every year.

**Errata 1.0.1**, applied to version 1.0.0 without re-harvesting. Two PDB rows were removed because their producing papers predate the structures' deposit (5CC7 and 5HRS). Papers the repository itself credits for a GEO or PDB deposit became co-producing papers rather than outside papers (45 on dataset rows). Mentions that use a PDB identifier's characters for something else became false matches (6), and duplicate records of one work were merged (15). Together these removed 41 of 1,797 independent paper-dataset pairs.

**Errata 1.0.2**, applied to version 1.0.1 without re-harvesting. The production review (section 5) found 79 BioProject, SRA, dbGaP, PRIDE, EGA and ArrayExpress rows that the NCI paper cited rather than produced. They moved from `datasets` to `removedByErrata`, each with its reason and evidence, and 19 rows the rule flagged were kept because the repository page shows the NCI paper's team deposited the data. The sample went from 1,498 to 1,419 datasets, from 1,756 to 1,624 independent paper-dataset pairs, from 1,678 to 1,557 distinct independent papers, and from 415 to 389 datasets with independent use (`errata.datasetsEffect`).

Every change in both errata is listed in `errata.changes` with its evidence, and `errata.releases` records the digest of each version's source file. Rows removed by either errata stay in `removedByErrata`; versions 1.0.0 and 1.0.1 themselves are not included. After the corrections, the 95% upper bound on the share of remaining pairs that use an identifier's characters for something else is 0.77% (`errata.residualFalseMatch`, from the reading in section 9).

**Rerun.** `python code/build_reuse_cohort.py harvest --years 2014-2025 --workdir <work>`, then `python code/build_reuse_cohort.py merge --years 2014-2025 --workdir <work> --out <cohort.json>`. The harvest checkpoints every stage and resumes. Europe PMC changes daily, so a new harvest reproduces the method and the rates within sampling error, not every count. `correct_reuse_cohort.py` applies errata 1.0.1 (`--cohort`, `--out`, `--cache`) and errata 1.0.2 (`--review`, which also reads the review's write-up; not included).

## 5. Production review of the other databanks

Script `review_unchecked_databank_production.py`; output `2026-09-26-unchecked-databank-production-review-v1.0.0.json`. It ran on cohort version 1.0.1, and its findings became errata 1.0.2.

**Rows.** The 387 cohort rows in databanks the harvest had not checked against the repository: BioProject 153, SRA 37, dbGaP 43, PRIDE 99, EGA 37 and ArrayExpress 18.

**Method.** The GEO and PDB rule, applied from each repository's own public record (`rule`). A record naming the producing paper, by PubMed ID, DOI or a matching title, confirms it. A record naming a different paper first published earlier sets the row aside; for EGA, a study page linking an earlier paper also sets it aside but never confirms one. Otherwise the release must fall between 12 months before and 30 months after the producing paper, or its preprint. An accession the repository does not hold is dropped, and a producing paper PubMed types as a review is set aside unless the repository names it. Release dates are each repository's analogue of GEO's public date (`rule.releaseDates`).

**Readings.** A seeded random sample of 48 rows (seed 20260926: 24 flagged, 12 confirmed and 12 kept on their release date) and 43 rows chosen for doubt were read on the repository web pages by the analysis agent (Claude Opus 5.5). The final rule agreed with the page on 45 of the 48 random rows (95% interval 83.2% to 97.9%). The recommended correction is the rule's removals, less the rows the pages show the NCI paper's own team deposited, plus the kept rows they show another study produced.

**Result.** The rule flags 96 rows. The recommended correction removes 79 (`removalSets`, `errataCandidates`) and keeps 19 on page evidence (`flaggedButKeptOnPageEvidence`). The file recomputes the reuse clock's headline figures before and after the removal (`headlinesBefore`, `headlinesAfterRecommendedErrata`) and counts what the removal touches in the other analyses (`downstream`). The v1.1.0 result files are those analyses rerun on the corrected cohort.

**Rerun.** `python code/review_unchecked_databank_production.py <cohort version 1.0.1> --out <result.json>`. Cohort version 1.0.1 is not included; version 1.0.2 holds the same rows with the 79 moved to `removedByErrata`.

## 6. The reuse clock

Script `measure_reuse_clock.py`; output `2026-09-26-nci-reuse-clock-v1.1.0.json`. Pure computation on the cohort file, with no network access.

**Method.** Kaplan-Meier cumulative first use, by any other paper and by an outside team, by years since the producing paper, with pointwise 95% confidence intervals (Greenwood, log-log) and a cluster bootstrap over producing papers (`bootstrap95`): pooled, by databank and by production period. The clock starts at the producing paper's first publication date and runs to September 23, 2026. The file also gives outside papers per dataset-year by age, the share of datasets gaining a new outside paper in the last 24 months, the concentration of use, and the directly observed history of the 363 datasets produced in 2014 to 2016, all followed at least 9.73 years (`oldestCohort`).

**Result.** For the 2014 to 2016 datasets, independent use is 42.4% today (154 of 363; 37.5% to 47.6%) and was 25.1% at five years (20.9% to 29.8%). At five years, 59% of today's independent use and 39% of today's outside papers were already visible. Pooled Kaplan-Meier independent use is 4.7% at one year (3.8% to 6.0%), 16.8% at three (14.9% to 18.9%), 24.1% at five (21.8% to 26.6%) and 39.7% at 11.5 (35.9% to 43.7%). Independent use by five years is 46.2% for PDB entries, 21.8% for GEO series and 6.1% for the other databanks (`byDatabank`). Among the 929 datasets at least five years old, the most-used 10% carry 69.9% of outside papers (`concentration`).

**Checks.** Kaplan-Meier estimates against crude shares, a design-weighted estimate, an alternative release rule (release within 30 months either way), the sample without preprints, the 2019 stratum against an earlier pilot sample, the cohort build year by year, and a later re-query of Europe PMC for 60 randomly chosen datasets, all 60 unchanged (`checks`).

**Rerun.** `python code/measure_reuse_clock.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --out-prefix <prefix>`. Rerun on this package, it reproduced the result file exactly, apart from `generatedAt` and `checks.laterRequery`, which needs the output of `build_reuse_cohort.py verify` (`--verification`).

## 7. Early signals of independent use

Script `measure_reuse_predictors.py`; output `2026-09-26-nci-reuse-early-signals-v1.1.0.json`.

**Analysis set.** Datasets with at least five years of follow-up (1,116), less those whose repository release date contradicts production (187): 929 datasets from 763 producing papers (`analysisSet`).

**Outcomes.** Independent use within five years, meaning an independent paper names the accession within 5.0 years of the producing paper: 236 datasets (25.4%). New independent use in years three to five, so that a signal measured over the first two years cannot count toward its own outcome: 192 datasets (20.7%).

**Features.** Visible at deposit: the databank; for GEO, sample count, series type, platform, organism and processed files (NCBI E-utilities); for PDB, method, resolution and bound small molecules (RCSB PDB Data API); the producing paper's MeSH cancer headings, author count and open-access status. Visible at year two: the producing paper's citations in its first two years (Europe PMC), and use by any other paper or by an outside team within two years.

**Statistics.** Risk ratios with 95% intervals that allow for several datasets from one paper (cluster-robust); Mantel-Haenszel estimates within databanks; a Firth-penalized logistic regression with six terms chosen before fitting and 1,000 bootstrap resamples of producing papers (`model`); and a held-out test that fits on the even publication years (2014, 2016, 2018 and 2020) and scores the odd ones (`validation`). A signal holds when its interval excludes no difference in the fitting years and again, in the same direction, in the held-out years.

**Result.** Of the datasets an outside team named within two years, 60.4% went on to new independent use in years three to five, against 15.3% of the rest (risk ratio 3.95; 3.13 to 4.99). For independent use within five years, PDB entries against GEO series give a risk ratio of 2.15 (1.68 to 2.74), human GEO series 30.2% against 14.0% for mouse, and an open-access producing paper 0.98 (0.77 to 1.26). The model fitted on the even years ranks the odd years' datasets with an area under the curve of 0.75 (0.70 to 0.81).

**Checks.** `secondSource` compares GEO sample counts and series types with GEO's pages, outside-paper dates with PubMed, citation counts with Europe PMC's citation endpoint and with OpenAlex, author counts with PubMed, and PDB fields with RCSB entry records.

**Rerun.** `python code/measure_reuse_predictors.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --out <result.json> --cache <dir>` (seed 20260923).

## 8. Where the data goes

Script `measure_reuse_reach.py`; output `2026-09-26-nci-reuse-reach-v1.1.0.json`.

**Papers.** One row per unique independent paper: 1,557, of which 1,553 resolved in OpenAlex, by PMID or DOI and never by search.

**Method.** Two sources place each paper: OpenAlex institutions (with ROR IDs, countries, types and US states) and the paper's own author addresses from Europe PMC, parsed for countries and US states without any OpenAlex input. A country or state counts for a paper only when both sources place an author there; 1,533 papers are placed by both. An OpenAlex institution counts toward institution types, minority-serving institutions and IDeA states only when an address on the paper names it with no conflicting country or state. Topics are OpenAlex primary topics, and a paper is cancer-focused when its topic sits in oncology or cancer research or names a cancer. Minority-serving institutions are matched by exact name and state to the US Department of Education's FY 2025 eligibility matrix, whose digest is pinned in the script. IDeA states follow the NIGMS list.

**Result.** Outside teams in 43 US states and DC and in 91 countries and territories. Of the 1,552 papers with a topic, 1,193 (76.9%) are about something other than cancer. 65 papers have an author at a minority-serving institution, and 39 papers an author at an institution in one of 17 IDeA states.

**Checks.** `secondSource` compares the two sources paper by paper: the country sets are identical for 1,418 of the 1,533 papers both sources place (92.5%).

**How version 1.1.0 was computed.** It is a recount on cohort 1.0.2 from the per-paper records of the first reach run, which used cohort 1.0.1. The script's own summary functions ran unchanged, with only the API fetch steps replaced by those records, and no API was called (`recount`). The recount helper and the first run's file are not included. Running `python code/measure_reuse_reach.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --out <result.json>` applies the same method to today's OpenAlex records.

## 9. What outside teams did with the data

Script `classify_reuse_outcomes.py`; output `2026-09-26-nci-reuse-outcomes-v1.1.0.json`.

**Sample.** A seeded simple random sample of 557 of the 1,624 independent paper-dataset pairs: the pairs with the lowest SHA-256 of the seed (20260924), the accession and the paper (`selection`). A pair's rank never changes, so this is the earlier sample less the pairs of datasets removed by errata 1.0.2.

**Text.** Free full text where it exists (Europe PMC's open-access XML, or NCBI PMC XML for NIH author manuscripts), otherwise the title and abstract. Europe PMC's accession annotations locate the identifier independently. Each row keeps the passages naming the accession, with their sections (`passages`).

**Labels.** 25 documented keyword rules propose labels (`method.rules`). A rule settles a row only when exactly one use label fires on a strong cue, a databank word stands near the identifier, the paper is not a review, and the mention is not confined to the introduction, discussion or references (`method.certainWhen`). Every uncertain or multi-labelled row was read by a generative AI model, Claude Opus 5.5 (Anthropic; model ID claude-opus-5-5), working as a Claude Code agent on September 23, 2026. It read the passage and, where needed, the paper's title and abstract, and recorded labels, a primary label and a one-line reason quoting the passage. It also read the rows the rules settled and changed 21 primary labels. The script itself calls no model: it reapplies a stored decision only while the SHA-256 of the passage it was made on is unchanged. In all, 155 labels come from the rules and 402 from the model's reading (`summary.labelSources`). A mention never counts as a use.

**Result.** Among the 545 sampled pairs whose production was confirmed against the repository's record, 428 (78.5%; 74.9% to 81.8%) show analytic use, and 86% of those whose text shows whether the data were analysed. Primary uses among them: drug design or docking 120, structural comparison or modelling 110, discovery by reanalysis 107, methods or tools 32, pan-cancer or meta-analysis integration 27, validation in an independent cohort 21, and AI or machine learning 11. At least 172 of the 389 datasets with independent use were analysed by an outside team (`summary.confirmedOnly`).

**Checks.** Europe PMC's annotations locate the identifier in all 557 pairs (`summary.extractionCheck`). A human agreement check on a stratified sample of 45 rows (seed 20260924) is pending (`agreementCheck`).

**Rerun.** `python code/classify_reuse_outcomes.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --work <dir> --reviews docs/evidence/2026-09-26-nci-reuse-outcomes-v1.1.0.json --max-rows 557 --out <result.json>`. Passing the result file as `--reviews` reapplies the recorded decisions. The optional `--workbook` needs a helper that is not included.

## 10. Replacement value

Script `estimate_replacement_value.py`; output `2026-09-26-nci-reuse-replacement-value-v1.1.0.json`.

**Method.** Following Milham and colleagues (Nature Communications, 2018; https://doi.org/10.1038/s41467-018-04976-1), units of reused data are multiplied by a cited cost per unit. Datasets with independent use are first checked again against their repository record, and those whose production the record contradicts are left out of the headline (`productionStatus`: 338 confirmed, 39 not contradicted, 12 contradicted). Units come from the repositories: GEO samples (a sequenced sample keyed to its SRA experiment, so a library is never counted twice), SRA and BioProject sequencing libraries, and one structure per PDB entry. For each assay the low and high costs per unit are the lowest and highest figures quoted by at least two independent public price sources, and the central value is their midpoint (`costTable`). Sequencing and array costs are assay costs only; protein-structure costs are published whole costs per structure. There are two conventions: each dataset counted once, with a unit shared by two datasets counted once; and a dataset counted again for every independent paper naming it, with units de-duplicated within a paper.

**Result.** 377 datasets and 1,455 independent papers. Each dataset counted once: $18,862,593 (range $11,836,075 to $25,889,110). Counted for every reusing paper: $111,782,649 ($71,661,029 to $151,904,270). Sequencing and arrays only, counted once: $2,880,093 (`sensitivity.headlineWithoutStructures`).

**Checks.** Sample counts of 10 randomly drawn GEO series agree with GEO's series pages (10 of 10), and the methods of 60 structures agree with PDBe (60 of 60) (`secondSource`).

**Rerun.** `python code/estimate_replacement_value.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --out <result.json> --cache <dir>` (seed 20260923). Always pass `--cache`, because the default is a folder on the authors' workstation.

## 11. Independent reuse per NCI dollar

Script `measure_reuse_per_dollar.py`; output `2026-09-26-nci-reuse-per-dollar-v1.1.0.json`.

**Method.** Datasets with at least five years of follow-up are split equally across the NCI core projects on their producing paper that RePORTER confirms received NCI funding. For each of its NCI grants, a producing paper is charged the grant's lifetime NCI funding in RePORTER divided by the publications RePORTER links to the grant, and its distinct independent papers are credited to its grants in equal fractions. A group's yield is its credited independent papers per $1 million charged. The dollar is a whole-grant dollar, never a cost of producing data. Intervals resample producing papers. Only aggregate groups are reported, and no investigator is named or ranked.

**Result.** 880 datasets, 1,167 grants and 728 producing papers, with $150,673,228 charged. 35.2% of the datasets were used independently (31.7% to 38.7%), and each $1 million charged was followed by 8.5 independent papers (6.6 to 10.6). By award type the yield is 13.5 per $1 million for career and training awards, 11.9 for centers and program projects, 6.4 for research project grants and 2.5 for cooperative and network awards (`tables.fractional.byActivityGroup`).

**Checks.** All 9 checks pass. Among them: RePORTER's activity code agrees with every core project number; FY2026 NCI funding in RePORTER matches the funding export for 349 of the 362 cohort grants in both; RePORTER's publication counts track PubMed's grant-number counts on a seeded sample of 40 grants; and every dataset and every producing paper is credited exactly once.

**Rerun.** `python code/measure_reuse_per_dollar.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --work <dir> --export export-award.csv --out <result.json>`.

## 12. Award size, career stage and institution

Script `measure_payroll_vs_wins.py`; output `2026-09-26-payroll-vs-wins-v1.1.0.json`.

**Method.** The units, charges and outcomes of section 11, unchanged; its base reproduces section 11 exactly (`base`). New groupings (`definitions`): the grant's average annual NCI funding, in fifths and tenths, with lifetime NCI dollars and thirds within research project grants as alternatives; the contact principal investigator's career stage when the producing paper appeared, from RePORTER (years since the investigator's first NIH award; early career means under 10 years) and, as a second definition, from OpenAlex (years since the last author's first publication; early career means under 15 years); the grant-holding organization's rank by NCI grant dollars in fiscal years 2014 to 2021 (top 20, ranks 21 to 100, beyond 100); and whether the organization held an NCI Cancer Center Support Grant (P30). Intervals come from 2,000 resamples of producing papers (seed 20260923). No investigator is named, ranked or written out.

**Result.** Across fifths of award size, the RePORTER career stage, the top 20 institutions against the rest, and institutions with and without a cancer center grant, 29.6% to 39.6% of datasets were used independently. 14 of the 15 reuse-rate comparisons have 95% intervals that include no difference (`main.contrastIntervals`); the exception is the OpenAlex career definition within research project grants (+18.0 percentage points; +0.2 to +32.1). The OpenAlex definition's early-career group overall was used at 42.1%. Per $1 million, the top 20 institutions were followed by 7.8 independent papers and the rest by 9.0 (ratio 0.87; 0.57 to 1.37). P30 grants themselves return 19.0 per $1 million (13.5 to 25.7), which reflects how widely center grants are acknowledged.

**Checks.** All 18 checks pass.

**Rerun.** `python code/measure_payroll_vs_wins.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --work <dir> --per-dollar-json docs/evidence/2026-09-26-nci-reuse-per-dollar-v1.1.0.json --out <result.json>`.

## 13. Ranking datasets at age two, late bloomers and release timing

Script `measure_draft_board_and_release_speed.py`; output `2026-09-26-draft-board-and-release-speed-v1.1.0.json`.

**Ranking at age two.** The early-signals analysis set is ranked within each publication year at age two in three ways: by the section 7 model fitted on the even years (imported from `measure_reuse_predictors.py`), by the single question "has an outside team used it yet", and by databank alone. On the held-out odd years, each ranking is scored by the share of later outside papers (after year two) that its top 10%, 20% and 30% of datasets capture. Datasets tied at a cut share it pro rata. Intervals come from 1,000 resamples of producing papers within publication years, refitting the model each time.

**Late bloomers.** Among the 565 datasets at least eight years old, the most-used tenth (57 datasets, ties shared), and how many of them had no independent use at year three or at year five.

**Release timing.** Days from the producing paper to public release: for GEO and PDB from the cohort's repository check, and for PRIDE and ArrayExpress from their own APIs. The association with use by year five is estimated within databank and publication year.

**Result.** On the 453 held-out datasets and their 948 later outside papers, the model's top fifth captures 53.9% of the papers (37.0% to 67.1%) and the single question's top fifth 52.4% (36.2% to 64.8%), against 20% by chance. About 7.5 of the 57 most-used old datasets (13%; 4% to 24%) had no independent use at year five, and datasets with no independent use at year five carry 20.1% of eventual outside papers (14.4% to 27.0%). Of 1,111 GEO and PDB datasets, 91.3% were public by three months after the producing paper, and the median gap is 0 days.

**Checks.** `checks` compares paper totals with the cohort's errata summary, outcomes with the section 7 rows, the refitted model with the section 7 fit, release dates with the section 7 fetch and a fresh sample of GEO pages and RCSB entries, producing-paper dates with PubMed, and the pro-rata tie rule with explicit random tie-breaks.

**Rerun.** `python code/measure_draft_board_and_release_speed.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --early-signals docs/evidence/2026-09-26-nci-reuse-early-signals-v1.1.0.json --out <result.json>`.

## 14. Paper citations and clinical reach

Script `measure_paper_fame_and_clinical_reach.py`; output `2026-09-26-paper-fame-and-clinical-reach-v1.1.0.json`.

**Question 1: do the producing paper's citations predict use of its data?** The producing paper's Relative Citation Ratio (RCR) from NIH iCite, and two journal rules fixed in advance, are set against the section 7 outcomes on the same 929 datasets. Intervals are clustered by producing paper; the headline comparisons are repeated with one dataset per paper and on held-out years, and every RCR result is repeated without citations from papers that name the paper's datasets. The same citation links, read from both sides, show how often a reusing paper cites the producing paper at all.

**Question 2: do reusing papers reach clinical work?** For the independent reusing papers: the share cited by a clinical article (iCite), the Approximate Potential to Translate and the median RCR. These are compared with three groups standardized to the same publication years: the cohort's own sample of NCI-funded papers, the producing papers, and a random sample of PubMed records drawn by day of entry (2% of each drawn day's records). The PMIDs of both samples are stored in the file (`frameSample`, `pubmedBaselineSample`).

**Result.** Datasets from the most-cited third of producing papers were used independently within five years 33.5% of the time, against 22.6% for the least-cited third (risk ratio 1.49; 1.12 to 1.98), and RCR alone ranks the datasets with an area under the curve of 0.593. In 698 of 1,582 independent uses (44.1%), the reusing paper does not cite the producing paper; among uses labelled analytic on repository-confirmed data, 215 of 421 (51.1%). The question 2 comparisons are in `part2.comparisons`.

**Checks.** `secondSource` checks the outcomes against the section 7 rows, iCite citation counts against Europe PMC, iCite's clinical flag against PubMed publication types, and a sample of citation links against Europe PMC's citation index.

**Rerun.** `python code/measure_paper_fame_and_clinical_reach.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --early-signals docs/evidence/2026-09-26-nci-reuse-early-signals-v1.1.0.json --reach docs/evidence/2026-09-26-nci-reuse-reach-v1.1.0.json --outcomes docs/evidence/2026-09-26-nci-reuse-outcomes-v1.1.0.json --cache <dir> --out <result.json>`. Without the cohort harvest's work folder (`--paper-sample-dir`), copy the published result to the `--out` path first; the script reads the stored sample of NCI-funded papers from it.

## 15. Reuse beyond NCI's grantees

Script `measure_reuse_beyond_nci_funding.py`; output `2026-09-26-reuse-beyond-nci-funding-v1.1.0.json`.

**Part 1: institutions.** The US institutions that section 8 confirmed on independent papers are joined, by name within the same state, to the organizations NIH RePORTER records as receiving NCI funding in fiscal years 2014 to 2026, subprojects included (2,174 organizations). The join tries the institution's names and OpenAlex alternative names, then its OpenAlex parents and ROR child organizations, so a medical school funded under its university counts as funded. Each of the 155 institutions the automatic join left unmatched was searched again and checked one by one against the raw affiliation strings, RePORTER and ROR. These checks were made by the analysis agent (Claude Opus 5.5), not by a person, and each verdict is recorded in the script (`HAND_CHECKS`). RePORTER lists prime recipients only, so "no NCI award" means no direct NCI award.

**Part 2: funders.** Each independent paper's funders come from the MEDLINE grant list, Europe PMC's grant list and OpenAlex, with NIH institutes and centers identified by MEDLINE agency, NIH grant number or funder name. Acknowledging a grant does not mean the grant paid for the reuse.

**Result.** Of 339 US institutions outside NIH, 73 (21.5%) held no NCI award from FY2014 to FY2026, and 69 of the 517 papers with a US institution include one of them (13.3%; 10.4% to 16.7%). Of the 1,557 papers, 365 (23.4%; 18.9% to 28.3%) name an NIH institute or center other than NCI, 191 (12.3%) name NCI, and 442 (28.4%) acknowledge NIH. Intervals come from 2,000 resamples of producing papers (seed 20260923).

**Checks.** All 11 checks pass. Among them: the section 8 result ran on this cohort file; the funding export reconciles to its pinned totals; and every US institution has exactly one status. `part1.matchAudit` records a seeded sample of automatic matches checked the same way.

**Rerun.** `python code/measure_reuse_beyond_nci_funding.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --reach docs/evidence/2026-09-26-nci-reuse-reach-v1.1.0.json --out <result.json>`.

## 16. SHARE 2.0 score and independent use

Script `measure_share_score_predicts_nci_reuse.py`; output `2026-09-26-share-score-nci-reuse-v1.1.0.json`.

**Method.** Each cohort dataset's SHARE 2.0 score was read from the ShareScore public API by exact identifier: GEO series, BioProjects, and SRA studies through their BioProject. The outcomes and the analysis set are those of section 7. The file reports Firth-penalized logistic regression, clustered risk ratios, the area under the curve with the raw score as the ranking, and the same even/odd-year held-out test.

**Result.** 928 of the 1,419 cohort datasets have a score, including 779 of the 780 GEO series; ShareScore holds no PDB, PRIDE, EGA or ArrayExpress repository. On the 511 analysed GEO series, 116 of them used independently within five years, scores span 12 to 24 of a possible 44 points, and 293 series (57%) share the top score of 24. The score ranks used above unused series with an area under the curve of 0.54 (0.49 to 0.59), and its association with use does not hold on the held-out years. On these GEO deposits, the score does not predict independent use. The scores describe the records as ShareScore harvested them in July 2026, not on the day of deposit.

**Checks.** Outcomes match section 7 for all 1,116 datasets with five years of follow-up; the score in ShareScore's list response equals the record detail for all 779 GEO records; and the arithmetic of all 928 score records is checked (`secondSource`).

**Rerun.** `python code/measure_share_score_predicts_nci_reuse.py docs/evidence/2026-09-26-nci-reuse-cohort-v1.0.2.json --early-signals docs/evidence/2026-09-26-nci-reuse-early-signals-v1.1.0.json --out <result.json>`.

## Shared code

- `reuse_common.py`: per-host pacing and retries for the public APIs, key loading from environment variables, the core-project rule, and the complete Europe PMC listing of every record naming an accession. The listing follows the cursor to the end and raises an error when it ends short, rather than returning a partial list.
- `measure_route_coverage.py`, `measure_broad_reuse.py` and `measure_dataset_discovery_routes.py`: earlier scripts from the same work, included because the cohort and award-size scripts import their accession allow-list (`classify_accession`) and author-surname rule (`surname_key`). Their own outputs are not part of this release.

## Use of generative AI

Claude Opus 5.5 (Anthropic), working in Claude Code, wrote and ran the analysis code. It also made every recorded reading in this release: the passages behind the 402 model-reviewed use labels (section 9), the 73 links read in full in the link audit (section 3), the repository pages read in the production review (section 5) and the checks of the institutions the automatic name join left unmatched (section 15). Each reading is recorded, with its reason, in the result file or the script. Every input was public, and no controlled-access data went to any AI tool. The human agreement check of the use labels (45 rows) has not yet been completed.

## Limits

The reuse results are sample estimates, and every reuse rate is a floor. The surname rule sets aside some genuine outside teams. Public sources change daily, so a fresh run reproduces the methods and the rates within sampling error, not every count. The Index of NCI Studies figures describe its July 2026 release. RePORTER links a paper to every grant it acknowledges, so a link shows what a funder can trace, not which grant paid for a deposit.