Library / Benchmarks
Benchmarks
Page version 0.1.0How fast each library database answers, whether its records name their sources, and how closely Trials matches the records it copies.
Checked on 15 Nov 2026
The databases
Every night we time three requests to each database, five times each: a search with no words, the first record it lists and, where records have sections, that record's first section. Every week we open the first 20 records of that search and check that every row names its source. A database is listed here once it passes every check of its speed, its release and its records.
| Database | Release | Median answer | Slowest answer | Errors | Newest release served | Rows naming their source |
|---|---|---|---|---|---|---|
| Genes | 11 Nov 2026 | 39 ms | 91 ms | 0 of 15 | Yes | 997 of 997 |
| Trials | 12 Nov 2026 | 48 ms | 198 ms | 0 of 15 | Yes | 411 of 411 |
| Trial comparisons | 23 Mar 2026 | 46 ms | 79 ms | 0 of 10 | Yes | 497 of 497 |
| Instruments | 15 Nov 2026 | 26 ms | 57 ms | 0 of 15 | Yes | 961 of 961 |
| Drugs and targets | 18 Sep 2026 | 31 ms | 61 ms | 0 of 15 | Yes | 2,071 of 2,071 |
| Diseases | 18 Sep 2026 | 28 ms | 53 ms | 0 of 15 | Yes | 2,264 of 2,264 |
| Companies | 14 Nov 2026 | 28 ms | 39 ms | 0 of 15 | Yes | 1,960 of 1,960 |
| Cell lines | 14 Nov 2026 | 30 ms | 101 ms | 0 of 15 | Yes | 1,460 of 1,460 |
| Journal index | 6 Nov 2026 | 29 ms | 84 ms | 0 of 15 | Yes | 1,898 of 1,898 |
- Newest release served: the database serves the newest release built for it. The Release column gives its date.
- Rows naming their source: the rows of every section of the first 20 records, each naming its source with a record ID or a link.
- Pass marks: no errors, 95% of answers within 2,000 ms, the newest release served, and every row naming its source.
Trials
The checks ask whether a person can find the trial they mean, then read what it registered and posted as ClinicalTrials.gov states it.
Release of 12 Nov 2026, checked on 15 Nov 2026
Records compared with their sources
| Compared | With | Checked | Same | Updated at the source after our copy | Different | Pass mark |
|---|---|---|---|---|---|---|
| The facts at the top of a trial's record | ClinicalTrials.gov | 200 trials | 197 | 3 | 0 | 190 of 200 |
| Posted results of the first three outcomes | ClinicalTrials.gov | 406 values | 402 | 4 | 0 | 95% |
| Adverse event totals | ClinicalTrials.gov | 1,170 rows | 1,154 | 13 | 3 | 95% |
| Participants who started and completed each group | ClinicalTrials.gov | 150 trials | 149 | 0 | 1 | 95% |
| Papers linked to a trial, with their type | ClinicalTrials.gov | 200 links | 200 | 0 | 0 | 190 of 200 |
| NIH grants linked to a trial | NIH RePORTER | 100 links | 97 | 0 | 3 | 95% |
- The facts at the top of a trial's record are its title, status, phases, enrollment, lead sponsor, conditions, interventions and the day its results were first posted.
- Updated at the source after our copy: our value differs from the source's current one, and the source updated that record after the day of our copy.
- Pass mark: how many must be the same, or updated at the source after our copy.
- Posted results: 4 more values could not be judged, because ClinicalTrials.gov gave nothing to compare them with.
| Counted on the day of each copy | Our copy | ClinicalTrials.gov | Pass mark |
|---|---|---|---|
| Registered trials | 611,552 | 611,551 | Within 0.5% |
| Trials with posted results | 77,078 | 77,108 | Within 0.5% |
- Each count is made on the day of its own copy: one of the registry, and one of the posted results.
- Since our copy of the posted results, 3,921 more trials have posted their first results at ClinicalTrials.gov. Trials does not show those results yet.
Finding a trial
| Search by | Searches | Listed first | In the first five | Pass mark | ClinicalTrials.gov's own search, listed first |
|---|---|---|---|---|---|
| NCT number | 200 | 200 | 200 | All listed first | 200 of 200 |
| Title | 200 | 198 | 199 | 190 in the first five | 167 of 176 |
- NCT number: the trial's ID at ClinicalTrials.gov.
- Title: the trial's brief title, as ClinicalTrials.gov gives it.
- 24 of the 200 title searches sent to ClinicalTrials.gov's own search failed, so its column counts the 176 it answered.
- Search speed: 60 timed searches for conditions, drugs and outcome measures, the same each night. Half answered within 641 ms and 95% within 1,329 ms, with no errors. Pass mark: no errors, and 95% within 1,500 ms.
Questions it answers
| Question | Trials asked | Answered | Not found | Wrong answer | Could not be judged |
|---|---|---|---|---|---|
| What did this trial's primary outcome show? | 60 | 58 | 0 | 0 | 2 |
| Which papers report this trial's results? | 60 | 60 | 0 | 0 | 0 |
A question is answered when a search finds the trial and its record gives the answer ClinicalTrials.gov gives. It could not be judged when ClinicalTrials.gov gave nothing to compare with. These questions have no pass mark yet.
How the checks work
A separate program runs these checks through conductscience.org's public API, the same one anyone can use. It uses none of the code that builds the databases.
It draws the records it compares in an order set by a hash of each record's ID and the release's name, so no one chooses them. It compares each value with the source's record on the day of the check.
Speed and searches are checked every night. Records are compared with their sources every week, and again whenever a database serves a new release.
A check appears on this page once it passes its mark. Some checks, such as the questions asked of Trials, have no pass mark yet. We show them only after we have reviewed their numbers. A check that fails goes to the team that builds its database, and appears here once it is fixed.
Sequences, which finds the patents that disclose a sequence, keeps its own benchmarks.
Page history
0.1.0, 16 Nov 2026
The first page: speed, release and sources for nine databases, and how Trials compares with the records it copies.