Humanity Top 10K

Abstract

Humanity Top 10K is a model evaluation of documented human consequence. The Human Leaderboard decomposes historical biographies into contributions, compares their relative magnitude, assigns individual responsibility, and discounts overlapping effects before producing person-level estimates. The released snapshot contains 11,949 people and 131,478 compared contribution records, supported by 1,240,630 comparison observations. A probabilistic ordinal model places contributions on a common relative scale; separate rules account for scope, shared authorship, intended and unintended effects, deaths, and posthumous legacy. An independently fitted importance ranking provides a second view of historical significance.

This report specifies the computation and distinguishes fitted quantities from rubric choices. It documents conditional bootstrap intervals, comparison consistency, coverage checks and sensitivity diagnostics. The output is an explicit, inspectable synthesis of model judgements about historical evidence. Neither a high rank nor a narrow interval establishes causal identification, moral authority or an exhaustive account of a person’s life.

“Top 10K” names the release; all evaluated records are included, even beyond rank 10,000. This report describes the v5.4 estimator. The pilot README and earlier field-coefficient studies describe predecessor experiments, with different absolute-score transformations and judge protocols.

01What the model estimates

The target construct is the consequence of a person’s recorded actions for other people’s lives. Magnitude, direction and attribution are distinct questions. An invention can have wide consequences without all of them belonging to its most famous contributor; a political action can be large in impact while harmful in direction; an influential idea can be historically consequential even when its truth is disputed.

For person p, the release reports unsigned impact I, benefit B, harm H, and their signed balance N. Benefit and harm are non-negative internally. The interface prefixes harm with a minus sign and draws negative net values to the left of zero.

Net benefit sorts by descending N; net harm sorts by descending −N. Impact is constructed through the unsigned aggregation in §7 and need not equal benefit plus harm. Importance is a separate percentile, not an additional term in this identity. Keeping these outputs distinct prevents a large adverse consequence from being mistaken for a beneficial contribution.

The scores are relative units, not people saved, money, IQ, QALYs or a universal unit of welfare. Construct validity concerns the interpretation warranted by the evidence, rather than the mere existence of a reproducible number.¹ Choices about aggregation and compensation between dimensions are part of the measurement definition; composite-indicator methodology makes those choices explicit rather than treating them as facts supplied by data.²

02Population & evidence

2.1 Candidate frame

The v5.4 prescreen starts from Wikidata-linked biographies with an English Wikipedia article and coverage in at least 15 language editions. The recorded frame contains 65,233 deceased candidates, including 133 flagged as possibly fictional; dated deaths must be no later than 2023. Existing run members are retained, including seven outside that frame. A six-level preliminary judgement uses the name, description, lifespan and a short article lead. These are selection variables, not final contribution scores.

The uncertain selection band is then compared against 60 anchors, with eight scheduled comparisons per candidate. The stored prescreen records 282,164 answered comparisons and selects a pool of 12,000. The published evaluation contains 11,949 records after subsequent processing. Neither number makes the result a census of all consequential humans: inclusion depends on surviving documentation, multilingual coverage, extraction and eligibility.

Wikidata supplies structured identifiers and biographical metadata.³ Pantheon supplies a fame-related comparison signal for diagnostics, while the cross-verified notable-person database illustrates the broader problem of coverage and historical visibility.^{4, 5} These resources do not provide a ground-truth ranking of consequences. Research on gender inequality in Wikipedia further motivates treating the candidate frame as selected evidence, rather than a representative sample of human achievement.⁶

2.2 Contribution ledgers

Sources become structured ledgers distinguishing beneficial and harmful effects, changes of rule, war launches, defence, revolutions, unifications, religious foundings, ideas, a life’s work and posthumous legacy. Completion passes recover items beyond initial extraction limits and ask for substantial omissions. The idea pass captures intellectual work that action-focused summaries may miss. Completion means the specified pass was performed; it does not establish that every historical consequence has been recovered.

A record carries a description, time context and reached population, with assessments of magnitude, sign, scope and the person’s part. Shared achievements may map to one fitted node while retaining separate person-specific attribution. Thus the record count can exceed the comparison graph’s node count. A book recorded both as a work and as an idea should not automatically create two independent contributions.

2.3 Counterfactual interpretation

The rubric asks how much a deed changed lives beyond what people would otherwise have had at that time. This gives the comparison a counterfactual orientation. It does not identify a historical treatment effect from experimental or observational data: there is no fitted causal graph, justified exchangeability condition or observed control world. Formal causal inference requires assumptions connecting observations with counterfactual quantities.⁷ Minimal-change historical counterfactuals discipline the question but cannot make an alternative history directly measurable.⁸

One deliberate asymmetry is retained: intended harm is counted without automatically discounting it because another actor might have inflicted similar harm. Benefits are assessed against what was already available or likely to emerge. This is a responsibility rule in the rubric, not a theorem of causal inference.

03Judgement protocol

The main deed-magnitude comparisons use the run’s Mercury judge. The surrounding pipeline contains extraction, grounding and audit stages with other configured models. It is inaccurate to describe this release as an average of three independent judges merely because the earlier pilot used that protocol. Repeated prompts from one model also do not constitute independent historical experts.

A comparison asks which contribution had the larger effect and by approximately how much. Seven substantive responses span the following ratio intervals; an insufficient-evidence response removes weight rather than assigning a tie or a low score. The fit retains probability mass over categories instead of replacing each answer with its most likely option.

Comparisons combine nearby contributions, random cross-scale links and additional coverage or adaptive pairs. Connectivity is essential: disconnected groups cannot acquire a common scale from within-group comparisons alone. Sparse comparisons are useful only when they form a sufficiently connected design. Shared source material and shared judge parameters still introduce dependence.

A contribution set out within a faith, church, school, movement or other body already established is compared with that status stated. A gospel, a creed, a council’s canon, a rite, a monastic rule or a commentary then counts what it added to what the body’s members would have believed, done or had anyway; the body’s founding counts its members, and a text or rule written within it is not read as taken up by all who belonged to it. The same classification removes the follower count that idea records otherwise carry into the comparison. Whether a deed founded a body, was set out within one or was taken up outside any is itself a judgement by the model, pooled over two option orders, and a deed is treated as within a body only when that reading carries at least half the probability. Findings, inventions and scientific, political or moral ideas taken up by people on their own judgement fall outside this rule even where a discipline or a party later built on them.

Attribution probabilities are pooled over four option orders: forward, reverse and seeded permutations. For option q and order r, the pooled probability is the arithmetic average of within-order normalized probabilities:

Pooling and a fitted first-position term address specific presentation effects. They do not remove all evaluator bias. LLM-as-judge research documents position effects, verbosity effects and self-preference; task-specific agreement with people cannot be transferred directly to historical evaluation.^{9, 10, 11} These failure modes inform the protocol. The cited studies do not validate this leaderboard.

04Relative magnitude

4.1 Interval-censored comparative model

The deed model belongs to the comparative-judgement and ordinal-response traditions.^{12, 13} Let s_i and s_j denote log magnitudes and b a shared first-position effect. The latent comparison is a difference in log magnitude with Gaussian noise:

For category c with log-ratio limits ℓ_c and u_c, its probability is an interval probability under the standard normal cumulative distribution function Φ.

The objective is the probability-weighted negative log likelihood plus Gaussian regularization. Here θ denotes the coordinates actually optimized; m_d and τ_d are their prior means and standard deviations. The default node prior has mean zero and standard deviation 10 in those coordinates. Free thresholds entail a later rescaling, so this width must not silently be interpreted as identical across calibrated scales.

Weights are not renormalized after insufficient mass is dropped. An observation carrying little substantive probability consequently exerts less influence. Regularization stabilizes weakly constrained nodes; it is not evidence that an undocumented contribution was negligible. The implementation uses Newton updates, conjugate-gradient solutions for the sparse node Hessian, and alternating threshold updates.

4.2 Estimated thresholds and scale calibration

The code supports fixed log-ratio thresholds and a free-threshold probit model. In the latter, noise initially has standard deviation one and position asymmetry is absorbed into six ordered thresholds. Estimated thresholds t are aligned with stated thresholds z by least squares:

The equivalent bias-adjusted thresholds are (t_c−â)/m̂; they need not equal the stated thresholds. Calibration makes the stated intervals hold approximately across the scale. Fivefold held-out likelihood is available for selection, following the logic of cross-validation.¹⁴ The stored large-run analysis explicitly selects the free model through configuration and reports that no cross-validation was run at this scale. This is not a fresh validation result.

The board reports a calibrated noise scale of 1.774 and a second-position factor of 1.256. A factor above one indicates that the second-presented deed is favoured before the fitted adjustment. Readers should not apply that adjustment again to displayed scores.

4.3 Reference unit

The pinned reference is one contribution associated with James Levine: music director of the Metropolitan Opera, conducting 2,577 performances, for operagoers and audiences of Met broadcasts and recordings. Its relative fitted magnitude is one. This anchors units, not a moral assessment of the whole person. Another fixed reference would rescale comparable raw magnitudes without, by itself, changing the relative evidence.

05Scope & calibration

5.1 Reach beyond the local setting

Scope has five ordered levels: a town or small group; a province or city; a country, empire or people; a civilisation or continent; and the world. The expected level uses the full probability distribution. Shared fitted achievements use the mean of their available record-level assessments.

The reference level is 2.733. The factor K = 3 is an explicit project choice, not a parameter estimated from the magnitude likelihood or supplied by a paper. One extra scope level multiplies size by three; two levels multiply it by nine. Fractional levels interpolate multiplicatively.

Because the magnitude prompt already mentions reach and duration, this extra multiplier can amplify reach twice. Its rationale is to make the release’s emphasis on broad consequence explicit; its cost is sensitivity to scope classification. The stored sweep evaluates K ∈ {1, 2, 3, 5, 10}. Sweep results and final rankings can differ because later calibration and lifetime-prior refits alter scores. A sweep-stage top-50 count is not necessarily the final top-50 count.

5.2 A common line for deaths

For eligible killing records with at least 1,000 deaths, independently fitted size is replaced by a common log-linear calibration. The intercept is fitted from source-stated counts; the slope is fixed at one. The same line is used for qualifying records whose counts were estimated by the judge.

The published calibration uses 545 source-stated anchors and an intercept of -9.04. It implies about 8,440 deaths per reference unit, with a bootstrap range of 6,690–10,100. This is a conversion inside this model, conditional on its comparisons and reference. It is not an empirical price of a life, a welfare valuation or an independently verified death toll.

The line is applied after scope scaling and re-estimated in bootstrap calculations. Attribution and responsibility remain separate. The audit distinguishes source-stated counts from model estimates; filling every record is a completeness result, not historical precision. Wars also distinguish the consequences of launching conflict from its fighting deaths and apply explicit rules about sides and responsibility. These choices require scrutiny independently of the fitted line.

06Individual attribution

6.1 Standing, collaborators and responsibility

Size comparisons concern the whole deed. Attribution asks how much belongs to the person. Standing distinguishes sole originators, heads, near-equal collaborators, executors, performers, advocates and sponsors. Responsibility is assessed separately for benefit and harm. Keeping these terms distinct prevents an institution’s consequence from automatically being assigned in full to each participant.

Here g is the effective share after applicable standing, collaborator and counterfactual adjustments; r is responsibility; and w is the sign weight. Effective factors also encode type-specific rules. Unsigned contribution uses its own multiplier c_pd; it should not be reconstructed by assuming every record’s displayed positive and negative numbers add to its impact value.

Collaborator-count categories use representative values 2, 3, 5, 9, 20 and 60. Shares are averaged over pooled probability distributions rather than assigned from the most likely category. A count-dependent standing averages its share function over the count distribution; this is generally not the function evaluated at the mean count.

This is the base expectation; necessity and substitute rules modify relevant entries of f. Fixed standing values ignore the count distribution. The fallback where no count assessment was requested is three collaborators. These constants are rubric assumptions, not all deductions from cooperative game theory. Attributions are not guaranteed to sum to one across every participant recorded in the dataset.

6.2 Necessity and available substitutes

Executors, advocates and sponsors may receive more credit where their input was necessary. The counterfactual distinguishes a deed that would not have occurred, one that would have occurred later or less fully, and one that would have occurred much the same. Later-or-less uses one tenth; “anyway” retains the standing’s base value. This remains a model’s reading of evidence, not a historical experiment.

For an indispensable input with plausible alternative suppliers, a particular cooperative game supplies a reference share. A unit-value achievement needs a makers’ coalition and any one of m + 1 interchangeable suppliers. In a random ordering, one supplier is pivotal when the makers precede it and all other suppliers follow it:

This Shapley value is 1/2 with no alternative, 1/6 with one and 1/12 with two. Uniformly averaging the one-or-two band gives 1/8; averaging three through six gives 1/32. The implementation floors the result at the standing’s base share and uses that base for “many” substitutes. The formula follows the stated game’s assumptions;¹⁶ its historical application remains a choice. The floor, band averages and representation of the makers as one coalition are additional assumptions, not general Shapley theorems.

Further audits adjust particular cases, including the benefit attributed to a defensive leader when a plausible replacement would have led the same successful defence. Benefit and harm can receive different effective shares. A scalar labelled “part” is therefore not an invariant ownership percentage across all outcomes.

6.3 Decision and execution in acts of state

For changes of rule, war launches, defence, revolutions, unifications and recorded harms, a standing that implies decision is followed by a second question read against the source’s passages: who decided that the deed would be done, and what part the person had in that decision. Six readings are distinguished. The person decided it; decided it together with others in a council, cabinet, junta, parliament or alliance; directed the means, places, pace or people of what another had decided; executed a part of it under higher command, however high their office; urged or advised it; or had no part in the decision. The effective part is the expectation over the pooled readings:

One who decided keeps the graded share; one who decided together keeps it where the standing already counts the deciding body. Direction, execution, urging and absence are capped as shown. The rule under which a head bears a killing’s whole harm applies only in the measure that the person is read as having decided it. A minister who signed, a commander who carried out and an envoy who transmitted what a head of state had decided therefore no longer carry a head’s share of the consequence; the head of state does. Where the source is silent, the model reads the office and the event as historians now judge them, which remains a reading rather than a finding of fact.

A second pass re-read the deeds first read as urged or as without part under two further rules: a member of the council, cabinet, conference or leadership that took the decision, who proposed or pressed for it there, decided it together with the others however the source words it, since urging is of one outside the body that decided; and one who proposed a deed to the one who decided and then ordered or organised its carrying out directed it rather than urged it. Of 920 such readings, 51 became together, 78 directed, 29 decided and 3 executed, and 759 stayed as first read. The pass corrected readings in which a chief of staff pressing for war in the conference that declared it, or a minister who proposed a massacre and had his own service carry it out, had been read as advisers.

6.4 Attribution by record rather than by title

Standing and necessity grades for the deeds of every woman in the population, and for a seeded control sample of men drawn at a fixed rate, were asked again with the source’s passages under one further instruction: judge by what the record shows each person did, as historians now judge, not by whose name stood first, who held the post, chair or title, who was awarded or honoured, or who was the husband, father, brother or superior of another who did it; where a collaborator’s part was later found to have been slighted in the credit given at the time, grade the part the record now shows, no more and no less. Shared achievements with a woman among their people were graded again under the same instruction, for every person in them.

This is a reading rule applied to evidence, named after the pattern Rossiter documented in the history of science.^{15, 6} It adds no weight, quota or bonus keyed to sex, and the control sample shows how much of any movement the instruction produces for men’s deeds as well. Where the record itself omits a contribution, the rule cannot restore it; the biographical sources carry their own documented imbalance.

07Overlap & aggregation

7.1 Discounting repeated effects

Simple addition rewards verbose ledgers: one effect can appear as an invention, a publication, an application and a lifetime achievement. The Human Leaderboard groups records identified as the same event. Within a group, the record with the largest unsigned attributed value represents that event across benefit and harm, preventing different copies from independently claiming opposite sides.

For a chosen quantity, surviving positive-valued records are ordered from largest to smallest. A record’s residual fraction is the product of the fractions not already included in earlier records. The overlap-adjusted union is:

Overlap is directional: an episode may lie almost wholly within a campaign without the campaign lying wholly within that episode. Benefits and harms use their own value ordering after common duplicate-group representatives have been selected. The unsigned union orders by unsigned value. Residual fractions for the same deed can therefore differ between outputs.

This product is a pragmatic discount, not exact inclusion–exclusion over identifiable affected populations. Pairwise overlaps do not determine all higher-order intersections. Results can depend on order, deed boundaries and duplicate detection. War-launch effects and their separately recorded fighting deaths are distinct by construction; same-death audits can raise overlap where casualty descriptions would otherwise be counted twice.

7.2 Life’s work and legacy

A life’s-work record supplies a broad estimate that can cover effects omitted by a finite ledger. It is not added wholesale to the ledger. The analysis compares the deed union with attributed lifetime value plus legacy outside the other deeds, taking the larger branch:

Legacy concerns effects of the person as a posthumous figure, separately from continuing effects of their own works and ideas. Lifetime attribution is capped by the size-weighted attribution of its component deeds when this is lower than the direct lifetime assessment. A broad career description consequently cannot simply bypass the collaboration discounts on individual work.

The analysis also refits lifetime log magnitudes under empirical priors centred on the person’s deed union. A preliminary fit supplies the median log lifetime-to-union ratio δ, a robust spread based on the median absolute deviation, and a bootstrap-noise estimate. The prior width subtracts that noise variance, subject to a minimum width:

The subtraction in (18) returns the scope-adjusted target to comparison-model coordinates. Priors are rescaled into free-probit coordinates during fitting. This is an empirical regularizer estimated from the same data, not an independent lifetime observation or a fully integrated hierarchical posterior. Its hyperparameters remain fixed in the subsequent bootstrap fits.

7.3 Signed totals

Signed totals use eligible signed deeds and side-specific shares. The lifetime fallback supplies unsigned impact; it does not manufacture a signed balance for the entire life. Records withheld from signed presentation are excluded from signed ranking rather than assigned zero. This distinction matters for both ranks and missing values.

08Independent importance

Importance is independently fitted from person-level pairwise judgements using a regularized Bradley–Terry model.¹⁷ It asks a broader significance question and does not sum the deed ledger. A position parameter h accompanies person parameters η:

The soft target allocates half the “close” probability to each side, while insufficient mass reduces the observation’s weight. This is a weighted binary model, not a separately fitted tie process.

Zero-weight observations contribute no evidence. Latent parameters become empirical percentiles using ascending midranks for ties. For M people and one-based ascending midrank r̄:

A value of 99 means a position near the top of this evaluated population. It is neither a 99% probability of being important nor 99 units of effect. Disagreement between importance and impact can reveal a mismatch between holistic significance and the explicit ledger; neither output automatically validates the other.

09Uncertainty & reliability

9.1 Conditional bootstrap intervals

The export reports 100 bootstrap fits. Each draw resamples comparison observations with replacement, refits the model and recomputes relative sizes and downstream quantities. Displayed intervals use the 10th and 90th percentiles:

The bootstrap approximates sampling variation by resampling observed data.¹⁸ Here its interpretation is conditional on the judge, sources, prompts, extracted deeds, scope, sign, overlap and attribution assessments. Those inputs are not all regenerated in every draw. Major uncertainties remain outside the interval: omissions, source errors, alternative counterfactuals and systematic evaluator bias.

The deed bootstrap resamples rows, not independent people or historical events. Related prompts and repeated comparisons can remain dependent, so nominal coverage is not established by the procedure alone. The separate importance analysis resamples pair keys, keeping associated observations together. These resampling units must not be described interchangeably.

Rank intervals rank each bootstrap draw before taking quantiles. They summarize positional instability within this candidate set. Overlapping score intervals are not a formal equality test; non-overlap does not replace a prespecified contrast. Net uncertainty is calculated from joint draws of benefit and harm, not by subtracting endpoints of separate intervals.

9.2 Split-half consistency

Comparison observations are randomly divided into halves. Each half is fitted, and Spearman correlation is calculated over nodes appearing in both. Ten splits are used. The published corrected reliability is 0.986; the underlying half-split result appears in the audit below. The Spearman–Brown transformation is:

This correction relates reliability under an idealized change in test length.¹⁹ Here it is a consistency diagnostic: halves share the judge, rubric and source world. Global correlation can be dominated by broad differences between very large and small deeds. A high value neither verifies every biography nor guarantees stable ordering of adjacent people at the top.

10Validation & sensitivity

Validation separates three questions: whether required inputs exist, whether comparisons behave consistently, and whether outputs support their intended interpretation. Coverage checks address the first; split-half fits help with the second; the third needs external historical and substantive evidence. A completeness threshold cannot answer all three.

10.1 Published audit checks

The export records 18 passing checks out of 18. They cover signs, contribution types, casualty and side assessments, connectivity, comparisons, overlap, scope, ideas, fields, completion, attribution and selected distributional diagnostics. Passing means the recorded rule’s threshold is met; it does not imply independent historical endorsement of each judgement.

10.2 Fame-adjusted residual checks

Era and field diagnostics regress log impact on Pantheon’s Historical Popularity Index. Residual comparisons ask whether groups are unusually high or low conditional on that fame measure. Schematically:

The era comparison contrasts people active from 1946 with those active before 1500. Other checks examine thought and letters and a recorded saints subset. These are conditional distribution checks within the HPI-covered subset, not causal estimates of discrimination, proof of fairness or estimates of missing contributions. Popularity can share the visibility biases of the biographies used to construct the ledger.

10.3 Sensitivity to assumptions

Useful sensitivity analyses vary a substantive assumption while keeping evidence fixed: scope strength, standing shares, substitute availability, casualty allocation, overlap or the lifetime branch. Results should identify rank movements, sign reversals and which deeds caused them. Two runs with different ledgers and rubrics cannot isolate the effect of one parameter.

A check on sex cues was run on this release. Seeded samples of 1,500 comparisons between a woman’s deed and a man’s, 1,500 between two men’s deeds as a control, and 160 comparisons of the deeds of Nightingale, Curie, Lovelace and Lamarr were asked again in two forms: blind, with both names removed and pronouns and sex-marked titles neutralised, and swapped, with the target’s person presented under a given name, pronouns and titles of the other sex. The quantity compared is the expected log ratio towards the target deed, new answer minus recorded answer. Blind, women’s deeds read 0.077 ln smaller (standard error 0.019) and the control’s 0.013 larger (0.019); swapped, women’s deeds presented as men’s read 0.199 smaller (0.016) and men’s presented as women’s 0.120 smaller (0.015). Removing or reversing a woman’s cues therefore lowers the reading of her deed by about nine per cent relative to the control, and the four named women’s deeds depend on the name more than most (Lovelace −0.59 ln blind). The judge shows no surface penalty on women’s deeds in this release; the recorded sizes stand and no blind re-ask was made. The check measures presentation, not the completeness of the record, which sections 6.4 and 11.3 address.

The repository includes order, granularity, drift, sensitivity and sex-cue experiments. An earlier small-run test cannot be promoted to a result for every person in this release. Likewise, a parameter chosen after inspecting prominent rankings remains a development choice. Stronger future validation would preregister criteria and evaluate them using independent adjudications and held-out historical cases.

11Limits of interpretation

11.1 Visibility is not consequence

Wikipedia language coverage helps identify documented biographies but favours people visible to later institutions and editors. Lost texts, colonial documentation practices, unnamed collective work and unequal access to publication shape the input. Demographic filters describe evaluated records, not the distribution of consequence in the world population.

Collective-memory research models the decline of attention;²⁰ citation research addresses differences between scientific fields;²¹ science–patent links trace one route from scientific work to technology.²² These distinctions motivate separating attention, intellectual dependence and downstream effect. They do not justify using a citation half-life as an effect half-life or transferring a universal culture-to-welfare coefficient into this model.

11.2 Common units involve judgement

Comparing a scientific advance, an institution, a musical career and a violent campaign requires commensurability assumptions. The ordinal likelihood makes comparisons coherent on a fitted scale conditional on the judge. It cannot prove that every consequence has a uniquely correct ethical magnitude. Netting also assumes that benefit and harm can compensate in the way the rubric permits; readers with different values can prefer different aggregations.

The magnitude prompt considers reach, intensity and duration jointly. The current estimator does not separately measure these and integrate verified person-years. It should not be described as a validated population integral because an earlier experiment explored one. The anchor unit is conventional and the scope coefficient chosen; neither acquires empirical authority from formal notation.

11.3 Historical attribution is incomplete

Named individuals often stand for organizations, accumulated knowledge and work by people absent from surviving records. Shared nodes and collaborator discounts reduce some overcounting but do not reconstruct all those contributions. Necessity can be fragile when substitutes are poorly documented. A model can produce precise probabilities over ambiguous accounts without those probabilities being calibrated historical frequencies.

Large estimates may depend on a few high-consequence hypotheses. They deserve deed-level inspection: what happened, what is inferred about the alternative, and how far downstream consequences are assigned. The interface exposes this chain so the model’s judgement can be examined; it does not replace specialist historical research.

11.4 Intended interpretation

The release supports exploring comparative judgements, inspecting influential assumptions and identifying records for review. It does not support conclusions about innate worth, intelligence or the merit of demographic groups. A rank is an output of this dataset and rubric, not an intrinsic property of the person.

12Reproducibility

Reproduction requires retaining evidence, judgement protocol and configuration together. Dataset documentation, model reporting and auditing frameworks emphasize provenance and intended use.^{23, 24, 25} The relevant chain is:

1. Selection and source capture. Retain prescreen decisions, identifiers, source revisions and extraction text. Keep inclusion and availability decisions separate from scores.
2. Ledger construction. Preserve original and completed deeds, ideas, shared-work mappings, duplicates and audit amendments. A changed ledger is a changed evaluation even if the model name stays the same.
3. Judgement records. Retain prompts, option orders, model identifiers, probability vectors and pair keys. Run identifiers record configurations, not independently verified provider capabilities.
4. Analysis settings. Preserve the reference, scope factor, model-selection override, thresholds, priors, bootstrap count and seeds. Replaying stored judgements differs from making new requests to a model endpoint.
5. Export and presentation. Keep board, contribution chunks and diagnostics together. A later analysis file must not silently replace the export’s counts, intervals or ranks.

The frontend preserves published scores and displays 100 people per page. The net bar uses a fixed scale across pages and filters with the signed transform below, compressing wide ranges without changing score order:

Missing signed values stay missing. Contribution colours are categorical, not additional weights. Sex labels are grouped for navigation while source labels remain in the data. Tier shortcuts partition result positions: S+ (1–100), S (101–500), A (501–1,000), B (1,001–2,500), C (2,501–5,000) and D (5,001 onward). They follow the current sorting and filtered result set, and are navigation bands rather than additional model grades. These choices affect exploration, not fitted scores.

13References

These publications support the statistical methods, conceptual distinctions and limitations discussed above. They do not endorse the leaderboard’s coefficients or individual historical scores.

1. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist 50, 741–749.
2. OECD, European Union & EC Joint Research Centre (2008). Handbook on Constructing Composite Indicators: Methodology and User Guide. OECD Publishing.
3. Vrandečić, D. & Krötzsch, M. (2014). Wikidata: A free collaborative knowledgebase. Communications of the ACM 57, 78–85.
4. Yu, A. Z., Ronen, S., Hu, K., Lu, T. & Hidalgo, C. A. (2016). Pantheon 1.0, a manually verified dataset of globally famous biographies. Scientific Data 3, 150075.
5. Laouenan, M. et al. (2022). A cross-verified database of notable people, 3500BC–2018AD. Scientific Data 9, 290.
6. Wagner, C., Garcia, D., Jadidi, M. & Strohmaier, M. (2015). It’s a Man’s Wikipedia? Assessing gender inequality in an online encyclopedia. Proceedings of ICWSM 9.
7. Hernán, M. A. & Robins, J. M. (2020). Causal Inference: What If. Chapman & Hall/CRC.
8. Tetlock, P. E. & Belkin, A. (eds.) (1996). Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives. Princeton University Press.
9. Zheng, L. et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. Advances in Neural Information Processing Systems 36.
10. Wang, P. et al. (2024). Large Language Models are not Fair Evaluators. Proceedings of ACL.
11. Panickssery, A., Bowman, S. R. & Feng, S. (2024). LLM Evaluators Recognize and Favor Their Own Generations. Advances in Neural Information Processing Systems 37.
12. Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review 34, 273–286.
13. McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society, Series B 42, 109–127.
14. Stone, M. (1974). Cross-validatory choice and assessment of statistical predictions. Journal of the Royal Statistical Society, Series B 36, 111–147.
15. Rossiter, M. W. (1993). The Matthew Matilda Effect in Science. Social Studies of Science 23, 325–341.
16. Shapley, L. S. (1953). A value for n-person games. Contributions to the Theory of Games, Volume II. Princeton University Press.
17. Bradley, R. A. & Terry, M. E. (1952). Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39, 324–345.
18. Efron, B. (1979). Bootstrap methods: Another look at the jackknife. The Annals of Statistics 7, 1–26.
19. Spearman, C. (1910). Correlation calculated from faulty data. British Journal of Psychology 3, 271–295.
20. Candia, C., Jara-Figueroa, C., Rodriguez-Sickert, C., Barabási, A.-L. & Hidalgo, C. A. (2019). The universal decay of collective memory and attention. Nature Human Behaviour 3, 82–91.
21. Radicchi, F., Fortunato, S. & Castellano, C. (2008). Universality of citation distributions: Toward an objective measure of scientific impact. Proceedings of the National Academy of Sciences 105, 17268–17272.
22. Ahmadpoor, M. & Jones, B. F. (2017). The dual frontier: Patented inventions and prior scientific advance. Science 357, 583–587.
23. Gebru, T. et al. (2021). Datasheets for datasets. Communications of the ACM 64, 86–92.
24. Mitchell, M. et al. (2019). Model cards for model reporting. Proceedings of FAT*, 220–229.
25. Raji, I. D. et al. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. Proceedings of FAT*, 33–44.