Medicines Security Index methodology

Purpose and interpretation

The Medicines Security Index assesses a jurisdiction’s demonstrated capability to sustain access to a defined basket of quality-assured medicines during routine conditions and disruption. The construct covers governance and operational capabilities; it is not a prediction of individual patient outcomes, a sovereign credit rating or an accreditation scheme. A high capability score cannot establish that medicines are affordable or universally accessible.

The ten domains form a formative framework: different capabilities jointly define medicines security. They are not assumed to be interchangeable manifestations of one latent trait. High internal consistency is therefore not a stand-alone validation criterion. Report the domain profile and patient-access outcomes alongside any composite.

Version 0.2 is a proposed research protocol dated 22 October 2026, developed for Medicines Security Institute. Candidate indicators, thresholds and release criteria are MSI design proposals requiring consultation and pilot evidence. Citing OECD, WHO or JRC material does not imply their endorsement. No independent validation, reviewer appointment or institutional partnership has been obtained.

Unit basket and observation window

Use jurisdiction × fixed medicines basket × calendar year as the primary unit. Name the jurisdiction, delivery setting and public/private coverage. Define each basket item by international nonproprietary name, strength, dosage form, route and age-appropriate formulation. Publish the basket before extraction; never choose it after seeing country performance.

For an initial pilot propose 30 clinically prioritised tracer formulations selected by an independent clinical and patient panel from the then-current WHO Model List of Essential Medicines. Agree treatment-area and paediatric representation during consultation. The exact common basket must be frozen before scoring; no definitive basket has yet been selected. Keep country-specific supplementary items outside cross-country scores.

Use the same completed calendar year for comparisons. Register the extraction cut-off and source vintages. Process and performance evidence must cover that year; standing policies may be up to 36 months old only with current confirmation of applicability. Exercises may span 12 or 24 months as specified in the dictionary. Do not carry older operational values forward as current.

Use all eligible records when auditable census data exist. Otherwise preregister a stratified facility and product sample covering regions, urban/rural settings, delivery levels and relevant public/private provision. Publish frame, selection probabilities, exclusions, nonresponse and design weights. A convenience sample supports only an explicitly bounded pilot profile, not a national league table.

Indicator specification and extraction

The dictionary defines 30 candidate indicators, three per domain. All are proportions with a positive direction. Each record must contain the indicator ID, jurisdiction, basket version, numerator, denominator, observation dates, target population, sample frame, source locator, archive hash where lawful, extraction date, extractor, second assessor, uncertainty bounds and adjudication decision.

Freeze all critical-role lists, plans, risk registers, activity schedules, response deadlines and denominator universes before the observation window. Record whether each denominator is externally anchored or institutionally self-selected. Independent reviewers must check for artificially short plans, lenient deadlines or omitted liabilities. These gaming risks can invalidate an indicator even if its arithmetic is correct.

Two assessors independently extract every pilot observation. Preserve both raw values and rationales before adjudication. A third assessor resolves unresolved differences. Do not average conflicting claims without resolving the underlying denominator and scope. One source may inform several indicators only when it supplies distinct observations; assess conceptual overlap explicitly.

Ten domains and thirty candidate indicators

Each indicator is a percentage with an explicit numerator and denominator. All three indicators must be assessable to produce a domain score. The complete dictionary is available as CSV and within the printable methodology.

Evidence quality and missing observations

Grade A evidence is independently audited or directly reproducible official microdata with matching scope and period. Grade B is traceable primary administrative evidence independently checked by the second assessor. Grade C is unverified self-report, unclear scope or stale evidence; retain for context and do not use in the primary score. Grade D is unavailable or unusable. These grades are proposed evidence rules, not externally certified categories.

Every scored observation needs a valid positive denominator and numerator between zero and denominator. Zero numerator is valid only with affirmative evidence of no successful units; absence of documents is missing, not zero. A zero denominator, including no eligible events, is not automatically 100 percent. Use prespecified standardised exercises where allowed; otherwise mark not assessable.

Publish a descriptive domain mean only if all three indicators are assessable with A or B evidence; publish a composite only when all 30 meet this rule. Never redistribute weights around missing indicators for the primary score. Incomplete profiles show coverage and domain gaps. This strict rule may make country ranking infeasible; the feasibility pilot must determine that honestly.

For mixed evidence quality, display A/B shares separately rather than multiplying scores by confidence. Publish sampling uncertainty using the actual sampling design where microdata permit it. Administrative data without an estimated sampling error still have measurement and reporting error. Do not report scenario ranges as statistical confidence intervals.

Scoring weighting and aggregation

For indicator j, calculate s_j = 100 × n_j / d_j using eligible numerator n_j and denominator d_j. No sample-dependent min-max transformation, winsorisation or imputation is used in the primary protocol. Retain unrounded values through aggregation; display one decimal only at the end.

Each domain score D_k is the arithmetic mean of its three indicator percentages. The composite C is the arithmetic mean of the ten domain scores. Each domain has weight 10 percent; each indicator has effective weight 1/30. Equal weighting is a transparent normative starting point, not empirical proof of equal importance.

An arithmetic composite permits compensation across domains. Therefore always display the minimum domain and flag any domain below a provisional 40/100 floor; this floor is a consultation proposal, not an empirically calibrated safety threshold. Compare geometric aggregation and plausible alternative weights in sensitivity analysis. Do not describe a composite as safe, resilient or investment grade on a numerical cut-off alone.

The existing ten-slider 0–4 workbench is a legacy scenario tool. It cannot be treated as this 30-indicator protocol or used to claim validated country results. The supplied v0.2 calculator is separate and requires raw numerator/denominator evidence. Existing research records are not retrospectively rescored.

Patient access outcomes and external benchmarks

Report affordable essential-medicine availability, stockout-days, unmet treatment need and treatment interruptions as companion outcomes when measured with comparable population coverage. WHO SDG 3.b.3 metadata inform access measurement; do not equate its definition with the new MSI capability score. If C3 stockout-free days enter the composite, exclude C3 and recompute a prespecified diagnostic score when comparing the index with stockout outcomes to avoid circular validation.

WHO regulatory benchmarking and UHC service coverage may supply external context, with exact product scope and year. Do not convert absence from a WHO maturity list into regulatory failure. UHC is a broad service-coverage benchmark and neither validates MSI nor substitutes for medicines-security evidence.

FDA recall reports and other live alerts remain a surveillance layer. Counts reflect reporting and regulatory activity as well as underlying events. They do not directly become country scores, shortage prevalence or negative quality ratings.

Robustness and ranking protocol

Preregister weight sensitivity with 10000 reproducible draws: multiply each domain’s equal weight by an independent uniform factor between 0.5 and 1.5, then renormalise. This is a declared plausibility scenario, not a probability model of the true weights. Also compare equal-domain geometric aggregation and each leave-one-domain-out variant.

For each country use assessor-supported indicator lower and upper values to calculate monotone composite bounds. If defensible bounds are absent, say uncertainty is unquantified. Jointly perturb measurement assumptions and weights in a documented pilot analysis; retain seed, code, draws, inputs and full rank distributions. Sensitivity results cannot establish validity without real evidence.

For a published country table require an identical basket, period, evidence rules and measurement definitions; complete assessed inputs; independent release review; and a named dataset version. Use competition ranks on unrounded scores, show one-decimal scores, and display exact ties. Label rounded ties and overlapping uncertainty ranges prominently; do not imply statistically distinct performance from a rank difference.

Preregister a provisional robustness trigger: withhold a definitive ordinal league table if more than 20 percent of countries move over 10 percentile points between the 5th and 95th sensitivity rank percentiles, or if any major construct-validity or comparability objection remains unresolved. Show domain profiles and sensitivity ranges instead. This decision rule requires panel agreement and must not be tuned after results are seen.

Independent validation study

Content validity: recruit a proposed panel of 8–12 independent reviewers spanning pharmaceutical systems, regulation, supply chains, composite-index statistics, health economics and patient representation. Include several income settings and regions; require conflict-of-interest declarations. Two anonymous rating rounds assess relevance, clarity, feasibility and duplication on a four-point scale. Proposed retention criterion: at least 80 percent rate each item 3 or 4; revise and rerate other items. This is a prespecified consultation rule, not universal certification.

Feasibility and reliability: pilot six jurisdictions selected for diversity of income, geography, supply model and data infrastructure, without claiming representativeness. Double-extract all 180 country-indicator cells. Report missingness, extraction time, evidence grades, absolute percentage-point disagreements and a two-way random-effects absolute-agreement single-measure ICC with a 95 percent interval where estimable. Proposed targets: at least 90 percent within five points and lower ICC confidence bound at least 0.75. Six jurisdictions give imprecise reliability estimates; expand the sample using precision simulations before making a general reliability claim.

Construct validity: preregister directional hypotheses linking relevant domains to independently measured access, disruption duration and recovery performance. Examine income, infrastructure, reporting intensity and selection effects as confounders. Inspect indicator correlations for duplication; do not choose weights simply to maximise association with GDP. A feasibility pilot cannot support a 30-variable factor model; determine a larger sample using model-specific power and precision analysis.

Prospective validation: lock baseline inputs and follow prespecified subsequent disruption and patient-access outcomes. Record event definitions and denominators before follow-up; use holdout jurisdictions or time periods and report uncertainty. Baseline capability associations are not causal effects. No predictive claim is warranted until this study is completed.

Independent replication: a reviewer unconnected to scoring reruns extraction checks, calculations and robustness scripts from the frozen evidence archive. Publish the review report, author responses, unresolved issues, reviewer consent and conflicts. Institutional endorsement may only be stated with explicit written permission.

Validation status and reproducible technical checks

16 technical checks passed, including 1,000 seeded monotonicity trials and 10,000 weight scenarios. All inputs in these checks are synthetic fixtures. No country or reviewer results are implied.

Governance release and correction

MSI owns the research protocol. Appoint an independent methods chair and a separate country-evidence review lead; proposed roles are not appointments. David Yusuf, Executive Director, may sponsor the programme but must not be its sole validator. Funders and commercial clients have no right to select scores or suppress findings; disclose funding and methodological influence.

Before external research involving people, assess ethics review, consent, data protection and secure retention requirements. Publish only necessary aggregate evidence. Preserve confidential commercial source access under controlled review rather than exposing supplier contracts or personal data.

Release gate one is a documented consultation decision and frozen v1.0 protocol. Gate two is an auditable pilot dataset and inter-rater analysis. Gate three is robustness, construct analysis and independent replication. Gate four is signed external review with a response-to-reviewers log and a scoped release decision. Prospective validation remains separately reported even if an initial descriptive release is authorised.

Give jurisdictions 20 working days to challenge factual extraction errors before release, with the same procedure for all. They cannot veto publication. Log decisions and material corrections publicly, version affected datasets and recalculate all linked scores. Review the protocol annually; major construct or basket changes create a new series and require bridging analysis rather than silently rewriting trends.

External review route and present status

Current status: protocol developed; technical checks of the supplied calculator completed; no real-country pilot populated under v0.2; no external reviewer appointed; no independent scientific validation obtained. The existing WHO/World Bank comparison remains an external benchmark.

Approach JRC COIN for guidance on the proposed pilot and later methodological review. Its published Open Days guidance recommends stakeholder consultation, an agreed framework and populated data before a full session. Acceptance, timing and the scope of any review are theirs to decide. A COIN discussion is not accreditation or WHO endorsement.

A review request and reviewer assessment form are included in the downloadable package. They are prepared for approval and have not been sent. A realistic programme depends on recruiting assessors and securing comparable source data; no completion date or successful validation can be promised in advance.