In 2012, scientists at the biotech company Amgen tried to reproduce the findings of 53 landmark cancer biology papers. They succeeded with 6. That is an 11% reproducibility rate for the foundational studies that drug development pipelines were built on. Bayer scientists reported similar numbers around the same time. The pattern has shown up across multiple fields and replication projects in the years since, and the cost is not small. A 2015 analysis in PLOS Biology estimated that irreproducible preclinical research costs the United States roughly $28 billion per year.
What is striking about that analysis is what causes the waste. The largest single contributor is not bad statistics or sloppy experimental design. It is the reagents themselves. This article walks through why that finding matters, what reagent traceability actually means, and what to look for when sourcing research materials.
What The Reproducibility Crisis Actually Looks Like In Numbers
The headline statistics are stark:
- Amgen 2012 (Begley & Ellis, Nature): 6 of 53 landmark cancer studies reproduced. 11% success rate.
- Bayer 2011 (Prinz et al, Nature Reviews Drug Discovery): fewer than 25% of published findings reproduced internally. ~25% success rate.
- Reproducibility Project: Cancer Biology (eLife, 2021): of 50 replications conducted on 23 high-impact cancer papers, 8 produced results consistent with the originals. 16% success rate, with median replication effect sizes 85% smaller than original published effects.
- Baker 2016 (Nature survey, 1,576 researchers): more than 70% had tried and failed to reproduce another scientist’s experiment; more than 50% had failed to reproduce their own.
These numbers are not anomalies from one bad field. They span oncology, psychology, cell biology, pharmacology, and the survey responses came from researchers across all major biomedical disciplines.
The 36% Number That Should Change How Labs Source Reagents
Freedman and colleagues at the Global Biological Standards Institute, working with two Boston University economists, broke down the root causes of irreproducibility into four categories and asked which contributed most to the wasted research spend. Their findings:
- Biological reagents and reference materials: 36%
- Study design: 28%
- Data analysis and reporting: 25%
- Laboratory protocols: 11%
Reagent problems are the single largest driver of irreproducible preclinical research. Bigger than bad experimental design. Bigger than statistical errors. Bigger than protocol variability. The materials used in the experiment cause more failures than every other category individually.
This finding gets less attention than the headline reproducibility numbers because it points to a less satisfying culprit. It is easier to argue that other scientists need better stats training than to acknowledge that contaminated cell lines, misidentified antibodies, and inconsistent peptide batches are quietly producing data that nobody can replicate.
Why Reagents Cause So Much Trouble
Reagents are the active ingredients of biomedical research. They include antibodies, peptides, cell lines, chemical compounds, growth factors, enzymes, buffers, and cell culture media. Each one introduces variables that can change experimental outcomes:
- Cell line misidentification: Studies have repeatedly shown that 20% or more of cell lines in use are misidentified or contaminated with other cell lines. A “HeLa” experiment using a different cervical cancer line produces different results.
- Antibody non-specificity: Antibodies bind off-target proteins more often than published methods sections acknowledge. The Antibody Registry exists because the same catalog number can refer to different lots with different binding profiles.
- Peptide purity and identity: A peptide labeled at 99% HPLC purity may still contain truncated sequences, deletion variants, or oxidized residues that alter biological activity. Different lots from the same vendor can vary in residual solvents, salt counterions, and endotoxin levels.
- Reagent stability: Many reagents degrade over time in ways that are not visible to the user. A peptide stored at -20°C for two years may still look fine but have lost 30% of its activity through oxidation or deamidation.
- Supplier-to-supplier variation: Two vendors selling “BPC-157” or “thymosin beta-4” can produce material that differs in salt form, purity profile, endotoxin burden, and impurity spectrum. Researchers comparing data from labs using different sources are often comparing apples to oranges.
Each of these issues looks small in isolation. Aggregated across thousands of experiments, they explain the 36% figure.
What Reagent Traceability Actually Means
Reagent traceability is the documented chain of information that lets a researcher know exactly what is in the vial in front of them, where it came from, when it was made, and how it was tested. A minimum useful traceability record includes:
- Catalog number and lot/batch number: specific to the exact production run
- Certificate of analysis (COA): purity by HPLC, identity confirmation by mass spectrometry, specific endotoxin level in EU/mg, residual solvent measurements
- Manufacturing date and expiration date
- Storage conditions and recommended handling: including specific temperature ranges, light sensitivity, and reconstitution guidance
- Source materials disclosure: which precursors, reagents, and water sources were used
- Quality control documentation: which tests were performed, by whom, and what method
Without these, a researcher cannot meaningfully replicate an experiment that used the same product six months later. The product has changed, and there is no way to compare.
RRIDs And The Push For Standardized Reporting
The Resource Identification Initiative, launched in 2014, addressed one specific gap: when published papers cite a reagent, readers should be able to identify the exact resource that was used. The initiative created Research Resource Identifiers (RRIDs), unique persistent codes that link a publication to a specific antibody, cell line, plasmid, mouse strain, or software tool in an authoritative database.
The format is consistent: RRID followed by a database prefix (AB for antibodies, CVCL for cell lines from Cellosaurus, Addgene for plasmids, MMRRC for mouse models, SCR for software) and a unique number. The system is machine-readable and works across publishers.
Since 2016, NIH grant guidelines have required RRIDs in publications from NIH-funded research. More than 40 biomedical journals now request or require them. The initiative is voluntary in some venues and mandatory in others, but the direction is clear: granular reagent identification is becoming a baseline expectation for publishable research.
Peptides as a category are not yet covered by the formal RRID system the way antibodies and cell lines are. Researchers working with peptides currently rely on catalog numbers, lot numbers, vendor identity, and certificates of analysis as their traceability layer. The principle is the same even when the specific RRID label does not apply.
Sterility Is Not Quality. Purity Is Not Identity.
Three quality measures are often conflated in research peptide sourcing, and they should not be:
- Sterility means no live microorganisms. It does not address endotoxin, chemical contaminants, or peptide purity.
- Purity measures the chemical composition of the peptide population. A 99% pure preparation contains 1% impurities that may include sequence variants, oxidation products, or truncated peptides.
- Identity confirms that the molecule is what the label says. Mass spectrometry and amino acid sequencing answer this question; HPLC does not.
A peptide can be sterile, 99% pure, and the wrong sequence. A peptide can be the correct sequence and contain biologically significant endotoxin contamination. A peptide can be pure, identified, and stored under conditions that degraded it before it reached the buyer. Each of these failure modes requires a different documentation step to detect.
Suppliers who provide COAs that cover only one of these dimensions are providing partial traceability. Sourcing decisions should reflect what is reported and what is not.
What Funders And Journals Now Expect
The reproducibility crisis has shifted institutional incentives in measurable ways:
- NIH guidelines (effective May 2016): All grant applications must address rigor and transparency in research design, including reagent authentication. Funded research must report RRIDs in publications.
- Nature, Science, Cell, and most biomedical journals: Reagent reporting checklists are now part of submission requirements. Authors must specify catalog numbers, lot numbers, and authentication procedures for key materials.
- FDA and EMA preclinical submission guidance: Drug development applications increasingly require traceable provenance for reagents used in preclinical studies that support clinical trial applications.
- Open science initiatives: Platforms like protocols.io, the Open Science Framework, and the Resource Identification Portal are integrating into standard research workflows for sharing methods and reagent information.
Labs that cannot produce reagent traceability records are increasingly disadvantaged in publication, grant renewal, and downstream regulatory submissions. The cost of compliance has become smaller than the cost of non-compliance.
What This Means For Research Peptide Sourcing
For researchers ordering peptides, traceability translates into specific things to look for on the certificate of analysis and the supplier’s documentation:
- HPLC purity reported with the chromatogram, not just a percentage
- Mass spectrometry identity confirmation with the measured mass
- Specific endotoxin level in EU/mg from a documented LAL assay method
- Lot number and manufacturing date on every vial
- Storage and reconstitution guidance specific to the peptide, not boilerplate language
- Batch-to-batch consistency data where the same peptide is purchased repeatedly
- Documented sourcing of raw materials (amino acids, water, solvents)
Peptide vendors that provide this level of documentation cost slightly more than vendors that do not. The cost difference is small relative to the cost of a failed experiment or unreproducible published data.
Frequently Asked Questions
What is the difference between reproducibility and replicability?
Reproducibility usually refers to obtaining the same results when applying the same methods and materials to the same or comparable data, often by the same lab or with full access to the original setup. Replicability refers to obtaining consistent results when an independent lab performs a new experiment using the same methods. Both depend on detailed reporting of materials and protocols, which is why reagent traceability matters for both.
Why are reagents specifically called out as the biggest cause of irreproducibility?
The 2015 Freedman analysis in PLOS Biology grouped causes into four categories and found that biological reagents and reference materials contributed 36% of the irreproducible spend, more than any other single category. The reasoning is that reagent variation directly produces different experimental outcomes, while many other sources of irreproducibility can be partially compensated for through statistical analysis or replication.
What is an RRID and does it apply to peptides?
A Research Resource Identifier is a unique persistent code that links a publication to a specific reagent, cell line, model organism, or software tool in an authoritative database. RRIDs currently cover antibodies, cell lines, model organisms, plasmids, and software. Peptides are not yet part of the formal RRID system, but the same traceability principles apply: researchers should cite the specific vendor, catalog number, lot number, and certificate of analysis when reporting peptide-based experiments.
Does a high purity number on the COA mean a reagent is high quality?
Not by itself. HPLC purity is one dimension of quality and does not address identity confirmation, endotoxin contamination, residual solvents, storage history, or batch consistency. A complete quality assessment requires HPLC purity, mass spectrometry identity, endotoxin testing, and documented storage and handling. Vendors who report only HPLC purity are providing partial information.
How does NIH guidance on rigor and transparency affect peptide researchers?
NIH requires applicants for federal funding to describe how reagents and materials used in proposed research will be authenticated and validated. Researchers using peptides need to document the source, identity confirmation, and quality testing of materials in their methods sections and publications. Vendors providing traceable documentation make this compliance significantly easier.
What is the simplest way to start improving reagent traceability in a lab?
Three steps cover most of the value. First, record the catalog number, lot number, and vendor of every reagent used in every experiment, in the lab notebook or LIMS at the time of use. Second, request and retain the certificate of analysis for every batch. Third, when publishing, include this information in the methods section even when the journal does not explicitly require it. These three habits address the reagent-traceability gap that drives much of the reported irreproducibility in preclinical research.
Why Traceability Is the Fix
The reproducibility crisis is often framed as a problem with how researchers analyze data or design experiments. The numbers say something different. More than a third of irreproducible preclinical research traces back to the reagents themselves, not the methods used to study them.
For peptide research specifically, where small differences in sequence, purity, salt form, or endotoxin level can change biological outcomes, traceability is the single highest-leverage place to invest in better data. The labs that build documentation habits now will be better positioned when funding agencies, journals, and regulators continue tightening reagent reporting requirements.
Resources
The references below are the foundational peer-reviewed sources and authoritative documents cited above. They provide the underlying data on the reproducibility crisis and the institutional response that has followed.
- Begley CG, Ellis LM (2012). Drug development: Raise standards for preclinical cancer research. Nature 483:531-533. The original Amgen replication study. PMID 22460880.
- Prinz F, Schlange T, Asadullah K (2011). Believe it or not: How much can we rely on published data on potential drug targets? Nature Reviews Drug Discovery 10:712. The Bayer replication study.
- Freedman LP, Cockburn IM, Simcoe TS (2015). The Economics of Reproducibility in Preclinical Research. PLOS Biology 13(6):e1002165. The $28 billion analysis with category breakdown. https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.1002165
- Baker M (2016). 1,500 scientists lift the lid on reproducibility. Nature 533:452-454. The large-scale researcher survey on reproducibility failures.
- Reproducibility Project: Cancer Biology (2021). Investigating the replicability of preclinical cancer biology. eLife 10:e71601. https://elifesciences.org/articles/71601
- Bandrowski A et al. (2016). The Resource Identification Initiative: A cultural shift in publishing. Journal of Comparative Neurology 524(1):8-22. The foundational RRID initiative paper. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4684178/
- NIH Office of Research Infrastructure Programs. Research Resource Identifiers (RRIDs). Official NIH guidance. https://orip.nih.gov/division-comparative-medicine/research-resources-directory/research-resource-identifiers-rrids
Disclaimer: All Koi Peptides products are sold for research use only. Not for human or veterinary use. The content above is for educational purposes and does not constitute medical advice.
