Appendix A: Glossary of Terms
Adverse Event Reporting (AER)
Reports filed by patients and medical professionals to national and international regulatory bodies regarding adverse reactions to medical drugs and devices. See, for example, the US FDA’s system here.
Biological markers refer to a broad subcategory of medical signs — that is, objective indications of the medical state of a patient, observed from outside the patient — and which can be accurately measured and the findings reproduced. For example, a blood test result. Medical signs stand in contrast to medical symptoms, which are limited to those indications of health or illness perceived by patients themselves.
When study participants do not know if they are receiving an intervention. If a study is double-blinded, medical providers administering the treatment do not know either. Many studies have shown that if you know what treatment you are receiving, you’ll eventually experience greater placebo effects than a subject who doesn’t know what treatment s/he is having.
Confounding Factors and Bias
A confounding factor is an element that might influence the results of a study or analysis. For instance, comparing the epidemiological situation of two countries, without adjusting for population age; there are countries with a much older population than others, and this can greatly influence an analysis. In this example, population age is the confounding factor. When analyzing a dataset or a study result, researchers try to make adjustments to exclude any confounding factor, thus achieving a more reliable answer to the research question. Bias is defined as any “non-random error” that interferes with an accurate assessment of a study outcome. More: Identifying and Avoiding Bias in Research.
Correlation and Causation
Correlation: when two phenomena are linked by a relationship. For instance, it’s raining and at the same time, the lights go off in your apartment. These two events happen at the same time, so there is a sort of relationship, but you can’t say if the lights went off because of the rain. Causation, conversely, indicates that two phenomena are linked by a cause and effect relationship. In the example, you call up a specialist to fix your electricity problem and he explains to you that clearly a leak in your building meant the rain shut the lights off. In scientific methodology, the difference between correlation and causation is key. Many phenomena can seem linked by a cause and effect relationship, but unless proven we must assume that they are instead connected because of correlation. Attributing a causation value to a phenomenon is a very common and problematic bias.
One definition in relatively plain language: “Critical appraisal is the process of carefully and systematically assessing the outcome of scientific research (evidence) to judge its trustworthiness, value, and relevance in a particular context.” Source: What is Critical Appraisal?— Center for Evidence Based Management.
One definition of disease mongering is the “selling of sickness” in order to promote drug sales. It is a strategy used by pharmaceutical companies to enlarge the potential market for their products, either by inventing new illnesses, or by framing a normal phenomenon as such. Common symptoms, though not pathological, can become the indication for a new drug, or a new indication for an already marketed product.
Endpoints: Primary, Secondary, Surrogate, Combined
A clinical trial aims to answer a research question. The “endpoints” are indicators established to measure the trial’s result. For instance, a trial for a new oncological drug could set as its primary endpoint prolonging the patient’s life, and as secondary endpoint the time without symptoms. Surrogate endpoints are inherently a red flag; in this example that could be the size of the tumor. Combined endpoints use two or more outcomes as a single measure; beware of this as it might bias the study results.
European Medicines Agency (EMA)
The EMA is an agency of the European Union in charge of the evaluation and supervision of medicinal products. Along with the US Food and Drug Administration, the agency’s decisions on drugs have great influence on the worldwide pharmaceutical market.
Evidence-Based Health Care
The term indicates an approach used by health care providers to decide on appropriate intervention to treat a condition, basing the decision-making process on the “best evidence available.” More: Evidence Based Health Care.
Experimental (also called Interventional) Study
A study where all the conditions are pre-determined and controlled by the person carrying out the experiment. More: Experimental Study definition.
Exposure in Absolute Values (Absolute Risk)
Conversely to relative risk, the absolute risk gives an indication of the likelihood over time of an event happening. See this website for accurate but plain language definitions, and many examples: Relative Risk and Absolute Risk: Definition and Examples.
Field Safety Notice
The notice a manufacturer releases to acknowledge that a serious safety concern regarding the use of its product has emerged.
Food and Drug Administration (FDA)
The FDA is a US regulatory agency responsible for supervision of pharmaceutical drugs, vaccines, and medical devices. Along with the European Medicines Agency, the FDA’s decisions on drugs have great influence on the worldwide pharmaceutical market.
An indication describes the approved use of a pharmaceutical product (such as a drug, test, or vaccine). It means that regulatory authorities have reviewed the evidence submitted by the manufacturer for the treatment or test of a condition or a disease, and allowed the company to market the product for this specific use. Widening the indication is a common practice to expand a product’s market, and it often happens step by step, with the industry submitting new data to the regulators.
The WHO defines a health intervention as: “An act performed for, with or on behalf of a person or population whose purpose is to assess, improve, maintain, promote, or modify health, functioning, or health conditions.”
The full definition of a “medical device” by the WHO: “Any instrument, apparatus, implement, machine, appliance, implant, reagent for in vitro use, software, material or other similar or related article, intended by the manufacturer to be used, alone or in combination, for human beings, for one or more of the specific medical purpose(s) of diagnosis, prevention, monitoring, treatment, or alleviation of disease; diagnosis, monitoring, treatment, alleviation of or compensation for an injury; investigation, replacement, modification, or support of the anatomy or of a physiological process; supporting or sustaining life; control of conception; disinfection of medical devices; providing information by means of in vitro examination of specimens derived from the human body; and does not achieve its primary intended action by pharmacological, immunological or metabolic means, in or on the human body, but which may be assisted in its intended function by such means.” More: Medical Device — Full Definition.
A meta-analysis is a statistical analysis that combines the results of several, comparable studies.
Multicentric Double-Blind Randomized Controlled Trials
Multicentric indicates that a trial is performed in several, different locations (hospitals, for instance). For double-blind and randomized controlled trial (RCT) see nonrandomized, nonblinded, below.
Randomization is a key concept in scientific studies that aim at comparing two groups of subjects that are receiving different interventions (such as a tested drug and another drug for comparison). By randomly assigning patients to one of the two groups, bias and confounding factors are minimized so that the two groups are as much alike as possible at the onset of the study. Nonetheless, by chance alone, baseline imbalances, such as one group being older than the comparator group, do occur and can skew the outcome.
Observing phenomena as they take — or took — place and trying to draw conclusions on their underlying causes. Although observational studies can provide valuable hints, they can’t provide solid and final evidence, as they are especially prone to bias and confounding factors. The best course of action is normally to use observational studies to formulate hypotheses, to be confirmed with randomized controlled trials that allow firmer conclusions on the intervention’s effect.
Outcome switching is a questionable practice in which researchers modify the aim of the study after it has started. Researchers are supposed to identify the expected outcome of the medical intervention being tested in a study protocol before the trial is launched. If that outcome is dropped part way through a study and another outcome is substituted, it could be because the researchers looked at the data and did not get the positive outcome they had hoped for.
Percentage Exposure (Relative Risk)
The US National Institutes of Health defines relative risk as “a ratio of the probability of an event occurring in the exposed group versus the probability of the event occurring in the non-exposed group. For example, the relative risk of developing lung cancer (event) in smokers (exposed group) versus non-smokers (non-exposed group) would be the probability of developing lung cancer for smokers divided by the probability of developing lung cancer for nonsmokers. The relative risk does not provide any information about the absolute risk of the event occurring, but rather the higher or lower likelihood of the event in the exposure versus the non-exposure group.” Source: Relative Risk.
According to the WHO, “the science and activities relating to the detection, assessment, understanding, and prevention of adverse effects or any other drug-related problem.”
PICO is a tool to break down a research question. The acronym stands for Patient/Population; Intervention; Comparison; Outcomes. Using it allows for a well-structured search in the biomedical literature and it can also be extremely helpful to journalists who need to dig deeper on any medicine-related interventions. More: PICO — Evidence Based Medicine.
The monitoring of a drug or vaccine safety after it has been put on the market.
Pre-Identified Outcome Measures
See outcome switching.
Premarket Approval Application
In order for a drug or a vaccine to be approved for the market, the manufacturer must submit a Premarket Approval Application, whose aim is to give the regulators all relevant information about the product’s expected effectiveness and safety, as measured in clinical trials.
Randomized Controlled Trials (RCTs)
A randomized controlled trial can be one of the most rigorous ways of determining whether a cause-effect relationship exists between treatment and outcome, and for assessing the cost effectiveness of a treatment. Comparable groups of subjects are randomly assigned to receive a new medical intervention, or an intervention that is already standard, a placebo, or no intervention at all. If they are single-blinded, the receiver doesn’t know which group they are in, and if they are double-blinded, those administering the intervention do not know either. Such design allows for comparisons between interventions, and it is considered the gold standard among medical studies. More here: Understanding Controlled Trials: Why Are Randomised Controlled Trials Important?
These are performed looking back at previously collected data. Retrospective studies are observational studies prone to bias, because data was not collected to answer that specific research question.
Surrogate Endpoint or Surrogate Outcome
Surrogate endpoints are indicators (often bio-markers) chosen by researchers because they are considered important contributors in the mechanism of a disease. For example, blood pressure may be used as a surrogate endpoint in a trial on cardiovascular drugs, because it is a known risk factor for heart attacks and strokes. The hypothesis is that if the drug shows an effect on the surrogate endpoint, high blood pressure, it will also have an effect on the clinical outcome (heart attacks and strokes). Unfortunately, in many cases a drug’s effect on a surrogate outcome won’t bring the expected benefit to patients, and may even harm them, because other aspects of the mechanism of the disease have not been well understood yet. That’s why any results obtained in a study that was designed with a surrogate endpoint must be taken with much caution.
World Health Organization (WHO)
The WHO operates under the United Nations to coordinate international responses to public health. Based in Geneva, Switzerland, the agency has six regional offices and 150 field offices worldwide.
Appendix B: Regulatory Agencies
- Argentina: National Administration of Drugs, Food, and Medical Devices (ANMAT)
- Belize: Ministry of Health
- Bolivia: Ministry of Health and Social Welfare
- Brazil: Brazilian Health Surveillance Agency (ANVISA)
- Chile: Public Health Institute of Chile (ISPCH) (ISPCH)
- Colombia: Ministry of Health and Social Protection / INVIMA Instituto Nacional de Vigilancia de Medicamentos y Alimentos
- Costa Rica: Ministry of Health
- Cuba: CECMED
- Ecuador: National Agency for Regulation, Control, and Sanitary Surveillance (ARCSA)
- El Salvador: Ministry of Health
- Guatemala: Ministry of Health
- Guyana: Ministry of Health
- Jamaica: Ministry of Health
- Mexico: Federal Commission for the Protection Against Sanitary Risk (COFEPRIS)
- Nicaragua: Ministry of Health
- Panama: Ministry of Health
- Paraguay: Ministry of Public Health
- Peru: General Directorate of Medicines, Supplies, and Drugs (DIGEMID)
- Trinidad and Tobago: Ministry of Health / Bureau of Standards
- Uruguay: Ministry of Public Health
- Venezuela: National Institute of Hygiene “Rafael Rangel” / Autonomous Health Service Comptroller (SACS)
EU and EU Member States
There are several options for applying for a marketing authorization within the EU.
- European Union: European Medicines Agency (EMA)
- Austria: Bundesamt für Sicherheit im Gesundheitswesen (BASG / AGES)
- Belgium: Federal Agency for Medicines and Health Products
- Bulgaria: Bulgarian Drug Agency (BDA)
- Croatia: Agency for Medicinal Products and Medical Devices of Croatia
- Cyprus: Ministry of Health
- Czech Republic: State Institute for Drug Control (SUKL)
- Denmark: The Danish Medicines Agency
- Estonia: Agency of Medicines
- Finland: Finnish Medicines Agency (FIMEA)
- France: Agence nationale de sécurité du médicament et des produits de santé (ANSM)
- Germany: Bundesinstitut für Arzneimittel und Medizinprodukte (BfArM)
- Greece: National Organization for Medicines
- Hungary: National Institute for Pharmacy and Nutrition (OGYEI)
- Ireland: Health Products Regulatory Authority (HPRA)
- Italy: Agenzia Italiana del Farmaco (AIFA)
- Latvia: State Agency of Medicines
- Lithuania: State Medicines Control Agency (SMCA)
- Malta: Medicines Authority
- Netherlands: Medicines Evaluation Board (MEB)
- Poland: Office for Registration of Medicinal Products, Medical Devices, and Biocidal Products (UPRL)
- Portugal: National Authority of Medicines and Health Products (INFARMED)
- Romania: The National Agency for Medicines and Medical Devices (NAMMD)
- Slovakia: State Institute for Drug Control
- Slovenia: Agency for Medicinal Products and Medical
- Spain: Ministry of Health, Social Services, and Equality
- Sweden: Medical Products Agency (MPA)
Non-EU Member States
- Albania: National Agency on Drugs and Medical Devices
- Andorra: Ministry of Health and Welfare
- Armenia: Scientific Center of Drug and Medical Technologies Expertise (SCDMTE)
- Azerbaijan: Center of Drug Analytical Expertise of the Ministry of Health of Azerbaijan
- Belarus: Center for Examinations and Tests in Health
- Bosnia and Herzegovina: Agency for Medicinal Products and Medical Devices of Bosnia and Herzegovina
- Georgia: Ministry of Labor, Health, and Social Protection
- Iceland: Icelandic Medicines Agency
- Kosovo: Kosovo Medicines Agency
- Macedonia: Agency for Medicinal Products and Medical Devices
- Montenegro: Agency for Medicines and Medical Devices
- Norway: Norwegian Medicines Agency
- Russia: Federal Service on Surveillance in Healthcare and Social Development
- San Marino: Ministry of Health and Social Security
- Serbia: Medicines and Medical Devices Agency of Serbia
- Switzerland: Swissmedic
- UK: Medicines and Healthcare Products Regulatory Agency (MHRA)
- Ukraine: State Inspectorate for Quality Control of Medicines
Asia and the Pacific
- Australia: Therapeutic Goods Administration (TGA)
- Bangladesh: Directorate General of Drug Administration (DGDA)
- Bhutan: Drug Regulatory Authority
- China: China Food and Drug Administration (SFDA)
- Hong Kong: Department of Health / Central Drugs Standard Control Organization
- India: Central Drugs Standard Control Organization
- Indonesia: Ministry of Health Medical Device Regulatory & CRO
- Japan: Pharmaceuticals and Medical Devices Agency (PMDA)/Ministry of Health, Labour and Welfare (MHLW)
- Kyrgyzstan: Department of Medicines Supply and Medical Equipment of the Ministry of Health of the Republic of Kyrgyzstan
- Laos: Food and Drug Department
- Maldives: Ministry of Health
- Malaysia: National Pharmaceutical Regulatory Agency (NPRA)
- Nepal: Department of Drug Administration
- New Zealand: New Zealand Medicines and Medical Devices Safety Authority (MEDSAFE)
- Pakistan: Drug Regulatory Authority of Pakistan
- Philippines: Department of Health
- Singapore: Health Sciences Authority
- South Korea: South Korea Ministry of Food and Drug Safety
- Sri Lanka: Cosmetics, Devices, & Drugs Regulatory Authority
- Taiwan: Taiwan Food and Drug Administration
- Thailand: Thailand Food and Drug Administration
- Vietnam: Drug Administration of Vietnam / Medical Devices Department, Ministry of Health
- Bahrain: National Health Regulatory Authority
- Iran: Ministry of Health
- Iraq: Ministry of Health
- Israel: Ministry of Health
- Jordan: Jordan Food and Drug Administration (JFDA)
- Kuwait: Kuwait Institute for Medical Specialization
- Lebanon: Ministry of Public Health
- Morocco: Ministry of Health
- Oman: Ministry of Health Sultanate of Oman
- Qatar: MOH Pharmacy and Drug Control Department
- Saudi Arabia: Saudi Food and Drug Authority (SFDA)
- Syria: Directorate of Drug Control
- Turkey: Turkish Medicines and Medical Devices Agency
- United Arab Emirates: Ministry of Health
- Yemen: Supreme Commission for Drugs and Medical Appliances
- Algeria: Ministry of Health, Population, and Hospitals
- Angola: Direcção Nacional de Medicamentos e Equipamentos / Ministry of Health
- Benin: Direction de la Pharmacie et des explorations diagnostics
- Botswana: Drug Regulatory Services / Ministry of Health
- Burkina Faso: Direction Générale de la Pharmacie, du Médicament et des Laboratoires (DPMED)
- Burundi: Direction de la pharmacie, du médicament et des laboratoires / Ministry of Health
- Cabo Verde: Entidade Reguladora Independente da Saúde (ERIS)
- Cameroon: Direction de la Pharmacie, du Médicament et des Laboratoire / Ministry of Health
- Chad: Direction de la pharmacie, du médicament et de la pharmacopée / Ministry of Health
- Congo: Direction des pharmacies, du médicament et des laboratoires / Ministry of Health
- Côte d’Ivoire: Direction de la pharmacie, du médicament et des laboratoires (DPML) / Ministry of Health
- DR Congo: Direction de la pharmacie et du médicament (DPM) / Ministry of Health
- Djibouti: Direction du médicament, de la pharmacie et des laboratoires (DMPL) / Ministry of Health
- Egypt: Egyptian Drug Authority (EDA)
- Equatorial Guinea: Direction générale de pharmacie et médecine traditionnelle / Ministry of Health
- Eritrea: National Medicine and Food Administration / Ministry of Health
- Ethiopia: Ethiopian Food and Drug Authority (EFDA)
- Gabon: Direction du médicament et de la pharmacie / Ministry of Health
- Gambia: Medicine Control Agency (MCA) / Ministry of Health
- Ghana: Food and Drugs Authority
- Republic of Guinea: Direction nationale de la pharmacie et du laboratoire / Ministry of Health
- Guinea Bissau: Ministry of Health
- Kenya: Pharmacy and Poisons Board (PBD)
- Lesotho: Ministry of Health
- Liberia: Liberia Medicines and Health Products Regulatory Authority / Ministry of Health
- Libya: Pharmacy Management Department / Ministry of Health
- Madagascar: Direction de la pharmacie, des laboratoires et de la médecine traditionnelle / Ministry of Health
- Malawi: Pharmacy, Medicines, and Poisons Board / Ministry of Health
- Mali: Direction de la pharmacie et des médicaments / Ministry of Health
- Mauritania: Direction des médicaments et de la pharmacie / Ministry of Health
- Mauritius: Pharmacy Board / Ministry of Health
- Morocco: Direction du médicament et de la pharmacie / Ministry of Health
- Mozambique: Direcção Nacional de Farmácia / Ministry of Health
- Namibia: Namibia Medicines Regulatory Council / Ministry of Health
- Niger: Direction de la pharmacie, des laboratoires et de la pharmacopée traditionnelle / Ministry of Health
- Nigeria: National Agency for Food and Drug Administration and Control (NAFDAC)
- Rwanda: Department of Pharmaceutical Services / Ministry of Health
- São Tomé and Príncipe: Direction de la pharmacie et du médicament / Ministry of Health
- Senegal: Direction de la pharmacie et du médicament / Ministry of Health
- Seychelles: Medicines Regulation Unit / Ministry of Health
- Sierra Leone: Pharmacy Board of Sierra Leone / Ministry of Health
- Somalia: Pharmaceutical and Medical Department / Ministry of Health
- South Africa: South African Health Products Regulatory Authority (SAHPRA)
- South Sudan: Food and Drugs Control Authority, South Sudan / Ministry of Health
- Sudan: National Medicine and Poisons Board
- Swaziland: Pharmaceutical Services Department / Ministry of Health
- Togo: Direction de la pharmacie, du médicament et des laboratoires / Ministry of Health
- Tanzania: Tanzania Food and Drug Authority (TFDA) / Ministry of Health
- Tunisia: Direction de la pharmacie et du médicament (DPM)
- Uganda: National Drug Authority
- Zambia: Zambia Medicines Regulatory Authority (ZAMRA)
- Zimbabwe: Medicines Control Authority (MCAZ)
Appendix C: Books
Marcia Angell, M.D., “The Truth About Drug Companies: How They Deceive Us and What to Do About It.” (2004)
Michael Blastland, Andrew Dilnot, “The Tiger That Isn’t: Seeing Through A World Of Numbers.” (2007)
Nellie Bly, “Ten Days in a Mad-House.” (1887)
J. Douglas Bremner, M.D., “Before You Take That Pill.” (2008)
Daniel Carpenter, “Reputation and Power: Organizational Image and Pharmaceutical Regulation at the FDA.” (2010)
John Carreyrou, “Bad Blood: Secrets and Lies in a Silicon Valley Startup.” (2018)
Archie L. Cochrane, “Effectiveness And Efficiency: Random Reflections on Health Services.” (1972)
Katherine Eban, “Bottle of Lies: The Inside Story of the Generic Drug Boom.” (2019)
Allen Frances, “Saving Normal: An Insider’s Revolt against Out-of-Control Psychiatric Diagnosis, DSM-5, Big Pharma, and the Medicalization of Ordinary Life.” (2014)
Gerd Gigerenzer, “Calculated risks: How to Know When Numbers Deceive You.” (2003)
Gerd Gigerenzer, “Reckoning With Risk : Learning to Live With Uncertainty.” (2002)
Ben Goldacre, “Bad Pharma: How Medicine is Broken, and How We Can Fix It.” (2012)
Merrill Goozner, “The $800 Million Pill: The Truth Behind the Cost of New Drugs.” (2005)
Peter C. Gøtzsche, “Mammography Screening: Truth, Lies and Controversy.” (2012)
Peter C. Gøtzsche, “Deadly Medicines and Organised Crime: How Big Pharma Has Corrupted Healthcare.” (2013)
Katharine Greider, “The Big Fix: How the Pharmaceutical Industry Rips Off American Consumers.” (2003)
Iona Heath, “Matters of Life and Death: Key Writings.” (2007)
Darrell Huff, “How To Lie with Statistics.” (1954)
Robert M. Kaplan, “Disease, Diagnoses, and Dollars: Facing the Ever-Expanding Market for Medical Care.” (2009)
Jerome P. Kassirer, M.D., “On The Take: How Medicine’s Complicity with Big Business Can Endanger Your Health.”
Jacky Law, “Big Pharma: How The World’s Biggest Drug Companies Control Illness.” (2006)
James Le Fanu, “The Rise and Fall of Modern Medicine.” (2000)
John Le Carré, “The Constant Gardener.” (2005)
Jeanne Lenzer, “The Danger Within Us: America’s Untested, Unregulated Medical Device Industry and One Man’s Battle to Survive It.” (2017)
Barry Meier, “Pain Killer: An Empire of Deceit and the Origin of America’s Opioid Epidemic.” (2018)
Ray Moynihan, Alan Cassels, “Selling Sickness: How the World’s Biggest Pharmaceutical Companies Are Turning Us All into Patients.” (2006)
Siddhartha Mukherjee, “The Laws of Medicine: Field Notes from an Uncertain Science.” (2015)
Vinayak K. Prasad, Adam S. Cifu, “Ending Medical Reversal: Improving Outcomes, Saving Lives.” (2015)
Vinayak K. Prasad, “Malignant: How Bad Policy and Bad Evidence Harm People with Cancer.” (2020)
Marc A. Rodwin, “Conflicts of Interest and the Future of Medicine.” (2011)
Peter Rost, “The Whistleblower: Confessions of a Healthcare Hitman.” (2006)
Sergio Sismondo, “Ghost-Managed Medicine: Big Pharma’s Invisible Hands.” (2018)
H. Gilbert Welch, “Should I Be Tested for Cancer?: Maybe Not and Here’s Why.” (2006)
H. Gilbert Welch, Lisa Schwartz, Steve Woloshin, “Overdiagnosed: Making People Sick in the Pursuit of Health.” (2012)
Steven Woloshin, Lisa M. Schwartz, H. Gilbert Welch, “Know Your Chances: Understanding Health Statistics.” (2008)
About the Authors
Serena Tinari and Catherine Riva are medical investigative journalists and co-founders of Re-Check.ch, a Swiss nonprofit organization dedicated to investigating and mapping health affairs. Re-Check.ch produces multi-language publications, supports newsrooms on their health investigations, and gives workshops on how to perform in-depth research on medicine and public health.
Ms. Tinari has been working across print and electronic media, radio, and television since 1994. She authored dozens of investigative stories for the Swiss public broadcaster. Tinari specializes in drug safety, Evidence-Based Medicine and conflicts of interest. A trainer, speaker, and moderator, she has reported extensively on Swine Flu and Tamiflu.
Ms. Riva specializes in the design and methodology of clinical trial, public health, and Evidence-Based Medicine. A trainer and a speaker, Riva has reported on the Mediator case, breast cancer screening programs, the controversies related to HPV vaccination, and on conflicts of interest in healthcare and medical research. She has also published in biomedical journals.
The Global Investigative Journalism Network serves as the international association for the world’s investigative reporters, with 203 member groups in 80 countries. GIJN provides training, resources, and networks, with a core mission to strengthen investigative and data journalism worldwide. You can reach GIJN through its website gijn.org or write to us at email@example.com.