How to Tackle the Global Undercount in COVID-19 Deaths: Reporters Offer Tips and Techniques

Crematorium workers collect a body in Lima, Peru. Photo: Omar Lucas. Image: Courtesy IDL-Reporteros

In April, Somali freelance reporter Abdalle Ahmed Mumin decided to interview ambulance drivers in Mogadishu to check the city’s low death toll figures for COVID-19. He discovered a massive undercount simply from the fact that those drivers were carrying four times as many bodies as normal for two straight weeks.

At the same time, two reporters for The Guardian were doing a check of similarly low official pandemic death figures in the city of Kano, Nigeria. They interviewed five gravediggers, and not only found hundreds of deaths above the average, but an alarming death toll among gravediggers themselves.

Meanwhile, across the Atlantic, two data reporters at ProPublica were determined to get around inadequate official pandemic data to estimate how many Detroit residents were dying in places other than care settings. So they dug into a seldom-used data tool that archives Detroit’s 911 service calls, and found a “dead person observed” category that showed 150 at-home deaths had occurred in 10 days, compared to about 40 in previous years.

Some 3,700 miles to the south, in Lima, Peru, journalists with IDL-Reporteros found that crematoria were incinerating bodies at three times the normal rate. They also discovered that handwritten registers for crematoria companies subcontracted by health authorities actually confirmed COVID-19-related causes of death.

Around the world, investigative reporters are discovering new methods and tools to build a clearer picture of direct and indirect deaths from the coronavirus pandemic.

They are doing so in response to the palpable limits of government death count data, with widespread testing deficiencies, delays in official data, a lack of probable cause criteria on death certificates, and in some cases deliberate undercounts or cover-ups.

The result has been a global undercount in COVID-19-related deaths, and one which has serious consequences in terms of resource allocation, the use of public funds, and government accountability. Investigative reporters are helping to fill that gap.

In March, journalists with China’s Caixin media outlet focused on supply orders to funeral homes in Wuhan to gauge the scale of the death toll. Their images showed a truck delivery of about 2,500 urns to one funeral home. The truck driver told of a similarly large delivery the day before. 

Emmanuel Akinwotu, who reported the Guardian story in Kano, told GIJN: “If there isn’t reliable statistical evidence, anecdotal evidence can be very useful, so long as people stick to what they know. It’s also quite evocative. I spoke to gravediggers in three cemeteries in Kano, and they all said that deaths had gone up alarmingly. Hundreds more than usual in a week. Yet the official COVID death toll for Kano was 26.”

“What frustrated us,” said  Jack Gillum, a ProPublica data reporter, “was all these anecdotes suggesting undercounts, and our focus was on at-home deaths, so we were trying to think of ways to get numbers to fill in the gaps … Michigan didn’t really provide up-to-date data, so we turned to things like 911 call data.”

In a snapshot survey of experts and reporters, GIJN found 12 methods and six tools that are proving effective in counting the ultimate cost of the pandemic, beyond government-issued statistics:

Methods for Counting Deaths Beyond Official Figures

• For countries with relatively reliable registry statistics, compare total deaths, or all-cause mortality, to the same period in previous years, as the Financial Times did for 14 countries in late April. Where the most recent data from statistics offices are incomplete, reporters can say so and still illustrate a surge due to total deaths jumps that have been multiples of the average whether they are in Ecuador or New Jersey.
• Use “proxy sources” to estimate or illustrate excess deaths, including interviews with health workers, mortuary officials, religious funeral officiants, gravediggers, and ambulance drivers.
• Learn from business people how local authorities are classifying deaths when dealing with subcontracted companies. IDL-Reporteros found that Peruvian health authorities were candid about COVID-19-related deaths in registries used by private crematoria.
  

  Workers from a crematorium collect a body from a Lima hospital. Image: Courtesy of Omar Lucas/ IDL-Reporteros

• Look for leaks of confidential local government data. Some local authorities are conducting internal surveys of COVID-19-related deaths at public hospitals, and sending the data to national authorities, where it is sometimes suppressed for propaganda reasons. This data is ripe for leaks to reporters from local sources. On May 8, The New York Times found that the true death toll for Mexico City was more than three times higher than federal government claims, after three sources familiar with the figures revealed a confidential count ordered by the city’s mayor, Claudia Sheinbaum. A week later, a Sky News investigation found the true toll to be even higher, after visiting “dozens” of hospitals and funeral parlors, and confirming the findings with local sources.  
• Work backwards from open data portals to find audio of 911-type emergency service calls, and, where it is legal to do so, use the information from dispatchers to identify cases and affected homes. Also, look out for phrases that might be code or synonyms for deaths in certain categories.
  

  An audio file from a Detroit 911 police service call on a suspected COVID-19-related death. Image: Courtesy ProPublica

• Look for well-positioned individuals who are trying to count deaths for their own reasons. The New York Times found a demographer in Russia who had discovered 1,700 excess deaths for Moscow in April from an obscure registry, which transformed the picture of Russia as a low coronavirus-count nation.
• Use social media searches in as many languages as possible to note references to suspected COVID-19-related deaths, follow up with families and request any official documents they may have been sent.
• Focus on supposedly low-count areas that border on, or are otherwise similar to, cities or districts where high or reliable counts have already been found.
• Model deaths in a small sample district — including by going door to door, safely — and extrapolate the findings to larger areas.
• In general, look for raw, “transactional” data sources — like police calls — rather than curated data. ProPublica’s Gillum says the information computer systems spit out tend to be more reliable on sensitive topics like deaths.
• Look for bulk orders of supplies to funeral homes, like urns and caskets.
• Use existing, accurate, non-COVID excess death ratios as a guide for investigating indirect or “collateral damage” deaths elsewhere. The Oxford COVID-19 Evidence Service has found that a third of deaths above the norm in England and Wales were not COVID-19-related.

Recognizing the Data Gap

In presenting anecdotal evidence and proxy indicators of mortality surges, experts say it’s crucial that reporters clearly communicate the limits of these methods — both in terms of their accuracy and their ability to describe COVID-19-related causes — and how their conclusions were reached. Some leading stories on all-cause mortality data have featured nothing but the figures.  

While obituaries can be useful, an analysis by The City news site and Columbia Journalism Investigations found that only 5% of New York’s pandemic victims, for instance, had been memorialized in obituaries, and that the numbers were skewed toward wealthier and younger people. 

And although the internationally-recognized medical certificate of cause of death (MCCD) can be a useful classification guide, the United Nations Economic and Social Commission for Asia and the Pacific has warned that only 29% of deaths in countries in this region, for instance, have been assigned MCCDs in recent years.

Gustavo Gorriti, the director of Peru-based IDL-Reporteros, told GIJN that their counting breakthrough happened when reporters discovered that private crematoria companies were being asked by local authorities to physically collect bodies. This was in response to new regulations requiring that people whose deaths were classified as COVID-19-related be quickly cremated.

Gorriti said public hospital staff noted the true cause of death in handwritten registries, so IDL-Reporteros journalists embedded with those collection crews could record the COVID-19 classifications whenever they had a new pick-up. Two funeral homes in Lima and Callao received 1,073 bodies officially designated as COVID-19 victims by April 26, compared to the figure of 330 COVID-19 deaths announced by the government for those two cities.

“We found that a lot of incinerations were being carried out by one company; a horribly overworked company in this sad and depressing business,” he said. “We took the notes from the handwritten registers and compared them with the official numbers. There were glaring disparities.”

Gorriti added: “Our findings were not well-received by the government.”

Evaluating Excess Deaths

Dr. Carl Heneghan, a British epidemiologist and contributor to the Oxford COVID-19 Evidence Service, says “excess deaths” — total mortality above the norm for a defined period — do offer investigators advantages over other data points in the pandemic’s messy information landscape, simply because death is a fact, and bodies can be counted accurately.

Speaking in a recent GIJN webinar, Heneghan said reporters should be cautious about assuming the excess deaths over a period of many months in 2020 are definitively COVID-19-related, since, for instance, England and Wales also showed an excess deaths spike for the winter period of 2017/2018, which was not related to any pandemic. However, he believes a spike in excess deaths compressed into a period of weeks during the pandemic could be a good indicator of the impact of COVID-19.

But many people are dying from other diseases, or from a lack of hospital care, as a result of the lockdowns imposed by different governments. In a few isolated cases, in places like central Mumbai and Ahmedabad in India, fewer overall deaths were reported early in the pandemic, with some local authorities suggesting that restrictions have led to fewer traffic and rail fatalities, and even drops in deadly forms of crime.

While individual countries have different levels of reliability in their data — both on overall deaths and any COVID-19-related classification — reporters in developing countries, for example, can use trends from accurate data elsewhere as a guide to investigate local hotspots. For instance, deaths explicitly classified as “non-COVID” in non-care settings, hospitals, and care homes in England have been accurately recorded at more than twice the norm in recent weeks, Heneghan said.

“If I was to do just one thing, I’d start with all-cause mortality; try to understand how it’s reported in your country, and how that data is captured,” said Heneghan. “Then compare that to other years. You’re always looking for the excess, then you can ask questions about (causes).” 

Tools to Tackle the Undercount

• • City open data portals. Many cities around the world — including some in the developing world, which can be found in lists like this — have data portals that reporters can use to assess at-home deaths and other service call details. Detroit’s portal is one is one example, but many have their own key category phrases, like “dead person observed,” that reporters can discover by reading the tables. Many also have a “data” prefix, like the sites for Buenos Aires or San Francisco. Note that some searches require key words in all caps, and some cities, like Chicago for instance, require public records requests for call data.
    

    Image: Screenshot

  • Apps that broadcast and archive audio feeds from police and emergency medical service radio traffic. In the United States, Broadcastify — which is governed by a Creative Commons license — has helped reporters identify the fate and even home addresses of COVID-19 victims.
  • “Chain ladder” actuarial tables. Reporters can correct underestimates from authorities of deaths on particular days — underestimates that the Oxford COVID-19 Evidence Service says happen routinely, due to reporting lags — by using a tool from the world of insurance claims. You can use spreadsheet columns reflecting growing death figures for a particular date as they dribble through in updates from health authorities, and calculate a more accurate number by adding the inflation factor that the table spits out.

Screenshot. Image Courtesy Oxford COVID-19 Evidence Service

•  TweetDeck and Who Posted What can be used to search major social media platforms for references to deaths within defined areas and dates.
• The Associated Press-supported data.world platform, which integrates several major COVID-19 data bases — mostly in the US and Europe — offers free access and map-embed tools for reporters.
• Satellite imagery services with a track record of working with investigative journalists, which can supply overhead images revealing COVID-19 fall-out, including, for instance, burial trenches and unclaimed remains. A guide to satellite resources for investigative reporters — including several services that offer satellite images to newsrooms at no charge — can be found at this GIJN link. 
  

  Burial trenches dug during the COVID-19 outbreak in Qom, Iran. Credit: Satellite image ©️2020 Maxar Technologies

Fresh Leads Reporters Can Tackle Today

In one of the first investigations to expose spikes in deaths outside hospital and nursing home settings, this ProPublica story recorded jumps in at-home deaths in New York City, and parts of Michigan, Massachusetts and Washington State.

A team of three reporters used data from local vital records departments, health agencies, and 911 call centers. They also listened to audio from those calls via an archiving app, where they heard telling phrases from dispatchers, like “use your precautions here,” for  “dead on scene” police calls.

Gillum said the reporting process also revealed several promising leads and tools on excess death that they could not follow simply because time ran out.

One of these leads, he said, was a surge in certain phrases – like “medical examiner” – in “final disposition” fields on city data registries, which could indicate additional deaths, but which they did not have time to confirm.

“If I was a local reporter in the San Francisco Bay Area, for instance, I’d want to know what that means,” he said, noting that their reporting had flagged some anomalies in that area.  

Another lead for other reporters emerged from the fact that, in many cities, the ProPublica team noticed that total calls for emergency medical or paramedic assistance had plummeted by as much as 25%. Are large numbers of people opting not to call for help out of fear of hospitals, or interaction with paramedics? And has this contributed to non-COVID-19 excess deaths? Gillum said these were questions in need of investigative answers, and that they could apply in other countries as well.

Jason Burke, Africa correspondent for the Guardian, said investigative reporters in Africa should approach death count estimates with two things in mind: The narrative that Africa really does have low death counts is likely wrong, and that many families around the continent were failing to disclose COVID-19-related deaths because they did not want their relatives’ funerals to be disrupted by pandemic restrictions.

“This narrative, basically supported by the World Health Organization, that Africa has somehow dodged a bullet because of its youthful population, etcetera, is just nonsense,” he said. “The worst offender has been the African Union, which put out a statement last week basically saying that testing was robust and the figures were robust. You don’t have to know much about how states function on the continent to know that simply cannot be the case. The question is how to show the misrepresentation.”

Burke has been asking a network of stringers around the continent to focus on proxy sources to assess real death rates, including ambulance drivers, casket makers, and gravediggers. 

Burke said he was worried about major potential undercounts in countries like Burkina Faso, Zimbabwe and Burundi. By contrast, he said he initially thought the Democratic Republic of the Congo (DRC) would have a disproportionately high death toll, but recent indications suggested that both its response and its data reporting systems had been better than similar nations. 

“So far, I haven’t found a massive surge in mortality that’s going unreported in Kinshasa,” he said. “I think the DRC is (surprisingly) robust, not least because it has a really good body of epidemiologists and infectious disease specialists and systems to cope with this kind of thing, because it gets hit by so many.”

To investigate the death toll in Kano, Nigeria, Akinwotu said his colleague, Mustapha Hodi, spoke to three gravediggers while wearing a face mask, and preserving social distance. These included 75-year-old Musa Abubakar, who said cemetery workers had not been issued gloves or face masks, and noted: “I have never witnessed mass deaths like this.” Local clerics and funeral officiants also proved helpful to their investigation.

Akinwotu himself was trapped in London by pandemic travel restrictions. So he worked the phones, asking a person in Kano with personal protective equipment and the relevant language skills to serve as a conduit for his questions to gravediggers at other cemeteries. He said a chronic shortage of COVID-19 tests in the city in April had made a large sample of anecdotal evidence essential.

Akinwotu said the next tool he planned to explore in searching for excess deaths was Facebook Messenger.

“It is really effective in terms of getting responses from strangers, but the language barrier is a problem,” he said, “because I only speak English and Yoruba.” 

Rowan Philp is a reporter for GIJN. Rowan was formerly chief reporter for South Africa’s Sunday Times. As a foreign correspondent, he has reported on news, politics, corruption, and conflict from more than two dozen countries around the world.