Surveillance and Data Quality

This essay provides many specific examples of new knowledge available from the NVDRS, some of which has already informed policy.

Barber, Catherine; Azrael, Deborah; Hemenway, David. A truly national National Violent Death Reporting System. Injury Prevention. 2013; 19(4):225-6.

A broadened reporting system, not only for firearms but for all violent deaths (all suicides and homicides) will provide more useful data, at only a small increased cost. This article summarizes the need for such a surveillance system, and its status as of 2001.

Azrael, Deborah; Barber, Catherine; Mercy, James. Linking data to save lives: Recent progress in establishing a National Violent Death Reporting System. Harvard Health Policy Review. 2001; 2:38-42.

This article highlights the benefits of surveillance systems for various social issues (e.g., economics, crime, public health). It shows how the additional information provided by a national violent death reporting system can be used for policy evaluation.

Azrael, Deborah; Barber, Catherine; Hemenway, David; Miller, Matthew. “Data on violent injury.” In: Jens Ludwig and Philip J. Cook, eds. Evaluating Gun Policy: Effects on Crime and Violence. Washington D.C.: Brookings Institution. 2003.

We describe the effort of many groups, which led to the creation of this data system.

Hemenway, David; Barber, Catherine W; Gallagher, Susan S; Azrael Deborah. Creating a national violent death reporting system: A successful beginning. American Journal of Preventive Medicine. 2009; 37:68-71.

Do the vital statistics provide an accurate count of unintentional firearm deaths? We compared the Supplemental Homicide Report data on “manslaughter by negligence,” which are considered to be accidents with the vital statistics data. We found that only 23% of the negligence manslaughters were classified as accidents on the death certificates. Official vital statistics data almost certainly undercount firearm accident deaths when the victim is shot by another person.

Barber, Catherine; Hemenway, David; Hochstadt, Jenny; Azrael, Deborah. Underestimates of accidental firearm fatalities: Comparing Supplementary Homicide Report data with Vital Statistics. Injury Prevention. 2002; 8:252-256

We carefully read all the circumstances of any death characterized as an unintentional firearm fatality by the state vital statistics registry (death certificate), the medical examiner or coroner, the police, or the National Violent Death Certificate (NVDRS) abstractor. We found that the NVDRS data were extremely accurate but the Vital Statistics data were not. The Vital Statistics data seriously under-report accidental deaths to children (many true accidents are reported as homicides) and over-report accidental deaths to adults (many homicides and suicides are reported as accidents).

Barber, Catherine; Hemenway, David. Too many or too few unintentional firearm deaths in official U.S. mortality data? Accident Analysis and Prevention. 2011; 43:724-31.

Various proxy measures for the prevalence of firearm ownership were compared with surveys-based estimates. One proxy, the percentage of suicides with a firearm, performed consistently better than other measures in cross-sectional comparisons.

Azrael, Deborah; Cook, Philip J; Miller, Matthew. State and local prevalence of firearms ownership: Measurement, structure and trends. Journal of Quantitative Criminology. 2004; 20:43-62. Also see National Bureau of Economic Research Working Paper #8570.

This commentary emphasizes that cost of injury estimates that focus on medical costs and productivity losses of the victims (e.g., WISQARS) dramatically underestimate the costs of street gun crime. Costs of street gun crime also include costs to the shooters family, criminal justice costs, trauma of witnesses, the costs of avoidance (e.g., children not being able to go out and play), the likelihood of retaliation, and the destruction of neighborhoods

Hemenway, David. Measuring the cost of injury: Underestimating the costs of street violence. Injury Prevention. 2011; 17(5):289-90.

Homicides by police are vastly undercounted by police reports and by Vital Statistics. The National Violent Death Reporting System will be an excellent source for good data on police homicides but all states need to be part of the system.

Police killings are constantly in the news. This article provides another endorsement for the NVDRS, and shows that the U.S. doesn’t need to create another data system for the sole purpose of tracking police homicide.

Barber C, Azrael D, Cohen A, Miller M, Thymes D, Wang D, Hemenway D. Homicides by police: Comparing counts from the National Violent Death Reporting System, Vital Statistics, and Supplementary Homicide Reports. Journal of Public Health. 2016; 106(5):922-7.

This essay in a statistical journal highlights the importance of data systems (i.e., data collected consistently and comparably across sites and over time), provides examples of the need to recognize the current limitations of each system (e.g., NVDRS) along with the importance to keep improving them, and bemoans the lack of both data and funding for the analysis of firearm issues.

Hemenway, David. Firearms data and an ode to data systems. Chance (American Statistical Association) 2018 Feb; 31(1):7-11.

We argue that as anti-science sentiment sweeps the world, it is vital to stop the suppression of firearms studies. “The attempt to muzzle research requires constant push-back. I am always shocked to remember how recent the Enlightenment was, and how fragile is the freedom to be able to make careers out of the search for truth.”

Hemenway, David. Fighting the silencing of gun research. Nature. 2017; 546: 345-47.

Virtually all existing evidence on firearms and health come from ecological and case-control studies. Because California has been collecting data on all lawful handgun transfers, HICRC has been helping to create a large cohort of more than 28 million California adults by linking the handgun data with voter registration files and all-cause mortality data. Assembly of the LongSHOT cohort was completed in 2019. This dataset will provide incredibly useful longitudinal data on risk factors for violent death.

Zhang Y, Holsinger EE, Prince L, Rodden JA, Swanson S, Miller, M, Wintemute G, Studdert D. Assembly of the longshot cohort: public record linkage on a grade scale. Injury Prevention. 2020; 26:153-158.

We cross-linked individual-level mortality data from the 2015 NVDRS and 5 open-source data sets (FatalEncounters.org, Mapping Police Violence, the Guardian’s “The Counted,” Gun Violence Archive, and the Washington Post’s “Fatal Force Database.” Of all the individual fatal shootings from these 5 other data bases, NVDRS captured 97%. NVDRS provides a comprehensive national count of fatal police shootings and detailed circumstantial information about these deaths.

Conner A, Azrael D, Lyons VH, Barber C, Miller M. Validating the National Violent Death Reporting System as a source of data on fatal shootings of civilians by law enforcement officers. American Journal of Public Health. 2019; 109:578-584

In red state Utah, diverse stakeholders backed a state-funded study of firearm suicide. Linked data showed the proportion of suicide decedents who (a) could

have passed a background check, (b) had a concealed carry permit, and (c) had been seen for a previous suicide attempt. Informed by the report’s findings, the legislature, health care and religious groups, and state agencies all launched major initiatives to reduce firearm suicide. The Utah experience provides a case study in bringing diverse stakeholders—particularly gun owners—together to find common ground and in using linked data to support and guide their efforts.

Paper selected as Editor’s Choice for Top Ten Papers of the Year

Barber C, Berrigan JA, Sobelson-Henn M, Myers K, Staley M, Azrael D, Miller M, Hemenway D. Linking public safety and public health data for firearm suicide prevention in Utah. Health Affairs. 2019; 38:1695-1701.

This commentary argues that while injury researchers often complain about the lack of quality data, we don’t do nearly enough to create and improve data systems.

Hemenway D. Let’s make it a priority to improve injury data. Injury Prevention. 2020; 26:395-396.

The US firearms data infrastructure is limited and fragmented. In the absence of federal leadership, we propose that states evaluate their data gaps, expand data collection, and improve data presentation and availability.

Durkin A, Willmore CN, Sarnoff CN, Hemenway D. The firearms data gap. Journal of Law, Medicine, and Ethics. 2020. S248:32-38.

Half of non-fatal gunshot wounds are not unintentional. In this Commentary, we explain why miscoding occurs, and how it can be corrected.

Barber C, Goralnick E, Miller M. The problem with ICD-coded firearm injuries. JAMA Internal Medicine. 2021; 181:1132-33.

Examining 1227 cases of firearm injury from hospital discharge data, we found 68% to be assaults. But more than a quarter of these were coded as unintentional injuries in hospital discharge data. The Nationwide Emergency

Department Sample (NEDS) cannot be used to reliably count intent-specific firearm injuries.

Miller M, Azrael D, Yenduri R, Barber C, Bowen A, MacPhaul E, Mooney SJ, Zhou L, Goralmich E, Rowhani-Rahbar A. Assessment of the accuracy of firearm injury intent coding at 3 US hospitals. JAMA Network Open. 2022; 5:e2246429.

The natural language processing model was used to evaluate 381 patients with firearm injuries. The model proved more accurate than medical coders in assigning intent.

MacPhaul E, Zhou L, Mooney SJ, Azrael D, Bowen A, Rowhani-Rahbar A, Yenduri R, Barber C, Goralnick E, Miller M. Classifying firearm injury intent in electronic hospital records using natural language processing. JAMA Network Open. 2023; 6:e235870.

Carpenito T, Miller M, Manjourides J, Azrael D. Using multiple imputation by super learning to assign intent to non-fatal firearm injuries. Preventive Medicine 2022; 165:107324.

The US currently does not have a system that accurately tracks nonfatal shooting injuries. This paper characterizes the strengths and weaknesses of existing federal data systems (i.e., hospital and police data) and explains how they can be improved. For example, for data from health care agencies, challenges currently being addressed include public access, timeliness, and accuracy of coding of intent (hospitals misclassify many firearm assaults as accidents)..

Barber C, Cook PJ, Parker ST. The emerging infrastructure of US firearms injury data. Preventive Medicine 2022; 165:107129.

Reading the NVDRS (2009-2018) narratives, we found that abstractors too often incorrectly classify cases of accidental gun deaths of children as homicides if the narrative explicitly noted adult negligence as contributing to the shooting death, NVDRS coding guidelines should explicitly state that negligent manslaughter is not a contraindication of unintentional shootings provided the firearm was not used to intentionally harm, threaten or coerce.

Fischer S, Miller M, Barber C, Azrael D. Accuracy of the National Violent Death Reporting System in identifying unintentional firearm deaths to children by children. Injury Epidemiology. 2024:11:29.