As part of our research at the Puppycide Database Project, our volunteers read a lot of news stories about police using deadly force. Many of these articles are well-researched, hard-hitting news stories that provide real accountability. Other articles ... not so much.
Over and over again we come across the simple republication of a police-issued press release as a news story. Just as popular is a trend in which reporters riding along with police and breathlessly retelling the story. Despite two Supreme Court cases holding that police bringing reporters along to, for example, execute search warrants violates the Fourth Amendment, ride-alongs have not just continued as a normal practice, but they have blossomed. There are many, many TV shows based entirely around the practice (TV shows that we will refrain from providing free advertising to by linking them in our post).
Such obviously problematic practices aren't the only issues we have encountered police reporting. Perhaps the biggest problem is the numbers.
It's the reason why this project was started: there is no national or even a single state database of police use of deadly force. As a result, the public relies largely on investigative journalists to research and uncover the number and reasons for police killings.
Unfortunately, we regularly see such reports getting the numbers wrong.
A recent report by TIME Magazine titled "Pet Owners Look to Muzzle Police Who Shoot Dogs" reviewed the rising tide of widely publicized and controversial police killings of pets. The report was sympathetic toward animal rights activists and the families of dogs who had been killed by police: TIME argued that police are poorly trained to handle their interactions with animals, and that puppycide is a result of the failure of police departments to prepare their officers for those interactions.
Despite the tone of the article, TIME included a preposterously low estimate of lethal force used toward dogs:
"There are no official tallies of dog killings by police, but media reports suggest there are, at minimum, dozens every year, and possibly many more."
Such a claim is essentially meaningless. In the statement above, the only reference that TIME made to the number of killings of dogs by police, TIME claims that the number is somewhere between zero and infinity. TIME could have easily confirmed a much higher number of killings using government records that are easily accessible through the public domain. Many municipalities maintain a Civilian Review Board that publishes reports of police shootings, and many reports cover the shootings of animals. For those without a Civilian Review Board, other municipalities publish an Annual Use of Force Report that includes the total number of police shootings. Finally, a growing number of police departments that have historically refused to entertain either police are being forced to through "Consent Decree" arrangements with the Department of Justice. The Puppycide Database Project has recovered files from police agencies bound by one or more of the situations stipulated above in Albuquerque, Chicago, Cleveland, Denver, Los Angeles, Milwaukee, New Orleans, New York, Seattle and many others. In any one of these cities, police shoot "dozens" of dogs each year. There are over 18,000 police departments across the United States.
The error was pointed out to TIME on their website using the online comments section. TIME did in fact end up running a correction on the story, but the correction had nothing to do with the statistics they cited, it was for a misquote. The comments section pointing out the error appears to have been removed from the website as of this writing.
Even reporters engaging in more thorough analysis that cite accurate numbers can mislead their readers. In mid-2014, the Washington Times ran a piece from the Associated Press seeking to draw attention to concerns similar to those raised by the TIME article: "Training helps to reduce Milwaukee pet shootings".
"Milwaukee police shoot and kill dozens of dogs each year, often saying the animals appeared to be a threat to officers or to the public. But in response to criticism and two lawsuits, police have been able to reduce that number with improved training, which began in 2012 and expands this fall. The sessions teach officers to recognize when dogs’ actions are nonthreatening or when less-lethal defense might be more effective."
In essence, the article is making the claim that the Milwaukee Police responded to public pressure and a series of lawsuits to stop killing pets by introducing new training, and that training worked. To back up this claim, the article cites reports from the Milwaukee Fire and Police Commission:
"There were 41 dogs killed in 2009, and by 2012 the number was down to 28, according to the Fire and Police Commission."
The quote purports to show a 32% drop in dogs killed by police after the training. There is an immediate issue of the confusion of a correlation in a causation which plagues discussions surrounding police training. Was the new training accompanied by direct orders from police superiors to stop killing dogs that replaced prior standing orders? Did department brass tell patrolman that there would be disciplinary consequences for killing dogs, where none had existed before? How do we separate the effects of such efforts, where they exist, from the impact or non-impact of training itself? Such concerns apply even where significant drops in animal killings exist following training. However, the issue here is that the dramatic reduction that the AP claimed occurred did not.
The Puppycide Database Project obtained and reviewed the same reports that the AP & Washinton Times cited (Readers are welcome to review those reports as well - they are available for download on our Datasets Page, as well as from the Milwaukee Fire and Police Commission directly).
In order to show a dramatic shift, the AP focused on a very specific metric: the number of dogs killed by the police. While it seems at first blush like this would be the only relevant number, it ignores dogs that were shot but not killed by police, as well as other animals that were killed by police. When we look at all of the animals that the police shot, the reduction is not as significant as it initially appears.
Instead of a 32% reduction, there was a 13.12% reduction.
Two factors contributed to this discrepancy. The number of dogs that were shot by police, but survived, was much higher in 2012 than in 2009. In 2012, 28 of 37 dogs shot were killed: a death rate of 76%. By contrast, in 2009, 45 of the 47 dogs shot were killed: a death rate of a much higher 96%. It is unlikely that training played a role in reducing a near-total rate of death to the lower rate seen in 2012. Likelier explanations would include changes in policy for providing medical care to dogs who have been shot, or changes in how animal deaths are recorded. The records provided by Milwaukee do not specify whether dogs who have been euthanized after being shot qualify as "use of force" deaths or not: including such deaths for one year and excluding them the next could easily account for such a drastic difference.
The other key issue in the reports is something that could easily be overlooked. The Fire and Police Commission deliberately omit a significant number of police shootings of animals - so long as a the reporting officer states that the shooting was a "mercy killing" or euthanasia, the use of force is completely ommitted from the statistics in the report. Here is how the 2009 report phrases the omission, which is mentioned only once in the entire report, and not included in any statistical report analysis:
"From January 1, 2009 to December 31, 2009, there were 485 use of force incidents recorded by the MPD. Of these 485 incidents, six were accidental discharges of weapons and 20 were for the purpose of euthanizing an injured or diseased animal. As these 26 incidents are fundamentally different from other use of force incidents in the purpose and intent of the force, these incidents are excluded from all subsequent analyses. Accordingly, 459 incidents are analyzed in this report."
The Puppycide Database Project includes these killings in our own review, and in the numbers we provided above. While it may be true that shootings conducted for the purposes of euthanasia are "fundamentally different from other use of force incidents in the purpose and intent of the force", completely omitting these results from further analysis is nonsensical. All uses of lethal force require investigation and independent review. Most of the animal shootings involved in the Milwaukee reports involve only a single police officer. The idea that, police officer or not, Milwaukee is simply taking shooters at their word for what occured without further inquiry is truly shocking. It is inexcusable to allow a self-reported claim of medical necessity to over-ride not just an investigation but the recording of the event in shooting statistics - but that is exactly how the Milwaukee Police handle their reports, and as a result the Associate Press/Washington Times sold the public on numbers divorced from reality.
The AP/Washington Times was not attempting to triviliaze the amount of killings in Milwaukee. Their report shows how even the reduced intepretation of the numbers plays out to more than twice the number of dogs supposedly killed by the New York City Police Department:
"The 28 dogs killed by Milwaukee police in 2012 were more than twice the 12 dogs killed by New York City police."
We were surprised to find that even the comparison also came from the Milwaukee Report and not from the journalists. It is very unusual for a municipal police report to compare use of force numbers directly with those of another police department.
Although we have treated TIME, AP, and the Washington Times harshly in this post, it is not because we wish to give them a bad name or to suggest that they should stay away from reporting on these issues. Quite the opposite - of the many, many mistakes the Puppycide Database Project has found in reporting on the use of lethal force by police, these news organizations represented some of the most well-respected sources of news. By focusing on these organizations, we hope to point out how easy it is to provide incorrect information that can alter public perception of this critical problem - even when the intention is not to be dishonest or manipulate public opinion.
Making mistakes is part of being human - and those of us here at the Puppycide Database Project are no exception. We make mistakes as well. With that said, we have developed some fairly clear cut rules for correcting errors - we place corrections to stories above the fold, on the very first line of the article being corrected. We also create separate articles for each correction, linking between the new article and the issue being corrected. Finally, we post to social media to make sure those following us are aware of the latest revisions.
Transparency is vital, not just for police and government agencies, but for media and research organizations as well.