Given that it’s been a long time since I last explained this, and since there have been more people sending links to reports lately in an effort to help, I thought it was a good time to go over our reporting methodology again.
Why is methodology important?
Our reporting methodology is very important for the statistical generation portion of this project. In order to keep our data consistent and reliable we have to maintain a specific process by which we determine whether a report qualifies for inclusion into our database for statistical analysis, how those reports should be categorized, when an update to a report qualifies for inclusion, and how to deal with those updates in order to avoid duplicate reports.
We don’t just record every single report and update then count it all at the end of a cycle, there is a set and defined process by which everything has to happen in order to assure everyone that when we publish statistical reports and comparative reports that they use standardized methodology and, thus, are actual “apples to apples” comparisons between different sets of statistical data.
For example, in order to have an accurate comparison between the FBI/DOJ UCR crime rate statistics and our NPMSRP police misconduct rate statistics, we have to make sure that we use the same methodology tracking police misconduct that the FBI/DOJ uses to track crime or else it wouldn’t be a valid comparison and there would be no chance for our numbers to have any validity whatsoever.
What qualifies as a valid report?
Generally, there is a set of criteria a report must meet in order to be included in the database that includes:
- that the report is about a sworn law enforcement officer (must be a sheriff, police officer, or constable with full police powers of arrest and use of force)
- that the law enforcement officer is currently employed as such at the time of the alleged incident
- a determination of whether the report is credible
- existence of supporting evidence or third-party statements
- that the report is within the recording period time frame
- that the report is publicly available for review
- that the report does not already exist in the database and, if it does, that it has an update status that qualifies to overwrite the previous report.
Why only sworn officers?
Now, one of the most common questions we have is why reports must be about sworn officers and why we don’t include reports about corrections officers or a sheriff’s secretary. The reason for this is that we have the data for the number of sworn officers in the US so we can create statistics around that. Also, these are the people with the authority granted by the state to use force to enact the detention of citizens. They are the ones with the authority that we are concerned about abusing, not the clerk in the department filing reports.
What are “Qualifying Updates”?
In order to avoid counting a single incident multiple times we use a hierarchical status methodology which allows us to track the status of incidents while avoiding duplicate reports. So, if a report is mentioned in a hundred different news papers it is only recorded once in our database and, if another paper reports on the same story later without a valid update, it’s not added as a new report.
In order to do this, we check each story we add to the database against the database to ensure it wasn’t already tracked and, if it was, we check the new report to see if it has a qualifying update status. If the new status qualifies as a status update we pair it with the original report and give the original report a null counter so that it isn’t added to the statistical data. This allows us to track the progression and outcome of incidents without risking duplicate reports.
Why do that?
So that we can track statistics on things like conviction rates, disciplinary rates, police misconduct costs via civil litigation, and other data without worrying that, in the process, our data would be skewed with duplicate reports or become unwieldy due to an excessive number of status updates.
So, we have a status update model which will exclude some updates while allowing others. This is why, sometimes, when you submit a report to us it won’t get published in the feed. Because it already exists and the status isn’t a qualifying update which ranks higher than the existing status of the report in the database.
What is this status hierarchy?
To qualify as a status update to an existing report the new status must rank higher than the currently recorded status in order to be added to the database as an update that overwrites the current status. In order from lowest to highest rank:
- Allegation
- Investigation
- Lawsuit
- Disciplinary
- Termination/Resignation
- Charges
- Settlement/Judgment
- Conviction/Plead
- Sentencing
Now, there are exceptions. If the current status ranks higher than “judgment/settlement” we still record the data of that status in order to track police misconduct costs, but it doesn’t overwrite the current status, it’s added as a null value.
Also, if a new status includes a new allegation concerning a new incident it’s added as a new incident… but if it concerns a new allegation that is part of the same incident it won’t qualify unless that new allegation is ranked higher in the catagorization hierarchy than the current catagorization…
What am I talking about now?
Categorization Hierarchy
In order to maintain a parity with the US DOJ/FBI UCR reporting methodology we use a hierarchical categorization methodology in order to assign each incident to a specific category of incidents for statistical analysis.
When a report concerns a single incident with multiple allegations we only categorize that incident in association with the most severe allegation made within that incident… which is the same thing the UCR does.
For example, in the UCR, if an incident is recorded where a suspect kidnapped then murdered a victim that would be recorded only as a murder, not a kidnapping.
So, if we get a report where an officer racially profiled someone, beat them, then falsely arrested them to cover for it. It would be recorded as an excessive force report.
Now, the exception here is when we’re dealing with an incident involving multiple officers and each officer is accused of a different act of misconduct within that single incident. So, if one cop beats a suspect and the other only assists with the cover up then it is recorded as two incidents, one excessive force case against one officer and one accountability case against the other.
So, how do these rank? From lowest to highest categorization:
- General
- Accountability
- DUI
- Drugs
- Animal Cruelty
- Bias
- Sexual Harassment
- Dishonesty
- Color of Law
- False Arrest
- Assault
- Sexual Misconduct
- Sexual Misconduct with Minor
- Excessive Force
- Taser Abuse
- Firearms/Shooting
- Murder
Now, there is some variance here in that some allegations of a certain type are more serious than a less severe allegation of a higher ranking type. So, sometimes there is some play in what category a report will be classified as, but these are the general guidelines.
There’s more…
There are other miscellaneous rules that apply to recording these for statistical analysis. Such as how we assign the number of officers to an incident when the number is unspecified but clearly indicative of multiple officer involvement. How we record fatalities that are not officially recorded as murders, victim tracking, and other issues.
The key here is that the process of recording reports for statistical analysis and the statistical analysis itself is a complex and carefully thought-out process that has to abide by a specific set of guidelines in order to maintain parity with other compatible statistical models so that we can perform reliable and verifiable comparisons with the other statistical reports out there and glean information about the severity of police misconduct in the US.
Still, we are always working on ways to improve the quality of data that we track and include more data in our tracking capabilities. As time progresses and, if I get more time and resources to do better tracking, we might expand the scope of what we track with each report, expand classifications, and even begin to track detainee and prisoner abuse reports as a separate reporting effort. So we’re always examining what we do to see if we can do it better.
The key is that this is not a simple and easy process, it’s not just a matter of recording every single report we see and adding up that number at the end of the month without rhyme or reason. So if we don’t put a report you tell us about on the feed it’s not personal, there’s a process in place which determines when a report qualifies and when it doesn’t. So, please don’t take it personally when a report you tell us about doesn’t make it on the feed and don’t let it discourage you from continuing to let us know when you see a report you think qualifies for inclusion in our database.
Thanks!
Recent Comments