Methodology

Movement Alliance Project and MediaJustice created this site as a tool to support organizers and movements seeking to end pretrial incarceration. We believe organizers need to pay close attention to exactly how these tools are used, and approach risk assessment with strong skepticism when it is presented as a solution to money bail reform and pretrial incarceration.

Once they learn the status of their risk assessment, we would strongly encourage you to refer to Community Justice Exchange’s Organizing Guide for Confronting Pretrial Risk Assessment Tools in Decarceration Campaigns.

This site serves to both document important details about the pretrial risk assessment tools in use in the communities we surveyed and researched – their factors, weights, and impact – as well as our arguments about these factors’ correlations with race and poverty as applied nationwide.

Our research framework:

1. We looked at how tools are designed: different variables within a tool, the tool’s developer, the tool’s rate of predictive “validity,” and how tools define recidivism (i.e. new criminal arrest and failure to appear in court).
2. We searched for breadth of tool usage: which tools are used where, all across the country.
3. We assessed community impact: how tools impact jail populations and pretrial jail demographics. 
4. We examined the policies and practices of implementation: which individuals are assessed by these tools in any particular jurisdiction, how the results are used in the pretrial decision-making process, and availability and access of results during trial to different people who are a part of the decision-making.

Our initial research phase started with a surface level literature review of governmental reports, validation studies from research institutions, and county pretrial websites. This review, as well as our engagement in community organizing around these tools in Philadelphia and in other communities, set the basis for our understanding of these tools and informed the design of our interview questions and research framework.

We then started searching for common pretrial RATs and began to understand the basics of how they work.

We also interviewed pretrial service agencies and others who conduct assessments, to get to know the tools and how they are used, and to better understand the effect these systems have on local populations. Early interviews gave our researchers a basic outline of which tools exist and how they function in pretrial processes, and as we learned more, our interviews covered deeper questions of tool impact and institutionalization.

Our full interview summary is available for download alongside our database. The details of individual interviews are available upon request.

 As our research developed, we also turned to more academic sources on the functioning and impact of pretrial RATs as well as the policies and practices of many criminal legal systems.

We also dug deep to understand how these tools worked, with regard to their variables as well as how they are validated. As algorithms, many pretrial RATs are complex and opaque to the public, but our researchers believe we can understand the impacts of RATs without the actual algorithm models or advanced statistical and computer science backgrounds.

Sources

Interviews

Movement Alliance Project developed a survey with 27 questions to gather feedback from on-the-ground risk assessment tool implementers. Telephone interviews lasted 30 to 45 minutes with governmental staff and other stakeholders who conduct risk assessments.

Interviewees provided both quantitative and qualitative feedback about the tool they use, the factors and scoring of the tool, and if it had been validated. We also asked about their pretrial services procedures, including who is assessed, who can see the results of the tool, and how the tool impacts the pretrial process, as well as the pretrial jail population and demographic statistics, to determine the impact of the tool.

Our team conducted 38 interviews, mostly over the phone, with pretrial service agencies and others that conduct assessments across the United States. Jurisdictions include New Orleans, Los Angeles, Harris County, Texas, Connecticut, New Jersey, Utah, and several counties in Kansas and Tennessee.

These interviews gave us an in-depth understanding of how pretrial risk assessment tools are implemented locally, and the diverse ways in which jurisdictional staff see those implementations.

We also emailed over 530 individuals at county and state governments to inquire about the tools they use and to build out our national map of risk assessments utilized.

Secondary Sources

We also used secondary sources, like documentation sites for risk assessment tools and academic research sites, along with county and state webpages, to build our database. Some jurisdictions have publicly available information on the tool they use, as well as some details on how the tool interacts with their criminal legal system.

Alaska, for example, has information on their Department of Corrections¹Pretrial Enforcement Division: Assessment Tool Information, Alaska Department of Corrections site as well as articles available²Pamela Cravez: Pretrial Risk Assessment Tool developed for Alaska, Alaska Justice Forum that detail how the tool was developed and its uses, as well as the factors it employs.

Orange County, North Carolina, also has detailed information on the Pretrial Release portion³Pretrial Release, Criminal Justice Resource Department, Orange County, North Carolina of their county website describing how the VPRAI-R is implemented there, as well as an interview with the Pretrial Release Case Manager⁴Jessica Smith: Bail Reform in North Carolina: Orange County Reforms, North Carolina Criminal Law, UNC School of Government Blog that breaks down the reform and assessment process.

We also utilized publicly available FOIA (Freedom of Information Act) requests from MuckRock to gather data around county tool usage.