Pretrial RATs and the decisions they prescribe can have an enormous impact on an accused person’s life, and the lives of their family and community.

All over the country, pretrial RATs help judges and magistrates decide if someone gets to come home before their trial.

In many places, the risk scores from pretrial RATs influence if a judge or magistrate decides to release or detain someone pretrial, what their bail will be or if the court will set a bail, or what kind of conditions or supervision requirements the court will require them to adhere to if released.

There is a lot of research showing that being detained pretrial is extremely harmful.

People who are detained are more likely to lose their jobs and future employment prospects,1Will Dobbie, Jacob Goldin, and Crystal Yang: The Effects of Pretrial Detention on Conviction, Future Crime, and Employment: Evidence from Randomly Assigned Judges, American Economic Association as well as their homes, access to medication, and even their children.2Paul Heaton, Sandra Mayson, and Megan Stevenson: The Downstream Consequences of Misdemeanor Pretrial Detention, University of Pennsylvania Law School 

Those detained pretrial are also much more likely to be convicted3Emily Leslie and Nolan G. Pope: The Unintended Impact of Pretrial Detention on Case Outcomes: Evidence from New York City Arraignments, Journal of Law and Economics and receive longer sentences than those who are released,4Megan Stevenson: Distortion of Justice: How the Inability to Pay Bail Affects Case Outcomes, Journal of Law, Economics, and Organization often because they plead guilty and take a sentence just to get out of jail.5Paul Heaton, Sandra Mayson, and Megan Stevenson: The Downstream Consequences of Misdemeanor Pretrial Detention, University of Pennsylvania Law School Those detained pretrial are also more likely to be arrested again in the future.6Paul Heaton, Sandra Mayson, and Megan Stevenson: The Downstream Consequences of Misdemeanor Pretrial Detention, University of Pennsylvania Law School

Those who are released are able to handle their cases in partnership with their lawyers, with the support of their families and communities.

As with most oppressive systems, the impacts fall primarily on people of color and poor people. That’s why we want to make sure that RATs helping determine if people are free or not before their trial do not uphold and reproduce the same ugly biases already embedded in our criminal legal system.

As noted, some decision-makers propose RATs as the necessary step in removing money bail. We reject the idea that money bail, a system that targets and harms poor people and people of color, needs to be replaced with another system that does the same thing, especially as we see some jurisdictions expand their pretrial incarceration and fail to reduce racial disparities after implementing RATs.

Some of our communities’ main concerns with pretrial RATs include:

  1. Many of these RATs have a disproportionate racial impact: they draw from racially biased data and racially biased systems, so they are likely to reproduce inequalities in their outcomes.
    • The factors that feed into many of the pretrial RATs used across the country are proxies for race or class, and they often perpetuate racial and economic bias. For instance, many tools include factors like whether or not someone is employed or has stable housing in making their predictions, which are clearly connected to a person’s economic status.
    • Scholars have even stated: “the problem of bias in criminal justice data is such a substantial, serious threat that it calls into question the entire endeavor of data-driven risk assessment.”7David G. Robinson and Logan Koepke: Civil Rights and Pretrial Risk Assessments, Upturn Inc.

  2. There is a severe lack of clarity and transparency in jurisdictions around what training data goes into RATs and how they are used – especially when tools are proprietary or complex machine-learning algorithms.  
    • People charged with crimes have a right to know what tools are used to evaluate them and how these tools impact what happens to them. In many initial arraignment hearings, accused people lack basic court safeguards to protect their rights. 8ACLU Kansas: Challenging Pre-Trial Risk Assessment Tools
    • At many arraignments, which happen in a matter of minutes, accused people don’t have a lawyer present. If a defense representative is there, they have likely not had a chance to talk to their accused client. The accused person does not generally have any chance to review a RAT score or the factors that went into it, or to contest a RAT score before an arraignment hearing.
    • As we saw in our interviews, in some cases, decision-makers who use the results of RATs may or may not understand how the scores are calculated, or even know all the inputs that go into their tool. They won’t necessarily know what particular inputs pushed an overall score to be especially high or low, or what the result is meant to communicate to a decision-maker.

  3. RATs provide decision-makers in the criminal legal system with a false sense of scientific fairness when making their decisions.
    • No tool is created or used in a vacuum. The factors used, how they are weighed, and the definition of “risk” are all subject to human bias. Most risk assessments are tied to decision-making frameworks — sets of policy guidelines meant to interpret the results of risk assessment tools. Moving these decisions to a computer and a new decision-making framework merely bypasses or buries the question of human responsibility.
    • Because RATs rely on criminal legal system data from the past few decades, many of which were collected before more recent pretrial reforms, they create what researchers have termed “zombie predictions9John L. Koepke and David G. Robinson: Danger Ahead: Risk Assessment and the Future of Bail Reform, Washington Law Review that overestimate risk. RATs also attempt to turn “dangerousness” into a measurable output,10John L. Koepke and David G. Robinson: Danger Ahead: Risk Assessment and the Future of Bail Reform, Washington Law Review which some argue is an impossible task. RATs often have to choose between statistical accuracy and fairness in terms of equity for anyone being assessed11Chelsea Barabas, Karthik Dinakar, and Colin Doyle: The Problem with Risk Assessment Tools, The New York Times – a process that is not clear to those using or being judged by these tools.
    • These tools are supposed to be “evidence-based,” meaning they are backed by some kind of data and research. But without actively questioning the biases baked into that data collection, one article argues, bias does not disappear; the use of “objective” methodologies does not undo structural inequality.12Kay Whitlock and Nancy A. Heitzeg: Billionaire-Funded Criminal Justice Reform Actually Expands Carceral System, Truthout
    • As one research team found, the statistical issues of fairness within risk assessments creates its own bias, but these tools also inherit the bias from the underlying data they use and the inherent bias from the way we predict individual behavior based on group models.13Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook, and Julie Ciccolini: Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment, Criminal Justice and Behavior

  4. Validation scores and statistics show that RATs are not necessarily highly accurate or predictive on the questions they are trying to answer, especially on questions of violence, which is arguably the result that decision-makers and communities are most concerned with predicting. In many places, the tools are not validated at all, haven’t been validated for years, or have not been validated on local systems or  populations.
  5. Regardless of the input factors or intention of the developers, RATs are not always used as intended. Judges and magistrates have discretion over pretrial outcomes, and studies have shown that they often deviate from the tool’s recommendations to push for pretrial detention16Brandon Buskey and Andrea Woods: Making Sense of Pretrial Risk Assessments, The Champion and over-detain those who receive low scores.17Megan Stevenson: Assessing Risk Assessment in Action, George Mason University One scholar even argued that judges follow RAT recommendations when it offers rationalization for the judge to make the choice they wanted anyway.18Andrew Van Dam: Algorithms were supposed to make Virginia judges fairer. What happened was far more complicated, The Washington Post
    • The Pretrial Justice Institute19Pretrial Justice Institute: Scan of Pretrial Practices has argued that many jurisdictions use RATs to make set bond amounts and make detention and release decisions, which “raises legal concerns” because detention decisions should happen at a full hearing in front of a judge. Pretrial RATs were designed to set release conditions; only clear evidence that no release conditions could ensure the safety of the individual and the public should lead to detention. Pretrial detention, the Pretrial Justice Institute argues, “is not a decision that should be based simply on probable cause and an assessment score.”20Pretrial Justice Institute: Scan of Pretrial Practices

  6. We don’t need risk assessments in order to set policies that decarcerate our jails, to release the vast majority of people pretrial, and to support those people who need non-punitive, non-surveilling, free help to come safely back to court. Each accused person deserves a strong individual hearing based on their actual circumstances and needs — not a dataset trained on other people’s cases.
    • Despite claims to the contrary, there is no clear evidence that using these tools actually leads to substantive decarceration or a reduction in racial disparities in jails, and many other reforms and interventions exist that can keep communities safe and reduce pretrial detention.21David G. Robinson and Logan Koepke: Civil Rights and Pretrial Risk Assessments, Upturn Inc.

See a detailed breakdown of key issues with algorithmic decision-making in this Statement of Concern on pretrial risk assessments released by the Leadership Conference on Civil and Human Rights and over 100 community and national organizations.22Six Things to Know About Algorithm-Based Decision-Making Tools

Read more about how bias operates in pretrial RATs in The Danger section of our site.

The goal of our research is to increase the community’s understanding of how these tools influence pretrial detention decisions, especially communities engaging in campaigns or wrestling with potential changes to their pretrial legal systems. We are particularly focused on whether these tools reinforce pre-existing racial and socioeconomic biases in these systems.