RATs Don’t Solve Pretrial Injustice

Pretrial RATs and the decisions they prescribe can have an enormous impact on an accused person’s life, and the lives of their family and community.

All over the country, pretrial RATs help judges and magistrates decide if someone gets to come home before their trial.

In many places, the risk scores from pretrial RATs influence if a judge or magistrate decides to release or detain someone pretrial, what their bail will be or if the court will set a bail, or what kind of conditions or supervision requirements the court will require them to adhere to if released.

There is a lot of research showing that being detained pretrial is extremely harmful.

People who are detained are more likely to lose their jobs and future employment prospects,¹ as well as their homes, access to medication, and even their children.²

Those detained pretrial are also much more likely to be convicted³ and receive longer sentences than those who are released,⁴ often because they plead guilty and take a sentence just to get out of jail.⁵ Those detained pretrial are also more likely to be arrested again in the future.⁶

Those who are released are able to handle their cases in partnership with their lawyers, with the support of their families and communities.

As with most oppressive systems, the impacts fall primarily on people of color and poor people. That’s why we want to make sure that RATs helping determine if people are free or not before their trial do not uphold and reproduce the same ugly biases already embedded in our criminal legal system.

As noted, some decision-makers propose RATs as the necessary step in removing money bail. We reject the idea that money bail, a system that targets and harms poor people and people of color, needs to be replaced with another system that does the same thing, especially as we see some jurisdictions expand their pretrial incarceration and fail to reduce racial disparities after implementing RATs.

Some of our communities’ main concerns with pretrial RATs include:

Many of these RATs have a disproportionate racial impact: they draw from racially biased data and racially biased systems, so they are likely to reproduce inequalities in their outcomes.
- The factors that feed into many of the pretrial RATs used across the country are proxies for race or class, and they often perpetuate racial and economic bias. For instance, many tools include factors like whether or not someone is employed or has stable housing in making their predictions, which are clearly connected to a person’s economic status.
- Scholars have even stated: “the problem of bias in criminal justice data is such a substantial, serious threat that it calls into question the entire endeavor of data-driven risk assessment.”⁷
There is a severe lack of clarity and transparency in jurisdictions around what training data goes into RATs and how they are used – especially when tools are proprietary or complex machine-learning algorithms.
- People charged with crimes have a right to know what tools are used to evaluate them and how these tools impact what happens to them. In many initial arraignment hearings, accused people lack basic court safeguards to protect their rights. ⁸
- At many arraignments, which happen in a matter of minutes, accused people don’t have a lawyer present. If a defense representative is there, they have likely not had a chance to talk to their accused client. The accused person does not generally have any chance to review a RAT score or the factors that went into it, or to contest a RAT score before an arraignment hearing.
- As we saw in our interviews, in some cases, decision-makers who use the results of RATs may or may not understand how the scores are calculated, or even know all the inputs that go into their tool. They won’t necessarily know what particular inputs pushed an overall score to be especially high or low, or what the result is meant to communicate to a decision-maker.
RATs provide decision-makers in the criminal legal system with a false sense of scientific fairness when making their decisions.
- No tool is created or used in a vacuum. The factors used, how they are weighed, and the definition of “risk” are all subject to human bias. Most risk assessments are tied to decision-making frameworks — sets of policy guidelines meant to interpret the results of risk assessment tools. Moving these decisions to a computer and a new decision-making framework merely bypasses or buries the question of human responsibility.
- Because RATs rely on criminal legal system data from the past few decades, many of which were collected before more recent pretrial reforms, they create what researchers have termed “zombie predictions“⁹ that overestimate risk. RATs also attempt to turn “dangerousness” into a measurable output,¹⁰ which some argue is an impossible task. RATs often have to choose between statistical accuracy and fairness in terms of equity for anyone being assessed¹¹ – a process that is not clear to those using or being judged by these tools.
- These tools are supposed to be “evidence-based,” meaning they are backed by some kind of data and research. But without actively questioning the biases baked into that data collection, one article argues, bias does not disappear; the use of “objective” methodologies does not undo structural inequality.¹²
- As one research team found, the statistical issues of fairness within risk assessments creates its own bias, but these tools also inherit the bias from the underlying data they use and the inherent bias from the way we predict individual behavior based on group models.¹³
Validation scores and statistics show that RATs are not necessarily highly accurate or predictive on the questions they are trying to answer, especially on questions of violence, which is arguably the result that decision-makers and communities are most concerned with predicting. In many places, the tools are not validated at all, haven’t been validated for years, or have not been validated on local systems or populations.
- RATs have a tendency to overpredict risk¹⁴ and overrepresent the likelihood and intentionality of their predicted outcomes: failure to appear, rearrest, and pretrial violence, which are actually fairly rare.¹⁵
Regardless of the input factors or intention of the developers, RATs are not always used as intended. Judges and magistrates have discretion over pretrial outcomes, and studies have shown that they often deviate from the tool’s recommendations to push for pretrial detention¹⁶Brandon Buskey and Andrea Woods: Making Sense of Pretrial Risk Assessments, The Champion and over-detain those who receive low scores.¹⁷Megan Stevenson: Assessing Risk Assessment in Action, George Mason University One scholar even argued that judges follow RAT recommendations when it offers rationalization for the judge to make the choice they wanted anyway.¹⁸Andrew Van Dam: Algorithms were supposed to make Virginia judges fairer. What happened was far more complicated, The Washington Post
- The Pretrial Justice Institute¹⁹ has argued that many jurisdictions use RATs to make set bond amounts and make detention and release decisions, which “raises legal concerns” because detention decisions should happen at a full hearing in front of a judge. Pretrial RATs were designed to set release conditions; only clear evidence that no release conditions could ensure the safety of the individual and the public should lead to detention. Pretrial detention, the Pretrial Justice Institute argues, “is not a decision that should be based simply on probable cause and an assessment score.”²⁰
We don’t need risk assessments in order to set policies that decarcerate our jails, to release the vast majority of people pretrial, and to support those people who need non-punitive, non-surveilling, free help to come safely back to court. Each accused person deserves a strong individual hearing based on their actual circumstances and needs — not a dataset trained on other people’s cases.
- Despite claims to the contrary, there is no clear evidence that using these tools actually leads to substantive decarceration or a reduction in racial disparities in jails, and many other reforms and interventions exist that can keep communities safe and reduce pretrial detention.²¹

See a detailed breakdown of key issues with algorithmic decision-making in this Statement of Concern on pretrial risk assessments released by the Leadership Conference on Civil and Human Rights and over 100 community and national organizations.²²

Read more about how bias operates in pretrial RATs in The Danger section of our site.

The goal of our research is to increase the community’s understanding of how these tools influence pretrial detention decisions, especially communities engaging in campaigns or wrestling with potential changes to their pretrial legal systems. We are particularly focused on whether these tools reinforce pre-existing racial and socioeconomic biases in these systems.