Pretrial RATs are used in different parts of the pretrial process, depending on jurisdiction and local policies and practices. Tools are not always used as intended, and judges and magistrates do not always follow the tools’ recommendations.
How Assessments are Used
Pretrial RATs come into play in various parts of the pretrial process.
Most of the time, an accused person is assessed after they are arrested and charged, but before their first appearance in front of a judge or magistrate. Risk levels generated from the RAT may be presented alongside a specific pretrial recommendation, often generated from a decision-making framework, which helps judges or magistrates decide whether or not the accused will be released pretrial or what bail or conditions they may receive.
For example, our interviewee from Anne Arundel County, Maryland, specified that RAT scores help determine if the accused individual meets criteria for release, conditional release, or continued detention.1Interview with Movement Alliance Project, 8/13/2019. See Interview Summary for more information
And in Contra Costa, California, our interviewee told us that a low risk level designation means release on recognizance. But below average, average, and above average levels mean increasing check-ins and more intensive pretrial supervision, and a high risk level designation means either release is not recommended or electronic monitoring is recommended if the person is released.2Interview with Movement Alliance Project, 7/13/2017. See Interview Summary for more information
In several jurisdictions we interviewed, a low RAT score automatically leads to or recommends release. Some interviewees described the RAT as a check on the power of judges’ and magistrates’ decision-making, especially when low scores lead to automatic release or give defense attorneys the opportunity to argue for release.
According to our interview, in Cass County, Minnesota, judges grant release on recognizance to those with a low-risk designation.3Interview with Media Mobilizing Project, 7/18/2019. See Interview Summary for more informationIn Los Angeles, probation staff conduct the assessment and recommend release for scores below 15, on a scale that goes up to 99.4Interview with Movement Alliance Project, 4/7/2017. See Interview Summary for more information
For some jurisdictions we interviewed, a high RAT score leads to an additional hearing. In Napa County, California, according to our interviewee, judges make release decisions based on risk scores prior to an arraignment, and those who get high risk scores receive a formal arraignment.5Interview with Movement Alliance Project, 7/11/2017. See Interview Summary for more information
According to our interview with Hennepin, Minnesota, the results of their RAT connect to both automatic release and hearings as appropriate. The pretrial unit is able to release those who receive low or moderate risk levels without judicial review: low risk leads directly to release, and moderate risk leads to release with conditions. A high risk level leads to a pretrial hearing and the judge ultimately makes a decision about bail.6Interview with Movement Alliance Project, 11/3/2017. See Interview Summary for more information
However, RATs are used in many jurisdictions at other times or for other purposes besides guiding the initial arraignment decision after arrest and charging and an initial holding period. RATs may be used after the first appearance, upon entrance into a pretrial service program, or on a voluntary basis to receive additional services. In several places, RATs are used for those who could not make bail, as a secondary opportunity for pretrial release.
Our interviewee stated that Shasta County, California, uses the VPRAI to place individuals who receive high scores on a supervision program, often with regular check-ins and possibly electronic monitoring.7Interview with Movement Alliance Project, 11/13/2018. See Interview Summary for more information
In Virginia, according to one researcher, the VPRAI-R is often not utilized in the pretrial process until after an initial decision is made to release or detain, as seen below.
Image generated by Stacey Feindt, University of Mary Washington, Student Research8Stacey Feindt: Detained by Data: A Critical Analysis of the Virginia Pretrial Risk Assessment Instrument, University of Mary Washington
Some jurisdictions use a risk/needs tool that might help produce a risk level but are also supposed to predict what kind of services or supports an individual might need. Some of these RATs are designed for post-conviction purposes, and are used to determine the type of pretrial supervision or programming someone might receive, rather than whether or not they should be released. These tools may be administered upon entrance to jail or pretrial service programs.
For example, Somerset, Maine, uses the LSI-R, a tool not calibrated for pretrial purposes,9Cynthia A. Mamalian: State of the Science of Pretrial Risk Assessment, Pretrial Justice Institute once an individual has been placed in a pretrial program to help determine the level of supervision required.
Some RATs have been in place for decades, while others are newly introduced. Around the country, tools may be well-incorporated into the pretrial process, or jurisdictions may be launching a new pilot program to test out a tool.
The RAT only recommends a pretrial outcome. Most interviewees stressed that the RAT does not take the place of a judge or magistrate in ultimate decision making. The RAT’s designation of high or low risk can have a huge impact on a judge or magistrate’s impression – or none at all, as seen below.
Tool Misuse
Many tools were developed for a particular purpose and are being used in another manner. This is more likely especially when a jurisdiction, such as a county, adopts a national tool like the PSA without actually validating it for local conditions.
Some proponents of RATs claim they can be decarceral, but many tools are not actually being implemented in a way that reduces jail populations. Some tools may never have been designed or deployed to reduce jail populations, especially those that have been in place for decades – contrary to the popular narrative around RATs.
Several jurisdictions use a tool designed for post-adjudication in a pretrial setting, such as the LSI-R, which some argue is inappropriate because these tools use factors that are not linked to predicting pretrial risk.10Cynthia A. Mamalian: State of the Science of Pretrial Risk Assessment, Pretrial Justice Institute
Plus, judges and magistrates might not even use the tools as intended.
The Pretrial Justice Institute’s 2019 Scan of Pretrial Practices11Pretrial Justice Institute: Scan of Pretrial Practices surveyed 91 jurisdictions and found that 73% of them were using a RAT to make decisions about release or detention – a use of the tool that they said “raises legal concerns” because detention decisions should be made through individual hearings that give due process and prove that no conditions of release would protect public safety.
Megan Stevenson’s research on Kentucky’s use of the PSA showed that many judges disregarded release recommendations from the tool and detained supposedly low-risk people anyway.12Megan Stevenson: Assessing Risk Assessment in Action, George Mason University
Stevenson pointed out that if judges followed risk assessment recommendations, 90% of defendants should be granted immediate non-financial release, while in reality only 29% of defendants receive this outcome.
Independent research on algorithms used in courtrooms found that algorithms like RATs are often not used as intended, and legal professionals ignored or resisted RATs in favor of their own judgement.13Angèle Christin: Algorithms in practice: Comparing web journalism and criminal justice, Big Data & Society
Evidence from early RAT usage showed that in some cases, judges were more likely to override recommendations for release than those for detention,14Brandon Buskey and Andrea Woods: Making Sense of Pretrial Risk Assessments, The Champion meaning judges only go against the tool’s recommendation when it means pushing for more pretrial detention, rather than less.
One study of RAT use in sentencing found that risk assessments increased the likelihood of incarceration for poorer defendants while decreasing the likelihood of incarceration for wealthier defendants. The researchers concluded that “under some circumstances, risk assessment information can increase sentencing disparities”15Jennifer L. Skeem, Nicholas Scurich, and John Monahan: Impact of Risk Assessment on Judges’ Fairness in Sentencing Relatively Poor Defendants, Virginia Public Law and Legal Theory Research Paper No. 2019-02 – which is certainly contrary to the aim of decreasing bias in the criminal legal system and could happen with pretrial RATs as well.
A 2019 report on Virginia’s sentencing tool found that judges were regularly using their own judgement to deviate from the tool’s recommendations.16Megan T. Stevenson and Jennifer L. Doleac: Algorithmic Risk Assessment in the Hands of Humans, SSRN Another researcher studying Virginia found that judges followed the sentencing tool’s suggestions less than half the time and that judges did not trust the RAT or were not trained properly in how to use it.17Andrew Van Dam: Algorithms were supposed to make Virginia judges fairer. What happened was far more complicated, The Washington Post
This scholar even suggested that judges tend to follow RAT’s recommendations when RATs provide a rationalization for the judge to make the decision they wanted to make anyway.18Andrew Van Dam: Algorithms were supposed to make Virginia judges fairer. What happened was far more complicated, The Washington Post
Jurisdictions also differ in terms of how easy or difficult it is for judges and magistrates to deviate from a tool’s recommendation. In some places, it is very easy for a judge or magistrate to ignore a RAT score, while in others it requires a complex override process.
Most of the jurisdictions we asked did not have any data on how often their judges and magistrates followed the recommendation from a RAT. Some jurisdictions stated that judges or courts followed recommendations, but only Dekalb County, Illinois, Polk County, Iowa, and Douglas County, Kansas had statistics to back this claim up.
Douglas County19Interview with Movement Alliance Project, 7/18/2019. See Interview Summary for more information said that judges follow pretrial service recommendations 93% of the time, and Polk County20Email communication with Media Mobilizing Project, 9/18/2018. gave data for the month of August 2019, when judges followed release recommendations at the initial appearance hearing for 76% of cases. Dekalb County21Interview with Movement Alliance Project, 9/15/2017. See Interview Summary for more information stated that judges only follow recommendations 80% of the time – the other 20% was “overwhelmingly cases in which judge decides to detain arrestee in spite of recommendation for supervised release.”
The rest did not have any data about how often judges follow RAT recommended outcomes.
The Pretrial Justice Institute’s 2019 Scan of Pretrial Practices22Pretrial Justice Institute: Scan of Pretrial Practices report also found, of the jurisdictions they interviewed that use decision-making frameworks, only 52% actually track how often judges’ release decisions match framework’s recommendations.
Our interviews indicated that some pretrial service agencies use tools to keep who they feel are the “right” people in jail, rather than to decarcerate, which can mean keeping individuals whom the tools indicate are “dangerous” behind bars. Pretrial services may also use RATs in other ways contrary to the tools’ intended purposes.
