Abstract
Excerpted From: David K. Hausman, Risk Assessment as Policy in Immigration Detention Decisions, 68 Journal of Law & Economics 103 (February, 2025) (6 Footnotes) (Full Document)
How do judges and other decision-makers incorporate the recommendations of algorithmic tools when making bail decisions? A large and fast-growing literature examines this question in the criminal justice context, where judges are the decision-makers. This article considers the same question in the immigration detention context. Immigration and Customs Enforcement (ICE) officers' decisions carry stakes similar to those in the criminal bail context. Although ICE officers lack the prestige and independence of judges, they, like judges making bail decisions, consider flight risk and danger to decide whether to detain or release someone.
On June 5, 2017, ICE altered its risk-assessment algorithm. Before the change, the tool could recommend release, detention, or referral to a supervisor. After the change, the tool could recommend only detention or referral to a supervisor; release was no longer a possible recommendation. When ICE removed the release possibility, it also changed the tool so that it referred cases to a supervisor two-thirds less often. After these changes, the tool began recommending detention in more than 9 of 10 cases. In other words, ICE made a policy decision to increase the use of detention, and it implemented that policy change through a change to its risk-assessment software.
I find, using a regression discontinuity in time design, that ICE's change to its risk-assessment tool decreased noncitizens' chance of release (including both release outright and release on bond) by about half, from around 10 percent to around 5 percent. Officers and supervisors continued to rely on the tool's recommendations after the tool stopped recommending release: The probability that an officer would override the tool's recommendation of detention increased only slightly after June 5, 2017, even as detention became a much more common recommendation. And supervisors counteracted officers' slightly increased frequency of disagreement with the tool, becoming more likely to overrule officers' release decisions.
These findings advance the growing literature on human-algorithmic decision-making in the context of government detention. A key problem identified by the human-machine decision-making literature is that decision-makers often discount algorithms' predictions and, in the bail context, choose to set a high bond (or no bond) even when the algorithm predicts little risk--particularly when the defendant is Black. This problem persists despite evidence that algorithms may outperform humans' predictive judgments in many contexts, including predictions about flight risk and recidivism in the bail context.
The results in the present paper, by contrast, suggest that lower level officials, at least, may do little to calibrate their reliance on algorithms' predictions and recommendations. As noted above, ICE officers and supervisors continued to follow the algorithm's recommendations even as release recommendations ceased to exist. These results match those of Albrigh, who finds that humans respond strongly to changed algorithmic recommendations.
More broadly, however, ICE officials were far more likely to follow algorithmic recommendations--when those recommendations indicated detention--than judges considering criminal bail. Officers overrode the release recommendation (before it was eliminated) well over half the time but virtually never overrode the detention recommendations by the risk-classification assessment (RCA). By contrast, while judges' rates of overriding risk assessments in the criminal context have varied--for example, judges overrode 12 percent of release recommendations and over half of detention recommendations, and judges overrode 57 percent of recommendations for diversion and 27 percent of recommendations against diversion in Stevenson and Doleac-I am not aware of any context in which judges virtually never overrode a risk-assessment tool's detention recommendation. The near-complete unwillingness by ICE officers to override the RCA tool's recommendation in favor of release is therefore notable. That strong pattern might reflect ICE officers' lack of decisional independence.
These results also illustrate that, for the same reasons that algorithmic recommendations may help influence decision-makers to improve predictions, such recommendations may accomplish nontransparent administrative policy change. The effectiveness of policy change by algorithm fits the growing body of evidence demonstrating that line-level law enforcement officers are sensitive to the incentives set for them by their supervisors. Mummolo shows that a procedural change in the New York City Police Department-- requiring additional documentation for street stops--decreased the number of those stops and increased the rate at which searches yielded contraband; Ba and Rivera show that police union memos in Chicago reduced complaints against the police; Mas finds that when police pay declines as a result of a union arbitration loss, arrest rates fall and crime rates rise. Like police supervisors, ICE management was able to change officers' behavior: Decision-makers “did not calibrate their reliance on the risk assessment based on the risk assessment's performance.
Finally, these findings also add to the small existing empirical literature on algorithmic decision-making at ICE. That literature has not examined the causal effect of the 2017 shift. Koulish offers the first quantitative overview of the risk-assessment tool, and Noferi and Koulish examine an earlier version of the RCA data used in the present article and conclude that ICE engages in significant overuse of detention. Koulish and Calvo examine the contextual determinants of ICE officers' decisions to override the RCA tool's recommendations. Koulish determines that noncitizens who are mandatorily detained are no more likely to pose a significant risk than those whom ICE has discretion to release. Evans and Koulish and Koulish and Evans document the many versions of the ICE risk-assessment tool and the determinants of its predictions. I add to this body of work by evaluating, for the first time, the consequences for release rates of the 2017 change to the risk-assessment tool. As scholars and policymakers work to improve algorithmic decision-making tools, they should bear in mind the likely effects of nontransparent changes.
[. . .]
As scholarly interest in risk-assessment tools in the criminal bail and sentencing contexts has grown, the similar but distinct immigration detention context has received less attention. When ICE changed its risk-assessment tool for detention decisions on June 5, 2017, it entirely removed the possibility that the tool would recommend release. After the change, the tool could recommend only detention or referral to a supervisor. Officers became only slightly less like to follow the recommendation, and detention therefore became more common. In other words, the software change accomplished a policy change.
The ICE detention decision-making context differs from the typical criminal bail context: Independent judges set criminal bail, whereas ICE officers, who answer to their supervisors, make detention decisions for noncitizens. The hierarchical ICE context might have lent additional weight to the tool's changing recommendations. But these results are consistent with those of Albright, who finds that changes to algorithmic recommendations can lead to large real-world changes in the criminal bail context. One possible normative implication is that such policy changes, like nonalgorithmic policy changes, should be made transparently.
These results also have limitations. Because the ICE dataset lacks demographic information, I am unable to examine the relative effects of the change by nationality, race, or demographic characteristics. And I am also unable to measure the relative importance of the changes to the flight-risk scores and the changes to the detention recommendations themselves. This article does, however, present strong evidence that the 2017 changes to ICE's risk-assessment software had real-world effects that led officers to detain noncitizens more often.
