Criminal Justice
Research & Evaluation
Assessment of Risks and Needs
For the past 10 years, my research team has conducted research on how to assess risks and needs (mental health and substance use) in a criminal justice context, as well as response to intervention (called “responsivity” in the criminal justice literature).
Below you will see the dissertations my students conducted evaluating risk, needs, and response to intervention. Curtis (2018) and Meeks (2020) both studied risk, Gerber (2022) studied needs (specifically substance use), Bell (2020) studied response to intervention (“responsivity”), and Cochran (2018) studied how to conduct cost-benefit studies in criminal justice settings. Chroback (2022) is completing her dissertation studying violence risk and substance use, which she will defend in early August.
In addition to those dissertations, I will summarize other unpublished research I have led in criminal justice risk, needs, and responsivity, and what I see as current best practices and future directions.
Since my dissertation in 1998 I have studied the assessment of criminal risk, with a particular focus on risk of violence. In 1996 I was the Principal Investigator (PI) in a series of validation studies of the Texas Risk Assessment System (TRAS), and in 2021 the Texas Department of Criminal Justice (TDCJ) asked me to specifically study sexual violence and the utility of continuing to use the assessment of sexual violence risk vs. creating a new assessment using modern statistical modeling techniques (sometimes called data mining or predictive modeling or machine learning). [I prefer the more general term “statistical modeling,” but because “machine learning” has gained the most acceptance, I will use that term to represent statistical techniques which are used to predict a specific outcome—in this case criminal behavior.]
Because I have studied assessment of risk the most, I will focus this first blog post on what we know about assessing risk of felony arrest, violence, and sexual violence:
Using actuarial assessment tools such as the TRAS, Level of Supervision Inventory – Revised (LSI-R) or any of a handful of other similar instruments is more accurate than using structured or unstructured human judgment to assess criminal risk.
The most accurate and least biased risk assessment tools are created using machine learning algorithms on large, representative samples. These algorithms accurately identify important predictors/items as well as more accurately weight these predictors/items in order to assess risk. See Curtis (2018) and Meeks (2020) below for detailed, technical explanations.
For general recidivism (such as arrest or conviction for a felony), actuarial assessment tools are able to accurately predict at roughly 60% accuracy, while our research shows that machine-learning-based assessments can accurately identify over 80%. See reports from the TRAS Validation Studies for more information.
For risk of violence, the discrepancy in accuracy between actuarial assessments and machine-learning-based assessments, is larger because actuarial assessments are less accurate predicting violence (due to a lower base rate of violence relative to felony arrest). See TRAS reports, Curtis (2018) and Meeks (2020).
For risk of sexual violence, the discrepancy is even wider (again because the base rate of sexual violence is lower than for “general” violence). For example, our latest research shows that actuarial tools used specifically to identify sexual violence have a 20% accuracy rate, while our machine-learning-based assessments have approximately 85% accuracy. See the TDCJ Sex Offender Study for more details.
In addition to substantial accuracy benefits, machine-learning-based assessments have the added advantage of being less biased for both race and gender across all types of criminal risk (i.e., felony arrest, violence, sexual violence), and because of its operational efficiencies machine-learning-based risk can be conducted quickly and cheaply (and thus more frequently).
In summary, current best practices for assessing criminal risk for felony arrest, violence, and sexual violence all involve creating a machine-learning-based risk assessment. These machine-learning-based approaches are more accurate, less biased, fast to complete, and cheaper than traditional actuarial risk assessments such as TRAS or LSI-R.
— Dr. Eugene Wang
June 2022
Dissertations
“Numerous studies have been conducted over the last twenty years investigating the effectiveness of cognitive intervention programs on reducing reconviction rates among offenders. Whereas cognitive intervention programs have demonstrated promising results, there is a gap in the literature surrounding who would be most responsive. Knowing what type of offender would most benefit from attending this program could potentially reduce recidivism by providing the proper rehabilitation programming to offenders before they are released.”
Bell, B. K. (2020). Responsivity in rehabilitation programming for offenders [Doctoral dissertation, Texas Tech University]. TTU DSpace. https://hdl.handle.net/2346/86565
Abstract coming soon!
Chroback, K. (2022). The effects of substance use on violence [Doctoral dissertation, Texas Tech University].
“This study is an examination of the effectiveness and cost-effectiveness of correctional Career and Technical Education (CTE) programs administered by the Windham School District (WSD) within the Texas Department of Criminal Justice (TDCJ). The study focuses on offenders who took CTE courses in the 2014–2015 school year through the first half of the 2015–2016 school year and were subsequently released into their communities on parole. The measures of effectiveness included recidivism (measured as re-incarceration within TDCJ) and employment (measured as being employed for at least one day during the observation period).”
Cochran, D. B. (2018). Cost–Benefit Analysis of the Windham School District’s Correctional CTE Program [Doctoral dissertation, Texas Tech University]. TTU DSpace. https://hdl.handle.net/2346/86823
“Parole board members use predictions of an offender’s risk of recidivating to inform early release decisions. As of yet, we have been only moderately accurate in our attempts to predict recidivism. Machine learning has the potential to increase our predictive ability. Despite this, machine learning methods are relatively unexplored by criminal justice researchers. The goal of this dissertation was to provide recommendations about which combination of modern machine learning method (random forests or Extreme Gradient Boosting) and set of predictors is optimal for predicting recidivism at parole hearings.”
Curtis, J. (2018). On using machine learning to predict recidivism [Doctoral dissertation, Texas Tech University]. TTU DSpace. http://hdl.handle.net/2346/73945
“Drug use has been shown to have an effect on recidivism, but the extent of that effect, the types of drugs that produce that effect, and what types of crimes are being affected has not been addressed. This study attempted to answer some of those questions. The purpose of this dissertation is dissertation is to determine the effect drug type and intensity has on drug crime and property crime recidivism. Also, thresholds for identifying criminogenic need of substance use were explored.”
Gerber, W. (2022). The Effects of Substance Type and Use on Public Safety: A Machine Learning Analysis [Doctoral dissertation, Texas Tech University]. TTU DSpace. Template: Front Matter (tdl.org)
“The current study will evaluate the use of machine learning as a form of risk assessment as it fits within the risk-need-responsivity framework. Specifically, the risk of violent reconviction will attempt to be predicted by multiple machine learning algorithms. As violent reconviction has significant class imbalance, as well as asymmetric error cost, methodologies accounting for these potentially problematic situations will be evaluated.”
Meeks, S. F. (2020). Evaluating class imbalance and asymmetric costs using machine learning [Doctoral dissertation, Texas Tech University]. TTU DSpace. https://hdl.handle.net/2346/86573
Published Articles
“Over the last decade state prisons have experienced unprecedented growth and many demographic changes. At the same time, courts are requiring states to provide mental health screening and treatment to prisoners. Findings from recent studies indicate that the prevalence of mental illness is higher in prisons than in the community, and comorbidity is common. Our ability to generalize from these studies is limited, however, because of major shifts in the demographic mix in prisons during the past decade. New studies on the prevalence of mental illness in prisons, which consider these recent changes would help planners allocate funds and staff to more effectively meet the needs of these individuals.“
The prevalence of mental illness in prison: Review and policy implications. Administration & Policy in Mental Health, 29(1), 21-40. Diamond, P. M., Wang, E. W., *Smith, J. L., Holzer, C. E. III, Cruser, d. A., & Thomas, C. R. (2001).
“In 1990, the Texas Department of Criminal Justice instituted a specialty aggression treatment program called the Program for the Aggressive Mentally Ill Offender (PAMIO). It has beds for 458 violent male offenders and offers multidisciplinary medical and behavioral treatment. This study examined the effectiveness of this treatment program by comparing the annual rates of disciplinary offenses (e.g., staff assaults, inmate assaults, and good time lost) of 66 offenders prior to and following treatment. Paired t tests showed a significant decrease after PAMIO treatment in the annual rates of total disciplinaries, staff assaults, inmate assaults, and good time lost. This improvement appeared to be directly related to treatment effects.“
The effectiveness of rehabilitation for persistently violent male prisoners. International Journal of Offender Therapy and Comparative Criminology, 44(4), 505-514. Wang, E. W., Owens, R. M., *Long, S. A., Diamond, P. M., & *Smith, J. L. (2000).
“The authors used structural modeling to predict institutional aggression among male mentally ill offenders using the predictors of anger, antisocial personality style, current violent offense, ethnicity, and impulsivity. Measures included the Barratt Impulsiveness Scale, the Buss-Perry Aggression Questionnaire, the Personality Assessment Inventory, age, ethnicity, current violent offense, victim injury from current offense, and institutional incidents of physical and verbal aggression. The model fit the data, and accounted for 94% and 87% of the variance of physical and verbal aggression, respectively. Results indicated anger, antisocial personality style, and impulsivity are stronger predictors of institutional aggression than are ethnicity and current violent offense; anger was the best predictor. Results suggest dynamic variables such as anger can be targeted for clinical intervention to reduce institutional violence.“
Empirically identifying factors related to violence risk in corrections. Behavioral Sciences and the Law, 17, 377-389. Wang, E. W., & Diamond, P. M. (1999).
“Indicators of feigned PAI profiles were derived from comparisons of simulators instructed to feign and genuine patient groups. Concerns are raised regarding whether these indicators are applicable to forensic and correctional populations and can be cross validated with a known-groups comparison. Compiling data on 57 malingerers and 58 genuine patients from two forensic and correctional sites, three primary indicators of feigning, Negative Impression (NIM) scale, Malingering Index, and the Rogers Discriminant Function (RDF) were investigated. Results suggested that the RDF was not applicable to forensic referrals. However, NIM 2 77T appeared to be a useful screen for forensic samples. In addition, convergent evidence of feigning was found across designs (simulation and known-groups) and samples (non-forensic and forensic) for extreme elevations on NIM (> 1101) and Malingering Index (? 5).“
The PAI and feigning: A cautionary note on its use in forensic-correctional settings. Assessment, 5, 399-405. Rogers, R., Sewell, K. W., Cruise, K. R., Wang, E. W., & Ustad, K. L. (1998). doi:10.1177/107319119800500409.
“Examined the usefulness of the PAI (L. C. Morey, 1996), in assessing problematic behaviors in a corrections-based psychiatric hospital. As evidence of criterion related validity, selected PAI scales were compared to (1) evidence of malingering on the Structured Interview of Reported Symptoms (R. Rogers, et al 1992), (2) suicidal threats and gestures, and (3) ratings of aggression on the Overt Aggression Scale (S. C. Yudofsky, et al 1986). Clinical records were reviewed systematically for the time period of September 1, 1996–February 10, 1997. All records containing PAI protocols were examined in relation to other psychometric measures. In general, results support the use of the PAI for the assessment of these problematic behaviors. The PAI scales, subscales, and derived scales are appended. (PsycINFO Database Record (c) 2016 APA, all rights reserved).“
A pilot study of the Personality Assessment Inventory (PAI) in corrections: Assessment of malingering, suicide risk, and aggression in male inmates. Behavioral Sciences & the Law, 15, 1-14. Wang, E. W., Rogers, R., Giles, C. L., Diamond, P. M., Herrington-Wang, L. E., & Taylor, E. R. (1997).
Reports
2019
“We looked at whether or not the offender got employed, how long it took, whether the individual was employed one year after gaining initial employment, and whether or not the individual was rearrested or reincarcerated within three (3) years post-release.”
2017
“The bulk of this report evaluates WSD academic, career and technical education (CTE), and life skills programs against these outcomes: academic achievement gains, HSED and industry certifications, and reentry outcomes.”
2015
“S.B. 213 Sec. 19.0041, titled “Program Data Collection and Biennial Evaluation and Report”, requires Windham School District (WSD) to compile and analyze information to determine whether its programs are meeting its goals, to make changes to the programs as necessary, and to submit a report to the Board, the Legislature, and the Governor’s office.”