Accounting for Individual Differences Among Fingerprint Examiners Using Tree-Based Item Response Theory by Amanda Luby, Carnegie Mellon University

Accounting for Individual Differences Among Fingerprint Examiners Using Tree-Based Item Response Theory by Amanda Luby, Carnegie Mellon University, at the Statistics Colloquium today, 1:10 – 1:50 pm, Stetson Court Classroom 101

Abstract: Fingerprint examination is perhaps the most well-known of the forensic science methods, yet remains one of the most subjective. Even as automated systems (e.g. IAFIS) become more prolific and accurate, final source determinations are left to individual forensic examiners. Given the same comparison task, different fingerprint examiners may come to different conclusions and/or use different decision criteria. There has been an influx of studies attempting to better estimate overall error rates as well as understand decision-making processes in forensic analyses. Item response theory (IRT) can provide valuable insight into these studies by accounting for both individual differences among examiners as well as comparison tasks of varying difficulty.

There are, however, fundamental differences between forensic science applications and traditional IRT settings. Namely, responses do not naturally conform to one of the standard response schemes because (a) each question is made up of series of structured responses, and (b) there is no keyed correct response. I use IRTrees, a tree-based IRT framework, to account for these differences. Using a tree structure, each sub-decision in a response is modeled with IRT, allowing for separation of decision-maker and item effects at each step in the decision-making process. I analyze the FBI “black box” study using the IRTrees framework, demonstrating the utility of such an approach in the forensic science domain.