Human Examiner Vs Al in Firearm Toolmark Comparison

Abstract

Forensic examination of firearms is evolving from microscope comparison towards 3D imaging, computer scoring, and artificial intelligence technologies such as convolutional neural networks (CNNs). The review found that Al tools perform highly accurately in certain applications. Specifically, CNN models for classification of firing pin impressions scored 82.6 92.8% accuracy in controlled tests, while machine learning models for bullet marks showed about 84-86% accuracy. On the other hand, "black box" research shows high, albeit imperfect, accuracy of trained human examiners. In one study 173 qualified examiners completed 8,640 comparisons of bullet and cartridge case marks with extremely low false positive rates (about 0.66-0.93%) and low false negative rates (about 1.87-2.87%).
The review found, Al and humans demonstrate complementary capabilities. Human experts were more capable than computer tools in correctly excluding different source comparisons, while algorithms were better in confirmation of same source comparisons. Examiners performed better in difficult situations of overlapped marks, where some algorithms had difficulties. It appears that Al technologies should assist examiners but not replace them.
On the other hand, there are several important limitations which we have discussed in this review paper, including: Studies based mostly on limited, controlled test sets; hence the performance may decrease in case of new gun models, calibres, mark types etc., that are not included in training. Problems of trustworthiness, explainability, and integration of Al in forensic work remain open. Also, there are some studies based on non-experts or limited fragments of images, which may artificially increase accuracy. We believe that Al should be used as a decision support tool by experienced examiners. Further research should use open accessible datasets and involve practicing forensic examiners.