I. Technical Goals – Achievements and Failures
II. Hardware Development Summary
III. Characterization of Spectral Properties of Latent Blood
IV. Detection of Latent Finger Prints
V. Impact of the project and Recommendations Concerning Further Technology Development
The goal was to develop a compact, lightweight camera which would be able to simultaneous capture 20 wavelengths from the visible to the near infrared (400-1000nm) at video rates. The proposed approach focused on the camera development on the use of commercially available “monochrome” CMOS detectors. We used commercial-off-the-shelf (COTS) silicon CMOS detector arrays which will enable spectral detection throughout this range. The use of all COTS components during the hardware development enabled us to minimize the cost of development as we optimize the optical design, minimize size, weight and power requirements, and develop the software interface.
The overall goals from the original proposal were: Year 1 – Design, develop, and construct a prototype multispectral camera and illuminator. Year 2 – Demonstrate the utility of the camera system on latent blood stains and finger prints. Year 3 – (Option) Design and fabricate miniaturized packaging of optical components and electronics into a field deployable hand-held unit. Details for Year 3 are discussed in Section V.
Below we list some of the technical goals and metrics for the multispectral camera as discussed in the approved application as well as their final status to assess technical success.
The main technical issue in Year 2 was to demonstrate the utility of the camera system on latent blood stains and fingerprints. Multispectral images were acquired under various conditions for various surfaces (eg. dark cloth, leather, paint):
In the original proposal, a new optical approach (Spectral Imaging Cube (SI-Cube)) was proposed to separate incoming white light images into multiple spectral mages. However, in the course of developing this design and applying it to our needs for a multispectral camera, several technical shortcoming of this approach became apparent. In summary, the restrictions imposed by the SI-Cube design (eg. restricted field of view, limited spectral tuning, difficulty in aligning relay mirrors) could only be rectified by increasing the physical size of the SI-Cube. However, if one increases the size of the SI-Cube to accommodate improved imaging, then one could use alternative optical designs. Rather than pursuing the SI-Cube option, we instead focused on both the dichroic camera and grid camera approaches which should provide similar performance to what was originally anticipated for the SI-Cube.
Figure 1: (a) A set of targets including leather (far left of middle row) and various patterned cloth upholstery stained with pig blood. The white disks in the image are used to calibrate the spectral content of the illumination source. Image acquired with a cell phone camera. (b) False color RGB image which retains the Blue (420 nm) and Green (500 nm) channels but replaces the Red channel with IR spectral data at 780nm. The contrast in the blood stains relative to the black shirt and patterned materials improves relative to ‘normal’ RGB images.
As an example, we created a ‘crime scene’ of various targets in Figure 1a with patterned and/or textured materials including leather and various cloth upholstery samples. Each target was stained with pig blood. The spectral data of this set of targets was acquired with the 8 spectral channel dichroic camera. Using our very basic analysis of replacing the ‘red’ channel with the infrared data from 780nm in an RGB false color image, one can improve the contrast in the blood stains on many of the fabrics as seen in Figure 1b. For example, the blood stain on the black shirt, which is invisible in Figure 1a, is now readily apparent in Figure 1b. Moreover, the blood stains in many of the patterned materials which are harder to visualize in RGB are readily apparent in Figure 1b.
Further exploration to determine what the spectral limitations in detecting blood stains on a variety of substrates yields interesting conclusions concerning the ability of multispectral imaging to detect latent blood stains. When the blood stains are on white or off-white substrates, there is sufficient ‘color difference’ between the substrate and blood stain to identify not only the presence of the stain, but also spectroscopically identify the stain as blood. As one would expect, the difficulty in identifying a stain as blood arises on darkly colored substrates. The central point is that ‘difficult to see stains’ result from the fact that the spectra of the substrate and the blood are similar. The technical challenge then is to de-convolve the ‘blood’ feature from the substrate spectral feature [1, 2].
For ‘hard to see (visible light)’ stains on dark clothing, the magnitude of the infrared reflection from blood is significantly different from the reflectivity of the substrate fabrics so that one can use the infrared reflectance to improve the contrast in the images to visualize the presence of the stain on the fabric. However, the reflection spectra of blood in the near-infrared range from 700- 1000nm only varies slowly with no distinguishing features: While multispectral imaging of blood stains in the 700-1000nm range can be used to detect the presence of a stain, there are not any distinguishing spectral features which enable identification of the stain as blood. Unless the IR spectra extends beyond ~1600nm, there are not any spectral features in the 700-1000nm near-IR spectral range in which our multispectral system operates which can identify the chemical composition of the stain as a blood stain (this is the same conclusion of Refs. [1, 2]).
Using the multispectral camera, we were able to demonstrate video-rate ‘stain detection’. The complete videos are included as electronic attachments to the final report. The person in the image has a blood stain on his black tee shirt just to the right of the armpit area. Again we emphasize that these video demonstrates the ability of the multispectral camera (a) to function in a ‘stain detection’ mode (b) to acquire video-rate multispectral images and (c) survey a large area in a prospective crime scene for evidence of stains. In this example, due to the video-rate acquisition, the stain detection can be performed on a moving test subject.
The results with our multispectral camera on latent fingerprints exhibit some successes, some failures, and some inconclusive results. One success is that the spatial resolution/ pixel resolution of the monochrome cameras have more than enough spatial/ pixel resolution such that the images can be compared using computer software to a library of fingerprints.
The failure in the multispectral methodology for fingerprints is generally there is no advantage in synthesizing a multispectral (400-1000nm) false color image in our attempts to achieve better contrast in identify fingerprints. The first limitation is that optimal contrast depends strongly on the angle of illumination of the target relative to the background surface. This limitation renders our envisioned modality of a ‘quick survey’ of a large area to identify the presence of latent prints problematic. More importantly, since the reflectivity spectra of fingerprints is essentially featureless in the 400-1000nm range, the contrast is entirely predicated on the spectral content of the background material. Unfortunately, since the ‘color’ of the background material is highly variable, one can not a priori pick a single color for which one achieves the best contrast. For some backgrounds, a single monochromatic image may be ideal in improving the contrast between the fingerprint and the background substrate on which it is located. On other background materials, there is little difference in the choice of spectral color. The choice of the color to achieve the best contrast will be substrate dependent. Given the wide variety of background/ substrate materials and the need to adjust the illumination conditions to obtain the best contrast, multispectral imaging of fingerprints in the 400-1000nm does not appear to be a promising technology for crime scene investigation.
In our initial proposal, our work was inspired by previous multispectral camera development of Teledyne Scientific & Imaging (TSI) in 2012 [3, 4] : “… crime scene investigators are tasked with detecting and collecting physical evidence present at the scene, including body fluids, hairs, fibers and latent prints. These are often difficult to distinguish from the background and may present a formidable challenge to detect and identify.” Our view in the original proposal was that despite the clear value of multispectral detection capabilities, until now, multispectral technology has not been available in a form suitable for deployment to the crime scene: the technical challenges to be overcome are weight, cost, complexity, ruggedness, portability and user friendliness.
In comparing our multispectral camera to that which was developed by Teledyne, our multispectral camera shows several major improvements compared to Teledyne: (1) the utilization of 16 colors (ie. multispectral) acquired simultaneously from the visible into the near-infrared and processed into real-time images. Teledyne’s system had only a maximum of 6 colors, which are acquired sequentially and require after-the-fact processing. (2) acquiring the multispectral images at video-rates. (3) with the advance in monochrome cameras with increased sensitivity, we did not need to build a specialized multicolor illumination source as was done by Teledyne, but instead could use inexpensive halogen lamps (4) the camera design ensures a small SWaP camera.
We found that our work on utilizing multiple cameras simultaneously was at the fore front of technology. There appears to be a push by vendors to solve many of the computer/ camera problems which we encountered and eventually solved: For example, E-CON Systems (which supplies our cameras) and NVIDIA Jetson (one of vendors whose minicomputer we evaluated) has recently teamed up to stream 6 synchronized cameras on a single Jetson. Our technical assessment is that with the continued development in the integration of multiple cameras with minicomputers, a small SWAP, hand-held multispectral camera is achievable in the next phase.
The merit of the decision to further miniaturize our approach for multispectral imaging depends more on the utility of the multispectral approach to crime scene investigators in the field than on the technical feasibility of a small SWAP camera. Specifically, as detailed in the final report, spectroscopic identification of blood stains using the multispectral camera is not always possible due to the strong impact of the substrate’s color on the spectral analysis. However, the multispectral approach from 400-1000nm would make rapid ‘stain detection’ viable. Essentially the merit of further miniaturization depends on the answer to the following: Would the rapid (ie. video rate) survey of a crime scene for ‘stains’ which might possibly be blood (or semen or other stain of interest) be valuable to the crime scene investigator? While multispectral imaging of blood stains in the 700-1000nm range can be used to detect the presence of a stain, there are not any distinguishing spectral features which enable identification of the stain as blood. Essentially, if the blood stains are on light colored materials, the blood stains can be spectrally identified as blood. However, if the stains are on dark clothing, the multispectral forensic camera can detect the presence of a stain, but not positively identify the stain as blood.
In a field crime scene investigation, immediate feedback is required to quickly identify the presence and location of latent evidence. There is a huge practical advantage for efficiency in allocation of an investigator’s time and reduced cost in rapidly performing forensic imaging at the crime scene. This trend for rapid screening and investigative feedback can be accelerated by the deployment of our handheld multispectral camera. We envision two modes for our new forensic multispectral camera: Stain mode and Spectral Detection mode. In the ‘Stain’ mode, the camera system processes the multispectral data to enhance the contrast and identifies the presence of stains. While this mode does not identify the type of stain, we envision that this mode will enable the rapid processing of a crime scene to identify the location of stains for further analysis. Once the locations of stains in a crime scene have been identified, the camera system can also operate in a ‘spectroscopy’ mode to attempt to spectroscopically identify the composition of the stain and determine if it is a blood stain.
As described in Refs [1, 2], if one could acquire multispectral data in the 1600-2500nm range (rather than the 700-1000nm range of our instrument), one could spectroscopically identify blood stains even on dark clothing. In principle, our instrument design could be easily modified for the 1600-2500nm range. In that range, the predominate hardware changes would be the appropriate bandpass filters (which is straight forward) as well as monochrome cameras which are sensitive in this wavelength range. The issue for infrared cameras beyond 1600nm is one of cost and image format size. If as a rough bench mark, we use the parameters of a commercially sold 1600nm camera, we expect that there would be a significant increase in cost compared to our system. Infrared cameras (Edmund Optics) operating up to 1600nm cost approximately $2000 each: replacing each of our 16 monochrome cameras (cost about $200 each) with longer wavelength IR cameras would push the price tag of the multispectral camera system to a cost roughly ten times higher than an equivalent camera operating up to 1000nm. Moreover, the pixel size of the 1600nm cameras is only 768 x 494 pixels which is significantly less than the 5M format which we are currently using. The illumination source is not a problem. Halogen lamps have sufficient brightness in the 1600-2500nm range.
Lastly, consider the prospects for using our 400-1000nm multispectral camera for detection of latent evidence other than blood stains. If one were to use UV lights for illumination, our multispectral camera could easily detect the fluorescence of stains thereby enabling their detection. For example, stains of sweat, saliva and semen could be detected with our multispectral camera with a UV light source. Unfortunately, blood stains do not fluoresce. With regards to latent finger prints, due to the featureless spectrum of fingerprint oils in the visible to near-infrared range and the strong influence of the spectral content of the substrate material, the utility of multi-spectral imaging of fingerprints in the 400-1000nm range is questionable.
The Crime Scene Investigator Network gratefully acknowledges the Office of Justice Programs’ National Criminal Justice Reference Service for allowing us to reproduce the article .
Article posted December 28, 2018