The Viper system detects the muzzle flash of a sniper's weapon with a focal plane array staring midwave infrared camera. The video signal from an Inframetrics MilCam is fed to a Datacube MV-250 real-time image processor. Detection algorithms incorporate a histogram based automatic gain and pedestal control routine, a temporally matched flash filter, spatial filters, adaptive background estimation, adaptive thresholding, track filtering and a final detection locator. The Viper system that was described in earlier publications (1,2) was designed for use with a stationary camera. Thus, if the camera was moved or jarred the system would produce a very large number of false alarms while the camera motion lasted.

It was decided to test the feasibility of using the Viper system from moving platforms, such as aircraft and land vehicles. Algorithm modifications for airborne deployment are generally easier to implement than for a land vehicle. There are two main reasons for this. The first is that motion characteristics of an airborne platform can be considered to be smoother than that of a land vehicle. Thus, the camera is subjected to less of a jarring motion if placed on an aircraft. The second reason is that if deployed on an aircraft, the objects in the field of view are essentially in the far field. Nearby objects, such as trees or buildings within the camera field of view complicate deployment on moving land vehicles.

Objects in the near field of view make algorithm design for a moving camera system significantly more difficult. This is because as the camera is translated, the relative motion of objects in the far field and in the near field are not identical, resulting in new areas of the scene being occluded or becoming visible before the camera. For example, if the camera is deployed on a Hummv driving down a road, trees in the near field will result in areas in the background which will alternately become visible and then be blocked from the camera. Since the Viper system uses an adaptive background estimation method for signal detection, these effects of near field objects would have to be remedied. One approach would be to implement a routine to estimate a range to each object in the image, essentially producing a three dimensional map of the scene. Algorithms of this type to derive three dimensional information from a moving camera have been developed by others. However, these methods require more processing power than available in the current Viper system. Therefore, it was decided to begin implementation of a moving camera Viper system with an airborne version.
 

The first task in modifying the Viper system for airborne operation was to modify the detection algorithms to compensate for the image changes due to camera motion. Since all objects seen by the camera are essentially at the far field, successive video fields could be registered to each other just by using the warping, translation and rotation operations. Furthermore, by using a relatively narrow field of view lens (32 degrees x 24 degrees) pincushion distortion was minimized and the need for the image warp operation was eliminated. For the initial flight tests translation and rotation operations were sufficient to register successive image frames.

The Viper temporal filter is a four frame long finite impulse response matched filter. This means that as each frame of video is acquired (at 60 frames per second) it has to be registered with the three previous frames, each of which is offset by its own translation and rotation parameters. Furthermore, the background activity estimate has to be updated at each frame by the translation and rotation parameters for that frame.

There are several ways to estimate the motion of the video camera from one frame to the next. One method is to use an attitude measurement device, such as an inertial measurement unit (IMU) or fiber optic gyro, to measure the motion of the camera; and applying the known mechanical relationship between the camera and the attitude reference to calculate the translation and rotation parameters. This would probably be the approach taken in the design of an operational system, where it is likely that an attitude reference is already built into the aircraft.

A second approach is to use the information contained in the video signal itself to estimate camera motion parameters. This is done via correlation methods to compare two successive frames and find translation and rotation parameters that give a best match. Since in Viper the target to be detected is a point source, the presence or absence of the flash would not significantly effect the correlation based image registration routines. The problem however is that these methods require significant amounts of processing power and were considered not feasible with the existing hardware.

While the problem of frame registration was being considered another issue became apparent. The payload size and weight limitations of the NRL Swallow UAV allowed only a sensor package and a downlink to be flown. The integration of an IMU package with its own downlink was not practicable. It was thus determined that video only could be downlinked and recorded. Camera motion estimation would be performed only based on downlinked video.

It was desired to collect the downlinked video in digital format. To do this a personal computer (PC) based video recorder was designed and built. This system used a frame grabber and a dual channel wide SCSI architecture to strip incoming video to four streams, each of which was recorded on a separate hard disk, in a redundant array of inexpensive drives (RAID) implementation. This custom developed RAID recorder solution was chosen as a practical solution since it was much less expensive than purchasing a commercial off the shelf RAID recorder. This recorder also provided a live display with overlaid frame number and timestamp which was recorded on to video tape along with an audio track. Radio communications and input from a microphone were recorded on this audio track, greatly simplifying flight test documentation and providing ground truth data as to when each shot occurred.

Test flights were flown at a small private airfield at Airlie, Virginia. On the days of testing the airport was closed to all other traffic. The system was first tested on a T-34 aircraft. An advantage of this aircraft is that it is a tandem seater with a canopy that can be opened in flight. The camera was handheld with the operator holding it steady out the port side of the aircraft. The operator also was responsible for setting the pedestal and gain of the camera by monitoring the IR video through a viewfinder. To make integration with the T-34 as simple as possible two battery belts were used to power the camera and UHF video transmitters. The transmitters and antennas were attached to the inside of the canopy using suction cups. This allowed an unobstructed line of sight to the ground receiver antenna.

To make the motion estimation problem easier it was decided to place marker beacons on the ground near the area of interest. These simply consisted of four coal fueled grills that were on the ground and which were lit before each flight.[Figure 1] These provided a fixed frame of reference for objects on the ground. For each frame of video the location of each marker beacon could be automatically tracked based on its position on the previous frame. A very simple correlation tracker searching a small area was implemented to automatically track each beacon on the image.

The ground station consisted of a vehicle mounted video recorder system and the UHF receiver system with an 18 inch dish antenna. One person was assigned the task of manually pointing the receive dish at the aircraft as it flew overhead.

During each flight the pilot would keep the aircraft at a constant altitude and roll angle, and by varying power would control his position to orbit the target area in a circular path. The ground team consisted of a recorder operator, an antenna tracker, a rifleman, and a safety coordinator.

When the recorded data was analyzed and algorithms for motion compensation based on the four marker beacons were written it was apparent that the automatic detection of rifle muzzle flashes from an aircraft is feasible. It was decided to continue on to demonstrate the deployment of Viper on a UAV.

For data collection flights from the UAV platform required knowing whether the shooter was within the field of view of the sensor and to time rifle shots appropriately. In flight tests with a manned aircraft the camera operator essentially kept the camera pointed at the target, and was in radio communications with the ground. With a UAV however, ground personnel would not easily have known if the sniper area was within the sensor field of view. Additionally it was desired to provide a means of visible feedback to the UAV operator to guide him in establishing the flight path. To do this a visible light video camera was incorporated into the sensor package with its own downlink [Figure 2]. The two video streams (IR and visible) were synchronized to each other by using the RAID recorder frame counter and mixing this with the visible video which was recorded on tape. The visible tape would later be digitized with the RAID recorder and a frame accurate (at 30 frames/sec) alignment with the corresponding infrared video would be made [Figure 3]. Furthermore, the two cameras were boresighted and camera coordinates were calibrated for further analysis.

The data collected at the UAV test flights were analyzed by running the automatic marker beacon tracking and detection algorithms. The detections from the infrared sequences were then overlaid as target symbology onto the visible video collected during the flights. This was done by using the cross camera calibration parameters. This allowed the tracking of the shooter location once it was detected as long as it and the marker beacons were within the field of view.

These experiments demonstrated that the Viper system can be effective when deployed on an airborne platform and that algorithms can be developed to detect and track the location of muzzle flashes from manned and unmanned aircraft. Although in this case the algorithms were developed to run offline with recorded data, their real time implementation is practical.
 

In addition to airborne deployment several improvements to the original Viper system were implemented. In 1996 four Viper systems were delivered for evaluation by the US Army Dismounted Battlespace Battlelabs and by the US Marine Corps. Upon receiving comments from the counter sniper evaluation teams several design enhancements were made. One of these was the incorporation of an acoustic detector to verify IR detections and to provide passive range information for a muzzle flash. Another improvement was the development of two new simplified methods for calibrating counterfire rifle coordinates to the camera.
 

A simple acoustic detector was built with a COTS microphone, an amplifier and a PC mounted data acquisition system. The microphone output was sampled at 6000 Hz and a digital filter was implemented to estimate the energy of the acquired signal. A simple infinite impulse response filter was used to estimate background energy levels and to detect the presence of a muzzle blast. Since this acoustic filter is to be used in conjunction with Viper, and to verify IR based detections, it was considered important not to miss any detections. Since the acoustic detector would not be the primary sensor its stand alone false alarm rate was considered to be of secondary importance. Thus while the acoustic filter may give false alarms from car doors closing and other impulsive noises, it was considered that in an operational situation with a counter sniper team these would not degrade the performance of the primary IR flash detection sensor.

The Viper system alarm output was connected to the PC and each IR detection opened a predetermined range gate. Knowing the speed of sound at the ambient temperature, if an acoustic detection was received within the range gate the detection was considered to be verified, and the range calculated to the target by measuring the flash-bang delay time.

The system was tested at the Fort Meade, Maryland rifle range and demonstrated that acoustic verification for IR detection is highly feasible. One of the unforeseen situations encountered was that on one of the test days an adjacent rifle range was in use with rapid fire weapons and that the acoustic signatures from the other range were being detected. Thus, in environments where other weapons' blasts are audible within the range gate triggered by an IR event the usefulness of acoustic verification may be diminished.

Work is currently underway to implement the acoustic detection algorithms on a digital signal processor platform for incorporation into the Viper system.
 

The Viper system has a magnetometer / inclinometer which attaches to counterfire rifle. The calibration procedure was to choose three points in the infrared video display. The observer moved a cursor onto these points in the infrared image and had the shooter point his rifle at the chosen points. A comment encountered during the evaluation of the system was that under some situations it was difficult to find three points which showed up at high contrast on the IR image. It was therefore decided to develop alternative procedures to select three calibration points.

The first approach that was implemented was to attach a second video camera in the visible spectrum to the IR camera. This would allow the observer to choose aim points in the visible light image. The two cameras were cross calibrated (by the same procedure as used in the UAV deployment) and the rifle to visible calibration would provide the rifle to IR camera calibration.

This system was built and evaluated in the field. While it worked well under most lighting situations, there were cases, such as when positioned toward the sun or looking at very high contrast scenes, when it was deemed to be unacceptable.

To solve the calibration problem another approach was developed. This involved the construction of a miniature gimbal with azimuth and elevation axes that incorporated a standard sniper's scope. This removable assembly was attached to the camera and was operated in the same manner as a theodolite. By aiming the calibration scope at three fixed points, the camera coordinates of these points were determined. The shooter aimed at these points, giving the magnetometer / inclinometer readings for these calibration points. Thus the system was calibrated.

The scope based calibration consistently produced accurate results. And since the calibration device is a sniper's scope, identical to the one on the shooter's rifle, it always provided an unambiguous calibration. The only negative side to this approach is that there is one extra piece of equipment, the gimballed calibration head, as part of the system. The calibration gimbal which was manufactured as a prototype is rather unwieldy, but this item can be miniaturized for production versions.
 

The feasibility of infrared detection and location of muzzle flashes from an airborne platform has been demonstrated and significant improvements have been made to the Viper system to enhance its performance on the ground. A study of the application of these methods to land based vehicle deployment is currently under way.
 

We would like to thank Dave Youn, who was the camera operator aboard the T-34; Ron Gilman, who was the T-34 pilot; and Chris Bovais, who was the pilot of the Swallow UAV. This work was funded by the Naval Research Laboratory under contract number N00014-97-D-2006.