Page 166 - Automated Fingerprint Identification Systems (AFIS)

BUYING AN AFIS SYSTEM: THE BASIC DOCUMENTS NEEDED 151

• Government furnished property, information, or access to government
facilities and information systems [what]
• Security considerations—what level of clearance, if any, will be required for
contractors and government personnel on the project? [who and what]
• Project milestones to include preparation of acquisition phase documents,
their approval, and release [when]
• Who will run the acquisition? [who]
• Who approved the Con Ops? [who]

As part of developing an acquisition strategy, many decisions will have to be
made. Some of the key decisions can be addressed with a decision tree. They
include the following:

• Should there be a benchmark or not?
• How should vendors be distinguished by use of mandatory versus rated
requirements?
• Should the evaluators be given a chance to see the bidders as part of an oral
presentation or not? 2
• What weights should be assigned to the following vendor proposal con-
stituents: price, experience, benchmark results, orals presentation quality,
and rated requirements?

The next subsection addresses the pros and cons of benchmarking.

7.4.3 BENCHMARKING

A benchmark is a documented procedure that will measure an AFIS in the exe-
cution of a well-defined set or sets of tasks. Many different aspects of an AFIS,
ranging from response time and false match rate to ease of the user interface,
can be measured. It is assumed that these metrics relate to the anticipated per-
formance in a particular application. Thus, there is a need to carefully align
any benchmark with the particulars of the intended use. Obviously, nobody
else’s benchmark results are fully indicative of results you would experience.
A well-defined benchmark can be applied to several vendors’ systems so that
comparisons can be made between different proposed systems based on your
anticipated use and your data.
An ANSI-IAI Standard for Benchmarking AFIS Systems was published in
1985. Since it was not reviewed and updated at the end of the 5-year nominal
life of ANSI standards, it was dropped. It is still worth reading just to know what
performance metrics can be benchmarked.

2
An outline for an oral presentation can be found in Ch. 9 of Biometrics: Identity Assurance in the
Information Age, Woodward, Orlans, and Higgins, 2002, by Osborne, a McGraw Hill Company.