Page 150 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

P. 150

CHAPTER

Pitfalls and Opportunities

in the Development and 7

Evaluation of Artiﬁcial

Intelligence Systems

David G. Brown, Frank W. Samuelson
US Food and Drug Administration, Silver Spring, MD, United States

CHAPTER OUTLINE
1. Introduction .......................................................................................................139
2. AI Development..................................................................................................142
2.1 Our Data Are Crap ............................................................................... 142
2.2 Our Algorithm Is Crap.......................................................................... 146
3. AI Evaluation .....................................................................................................148
3.1 Use of Data ........................................................................................ 148
3.2 Performance Measures ........................................................................ 149
3.3 Decision Thresholds ............................................................................ 154
4. Variability and Bias in Our Performance Estimates ...............................................154
5. Conclusion ........................................................................................................157
Acknowledgment.....................................................................................................158
References .............................................................................................................158

1. INTRODUCTION

Sergei, my appliance repairman, tells me my washing machine needs a new mother
board. I guess it does the crossword puzzle between loads. I look over at the car with
the funny hat in the next lane and there’s no driver. I read that it has no audio sensor
and can’t hear me honk at it. What else can’t it do? Does it understand that if a ball
rolls across the road a child may be in pursuit? This is the century of artiﬁcial intel-
ligence (AI), when computational intelligence (CI) algorithms show great promise
to tremendously improve the quality of our lives. Unfortunately, of course, it is
also the age of AI hype, in which systems without the intelligence of a cockroach
(Fig. 7.1) are touted as marvelous advances and cavalierly ceded the authority
over life-and-death decisions. We are warned that AI may be an existential threat
to our species; however, the dangers of AI at least for the foreseeable future pale
in comparison to those of AS (artiﬁcial stupidity), whereby CI algorithms are
used that are confusing, dysfunctional, and, yes, dangerous. In order not to become

145 146 147 148 149 150 151 152 153 154 155