Page 289 - Artificial Intelligence in the Age of Neural Networks and Brain Computing

282 CHAPTER 14 Meaning Versus Information, Prediction Versus Memory

fooled by slightly altered inputs such as adversarial inputs), and interpretability
(deep neural networks are basically a black-box, and humans do not understand
why or how they work so well).
In this essay, I will talk about some of the concepts that are central to brain
science and artificial intelligence, such as information and memory, and discuss
how a slightly different view on these concepts can help us move forward, beyond
current limits of our understanding in these fields.
The rest of this chapter is organized as follows: in Section 2, I will discuss mean-
ing versus information, with an emphasis on the need to consider the sensorimotor
nature of brain function. In Section 3, I will talk about prediction and memory, in the
context of synaptic plasticity and brain dynamics. In Section 4, I will move on to a
broader topic of question versus answer, and discuss how this dichotomy is relevant
to both brain science and artificial intelligence. Section 5 will include some further
discussions, followed by conclusions.

2. MEANING VERSUS INFORMATION

The concepts of computing and information have fundamentally altered the way we
think about everything, including brain function and artificial intelligence. In
analyzing the brain and in building and interpreting intelligent artifacts, computing
has become a powerful metaphor, and information has become the fundamental unit
of processing and measurement. We think about the brain and artificial neural
networks in terms of computing and information processing, and measure their
information content. In this section, I will talk mostly about information.
First of all, what is information? We tend to use this word in a very loose sense,
and information defined in this way is imbued with meaning (or semantic content).
In a scientific/engineering discussion, information usually refers to the definition
1
given in Claude Shannon’s information theory. In information theory, information
is based on the probability of occurrence of each piece of message, and the concept
is used to derive optimal bounds on the transmission of data.
However, there can be an issue if we try to use Shannon’s definition of informa-
tion in our everyday sense, as Shannon explicitly stated in his primary work on in-
formation theory that information defined as such does not have any meaning
attached to it. So, for example, in an engineered information system, when we store
information or transmit information, the data themselves do not have meaning. The
meaning only resides in the human who accesses and views the information. All the
processing and transmission in the information system is at a symbolic level, not at a

1
“Category: Information Theoryd Scholarpedia.” May 26, 2011. http://www.scholarpedia.org/article/
Category:Information_Theory.