Page 56 - Introduction to Information Optics

1.6. Trading Information with Entropy 41

On the other hand, if the signal is a single pulse function, such as

u(t) = <5(r - t 0),

then the corresponding WDF would be

Mr, v) - <5(r - a

which is plotted in Fig. 1.14b.

1.6. TRADING INFORMATION WITH ENTROPY

Let us consider a nonisolated system, in which the structure complexity has
been established a priori in N equiprobable status; the entropy of the system
can be written as [1.10]

S 0 = k\nN, (1.135)

where k is Boltzmann's constant. If the system structure is reduced by outside
intervention to M state, (M < N), then its entropy would be

S,=fclnAf. (1.136)

Since S g > S l5 the decrease in entropy in the system is obviously related to the
entropy information / that can be acquired from external sources:

AS - Si - S 0 = -fc/ln2, (1.137)

where / = Iog 2 N/M. Thus, we see that the amount of information required for
this reduction should be proportional to amount of entropy as AS decreases in
the system. One of the most intriguing laws in thermodynamics must be the
second law [1.11], in which it stated that for an isolate system its entropy can
only be increased or remain constant; that is,

AS t = A(S 0 - Win 2) ^ 0. (1.138)

In other words, any further increase in entropy AS t can be due to AS 0 or A/,
or both. Although in principle it is possible to distinguish the changes in AS 0
and A/ separately, in some cases the separation of the changes due to AS 0 and
A/ may be difficult to discern.
It is interesting to note that, if the initial entropy S 0 of the system