Page 89 - Artificial Intelligence in the Age of Neural Networks and Brain Computing
P. 89

References     77




                     Substituting Eq. (3.21) into Eq. (3.17) we obtain finally overall iterative algo-
                  rithm of unsupervised learning weight adjustment over time step t is driven by
                  backprop
                                             .  . 0
                       ½W ji ðt þ 1ފ ¼ ½W ji ðtފ þ h g S þ a mom ½W ji ðtÞ  ½W ji ðt   1ފŠ;  (3.29)
                                               j
                                                 i
                  where we have followed Lipmann the extra momentum term to avoid the Mexican
                  standoff ad hoc momentum a mom to pass the local minimum.



                  ACKNOWLEDGMENT

                  Sponsored by ONR Grant N00014-17-1-2597.



                  REFERENCES
                   [1] H. Szu, G. Moon, How to avoid DAD? MOJ Applied Bionics and Biomechanics 2 (2)
                      (2018) 43e58.
                   [2] A. Ng, The State of Artificial Intelligence, MIT Review, YouTube, Similar to Internet
                      Company: Product-Website-Users (e.g., Google, Baidu); AI Company: Data-
                      Products-Users Positive Cycle.
                   [3] J. Mervis, No so fast, Science 358 (2017) 1370e1374;
                      (a) M. Hutson, A matter of trust, Science 358 (2017) 1375e1377.
                   [4] G. Cybenko, Approximation by superposition of a sigmoidal functions, Mathematics of
                      Control, Signals, and Systems 2 (1989) 303e314;
                      (a) S. Ohlson, Deep Learning: How the Mind Overrides Experience, Cambridge Univ.
                      Press, 2006.
                   [5] A.N. Kolmogorov, On the representation of continuous functions of many variables by
                      superposition of continuous function of one variable and addition, Doklady Akademii
                      Nauk SSSR 114 (1957) 953e956.
                   [6] R. Lipmann, Introduction to computing with neural nets, IEEE ASSP Magazine (April
                      1987).
                   [7] G. Hinton, Y. LeCun, Y. Bengio, Deep learning, Nature 4 (2) (2015) 4e22.
                   [8] H. Szu, M. Wardlaw, J. Willey, K. Scheff, S. Foo, H. Chu, J. Landa, Y. Zheng, J. Wu,
                      E. Wu, H. Yu, G. Seetharamen, J. Cha, J. Gray, Theory of glial cells & neurons
                      emulating biological neural networks (BNN) for natural intelligence (NI) operated
                      effortlessly at a minimum free energy (MFE), MOJ Applied Bionics and Biomechanics
                      1 (1) (2017) 1e26.
                   [9] J. McCelland, D. Rumelhart, PDP Group, MIT Press, 1986.
                  [10] S.-Y. Lee, H. Szu, Design of smartphone capturing subtle emotional behavior, MOJ
                      Applied Bionics and Biomechanics 1 (2) (2017) 1e10.
   84   85   86   87   88   89   90   91   92   93   94