Page 404 - Mechatronics for Safety, Security and Dependability in a New Era

Ch78-I044963.fm Page 388 Monday, August 7, 2006 11:30 AM
Ch78-I044963.fm
388
388 Page 388 Monday, August 7,2006 11:30 AM - & •
and the total moved distance by 100 trials. Table 2 shows the attainment rate of task at 100th trial and
the total moved distance. The attainment rates were low because the number of trials was a little. The
learning methods with fuzzy ART were superior to the normal actor-critic learning method.
Furthermore, the learning speed had quickened most and the total moved distance was largest in using
inheritance of the state-value. The efficiency of the proposed method was confirmed in the experiment
of multi-link mobile robot though the number of trials was a little.

TABLE2
PERFORMANCE OF EACH LEARNING METHOD FOR A MOBILE ROBOT
Module2 Module4
Modulel / Module3 / Module5 Cvclomctcr Fuzzy ART Fuzzy ART
Normal with
method without
inheritance inheritance
Total moved distance 429 mm 485 mm 578 mm
th
Attainment rate of taskat 100 trials 29% 34% 36%

CONCLUSION

We proposed a reinforcement learning method that used fuzzy ART for segmentation of state-space.
And we proposed a generating method of a new category node that inherited the state-value of the
similar node. The efficiency of proposed method was estimated in the simulations of hand reaching
problems and the movement experiments. The learning efficiency was improved more by inheriting
the sate-value in the fuzzy ART. The learning speed of proposed method is about 20 times the speed of
normal actor-critic method in the hand reaching problems. The size of state-space was decreased very
much in proposed method. The efficiency of proposed method was confirmed in the experiments of
multi-link mobile robot. Thus, it was confirmed that the proposed method was able to apply to the
learning with the real robots.

References

Y. Takita, M. Nunobiki, et al. (1999). An investigation of climbing up stairs for inchworm robot,
Proceedings ofTITech COE/Super Mechano-Systems Workshop'99, 133-138
M. Nunobiki, T. Takita, et al. (1999). Study on the gaits of an inchworm robot through a narrow
path. Advanced Robotics, 13: 3, 329-330
M. Nunobiki, et al. (2001). An investigation of mobility of inchworm-type mobile robot,
Proceedings of the France-Japan Congress of Mechatronics, 113-118.
M. Nunobiki, et al. (2004). Returning motion from the state of falling sideways for articulated
mobile robot, JSME International Journal, Series C, 47:1, 225-232.
R.S. Suttun and A.G. Barto. (1998). Reinforcement Learning; An Introduction , The MIT Press.
A.G. Barto, R.S. Suttun and C.W. Anderson. (1983). Neuronlike adaptive elements that can solve
difficult learning control problem, IEEE Trans. Syst., Man, Cyber., SMC-13:5, 834-847.
J. Morimoto and K. Doya. (2001). Acquisition of stand-up behavior by a real robot using
hierarchical reinforcement learning. Robotics and Autonomous Systems, 36, 37-51.
J. Morimoto and K. Doya. (1998). Reinforcement learning of dynamic motor sequence: Learning
to stand up, Proc. of IEEE/RSJ Int. Conf. On Intelligent Robots and System, 3, 1721-1726.
G.A. Carpenter, S. Grossberg and D.B. Rosen. (1991) Fuzzy ART: Fast stable learning and
categorization of analog patterns by an adaptive resonance system. Neural networks, 4, 759-711.

- &