|
Abstract : |
RoboCup simulated soccer presents many challenges to reinforcement learning methods, including a large state space, hidden and uncertain state, multiple agents, and long and variable delays in the effects of actions. We describe our application of episodic SMDP Sarsa() with linear tile-coding function approximation and variable to learning higher-level decisions in a keepaway subtask of RoboCup soccer. In keepaway, one team, "the keepers, " tries to keep control of the ball for as long as possible despite the efforts of "the takers. " The keepers learn individually when to hold the ball and when to pass to a teammate, while the takers learn when to charge the ball-holder and when to cover possible passing lanes. Our agents learned policies that significantly out-performed a range of benchmark policies. We demonstrate the generality of our approach by applying it to a number of task variations including different field sizes and different numbers of players on each team. 1., |
