荒井 幸代

Recently, inverse reinforcement learning, which estimates the reward from an expert's trajectories, has been attracting attention for imitating complex behaviors and estimating intentions. This study proposes a novel deep inverse reinforcement learning method that combines LogReg-IRL, an IRL method based on linearly solvable Markov decision process, and ALOCC, an adversarial one-class classification method. The proposed method can quickly learn rewards and state values without reinforcement learning executions or trajectories to be compared. We show that the proposed method obtains a more expert-like gait than LogReg-IRL in the BipedalWalker task through computer experiments.

In recent years, the application field of drones has rapidly expanded, and they are attracting attention as a means of gathering information in disaster areas and as a means of disaster relief. Currently, drones are mainly controlled manually or by programmed control with pre-defined actions. Manual control limits the flight range to the transmitter's communication range and requires constant monitoring of the drone in flight, making it difficult to operate the drone when access to the disaster area is difficult or visibility is poor. In addition, since programmed control sets the drone's behavior in advance, stable control in response to changes in the flight environment due to a disaster may be difficult in emergency surveys during a disaster. The purpose of this study is to verify the applicability of model-based control, which is based on a mathematical model of the environment, and model-free control, which obtains optimal control rules from interaction with the environment, to drone’s controller for realizing autonomous drone flight in environments that are difficult to operate with existing control methods. Specifically, we compared the performance of model predictive control as model-based control and reinforcement learning as model-free control by computer experiments, and verified that the characteristics of both control system differ depending on the nature of the disturbance.

This study is intended to encourage appropriate social norms among multiple agents. Effective norms, such as those emerging from sustained individual interactions over time, can make agents act cooperatively to optimize their performance. We introduce a "social learning" model in which agents mutually interact under a framework of the coordination game. Because coordination games have dual equilibria, social norms are necessary to make agents converge to a unique equilibrium. As described in this paper, we present the emergence of a right social norm by inverse reinforcement learning, which is an approach for extracting a reward function from the observation of optimal behaviors. First, we let a mediator agent estimate the reward function by inverse reinforcement learning from the observation of a master's behavior. Secondly, we introduce agents who act according to an estimated reward function in the multiagent world in which most agents, called citizens, have no way to act. Finally, we evaluate the effectiveness of introducing inverse reinforcement learning.------------------------------This is a preprint of an article intended for publication Journal ofInformation Processing(JIP). This preprint should not be cited. Thisarticle should be cited as: Journal of Information Processing Vol.22(2014) No.2 (online)DOI http://dx.doi.org/10.2197/ipsjjip.22.299------------------------------This study is intended to encourage appropriate social norms among multiple agents. Effective norms, such as those emerging from sustained individual interactions over time, can make agents act cooperatively to optimize their performance. We introduce a "social learning" model in which agents mutually interact under a framework of the coordination game. Because coordination games have dual equilibria, social norms are necessary to make agents converge to a unique equilibrium. As described in this paper, we present the emergence of a right social norm by inverse reinforcement learning, which is an approach for extracting a reward function from the observation of optimal behaviors. First, we let a mediator agent estimate the reward function by inverse reinforcement learning from the observation of a master's behavior. Secondly, we introduce agents who act according to an estimated reward function in the multiagent world in which most agents, called citizens, have no way to act. Finally, we evaluate the effectiveness of introducing inverse reinforcement learning.------------------------------This is a preprint of an article intended for publication Journal ofInformation Processing(JIP). This preprint should not be cited. Thisarticle should be cited as: Journal of Information Processing Vol.22(2014) No.2 (online)DOI http://dx.doi.org/10.2197/ipsjjip.22.299------------------------------