The MDP formalism and its variants are usually used to control the state of a system through an agent and its policy. When the agent confronts incomplete information, its policy can perform actions to gather information, such as in (1) the partially observable state case, or in (2) the reinforcement learning scenario. However, the acquired information is only a means to better control the system state, so the information gathering is only a consequence of maximizing the expected return. On the contrary, the purpose of this dissertation is to study sequential decision problems where acquiring information is an end in itself. More precisely, it fi rst covers the question of how to modify the POMDP formalism to model information-gathering problems and which algorithms to use for solving them. This idea is then extended to reinforcement learning problems where the objective is to actively learn the model of the system. Also, this dissertation proposes a novel Bayesian reinforcement learning algorithm that uses optimistic local transitions to efficiently gather information while optimizing the expected return. Through bibliographic discussions, theoretical results and empirical studies, it is shown that these information-gathering problems are optimally solvable in theory, that the proposed methods are near-optimal solutions, and that these methods off er comparable or better results than reference approaches. Beyond these specific results, this dissertation paves the way (1) for understanding the relationship between information-gathering and optimal policies in sequential decision processes, and (2) for extending the large body of work about system state control to information-gathering problems.
