An Exploration of a Hierarchical Approach for MacDec-POMDPs

Abstract

In order to do something useful, you should know what to do and how to do it. The same goes for robots or other machines, which are also referred to as agents. Advances in e.g. technology and science have made such agents more and more sophisticated and capable. This opens up a plethora of possibilities for solving problems or automating processes.
However, as agents and the problems they solve become more and more sophisticated, the software that controls these agents naturally also becomes more complex. Especially when considering systems of multiple cooperating agents. With rapidly increasing amounts of agents, observations and capabilities to take into account, the plans used to come to decisions become increasingly infeasible to create by hand. Thus automated methods are necessary to create such plans.
The approach this thesis uses is inspired by the ability of human intelligence to abstract away pedantic details when planning. We do not for example include which muscles we will utilize, when we make a plan for picking up groceries. Deciding to get groceries is on a completely different level than deciding how many centimeters we want to move a limb in a particular direction. Abstracting away such details greatly reduces the size of any potential plan. Which makes plans easier to construct, while the scope of the problem remains the same.
Thus, utilizing this notion of different levels of abstraction during planning, is the approach this thesis uses to improve planning for agents. This is done by extending the Macro Decentralized Partially Observable Markov Decision Process (MacDecPOMDP) framework to allow for an arbitrary number of levels of abstraction.