DOCTORAL CANDIDATE
Ardianto Wibowo

SUPERVISORS
Amer Baghdadi, Institut Mines Telecom Atlantique (IMT)
Paulo Santos, Flinders University
Karl Sammut, Flinders University

Artificial Intelligence, Multi-agent systems, Embedded Systems, Maritime Engineering.

This Ph.D. project has two primary objectives. First, it aims to enhance the implementation of an existing novel algorithm, Qualitative Case-Based Reasoning and Learning (QCBRL), through the integration of MARL, with a focus on autonomous maritime vessels performing search and rescue tasks. The second objective is to evaluate and compare the proposed solutions for efficiency and adaptability using a hardware-in-the-loop approach, leveraging real-life embedded boards and a sophisticated simulation platform.

In the context of QCBRL, new cases are generated using a partial Reinforcement Learning (RL) method. When there are no similar cases available, QCBRL iterates the RL method through multiple simulations until it achieves a successful episode. The state-action set from this successful episode is then stored as new cases. Maintenance of these cases involves assigning trust values, which are updated based on whether the retrieved case successfully solves the problem. Cases with trust values meeting removal criteria are deleted. This process follows the complete Case-Based Reasoning (CBR) cycle, encompassing Retrieval, Reuse, Revision, and Retention, with an explicit world representation for human understanding and system accountability.

In the initial work, the environment was observed by multiple agents. However, the reasoning was centralized within a single agent. Based on this, this research aims to explore team-agent reasoning by investigating the capability of MARL to learn typical cases and retrain the distributed system within the maritime domain. Recent theoretical findings indicate that decentralized MARL can converge effectively. Navigation tasks may rely on pre-trained models, but they can benefit from online learning, especially when identifying obstacles and targets, which are used to asynchronously update a partially shared environmental model.

Furthermore, this research also aims to evaluate and compare the proposed solutions in terms of efficiency, adaptability, and feasibility of implementation using a hardware-in-the-loop approach. This approach involves implementing each agent on real-life embedded boards, such as NVIDIA Jetson boards, connected to a virtual reality simulator. The choice of the AirSim simulation platform, based on the Unreal Engine, is promising. It provides a scalable and photorealistic training environment, which is crucial for ensuring that the research findings can be effectively translated into real-world maritime applications.

This research is expected to hold significant potential for improving the capabilities of autonomous maritime vessels, making them more efficient and adaptable in search and rescue missions, ultimately contributing to the safety and security of maritime operations.