Hiring Institution
Université Grenoble Alpes (UGA)

PhD-Awarding Institutions
Université Grenoble Alpes (UGA)
Queensland University of Technology (QUT)

Download the full Position Description

Option 1

Identifying communication materials for pictorial communication in context

This project will explore the affordances of emerging AI models and algorithms for Augmented Alternative Communication (AAC) approaches respectful of the competencies and preferences of people with intellectual disability.
In this project, the PhD candidate will establish algorithms for recommending image sets from diverse everyday contexts, such as a picture of a scene, the content of a book, or a restaurant menu. There are a number of approaches that can be considered here, from using keyword selection and large-scale lexical resources such as WordNet to generating captions and summaries using deep learning approaches.
The candidate will create flexible and configurable prototypes in order to engage with participants with intellectual disability in order to establish the types of images that are useful, and how many should be recommended, or how they may be organized. The prototype will be trialed with people with intellectual disability as part of the thesis, and refined based on the preferences and usage of research participants.

Option 2

Generating personalised images for use in communication by with minimally-verbal adults

This project will explore the affordances of emerging AI models and algorithms for Augmented Alternative Communication (AAC) approaches respectful of the competencies and preferences of people with intellectual disability.
In this project, the PhD candidate will consider the potential of emerging AI generative models, such as Stable Diffusion, to fabricate visual communication materials that are personalized to individuals with intellectual disability for use in AAC systems or in shared communication contexts. These images are generated, or can be retrieved, based on elaborated description. As a result, the project will investigate not only the image generation process itself, but more so the caption generation. The generated images will account for short history of image selection by a person in order to refine a model of style and content preferences, as well as a preferred complexity of visual content (eg. Number of concepts carried by images). Indeed, some images convey a single unit of meaning (eg. An object), while others can represent complex scenes.
The candidate will create flexible and configurable prototypes in order to engage with participants with intellectual disability in order to establish the types of images that are preferred. The prototype will be trialed with people with intellectual disability as part of the thesis, and refined based on the preferences and usage of research participants.

Option 3

Deep learning methods for Augmented and Alternative Communication

This project will explore the affordances of emerging AI models and algorithms for Augmented Alternative Communication (AAC) approaches respectful of the competencies and preferences of people with intellectual disability.
In this thesis, we want to study how alternative and augmentative communication can benefit from modern text-based artificial intelligence methods and how in return modern text-based artificial intelligence methods can benefit from alternative and augmentative communication. The aim is to create multimodal and multilingual models (e.g. texts, images, pictograms…) based on neural networks exploiting transforming architectures. To do this, the team in Grenoble will benefit from its experience in building such models for written and spoken language as well as from the GPU computing grids available in France.
The candidate will create flexible and configurable prototypes in order to engage with participants with intellectual disability. The prototype will be trialed with people with intellectual disability as part of the thesis, and refined based on the preferences and usage of research participants. Targeted AAC Applications could be automatic organization of pictogram grids, speech/text to picto, automatic design of visual scenes.

Didier Schwab
Laurianne Sitbon

Informatics, Computer Science