This Thursday, November 21, from 2:00 PM to 4:00 PM, the Thesis Monitoring Committee (CAT) will discuss the proposal "Generalizable and Explainable Real-time Stroke Rehabilitation Support System with Accessible Technology," by student Ana Rita Cóias.

Abstract: "AI-based systems can support rehabilitation decision-making procedures and exercise training. With the advances in motion-tracking technology (e.g., vision-based motion capture systems) and machine learning, such systems can provide therapists with objective data about patients' impairments and progress towards a more informative decision process on therapy adjustments. As a virtual training coach, a support system can aid patients in exercising autonomously by providing instructions and feedback on performance. To fulfill its purpose, the virtual coach should incorporate a set of requirements: the objective motion assessment should be comprehensive, covering a wide range of exercises and various levels of impairment; it should offer explanations for its decision, promoting user trust and understanding; deliver real-time corrective feedback to encourage motor function improvement, keeping patients motivated and engaged in therapy; and its overall technical setup should be accessible and suitable to diverse settings (e.g., home).

In this thesis, we seek answering to the following research questions:

1) How can we achieve real-time exercise assessment, avoiding a costly and time-consuming labeling process?

2) How can we achieve a more explainable virtual coach?

3) How can we successfully evaluate new exercises with just a small sample of new data?

4) Can we successfully assess exercise performance with off-the-shelf accessible technology?

We will address these questions by developing: machine learning approaches to assess exercise performance in real time, relying only on video-level annotations; algorithms with a great capability of generalization to different motion patterns and exercises; a virtual coach prototype that provides explanations about its decision for therapists and patients; investigate the level of accessibility of the proposed solution and collect findings on how to make it more accessible. So far, with our contributions, we have acquired some insights that bring us closer to our goal. In a previous contribution, we developed a prototype of a virtual coach that provides visual markers highlighting body joints requiring attention while performing exercises. We have collected findings on the prototype usability with healthy and a post-stroke participant. We identified suitable motion-tracking solutions that enable the application of regular RBG cameras for exercise assessment, enhancing system accessibility. We collected a new dataset of 20 post-stroke patients performing five rehabilitation exercises. We explore a framework for frame-level pseudo-labeling generation leveraging an Explainable AI technique. Besides our contributions, we still need to take some steps forward to achieve our goal. This document details the work developed towards answering the presented research questions and the research proposal for the next year and a half. We describe our research plans and how we will approach them."

The session will be held in mixed mode:

- In presence in the ISR Meeting Room, IST-North Tower, Floor 7.

- Via Zoom link for the external participants