Feedback strategies for enhancing Human Movement Augmentation

Human Movement Augmentation (HMA) aims to further enhance human abilities to move and interact with the environment. HMA is achieved through the use of an augmented device that can be tailored to the specific application and task. In this thesis, we will mainly focus on Supernumerary Robotic Limbs (SRLs): robotic manipulators used in close proximity to the user or worn to increase the user's available degrees of freedom. The success of HMA relies on the design of both a control interface to send commands to the SRL, and a feedback system to provide the user with supplementary sensory information. While one might argue that vision alone provides a sufficiently rich sensory modality to close the sensorimotor loop, we believe that supplementary feedback can offer important benefits. Vision can be occluded, and in some cases (e.g., estimating interaction forces) may not offer reliable cues. According to motor control theory, every movement emerges from the continuous interaction between motor commands and sensory information: the sensorimotor loop. In the context of HMA, this loop also includes supplementary feedback. Afferent and efferent signals are also linked, in the central nervous system (CNS), through dynamic and kinematic representations (i.e., body schema) that summarize beliefs and knowledge about the body, and tools. The concept of body schema, and more broadly, body representation, is fundamental for both motor control and embodiment i.e. the sense of owning (i.e., ownership) and controlling (i.e., agency) a body part. These internal models allow for the planning and execution of movements, while also enabling users to experience tools or external devices as part of their own body. In the context of HMA, the goal is to control SRLs as if they were natural extensions of the body. Achieving this requires not only accurate motor output but also sensory feedback. Importantly, motor control and embodiment are not independent but they partially overlap, supporting both coordinated action and the subjective experience of embodiment While most of the literature has focused on the design of control interfaces, comparatively little attention has been given to feedback strategies and devices. In particular, the impact of adding supplementary feedback to close the sensorimotor loop, and how this influences user behavior, remains underexplored. This thesis seeks to address that gap, offering concrete examples of feedback design and evaluating their behavioral effects in augmented tasks. Designing and implementing effective supplementary feedback is inherently complex. It involves identifying which information should be provided, and how it should be delivered. In addition, task constraints may restrict the possible design choices. Factors such as the type of technology used and the location on the body where feedback is delivered further influence the effectiveness of the system and must be considered to optimize the flow of information. Given the complexity and relevance of designing a proper feedback strategy, a reasonable approach is to first characterize the users ability to discriminate that particular strategy, and then apply it in a closed-loop scenario to assess behavioral outcomes. For this reason, we carried out some psychophysical preliminary studies to characterize the perceptual properties of the feedback. First, we developed a biofeedback method using eccentric rotating mass (ERM) motors to convey motor units activity (i.e., firing rate). Results showed that motor placement significantly influenced users' discrimination abilities. Specifically, placing motors on different arms, thereby increasing spatial separation, improved discrimination. Nevertheless, the feedback coding strategy proved to be an even more decisive factor. In particular, the continuous vibration encoding of firing rate outperformed the burst-based (i.e., spike strategy) encoding of individual action potentials. These findings highlight the necessity of thoroughly characterizing feedback modalities prior to implementation. The importance of properly characterizing the feedback was highlighted also in the second pilot study. When using electrotactile stimulation via a high-density electrode matrix, it is possible to enhance spatial resolution by exploiting the funneling illusion (i.e., when two pads are activated simultaneously, a phantom stimulus is perceived in a location midway between them). However, to produce a smooth transition in perceived intensity between single-pad and dual-pad phantom stimulation, our results showed that the two pads must be activated with a combined intensity higher than that of the single pad. While these initial results address the perceptual efficacy of the feedback strategies, it remains essential to evaluate their impact in functional, closed-loop contexts. To this end, we tested two different scenarios: a force regulation task and a trimanual coordination task. In the force regulation task, users controlled the interaction force of a wearable robot's end-effector with the environment. Even a simple vibrotactile feedback, proportional to the exerted force, significantly improved their ability to apply and control the desired force. In the trimanual task, the effectiveness of the supplementary feedback was influenced by the coordination required to complete the task. The greatest benefit was observed when all effectors were coupled (i.e., tri-couple task) to achieve a common objective. The results presented so far emphasize the complexity of designing appropriate feedback strategies and highlight their positive influence on motor performance in HMA. Still, these experiments primarily explore the perceptual and behavioral dimension. However, Human-robot interaction occurs within a continuous loop of afferent and efferent signals which, over time, through multisensory integration, may reshape our internal body representations and users may begin to embody the SRL. While previous work has demonstrated embodiment with simple tools like sticks, evidence concerning the embodiment of SRLs is still limited and inconclusive. To address this, we validated a platform where the SRL is user controlled and congruent supplementary feedback is provided to close the sensorimotor loop. Preliminary results suggest the feasibility of integrating the SRL into the user's body representation and embody it.