A DIGITAL TWIN FRAMEWORK FOR HUMAN MOBILITY MEASUREMENT IN THE HOME SETTING

This technology, which will be supported by advanced deep neural networks, will facilitate accurate measurement of joint angles, as well as identification of motor tasks and task performance continuously over extended periods of weeks or months. Deployed on an iPad, the digital twin will allow a person, without any technical expertise, to use wearable technology to collect accurate and robust mobility data independently and with ease in almost any location. Integrating the 3D living environment with human movement data has not been achieved to date and will provide next-level human movement visualisation capability. It will extend our understanding of how individuals move within their home (beyond conventional clinical joint motion assessment), by revealing what tasks individuals do and how, as they physically interact with their own living space.

This digital twin technology is low cost and will advance telehealth and telerehabilitation, particularly for those in remote or disadvantaged communities. It will help improve design, development and testing of aged care infrastructure to ensure safe mobility and healthy ageing for Australians. More broadly, the monitoring, visualisation and data processing capability will have applications, including defence/military (monitoring front-line soldiers), elite sports (sports training and game strategy), film and animation, and space/aerospace/naval industries (vehicle design, and habitual motion monitoring in confined spaces).

The tasks required to achieve this project’s objective are:

Aim 1: Develop a ‘digital twin’ framework of an individual in their home setting by combining accurate human motion from IMUs, UWB data and 3D digital mapping

Aim 2: Discover and validate algorithms to calculate joint angles using IMUs, and automatically classify activities of daily living in the home using the digital twin, employing the minimum number of IMUs possible.

Aim 3: Deploy the use of wearable motion measurement technology and 3D indoor digital mapping for the home and residential aged care facility of older adults for real-time mobility measurement, visualisation and data logging over a long-term continuous data acquisition period.

Our proposed digital twin computational modelling framework will enable:

(i) Fast and accurate machine learning algorithms that convert raw IMU data acquired from a subject (linear acceleration and angular velocity), into joint angles. Joint angles are the standardised clinically relevant measure to quantify inter-segment angles at a joint (e.g. joint flexion and axial rotation) and are critically important for interpreting joint function and consolidating and curating data, for instance, across different subjects, laboratories or motion measurement modalities. This capability has applications in discerning onset or progression of musculoskeletal conditions, evaluating fall risk, and assessing the effectiveness of orthopaedic surgery or conservative intervention such as physiotherapy.

(ii) Generation of 3D geospatial data of a person’s home and integration of IMU and UWB data to achieve graphic visualisation of an individual (avatar) virtually moving and interacting with their home environment. Visualisation of whole-body motion using our digital twin technology will provide a leap-forward in remote human movement monitoring capability using wearables. Our user-friendly framework will be deployed on an iPad, allowing clinicians, family, care-givers and subjects themselves to collect motion data without technical expertise.

(iii) Real-time quantification of mobility over extended periods in the home, including classification of specific activities of daily living, as well as activity frequency and performance. Continuous human movement measurement in the home over larger time durations (weeks, months), and intuitive consolidation and interpretation of the resulting ‘big data’, will provide definitive measures of habitual movement at both the level of the overall motor task, and specific joint movement. It will reveal the way specific motor tasks of daily living are carried out over time, and generate new knowledge of how people adapt their physical movement patterns to their living environment.