Prospects for diagnosis of movement disorders using computer vision methods based on a mobile device

Authors

  • M.A. Andrushchenko Kharkiv National University of Radio Electronics
  • K.G. Selivanova Kharkiv National University of Radio Electronics

DOI:

https://doi.org/10.31649/1681-7893-2024-48-2-96-103

Keywords:

healthcare, healthcare systems, physical rehabilitation, medical information systems, computer vision, mobile applications, markerless motion tracking, diagnostics, movement disorders.

Abstract

The research work investigated markerless video analysis technologies based on the analysis of the relative position of heterogeneous parts of object images in successive frames. Modern video motion capture systems are ready-made clusters of points in the form of plates with four markers for long limb segments, a ‘cap’ with markers for the head, etc. The main models used in mobile devices to estimate the movements of the upper limbs and the biomechanics of joint movement in real time were studied, namely PoseNet, MoveNet Thunder, MoveNet Lightning, and BlazePose in Light, Full, and Heavy versions. The models were evaluated for key characteristics such as speed, device impact, and support for hardware acceleration.

Author Biographies

M.A. Andrushchenko, Kharkiv National University of Radio Electronics

graduate student, Department of Biomedical Engineering

K.G. Selivanova , Kharkiv National University of Radio Electronics

Ph.D., Associate Professor, Department of Biomedical Engineering

References

Kim, Bokkyu, and Christopher Neville. "Accuracy and Feasibility of a Novel Fine Hand Motor Skill Assessment Using Computer Vision Object Tracking." Scientific Reports 13, no. 1 (2023): 1-14. doi: https://doi.org/10.1038/s41598-023-29091-0.

Gupta, S., & White, M. (2021, November 8). Improved On-Device ML on Pixel 6, with Neural Architecture Search. Google Research Blog. Retrieved from https://blog.research.google/2021/11/improved-on-device-ml-on-pixel-6-with.html

Apple Machine Learning Research. (n.d.). Deploying Transformers on the Apple Neural Engine. Retrieved from https://machinelearning.apple.com/research/neural-engine-transformers

Bazarevsky Valentin, Grishchenko Ivan, Raveendran Karthik, etc. (2020). BlazePose: On-device Real-time Body Pose tracking. arXivDOI: doi: https://doi.org/10.48550/arXiv.2006.10204

Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, Andrei Tkachenka, George Sung, Chuo-Ling Chang, Matthias Grundmann. MediaPipe Hands: On-device Real-time Hand Tracking. arXiv, 2020. doi: https://doi.org/10.48550/arXiv.2006.10214

TensorFlow. Pose Detection. GitHub. URL: https://github.com/tensorflow/tfjs-models/tree/master/pose-detection

Votel, R., & Li, N. (2021, May 17). Next-Generation Pose Detection with MoveNet and TensorFlow.js. The TensorFlow Blog. Retrieved May 30, 2024, from https://blog.tensorflow.org/2021/05/next-generation-pose-detection-with-movenet-and-tensorflowjs.html

Google AI Edge. (n.d.). Pose Landmarker. GitHub. Retrieved May 30, 2024, from https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/pose_landmarker

TensorFlow. (n.d.). Pose Estimation iOS Demo. GitHub. Retrieved May 30, 2024, from https://github.com/tensorflow/examples/tree/master/lite/examples/pose_estimation/ios

Selivanova K. and Avrunin O., (2022). Method of Hand Movement Disorders Determination based on the Surgeon's Laparoscopic Video Recording, 2022 IEEE 3rd KhPI Week on Advanced Technology (KhPIWeek), Kharkiv, Ukraine, p. 1-4, doi: https://doi.org/10.1109/KhPIWeek57572.2022.9916457

Selivanova Karina, Avrunin Oleg, etc. (14 October 2020). The tracking system of a three-dimensional position of hand movement for tremor detection", Proc. SPIE 11581, Photonics Applications in Astronomy, Communications, Industry, and High Energy Physics Experiments 2020, 115810I doi: https://doi.org/10.1117/12.2580330

Tymkovych, M., Avrunin, O., Selivanova, K., Kolomiiets, A., Bednarchyk, T., & Smailova, S. (2024). CORRESPONDENCE MATCHING IN 3D MODELS FOR 3D HAND FITTING. Informatyka, Automatyka, Pomiary W Gospodarce I Ochronie Środowiska, 14(1), 78–82. DOI: https://doi.org/10.35784/iapgos.5498

Tymkovych Maksym, Selivanova Karina, Avrunin Oleg, etc. (20 December 2023). 3D scanning technologies by optical RealSense cameras for SIREN-based 3D hand representation", Proc. SPIE 12985, Optical Fibers and Their Applications 2023, 129850O https://doi.org/10.1117/12.3022737

Selivanova Karina, Avrunin Oleg, etc. (6 November 2019). Biometric hand tremor identification on graphics tablet", Proc. SPIE 11176, Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments 2019, 111762H DOI: https://doi.org/10.1117/12.2536421

Park KW, Mirian MS, McKeown MJ. (2024). Artificial intelligence-based video monitoring of movement disorders in the elderly: a review on current and future landscapes. Singapore Med J. Mar 1;65(3):141-149. doi: 10.4103/singaporemedj.SMJ-2023-189. Epub 2024 Mar 26. PMID: 38527298; PMCID: PMC11060643

Morinan G, Dushin Y, Sarapata G, Rupprechter S, Peng Y, Girges C, et al. Computer vision quantification of whole-body Parkinsonian bradykinesia using a large multi-site population. Npj Park Dis. 2023;9:10

Guo Z, Zeng W, Yu T, Xu Y, Xiao Y, Cao X, et al. Vision-based finger tapping test in patients with Parkinson’s disease via spatial-temporal 3D hand pose estimation. IEEE J Biomed Health Inform. 2022;26:3848–59

Wang X, Garg S, Tran SN, Bai Q, Alty J. (2021). Hand tremor detection in videos with cluttered background using neural network based approaches. Health Inf Sci Syst. 9:30

Lam WWT, Tang YM, Fong KNK. (2023). A systematic review of the applications of markerless motion capture (MMC) technology for clinical measurement in rehabilitation. J NeuroEngineering Rehabil.; 20:57

Cao Z, Hidalgo G, Simon T, Wei S-E, Sheikh Y. OpenPose: Realtime multi-person 2D pose estimation using part affinity fields. 2018. DOI: 10.48550/ARXIV.1812.08008

Gomes N, Pato M, Lourenço AR, Datia N. (2023). A survey on wearable sensors for mental health monitoring. Sensors.; 23:1330

Khan MH, Schneider M, Farid MS, Grzegorzek M. (2018 Sep 21). Detection of Infantile Movement Disorders in Video Data Using Deformable Part-Based Model. Sensors (Basel).;18 (10):3202. doi: https://doi.org/10.3390/s18103202. PMID: 30248968; PMCID: PMC6210538.

Colyer S.L., Evans M., Cosker D.P., Salo A.I. (2018). A Review of the Evolution of Vision-Based Motion Analysis and the Integration of Advanced Computer Vision Methods Towards Developing a Markerless System. Sports Med. Open.;4:24. doi: https://doi.org/10.1186/s40798-018-0139-y.

Mehrizi R., Peng X., Xu X., Zhang S., Metaxas D., Li K. (2018). A computer vision based method for 3D posture estimation of symmetrical lifting. J. Biomech. 69: 40–46. doi: https://doi.org/10.1016/j.jbiomech.2018.01.012

Penelle B., Debeir O. (11–13 September 2013). Human motion tracking for rehabilitation using depth images and particle filter optimization; Proceedings of the 2013 2nd International Conference on Advances in Biomedical Engineering (ICABME); Tripoli, Lebanon.; pp. 211–214.

Tseng Y.C., Wu C.H., Wu F.J., Huang C.F., King C.T., Lin C.Y., Sheu J.P., Chen C.Y., Lo C.Y., Yang C.W., et al. (18–20 May 2009). A wireless human motion capturing system for home rehabilitation; Proceedings of the 10th International Conference on Mobile Data Management (MDM’09): Systems, Services and Middleware; Taipei, Taiwan.; pp. 359–360.

Williams S, McKay J, Bernhard D, et al. (2022). Quantitative motion analysis and clinical characteristics of Holmes tremor as compared to other tremor types (S32.008). Neurology.; 98(Suppl 18).

Boccagni C, Carpaneto J, Micera S, Bagnato S, Galardi G. (2008). Motion analysis in cervical dystonia. Neurol Sci.; 29(6): 375-381. DOI: https://doi.org/10.1007/s10072-008-1033-z

B, Agostino R, Bologna M, et al. (2008). Fast voluntary neck movements in patients with cervical dystonia: a kinematic study before and after therapy with botulinum toxin type A. Clin Neurophysiol.;119(2):273-280. DOI: https://doi.org/10.1016/j.clinph.2007.10.007

Belic M, Bobic V, Badza M, Solaja N, Duric-Jovicic M, Kostic VS. (2019). Artificial intelligence for assisting diagnostics and assessment of Parkinson’s disease: a review. Clin Neurol Neurosurg.;184:105442. doi: https://doi.org/10.1016/j.clineuro.2019.105442.

Wójcik Waldemar, Smolarz Andrzej (2017). Information Technology in Medical Diagnostics, July 11, 2017 by CRC Press, 210 Pages.

Highly linear Microelectronic Sensors Signal Converters Based on Push-Pull Amplifier Circuits / edited by Waldemar Wojcik and Sergii Pavlov, Monograph, (2022) NR 181, Lublin, Comitet Inzynierii Srodowiska PAN, 283 Pages. ISBN 978-83-63714-80-2

Pavlov Sergii, Avrunin Oleg, Hrushko Oleksandr, and etc. (2021). System of three-dimensional human face images formation for plastic and reconstructive medicine // Teaching and subjects on bio-medical engineering Approaches and experiences from the BIOART-project Peter Arras and David Luengo (Eds.), , Corresponding authors, Peter Arras and David Luengo. Printed by Acco cv, Leuven (Belgium). - 22 P. ISBN: 978-94-641-4245-7.

Pavlov S.V., Avrunin O.G., etc. (2019). Intellectual technologies in medical diagnosis, treatment and rehabilitation: monograph / [S. In edited by S. Pavlov, O. Avrunin. - Vinnytsia: PP "TD "Edelweiss and K", 260 p. ISBN 978-617-7237-59-3.

Romanyuk, O., Zavalniuk, Y., Pavlov, S., etc. (2023). New surface reflectance model with the combination of two cubic functions usage, Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Srodowiska, , 13(3), pp. 101–106

Kukharchuk, Vasyl V., Sergii V. Pavlov, Volodymyr S. Holodiuk, Valery E. Kryvonosov, Krzysztof Skorupski, Assel Mussabekova, and Gaini Karnakova. (2022). "Information Conversion in Measuring Channels with Optoelectronic Sensors" Sensors 22, no. 1: 271. https://doi.org/10.3390/s22010271.

Downloads

Abstract views: 2

Published

2024-11-19

How to Cite

[1]
M. Andrushchenko and K. Selivanova, “Prospects for diagnosis of movement disorders using computer vision methods based on a mobile device”, Опт-ел. інф-енерг. техн., vol. 48, no. 2, pp. 96–103, Nov. 2024.

Issue

Vol. 48 No. 2 (2024)

Section

Systems Of Technical Vision And Artificial Intelligence, Image Processing And Pattern Recognition

Metrics

Downloads

Download data is not yet available.

License

Автори, які публікуються у цьому журналі, погоджуються з наступними умовами:

  1. Автори залишають за собою право на авторство своєї роботи та передають журналу право першої публікації цієї роботи на умовах ліцензії Creative Commons Attribution License, котра дозволяє іншим особам вільно розповсюджувати опубліковану роботу з обов'язковим посиланням на авторів оригінальної роботи та першу публікацію роботи у цьому журналі.
  2. Автори мають право укладати самостійні додаткові угоди щодо неексклюзивного розповсюдження роботи у тому вигляді, в якому вона була опублікована цим журналом (наприклад, розміщувати роботу в електронному сховищі установи або публікувати у складі монографії), за умови збереження посилання на першу публікацію роботи у цьому журналі.
  3. Політика журналу дозволяє і заохочує розміщення авторами в мережі Інтернет (наприклад, у сховищах установ або на особистих веб-сайтах) рукопису роботи, як до подання цього рукопису до редакції, так і під час його редакційного опрацювання, оскільки це сприяє виникненню продуктивної наукової дискусії та позитивно позначається на оперативності та динаміці цитування опублікованої роботи (див. The Effect of Open Access).