Multimodal Content Analysis for Enhanced User Experience

Authors

  • Xiang Li

    International College of Digital Innovation, Chiang Mai University, Chiang Mai, Thailand

  • Tamprasirt Anukul

    International College of Digital Innovation, Chiang Mai University, Chiang Mai, Thailand

  • Fangli Ying

    International College of Digital Innovation, Chiang Mai University, Chiang Mai, Thailand

How to Cite

Li, X., Anukul, T. ., & Ying, F. . (2025). Multimodal Content Analysis for Enhanced User Experience. International Journal of Basic and Applied Sciences, 14(SI-3), 63-72. https://doi.org/10.14419/txmrrk92

Received date: April 29, 2025

Accepted date: July 22, 2025

Published date: August 15, 2025

DOI:

https://doi.org/10.14419/txmrrk92

Keywords:

Improving the Quality of the Platform; Interaction of Participants; Interactive Systems; Multimodal Review; Study of Materials

Abstract

The aim of the study was to develop methods of multimodal content analysis to improve the user experience in interactive systems. The study reviewed existing approaches to multimodal content analysis, established criteria for developing new methods, and provided specific examples of practical application of the developed methods in various fields, demonstrating their effectiveness and potential in real-world conditions. The main results consisted of the development of methods, in particular, integration and synchronization of modalities, which can demonstrate high efficiency in medical diagnostics. In turn, the personalization of user experience in streaming services increases user satisfaction through relevant recommendations, while ensuring data privacy and security meet modern regulatory requirements in the healthcare sector. The paper also examines the integration of modalities, which includes convolutional neural networks for image and video processing, recurrent neural networks for text and audio processing, and attention mechanisms for highlighting important parts of the data. Multimodal analysis involves the processing and integration of data from different sources, which provides a complete picture to improve the analysis. The developed methods outperform most existing approaches, providing higher accuracy, speed, robustness to noise, and incomplete data.

References

  1. Ahmad N, 2024. Revolutionizing ERP integration: AI-powered solutions for effortless usability and enhanced user experience. http://dx.doi.org/10.13140/RG.2.2.33419.30247
  2. Alzubi TM, Alzubi JA, Singh A & Alzubi OA 2023. A multimodal human-computer interaction for smart learning system. International Journal of Human-Computer Interaction. https://doi.org/10.1080/10447318.2023.2206758
  3. Androshchuk A 2023. Cyber protection in programming languages Java and C#. Technologies and Engineering, 24(3):9–14. https://doi.org/10.30857/2786-5371.2023.3.1
  4. Aviv I, Gafni R, Sherman S, Aviv B, Sterkin A & Bega E 2023. Cloud infrastructure from python code–breaking the barriers of cloud deployment. In: European Conference on Software Architecture, ECSA (pp.1–8). Available at: https://www.researchgate.net/profile/Itzhak-Aviv/publication/373897534_Cloud_Infrastructure_from_Python_Code_-breaking_the_Barriers_of_Cloud_Deployment/links/6501edd2808f9268d573dea5/Cloud-Infrastructure-from-Python-Code-breaking-the-Barriers-of-Cloud-Deployment.pdf
  5. Awada IA, Mocanu I & Florea AM 2018. Exploiting multimodal interfaces in eLearning systems. International Scientific Conference: eLearning and Software for Education Conference, 2:174–181. https://doi.org/10.12753/2066-026x-21-094
  6. Azieva G, Kerimkhulle S, Turusbekova U, Alimagambetova A & Niyazbekova S 2021. Analysis of access to the electricity transmission network using information technologies in some countries. E3S Web of Conferences, 258:11003. https://doi.org/10.1051/e3sconf/202125811003
  7. Bezshyyko O, Dolinskii A, Bezshyyko K, Kadenko I, Yermolenko R & Ziemann V 2008. PETAG01: A program for the direct simulation of a pel-let target. Computer Physics Communications, 178(2):144–155. https://doi.org/10.1016/j.cpc.2007.07.013
  8. Bisenovna KA, Ashatuly SA, Beibutovna LZ, Yesilbayuly KS, Zagievna AA, Galymbekovna MZ & Oralkhanuly OB 2024. Improving the effi-ciency of food supplies for a trading company based on an artificial neural network. International Journal of Electrical and Computer Engineering, 14(4):4407–4417. https://doi.org/10.11591/ijece.v14i4.pp4407-4417
  9. Bouchey B, Castek J & Thygeson J 2021. Multimodal learning. In: Learning Environments in STEM Higher Education (pp.35–54). Cham: Springer. https://doi.org/10.1007/978-3-030-58948-6_3
  10. Breitschaft SJ, Pastukhov A & Carbon C-C 2021. Where's my button? Evaluating the user experience of surface haptics in featureless automotive user interfaces. IEEE Transactions on Haptics, 15(2):292–303. https://doi.org/10.1109/TOH.2021.3131058
  11. Chaiyaphum T 2022. Developing supply chain innovation model for operation and adaptation of logistics service providers in the disruptive era. Available at: http://www.dspace.spu.ac.th/handle/123456789/8697
  12. Chiu WHK, Ko WSK, Cho WCS, Hui SYJ, Chan WCL & Kuo MD 2024. Evaluating the diagnostic performance of large language models on complex multimodal medical cases. Journal of Medical Internet Research, 26:e53724. https://doi.org/10.2196/53724
  13. Choudhary M, Singh S & Rathore SS 2024. Beyond text: Multimodal credibility assessment approaches for online user-generated content. ACM Transactions on Intelligent Systems and Technology. https://doi.org/10.1145/3673236
  14. Dewangan O & Mishra M 2023. TLMAEMS: Design of an efficient transfer learning model with auto encoders for multimodal sentiment analysis via deep sentiment networks. European Chemical Bulletin, 12(Special Issue 5):1453–1467. http://dx.doi.org/10.48047/ecb/2023.12.si5.1362023.30/05/2023
  15. Fang H, Liang J & Sha L 2024. Enhanced multimodal recommendation systems through reviews integration. Research Square. https://doi.org/10.21203/rs.3.rs-4333408/v1
  16. Henrique BM, Sobreiro VA & Kimura H 2023. Practical machine learning: Forecasting daily financial markets directions. Expert Systems with Ap-plications, 233:120840. https://doi.org/10.1016/j.eswa.2023.120840
  17. Heraz A, Chen L & Bhyravabhottla KKA 2024. Enhancing user receptivity in e-learning environments through gesture-based emotion tracking as a pathway to mental well-being and health equity. JMIR Formative Research. https://doi.org/10.2196/preprints.57293
  18. Iklassova K, Aitymova A, Kopnova O, Shaporeva A, Abildinova G, Nurbekova Z, Almagambetova L, Gorokhov A & Aitymov Z 2024. Ontology modeling for the automation of questionnaire data processing. Eastern-European Journal of Enterprise Technologies, 5(2-131):36–52. https://doi.org/10.15587/1729-4061.2024.314129
  19. Ince EB, Cha K & Cho J 2024. Towards a conceptual model of users’ expectations of an autonomous in-vehicle multimodal experience. Human Behavior and Emerging Technologies, 2024(1):7418597. https://doi.org/10.1155/2024/7418597
  20. Jagnade G, Sable S & Ikar M 2023. Advancing multimodal fusion in human-computer interaction: Integrating eye tracking, lips detection, speech recognition, and voice synthesis for intelligent cursor control and auditory feedback. International Conference on Computing Communication and Networking Technologies. https://doi.org/10.1109/ICCCNT56998.2023.10306457
  21. Kamil MAF 2023. User Experience analysis of linkedin social media using usability metric for user experience (UMUX). Journal of Information Engineering and Educational Technology, 7(2):78–82. https://doi.org/10.26740/jieet.v7n2.p78-82
  22. Kamtab P & Wiriyanon T 2022. Design a digital multimodal user interface for senior citizens. Journal of Education Studies, 50(1):6. https://doi.org/10.58837/chula.educu.50.1.5
  23. Kerimkhulle S & Aitkozha Z 2017. A criterion for correct solvability of a first order difference equation. AIP Conference Proceedings, 1880:040016. https://doi.org/10.1063/1.5000632
  24. Kerimkhulle S, Kerimkulov Z, Bakhtiyarov D, Turtayeva N & Kim J 2021. In-Field Crop-Weed Classification Using Remote Sensing and Neural Network. In: SIST 2021 - 2021 IEEE International Conference on Smart Information Systems and Technologies (9465970). https://doi.org/10.1109/SIST50301.2021.9465970.
  25. Kerimkhulle S, Obrosova N, Shananin A & Tokhmetov A 2023. Young Duality for Variational Inequalities and Nonparametric Method of Demand Analysis in Input–Output Models with Inputs Substitution: Application for Kazakhstan Economy. Mathematics, 11(19):4216. https://doi.org/10.3390/math11194216
  26. Khoda V, Leshchuk N, Topalov A, Robotko S, Klymenko O & Nekrasov S 2024. Computerized Lathe Control System based on Internet of Things Technology. In: Proceedings - International Conference on Advanced Computer Information Technologies, ACIT (pp.674–677). Ceske Budejovice: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ACIT62333.2024.10712548
  27. Khrulov M 2025. Analysis of the current state of computer systems in the field of virtual healthcare. Technologies and Engineering, 26(1): 55–66. https://doi.org/10.30857/2786-5371.2025.1.5
  28. Kolodiziev O, Krupka M, Shulga N, Kulchytskyy M & Lozynska O 2021. The level of digital transformation affecting the competitiveness of banks. Banks and Bank Systems, 16(1):81–91. https://doi.org/10.21511/bbs.16(1).2021.08
  29. Konurbayeva ZT, Madiyarova ES & Rakhimberdinova MU 2015. Algorithm for generating competitive potential of engineering within the regional economy. Actual Problems of Economics, 168(6):236–247.
  30. Kovalchuk D 2025. Utilising large language models for automated real-time cyber threat analysis. Bulletin of Cherkasy State Technological Univer-sity, 30(1):48–58. https://doi.org/10.62660/bcstu/1.2025.48
  31. Maksymov A & Tryus Yu 2025. Information technology for solving the multi-criteria decision-making problem using the modified Fuzzy TOPSIS method. Bulletin of Cherkasy State Technological University, 30(1):91–106. https://doi.org/10.62660/bcstu/1.2025.91
  32. Manmothe SR & Jadhav JR 2024. Integrating multimodal data for enhanced analysis and understanding: Techniques for sentiment analysis and cross-modal retrieval. Journal Of Advanced Zoology, 45(S4):22–28. https://doi.org/10.53555/jaz.v45iS4.4144
  33. Mao X & Zhang J 2021. On the power of SVD in the stochastic block model. https://doi.org/10.48550/arXiv.2309.15322
  34. Marques J & Marques RP 2023. Digital transformation of the hotel industry. Cham: Springer.
  35. Meier S 2022. Digital storytelling: A didactic approach to multimodal coherence. Frontiers in Communication, 7:906268. https://doi.org/10.3389/fcomm.2022.906268
  36. Moradi P 2022. Ageing with technology – An extended multimodal design study of active ageing users’ emotional experience with social robots. Available at: https://hdl.handle.net/10292/15512
  37. Mullick T, Shaaban S, Radovic A & Doryab A 2024. Framework for ranking machine learning predictions of limited, multimodal, and longitudinal behavioral passive sensing data: Combining user-agnostic and personalized modeling. JMIR AI, 3:e47805. https://doi.org/10.2196/47805
  38. Naegelin M, Weibel RP, Kerr JI, Schinazi VR, La Marca R, Von Wangenheim F, Hoelscher C & Ferrario A 2023. An interpretable machine learn-ing approach to multimodal stress detection in a simulated office environment. Journal of Biomedical Informatics, 139:104299. https://doi.org/10.1016/j.jbi.2023.104299
  39. Orazbayev B, Zhumadillayeva A, Kabibullin M, Crabbe MJC, Orazbayeva K & Yue X 2023. A Systematic Approach to the Model Development of Reactors and Reforming Furnaces With Fuzziness and Optimization of Operating Modes. IEEE Access, 11:74980–74996. https://doi.org/10.1109/ACCESS.2023.3294701
  40. Panwar V 2024. Leveraging progressive web apps (PWAs) for enhanced user experience and performance: A comprehensive analysis. International Journal of Management IT and Engineering, 14(4):31–43.
  41. Pavlova D, Dovramadjiev T, Daskalov D, Mirchev N, Peev I, Radeva J, Dimova R, Kavaldzhieva K, Mrugalska B, Szabo G, Kandioglou A 2024. 3D Design of a Dental Crown with Artificial Intelligence Based in Cloud Space. Lecture Notes in Networks and Systems, 817:437–445. https://doi.org/10.1007/978-981-99-7886-1_37
  42. Porkodi S & Raman AM 2025. Success of cloud computing adoption over an era in human resource management systems: a comprehensive meta-analytic literature review. Management Review Quarterly, 75(2):1041–1075. https://doi.org/10.1007/s11301-023-00401-0
  43. Pradipta RAH, Widiani HN, Tama DA, Ramdani AL, Komariansyah KK, Nurfajri MI, Barus I & Fami A 2024. Analyzing user experience in the M-TIX application using the user experience questionnaire method. International Journal of Multilingual Education and Applied Linguistics, 1(3):61–71. https://doi.org/10.61132/ijmeal.v1i3.62
  44. Ragan LC & Villarin LJR 2021. Emergent guiding principles for STEM education. In: Innovative Learning Environments in STEM Higher Educa-tion (pp.107–119). Cham: Springer. https://doi.org/10.1007/978-3-030-58948-6_6
  45. Ramtohul A & Khedo KK 2023. An engaging user experience framework for mobile augmented reality. Research Square. https://doi.org/10.21203/rs.3.rs-3080680/v1
  46. Rexhepi BR, Kumar A, Gowtham MS, Rajalakshmi R, Paikaray MD & Adhikari PK 2023. An Secured Intrusion Detection System Integrated with the Conditional Random Field For the Manet Network. International Journal of Intelligent Systems and Applications in Engineering, 11(3s):14–21.
  47. Richas F & Kamal I 2024. Comparison analysis of user experience on m-BCA and BRImo mobile banking applications using the user experience questionnaire (UEQ) method. International Journal of Management and Business Economics, 2(3):47–51. https://doi.org/10.58540/ijmebe.v2i3.551
  48. Rust RT 2020. The future of marketing. International Journal of Research in Marketing, 37(1):15–26. https://doi.org/10.1016/j.ijresmar.2019.08.002
  49. Salim F 2023. Identifying user experience (UX) elements and emotional experiences on smartwatch devices. http://dx.doi.org/10.22677/THESIS.200000726145
  50. Santhosh P, Ashweej A, Agarwal G & Raj GD 2024. Personalized product ranking system for enhanced user experience. MATEC Web of Confer-ences, 392:01077. https://doi.org/10.1051/matecconf/202439201077
  51. Saputra FA, Yeza MP, Fahrezy MF, Fami A & Barus IRG 2024. Article comparing user experience maxim and in drive applications using the user experience questionnaire. ILKOMNIKA Journal of Computer Science and Applied Informatics, 6(1):79–90. https://doi.org/10.28926/ilkomnika.v6i1.614
  52. Smith-Harvey J & Aguayo C 2024. Modes of meaning: Multimodal media & 4E+ cognition in tech-enhanced learning. Pacific Journal of Technolo-gy Enhanced Learning, 6(1):10–11. https://doi.org/10.24135/pjtel.v6i1.181
  53. Taylor DL, Yeung M & Bashet AZ 2021. Personalized and adaptive learning. In: Innovative Learning Environments in STEM Higher Education (pp.17–34). Cham: Springer. https://doi.org/10.1007/978-3-030-58948-6_2
  54. Tkachenko O, Chechet A, Chernykh M, Bunas S & Jatkiewicz P 2025. Scalable Front-End Architecture: Building for Growth and Sustainability. Informatica (Slovenia), 49(1):137–150. https://doi.org/10.31449/inf.v49i1.6304
  55. Tokarieva KS, Kovalchuk OY, Kolesnikov AP, Dzyurbel AD, Bodnar-Petrovska OB & Predmestnikov OG 2024. The use of ai-language models in judicial proceedings: information and legal aspects. Revista Juridica, 2(78):520–538. https://doi.org/10.26668/revistajur.2316-753X.v2i78.6928
  56. UI/UX staffing 2019. Available at: https://evergreens.com.ua/ua/articles/ui-and-ux-personalization.html
  57. Varanitskyi D, Rozkolodko O, Liuta M, Zakharova M & Hotunov V 2024. Analysis of data protection mechanisms in cloud environments. Tech-nologies and Engineering, 25(1):9–16. https://doi.org/10.30857/2786-5371.2024.1.1
  58. Vijayakumar T 2024. Enhancing user experience through emotion-aware interfaces: A multimodal approach. Journal of Innovative Image Pro-cessing, 6(1):27–39. https://doi.org/10.36548/jiip.2024.1.003
  59. Wang C, Yu X, Xu L, Wang Z & Wang W 2022. Multimodal semantic communication accelerated bidirectional caching for 6G MEC. Future Gen-eration Computer Systems, 140:225–237. https://doi.org/10.1016/j.future.2022.10.036
  60. What is Data Protection and Privacy? 2024. Available at: https://cloudian.com/guides/data-protection/data-protection-and-privacy-7-ways-to-protect-user-data/#:~:text=By%20protecting%20data%2C%20companies%20can,%2C%20encryption%2C%20and%20endpoint%20protection
  61. Wildfeuer J & Lehmann C 2024. Drawing multimodality's bigger picture: Metalanguages and corpora for multimodal analyses. Lausanne: Frontiers Media SA. https://doi.org/10.3389/978-2-8325-5196-7
  62. Xhafka E, Sinoimeri D & Teta J 2024. Evaluating the Impact of E-Governance on Public Service Improvement in Albania: A Quantitative Analysis. Sustainability (Switzerland), 16(24):10896. https://doi.org/10.3390/su162410896
  63. Xu Y, Lin Y-S, Zhou X & Shan X 2023. Utilizing emotion recognition technology to enhance user experience in real-time. Computing and Artifi-cial Intelligence, 2(1):1388. https://doi.org/10.59400/cai.v2i1.1388
  64. Yasinthorn Y & Celadon S 2023. Security enhancement and performance optimization for WordPress-based Websites, a case study of WorayuthIT website. Available at: http://digital.csmsu.net:8080/library/handle/123456789/218
  65. Yasmin S & Jamal CQ 2023. Multisensory modeling of tabular data for enhanced perception and immersive experience. In: Advances in Visual Computing (pp.452–465). Cham: Springer. https://doi.org/10.1007/978-3-031-47966-3_36
  66. Zhang Y & Tan H 2021. Effects of multimodal warning types on driver’s task performance, physiological data and user experience. In: Cross-Cultural Design (pp.304–315). Cham: Springer. https://doi.org/10.1007/978-3-030-77080-8_25
  67. Zhetenbayev N, Zhauyt A, Balbayev G & Shingissov B 2022. Robot device for ankle joint rehabilitation: A review. Vibroengineering Procedia, 41:96–102. https://doi.org/10.21595/vp.2022.22507
  68. Ziker C, Truman B & Dodds H 2021. Cross Reality (XR): Challenges and opportunities across the spectrum. In: Innovative Learning Environments in STEM Higher Education (pp.55–77). Cham: Springer. https://doi.org/10.1007/978-3-030-58948-6_4

Downloads

How to Cite

Li, X., Anukul, T. ., & Ying, F. . (2025). Multimodal Content Analysis for Enhanced User Experience. International Journal of Basic and Applied Sciences, 14(SI-3), 63-72. https://doi.org/10.14419/txmrrk92

Received date: April 29, 2025

Accepted date: July 22, 2025

Published date: August 15, 2025