The Medical Metaverse, Part 1: Introduction, Definitions, and New Horizons for Neuropsychiatry
Editor’s note: This report is the first of a two-part series introducing the concepts of the medical metaverse. Part 1 covers the basic definitions and its current utility in mental health care. Part 2 will address the metaverse as it relates to clinical neurosciences and will appear in a subsequent issue of The Journal of Neuropsychiatry and Clinical Neurosciences.
The “metaverse” was initially used in literature by the science fiction writer Neal Stephenson in his 1992 dystopian novel Snow Crash (8, 9). The metaverse denotes a digital platform as a new and alternative environment manifested through digital content delivered via advanced technological devices employing artificial intelligence (AI) (2). This platform encompasses highly immersive and collaborative environments based on three-dimensional and real-time digital worlds in which multiple users can engage in activities (e.g., social, economic, and cultural). The users interact via avatars in versatile settings without physical limitations (10, 11). A metaverse research organization, the Acceleration Studies Foundation (ASF), proposed thinking about the metaverse as a link or connection point between the real and virtual worlds (2, 12). In the metaverse, however, users are not limited by the typical rules of nature (e.g., age, gender, appearance, and species). Freedom from other restrictions, such as time and space, is also enabled by virtual reality (VR) technologies (both hardware and software). Thus, individuals can engage in a wide array of activities that are impossible in real life (2).
Metaverse Technologies
The cybernetic progression continues to advance with support from major technology companies. According to the literature, within the next 5 years tens of billions of dollars will be invested to expand currently available platforms (13). This rapid progression of core technologies, including AI, “big data,” the Internet of things (IoT), edge computing, blockchain, digital twins (DT), extended reality (XR), and high-speed 5G Internet networks, underpins the metaverse’s existence, advancement, and sustainability (14–18). The coronavirus disease 2019 (COVID-19) pandemic stimulated additional advances (2). Therefore, it is necessary to understand the concept of the metaverse, its nuances, and its applications for medical education and health care delivery (19).
According to the ASF, the metaverse roadmap distinguishes technologies along two separate dimensions, augmentation versus simulation and intimate versus external, to characterize four general types of metaverse technologies: augmented reality (AR), lifelogging, mirror worlds, and virtual worlds (1). In the metaverse, augmentation and simulation are defined by whether the information is presented in physical reality (via AR) or in VR (via simulation) (Figures 1 and 2) (1, 2). Conversely, intimate (identity-focused) technologies are centered on the actions of the individual or object; for example, the user or object (i.e., semi-intelligent object) engages by using an avatar or digital profile. In contrast, the external (world-focused) technologies are centered on the outside world, and they typically focus on the control of aspects of the surrounding world, such as the ability to provide information about the conditions of the external environment and how the user can control these conditions (Figures 1 and 2) (1, 2).
FIGURE 1. Quadrant-based metaverse diagram.
The horizontal axis includes metaverse technologies that facilitate the empirical relationship between technology and user. The left side reflects the external environment, and the personalized experience of the user is intensified toward the right, intimate side (1, 2). In the external milieu, the surrounding world is emphasized, and this outward perspective is user centered and provides guidance to the user on how to engage with these environmental features. The intimate element of the metaverse provides an inward perspective and focuses on individualized behaviors guided by customized data. The vertical axis includes technologies that exemplify the relationship between reality and innovation. Augmentation increases the human sensory experience by incorporating new functions into the real-world environment via visual, auditory, and other sensory stimuli. Conversely, simulation duplicates the real world by modeling reality in a virtual world (1, 2).
FIGURE 2. Four categories of metaverse technology.
A. Augmented reality (AR). AR uses virtual imaging and fetching technologies to integrate digital information into the real-world environment (3). B. Lifelogging. This technology obtains digital records of the user’s personal data (e.g., heart rate, breathing, and sleep patterns) by capturing, storing, and sharing information by using multimodal digital sensors integrated in high-tech devices, such as smart watches, smart phones, and wearables (2, 4). C. Mirror worlds. In this type of technology, the real world is depicted (i.e., simulated) with digital innovations (e.g., satellites and interactive software) to reflect the characteristics of the external world in the cybernetic world. These simulations replicate an accurate version of the real-world structures and human environments (i.e., mirror duplicates). Google Earth, Foldit, and Microsoft Teams are examples of digital applications simulating mirror worlds (2, 5, 6). D. Virtual worlds. The virtual world incorporates virtual reality (VR) technology. In VR, the user is fully immersed and is completely blinded to the real environment; this immersion is facilitated by a head-mounted display (digital lenses) in which the user receives visual, auditory, and other sensory inputs (1, 2, 7).
COVER. Artistic depiction of the medical metaverse. On the right, various technologies of the digital biomedical world. On the left, the head-mounted display lenses and the brain symbolize the immersive experience of extended reality and its role in the metaverse.
All images created under the terms of the Creative Commons Attribution License. Created with VH Dissector, Canva, and BioRender.com.
Medical Metaverse
Metaverse technologies have opened new horizons for innovations in medicine. Medical education and health care practices are embracing technological advancements to improve patient care (2, 20–22). A wide range of subspecialties, such as cardiology, emergency medicine, gastroenterology, gynecology, oncology, ophthalmology, and radiology, are employing metaverse technologies (4, 22–26). Mental health disciplines are also benefiting from metaverse innovations, specifically XR technologies (VR, AR, and mixed reality) (27). VR technologies are among the most widely used advances in mental health interventions. VR exposure therapy (VRET) and AR exposure therapy have proven effective in treating specific phobias, posttraumatic stress disorder (PTSD), anxiety, attention-deficit hyperactivity disorder, substance-related disorders, depression, and eating disorders (20, 28–31).
VRET employs psychotherapeutic immersive paradigms that provide multisensory VR experiences to amplify the patient’s experiential engagement during treatment sessions (27). VRET interventions are effective in reducing PTSD symptoms of active-duty and combat soldiers after military trauma (32, 33). XR therapeutic benefits are based partly on its immersive capabilities that allow experiences at a digital multisensorial level and integrate a seamless human-computer interface (27). Deepak Chopra indicated that “the metaverse guides us past the notion that mind, body, and spirit are separate” because the metaverse incorporates technologies (e.g., AR and VR) to rectify one’s dysfunctional reality by using virtual (functional) reality (34). VR innovations allow for the reproduction of transcendental experiences (e.g., delivering joy, distraction, and positive affect in a soothing environment) (34–36). Higher-level cognitive functions (e.g., decision making and problem solving) may vary based on the body’s characteristics that the individual perceives as self-owned. A study performed by Banakou et al. used superintelligent (“Albert Einstein”) and average-intelligence virtual body avatars to illustrate this concept (37). Participants who embodied the Einstein avatar performed cognitively better than those who embodied an average-intelligence avatar of an age similar to their own (37). A similar influence of the owned body has been shown in social behavior studies. These experiments investigated whether acting through certain types of avatars (i.e., as players in video games) and the avatars’ actions could influence pro- or antisocial behaviors (194 subjects). Players choosing more heroic avatars were encouraged to behave in a more prosocial manner. The authors reported that after acting as a villain or superhero within the virtual world, participants modeled the behaviors of their chosen avatars when interacting with their fellow players in the real world. In addition, the investigators highlighted that role-taking activities supported behaviors consistent with the priming actions (i.e., activities of the heroic or villainous avatar) (38). The introduction of avatars may support treatment engagement during psychotherapy (39). Thus, the virtual world can serve as a vehicle for observation, simulation, and modeling, and avatars power these processes (38). Additional benefits of avatar-integrated therapies include increases in treatment seeking via anonymity, reductions in communication barriers, facilitation of expression and exploration of client identity, and improvements in interventional settings by allowing therapists to control and manipulate the treatment conditions within the virtual environment (39).
Metaverse technologies may establish an ideal scenario for mental health research (e.g., sense of body ownership) and the development of new therapies. The fact that real-life situations are difficult to replicate in the real world (i.e., during psychotherapy sessions) but can be safely recreated in cyberspace opens many possibilities for new therapeutic interventions (28). For example, VRET has proven effective in replicating human reactions (e.g., fear-inducing situations generating social anxiety) in real social environments (40, 41). Furthermore, VRET has been successfully used to reduce social anxiety associated with stuttering when speaking to those with authority or during public speaking (42).
The COVID-19 pandemic has resulted in significant lifestyle changes worldwide. By extension, health care systems have transformed traditional health education and clinical care settings in an effort to decrease face-to-face interactions. Metaverse technologies are an effective alternative for remote care of certain chronic conditions and biometrically enhance care models involving mental health treatment. These treatments include mindfulness and cognitive-behavioral therapy, either guided by a therapist or as a self-help program (43, 44). In addition, medical metaverse XR innovations may supplement current therapeutic approaches and expand mental health care access (31).
Conclusions
In conclusion, the world is experiencing a digital health care transformation with the development of core technologies (e.g., AI, IoT, DT, and XR). The metaverse is driving these innovations synergistically to increase access to health care services among multiple disciplines, supplement medical education and training, and optimize the experience of delivering and receiving medical care to meet the clinical needs of patients. Part 2 of this series will further explore the rapidly changing influence that the metaverse has on clinical neurosciences.
1. : Metaverse Roadmap: Pathways to the 3D Web. Acceleration Studies Foundation, 2007. https://www.metaverseroadmap.org/MetaverseRoadmapOverview.pdfGoogle Scholar
2. : Educational applications of metaverse: possibilities and limitations. J Educ Eval Health Prof 2021; 18:32Crossref, Medline, Google Scholar
3. : Augmented reality interface for complex anatomy learning in the central nervous system: a systematic review. J Healthc Eng 2020; 2020:8835544Crossref, Medline, Google Scholar
4. : A scoping review of metaverse in emergency medicine. Australas Emerg Care (Epub Aug 8, 2022). https://doi.org/10.1016/j.auec.2022.08.002Google Scholar
5. : Using immersive technologies to develop medical education materials. Cureus 2021; 13:e12647Medline, Google Scholar
6. : Crystal structure of a monomeric retroviral protease solved by protein folding game players. Nat Struct Mol Biol 2011; 18:1175–1177Crossref, Medline, Google Scholar
7. : A taxonomy of mixed reality visual displays. IEICE Trans Inf Syst 1994; 77:1321–1329Google Scholar
8. : Advertising in the metaverse: research agenda. J Interact Advert 2021; 21:141–144Crossref, Google Scholar
9. : Snow Crash. New York, Bantam Books, 1992Google Scholar
10. : An empirical investigation of virtual world projects and metaverse technology capabilities. ACM SIGMIS Database Adv Inf Syst 2011; 42:74–101Crossref, Google Scholar
11. : Networking the metaverses. IEEE Netw 2007; 21:2–4Google Scholar
12. : A study on the intention and experience of using the metaverse. Jahr–Eur J Bioeth 2022; 13:177–192Google Scholar
13. : What the metaverse is (really) and why we need to know about it. Cyberpsychol Behav Soc Netw 2022; 25:355–359Crossref, Medline, Google Scholar
14. : The metaverse as a virtual form of data-driven smart urbanism: platformization and its underlying processes, institutional dimensions, and disruptive impacts. Comput Urban Sci 2022; 2:24Crossref, Medline, Google Scholar
15. : Applications of blockchain in the medical field: narrative review. J Med Internet Res 2021; 23:e28613Crossref, Medline, Google Scholar
16. : Digital twins: from personalised medicine to precision public health. J Pers Med 2021; 11:745Crossref, Medline, Google Scholar
17. : The challenges of entering the metaverse: an experiment on the effect of extended reality on workload. Inf Syst Front (Epub Feb 12, 2022). https://doi.org/10.1007/s10796-022-10244-xGoogle Scholar
18. : Rethinking the meaning of cloud computing for health care: a taxonomic perspective and future research directions. J Med Internet Res 2018; 20:e10041Crossref, Medline, Google Scholar
19. : Can extended reality in the metaverse revolutionise health communication? NPJ Digit Med 2022; 5:132Crossref, Medline, Google Scholar
20. : Extended-reality technologies: an overview of emerging applications in medical education and clinical care. J Neuropsychiatry Clin Neurosci 2021; 33:A4, 173–177Link, Google Scholar
21. : Metaverse games: game changer for healthcare? Cyberpsychol Behav Soc Netw 2022; 25:267–269Crossref, Medline, Google Scholar
22. : CardioVerse: the cardiovascular medicine in the era of metaverse. Trends Cardiovasc Med (Epub May 11, 2022). https://doi.org/10.1016/j.tcm.2022.05.004Google Scholar
23. : Gastroenterology in the metaverse: the dawn of a new era? Front Med 2022; 9:904566Crossref, Google Scholar
24. : The metaverse and oncology. Clin Oncol (R Coll Radiol) (Epub Jul 13, 2022). https://doi.org/10.1016/j.clon.2022.06.011Google Scholar
25. : An interactive experiment combining ultrasound, magnetic resonance imaging, and force feedback technology to physically feel the fetus during pregnancy. Eur J Radiol 2019; 110:128–129Crossref, Medline, Google Scholar
26. : Metaverse and virtual health care in ophthalmology: opportunities and challenges. Asia Pac J Ophthalmol 2022; 11:237–246Crossref, Medline, Google Scholar
27. : Cranial nerve zero (CN 0): multiple names and often discounted yet clinically significant. J Neuropsychiatry Clin Neurosci 2022; 34:A4, 95–99Link, Google Scholar
28. : Virtual reality in metaverse for future mental health-helping profession: an alternative solution to the mental health challenges of the COVID-19 pandemic. J Public Health (Epub Apr 25, 2022). https://doi.org/10.1093/pubmed/fdac049Google Scholar
29. : Moving from virtual reality exposure-based therapy to augmented reality exposure-based therapy: a review. Front Hum Neurosci 2014; 8:112Crossref, Medline, Google Scholar
30. : Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol Med 2017; 47:2393–2400Crossref, Medline, Google Scholar
31. : Future of mental health in the metaverse. Gen Psychiatr 2022; 35:e100825Crossref, Medline, Google Scholar
32. : Clinical virtual reality tools to advance the prevention, assessment, and treatment of PTSD. Eur J Psychotraumatol 2017; 8:1414560Crossref, Medline, Google Scholar
33. : Effectiveness of virtual reality exposure therapy for active duty soldiers in a military mental health clinic. J Trauma Stress 2011; 24:93–96Crossref, Medline, Google Scholar
34. : Is the Metaverse the New Hope for Mental Health? SFGATE. https://www.sfgate.com/news/article/Is-the-Metaverse-the-New-Hope-for-Mental-Health-17221756.php. Accessed Sep 26, 2022Google Scholar
35. : Preliminary study of virtual-reality-guided meditation for veterans with stress and chronic pain. Altern Ther Health Med (Epub Sept 24, 2021). http://www.alternative-therapies.com/abstract/index.html?id=38043Google Scholar
36. : Meditating in virtual reality: proof-of-concept intervention for posttraumatic stress. Psychol Trauma 2020; 12:847–858Crossref, Medline, Google Scholar
37. : Virtually being Einstein results in an improvement in cognitive task performance and a decrease in age bias. Front Psychol 2018; 9:917Crossref, Medline, Google Scholar
38. : Know thy avatar: the unintended effect of virtual-self representation on behavior. Psychol Sci 2014; 25:1043–1045.Crossref, Medline, Google Scholar
39. : What role can avatars play in e-mental health interventions? Exploring new models of client-therapist interaction. Front Psychiatry 2016; 7:186Crossref, Medline, Google Scholar
40. : Meta-analysis of technology-assisted interventions for social anxiety disorder. J Anxiety Disord 2016; 42:71–84Crossref, Medline, Google Scholar
41. : Controlling social stress in virtual reality environments. PLoS One 2014; 9:e92804Crossref, Medline, Google Scholar
42. : Virtual reality exposure therapy for treating social anxiety: a scoping review of treatment designs and adaptation to stuttering. Front Digit Health 2022; 4:842460Crossref, Medline, Google Scholar
43. : Into the spine metaverse: reflections on a future metaspine (uni-)verse. Glob Spine J 2022; 12:545–547Crossref, Medline, Google Scholar
44. : Internet-based interventions for social anxiety disorder: an overview. Verhaltenstherapie 2013; 23:160–168Crossref, Google Scholar
