Building Skills in a Virtual World: Immersive Training for the Modern Workforce
Stephen Elms is a PhD candidate at The Bartlett Faculty of the Built Environment at University College London. His research focuses on workforce skill development in offsite construction, with an emphasis on vocational education and training.
In the opening moments of Steven Spielberg’s 2018 epic sci-fi Ready Player One, the main character, Parzival, introduces us to the virtual world known as OASIS. He describes it as “a place where the limits of reality are your imagination.” This not-too-distant future of an immersive universe filled with games, fantasy, and adventure is often how we are introduced to technologies like Extended Reality (XR), which includes Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and Tangible User Interfaces (TUI). When Ready Player One was first published in 2011, author and screenwriter Ernest Cline’s rich, immersive worlds—described through futuristic technology—were little more than a daydream.
Fast-forward a dozen years, and such science fiction has not only become reality, but is also relatively affordable. In 2024, both Meta (formerly Facebook) and Apple have released high-end consumer headsets that blend VR and AR—the Quest 3s and the Vision Pro, respectively. Most smartphones and tablets are AR-capable, with games like Pokémon Go.
It might seem like a stretch to connect a blockbuster film and a mobile game—one your kids probably cannot put down—to modular offsite construction. However, as we will explore, the real connection lies in the potential for gamified, immersive worker training that can be equally engaging.
As both the hardware and software for developing and deploying XR applications continue to become more affordable, the investment in this technology as a training tool is increasingly appealing. XR also offers several other benefits, including performing potentially hazardous tasks in a safe environment and acquiring new skills without the immediate need for physical materials. To better illustrate this, we will examine four academic studies that utilized XR technology—highlighting the benefits of each type: VR, AR, MR, and TUI—in training workers within the construction industry.
Stephen Elms was one of three award winners at the 2024 Transforming Construction with Off-site Methods and Technologies conference, presented by the University of New Brunswick’s Off-site Construction Research Centre and sponsored by the Modular Building Institute.
As both the hardware and software for developing and deploying XR applications continue to become more affordable, the investment in this technology as a training tool is increasingly appealing. XR also offers several other benefits, including performing potentially hazardous tasks in a safe environment and acquiring new skills without the immediate need for physical materials. To better illustrate this, we will examine four academic studies that utilized XR technology—highlighting the benefits of each type: VR, AR, MR, and TUI—in training workers within the construction industry.
Terms
Before diving into the case studies, it is important to understand the different types of XR technologies that are transforming worker training in construction. Many readers are probably already familiar with VR and AR, as these are the most common and accessible forms of XR. You have likely encountered VR in Hollywood depictions (as discussed earlier), or even in theme parks, where it is being incorporated into existing attractions and standalone rides—such as Disney’s Star Wars: The Secrets of the Empire. Generally, consumer headsets are becoming more common for gaming, watching movies, or even exploring the metaverse. Similarly, aside from mobile games, AR is used by companies like Amazon and Ikea, allowing you to view how products would look in your home through your phone or tablet.
These examples offer helpful context, but what truly sets these technologies apart is how users engage with their environments. Each interface creates a different kind of interaction and immersion.
VR creates a fully immersive digital environment, where users interact as if physically present by wearing headsets that block out the real world. This allows them to engage with simulations or experiences in a controlled virtual space. AR, on the other hand, overlays digital information onto the real world, enhancing your physical surroundings with additional layers of content. AR is most often experienced through smartphones, tablets, or AR glasses.
MR takes this immersion a step further by seamlessly blending both the physical and digital worlds in real time. This enables users to interact with both environments simultaneously, allowing digital objects to coexist and respond to the physical world. Finally, TUI introduces a hands-on approach, where physical objects are used to manipulate digital information. This allows users to interact with tangible items that control digital outcomes, creating a more intuitive, physical connection to digital content.
If the differences between these technologies still seem unclear, the upcoming case studies will provide greater clarity as we explore how each is applied in worker training.
Case Study: Virtual Reality for Construction Safety Training
In a study conducted by a team of German and Danish researchers (Jelonek et al., 2022), VR was explored as a tool for improving safety training in the construction industry. With 97 fatal accidents reported in the German construction sector in 2020 alone, the need for more effective training methods is necessary. The study aimed to address this by creating immersive simulations where workers could experience dangerous situations without real-world risks. The VR simulation guided participants—many of whom were already familiar with basic safety protocols—through a range of tasks. Along the way, they encountered various hazards, from selecting the right personal protective equipment (PPE) to avoiding slick surfaces on the job site. The simulation provided instant feedback, restarting whenever mistakes were made—much like resetting a level in a video game—giving workers the opportunity to safely learn through experience.
According to the researchers (Jelonek et al., 2022), workers who trained using VR showed a noticeable improvement in their understanding of safety procedures, especially with specific tools like the angle grinder. Beyond knowledge, many participants expressed a greater sense of confidence and awareness after their virtual experience. Some even preferred the immersive, hands-on nature of VR over traditional training methods. However, the study also underscored the importance of accuracy—any gaps or inaccuracies in the simulation could lead to the wrong lessons being learned.
Case Study: Augmented Reality in Construction Education
At the University of Florida’s School of Construction Management, researchers investigated the use of AR to enhance students’ understanding of complex construction processes (Shanbari et al., 2016). The focus was on how AR could help students grasp the spatial and timing constraints involved in masonry and roofing assemblies—an area often difficult to convey through traditional classroom methods.
The study used AR to superimpose virtual building elements onto real-world video footage of a construction site. This allowed students to visualize hidden details of masonry and roof construction that might otherwise be difficult to observe. The goal was to simulate the kind of job site exposure typically limited to field trips or internships, both of which are frequently disconnected from the specific topics being taught in the classroom.
Students who were exposed to AR-enhanced videos showed improved comprehension of key elements like masonry wall ties and flashing, as well as the sequencing involved in assembling roofing systems (Shanbari et al., 2016). The immersive nature of AR helped bridge the gap between theoretical knowledge and practical understanding, offering a more interactive learning experience. While the technology was found to be a valuable supplement to traditional teaching methods, it was not intended to fully replace them. The study concluded that integrating AR into construction management education has strong potential to deepen students’ understanding of complex processes, preparing them more effectively for the demands of the industry.
Case Study: Mixed Reality for Construction Education
At Bond University in Australia, researchers explored the potential of mobile mixed reality (MMR) to enhance construction education (Vasilevski & Birt, 2020). With the growing adoption of building information modeling (BIM) in the architecture, engineering, and construction (AEC) industries, the study aimed to determine if MMR could improve students’ understanding of complex construction processes by creating immersive learning environments.
The study used a combination of mobile devices and MMR technologies, such as the Microsoft HoloLens, to create interactive simulations. These tools allowed students to engage with both virtual and real-world elements, providing a more dynamic learning experience. Unlike traditional classroom methods, where students often struggled to visualize the spatial and timing constraints of construction projects, MMR enabled them to interact with 3D models and simulations in real time. This approach not only made the learning experience more hands-on but also fostered greater engagement and motivation among the students.
The results were promising (Vasilevski & Birt, 2020). Students reported feeling more connected to the material and better able to comprehend the complexities of construction management tasks. Many participants highlighted the increased motivation and engagement that came from the interactive, gamified nature of MMR.
However, the study also noted some technical challenges, such as motion sickness experienced by a small percentage of users. Overall, the researchers concluded that MMR has significant potential to enhance learning outcomes by bridging the gap between theoretical knowledge and practical, real-world application.
Case Study: Tangible User Interfaces for Carpenter Apprentices
In a joint research effort between the Swiss Federal Institute of Technology, the Swiss Federal Institute for Vocational Education, and Stanford University, researchers explored how TUIs could enhance the spatial skills of carpenter apprentices (Cuendet et al., 2014). Spatial skills are crucial in the carpentry profession, as tasks often require the ability to visualize and manipulate objects in three dimensions based on two-dimensional plans. The study aimed to assess whether TUIs—interfaces that allow users to interact with digital information through physical objects—could improve apprentices’ spatial reasoning and practical skills.
Using TUI technology, apprentices engaged in hands-on activities that mirrored the traditional methods of learning in vocational education. By manipulating physical objects, the apprentices were able to interact with digital models, offering a bridge between theory and practice. This approach allowed them to develop spatial skills in a more intuitive, interactive way, which is particularly important in carpentry where understanding spatial relationships is key.
The results showed that apprentices with access to TUI-enhanced training had a higher level of spatial skill development compared to those who relied on traditional learning methods alone (Cuendet et al., 2014). The study also noted that these tools could help bridge the gap for apprentices whose spatial skills might not be as strong initially, offering a more inclusive and effective approach to skill development. Overall, the use of TUIs in vocational training holds promise as a method to enhance both learning and practical application in trades that require high levels of spatial intelligence.
Conclusion: The Future of XR in Offsite Construction Training
The case studies presented, while not directly related to offsite construction, provide clear examples of how XR technologies—whether VR, AR, MR, or TUI—are already transforming worker training across various construction sectors. These technologies offer immersive, interactive learning experiences that bridge the gap between theory and practice, allowing workers to develop crucial skills in a safe, controlled environment. As the offsite construction industry embraces automation and robotics, these technologies offer a natural complement to the skills required for managing and operating advanced systems.
However, the academic literature has yet to fully explore how XR can be applied specifically to offsite construction training. This is an area that researchers, including myself, are just beginning to investigate. The potential is vast—XR could revolutionize how workers are trained to handle the complexities of offsite construction, from factory-based production to the onsite assembly of modular components. The integration of XR with automation and robotics represents a significant opportunity for innovation in both training and operational efficiency.
As we move forward, there is a growing need for collaboration between academia and industry to unlock the full potential of XR in this space. By working together, we can develop training solutions that not only meet the evolving needs of the workforce but also push the boundaries of what is possible in offsite construction. The future of worker training in this industry is immersive, interactive, and filled with opportunity—and XR stands at the forefront of this transformation.
Bibliography
Cuendet, S., Dehler-Zufferey, J., Arn, C., Bumbacher, E., & Dillenbourg, P. (2014). A Study of Carpenter Apprentices’ Spatial Skills. Empirical Research in Vocational Education and Training, 6(1), 1–16. https://doi.org/10.1186/s40461-014-0003-3
Jelonek, M., Fiala, E., Herrmann, T., Teizer, J., Embers, S., König, M., & Mathis, A. (2022). Evaluating Virtual Reality Simulations for Construction Safety Training. I-Com, 21(2), 269–281. https://doi.org/10.1515/icom-2022-0006
Shanbari, H., Blinn, N., & Issa, R. R. A. (2016). Using Augmented Reality Video in Enhancing Masonry and Roof Component Comprehension for Construction Management Students. Engineering Construction and Architectural Management, 23(6), 765–781. https://doi.org/10.1108/ecam-01-2016-0028
Vasilevski, N., & Birt, J. (2020). Analysing construction student experiences of mobile mixed reality enhanced learning in virtual and augmented reality environments. Research in Learning Technology, 28(0). https://doi.org/10.25304/rlt.v28.2329
More from Modular Advantage
How Rebel Concept Harnesses AI as a Design Tool to Improve the DFMA Process
“The complexity of projects and the size of project are so large, and the technology is so complicated that AI would do it so much faster than we are,” says Kris Droszcz, CEO at Rebel Concept. “Everybody’s struggling to control the manufacturing processes according to the design. And I think that’s what AI will do.”
One Answer from AI Can Provide Innumerable Opportunities, says Stack Modular’s Jim Dunn
The Stack AI Engine is not fully functional yet, but it offers an unlimited number of productivity benefits. Currently, the company is using AI for cost estimates, most recently for a project out in California.
GrayWolf: Building the Future of Modular Data Cnters to Power the AI Revolution
The growth of artificial intelligence (AI) and cloud-based applications has created an urgent need for data centers capable of managing massive amounts of data. As businesses across industries increasingly rely on digital infrastructure to power their AI tools, the demand for data centers has skyrocketed.
Artificial Intelligence: Yes, it is a Construction Game-Changer
Much like other sectors of the economy, AI is reshaping construction, revolutionizing how projects are planned and executed across the country. While it is obvious that AI cannot do certain things by itself, like laying bricks, paving roads, or shingling roofs, its impact stands to be both massive and positive.
The Potential and Possibilities of AI for Offsite
AI is everywhere. Many industries have incorporated AI into business as usual with new organizational structures and new dependencies to be more efficient and to automate repetitive tasks. Offsite construction will not be immune to AI’s ubiquity. The technology offers unique benefits to the offsite building process.
Moving from AI Hype to an AI Advantage
Learning about AI will help you see past the overinflated hype and identify the opportunities specific to your own business. Learning about AI is like peeling an onion – each layer reveals something new, and occasionally it might make your eyes water. But in the end, you’ll have all the ingredients you need to cook up some real innovation in your organization.
The Blueprint for Smart and Ethical AI Integration in Your Business
The journey toward smart and ethical AI integration involves crafting a strategic vision that aligns AI’s transformative potential with your organizational goals. True success with AI is measured by its capacity to enhance human capabilities, foster sustainable growth, and generate value for all stakeholders while actively avoiding ethical pitfalls and harmful risks.
Five Factors for Success When Shifting Your Business to Modular
As labor, competitive and cost factors push more contractors and developers to embrace the opportunity to go modular as a first choice and not a last resort, it’s become clear that adopting this new model requires more than just a willingness to make it happen. Here are five factors to consider as your company moves to a modular construction model:
UniCool-Edge: Optimized Cooling Solutions for Modular and Edge Data Centers
In an industry where efficiency, scalability, and security are crucial, the UniCool-Edge™ system is a revolutionary solution for modular data center manufacturers. By offering external mounting for maximum space utilization, energy-efficient cooling, and pre-commissioned reliability, UniCool-Edge helps manufacturers stay ahead in the rapidly evolving AI-driven data center market. As edge data centers continue to expand, solutions like UniCool-Edge will play a vital role in supporting the next generation of modular data centers and their data center infrastructure.
DORÇE Navigates Mountainous Terrain, Extreme Weather to Help Rebuild Türkiye
The February 6 earthquakes in 2023 were enormous. In response, DORÇE moved quickly, obtaining a government bid to help create housing and multipurpose structures—both temporary and permanent—for earthquake victims and laborers hired for reconstruction efforts.