A review on ergonomics evaluations of virtual reality

3.1.1Hardware components

Virtual reality system hardware components include input devices and output devices. Input devices mainly include data gloves, joysticks, and motion trackers. Output devices are used to present the VR environment to the user and provide feedback, including visual, auditory, and haptic displays [9]. Output devices mainly include virtual reality head-mounted display (HMD), cave (Cave Automatic Virtual Environment), VR glasses, and headphones. Compared with the traditional HMD, Virtual Reality All-in-one Headset is loaded with an independent processor to make the system wireless. Ahn et al. [10] suggested that future virtual reality devices should consider multi-user virtual reality environments and wireless connectivity issues. They believed that wireless and contactless virtual reality devices were the future trends of VR technology.

3.1.2Software components

Virtual reality system software mainly consists of 3D modeling software, a virtual reality open platform, and an engine. Virtual reality modeling based on 2D drawing software. 3DS Max®, AutoCAD®, Softimage 3D®, and Maya® are examples of commercially available commonly used 3D modeling software. The VR Open Platform has an accessible Virtual Reality Software Development Kit (VR SDK). Oculus provides a constantly updated SDK to create prototypes and engage the public in the application development process. And the engine is the universal development and creation tool for virtual reality. Examples of mainstream VR engines are Unreal Engine 4®, Unity 3D®, Cry ENGINE®, and VR Platform®.

3.2.1Physical ergonomics

3.2.1.1. Pressure. The weight, weight distribution, and wearing style of different HMDs can bring different pressure on the facial bearing point, thus affecting the overall subjective discomfort of the user [12]. Research on HMD pressure has focused on both physical load and contact pressure.

Neck joint torque is an important evaluation index of body load, significantly influenced by weight and center of mass, and increases with HMD mass [13]. The minimum position of the center of mass of the neck joint torque varies with the test posture and the recommended range of the center of mass is determined according to the posture. Weight and center of gravity position have a significant effect on the subjective perception of body load level [14]. LeClair et al. [15] mentioned that the maximum acceptable mass of a helmet is about 1000 g, so the maximum mass of a head-mounted display should not exceed 1000 g.

Contact pressure is mainly generated from seven major areas of the head and face contacting with the HMD, including the bridge of the nose, cheekbones, eyebrows, forehead, temporal bone, top of the head, and back of the head [16]. Head-facial contact pressure is more sensitive to the position of the center of gravity of the HMD. An HMD with a forward center of gravity produced significantly higher nasal contact pressure and overall discomfort than the one with a backward center of gravity [17]. Studies have shown that the overall and nasal subjective discomfort was closely related to the nasal contact pressure, and the ear was the most sensitive to the discomfort for the design with the center of gravity on the ear [17]. Lee et al. [18] found that virtual reality headsets with different shapes of curves in contact with the facial region exerted different levels of pressure on the nose. Yan et al. [19] investigated the relationship between the weight of the virtual reality headset and the subjective head discomfort and pressure load, and they concluded that the lighter weight can make users feel better. At the same time, the integrated headset is more comfortable than the soft band one.

3.2.1.2. Muscle fatigue. Fatigue is mostly associated with muscle activity [20]. Eye movement is a natural habit and muscle fatigue rarely occurs, so visual fatigue is inherently different from the usual muscle fatigue, which is related to the activity of the central nervous system [21]. Therefore, this paper discusses the HMD visual fatigue study separately from the HMD muscle fatigue study.

The weight of the HMD itself can cause fatigue. The added weight on the helmet can make the center of gravity of the head and helmet move forward. At the same time, it can also increase the inertia of the neck [22]. According to the analysis of the Surface EMG, during each training, the fatigue level of the last few hours is significantly higher than that of the first few hours after wearing HMD [22].

Different target positions in virtual reality interaction can also affect musculoskeletal load and task performance [23]. Redundant vertical target positions should be avoided in VR interaction to reduce musculoskeletal discomfort and injury risk. Nichols [24] indicated that long-time experiments should be conducted to effectively assess muscle fatigue problems.

In addition, during the interaction, free gesture movements of the upper limbs without arm support and prolonged static postures can lead to shoulder discomfort and fatigue [25]. Repetitive and continuous arm posture during interactions can also lead to shoulder muscle fatigue [26].

3.2.1.3. Thermal comfort. The thermal comfort of HMD is important. Previous studies have demonstrated that wearing headgear in warm conditions can lead to subjective thermal disorders (STD) [27, 28]. HMD insulates the head area and can cause thermal discomfort, which in turn reduces the user’s wearing intention [29]. The thermal comfort of the user’s head plays a crucial role in overall personal comfort. Based on the surface area, part of the body heat is dissipated through the head [30, 31]. In addition, virtual reality headsets generate a lot of heat during operation, which means that the user will feel hotter than wearing other headgear (such as a helmet) [32].

Infrared thermography (IRT) has attracted growing attention nowadays in physiological studies because of its great potential to quantify surface temperature distribution easily and non-invasively and generate corresponding thermal images [31, 33, 34]. Dotti et al. [31, 33, 34] compared the application of miniature data loggers and infrared thermography (IRT) in human comfort research. Wang et al. [35] studied the thermal characteristics and subjective thermal discomfort of virtual reality headsets. They measured microclimate temperature and relative humidity using miniature data loggers, and the temperature distribution between the user’s face and the contact point of the headset using an infrared thermal imaging camera. The study found that subjective thermal discomfort was positively correlated with usage time, microclimate temperature, relative humidity, and display coverage area. They suggested that the design of HMD should consider reducing the display coverage area of the user’s face, especially the area with a high sweating rate.

3.2.2Visual ergonomics

The immersion that users obtain in VR environments depends largely on their visual experience. As a result, the research of visual ergonomics of virtual reality is particularly important. Latency refers to the difference between the time required for a virtual reality device to respond to a behavioral signal input by the user and the time for a VR device to present the signal. In September of 2020, IEEE (Institute of Electrical and Electronics Engineering) developed an HMD standard to reduce virtual reality diseases. It mentioned that the latency of virtual reality can affect user immersion and cause inconvenience, which put forward new requirements for HMD hardware. Latency should be as low as possible, 20 ms or even less in latency will be acceptable [36]. Frame rate is the number of frames per second that the image is refreshed, and low frame rates can cause flicker. Since flickering and low frame rates may cause symptoms such as headaches, eye fatigue, and seizures to sensitive users, the frame rate in VR content must be synchronized with the refresh rate of the VR HMD. It is recommended that the minimum frame rate be no less than 30 fps (frames per second) for images, 60 fps fo r graphics, and 90 fps for interactive content. And the higher the pixels per inch (PPI) of the VR image resolution is, the clearer the content shown on the screen will be.

The visual ergonomics problems caused by VR are mainly reflected in two aspects of visual fatigue and motion sickness.

3.2.2.1. Visual fatigue. HMD can cause visual fatigue [36]. Sheedy et al. [37] classified eye strain symptoms into external and internal symptoms based on location. The main external symptoms are burning, irritation, tearing, and dryness. These symptoms are caused by a variety of factors, such as eyelid opening, staring, upward gazing, reading small print, and flickering at the front and bottom of the eye. Internal symptoms are mainly pain, tension, and headache.

Analyzing the reasons for visual fatigue caused by HMD, Peli [38] suggested that it was caused by the mismatch between the convergence distance of the eyes and the focal length. Yano et al. [39] proposed that when viewing stereoscopic images, the difference between the demands of conditioning and convergence can place a lot of stress on the visual system, leading to eye fatigue and measurable changes in visual function. Bando et al. [40] observed that experiments in the VR environment were more prone to visual fatigue than experiments in the LCD environment, mainly due to image distortion or crosstalk in the stereoscopic viewing and the proximity between the source of illumination and the eyes.

3.2.2.2. Visually induced motion sickness. Visually induced motion sickness (VIMS) may occur during or after exposure to a virtual environment, causing discomfort to the user, characterized by symptoms such as nausea, headache, and disorientation [41]. VIMS is the main obstacle to be overcome in virtual reality. It is estimated that about 30–80% of people will experience some degree of illness when using virtual reality [42]. Therefore, this section will focus on VIMS in a virtual environment.

Previous theories suggest that motion sickness stems from the body’s response mechanism to food poisoning - when toxins are detected in food, the brain triggers a perceptual disorder that forces the body to vomit up the toxic food. Of course, such claims are difficult to substantiate, and current general theories focus mainly on the confusion of visual-motion signals. The main causes of motion sickness include conflicts between visual information and limb movement information, conflicts in visual vergence regulation, excessive binocular parallax, and discontinuous changes in parallax [43]. Sensory conflict theory posits that motion sickness occurs when sensory signals, particularly signals related to self-motion, from the various sensory systems (e.g. visual system, vestibular system, proprioceptors) are either in conflict with one another or else strongly violate expectations based on previous experience [44]. Therefore, conflict reduction is essential to avoid motion sickness [45]. In order to reduce motion sickness, Mizukoshi et al. [45] developed a scaling method for master-slave remote control systems based on the gaze motion of the head towards or away from the object.

VIMS is affected by individual factors [46]. These factors include age (younger individuals more susceptible than older individuals), sex (females more susceptible than males), and personality factors (individuals low in extraversion, high in neuroticism, and/or high in anxiety all being more susceptible) [41, 47].

Many of the factors that induce motion sickness are related to VR simulator hardware and displays [44]. Studies on HMD hardware-related factors leading to motion sickness mainly include display device types (screen, monitor, and helmet display) [48–50], the field of view (FoV) [51, 52], time delay [53], frame rate [54] and flickering [55]. Field of view refers to the size of the area that the user can observe. A small FoV indicates a narrow viewing area and the user must move the screen frequently. While small FoV is characterized by reduced image immersion and visual cognition, large FoV can lead to screen distortion, causing users to feel dizzy oruncomfortable [56].

VR latency can distract users, affecting their comfort and the intensity of motion sickness [57]. Lee et al. used the Delphi methodology to evaluate the effect of HMD on motion sickness and found that latency was the most critical consideration for the helmet display comfort experience [56]. The accumulation of flicker triggers motion sickness in the user.

The evaluation of VIMS is mainly divided into the subjective evaluation and objective evaluation. Subjective evaluation is essentially a study of the opinions of the majority of the subject group and can directly reflect user feelings. The commonly used questionnaire for evaluating motion sickness is the Simulator Sickness Questionnaire (SSQ) proposed by Kennedy in 1993 [41]. The SSQ assesses motion sickness by a total score based on three factors: nausea, oculomotor, and disorientation.

The SSQ has been widely used to measure signs and symptoms associated with military virtual reality simulators [41, 58]. However, some items in the SSQ have little correlation with the measurement of motion sickness in virtual reality environments [59]. The SSQ has been continuously improved in recent years, Kim et al. [60] argued that some items in the SSQ are not relevant to symptoms in VR environments and proposed the Virtual Reality Sickness Questionnaire (VRSQ). The VRSQ is composed of two parts: eye discomfort and orientation disorder, which excludes nausea, as nausea has a small effect on motion sickness in the VR environment.

Subjective assessment is convenient and widely used. However, it is subject to individual influence and can only obtain rough changes. In contrast, objective assessment has the advantages of less measurement error and direct measurement of human body response. But it also has limitations of device use and unintuitive data. Therefore, researchers often use the method of combining subjective evaluation with objective measurement.

Postural instability has been identified as a key predictor of motion sickness [61]. The measurement of postural instability includes two types of methods: the center of gravity judgment method and the path judgment method. In the method of determining the center of gravity, the force plate can be used to test the instability of the body’s posture when standing [62]. Changes in the center of pressure position in the Anterior-Posterior (AP) and Medial-Lateral (ML) axes are recorded at a frequency of 50 Hz. The effects of the visual task and the motion sickness state are mainly concentrated in the anterior-posterior axis. In the path judgment method, a magnetic sensor is used to record the subject’s posture data and fix it to the center of the participant’s back [63], collecting the path data on the X and Y axes at a frequency of 120 Hz. The temporal complexity of pose instability is studied using sample entropy and normalized path length, and the size of pose instability is studied using elliptical area and path length.

Other objective physiological data such as electrooculogram (EEG), electrocardiogram (ECG), electroencephalogram (EOG), galvanic skin response (GSR) and photoplethysmogram (PPG), blood pressure (BP), heart rate (HR), pulse rate, blink rate can also be used as objective assessment indicators of motion sickness [43, 64, 65].

3.3.1Visual ergonomics

3.3.1.1. Visual fatigue: VR content can also affect visual comfort and cause visual fatigue. Choy et al. [67] demonstrated that participants viewing stereoscopic 3D (S3D) with a virtual reality device exhibited higher SSQ scores than participants using other devices. Kooi and Toet [68] investigated the effect of binocular image defects on the visual fatigue of the stereo vision system. They found that almost all binocular image asymmetries severely reduced visual comfort.

3.3.1.2. VIMS: In addition to individual differences and HMD hardware factors, factors that may cause motion sickness are sound and content [69]. Therefore, the occurrence of virtual reality-induced motion sickness is correlated with software content design. IEEE Standard [36] stated that unnatural and abrupt virtual camera movements and asynchronous behaviors that do not match the visual experience of VR content can cause user dizziness and discomfort, and suggests that the frame rate in VR content must be synchronized with the refresh rate of the HMD. And in the case of VR content with high image complexity, users are forced to recognize a large amount of visual information, which may also lead to VR sickness. Therefore, dynamic scenes are prone to cause motion sickness symptoms, resulting in users quitting the evaluation [70], while few significant discomforts are reported in staticscenes [71].

Keshavarz and Hecht [72] found that pleasant sounds in a simulator environment can reduce motion sickness, especially when the pleasant sounds make participants feel relaxed. However, the direction of the sound should be determined by the user’s head position, and mismatches between the sound source and the actual audio playback can also lead to virtual reality illness [36].

3.3.2Cognitive ergonomics

Cognitive ergonomics research for virtual reality software is focused on two aspects: task performance and cognitive load.

3.3.2.1. Task performance. Virtual reality environments can have an impact on users’ task performance. Rizzuto et al. [73] evaluated the performance of the pointing task in real and virtual environments and found that the target error in the virtual condition was significantly larger than that in the real condition. To compare walking in virtual reality and the real world, various aspects were studied, including types of systems such as video displays and helmet displays, 3D spatial recognition, speed recognition, environments such as space stations or buildings. Several scholars [74, 75] have compared navigation tasks in HMD and desktop environments, including the number of captures, distance traveled, and average speed. The experiments showed that, in general, people were more satisfied and intuitive with HMD, but performed better on the desktop environment for most tasks.

Task performance is closely related to access to information in the virtual environment. Lee et al. [76] investigated the influence of text information on the cognitive processing of visual information in HMD by obtaining user evaluations from three dimensions: contrast sensitivity, sentence length, and text size. They proposed that in a virtual reality environment, text size of 96 pixels or more, a background contrast sensitivity of 75% to 50%, and an effective sentence length ratio of 33.3% to 50% were used to ensure the readability of text information. Lambooij et al. [77] also conducted a user study to determine the visual discomfort associated with 3D stereoscopic displays compared to 2D displays and suggested that participants with a moderate binocular condition experienced more visual discomfort and showed decreased performance in reading tasks. By studying the effects of color mode (dark or light mode), peripheral illumination, and virtual illumination on reading text, Erickson et al. [78] found that using light mode under bright virtual illumination facilitates the legibility of text to the user, but switching to dark mode was beneficial when lowering the virtual illumination. They believed that this was partially due to a color bleeding effect that occurs when a light-colored letter was presented on a dark background, where the light from the letter partially illuminates neighboring background pixels and results in a letter that appears slightly larger [79].

3.3.2.2. Cognitive load. A particular challenge of virtual reality is the potential overload of visual input, which creates an unnecessary cognitive load [80]. Rhiu et al. [81] verified that users felt a higher workload when using the HMD while walking and driving. In particular, the scores of mental demand and frustration were significantly different between the two systems, as users felt dizzy or mentally stressed when participating in the experiment. Chang et al. [82] designed a driving system with embedded Stroop tasks. Stroop task had been used to assess cognitive processing and selective attention abilities, which asked an individual to distinguish whether a certain word’s meaning and visual color match [82]. They found that the average response time when users answered Stroop trials in the FSD (flat-screen displays) condition was shorter than that in the HMD condition. This indicated that HMDs might have caught more of the users’ attention for virtual driving, which led to their delayed responses to the Stroop trials. In terms of gender differences, they found that men outperformed women in virtual driving, especially at longer driving distances. They speculated that the reason for this may be that females have a higher cognitive load in virtual driving. Female users had a significantly lower average minimum oxygen saturation and a greater decrease in oxygen saturation during the use of the system. The virtual driving system generated more mental work for women, which resulted in greater oxygen consumption [82].