At the SC 24 Plenary Session held in Rapid City, the United States in August 2011, the scope of Virtual Reality (VR) technology was expanded to VR/AR/MR (VR/AR/MR), and the term "VR Continuum" was finally used to describe the above technologies [1]. Virtual reality can be defined as "the sum of hardware and software systems that attempt to perfect an omni-dimensional, sensory illusion existing in another environment" [2], where the environment is a virtual environment artificially constructed with the help of computer technology. The concept of virtual reality technology was put forward in the 1960s and 1970s, and began to form and develop in the 1990s [1]. Simulated reality technology has been applied in many fields such as technical training, production and assembly, audio-visual entertainment, etc., and has solved some difficult problems. Immersive experience, user presence and human-computer interaction are regarded as the core features of virtual reality technology [3]. At present, the problems faced by virtual reality technology mainly focus on three aspects: environment modeling, presentation technology and human-computer interaction.

First, environmental modeling

Virtual reality modeling involves many disciplines such as digital image processing, computer graphics, multimedia technology, sensing and measurement technology, simulation and artificial intelligence technology. It includes geometric modeling technology, motion modeling technology, physical modeling technology and model management technology.

1.1 Geometric modeling technology

Geometric modeling techniques include shape modeling and appearance modeling. Shape modeling refers to the 3D model of virtual objects with the help of Computer Aided Design (CAD) technology, which can be divided into three types: wireframe model, surface model and solid model. The number of points composing the model gradually increases from the wireframe model to the solid model, so does the amount of data, but the accuracy and details of the model are also improved. Therefore, most models only build the surface model of virtual environment in order to give consideration to the richness of details and the simplification of data.

Appearance modeling refers to adding the color, material, texture, gloss and other attributes of the model to the established shape model. These details are added in the form of polygons, but because VR system has high requirements for real-time performance during running. When all the details in the field of view are loaded at the same time, the real-time performance of the system is difficult to be guaranteed if the computing and rendering performance of the host is insufficient. Therefore, how to guarantee the real-time performance of the system without missing the details of the model has become a research hotspot.

1.2. Motion modeling technology

Geometric modeling only completes the modeling of the static model. In VR system, the dynamic relationship between the user subject and the virtual object, such as position change, collision, deformation, etc., should be included. To reflect the position relationship between the user subject and the virtual object in the virtual environment, it is necessary to establish an appropriate three-dimensional coordinate system, and realize the change of user perception through coordinate transformation of logarithmic data points. Collision detection is based on the established position relationship, and the data volume of wireframe model and surface model is relatively small, so the collision detection is time-consuming, difficult to guarantee real-time performance, and the phenomenon of "die piercing" is easy to appear in practical application. Real-time deformation of virtual objects is based on collision detection, and the realization method of real-time deformation is one of the current research hotspots. Few VR products that have been published contain real-time deformation function, and most of the deformation effect is presented in the form of animation.

1.3. Physical modeling technology

In order to make the virtual object model closer to reality, it is usually necessary to configure the physical parameters such as gravity, inertia, elastic modulus of virtual object and surface hardness of virtual object. After the physical model is established, the user can feel the weight and surface hardness of the virtual object when interacting with it. However, due to the lack of information feedback means to the user subject in the virtual environment at present, the physical quantity feedback such as weight, hardness and elasticity can not be realized, so the research progress of physical modeling is less.

1.4 Model management technology

Because the virtual environment contains a large number of models, it is difficult to load at the same time, so it is very necessary to build a model management system which can load gradually according to the usage of the main user. If the performance of the model management system is inadequate, the problem of missing models can occur.

Second, performance technology

The most important feature of virtual reality technology is Immersive. In order to enable users to get close to the real experience, appropriate performance technology is very necessary.

At present, all the VR systems that have been published need to use wearable AR devices to help users generate immersion. Wearable AR devices are usually head-mounted display (HMD), and the head-mounted digital helmets can be divided into optical see-through (OST) helmet and video see-through (VST) helmet [1]. Optical perspective headsets are usually used in Augmented Reality (AR) systems, typically Google Glass augmented reality glasses. The video perspective helmet displays a high frame rate and high resolution virtual environment in a closed environment that is isolated from external light, so that the user subject can get an immersive and immersive feeling in the virtual environment. At present, compared with the optical perspective helmet, the video perspective helmet has a lower technical threshold and wider application range. As a result, there are a large number of manufacturers launched video perspective type helmet products, such as HTC as the first to enter the realm of VR equipment manufacturing manufacturers, has launched a vive VR glasses series, its characteristic is to provide form a complete set of headphones for the helmet and handle, but vive series is an external device, its operation ability is insufficient, need external host to be used by the user; Pico has unveiled the Neo line of VR goggles, which feature a Snapdragon 845 processor embedded in the device itself, allowing it to run some big VR apps without being attached to the mainframe. In addition to traditional mobile device manufacturers, some Internet companies have also launched VR equipment, such as "Adventure 2Pro" launched by iQIYI. Since it is a product launched by the video website itself, it specially ADAPTS VR resources provided byiQIYI1 website.

The weight of existing wearable reality devices still gives users an extra feeling when using them. For example, the HTC Vive Pro weighs 500g, and the Pico Neo 2 weighs 320g. For reference, everyday glasses weigh between 20 and 80g. Therefore, the weight of VR display device itself is one of the important factors affecting user immersion, and how to achieve the lightweight and miniaturization of VR display device is still a problem to be solved.

a) HTC vive pro                          b) Pico Neo 2

Figure 1 video perspective helmet

Three, human-computer interaction technology

Another characteristic of virtual reality technology is interactivity. In order to enable users to interact with objects in the virtual environment, VR equipment manufacturers will provide pairs of interactive devices matching with display devices. Most of these interactive devices are designed in the form of joypads. The interactive device is equipped with gyroscope, position sensor, speed sensor, acceleration sensor and other components, and equipped with buttons with different functions such as push, pull and grip. Through these sensors and buttons, the system can display the relative position and movement of the user's body in the virtual environment, and realize the user's interaction with the virtual object. However, due to the lack of relevant motion feedback technology, the interaction details of handheld interactive devices are not perfect enough. For example, grasping and other actions can only be realized through buttons rather than the actual grasping actions of users. Therefore, grasping and other actions in a virtual environment will reduce users' immersion.

Handheld interactive devices can only realize the interaction between the upper limbs of users and virtual objects. For some scenes requiring whole-body interaction of users, Time-of-Flight (TOF) cameras or motion capture cameras can be used for acquisition. However, due to the need to collect and screen a large amount of video information, it is difficult to ensure the real-time performance of human-computer interaction using video collection, which will cause the user to have a sense of hysteria and vertigo.

Therefore, the research and development of new interactive devices that can provide users with force, temperature, touch and smell feedback is also one of the research directions of virtual reality technology.

IV. Application scenarios

As an emerging technology, virtual reality technology has been applied in many aspects such as skill training, manufacturing assembly, digital medical treatment, building construction, gaming entertainment and higher education.

Literature [4] introduces the application of virtual reality technology in the construction industry. By equipped construction workers with augmented reality equipment and adding safety alerts in the event horizon through visual identification, the occurrence of safety accidents can be effectively reduced [4]. In Literature [5], the Unity3D engine is used to build a virtual mechanical laboratory system including mechanical arm, reducer and other equipment, which can be used for teaching and practical training of mechanical students. In addition to improving the teaching efficiency, the hidden danger of safety in practical practice has been avoided [5]. Literature [6] uses the method of large-scale Geographic data analysis to develop a Geographic Information system (GIS) based on virtual Geographic scenes, filling the gap between GIS and actual geography [6]. Literature [7] established an interior decoration model using virtual reality technology, enabling users to intuitively feel the design scheme, alleviating decoration problems such as irrational decision-making of customers and information asymmetry of both sides, and reducing rework and material waste in the decoration process [7].

In addition to the above-mentioned literature, there are many other applications of virtual reality technology. For example, the US Army has developed a VR simulation training program called STE, which enables soldiers to familiarize themselves with the environment in a unified virtual system before entering the war zone. China's military and armed police have also used a similar virtual reality system for anti-terrorism training and flight training. Huawei unveiled CyberVerse, a platform based on augmented reality technology, at the 2019 Huawei Developer Conference. The platform uses the user's phone's own camera and chip to achieve centimeter-level spatial positioning, and adds text, pictures and animations to the images captured by the camera in real time. The platform has been used in places such as Nanjing Road in Shanghai and Mogao Grottoes in Dunhuang to provide guided Tours and help for tourists. In terms of entertainment, Beat Saber, one of the earliest VR games, has garnered 39,856 reviews on Steam, with a 96% positive rating. At the same time, Bilibili, iQIYI and other online video platforms have also launched related video resources for VR devices.

V. Summary and prospect

In recent years, virtual reality technology has made significant progress, and its user needs and application scenarios have become clearer. This paper briefly introduces the research status of virtual reality technology and the problems still to be solved from four aspects of environment modeling, presentation technology, human-computer interaction and application scenarios.

The most core feature of virtual reality technology is immersive experience, and all related technologies and functions should be developed to improve the user's immersive experience. However, the large size and weight of virtual reality equipment, the lack of authenticity of virtual environment model, the low richness and poor real-time performance of human-computer interaction are all factors that restrict user experience. How to solve these problems and enhance users' immersive experience will become a hot topic in the field of virtual reality research.

In general, significant progress has been made in the theoretical research, system development and application promotion of virtual reality technology. With the development of mobile Internet technology and communication technology, the application range of virtual reality technology will be broader.

reference

Acta Photonica Sinica, 2015, 45(2): 157 -- 180. (in Chinese with English abstract) [2] Xiao J, Zhou Y, Zhou Z. Augmented Virtual Environment and Augmented Reality[J]. Acta Photonica Sinica, 2015, 45(2): 157 -- 180.

[2]BIOCCA F, DELANEY B. Immersive Virtual Reality Technology[J]. Communication in the age of virtual reality, 1995.

[3]WALSH K R, PAWLOWSKI S D. Virtual Reality: A Technology in Need of IS Research[J]. Communications of the Association for Information Systems, 2002, 8.

[4]LI X, YI W, CHI H L, WANG X Y, CHAN A. A Critical Review of Virtual and Augmented Reality (VR/AR) Applications in Construction Safety[J]. Automation in Construction, 2018, 86(November 2017):  150-162.

[5] Lin Lin, Zhang Yongchun, Shan Lijun. Research on Mechanical Simulation Teaching System Based on Reinforcement Learning and Virtual Reality [J]. China Equipment Engineering, 2020(22): 228-229.

[6] Lu G N, Batty M, Strobl J, Lin H, Zhu A X, Chen M. Reflections and Speculations in GIS: A Survey of Western China. International Journal of Geographical Information Science, 2019, 33(2): 346 -- 367. (in Chinese)

[7] Wang Wenxin. Application of virtual reality technology in interior design and decoration [J]. House,2020(33):20-21+47.