Augmentation Technology in Virtual Reality Technology _ Dr. Xu Cuidong

2021-01-22

Virtual reality is a new computer technology developed from computer vision, simulation technology, human-computer interaction and other information technologies. By separating the vision from the real world, people can be immersed in a completely virtual world through the display device. The so-called augmented reality is to superimpose virtual information on a location based on the recognition object, and the virtual information can be exchanged in real time to enhance the visual effect.

1. The problem

Virtual reality technology establishes artificial three-dimensional virtual environment, and users interact and influence each other with objects in the virtual environment in a natural way, which greatly expands human's ability to understand, simulate and adapt to the world. Virtual reality technology is widely used in simulation training, industrial design, interactive experience and other fields, to solve many important and universal needs. The main scientific problems of virtual reality include modeling method, presentation technology, human-computer interaction and equipment, but at present, there are many problems, such as large modeling workload, high simulation cost, insufficient matching degree with the real world and credibility.

To solve these problems, there have been multiple virtual augmented reality, reality and virtual environment matching synthesis to achieve enhanced, including 3 d virtual object to the real world technology, called augmented reality show, the real object of superposition of information to the virtual environment rendering technology, called enhanced virtual environment. These two types of techniques can be described visually as "virtual in reality" and "virtual in reality" respectively. Through the synthesis of real world and virtual environment, virtual reality augmented technology reduces the workload of 3D modeling, improves user experience and credibility with the help of real scenes and real objects, and promotes the further development of virtual reality technology.

 image.png

Fig. 1 "Fake in Reality"

 image.png

Fig. 2 "Reality in Void"

 

II. Technical Features

Now the industry generally recognized: from the real world to the virtual environment through augmented reality and augmented virtual environment these two kinds of virtual reality augmented technology. Generally speaking, the real world (computer vision), augmented reality, augmented virtual environment and virtual reality (VR) are collectively referred to as the VR continuum. Compared with the early stage, the concept of augmented reality or augmented virtual environment has changed a lot, and the technical field has been greatly expanded. However, their technical characteristics are inseparable from the following three points :(1) the mixing of virtual and real environment; (2) Real-time interaction; (3) Three-dimensional registration.

Augmented reality

Augmented reality technology, through real-time and continuous calibration of moving cameras or wearable display devices, projects 3D virtual objects into the user's visual mouth stably and consistently, achieving the performance effect of "reality with virtual". As shown in Figure 1, the real world is the physical space or its image space where we live, in which people and VR cards are virtual objects. As the viewpoint changes, the virtual objects also carry out corresponding projection transformation, making the virtual objects look like they are located in the three-dimensional space of the real world.

There is a special branch of augmented reality called spatial augmented reality, or projection augmented models. The computer generated image information is directly projected onto the pre-calibrated physical environment surface, such as a curved surface, a dome, a building, a group of real objects that finely control the movement, etc. In essence, spatial augmented reality projects the virtual objects generated by calibration onto the entire region of the preset real world as the surface texture of the real environment objects. Unlike traditional augmented reality, where the user wears a camera or a display device, this approach does not require the user to carry a hardware device and can support multiple people to participate at the same time. However, its performance is limited to a given object surface, and because the projected texture is viewpoint-independent, it is slightly less interactive.

2. Enhance virtual environments

The technology of enhanced virtual environment establishes 3D model of virtual environment in advance, extracts 2D dynamic image or 3D surface information of real object through the prior or real-time calibration of camera or projection device, and fuses the image area or 3D surface of the object into the virtual environment in real time, so as to achieve the performance effect of "reality in virtual environment". As shown in Figure 2, real-time images from the real world appear in the virtual environment, in which the texture on the VR card and the human body are both from the images collected by the camera, and the human body can even be real-time 3D objects and surface texture images.

In contrast to the projection enhancement model technology existing in augmented reality, there is also a corresponding technology in enhanced virtual environment technology, which covers the entire virtual environment with images collected by the camera, that is, as the texture of the virtual environment model, users can carry out interactive three-dimensional browsing with high sense of reality. When the three-dimensional model is the inner surface of a universal shape such as a sphere, cylinder, cube, etc., the technique is the now-ubiquitous panoramic image or video. Panoramic video projects a fish eye or several conventional images of the real world onto the 3D model to construct a single point of omni-directional fusion effect. Mosaic between multiple images can be in the way of image feature point matching or camera calibration in advance.

III Hardware Facilities

Augmented virtual reality technology needs to be combined with the real world, which inevitably requires cameras and other sensing devices and display devices. Therefore, the development of related hardware devices has an important impact on this technology.

1. Camera: Camera is the most important hardware device of augmented reality technology. A large number of camera tracking and calibration technologies are based on a simple camera.

2. Position and Angle Sensor: Position and Angle Sensor can play an important auxiliary role in camera tracking.

3. Wearable enhanced display device: Wearable enhanced display device can be divided into headwear, hand-held and spatial projection.

4. Sensory interactive equipment: Sensory interactive equipment can collect three-dimensional motion data of human body, provide an important way to synthesize real world sensing data into virtual environment, and is an important equipment to enhance virtual environment technology. These motion-sensing interactive devices can accurately represent the real world human movement in a virtual environment in real time and enhance the interactive ability of virtual reality.

5, 3D display equipment: this kind of equipment needs to wear the corresponding 3D glasses, each frame includes two images of the corresponding left and right eyes. These binocular display methods can only view fixed visual images, and can not get the real viewing experience of following the head or eyes. In recent years, there have also been some "naked eye 3D" products.