• Home
  • Scientific results
  • Augmented Reality (AR) : An application opportunity that is hard to contain -- Dr. Ding Ru

Augmented Reality (AR) : An application opportunity that is hard to contain -- Dr. Ding Ru


1 quote

"What we need is to go beyond the current state of the art," said Matt Grob, chief technology officer of Qualcomm's chip division. Imagine you're sitting in your office with a 4K resolution monitor or monitor on the wall, and even if it's 50, 60, 70 inches, it's only part of your field of vision. If you want to see the whole field, 4K is not enough. You need 8K or more. It requires a full view, and as you move left and right, you want it to have extremely low latency. It has to be able to track your gaze, and like Magic Leap's technology, it has to track your focus, your vertices, and it has to provide gigabit speeds so that everyone can access the content at the same time. It requires light weight and low power consumption, so it requires several times the computing power compared to what we do today." -- Referring to two characteristics of 5G technology: fast speed and low delay.

Therefore, with the rapid development of 5G technology, AR/VR/MR has gradually become the "killer application" of 5G. According to the data in recent years, Augmented Reality (AR) has more obvious and outstanding application and development advantages. Augmented reality has received wide attention of academia and industry, in the hardware device, and the tracking precision, the actual synthesis is rapidly developing and improving. [1], for example, in 2013, the enhancement of the Expo (Augmented World Expo), the vast majority of exhibitors are concentrated in SLAM system eye tracking technology and wearable enhancement devices, such as 3 d glasses Atheer One they expect highly integrated portable devices, can stimulate the rapid application of existing Augmented reality. At the same time, the emergence of new products will also bring more research content to researchers, and promote the development of AUGMENTED reality technology towards a higher direction.

1.1 Relationship and distinction between VR and AR

Virtual Reality (VR) establishes an artificially constructed THREE-DIMENSIONAL Virtual environment in which users interact and influence with objects in the Virtual environment in a natural way, greatly expanding human's ability to understand the world, simulate and adapt to the world. Virtual reality technology began to rise from the 1960s to the 1970s, and began to take shape and develop in the 1990s. It has solved some major or universal needs in simulation training, industrial design, interactive experience and other application fields, and has made great progress in theoretical technology and application development. The main scientific problems of virtual reality include modeling method, presentation technology, human-computer interaction and equipment. However, problems such as large amount of modeling work, high simulation cost, insufficient matching degree with the real world, and credibility are common at present. [2]

Augmented Reality (AR) is a real-time computing technology for the position and Angle of camera images, supplemented by corresponding images. The technology uses holographic projections to overlay the virtual world with the real world on the display screen of the lens, allowing the operator to interact with the device.

VR and AR are both virtual imaging, but there are essential differences in the implementation technology. The visual presentation mode of VIRTUAL reality is to block the connection between human eyes and the real world, and to create a new world through the real-time rendering of images by devices. The visual presentation mode of AUGMENTED reality is to overlay holographic images to enhance the visual presentation of human eyes connected to the real world.

1.2 The distinction between augmented reality technology and augmented virtual environment technology

The technology of virtual reality augmentation is to match and synthesize the virtual environment and the real environment to realize the augmentation. The technology of superimposing the 3D virtual object to the real world display is called augmented reality, and the technology of superimposing the information of the real object to the virtual environment rendering is called augmented virtual environment. These two types of technologies can be described visually as "virtual in reality" and "virtual in reality" respectively. Virtual reality augmented technology reduces the workload of 3D modeling through the synthesis of real world and virtual environment, improves the sense of user experience and credibility with the help of real scenes and objects, and promotes the further development of virtual reality technology.


Figure 1: Enhanced virtual environment technology "virtual in reality"


Figure 2: Augmented Reality "reality in reality"

1.3 Relevant international standards

In August 2011, after discussion at the SC24 Plenary session held in Rapid City, THE United States, it was considered to add WG9 to conduct standardization work in the field of virtual reality/augmented reality/mixed reality, and to expand the "virtual reality" (VR) in SC24's original work scope to "virtual reality/augmented reality/mixed reality" (VR/AR/MR). The south Korean delegation submitted a formal definition of a new name, scope and detail [4], China and the United States delegation name and scope of the concrete Suggestions were proposed in the paper, especially the Chinese delegation suggest using "virtual reality Continuum" (VR Continuum) instead of lengthy and meaning of cross "VR/ar/mixed reality". The name AR Continuum was adopted by SC 24, and eventually WG6 was changed to AR Continuum Presentation and Interchange. The WG9 Working group was formed and named AR Continuum Concepts and Reference Model.

2. The popularity of AUGMENTED reality is growing year by year

Search fever represents the level of public interest in the word. Typically, a new technology leads to a search boom, then a slow decline, a surge in the event of a technology breakthrough or a hot event, and eventually a plateau.

2.1 Domestic hot word search and information trends

Figure 3: Comparison of Domestic search indexes between 2015 and 2020 (VR/AR/MR/HCI)image.png

We used Baidu Index to compare the global search popularity of words such as virtual reality, augmented reality, augmented virtual environment and mixed reality. For reference, human-computer interaction (HCI) was taken as the reference. The comparison of search results is shown in Figure 3. The explosive growth of VR search in 2016 is mainly based on the following symbolic events :(1) 2016 technology, media and telecommunications industry forecast released by deloitte believes that virtual reality will become the most important industrial trend in 2016, and 2016 will become the first year of VR industrialization. (2) On the eve of MWC 2016 mobile World Congress in Barcelona, Samsung held its spring 2016 new product launch event. Different from the past, Samsung released its new mobile phone S7 through virtual reality (VR) technology at the event site. Since then, THE MWC technology giant has led the VR craze and started the global melee. (3) The potential users of VR in China have reached 286 million, and the cultural industry and real estate industry are the most important in the layout of VR industry in China.

On January 17, 2017, AR really attracted public attention in the event of "Baidu unlocking AR+ new public welfare method, subway resurrecting 'Lao Jiu Men' -- using augmented reality (AR) technology to restore precious historical sites". Since then, the popularity of VR search gradually decreased, and the popularity of AR search was relatively stable. (Note: Baidu Index does not include full names of keywords, and search data can only be traced back to 2015 at the earliest)

Figure 4:2017 ~2020 Domestic Information Index Comparison (VR/AR/MR/HCI)


As can be seen from FIG. 4, the information heat of AR has been increasing year by year since 2017, with more and more peak events, and the average AR information index is slightly higher than VR. (Note: Baidu Index does not include full names of keywords, and information data can only date back to 2017 at the earliest)


Figure 5: Average Domestic information index (VR/AR/MR/HCI) from 2017 to 2020

2.2 Comparison of search trends of foreign hot words


Figure 6: Virtual reality, augmented reality, and mixed reality search traffic statistics (Source: Google Trends, 2004.01~2020.12)

It can be seen from Figure 6 that, like human-computer interaction, virtual reality technology is maturing and gradually being accepted by the public. The latest peak is Facebook's acquisition of Oculus, a virtual reality eyeglasses company. , by contrast, the attention of augmented reality in 2008 began to rise rapidly, and appear multiple consecutive heat peak, by looking at its Associated Press can be found that the heat is mainly 2007 Apple iPhone, mobile Internet rise driven, especially in the camera and gyro become the standard of smart phones, for augmented reality provides sufficient space for development. Among them, Google Glass and online virtual fitting have attracted a lot of attention. However, the search volume of augmented virtual reality, augmented virtual environment, augmented Virtuality and other words is far less than the magnitude of the above hot words, reflecting that the technology of augmented virtual environment has yet to make a breakthrough, and no important events have yet occurred.

It is worth mentioning that there are great differences in the sensitivity and follow degree of technology at home and abroad: the technical sensitivity of foreign countries is high, and the rise of big data statistical tools in foreign countries is earlier, so the inflection point appears earlier than in China.

3. Application practice and scenario of AUGMENTED reality technology

3.1 Industrial manufacturing and maintenance

This is the first industry where augmented reality has been used. The head-mounted display is connected to the specific object to be constructed, and then the image of the object is described on the display [5]. Microsoft's Hololens is capable of superimposing 3d images on real scenes.



Photo 7: A technician wearing Holoens glasses takes a view of a scene

3.2 Medical field

Doctors use of augmented reality, in the process of surgery patients, can see registered to patients [6] MRI and CT images, graphics drawing module can be dynamic real-time rendering of virtual model, at the same time will play video file, figure 8 Hololens glasses of augmented reality application interaction effect showed good biological experiments [7]. Moreover, augmented reality technology has been applied in medical education and training, greatly enhancing the treatment level of medical staff. In addition, AUGMENTED reality can also be applied to virtual surgical simulation, virtual human function, virtual human anatomy and remote surgery.


Photo 8: A scene in the vision of a medical worker wearing Holoens glasses

3.3 Military field

The military can use augmented reality to provide important information about the surrounding environment, such as showing the entrance on the other side of a building or showing troop movements, allowing soldiers to move out of sight of the enemy.

Experts at BAE Systems and the University of Birmingham in the UK have developed a prototype military combat guidance system. The system is equipped with the Oculus Rift virtual reality headset. With these technologies, officers can deploy strategies directly from a portable intelligence center and gain access to frontline operations.


Figure 9: Application of AR in the military field

3.4 The field of TELEVISION broadcasting

Augmented reality technology can provide real-time program information for TV stations when broadcasting programs. When CCTV5 broadcasts the 2018 World Cup, it uses graphic markers with natural features to locate the location of the stadium and its internal details, so as to display the 3d rendering of the World Cup stadium in the live program. After the camera captures the mark, the position information of the natural mark is calculated by the background computing system, and finally the 3D animation is drawn on the screen by the 3D rendering system.


Figure 10: Application of AR in TV retransmission

3.5 Game Field

Pokemon Go is a mobile game co-produced and developed by Nintendo, The Pokemon Company and Google Niantic Labs. It is a game that explores, captures, fights and exchanges pokemon that appear in the real world [7]. Players can use their smartphones to discover pokemon in the real world, capture and fight.


Figure 11: Application of AR in the field of entertainment and games

3.6 Field of education and training

(1) AR training: The content to be learned will be presented on the screen in 3D, or even 1:1 restore the model to show the details of the internal structure of the model, and interact with the real scene at the same time, or conduct online simulation operation. Applicable direction: Medical school, military school, architecture major, etc.


Figure 12

(2) AR modeling: Learners can intuitively see what a specific project looks like in different environments. Models can be quickly generated, manipulated, and rotated. Students receive immediate feedback, which allows them to spot flaws in their designs. Applicable direction: architecture major, design major, etc.


Figure 13

(3) AR books: THE appearance of AR books is the same as that of traditional books, but when scanning the book reservation content with the camera, 3D animation, video and sound will be displayed, and some interactive elements can even be done. Application direction: library, teacher-assisted instruction, publishing house, preschool education, etc.


Figure 14

(4) Discovery learning: Discovering new knowledge alone enables students to learn quickly and effectively in the process of solving problems. Using AR technology, combined with the real school scene or knowledge, design such as campus tour guide, historic site tracking and other applications. Application direction: library, museum, exhibition hall, campus, etc.


Figure 15

(5) Higher education VR training Room: Virtual reality technology laboratory is established according to the needs of the school's own scientific research and teaching. In these specific courses, teachers and students only need to wear VR equipment to see the virtual platform and teaching facilities, so as to simulate operation and practice.

(6) Special professional AR training: In the skill training of medical school, military school and higher vocational college, three-dimensional simulation modeling is carried out, such as human body structure, military equipment and all kinds of electrical appliances, etc., and flat panel display and interaction are conducted in class.

(7) 3D exhibition hall construction: Not only enterprises, but also more and more schools have the demand for 3D exhibition hall construction. School culture and characteristics are reflected in the exhibition hall in the form of AR, which can increase the fun of visiting and learning and improve the overall image of the school.

(8) AR tour planning: AR technology is used to provide campus tour, providing students with detailed information about various parts of the campus, which is especially convenient for students or visitors to get to know the school quickly.


Figure 16

3.7 Restoration of historic sites and digital heritage protection

Information about cultural relics is provided to visitors in the form of augmented reality. Users can not only get the estimated text explanation, but also see the virtual reconstruction of the incomplete parts of the ruins.

The application of digital reproduction of Yuanmingyuan landscape based on augmented reality technology is being studied in China. Visitors can point their mobile phone cameras at the site and see what it looked like before it was destroyed. Visitors can learn all about the site, including when it was built and who presided over it and what materials were used. [8]

ARCHEO GUIDE is a personal electronic tour GUIDE project. The system naturally seamlessly renders the scene of the ruins captured in the video with a virtual 3D model on the portable device screen. [9]

3.8 Home Furnishing

Ikea's 2014 product manual includes an augmented reality catalog, an app that uses Metai's image recognition technology to identify specific pages and images.

The app can use a physical catalog as a standard size to determine the general size of a piece of furniture, and then project it onto the device's display screen, allowing users to see how a particular piece of furniture they want looks in their room and whether it fits.


Figure 17

3.9 Media and entertainment fields

(1) Media: Compared with traditional print media, holographic imaging equipment relying on 3D display technology is widely used in advertising media, such as commercial advertising, exterior rendering, product display, sand table, exhibition, etc.

(2) Entertainment: The current mainstream parallax 3D display technology includes wearable parallax 3D display technology, raster naked eye 3D display technology and directional naked eye 3D display technology. Their applications include holographic cinema, stage rendering and holographic restaurant.


Figure 18


Figure 19


[1] Zhou Zhong, Zhou Yi, Xiao Jiangjian. Overview of Augmented virtual Reality technology [J]. Scientia Sinica: Information Science,2015, 45(02):157-180.

[2]Zhao Q P. A survey on virtual reality. Sci China Ser-F: Inf Sci, 2009, 52: 348–400.

[3]Azuma R, Baillot Y, Behringer R, et al. Recent advances in augmented reality. Comput Graph Appl, 2001, 21: 34–47.

[4]Jones A, McDowall I, Yamada H, et al. Rendering for an interactive 360 light field display. ACM Trans Graph, 2007, 26: 40.

[5]SAR Display [EB/OL]. http://www. spatialar. eom.

[6]Glombitxa G, Evers H, Hassfeld, Engelmann U, Meinzer. Vir¬tual surgery in a teleradiology framework. Information Technology in Biomedicine[J]. IEEE Transactions, 1999, 3 ( 9 ) • 186 -196.

[7]Bajura Michael, Henry Fuchs • Ryutarou Ohbuchi. Merging Virtu¬al Reality with the Real World:Seeing Ultrasound Imagery Within the Patient [A]. Proceedings of SIGGRAPH [C]. 1992, 153 -161.

[8] Cheng Zhi, Jin Yifu. Research on Mobile Phone-based Augmented Reality and Its Mobile Learning Application D1 Electrochemical Education, 2013 (2): 66-70.

[9]Vlahakis V, Karigiannis J, Tsotros M, et al. ARCHEOGUIDE: First results of an AugmentedReality [A]. In:Mobile Computing System in Cultural Heritage Sites, Virtual Reality, Archaeology, and Cultural Heritage International Symposium (VAST 2001 ) [C]. Glyfada, Nr Athens, Greece (2001).