• Home
  • Scientific results
  • Augmented Reality technology facilitates new infrastructure for smart transportation - Dr. Tao Jiang

Augmented Reality technology facilitates new infrastructure for smart transportation - Dr. Tao Jiang


On November 18, 2020, Realize Medical announced the launch of its new integrated remote network collaboration feature in its Medical virtual reality (VR) platform, Elucis. Users in healthcare and education can now meet their peers in the Elucis Collaborative VR environment -- and the Elucis environment offers a safe solution to enable transmission and sharing of 3D content during the Novel Coronavirus pandemic lockdown. Using Elucis, different users can access virtual sharing environments together and create and edit 3D models of medical images simultaneously. Internal remote collaboration can be achieved with the click of a mouse by storing the project database on a network drive or an organization-approved cloud storage drive. Elucis will also allow projects to be developed so that users can collaborate with outsiders and share their work securely [1] (see Figure 1).


Figure 1: Researchers will use Virtual sharing access to Elucis to perform surgery on pathological organs

Elucis, a virtual reality platform, shows a recent example of AR's use in private collaboration. In fact, unable to get consumers to pay, AR devices began to move to the enterprise market, where demand was more defined, starting in 2017. After several years of product iteration and market education, AR equipment has gradually gained a firm foothold in vertical industries such as education, industry and security, attracting more and more government and enterprise customers. After 2018, the demand for AR in the industry gradually converges to two core scenarios: remote collaboration and visual guidance and training. In the fields of security and tourism, AR plays a similar role. It mainly provides users with additional information through image recognition, including the identity information of people and the introduction of museum exhibits. The steady development of AR technology in the industry reflects the weakness of AR technology in the consumer field :(1) the price of the first generation Google glasses is $1500, its extremely limited functions and too short battery life, etc., which made it get off to a bad start when mass production entered the consumer market in 2014 and quickly became the industry's "martyr". (2) Magic Leap One, a star AR company funded by Google and alibaba, only released its first AR glasses for consumers in 2018 after seven years of establishment and burning more than $2 billion. The $2,295 product was poorly reviewed by the media at its initial launch, resulting in a backlog that blocked the company's financing and led to the ouster of its founder.

The root causes of the development trend of AR technology in the two application scenarios are: (1) the internal cause is the development maturity of AR technology; (2) the external cause is that the industry and consumer have completely different underlying logic.

First, it is the maturity of AR technology development. The current battery technology, chip energy consumption, wearable experience and other aspects will limit the range of AR device, making it difficult to popularize. At the 2017 China Development Forum, Apple CEO Tim Cook said that "THE maturity of AR technology depends on the joint evolution of sensors, displays, batteries and other fields. AR was in Phase 0 at that time, and a huge breakthrough had not yet been seen". It is also due to the immaturity of these key system components that the current consumer AR devices are mostly split designs: the glasses are responsible for displaying AR images and video collection, and are connected to the external battery and computing module through the data line. This split design obviously limits the portability of AR devices, and it is doomed that it cannot be popularized in the consumption field in the short term.

Secondly, different underlying logic in the consumer and industrial application fields resulted in different product planning routes for AR technology in the two markets. (1) The underlying logic of products in the consumer domain focuses on "user experience". However, due to the shortfalls in battery technology, chip energy consumption, wearable experience and other aspects at the current stage, AR devices at the consumption level have the following defects: they cannot be worn for a long time, and they also encounter problems such as sweating and difficulty in battery replacement. In terms of gesture recognition, installing one more 3D camera will not only increase the cost by thousands of yuan, but also increase the power consumption and user's learning cost. Therefore, in the consumer field, traditional technology giants such as apple and Google have quietly turned their focus on AR technology to software in recent years. (2) The underlying logic of products in the industry is "cost reduction and efficiency improvement" : if you are a manufacturing company, this device can increase your production by 1%, or 1,000 sets of HoloLens, you will buy it. Moreover, products focusing on industrial applications are mostly specialized products that only need to meet specific tasks in specific scenes. Therefore, we should focus, make choices and refrain from functional selection, hardware configuration and industrial design. That is to say, the industry applies the underlying logic of "cost reduction and efficiency improvement" to focus on cost performance. Although the scheme may not be perfect, as long as the overall scheme can save costs and bring benefits to be multiplied, enterprises are willing to install. This is also the reason why AR equipment is gradually gaining a foothold in vertical industries such as education, industry and security, attracting more and more government and enterprise customers.

The underlying logic of industry applications points to a potential direction for AR applications in other emerging industries, such as new infrastructure for smart transportation.

There are four core demands in the field of intelligent transportation [2] :

(1) Safety. In terms of safety, under the coordination and control of the artificial intelligence system, people, cars and roads will interact in real time, and the probability of traffic accidents will be significantly reduced. And with the advent of driverless cars, problems such as drunk driving, road rage, red light running and fatigue driving will be fundamentally solved.

(2) Convenience. In terms of convenience, the current transportation system lacks systematicness and coordination, and different transportation modes fail to play a linkage role. Taking transfer as an example, the unreasonable setting of subway station and bus station results in high time cost for people to transfer. After the application of artificial intelligence technology, urban traffic flow changes will be predicted through the integration and analysis of various traffic data, to help transport operators better set up bus and subway stations, reasonable arrangement of routes, etc., to bring a lot of convenience to people's production and life.

(3) Efficiency. The intelligent transportation system can be optimized as a whole. Through the "intelligent transportation brain", resources of all parties can be coordinated to help people make more scientific and reasonable travel plans and improve the carrying capacity of the traffic network and traffic operation efficiency.

(4) People first. "Putting people first and serving the people" is the fundamental way for smart transportation to create value, and meeting the people's ever-growing needs for a better life is the inevitable choice for building socialism with Chinese characteristics. As a high frequency rigid demand, traffic will have a direct impact on people's living standard and quality. In the intelligent transportation system, human needs will be fully respected, and the system will allocate resources from the perspective of the entire traffic ecology of the city, putting people first and realizing a high degree of harmony among people, vehicles and roads.

Around the four core in the field of intelligence traffic demands, with the industry application of "efficiency" as the core of "authors efficiency" of the underlying product logic, AR technology will likely be more efficient and convenient way of human-computer interaction, wisdom global awareness information network of intelligent transportation infrastructure, imported into the intelligent driving a storehouse, directly improve the passenger driving experience.

In MISSION: Impossible 4, the image on the front windscreen of the BMW I8 concept car can not only display navigation information, but also show pedestrians and obstacles on the road ahead beyond visual range before the naked eye sees them, and the touch operation is even more dazzling (as shown in Figure 2). In fact, reversing visual, HUD, panoramic image and so on are all mature applications of AR technology in the automotive field.

 image.png   image.png

Figure 2: AR HUD of BMW I8 concept car, with realistic navigation information on the front windshield

But even "mission: impossible 4" border on science fiction in the AR HUD (based on augmented reality of reality equipment), in front of our country is developing intelligent snatched the car still conservative like a little brother: for wisdom transport infrastructure can provide new global intellisense information of road network, this will make the car the degree of automation of operation, safety degree ascend to a new height. Autonomous driving currently requires three major infrastructures:

Is a smart made car, it is to point to pick up the advanced vehicle sensor, controller, actuator and other devices, and the integration of modern communication and network technology, realizes the car and car, car and road, cars and people, cars and the cloud is (server), intelligent information exchange and sharing, have complex environment perception, intelligent decision-making, coordination control and can realize the safe, efficient, comfortable, energy-saving, and eventually replace people to operate a new generation of car.

Second, it is necessary to provide various information to intelligent connected vehicles. Through the vehicle network constructed by "5G+ Beidou", V2 vehicle road and vehicle cooperation with the surrounding environment can be realized. It is a ubiquitous information network that provides accurate spatial-temporal location services to intelligent connected vehicles.

The third is autopilot high-precision map. Are maps infrastructure, too? Yes, our country's future as a smart planet, smart regional management and so on will depend on maps. For autopilot maps, high-precision maps contain static information just like the current maps, such as the shape and type of the road, which can control the speed of the curve. When entering under the viaduct or inside the tunnel, the inertial navigation system will be automatically activated to make up for the shortcomings of the Beidou navigation system.

With wisdom global intellisense new construction of road network traffic information, can greatly satisfy the intelligence transportation four core claims of "security", "convenient", "efficient", only without the "people-oriented" the key factor - because if global intellisense network information can only be involved in vehicle automated driving vehicle operating system the background, although unknown to improve the running efficiency of traffic, but the user is non-inductive. In order to let users can feel the wisdom of traffic convenience experience, intelligence traffic intelligent information must be provided by the new infrastructure in the process of human-computer interaction, must be impact to the user's "eyes" and "ears", which requires an effective interactive media - AR HUD is such a very appropriate human-computer interaction equipment. In this sense, AR HUD can effectively contribute to the new infrastructure of smart transportation. In the subsequent product prototype conceptual design, it is necessary to bear in mind the different underlying product logic of "industry application product" and "consumer product", which can not only reduce cost and increase efficiency for manufacturers, but also take into account user experience. In this process, what information is the most useful and audiovisual impact for users to access to AR HUD of smart cars through intelligent network connection facilities? In what ways does the autopilot high-precision map involve the user's driving behavior? These are all meaningful subjects.


[1]S. Sprigg, Realize Medical releases remote collaboration feature for its ‘Elucis’ VR platform for 3D content creation and sharing | Auganix.org. [Online]. Available: https://www.auganix.org/realize-medical-releases-remote-collaboration-feature-for-its-elucis-vr-platform-for-3d-content-creation-and-sharing/ (accessed: Dec. 1 2020).

[2] Zhao Guanghui, redefining the transportation: Artificial intelligence leads the transportation revolution, 1st Ed. Beijing: Machinery Industry Press, 2020.