On October 15, 2020, Librestream [1], a provider of Augmented Reality (AR) and remote collaboration solutions, announced that Siemens Energy, a leader in the field of energy infrastructure, will take Onsight Augmented Reality platform [2] based on Librestream as the development platform of its Connected Worker solution (Figure 1).

Figure 1: The "Connected Worker" solution is supported by Librestream's Onsight Augmented Reality platform, which can provide immediate, responsive services to the global energy customer base

According to Librestream [3], Siemens Energy is deploying Connected Worker solutions for its thousands of field service personnel and external contractors that collectively serve thousands of global Energy and power customers. Based on the Connected Worker solution, teams of specialists can remotely inspect, diagnose, and maintain equipment (such as industrial steam and gas turbines, onshore and offshore oil and gas platforms, factory instrumentation and controls) in a Connected scenario as shown in Figure 2.

Figure 2: Functional schematic diagram of Remote expert collaboration of Siemens Energy

Librestream pointed out that the AR based Connected Worker solution can significantly shorten the time to solve industrial field problems, improve production efficiency and Worker safety, thus ensuring the normal operation of industrial production facilities for a long time and bringing benefits to customers. Based on AR Connected Worker solution in digital way to make capital intensive front-line workers would be able to improve productivity and safety, through real-time remote observation can make "virtual expert" obtain first-hand information industrial field problem (and experts don't have to be personally on the scene) - solve time shorten mean huge cost benefits.

Connected Worker software includes the complete Librestream Onsight platform (system function block diagram is shown in Figure 3), including remote expert assistance, digital workflow program, workspace document management, artificial intelligence function, professional on-site hardware inspection. The Librestream solution is also integrated with Microsoft Teams to quickly scale and simplify collaboration across Siemens Energy. The Connected Worker solution of Siemens Energy consists of the following Librestream components:

• Remote expert collaboration optimized for complex field environments;

• Integrated artificial intelligence (such as computer vision-based target recognition using Internet of Things data);

• Digital workflow and documentation management to guide, access and disseminate Tribal Knowledge across all departments;

• Professional hardware, including Onsight Hub adapter and Onsight Cube, a certified thermal imaging wearable device;

• Integrate with collaboration systems such as Microsoft Teams to simplify communication across the organization.

Figure 3: Onsight Augmented Reality Platform

The Connected Worker of Siemens Energy is only at the starting stage. Although we do not yet know the specific details, we can infer the general appearance of the Connected Worker scheme through other similar projects. Although the Connected Worker solution of Siemens Energy is based on augmented reality technology, to some extent it can be regarded as an extension of the application of "virtual maintenance" in virtual reality technology [4].

Virtual maintenance is an important research direction of the virtual technology in recent years, the purpose is through the adoption of computer simulation and virtual reality technology, the real show on computer equipment maintenance process, strengthen the equipment life cycle, improve the stages on the various maintenance decision-making ability, including the maintainability design and analysis, the maintainability demonstration, maintenance process verification, maintenance training implementation, etc., as shown in Figure 4.

Figure 4: Virtual maintenance

The most typical application of VR in equipment maintenance is the virtual maintenance system of The Us Hubble Telescope (HST). Shortly after Hubble entered orbit in April 1990, astronomers discovered a flaw in its optical system, so NASA created a virtual space environment (the Hubble Telescope (HST) virtual repair system, shown in Figure 5) for mission astronauts to complete various simulated repair activities. After training, the HST maintenance mission was successfully completed in December 1993. This is the first large-scale application of virtual maintenance technology to complete the actual task, and achieved success.

Figure 5: Virtual maintenance system for the Hubble Telescope (HST)

An example of an immersive maintenance verification system is presented below, as shown in Figure 6. The system is mainly composed of: (1) server maintenance tasks/scene, (2) immersion simulation terminal maintenance, (3) to monitor/command terminal, modeling management terminal (4), (5) model conversion process, (6) maintenance terminal, (7) control/viewing terminal of several parts, such as main function of each part are described below.

Figure 6: Immersion Maintenance verification system

(1) Maintenance task/scenario server: it is composed of task/scenario monitoring server, file/database server and corresponding management service program. The function is to manage the model library/scenario library/task library data, monitor the running status of maintenance tasks and scenarios, manage user permissions, manage user maintenance records, and perform necessary coordination among clients.

(2) Immersive maintenance simulation terminal: it consists of two parts: capture processing and helmet image rendering. The former is composed of optical capture equipment, data garment (or optical identification equipment), data gloves and capture processing computer, while the latter is composed of data helmet and helmet graphics rendering computer. Among them, the role of the capture and processing computer is to capture the position of each part of the human body in real time and convert it into the data of human posture. The role of the helmet graphics rendering computer is to receive the captured 3D human posture data, combine the data of the maintenance scene, generate THE 3D graphics (graphics of left and right eyes), and output the data to the helmet.

(3) Monitoring/command terminal: it is mainly composed of projection graphics rendering computer, large screen stereo projection system and corresponding software. The function is to receive the captured 3D human posture data, combine with the data of the maintenance scene, generate 3d graphics, and output them to the double-channel 3d projection system for leaders and experts to inspect, evaluate and direct the maintenance simulation process. The stereo projection system is composed of stereo projector, projector large screen, stereo eyes, etc.

(4) Modeling management client: It mainly consists of modeling management computer, modeling management program and model transformation program. The function is to receive 3d digital model and product structure data from CAD/PDM software.

(5) Model transformation processing: generate model library data and store it to the server. At the same time, the maintenance scenario database data is generated and stored to the server by selecting the model and defining constraints. By selecting scenarios and default maintenance procedures, maintenance task database data is generated and stored to the server.

(6) Maintenance analysis terminal: Mainly maintenance analysis computer and maintenance analysis software. The function of this part is to record the virtual maintenance process, maintenance obstacle analysis, maintenance time analysis, maintenance space analysis and so on.

(7) Control/observation terminal: it is composed of graphics rendering computer and corresponding software. The role of this part is to control the start/end/pause of the maintenance simulation task, control the personnel involved in the simulation and observation, help the positioning of the personnel involved in the simulation, coordinate the behavior of relevant personnel, and provide the real-time display and graphic Angle change of the maintenance scene. This part can be divided into two operation modes: control and observation. The former can control and watch the scene in real time. The latter can only watch, not control the scene.

The above "Immersion Maintenance Verification System" mainly provides the following services:

(1) Maintenance task, scene establishment and management: Using the data interface to obtain the digital model and maintenance planning data of the product from the product data Management system (PDM) and maintainability design/analysis software, to build the 3D product model library, maintenance scene library and maintenance task library.

(2) Simulation and verification of maintenance process: In the maintenance task scene, operators carry out various simulation operations according to the equipment maintenance and maintenance order in the digital THREE-DIMENSIONAL scene with the help of virtual reality equipment, and experience the maintenance and operation process in person. In view of the established maintenance task, the rationality of the maintenance plan is preliminarily verified, which provides the basis for the improvement plan and the establishment of maintenance regulations.

(3) Analysis of maintenance visibility, accessibility, maintenance obstacles and maintenance space: In the virtual maintenance and maintenance scene, the operator can change the head attitude at will to investigate the visibility; The operator inspects the accessibility through the movement of the limbs. By checking the geometric interference between the human body and the equipment and facilities in the 3D scene, the maintenance obstacle is analyzed. The space distance between limbs and equipment and between tools and equipment is calculated to analyze whether the maintenance space is sufficient.

(4) Comprehensive preliminary evaluation of maintainability: record all kinds of key information in the maintenance simulation process, including time, route, frequency of abnormal conditions, etc., and conduct preliminary comprehensive evaluation of the rationality and efficiency of the maintenance planning. Through the operator in the simulation process of personal experience, such as work posture, load, mutual coordination, etc., the initial assessment of maintenance personnel's feelings.

(5) Production of maintenance training materials: The maintenance simulation process in the 3D digital environment is recorded to form an interactive 3d animation, which can be embedded into the equipment maintenance training manual to improve the intuitiveness of equipment maintenance training and reduce the training cost.

Through the functional services and hardware components of the above "immersive maintenance verification system", we can have a general prediction of the system structure and functional framework of Siemens Energy's Connected Worker solution based on augmented reality.

In fact, from the perspective of user group distribution, the applications of VR or XR (Extended Reality) [5] can be divided into two parts. One is a complex VR application system that requires software development and is aimed at B-end (enterprise level) users (The Connected Worker of Siemens Energy is an application of this industry). The other is relatively simple, content-only VR industry content aimed at C-end (ordinary consumer) users. C-terminal application is mainly content, which can be simplified by B-terminal application migration. B - terminal application is mainly based on the above plus industry specific content to complete. In other words, XR solutions for industry applications are more empowered by traditional industries themselves through XR technology, thus driving their own industrial upgrading. The difference of THESE XR solutions for applications in different industries is mainly based on the business requirements of different industries and the adoption of different customized services on the XR common technology stack.

Librestream's Announcement of a Connected Worker Solution on 15 October 2020 is the second time in a few weeks that Siemens has deployed an extended Reality (XR) solution across its business. VinciVR announced last week that through their partnership with Siemens, VinciVR and Siemens have received funding from the Massachusetts Clean Energy Center to develop VR training simulations for the emerging offshore wind industry. As a leading technology solutions company with deep expertise in infrastructure, energy, automation and manufacturing, Siemens is likely to continue to implement XR technologies throughout its future operations.

reference

[1] Librestream, Empowering Service Workers With Augmented Reality. [Online]. Available: https://librestream.com/ (accessed: Oct. 19 2020).

[2] Librestream, Augmented Reality Platform for the Connected Worker - Onsight. [Online]. Available: https://librestream.com/products/ (accessed: Oct. 19 2020).

[3] Librestream, Siemens Energy Selects Librestream to Help Enhance Worker Safety & Productivity Through Remote Services. [Online]. Available: https://librestream.com/company-news/siemens-energy-selects-librestream-to-help-enhance-worker-safety-productivity-through-remote-services/ (accessed: Oct. 19 2020).

Public Security systems (Procedia Manufacturing), public security (ISO), Public security (ISO), Public security (ISO), Public security (ISO), Public security (ISO), Public security (ISO) Vol. 19, pp. 156-163, 2018, doi: 10.1016 / j. romfg. 2018.01.022.

B. Marr, "What Is Extended Reality Technology? A Simple Explanation For Anyone, "Forbes, 12 Aug., 2019. https://www.forbes.com/sites/bernardmarr/2019/08/12/what-is-extended-reality-technology-a-simple-explanation-for-anyone/#3e8fe34b7249 (accessed: Oct. 20 2020).

[6] S. Sprigg, Vinci VR and Siemens Gamesa Renewable Energy win grant to develop Virtual Reality training for US Offshore Wind | Auganix.org. [Online]. Available: https://www.auganix.org/vinci-vr-and-siemens-gamesa-renewable-energy-win-grant-to-develop-virtual-reality-training-for-us-offshore-wind/ (accessed: Oct. 20 2020).