The speed and quality of maintenance and repair are critical in the modern, technology-packed built environment. Consequently, these were considered in an experimental project that tested how remote expert assistance using VR and AR technologies could help improve the productivity of field service.
I’m in a hall overlooking white mountain tops. It’s snowing. In front of me stands an avatar that explains to me what we can do together in this virtual space. He jumps away but I can still hear his voice from behind me. He fetches a chair and hands it to me. I grab it and inspect it. The next moment, a video starts playing on the wall. Later, my host shows me how to draw in three dimensions, how to make sticky notes, how to share a PC desktop, and how to use other collaboration tools.
This
experience took place at FAKE Production, a Helsinki-based digital image,
animation, and VR/AR studio. With VR glasses and hand-held controllers, I had
tried out Glue, their universal collaboration platform. This
is a soon-to-be-released service that you can use with VR/AR gear and on mobile
and desktop devices.
Glue is also one of the solutions tested in an experimental project called Expert assistance using VR and AR glasses. In this project, Sovelto, a Finnish educational company, wanted to explore the possibilities of using VR and AR solutions for field service. Over ten organizations took part in the project, which received funding from KIRA-digi, the national built environment digitalization program.
Maintenance and Repair
Need to Improve
In Finland
alone, maintenance and repair work costs society about 24 billion euros
annually. Sovelto’s project was focused on field service, which often involve critical
operations that take place at a customer’s premises.
According
to the report, typically the first field service visit fails in one out of four
cases, and on average, it takes 1.5 visits for the service worker to fix the
problem. This inefficiency stems mostly from poor preparation, not having the
right spare parts, and a lack of expertise.
Delayed
maintenance and repair can cause substantial financial losses for the customer.
Furthermore, incorrect fixes may lead to damage and can even make machines
hazardous to use. Also, as technical systems become ever more complicated and
globally distributed, it’s becoming harder to have a suitably skilled person
quickly available wherever they are needed.
Some companies
have started to address this problem through mobile and VR/AR technologies. For
instance, in 2016, thyssenkrupp announced the use of Microsoft HoloLens in their
worldwide elevator service operations. Using this system, their service
technicians can visualize and identify problems ahead of a job and can also have
remote access to technical and expert information on site.
Helping the Field Workers Virtually
One way to
improve the productivity of field workers is to offer them assistance remotely. Matti Pouhakka, the Sales Director
of FAKE Production, explains: “I see two specific situations where remote
assistance can come to the rescue. First, there’s the maintenance worker who
understands how to fix something, but needs an expert to tell them what to fix.
And then there’s a person—a technical cleaner, for example—who sees a problem
and could likely fix it themselves quickly if they could be guided by a professional
on what to do.”
As the Glue
example demonstrated, VR is a powerful tool for simulation, training, and
collaboration. In a maintenance scenario, an expert could show and train a
field worker or even several workers in a VR environment. A worker could then practice
their skills with a digital counterpart of the actual machine. The sessions could
even be recorded for further use.
Also, digital
twins of whole buildings and technical installations open up new uses for VR.
Real-time IoT data combined with accurate models could help planning changes
and identify problems and allow them to be fixed remotely. Pouhakka emphasizes
that digital twins are assets that grow in value over the years. Clients and
designers should understand this and see the creation of comprehensive digital
models as an investment rather than a cost.
Augmented or Mixed Reality
Applications
Kim Nyberg, Technology Director at Trimble, gave presentations, showed demos,
and carried out tests with prototypes and existing Trimble mixed reality
products at Sovelto’s workshops. Kim is an expert in BIM and mixed reality
solutions.
“Remote
expert assistance is already feasible with the commercial solutions that we currently
have,” says Nyberg. “For example, I showcased Trimble Connect HoloLens, which lets you collaborate with several people
simultaneously and see them as avatars. The attendees can study the same
virtual model of a machine or a building with HoloLens glasses and discuss it. You
can either be physically in the same room or can attend remotely.”
Nyberg also
demonstrated a mobile AR app, Trimble
SiteVision. With the app, a person inspecting a machine or a structure can place
a digital 3D model into its real environment. The user can then take a snapshot
of the combination, add notes, and upload the image to Trimble Connect. From
there, a field worker can carry out the necessary work.
Simplicity and Usability
Are the Keys to Success
Sovelto’s
project report and my interviews with Pouhakka and Nyberg point to a future
where remote expert assistance is as easy as calling on a phone today. Existing
solutions already make this possible, and at a reasonable price. However, for
remote expert assistance to really take off, both hardware and software
developers must provide solutions that work flawlessly in demanding field
service situations.
Nyberg
believes that in the next one or two years we’ll see lightweight MR glasses
that will be feasible even for consumer use. He also envisions systems that you
can control with eye movements and thoughts rather than needing to use the hands
or voice.
Simplicity
and usability are and will be the critical success factors. “The best test is
to be able to do something useful with a system when you use it for the first
30 to 60 seconds,” Nyberg stated, summing up.
The title image is courtesy of Trimble.
Responses