How do augmented reality navigation systems work?

According to The Swedish Club, navigational accidents made up 30% of claims by type and 33% of claims by cost between 2014 and 2018. Between 2013 and 2017, lack of situational awareness was cited as the most common cause of collisions (45%), and the second most common cause of the most expensive claims (27%). But why is this?

If you’ve never stood a navigational watch, the bridge watchkeeper’s job appears straightforward: don’t hit anything. However, as any bridge watchkeeper will tell you, it’s rarely that simple.

Many bridges are poorly designed, with incompatible equipment installed in inconvenient positions. Watchkeepers have to break their visual lookout to refer to their equipment. I’m sure many watchkeepers have fantasised about a unified system after a busy watch spent trying to match digital information with the view from the window.

What if AIS and automated radar plotting aids (ARPA) information were floating in a bubble above other ships, and the course line, buoys, shallow patches and no-go zones were visible in the real world? Well, those systems exist: it’s called augmented reality (AR) navigation, and it’s here.

What’s augmented reality?

AR overlays digital content on the real world, either on a screen or using AR glasses. Unlike in virtual reality (VR), AR can anchor digital information to real-world objects, even if they’re moving. For example, rather than constantly cross-referencing radar and AIS, information about a target vessel can be displayed in an AR bubble near the target when you look out of the window. 

If you know the smartphone game Pokemon Go, you know how AR works on a phone. Car manufacturers are already building AR navigation systems into car windscreens, so it should be no surprise that it’s coming to ships. You can read more about AR and related technology here.

What’s wrong with current navigational systems?

Endsley defines situational awareness as:

The perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.” 

This is critical to safe marine navigation, but too often it’s an aspiration rather than a reality.

The inefficiency associated with task-switching is a well-studied phenomenon. Even a well-rested person can only keep about seven different pieces of information in their working memory. Yet the bridge of an average merchant ship has more aids to navigation than navigators of the past ever dreamed of.

Despite that, we expect watchkeepers to track information from assorted equipment, match that disparate information to the changing situation, process it, draw the correct conclusions, and act on their decisions. All while monitoring three or four radios, filling in the logbook, and standing ready to respond to alarms. It’s no surprise they sometimes miss something, with often catastrophic—and expensive—results.

Why AR navigation?

Source: Raymarine

AR navigation systems make relevant information readily available in an accessible format at the point of use. This reduces the cognitive load on the navigator and allows them to focus on the big-picture, rather than on the constant task-switching of gathering and cross-referencing information.

How does AR navigation work?

The AR system overlays relevant information on stabilised video images from a forward-facing camera. Just as navigators can choose which information layers to display on an electronic chart display information system (ECDIS), AR navigation users can choose to display AIS, radar, ECDIS, gyrocompass and route information on their AR system. AIS and radar targets change colour according to their threat level, and users can increase or decrease range and take bearings, just as with a radar or ECDIS.

The systems alert users to buoys, vessels and other targets of interest, displays shallow water, no-go zones and the planned track, and even includes information from other navigational instruments. At a glance, the watchkeeper can check important voyage information such as the speed of other vessels, the closest point of approach and the time of closest point of approach of targets.

In restricted visibility, or at night in heavy coastal traffic, AR navigation systems offer a notable improvement in situational awareness, clearly indicating which lights are likely to be ships rather than shore lights, and where to look for targets in fog.

Who uses AR marine navigation systems?

Mitsui O.S.K Lines and Furuno

Mitsui O.S.K. Lines (MOL) and Furuno collaborated to develop an AR navigation system. In March 2019, MOL installed it on their car carrier Beluga Ace, and in October 2019 on their new very large crude carrier (VLCC) Suzukasan as a trial. They’re planning to expand it into their twenty-one VLCCs, then to their energy transport fleet, including LNG and dry bulk carriers.

At the same time as MOL’s second trial, in October 2019 Furuno announced the Envision AR Navigation System.

Raymarine

Raymarine takes a different approach, offering their Raymarine Clearcruise AR as a free software upgrade for their existing Axiom multifunction displays rather than a standalone system.

Conclusion

AR navigation systems allow easy access to all relevant electronic information on a single screen. This improves situational awareness, decreases cognitive load, and makes validating and cross-checking navigational information easier than ever. Improved situational awareness leads to a reduction in navigational accidents, safer ships, and inevitable cost savings in repairs and insurance premiums.

Given the constant, valid reminders that navigational watchkeepers need to spend less time looking at equipment and more time looking out of the bridge windows, it’s hard to argue against a system designed to let them do exactly that.