Augmented reality helps helicopter pilots perform complex maneuvers at sea

Be it fog, rain or darkness, landing in the open sea on the deck of a ship in conditions of poor visibility is a serious problem even for experienced helicopter pilots.

As part of the Helideck Landing Assistance (HEDELA) project, the German Aerospace Center (Deutsches Zentrum fur Luft- und Raumfahrt; DLR) is working with the Aviation Service of the German Federal Police to develop flight assistance systems that help pilots land safely and accurately on ships . This is reported newsky.com.ua with reference to terradaily.

Augmented reality glasses were used for the first time in flight tests of the research helicopter FHS of the DLR company.

Helicopters play an important role in the rapidly developing offshore industry. Their ability to take off and land vertically, hover in one place and reach high flight speeds means they are well suited for a variety of marine applications. However, problems arise in conditions of poor visibility, often caused by sea fog, precipitation and darkness.

"At sea, pilots lack fixed landmarks for orientation," says Stefan Levedag from the DLR Flight Systems Institute. "All they see around them is open sea, and a moving ship offers relatively little landing space." This is aggravated by wave turbulence on the ship's deck, which can make it even more difficult for the helicopter to land safely.

The visual assistance system helps pilots during approach.

As part of the HEDELA project, DLR developed and tested special displays for landing on the deck of ships.

The goal is to support pilots throughout the flight, including planning the flight path(s) without obstacles for landing, transfer and take-off.

Augmented reality (AR) glasses take the burden off pilots performing these tasks in poor visibility conditions by superimposing landmarks for orientation and other important information directly on their field of vision. In a thick cloud, for example, a floating ship is circled in bright green.

Important information about flight speed, altitude and location, as well as about the artificial horizon, is also visible in the pilot's field of vision. "The superimposed symbols are incredibly useful in helping pilots orient themselves," says project manager Malte-Jorn Maibach.

"The main advantage is that the pilot does not need to constantly switch his gaze between the instruments in the helicopter cabin and the view outside. This allows him not to be distracted from the landing point."

This takes the load off the pilot and increases his situational awareness. At the same time, it also helps to optimize the operational capabilities and safety of marine helicopters.

Current developments in the helicopter cabin are aimed at reducing and combining display devices. One example is the first limit indicator (FLI), which presents the most important data related to the state of the propulsion system (torque, turbine speed, and turbine outlet temperature) with the help of a single control device.

This type of flight assistance system, which was also implemented in the tested head-up display (HMD) described below, can significantly improve the overall flight and handling characteristics of the helicopter.

The researchers developed the display system design during a previous project, Helicopter Flight Safety in Maritime Operations (HELMA), and successfully tested it using a simulator.

DLR first tested the use of commercial augmented reality glasses in flight during tests using its Active Control Technology / Flying Helicopter Simulator (ACT / FHS) modified EC135 research helicopter.

HMDs - helmets with built-in displays worn by pilots - are already widely used in the military sector and have a number of advantages. However, such systems are very expensive.

Therefore, in the framework of their work, researchers rely on commercial systems and develop them taking into account the specific requirements of marine helicopters.

"The advantage is that commercial systems are cheaper to buy, but at the same time very advanced technologies are used," says Maybach.

However, head tracking in AR glasses is a notorious problem, as it is not intended for use in moving systems, such as by the driver of a vehicle.

"The system must be able to distinguish the movement of the pilot's head from the movement of the helicopter, which makes the outside world appear to be moving," says aerospace engineer Christian Valko of the Flight Systems Institute. For this purpose, DLR researchers have developed software that recognizes the position of the pilot's head in relation to the moving helicopter and takes it into account when displaying holographic symbols.

Currently, researchers are analyzing data from flight tests. They also investigate how the AR glasses work in the air in conjunction with a newly developed head tracking algorithm and try to determine the effect of helicopter vibration on the entire system configuration.

New display systems are of considerable interest to helicopter pilots who have to perform very complex flight maneuvers that require them to keep the helicopter under control while maintaining constant awareness of the mechanical flight parameters. Innovative displays allow you to integrate relevant information with standard interfaces and help make work within these limits as intuitive as possible.

The problem of landing on the ship's deck is three questions for the director of the Institute of Flight Systems DLR, Professor Stefan Levedag

Why is the Institute of Flight Systems DLR conducting research on landing helicopters on ship decks?

Landing on the deck of a ship is one of the most difficult maneuvers for helicopter pilots. In addition to the fact that they have to deal with a moving landing platform, they often face strong winds, turbulence caused by the contours of the ship, the lack of landmarks to determine position and speed, as well as poor visibility in the absence of the horizon. Providing pilots with an optimal system for working in such extreme conditions is a serious problem, but DLR researchers use advanced technologies to develop effective solutions.

Such assistance systems are already used in the military sphere. Why are there still no available alternatives for civil aviation? When will such a system become available to civilian operators?

Military systems have very strict additional requirements, such as compatibility with night vision technologies, and therefore they are more expensive. The integration of new functions is more feasible in such an environment than with a commercial product, which requires as low costs as possible.

New systems such as HoloLens must also be certified for future use in airplanes. For example, they must withstand strong vibrations, temperature changes and electromagnetic fields without compromising accuracy. When trying to implement innovative solutions from the field of consumer electronics, serious obstacles often arise, and overcoming them requires time and significant resources.

As a result, it is difficult to predict when such a technology can be successfully transformed into an aviation product. Obstacles are especially high, given the difficult period that the aviation sector is currently experiencing.

What is special about the ACT / FHS research helicopter?

The EC135 ACT / FHS helicopter is a unique research instrument. It has open interfaces that allow you to transfer all important data about the flight status to experimental systems, including new displays or even visualizations such as HoloLens. Due to the unique architecture of the system, these new elements do not require approval, as would be necessary for a commercial application. This even applies to new systems and functions that provide control commands to the helicopter.