A near-complete meltdown of the United Kingdom’s air traffic control IT system in August, which stranded around 700,000 passengers and cost airlines as much as £65 million, was made worse when the password details of an on-call engineer who tried to fix the mess while at home failed to work.
An official report into the debacle, which was commissioned by the Civil Aviation Authority (CAA), found that it took an hour and a half after the system had gone into meltdown before the engineer arrived at the offices where the errant computer was located, but even then the engineer didn’t have the technical expertise to fix the problem.
Despite the fact that thousands of flights were being canceled or delayed as a result of the technical hitch, it took at least three hours before the assistance of a more senior IT engineer was called upon, the authors of the official final report discovered.
The senior engineer, however, didn’t know what the fault code being thrown out by the computer meant, so they had to call on the expertise of the company that created the system. To get to this point, four hours had already passed since the UK’s airspace had gone into meltdown.
The CAA estimates that 300,000 passengers were stranded when their flights had to be canceled as a result of the meltdown, while 95,000 passengers were booked on flights that were delayed by four or more hours.
“The incident on 28 August 2023 represented a major failure on the part of the air traffic control system, which caused considerable distress to over 700,000 aviation passengers and resulted in substantial costs to airlines and airports,” commented Jeff Halliwell, Chair of the Independent Review Panel on Thursday.
The August 28 meltdown was traced back to a flight planning system called ‘NATS (En Route) Plc’s Flight Planning System’ or NERL for short. The NERL processes flight plans submitted by airlines, including data such as aircraft type, speed, and routing.
According to air traffic control operator NATS, the system failed when a one in 15 million chance event took place as the computer tried to process a flight plan for a flight from Los Angeles to Paris that passed through British airspace.
The flight plan was compiled, formatted, and submitted in full accordance with international norms, but when NERL attempted to process the plan, it ran into problems as it tried to validate a waypoint marker for the flight to leave UK airspace.
The first two potential waypoints the system identified weren’t in the original flight plan, so they were immediately dismissed, but a third identified waypoint was included in the original flight plan and the system went ahead and tried to validate the waypoint.
It turns out, however, that the waypoint referred to in the original flight plan was for a location in North Dakota and not France. The system had no way of knowing this, however, because both waypoints share the same three-letter abbreviation.
Realizing that the flight would exit British airspace many hours earlier than the flight plan said the plane would enter, the computer generated a critical error that activated a failsafe system.
The failsafe system meant that all flight plans entering or leaving British airspace couldn’t be automatically processed, and air traffic controllers had to manually check every flight plan until the error was cleared.
Automatic processing couldn’t resume until the error was cleared, but with a Bank Holiday in full swing, senior engineers weren’t rostered to work in the office and had to be called from home to help.
The final report on the incident set out 34 recommendations to prevent a repeat incident, although NATS says that it has already addressed a number of findings in the report.
On Thursday, Ryanair welcomed the findings in the final report but called on Transport Minister Louise Haigh to sack the “overpaid and underperforming” CEO of the NATS air traffic control service.
