# How a Geo Data Pipeline Bug Disrupted Skyscanner Flight Search

DevFeed: [How a Geo Data Pipeline Bug Disrupted Skyscanner Flight Search](<https://devfeed.tech/articles/what-happened-32392.md>)

Original publisher: [Read original article](<https://medium.com/@SkyscannerEng/what-happened-f7c06f078a64?source=rss-401f3b3c958f------2>)

Author: Skyscanner Engineering

Published: 2023-11-21T11:36:51Z

Content type: article

Language: en

Sources: [Stories by Skyscanner Engineering on Medium](<https://devfeed.tech/sources/stories-by-skyscanner-engineering-on-medium.md>)

Topics: [Graphs](<https://devfeed.tech/topics/graphs.md>), [data](<https://devfeed.tech/topics/data.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [systems](<https://devfeed.tech/topics/systems.md>), [Kubernetes clusters](<https://devfeed.tech/topics/kubernetes-clusters.md>)

Tags: [bug](<https://devfeed.tech/tags/bug.md>), [data-pipeline](<https://devfeed.tech/tags/data-pipeline.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [graph](<https://devfeed.tech/tags/graph.md>), [issue](<https://devfeed.tech/tags/issue.md>), [kubernetes-clusters](<https://devfeed.tech/tags/kubernetes-clusters.md>), [recursion](<https://devfeed.tech/tags/recursion.md>), [systems](<https://devfeed.tech/tags/systems.md>)

## AI overview

A bug in Skyscanner's geo data materialisation process made some locations their own parents, creating loops in the location hierarchy and disrupting flight search. The article explains the geo data model and the incident's technical context.

## Source excerpt

How recursion brought down flight search at Skyscanner On 14th September, 2023 8:05 AM UTC (all timestamps in UTC) there was a critical bug in the output of our geo data pipeline which resulted in a number of geo locations being set as parents of themselves. This caused disruption to our service, and for this we're sorry. It gave us the opportunity to evaluate what went wrong, what we learned, and ways we could prevent a situation like this from happening again. Let's take a deep dive and explain things further. What is Geo Data? Geo data is a key dataset at Skyscanner which is used to provide systems, industry partners and travellers with a complete and accurate representation of the world. In simpler terms, any time you see an Airport, City, Region or Country used in Skyscanner, it's originating from this dataset. The most visible example across our offering can be seen when searching for flights where you will specify an airport, city or country for your journey. We use the geo data to populate origins/destinations and look for flights What was the issue? Skyscanner has been on a journey to upgrade our geo data set. At this time there are two versions, two geo models, running in parallel. Flights generally need to know about airports, cities, countries, contrary to other parts of the business where we need to model more complex relations such as districts, countries, islands, etc. Those relationships form a complex graph where locations are related to each others as parents and children. For this reason we kept our original "heritage" data set, merged it with our canonical dataset, our source of truth. We then basically generate (or reconstruct) our heritage dataset from the canonical data every day at 8am UTC. This generation is referred to as materialisation. The Materialisation Process On 14th September, a bug in the materialisation process updated some locations to be the parent of themselves. For example Scotland is now parent of Scotland. We've created a lo