Meet LENNON
Latest Earnings Networked Nationally Overnight (LENNON) is an application used by the rail industry. It provides data, such as ticket sales and franchise earnings, which helps them better understand how the rail network operates. The application is hosted externally, but DfT maintains an in-house version containing a subset of the data for our own analysis.
It’s a huge system at more than 100 terabytes. And it’s the most heavily used system by our Rail Technical and Data Management team.
Slow query times on an application like this can be a real nuisance for those relying on frequent access to the data. And this lack of speed means we can’t exploit the data in the system to its full potential.
Google Cloud Platform
So what could we do to speed the system up and allow us to make better use of it?
As part of a wider transformation of the digital technology in use at DfT, we undertook a discovery exercise with Google. This gave us a clear route to shut down our in-house data centres, move our services to Google Cloud Platform (GCP) and transform existing products onto GCP. LENNON was one of these transformation candidates.
After the Google discovery, we set up a Cloud & Data Centre Transformation project. Google, their partner CTS and DfT’s Digital Service, alongside the Rail Technical and Data Management team, were tasked with speeding up slow query times and improving the system’s capabilities. Not an easy task, but certainly possible.
And that’s not all we were hoping to achieve. Backups and maintenance on GCP should be ‘frictionless’. By comparison, the current application requires frequent manual intervention from the colleagues using it. Moving it to GCP will free up time and resources that could be better used elsewhere. The transformation should also increase the security of the application, something never to be overlooked.
This would be our first transformation of an existing application onto GCP, hopefully proving it as a reliable, cost effective and efficient platform for data processing among other things. So how has it gone so far?
LENNON leads the way
Although the project is still in progress, we have already seen some clear and valuable benefits. According to the Rail Technical and Data Management team, processing speeds have fallen dramatically. Where it used to take several hours to execute a query, it is now taking less than 20 seconds.
Not only this but the application can now run multiple queries simultaneously without affecting its performance. These improvements have been a huge help to our colleagues in Rail IT who can now consistently receive the correct data, quicker.
From the practical side of things, there is now also the ability to use DfT’s instance of LENNON for all of our queries. Previously, we had to rely on Rail Delivery Group’s version of LENNON to run some queries. Removing this dependency will save time and effort both for us and RDG.
For DfT’s Digital Service, this project has demonstrated some huge benefits of using GCP. I hope and expect that this experience will enable us to further improve our efficiency through the transformation of similar applications.
In the words of our Chief Architect, Mark Lyons:
"The transformation of LENNON within GCP is a really exciting first step towards moving and transforming our on-premise services into Google Cloud Platform within the next six months. This has been a fantastic example of collaboration between ourselves, Google, their partners CTS and our Rail colleagues, and it has given us a design pattern that can be re-used for other data processing and analysis requirements across the wider department"
Looking forward, we have already started to reuse this approach for other projects in their early stages. It’s exciting to think that data processing as powerful as this should be available for even more projects down the line. Here’s to the cloudy future!
Follow DfT on Twitter and don’t forget to sign up for email alerts