Skip to main content

https://dftdigital.blog.gov.uk/2016/10/01/from-a-to-b-and-back-again-upgrading-a-flagship-open-dataset/

From A to B and back again: Upgrading a flagship open dataset

Posted by: and , Posted on: - Categories: Agile delivery, Data, Open Data

toy bus stop with people hailing a bus

It’s easy to take for granted journey planners and other data-driven services we use every day, but they’re only as reliable as the data behind them. So what do you do when an important database is over 10 years old and has passed through so many hands that no one really knows how it all fits together? In our case, you bring the whole thing in-house, learning and improving it as you go, involving users every step of the way.

The name of the Department for Transport’s NaPTAN (National Public Transport Access Nodes) database may not exactly trip off the tongue, but it contains 400,000+ public transport stops – that’s every bus stop, rail station and passenger terminal in Britain! These records are created, checked and kept up to date by local authorities before being uploaded to a centrally held database, where they are published as open data for anyone to use.

The jewel in the crown

You won’t know just by looking, but NaPTAN is an essential part of services such as Traveline, Citymapper and Google Maps. If you want to offer a nationwide journey planner that includes public transport, then you need a national system for identifying public transport stops. Yet many countries don’t have this and it’s no exaggeration to say that we are the envy of some of our European neighbours. At a recent workshop, NaPTAN was described by a stakeholder as “a flagship dataset, the jewel in the crown of UK open data”. Praise indeed!

However, being over 10 years old NaPTAN was definitely creaking at the seams. So last year it was decided that DfT would use its in-house expertise to develop a new and improved service. This would offer better value for money and help us to develop internal expertise in writing and maintaining databases.

Don’t make assumptions

Using an agile approach to ensure we focused on meeting user needs from the start, we recruited a small group of ‘super users’. They were invaluable in identifying errors and helping us discover what mattered most to NaPTAN users. As the GDS posters say, it’s OK to ask for help, make mistakes and say “I don’t know”.

There was little in the way of documentation, so it sometimes felt like one of those problems where you’re given the answer and have to work out the question. Some fields in NaPTAN are empty whereas others will stop services from working properly if not populated correctly. This meant it was important for us not to make any assumptions – something that is true when designing any digital service.

Fixing unexpected issues

Iced cake saying NaPTAN is live

There were a few fraught moments and false starts, but the service successfully went live on 25 May to much celebration (and a little relief!). We had to fix several unexpected issues quickly in the following days and weeks, but now the service is mostly running smoothly with no major issues. We now have a much improved system that really meets users’ needs and will help them to plan their journeys, even if none of them are even aware it exists!

What we learned

As you might expect, we found out a fair bit along the way about how to redesign a service. Some things (in hindsight) were rather obvious! We learned that:

  • you can’t test every aspect of your service before release, so you need to be able to fix things quickly once it’s live or in beta
  • if you want to fix things quickly and successfully, you need to give your developer precise instructions
  • to understand and prioritise faults, you need to encourage users to tell you exactly what is wrong and how it is affecting them
  • using a shared Google Sheet for a bug list works better than emailing different versions of an Excel document back and forth (one of the biggest improvements to how we worked)
  • putting issues on the bug sheet in order of priority is a big help, especially when there are several things awaiting a fix
  • it’s not for the developer to decide priorities, that’s for the product manager
  • if in doubt about what really needs fixing first, ask your users!

Further information on NaPTAN can be found at gov.uk. The data is available under an Open Government Licence and can be downloaded in XML or CSV format as a complete national dataset or in separate local authority files.

For further details, please email us at naptan.nptg@dft.gsi.gov.uk

Follow DfT on Twitter and don’t forget to sign up for email alerts

Sharing and comments

Share this page

2 comments

  1. Comment by Dan Saunders posted on

    Interesting blog and I completely agree, without NaPTAN we'd not be able to complete the weekly NCSD runs, TNDS or run most of our products, it really is an essential dataset!

  2. Comment by Mark Taylor posted on

    DfT policy advisers needs to address seriously the issue of NaPTAN data maintenance (and for it's sister dataset, NPTG, the National Public Transport Gazetteer).

    NaPTAN is the fundamental building block of consistent, unambiguous and efficient public transport journey planning. Yet, there appears to be a reluctance to ensure the continued maintenance by LTAs of their local data through regulation.

    Please, policy people, think again.