One of the great things about working at the Department for Transport (DfT) is that it’s a really outward-facing department. During my 3 years here, I’ve met people from all over the transport sector and this has broadened my horizons to the art of the possible in terms of data science.
This is also reflected in the number of public consultations we run, from topics like a third runway at Heathrow airport to more niche areas like reducing emissions from machinery. We’ve issued 34 consultation documents this year alone making us one of the most prolific departments in government. Go DfT!
So, it’s no surprise that one of the first things we did with Data Science in DfT was to identify ways we could improve this process to make the jobs of our policy colleagues easier and save money for the taxpayer.
But where to begin? Adopting Agile, we started with the user need and spent time with policy colleagues to understand how they operated consultations and the pain points in the process. We quickly came up with two:
- processing a large amount of responses via different media including letters, emails, documents and online survey responses
- analysing and summarising what respondents are saying
Prototyping is easy...
Armed with this information we set about developing a prototype that could extract the text from the various media, and then process this into meaningful analysis that policy colleagues could interpret. Working with a small number of responses, we were able to pull a proof of concept that:
- extracted the text from the documents
- saved it to the cloud database
- cleaned and processed the text
We built a topic model to automatically assign the text into topics and visualised this interactively in a grand total of 2 weeks. Easy peasy! Or so we thought …
But solutions are hard!
Our first attempt at developing a data science application was anything but easy and we quickly found out that scaling from a prototype to a product was a real challenge. There was a myriad of additional issues we had to contend with. Architecture, hosting, deployment, security, error handling and authentication were all challenges we had to solve.
As such, it took us longer than anticipated to develop the first iteration of the application, and it didn’t even work well enough to use. As the saying goes, ‘experience is what you get instead of what you wanted’ – and we learned that lesson the hard way.
Back to the drawing (Kanban) board
Photo by Daria Nepriakhina on Unsplash
Humbled yet resolute, we looked back on development and identified areas where we could improve, both in terms of how we worked and the application’s functionality. We reflected, talked, planned and started work on the second iteration of the application.
Instead of trying to make everything perfect we adopted a ‘good enough’ mentality, but with added controls around checking and quality assurance and a more stringent approach to what features we implemented. This worked a lot better and the second iteration was smoother and quicker than the first.
As a result, we deployed the second iteration of the application to our cycle safety consultation. Did it work perfectly? No. Were there bugs and issues? Yes. Were our policy colleagues happy? Yes. Was it ‘good enough’? Yes. We could process and analyse several thousand responses within 10 hours, giving policy and ministers a high-level overview of the topics, supplemented by collated response documents. Normally, this would have taken weeks of painstaking and repetitive work.
Success?
It’s worth noting that the application isn’t a full substitute for the detailed analysis that policy colleagues conduct, but there is scope to improve this process and make it quicker and easier. Also, the application was applied to 2 other consultations at the same time and it didn’t work as well on these as it could have. If we were involved in the process earlier we could have influenced how data was collected from respondents for a better outcome. One of our next challenges is to work with policy to make sure their business processes work well with our application and vice versa.
It’s also very easy for us to dwell on what could have gone better. Given the number of government consultations, it’s no exaggeration to say that something like this could save the taxpayer millions of pounds through efficiency and more data driven decision making. As such, we’re going to be running the application on more consultations in future. We’ll continue to iterate, learn and improve with the aim of making those savings a reality.
3 comments
Comment by david howard posted on
Hi Tom, it would be great to discuss this and share some work we are doing on consultation process, the use of AI and cognitive services to improve analytics and also digital inclusion. We are doing some work with a couple of agencies in central govt as well as part of DfT.
Comment by Henry posted on
I really enjoyed reading this blog and hearing how technology is saving people time at DfT. Do you have any plans for the future to allow the data visualisation to be available in some form to the general public? Thanks
Comment by Tom Ewing posted on
Hi Henry! We are looking at ways in which we can better communicate the results following the consultation. It's unlikely that we'll be releasing a web application as part of this, however I'm hoping that some of the analysis and visualisation work that we do will start making its way into consultation outcome documents.