UK Department for Work and Pensions meets Open Data challenge with STR
The Department for Work and Pensions (DWP) is responsible for welfare, pensions and child maintenance policy. As the UK's biggest public service department it administers the State Pension and a range of working age, disability and ill health benefits to more than 22 million claimants and customers.
Through its many and varied activities, the Department collects and generates vast amounts of data. From the personal records of recipients to reports on employment, benefit and health trends. This rich data provides significant opportunities for analysis in the development of social policy, new apps and services.
DWP Client Statistics Development Team Leader Michael Payne says traditionally the department had been heavily reliant on paper-based systems for the collection and retention of official statistics and other data.
As affordable computing power increased during the 1980s, retention shifted from paper to electronic records. Computerised spreadsheets replaced paper-based versions and information could more readily be drawn from the IT systems used by the department.
The challenge of Open Data
The Client Statistics team created an internal software system, called Tabulation Tool, which allowed the generation of hundreds of thousands of ready-made statistical tables. These tables opened up access to vast amounts of data and allowed users to access reports as required.
Payne says at the same time, it was also planned to open up the government's vast stores of collected data. The government believed such a move would encourage the creation of new ways to analyse and interact with the data which, in turn, could stimulate new business ventures and encourage future economic growth.
"The Open Data push meant we had a clear need to move beyond the Tabulation Tool and adopt a more powerful platform for data collection and analysis," says Payne. "We were also getting a lot of user feedback that our existing tool was fine but people wanted more flexibility. They wanted to be able to drill down into data in different ways rather than simply having pre-defined views."
The search for better tools
With rising user demands and the looming impact of the 2010 Open Data initiative, the DWP Client Statistics team realised it had to find a suitable replacement for the long-serving Tabulation Tool.
"We had more than 100,000 people in the department at this time and so needed something that could scale and be flexible enough to deliver on the requirements of different groups," says Payne. "This included the mass of general users as well as specialists such as the statistics team."
Payne says there was also real demand for local geographic information and the ability to drill down into local government areas.
"This would help groups that were trying to develop policies or programs for smaller, more targeted segments. For example you could target neighbourhoods which contain higher-than-average numbers of lone parents for extra support."
The team undertook significant market research while checking what type of software other statistics agencies had in place. As part of the process, the team also looked at the Australian Bureau of Statistics and how it was able to publish a wide range of reports based on census data.
Specific criteria for the new software included providing greater flexibility for users to define their own statistical tables, allow detailed local area statistics to be obtained, and enable easier re-use of statistics. It was also important that individuals could not be identified, thereby requiring a fine balance between open data and privacy.
The Client Statistics team examined a range of options before selecting STR's SuperWEB2 solution, rebadged internally as Stat-Xplore.
"The product met or exceeded all of our key criteria," says Payne. "The associated costs allowed a robust business case to be assembled and approved, and the built-in statistical disclosure control module met a critical mandatory requirement for us around privacy."
With Stat-Xplore in place in mid-2013, the DWP Client Statistics team noticed a range of immediate benefits. It was now possible to release higher volumes of data - and in much more detail - in response to requests from others in the department. At the same time, all personal details remained completely secure.
"The process for publishing data is now much more efficient," says Payne. "We are able to refer many requests for statistical information directly to Stat-Xplore, rather than having to integrate, compile and present data ourselves. We are also able to release official statistics in a much more timely fashion than was previously possible."
Since the launch of Stat-Xplore in May 2013, more than 2,700 users have registered with the system and have generated more than 40,000 statistical tables. Users have come from a range of places including central and local government, academia, the media, special interest groups and students.
"There is a wide acknowledgement that Stat-Xplore is a very big step forward in making data within DWP more open," says Payne. "Users are very pleased with the additional flexibility, including the ability to drill down to very local geographical areas. A good feedback loop has also been established with users that enables suggestions for improvements."
The statistics team can also now provide members of parliament with more detailed reports to support their motions and respond much more quickly to Freedom of Information (FOI) requests from the general public.
Payne says the team is now publishing statistics in half the time it took before Stat-Xplore, thanks to the power and automation of the new platform.
"We are now used regularly as a case study across the UK government," he says. "We have shown that it is possible to make data available to users while at the same time protecting the privacy of individuals."
Payne says the DWP Client Statistics team will continue to work with STR to improve the user experience and will shortly be updating the software to the latest version.
"We have a long term relationship in place that ensures we will be able to continue to evolve and improve our statistical offerings well into the future," he says.
- Dealing with rapidly rising demand for benefit reports and other key statistics
- Restrictive and inflexible Tabulation Tool limiting depth of data exploration
- Need to prepare for the era of Open Data in government
- STR SuperWEB2 – Stat-Xplore data discovery platform
- Improved flexibility in gathering, analysing and sharing large data sets
- Ability to drill down to local geographic levels
- Release of much higher volumes of data more efficiently – statistics can be published in half the time
- Provision of strict privacy controls to protect individual identities