Unlocking Data Assets: Encouraging Re-use and Access to Survey Microdata

Peter Ellis, Director of the Pacific Community's (SPC) Statistics for Development Division, reflects on the multi-million-dollar assets that are surveys; how to encourage their re-use; and developing questions to deal with today's pressing issues like climate change.

SPC's Statistics for Development Division seeks to support the whole statistical life cycle - from identifying an enduring need that data can support, through design and collection, and ultimately the analysis and dissemination of insight that feeds into decision-making.

A couple of events in the last month made me reflect on different aspects of that cycle. In February, I spoke to researchers at the Australian National University in Canberra. The workshop's topic was applied microeconomic research in the Pacific. I noted that while many of the papers made excellent use of the UNICEF-supported “Multiple Indicator Cluster Survey” (surveys completed in six Pacific countries and planned for four more), none were using the Household Income and Expenditure Surveys that in recent years SPC has been supporting.

The two surveys are complements not competitors, but for some microeconomic research - particularly anything relating to income and to detailed food consumption - the income and expenditure survey would be a more useful tool.

Analysis is cheap, collection is expensive

This sort of research needs to use "microdata", which is the responses from the household and individual level in the original survey (i.e. before being aggregated up to summary statistics). As these surveys typically cost millions of dollars to undertake and there are far fewer of them than most people would like, we really want them to be re-used to the degree possible

The Pacific is rated very low - lower than any region in the world - on international ratings such as Open Data Watch's "Open Data Inventory" and one of the reasons for this is the difficulty in obtaining microdata.

At the workshop I talked specifically about how the Pacific Data Hub Microdata Library, hosted by SPC, is addressing this. The library seeks to make microdata from censuses, income and expenditure surveys and other surveys more available to researchers, in ways that minimise the demands on Pacific Islands national statistical offices.

Unlocking the power of microdata

Unfortunately, access to microdata isn't just a matter of knocking off the name and address columns and handing over the data file. There are complex statistical disclosure control techniques to check that individuals and households can't be identified, international standards for documentation, and processes required to minimally vet researchers and their projects. All of these take resources and recognising this, the aim of the Microdata Library is to provide a pooled regional service to help the Pacific benefit from this in an efficient way.

There have already been some success stories made possible by the microdata library - such as updating the water and sanitation map of South Tarawa (Kiribati), helping identify the vulnerable in Papua New Guinea and Solomon Islands, and analysing gender equality and sustainable energy.

More than 100 requests for data access - of which 54 were approved - were received in 2022 and this is growing steadily. But the potential is still to do more. We want more researcher awareness of these valuable survey microdata assets, and improved efficiency in adding data to the library and approving (or, more rarely, denying) requests for access.

The questions that make the data

A few weeks later I was in New York talking about the other end of the cycle - identification of data needs and design of survey instruments to meet them. I presented on work SPC has been undertaking with World Bank funding to develop a survey module on climate change and natural disasters. Pacific Island national statistical offices are already starting to include climate questions in their surveys; we are aiming to help them do this in an efficient and standardised manner that will allow comparisons over time and across countries.

Building on international experience we have developed 10 questions on whether a household has been impacted by flood, drought or other disaster types; and the impact of these on the dwelling, agricultural assets, health, access to school and other issues. As well as informing local monitoring of climate change loss and damage, we aim to feed this data into global reporting in the internationally-agreed global set of climate change statistics and the Sustainable Development Goals.

We're planning to test these questions further this year and then roll them out as another standardised asset for the region's data collection.