Key Takeaways
- The New Zealand government’s Integrated Data Infrastructure (IDI) is undergoing an overhaul, but the business case for the expansion is largely redacted, with over two-thirds of the 95-page document blanked out.
- Officials say transparency is critical to building trust in the government’s data system, but the report’s redactions raise concerns about the lack of transparency.
- The IDI is a crucial tool for the government’s social investment approach, but its current state is "clunky and slow," with limited access and patchy data coverage and quality.
- The overhaul aims to transform the IDI into a more powerful and efficient system, but the scope of the transformation could take it into uncharted territory, requiring significant legislative change and social license.
- The lack of transparency and accountability in the business case has raised concerns among some groups, who question the government’s commitment to building trust and ensuring the responsible use of data.
Introduction to the Issue
The New Zealand government’s Integrated Data Infrastructure (IDI) is a critical tool for informing policy decisions and evaluating the effectiveness of social programs. However, the IDI is currently facing an overhaul, with a business case that is largely redacted, raising concerns about the lack of transparency and accountability. The report, which is 95 pages long, has over two-thirds of its content blanked out, including the options being considered, costs to taxpayers, and cost-benefit analysis. This lack of transparency is particularly concerning given the government’s emphasis on the importance of transparency in building trust in its data system.
The Need for Transparency
The business case report highlights the need for transparency in building trust in the government’s data system. The report notes that "transparency was highlighted as critical to building trust, with calls for clearer communication about data use, access, and safeguards." This emphasis on transparency is consistent with the government’s social investment approach, which relies on the use of data to inform policy decisions and evaluate program effectiveness. However, the redactions in the business case report raise concerns about the government’s commitment to transparency and accountability. The report’s statement that "trust and reciprocity were identified as foundational to any future data system" suggests that the government recognizes the importance of building trust, but the lack of transparency in the business case report undermines this effort.
The Current State of the IDI
The current state of the IDI is "clunky and slow," with limited access and patchy data coverage and quality. The system has 15 billion rows of data, but it can only be refreshed three times a year, and it takes 12-14 weeks to complete each refresh. The system is also labor-intensive, with limited access and patchy data coverage and quality. The business case report notes that "data integration is labor-intensive, access is limited, and data coverage and quality are patchy." This current state of the IDI is a significant barrier to the government’s social investment approach, which relies on the use of data to inform policy decisions and evaluate program effectiveness.
The Overhaul of the IDI
The overhaul of the IDI aims to transform the system into a more powerful and efficient tool for informing policy decisions and evaluating program effectiveness. The scope of the transformation could take it into uncharted territory, requiring significant legislative change and social license. The business case report notes that "any shift in how the IDI is used – for example, towards targeting services to individuals, would require significant legislative change and building strong social license for such a change." This raises concerns about the potential risks and challenges associated with the overhaul, particularly in terms of ensuring the responsible use of data and protecting individual privacy.
The Geographical Barrier to Researchers
The IDI is currently accessible through 40 Data Labs, which are located in various parts of the country. However, the business case report does not address the geographical barrier to researchers, which is a significant issue. The report notes that "real-time and on-demand access to integrated data was seen as critical" but does not provide any information on how this might be achieved. The lack of access to the IDI is a significant barrier to researchers, particularly those in regional areas. A trust in Tai Rāwhiti has told RNZ about the challenges it faced in accessing the IDI, including the need for expert help to draw up its application for research approval and the difficulty in getting hold of the coding and technical know-how to design ways to get the data it was after once it was inside the Lab.
The Delay in the Overhaul
The overhaul of the IDI has been slow-going, with the indicative business case report delivered a year overdue. Stats NZ has also been delayed in developing a multi-year data and statistical program, which was ordered by the government. The agency has also faced problems in setting up a dedicated data support team, which was meant to improve the data flow in the IDI. The delay in the overhaul has raised concerns about the government’s commitment to building trust and ensuring the responsible use of data.
Conclusion
The overhaul of the IDI is a critical issue that requires careful consideration and transparency. The lack of transparency in the business case report raises concerns about the government’s commitment to building trust and ensuring the responsible use of data. The current state of the IDI is "clunky and slow," and the overhaul aims to transform the system into a more powerful and efficient tool for informing policy decisions and evaluating program effectiveness. However, the scope of the transformation could take it into uncharted territory, requiring significant legislative change and social license. The government must prioritize transparency and accountability in the overhaul of the IDI to ensure that the system is used responsibly and that individual privacy is protected.