1 Context

The findings from the fourth edition of the Open Data Barometer show that while some governments are advancing towards these aims, open data remains the exception, not the rule.

Why does this matter? Everyone should be able to access and use open data on an open web to allow them to participate fully in civic life. Without good data, it is impossible to hold governments to account for the decisions that they make, the policies they pass, and the money they budget and spend.

In its fourth edition, the Open Data Barometer covers 115 countries and jurisdictions, a 25 percent increase on coverage from the last edition. The leaders for each region in our study are Canada, Israel, Kenya, Korea, Mexico, and the UK. Overall, these findings reveal that the regional champions have been improving steadily since the last edition.

Regional Rank	East Asia & Pacific Global Rank Score (/100)	Europe & Central Asia Global Rank Score (/100)	Latin America & Caribbean Global Rank Score (/100)	Middle East & North Africa Global Rank Score (/100)	North America Global Rank Score (/100)	Sub-Saharan Africa Global Rank Score (/100)
1	Korea 5th 81	UK 1st 100	Mexico 11th 73	Israel 28th 46	Canada 2nd 90	Kenya 35th 40
2	Australia 5th 81	France 3rd 85	Uruguay 17th 61	Tunisia 50th 32	USA 4th 82	South Africa 46th 34
3	New Zealand 7th 79	Netherlands 8th 75	Brazil 18th 59	UAE 60th 26		Mauritius 59th 26
4	Japan 8th 75	Norway 3rd 74	Colombia 24th 52	Kazakhstan 59th 26		Ghana 59th 26
5	Philippines 22nd 55	Spain 11th 73	Chile 26th 47	Qatar 74th 19		Tanzania 67th 22

Table 1: Barometer’s fourth edition regional champions with their respective overall rankings and scores.

[Full rankings are available at our online data explorer]

To deliver real change, open data must meet the principles set out in the Open Data Charter — adopted by more than 15 national and 25 local governments to date.

This report now takes a closer look at our key findings and recommendations, before taking a broader look at themes and trends in the open data space.

2 Findings & Recommendations

Overall, this year’s Barometer shows that governments are slowing and stalling in their commitment to open data. In some cases, progress has even been undone.

However, those countries that have formally adopted the Charter are generally making good progress on fulfilling its principles. Their performance has been improving in recent years, and Charter adopters such as the UK, France, Korea, and Mexico have even become regional open data leaders.

Finding One – Nine out of 10 government datasets are not open

In this edition of the Barometer, we assessed 1,725 datasets from 15 different sectors across 115 countries. Only seven governments include a statement on open data by default in their current policies. Furthermore, we found that only 7% of the data is fully open, only one of every two datasets is machine readable and only one in four datasets has an open licence. While more data has become available in a machine readable format and under an open licence since the first edition of the Barometer, the number of global truly open datasets remains at a standstill.

Table 2: Evolution of key open data indicators throughout the four editions of the Barometer.
(The number of countries covered has increased over time, which may also influence these figures)

If governments added an open licence to existing datasets that already meet all other criteria, the number of truly open datasets available would more than double to 15%. For instance, in Canada, the restrictive licensing of several datasets is one of the primary reasons it has not overtaken the UK’s longstanding leadership position in the ranking.

Finding Two – Government data is typically incomplete and low quality

Government data is usually incomplete, out of date, of low quality, and fragmented. In most cases, open data catalogues or portals are manually fed as the result of informal data management approaches. Procedures, timelines, and responsibilities are frequently unclear among government institutions tasked with this work. This makes the overall open data management and publication approach weak and prone to multiple errors.

• a Although 79 out of the 115 governments surveyed have an open government data portal, often the most complete data is published on a source other than the official open data portal. In such countries, the majority of the most comprehensive datasets (61%) are published by other government agencies.
• b A significant amount of reference data is published by national statistics offices (NSOs) — probably because they have longstanding data management practices that are usually better than those of open data catalogues. Overall, from the 115 governments surveyed, 24% of reference data is published by NSOs. Even in governments that have open data portals, more reference data derives from NSOs (22%) than from open data portals (17%).
• c Data is hard to use because there is no metadata or guidance documentation available. Less than a third (31%) of the published datasets have some supporting basic metadata or companion guidance documentation.

Finding Three – Sustained political will is what makes or breaks the success of open data

Political momentum is key to introducing and scaling up open data. The importance of political decisions is demonstrated by countries such as Ukraine, Argentina, the Philippines, Burkina Faso, and Tanzania — all of which experienced big improvements in Barometer scores and rankings in this edition.

However, political will needs to be translated into strong legal and policy foundations, as in the cases of Canada, Mexico, Japan and Korea — all of which have achieved steady progress in their Barometer rankings. Otherwise, open data initiatives (and the resources needed to advance them) will dry up when the political winds change, as seems to have happened in Costa Rica, Ecuador, and Rwanda. In these latter three countries, positive progress was initially made on open data, but now a lack of further government action is significantly derailing progress. Similarly, the Nordic countries, which were once open data leaders, do not seem to be prioritising open data as highly as before, as evident by their decreasing rankings:

Governments	Ranking ODB 4th ed.	3rd ed.	2nd ed.	1st ed.
Denmark	13th	5th	9th	5th
Finland	20th	11th	12th	14th
Iceland	36th	22nd	27th	13th
Sweden	14th	9th	3rd	3rd

Table 3: Ranking of Nordic Countries – First to Fourth Editions of Barometer.

Open data initiatives can also flounder in cases where the leaders who back them fail to advance wider reforms that encourage a culture of openness, or where political imperatives are not translated into proper data management approaches that ensure the sustainable resources and policies needed for open data to survive political change. This is even an issue in countries which currently rank highly on the Barometer, such as the USA and the UK. The new US administration has already removed certain key datasets from websites, leading to concerns about the future of open government data in the USA. Meanwhile, the UK appears to be softening some of its policy commitments through a new ‘open government data when appropriate’ default policy.

Finding Four – Governments are not publishing the data needed to restore citizens’ trust

Open data portals often do not contain the data people really want and need. Governments must invest in opening up the datasets that people do need (e.g.,data on budget, spending, contracting, and company registers). These datasets still tend to be highly opaque, and often the least open:

Table 4: % of governments publishing fully open accountability related datasets for the different editions of the Barometer.

As findings show in this edition, there are a limited number of governments that have truly open data on these topics and yet these are the datasets that are key to combat corruption and enable government accountability. Governments clearly need to step up their game.

Finding Five – Few open data initiatives actively promote inclusion and equity

As in previous years, our researchers found some evidence that open data is contributing to economic growth and the creation of new businesses, but little or no evidence that it is contributing to social inclusion (whether by enhancing excluded groups’ access to public services or increasing their participation in policy decisions). While it is great news that open data is helping to create jobs and growth, we should not simply assume that a rising tide lifts all boats.

Groups with lower income and/or less political power tend to be excluded from consultation and decision-making processes around open data, frequently lack internet connectivity and the skills to access open data, and may also be less visible in the data in itself. A key example is the ‘sexist data crisis’: women are less likely to be online than men; less likely to be consulted on the design of data policies and initiatives; under-represented among the ranks of data scientists; and often uncounted in official statistics. The table below shows the lack of sex-disaggregated data for a selection of key datasets.

Table 5: Availability of sex-disaggregated data for four different sectors.

3 Open Data Trends

The Web Foundation believes that open data should benefit everyone. Data people want and data people can use should be opened. Our findings, summarised above, show that in most countries this is not happening. In this section, we examine in more detail the trends behind this worrying fact, and explore some counter-examples of good practice.

3. 1. Data for everyone

Ultimately, data is owned by people. Government-held and collected data is funded by taxpayers who in turn have the right to this data. The Web Foundation believes that all people should have a Right to Data in the same way that they should have a Right to Information (RTI). Policies that deliver affordable broadband for all and ensure adequate data protection for citizens’ personal data used are also important to underpin these rights.

The Open Data Barometer findings show that a significant amount of government data is not available to the public yet, and is rarely in an open format. Moreover, even the data which is open is failing to serve the needs of all citizens.

In the 115 countries assessed, the impact of open data continues to be greatest in the area of economic growth and new business creation. There has been little impact on improving marginalised groups’ access to services and participation in decision-making. Given this, the potential of open data to promote equal opportunities for all remains underutilised.

Top 10 Barometer governments	Impact on entrepreneurship (out of 10)	Impact on economy (out of 10)	Impact on inclusion (out of 10)
UK	9	6	1
Canada	5	3	4
France	8	4	3
USA	8	4	2
Korea	7	5	2
Australia	6	5	3
New Zealand	8	3	2
Japan	7	3	2
Netherlands	6	3	0
Norway	7	4	0
Average top 10	7.1	4	1.9

Table 6: Comparison between impact on entrepreneurship, economy, and inclusion for the top 10 Barometer governments.

Data blind spots frequently make the needs and contributions of certain groups less visible to policymakers, so it is critical for data initiatives to devote effort to overcoming such blind spots — often best done by involving marginalised groups at the design stage. This is why on-going dialogue with civil society and citizens is so important — a more diverse array of actors can help spot potential unintended consequences and avoid data-driven discrimination. Recent examples of open data being misused include the use of open court records to blacklist low-income tenants in New York City, and the unforeseen erosion of women’s informal land rights as a result of the expansion of formal land registries.

How can governments ensure they are including everyone when they are designing their data initiatives? Statistics Canada launched the Aboriginal Community Data Initiative to provide these groups with important data for planning and understanding the demographics of their community and the population in surrounding areas. Other examples from Japan and Côte d’Ivoire (discussed in the text boxes) provide further ideas.

3. 2. The data people need

This edition of the Barometer found that governments are not releasing the data needed to restore or build citizens’ trust, such as detailed budgets or company ownership registries.

Data for Government Accountability

Data needed for citizens to hold governments to account is often missing or hard to find. It is crucial for governments to provide information about the use of public resources. This includes data on how taxes are spent, how government contracts are awarded, and how money is funnelled into political campaigns. This also means releasing data vital to fighting corruption, such as data on budget, spending, contracting, land ownership, company registries, legislation, and election results (see methodology):

Datasets	% of open datasets published by all governments	Total # of governments	Governments publishing these datasets as truly open data
Budget	10%	12	Australia Brazil Georgia Germany Jamaica Mexico Netherlands Norway Paraguay Sweden UK Uruguay
Spending	3%	4	Canada Greece UK Uruguay
Contracting	3%	3	Australia France Philippines
Land ownership	1%	1	Canada
Company registries	5%	6	Australia Bulgaria Canada Kazakhstan Latvia Norway
Legislation	3%	3	New Zealand Spain UK
Election results	11%	13	Australia Austria Canada Denmark France Ireland Korea Peru Slovakia Spain Sweden Turkey UK

Table 7: Availability of open datasets for government accountability.

For example, only one in 10 budget datasets are open. Unfortunately, open budget data for the USA was no longer available at the time of producing this report, showing how progress in this area may even be backsliding. Even in the only case where land ownership data is available (Canada), it is only available at the sub-national level.

Data for Social Policies

Open data has the potential to make key public services — such as health, education and environmental management — more effective and inclusive, which in turn helps to fight poverty and reduce inequality. In this section, we evaluate whether government open data is truly used to empower and include all citizens. Open data can improve service delivery directly by giving citizens more tools for choice and accountability — or indirectly, by helping policymakers identify and tackle system-wide social issues.

However, similar to data for accountability, less data is available and open in areas relevant to social policies than for innovation. On average, Barometer findings show declines in the availability of data on key public services. Worryingly, this includes a significant change for the worse in health and education data, for example.

Datasets	% of open datasets published by all governments	Total # of governments	Governments publishing these datasets as truly open data
National statistics	8%	9	Belgium Canada Denmark Finland France Italy Japan Norway UK
Health	7%	8	Denmark Finland France Jamaica Switzerland Tajikistan UK USA
Education	8%	9	Argentina Brazil Denmark France Georgia Jamaica Malaysia UK USA
Environment	6%	7	Bulgaria Denmark France Georgia Russia Sweden UK

Table 8: Availability of open datasets for social policies.

Data for Innovation

Open data has significant potential to foster innovation. It is used in applications by entrepreneurs, and can unlock significant value for enterprises. It can also help to increase the efficiency and productivity of current public services. For example, innovation and significant economic value can be created by using datasets such as map data, public transport timetables, and data on international trade or crime.

With innovation data the most abundant of the three clusters (Innovation, Social Policy, Accountability), we can reasonably assume that governments are prioritising these particular datasets. However, even in countries with strong open government data initiatives, the number of these datasets has declined since the last edition of the Barometer.

Datasets	% of open datasets published by all governments	Total # of governments	Governments publishing these datasets as truly open data
Map data	11%	13	Australia Austria Canada Denmark DR Congo Germany Iceland Japan Netherlands Norway Sweden UK USA
Public transport timetables	8%	9	Argentina Canada France Germany Italy Norway UK USA Uruguay
International trade	10%	11	Australia Austria Canada France Jamaica Netherlands Norway Rwanda Sweden UK US
Crime	8%	9	Canada Denmark Finland France Georgia Germany Moldova Russia UK

Table 9: Availability of open datasets for innovation.

3. 3. Data people can easily use

Open data principles matter for data accessibility and usability. For open government data to be usable and valuable, it must be comprehensive, accurate, and of high quality. Governments should also ensure that they have a response mechanism in place that allows users to provide feedback, and continue to make revisions to ensure data quality is improved as necessary.

Access to data

The current approach centered only on open data portals is not working. Data portals have left behind a ghost town of open data projects. Although the open data community has been discussing this issue for a long time, it has been unable to improve the situation. Many datasets that are the most complete and up-to-date are frequently found on other government ministry or agency websites and not on the official open data portals. This shows poor coordination between different government agencies and central open data catalogues.

We recommend that governments update their data management policies and enable a more automated process for data publishing to “increase user friendliness and limit overheads for stakeholders”. This automated process could then be extended to populate a central portal, if user needs dictate this.

Even though the practical solution for this issue is partially technical, we must keep in mind that this is inherently a political and organisational issue. This is not just about portals. Governments need to take data governance seriously and improve the way they create and use open data across all functions, departments, and agencies. Rather than focusing on creating portals, governments must first focus on the political and organisational reform needed to improve open data’s impact and long-term sustainability.

Data Usability

Too often government data may be available online, but the available data is still of poor quality, making it hard to use. Good quality open data needs to be:

• Available online so as to reach the widest practical range of users and uses. Less than three quarters (71%) of existing data is nowadays available online in any form.
• Machine-readable so that large datasets can be analysed efficiently. More and more data is becoming available in a reusable format — up to 53% in the current edition — but almost half of the data available is still published in non machine-readable formats.
• Available in bulk so that it can be downloaded as one dataset and easily analysed by a machine. Unfortunately, only 24% of the datasets in our study can be easily downloaded in bulk.
• Free of charge so that anyone can access it no matter their budget. 10 percent of the data that is available can not be fully accessed for free. Particularly worrying is the case of land ownership data, only free of charge for about one-third of the data available.
• Open-licensed so that anyone has permission to use and reuse the data. Most data is currently not available under open licences (only 26% is). This is an area where there could easily be a quick win — if open licenses were applied across the board, at least 15% of datasets in the Barometer would be considered fully open (more than double the current measure of 7%).

Table 10: Quality on implementation for data available online of the 15 sectors analysed by the Barometer.
(Green and red cells are highest and lowest values per sector respectively)

4 Conclusions

If the promised benefits of the open data movement are to be realised, the current open data agenda pursued by governments needs to shift its focus back to the basics — and to people. Governments must step up their efforts to ensure that open data is for everyone and that data being released is truly what people need and use.

The findings from this edition of the Barometer show that, unfortunately, this is not happening in the majority of jurisdictions surveyed. Data is simply not open in practice. In too many cases, open data is seen as an extra, rather than a whole-of-government responsibility. Open data initiatives are not outlasting the leaders or administrations that started them, and often remain siloed within just one government department or agency. Backsliding in Barometer scores — even among the top performers — reflects this reality.

All of this results in unsustainable policies which are not adding up to better and more open data that is relevant for people, or which is used to make more equitable and effective policy decisions.

Based on the Barometer findings, the Web Foundation calls for the following actions by governments:

• 1 Recommendation – Government data must be open by default
• 2 Recommendation – Governments must integrate open data across all agencies and departments
• 3 Recommendation – Governments must adopt the Open Data Charter to ensure open data practices are embedded beyond political mandates
• 4 Recommendation – Governments must consult citizens and intermediaries when prioritising which open data to publish first
• 5 Recommendation – Governments must invest in using open data to improve the lives of marginalised groups

The potential benefits of open data to build trust and better, more equitable policies can more likely happen if these recommendations are implemented. Otherwise open data will continue to remain an aspiration rather than a reality, and overall country performance on the Barometer will continue to get a failing grade.