Background

Launched in 2013, data.gov.au is the Australian Federal Open Data Portal that provides access to public datasets from government.
Initially, the open data portal was based on WordPress. Eventually, to provide a better experience both to data publishers and end users, it was decided to migrate to a specialised data publishing application. Link Digital was chosen to make this possible via the open source CKAN platform.

Challenge

The migration of data from legacy systems is always challenging. Additionally, best practices surrounding cloud based hosting of CKAN was not widely known at the time.

Solution

Link Digital developed a data migration tool specifically to achieve migration of data from the custom legacy system which did not have any export functionality. In addition, they developed a reference architecture with Amazon Web Services.
With the migration to CKAN, features such as programmatic access to data, links to externally hosted data services (APIs) and basic data visualisation like spreadsheet view and graphing/mapping of datasets were available out of the box.
As the data platform developed, the CKAN system has allowed new functionality to be established, such as data harvesting and spatial data hosting.

Results

Grown from 500 datasets to over 30,000 today, data.gov.au is one of the largest open data catalogues in Australia. The ability of the CKAN system to allow automated publishing has allowed several datasets to be updated on a daily basis.
Following the migration, a number of plugins and enhancements have been developed. Among them, KML JavaScript preview (spatial), AGLS metadata, Google Analytics dashboard, API metrics, statistics and GeoServer integration.
With focus on quality data publishing in the first place, the Department of the Prime Minister and Cabinet can be confident CKAN will better support future upgrades and tools that they will be addressing down the track.
All the extensions established on data.gov.au are available on GitHub.