Support Us

New Template for CKAN Extensions

Sean Hammond - November 21, 2014 in Extensions

We’ve just merged a new template for CKAN extensions. Whenever you create a new CKAN extension using the paster --plugin=ckan create -t ckanext ... command (as documented in the writing extensions tutorial) it’ll now use the new template, which gives you:

  • PyPI integration – and files are automatically generated for your extension, ready for publishing to PyPI
  • A tests directory including stub tests for you to get started writing tests for your extension
  • Travis CI integration – automatically run your tests in a clean environment each time you push a new commit to GitHub. A .travis.yml file and build and run scripts are automatically generated for your extension, you still need to log in to Travis and click the switch to turn on Travis for your extension though.
  • integration – track the code coverage of your tests. A .coveragerc file is automatically generated for your extension. Again, you still need to login to Coveralls and turn it on.
  • A .gitignore file
  • A LICENSE file (uses the GNU AGPL by default)
  • A reStructuredText README file with a skeleton documentation structure including generated installation and configuration instructions, how to run the tests, etc
  • Travis, Coveralls and README badges! Show the world that you have continuous integration, good test coverage, PyPI downloads, and your extension’s supported Python version, development status and license.

Screenshot from 2014-11-21 16:26:14

For an example of an extension built using this template, look at ckanext-deadoralive.

What we’re trying to do with this new template is:

  1. Save ourselves time, by not having to manually create all of this boilerplate every time we roll a new CKAN extension
  2. Help improve the quality of CKAN extensions by encouraging developers to write good tests and documentation, and to use services PyPI, Travis and Coveralls

More to come. If you have any ideas for things to add to the CKAN extension template, let us know on ckan-dev

Export Datasets from CKAN to Excel

Sean Hammond - November 9, 2014 in Data, Presentations, Tutorial

ckanapi-exporter is a new API script that we’ve developed for exporting dataset metadata from CKAN to Excel-compatible CSV files. Check out the short presentation below, and visit ckanapi-exporter for more details:

CKAN Extension Registry – Share and Find CKAN Extensions

Rufus Pollock - November 4, 2014 in Extensions, News

We are happy to announce the new CKAN Extensions Registry which lists available CKAN Extensions:

CKAN Extensions are a way to extend and alter the functionality of the base CKAN platform using the numerous extension points provided by CKAN. CKAN Extensions provide limitless possibilities from altering the site look and feel to adding site pages, from new validation methods to modifying or adding APIs.

There are currently 100 extensions already listed in the registry based on an initial survey of the extensions available “in the wild” (on github etc), and we will be adding more going forward.

CKAN Extension Registry Front Page

Add Your Extension

Instructions for adding your extension to the registry are here:


All About Extensions

CKAN Extensions are a way to extend the functionality of the base CKAN platform using the numerous extension points provided by CKAN.

Support for creating CKAN Extensions was first introduced in Autumn 2010 and has been extended multiple times ever since. Until now we have collected lists of extensions on the wiki but with the growing number of Extensions it is useful to have a proper registry (an Extension registry was one of the most requested items in the Roadmap consultation).

Examples include:

Next Steps

At present, the Registry is confined to “functional” extensions which add new functionality to CKAN and are not specific to a given site.

We are considering adding a section for theme oriented and site-specific extensions (e.g. support for metadata specific to a given site) since these extensions may be useful as inspiration and instruction to others even if they are not likely to be directly installed.

New Broken Link Checker Plugin for CKAN

Sean Hammond - October 21, 2014 in Extensions, Presentations

deadoralive is a new broken-link checker service that works with CKAN and other sites, and ckanext-deadoralive is a CKAN plugin that you can install to integrate your CKAN site with the link checker. The pair have been developed by Open Knowledge as part of our work on the new version of Öppnadata, the Swedish national open data portal.

This quick presentation gives an overview of the link checker’s features and design:

For more details, see these blog posts:

CKAN patch releases 2.0.5, 2.1.3 and 2.2.1 now available

Adrià Mercader - October 15, 2014 in Releases, Uncategorized

We are happy to announce that new patch releases for CKAN 2.0.x, 2.1.x and 2.2.x are now available to download and install.

These patch releases fix important bugs and security issues, so users are strongly encouraged to upgrade to the latest patch release for the CKAN version they are using.

For a list of the fixes included you can check the CHANGELOG:

Patch release upgrades are very straight-forward and do not contain any backwards incompatible changes or involve any change in the requirements, database or Solr schema. As stated in the release policy, the latest patch release is the only one officially supported.

For details on how to upgrade, see the following links depending on your install method:

If you find any issue, you can let the technical team know in the mailing list or the IRC channel.


CKAN High-Level Roadmap Consultation – Autumn 2014

Rufus Pollock - October 1, 2014 in Association, Roadmap

We are working to create a Roadmap of key technical and community work for the next 6 months and we want to hear from as many as possible of you in the CKAN community to find out what you think is important.

To aid this process we have created a long-list of Roadmap ideas in a Trello board which you can view, comment, vote on and add to – please take a look now and share your thoughts »

Roadmap Consultation Trello Board – vote, comment and discuss »

More Information

With the CKAN Association Steering Group and Technical Team in place, we can now think more about the CKAN “Roadmap”.

The goal here is to agree a short-list of 4-6 high-level technical and community items that CKAN Association will focus on in the next 6-12m.

The plan is as follows:

  • Build a “long-list” of potential roadmap item and review / prioritize with users and the community over the next few weeks
  • Create a final Roadmap selection (short-list) that will be the focus of our near-term work

Led by Ashley Casovan of the Steering Group, the Steering Group and the Technical Team have already prepped long-list of high-level roadmap items which can be found in trello here:

Please jump in and vote and comment.


How does this relate to Github Ideas and Roadmap Tracker

The current ideas and roadmap tracker was used as input for this long-list. The purpose of the trello long-list was:

To have a shorter high-level long-list that focused on key items (the issue tracker now has over 80 items of varying scope and detail) Provide a simple interface in which people (including non-developers) could vote and comment

How will the Roadmap be resourced

The aim of the Roadmap is to produce a list of things that the CKAN Association and general community can specifically focus and execute on. We aim to produce a list that is commensurate with current (and anticipated) resources available to the CKAN Association based on Member commitments.

Where can I find out more about CKAN Roadmaps

See the Roadmap page.

Community Meeting: Friday 26 September 2014

Rufus Pollock - September 25, 2014 in Association, Meetups, News

The next CKAN Association Community meeting is planned for this Friday 26th September at 4pm London / 11am EDT.

One major proposed topic is the Roadmap but please suggest additional topics in the meeting doc or here on list.

To help us manage the meeting and get an idea of numbers if you intend to come please add yourself to the participants list in the meeting doc.


  • When: Friday 26 Sepember 2014 – 4pm London (BST) / 11am US East Coast (EDT) / 5pm European (CET)
  • Where: online (likely via WebEx or similar – link to come)
  • Text chat:
  • Chair: Jeanne Holm (GSA / Data.Gov)
  • Meeting Doc: Google Doc here – please add your name if you plan to come!

Draft Agenda

Announcing the CKAN Association Steering Group

Rufus Pollock - September 8, 2014 in Association, News

We are delighted to announce that we have finalized the initial membership of the CKAN Association Steering Group. At present it consists of 4 organizations and their representatives as follows:

  • Antonio Acuña (*), Head of, UK Cabinet Office
  • Jeanne Holm, Evangelist,, U.S. General Services Administration (
  • Pat McDermott and Ashley Casovan, Open Government Secretariat, Treasury Board of Canada (
  • Rufus Pollock (+), President (Open Knowledge)

(*) indicates Chair of the Steering Group
(+) indicates Secretary of the Steering Group

Steering Board Members will serve an initial term of 2 years and serve “ex officio” representing their organizations. We will continue to review Steering Group Membership and to consider potential new members as we go forward.

More Information About the Steering Group

The Steering Group is a key part of the CKAN Association. It is made up of key stakeholders who have committed to oversee and steer the CKAN Association going forward. The initial selection of the steering committee was coordinated by Open Knowledge.

Full details of the Steering Group can be found on the Steering Group page.

Senior Python web developer wanted for world leading open source data portal

Sean Hammond - July 20, 2014 in Jobs

Working in the fast-growing area of open data, we build open source tools to drive transparency, accountability and re-use. Our flagship product CKAN runs the official national data portals from the UK to Brazil, US to Australia and many others. We also build data tools and OpenSpending browsers.

We’re looking for someone passionate about the technical challenges of building software that is used as the infrastructure for open data around the world, so come join our growing team to shape the future of the open data ecosystem!

Key Skills

  • Python, JavaScript, HTML, CSS
  • Python web frameworks (we use Pylons and Flask)
  • PostgreSQL and SQLAlchemy
  • Self motivated, self-starter, able to manage your own time

Extra bonus points for:

  • Open source projects/contributions
  • Front end skills, particularly in data-vis
  • You’ve written an app using open data before
  • Experience working in a distributed team

How to apply

Email, with the subject line “Python Developer – CKAN/Services”. Please include:

  1. Your CV
  2. A link to your GitHub (or similar) profile
  3. A cover letter

More about the Job

You will be working as part of a small, dynamic team in a modern, open-source development environment. This role is full-time and we are very happy with remote-working.

We generally work remotely (with strong contingents in London and Berlin), using asynchronous communication (email, IRC, GitHub) but with standups, developer meetings and demos most days (Skype, Google Hangout) and real-world gatherings more than twice a year including at the Open Knowledge Festival. We also try and ensure our developers can attend at least one open source conference a year.

At each level of our software stack we use best-in-class open source software, including Python, Nose, Travis CI and Coveralls, Sphinx and Read the Docs, Flask, Jinja2, Solr, PostgreSQL and SQLAlchemy, JavaScript and jQuery, Bootstrap, Git and Github, and Transifex.

We iterate quickly, and publish working, open-source code early and often.

All of our code is on github – it is open to public scrutiny, and we encourage contributions from third-party developers. This means that we have to write exceptionally clear, readable, well-tested code with excellent documentation.

All code contributions, whether from internal or external developers, are made with GitHub pull requests and we do code reviews in the open on GitHub.

We are engaged with a large and active community of users, developers and translators of our open source software, via our mailing lists, GitHub issues and pull requests, public developer meetings, Stack Overflow and Transifex. We support users in getting started with our software, encourage and mentor new developers, and take on feedback and suggestions for the next releases.

About the Open Knowledge Foundation

The Open Knowledge Foundation (OKF) is an internationally recognized non-profit working to open knowledge and see it used to empower and improve the lives of citizens around the world. We build tools, provide advice and develop communities in the area of open knowledge: data, content and information which can be freely shared and used. We believe that by creating an open knowledge commons we can make a significant contribution to improving governance, research and the economy. The last two years have seen rapid growth in our activities, increasing our annual revenue to £2m and our team to over 35 across four continents. We are a virtual organisation with the whole team working remotely, although we have informal clusters in London, Cambridge and Berlin.

The OKF is an international leader in its field and has extensive experience in building open source tools and communities around open material. The Foundation’s software development work includes some of the most innovative and widely acclaimed projects in the area. For example, its CKAN project is the world’s leading open source data portal platform – used by,, the European Commission’s open data portal, and numerous national, regional and local portals from Austria to Brazil. The award winning OpenSpending project enables users to explore over 13 million government spending transactions from around the world. It has an active global network which includes Working Groups and Local Groups in dozens of countries – including groups, ambassadors and partners in 21 of Europe’s 27 Member States.

We’re changing the world by promoting a global shift towards more open ways of working in government, arts, sciences and much more.

The Open Knowledge Data Packager

Sean Hammond - June 9, 2014 in Releases

Today we’re launching the Open Knowledge Data Packager!

Data Packager is a web app for quickly creating and publishing Tabular Data Packages from collections of CSV files on your computer. You can register for a free user account and start creating data packages now, or take a look at a sample data package.

With Data Packager’s simple interface you can create a data package, upload CSV files to it, enter some metadata, and get a web page where users can explore and download your data package. When you login, you’ll be taken to your dashboard, where you’ll see a list of any packages you’ve created so far and Add package button:

Data Packager dashboard screenshot

My Data Packager dashboard

Click the Add package button to create a new data package and you’ll be taken to a form where you can enter the title and other metadata for your package:

Creating a new data package

Creating a new data package

Click on Next: Add CSV files and you’ll be taken to a form where you can upload one or more CSV files to your data package:

Uploading CSV files to a new data package

Uploading CSV files to a new data package

Finally, click on Finish to create your data package. You’ll be taken to your data package’s page:

Browsing your newly created data package

Browsing your newly created data package

You can publish the URL of this page, or share it with anyone who you want to share your data package with.

Why Tabular Data Packages?

Tabular Data Packages (defined by the Tabular Data Package spec) are a simple and easy-to-use data publishing and sharing format for the web. A Tabular Data Package is a collection of CSV files with a datapackage.json file. The datapackage.json file contains metadata about the package (title of the package, description, keywords, license, etc.) and schemas for each of the package’s CSV files.

The format is a good compromise between CSV and Excel, providing the simplicity and ease-of-use of CSV with some of the expressivity of full-blown spreadsheets.

The schemas for the CSV files use the JSON Table Schema format, a simple format for tabular data schemas. It includes metadata for each of the CSV file’s columns (column name, type, description, etc.) and optional primary and foreign keys for the file.

Data Packager Features

After you’ve created your data package and uploaded some CSV files to it, Data Packager has a few nice features for you…

Download data packages

The Download Data Package button on your data package’s page will download a ZIP file including all of your package’s CSV files and the datapackage.json file containing the metadata you entered for your package and files, plus schemas for each of your files:

Screenshot of Download Data Package button

Schema browser

Data Packager automatically generates a JSON Table Schema for each CSV file that you upload. The generated schema includes:

  • Column names for each of the file’s columns (taken from the CSV file’s header row, if it has one)

  • The type of the data in each column (string, number, date…), inferred from the values in the columns

  • Some descriptive statistics calculated for numerical columns (minimum and maximum values, mean, standard deviation…)

  • Temporal extents (earliest and latest dates) for date and time columns

By clicking on one of the CSV files on your data package’s page, you can browse the file’s schema using the schema browser. Each file’s page shows a preview of the CSV file contents, and by clicking on the columns in the preview you can inspect the schema for each column:

The schema browser

The schema browser

Schema editor

By clicking the Edit button on one of your CSV file’s pages, you can edit the file’s JSON Table Schema and add your own custom attributes. Data Packager validates all the changes that you make and gives helpful error messages if you try to save an invalid schema.

The schema editor

The schema editor

Primary and foreign keys

If you add primary and foreign keys to a CSV file’s schema, they’ll also be shown on the file’s page.

Primary and foreign keys

Primary and foreign keys


All of Data Packager’s features can also be used via its JSON API.

Open Source

Data Packager is 100% open source. You can:

Built with CKAN

Data Packager is built using CKAN, the highly-customisable open source data portal platform. All Data Packager features are implemented by a CKAN extension, ckanext-datapackager.