Support Us

You are browsing the archive for Releases.

News from the CKAN team 8 Oct 2012

Mark Wainwright - October 9, 2012 in Feature, News, Releases

There have been some changes of plan and a lot of work behind the scenes since the last round-up of CKAN news back in July. Major areas of work include the new Datastore and significant overhauls to CKAN’s web user interface. The release of CKAN 1.8 has been delayed, in order that it can include the new Datastore. Some details are below.

New Datastore

CKAN’s Datastore provides structured storage and querying of data, and underpins CKAN’s previews as well as other custom-made applications to access and process data. In current releases of CKAN, the Datastore is built on elasticsearch, an open-source search engine which was quick to build and is good for providing full-text search of a data resource. However, elasticsearch is limited in what it can do, and we have wanted for a while to replace it with a full database-backed system. We’ve now had a chance to spend some time on this – David has built a new Datastore based on PostgreSQL, and Dominik has been adding improvements and working on the migrations.

The new Datastore will have a full SQL search API, enabling more complicated queries, including queries that join data across different data tables. (We’ve also found some reliability problems with elasticsearch which it will hopefully solve.)

Version 2.0

The new Datastore is one of several major changes in progress to the code base. As a result work has started on a code branch for version 2.0 to reflect the fact that it will be a significant change. The changes include the new Jinja templating system (and a resulting change to all the default templates), which will not cause much visible change for users, but will make life much easier for anyone wanting to run their own customised version of CKAN. Legacy support for the old templates will be included, so that instances with existing customisations can keep them until they switch to the new system.

New UI

CKAN’s default user interface hasn’t changed significantly in 5 years and sadly needed attention. That changed dramatically for the better recently when the team gained a front end developer – first Aron and now JohnM – who have worked with Toby on improving the UI. The results can be seen on the CKAN demo site. They will be integrated into CKAN’s default UI in version 2.0.

Organizations and authorization

CKAN 2.0 will also include a new access authorization system. At its heart will be Organizations, a new feature to enable a smoother workflow for data publishers. An Organization will be a collection of datasets and users, so that a dataset can only be changed by users in the relevant Organization. (For a community instance like the DataHub, all users will belong to a default Organization.)

Where’s 1.8?

The release of CKAN 1.8 has been delayed, in order that it can include the new Datastore. To anyone who’s been waiting eagerly for 1.8, apologies! On reflection, we decided that for existing sites running 1.7.1 to upgrade now, only to have to do another upgrade shortly when the new DataStore is added, would be an unnecessary effort for little gain. The good news is that the Datatore has now been added to the 1.8 branch and will be undergoing testing over the next couple of weeks. An added complication is that sites using the old elasticsearch Datastore will need a way to migrate to the new one.

Comings and goings

Our first front end developer Aron left recently, and John Martin has joined the team to replace him. Dominik, a university student from Potsdam who joined us as an intern for the summer, will be going back to college soon, but we hope to be seeing more of him! We also said farewell recently to Ross, who made a great contribution during his time on the team as a developer, and Toby too is imminently off to pastures new. All the best to them from the CKAN team.

Translators needed!

Sean Hammond - July 31, 2012 in Releases

We’re preparing to release CKAN 1.8 in about two weeks’ time. A few parts of the interface are new, and so some words and sentences need translating from English to a number of other languages before the release.

If you can translate into any of these languages, please help! You can contribute translations, and help keep CKAN open to people across the world, by signing up here.

If there is currently no translation of CKAN into your language, and you’d like to start or contribute to one, you can do that too by signing up. If you have any problems get in touch via the ckan-dev list.

CKAN 1.7.1 released

Adrià Mercader - June 20, 2012 in News, Releases

The CKAN team is pleased to announce the release of version 1.7.1. This is bug fix release, and does not introduce any backwards incompatible change with version 1.7 (It is our intention to do the same with all forthcoming point point or minor releases).

The new version is available on PyPI, the apt repository and of course via source install. See the administration guide for full installation instructions.

For those users that deploy CKAN via package install, we have updated the upgrading instructions to make the process clearer, whether upgrading from another major version or the same major version.

As usual, feel free to email the dev list or pop by on #ckan at irc.freenode.net if you need any help.

A full list of changes is below.

v1.7.1 2012-06-20

Minor:

  • Documentation enhancements regarding install and extensions (#2505)
  • Home page and search results speed improvements (#2402,#2403)
  • I18n: Added Greek translation and updated other ones (#2506)

Bug fixes:

  • UI fixes (#2507)
  • Fixes for i18n login and logout issues (#2497)
  • Date on add/edit resource breaks if offset is specified (#2383)
  • Fix in organizations read page (#2509)
  • Add synchronous_search plugin to deployment.ini template (#2521)
  • Inconsistent language on license dropdown (#2575)
  • Fix bug in translating lists in multilingual plugin
  • Group autocomplete doesn’t work with multiple words (#2373)
  • Other minor fixes

Announcing CKAN 1.7

Mark Wainwright - May 14, 2012 in News, Releases

The results of a lot of hard work from the CKAN team saw the light last week when CKAN version 1.7 was released. We’re excited about some of the changes, and it seems that some other people are too. Some highlights are mentioned below; the complete change list is here.

New ways to handle data

A major change is the improved DataStore, which was covered in this blog post, and related improvements to the data viewer, including a completely new map view:

[IMG: map view]

The crucial innovation making this possible is the data API, which also introduces the possibility of automated tools for handling data, including querying and updating data without having to download and upload potentially large data files.

For data publishers: Organizations and analytics

Another significant area is the new Organization configuration for CKAN groups. When this is enabled, groups include both datasets and users, and can be used to control access permissions in a smoother way, so that, for example, a new user added to a group automatically has permission to edit the group’s datasets. This will significantly ease the workflow of collaborative projects hosted on a CKAN instance, or where a datahub is used by different publishing organisations.

CKAN 1.7 also includes new analytics to help gauge levels of interest and activity by keeping count of views and dowloads.

[IMG: Analytics]

For data users: Change feeds and related items

Atom feeds let a user follow updates on datasets, groups, or other search terms that they are interested in. If you have a browser that handles Atom (such as Firefox) or another feed reader, you can see an example here. The Related Items extension gives a convenient way to link a dataset to other things – for example, a news article about it, an app that uses it, or a neat visualisation of the data.

Trying it out

CKAN 1.7 is live on the DataHub. To download and install your own copy, see the installation guide.

CKAN 1.6 released

Mark Wainwright - March 15, 2012 in Feature, Releases

We’re pleased to announce a major new release (v1.6) of CKAN. Version 1.6 delivers a host of improvements including:

  • Major improvements to datasets, resources, and groups: the dataset view and edit page have been further simplified and improved, resources now have their own dedicated page, and groups are now searchable.
  • User pages have been completely overhauled and now include proper activity streams.
  • The QA (quality assurance) extension, which automatically calculates ‘5 stars of openness‘ scores and keeps track of broken links.
  • A brand new, and much improved, version of the data viewer — which provides live data previews and visualisations — using the Open Knowledge Foundation’s new Recline Data Explorer.

A full list of changes is below. Thanks to the many users whose requests, suggestions, bug reports, and sometimes volunteer coding time have helped the new version along!

To install or upgrade your own CKAN instance, follow the instructions in the Administration Guide.

List of changes in CKAN v1.6

Major changes

  • Resources now have their own pages, as well as showing in the Dataset (#1445, #1449)
  • Group pages enhanced, including in-group search (#1521)
  • User pages enhanced with lists of datasets (#1396) and recent activity (#1515)
  • Dataset view page decluttered (#1450)
  • Tags not restricted to just letters and dashes (#1453)
  • Stats Extension and Storage Extension moved into core CKAN (#1576, #1608)
  • Ability to mounting CKAN at a sub-URL (#1401, #1659)
  • 5 Stars of Openness ratings show by resources, if ckanext-qa is installed (#1583)
  • Recline Data Explorer (for previewing and plotting data) improved and v2 moved into core CKAN (#1602, #1630)

Minor changes

  • ‘About’ page rewritten and easily customisable in the config (#1626)
  • Gravatar picture displayed next to My Account link (#1528)
  • ‘Delete’ button for datasets (#1425)
  • Relationships API more RESTful, validated and documented (#1695)
  • User name displayed when logged in (#1529)
  • Database dumps now exclude deleted packages (#1623)
  • Dataset/Tag name length now limited to 100 characters in API (#1473)
  • ‘Status’ API call now includes installed extensions (#1488)
  • Command-line interface for list/read/deleting datasets (#1499)
  • Slug API calls tidied up and documented (#1500)
  • Users nagged to add email address if missing from their account (#1413)
  • Model and API for Users to become Members of a Group in a certain Capacity (#1531, #1477)
  • Extension interface to adjust search queries, indexing and results (#1547, #1738)
  • API for changing permissions (#1688)

Bug fixes

  • Group deletion didn’t work (#1536)
  • metadata_created used to return an entirely wrong date (#1546)
  • Unicode characters in field-specific API search queries caused exception (since CKAN 1.5) (#1798)
  • Sometimes task_status errors weren’t being recorded (#1483)
  • Registering or Logging in failed silently when already logged in (#1799)
  • Deleted packages were browseable by administrators and appeared in dumps (#1283, #1623)
  • Facicon was a broken link unless corrected in config file (#1627)
  • Dataset search showed last result of each page out of order (#1683)
  • ‘Simple search’ mode showed 0 packages on home page (#1709)
  • Occasionally, ‘My Account’ shows when user is not logged in (#1513)
  • Could not change language when on a tag page that had accented characters or dataset creation page (#1783, #1791)
  • Editing package via API deleted its relationships (#1786)

CKAN 1.5.1

David Read - January 24, 2012 in Feature, Releases

CKAN release 1.5.1 is now available. A major new feature is supporting running background tasks, which we will be using for downloading the data for each catalogued dataset. This will allow us to more easily preview it, search within it, and provide an API too.

Also included is a security fix that affected just the CKAN 1.5 release. All known servers have now been upgraded. The nature of the issue is embargoed for two weeks, to allow any further installations of this version to be upgraded.

There are also UI improvements to the resource and group editing pages, plus CKAN’s language coverage now covers even more languages (thanks to the large international community of CKAN users).

OKF provide both an easy-to-use install package for Ubuntu 10.04 (at apt.ckan.org) as well as general source install (from Github or PyPi). For full install and upgrade instructions see: http://readthedocs.org/docs/ckan/en/ckan-1.5.1/

Here’s the full Change Log of changes since the previous release 1.5 (2011-11-07):

CKAN v1.5.1

Major:

  • Background tasks (#1363, #1371, #1408)
  • Fix for security issue affecting CKAN v1.5 (#1585)

Minor:

  • Language support now excellent for European languages: en de fr it es no sv pl ru pt cs sr ca
  • Web UI improvements:
    • Resource editing refreshed
    • Group editing refreshed
    • Indication that group creation requires logging-in (#1004)
    • Users’ pictures displayed using Gravatar (#1409)
    • ‘Welcome’ banner shown to new users (#1378)
    • Group package list now ordered alphabetically (#1502)
  • Allow managing a dataset’s groups also via package entity API (#1381)
  • Dataset listings in API standardised (#1490)
  • Search ordering by modification and creation date (#191)
  • Account creation disallowed with Open ID (create account in CKAN first) (#1386)
  • User name can be modified (#1386)
  • Email address required for registration (for password reset) (#1319)
  • Atom feeds hidden for now
  • New config options to ease CSS insertion into the template (#1380)
  • Removed ETag browser cache headers (#1422)
  • CKAN version number and admin contact in new ‘status_show’ API (#1087)
  • Upgrade SQLAlchemy to 0.7.3 (#1433)
  • SOLR schema is now versioned (#1498)

Bug fixes:

  • Group ordering on main page was alphabetical but should be by size (since 1.5) (#1487)
  • Package could get added multiple times to same Group, distorting Group size (#1484)
  • Search index corruption when multiple CKAN instances on a server all storing the same object (#1430)
  • Dataset property metadata_created had wrong value (since v1.3) (#1546)
  • Tag browsing showed tags for deleted datasets (#920)
  • User name change field validation error (#1470)
  • You couldn’t edit a user with a unicode email address (#1479)
  • Package search API results missed the extra fields (#1455)
  • OpenID registration disablement explained better (#1532)
  • Data upload (with ckanext-storage) failed if spaces in the filename (#1518)
  • Resource download count fixed (integration with ckanext-googleanalytics) (#1451)

CKAN 1.5 release

David Read - November 9, 2011 in Releases

We’re proud to announce a major new release of CKAN! 1.5 represents a big step forward since our last major release in May. Alongside our usual PyPI release, we are now also releasing regular Ubuntu 10.04 packages, for easy installation.

So now it’s (almost) as simple as “apt-get install ckan” and “ckan-create-instance” Here are the docs: http://docs.ckan.org/docs/ckan/en/ckan-1.5/install-from-package.html

Major changes since 1.4:

  • official government data stores are usually Read-Only to the public, but now we’ve added Moderated Edits, to allow the public to suggest changes and correct metadata. Once suggestions are moderated by the officials, everyone can benefit from the improvements.
  • we got rid of a form framework that was difficult to customise. Now web designers can play about with the form’s HTML/CSS/Javascript to get it exactly how they want. They can do take advantage of autocomplete and other tools in our new powerful Action API. And when the form submits, the data goes through a new schema validation system. We spent a long time planning this, evaluating all sorts of systems and now we’re really pleased with the results.
  • total visual overhaul for viewing and editing datasets
  • vastly improved informatinoa about CKAN and user documentation (ckan.org, wiki.ckan.org & docs.ckan.org)

Plus hundreds of little things like: Captcha to keep spammers at bay, calling them ‘datasets’ rather than pushing our concept of ‘packages’ and fixing the odd bug that crops up too.

We’re also really proud of the recent work in CKAN extensions, such as for handling geographic data, harvesting data from other data stores and providing APIs for data in the catalogue with Web Store.

Thank you to the amazing on-line community for their continued ideas, suggestions and support in the development of this open source data hub software.

Here’s the full changelog, listing all the changes since the most recent minor release (1.4.3.1, 30th September):

v1.5 2011-11-07

Major: * New visual theme (#1108) * Package & Resource edit overhaul (#1294/#1348/#1351/#1368/#1296) * JS and CSS reorganisation (#1282, #1349, #1380) * Apache Solr used for search in core instead of Postgres (#1275, #1361, #1365) * Authorization system now embedded in the logic layer (#1253) * Captcha added for user registration (#1307, #1431) * UI language translations refreshed (#1292, #1350, #1418) * Action API improved with docs now (#1315, #1302, #1371)

Minor: * Cross-Origin Resource Sharing (CORS) support (#1271) * Strings to translate into other languages tidied up (#1249) * Resource format autocomplete (#816) * Database disconnection gives better error message (#1290) * Log-in cookie is preserved between sessions (#78) * Extensions can access formalchemy forms (#1301) * ‘Dataset’ is the new name for ‘Package’ (#1293) * Resource standard fields added: type, format, size (#1324) * Listing users speeded up (#1268) * Basic data preview functionality moved to core from QA extension (#1357) * Admin Extension merged into core CKAN (#1264) * URLs in the Notes field are automatically linked (#1320) * Disallow OpenID for account creation (but can be linked to accounts) (#1386) * Tag name now validated for max length (#1418)

Bug fixes: * Purging of revisions didn’t work (since 1.4.3) (#1258) * Search indexing wasn’t working for SOLR (since 1.4.3) (#1256) * Configuration errors were being ignored (since always) (#1172) * Flash messages were temporarily held-back when using proxy cache (since 1.3.2) (#1321) * On login, user told ‘welcome back’ even if he’s just registered (#1194) * Various minor exceptions cropped up (mostly since 1.4.3) (#1334, #1346) * Extra field couldn’t be set to original value when key deleted (#1356) * JSONP callback parameter didn’t work for the Action API (since 1.4.3) (#1437)

CKAN release 1.4.3

David Read - September 14, 2011 in Releases, Uncategorized

Today we released a new version of CKAN (1.4.3) which covers a good number of bug fixes, speed ups and some new features. The release is on PyPI and source is at BitBucket.

For example, now you can view a dataset as it was in CKAN at a past date or revision using an @ parameter e.g. http://thedatahub.org/dataset/osm@2011-6-1. Also we’ve put in links to these old versions into the revision history view e.g. http://thedatahub.org/dataset/history/osm.

Another cool thing we’ve added is the “Action API” to try out. The existing v1 and v2 APIs are still there for main use, but we’re putting the v3 out in this release as a ‘beta’ for feedback. The key difference is that instead of being RESTful, it’s RPC style. It is more powerful than existing APIs and aims to give better help feedback when things, so good for using it in Javascript etc. The docs are here: Action API Docs and please let us know how you find it so far.

Here’s the changelog about this release:

v1.4.3 2011-09-13

Major:

  • Action API (API v3) (beta version) provides powerful RPC-style API to CKAN data (#1335)
  • Documentation overhaul (#1142, #1192)

Minor:

  • Viewing of a package at a given date (as well as revision) with improved UI (#1236)
  • Extensions can now add functions to the logic layer (#1211)
  • Refactor all remaining database code out of the controllers and into the logic layer (#1229)
  • Any OpenID log-in errors that occur are now displayed (#1228)
  • ‘url’ field added to search index (e9214)
  • Speed up tag reading (98d72)
  • Cope with new WebOb version 1 (#1267)
  • Avoid exceptions caused by bots hitting error page directly (#1176)
  • Too minor to mention: #1234,

Bug fixes:

  • Re-adding tags to a package failed (since 1.4.1 in Web UI, 1.4 in API) (#1239)
  • Modified revisions retrieved over API caused exception (since 1.4.2) (#1310)
  • Whichever language you changed to, it announced “Language set to: English” (since 1.3.1) (#1082)
  • Incompatibilities with Python 2.5 (since 1.3.4.1 and maybe earlier) (#1325)
  • You could create an authorization group without a name, causing exceptions displaying it (#1323)
  • Revision list wasn’t showing deleted packages (b21f4)
  • User editing error conditions handled badly (#1265)

Release 1.4.2

David Read - August 5, 2011 in Releases

Today we released CKAN 1.4.2 to PyPI, which includes support in the model for edit moderation (which is used by this CKAN extension) and a password reset facility. There are also a number of minor features and bugfixes we covered in the five weeks since the previous release.

CKAN v1.4.2 2011-08-05

Major:

  • Packages revisions can be marked as ‘moderated’ (#1141)
  • Password reset facility (#1186/#1198)

Minor:

  • Viewing of a package at any revision (#1236)
  • API POSTs can be of Content-Type “application/json” as alternative to existing “application/x-www-form-urlencoded” (#1206)
  • Caching of static files (#1223)

Bug fixes:

  • When you removed last row of resource table, you could’t add it again – since 1.0 (#1215)
  • Adding a tag to package that had it previously didn’t work – since 1.4.1 in UI and 1.4.0 in API (#1239)
  • Search index was not updated if you added a package to a group – since 1.1 (#1140)
  • Exception if you had any Groups and migrated between CKAN v1.0.2 to v1.2 (migration 29) – since v1.0.2 (#1205)
  • API Package edit requests returned the Package in a different format to usual – since 1.4 (#1214)
  • API error responses were not all JSON format and didn’t have correct Content-Type (#1214)
  • API package delete doesn’t require a Content-Length header (#1214)

CKAN release 1.4.1

David Read - June 27, 2011 in Feature, Releases

Today we have released version 1.4.1 of CKAN, which includes an important refactor to make it simpler to create custom package editing forms – see more about this exciting feature below. There are also a selection of minor improvements, for instance adding ‘nofollow’ to reduce link spam.

Changelog v1.4.1 2011-06-27

Major:

  • Refactor Web interface (including ‘package edit’ form) to use logic layer rather than model objects directly (#1078)

Minor:

  • Links in user-supplied text made less attractive to spammers (nofollow) (#1181)
  • Package change notifications – remove duplicates (#1149)
  • Metadata dump linked to (#1169)
  • Refactor authorization code to be common across Package, Group and Authorization Group (#1074)

Bug fixes:

  • Duplicate authorization roles were difficult to delete (#1083)

New forms

Editing packages is at the heart of CKAN, and a lot of organisations running CKAN have customised the edit form.

In the past this was done all in Python code, reusing or creating new ‘widgets’ based on FormAlchemy, and this generated the form HTML. This worked well, but you needed to know Python, and sometimes it was not clear how to tweak the resulting HTML.

Now with CKAN release 1.4.1 we have a new form infrastructure where the list of fields (and any extra validation required) is defined in a very simple schema, and the form is designed completely in an HTML template. This means that the much larger pool of web designers can tweak the drop-down combos etc. without having to touch any Python.

To give you a flavour of editing the package form, here is the default HTML template (as used on ckan.net): new_package_form.html and here are the instructions for the set-up required: Forms using Templates.

Legacy forms using FormAlchemy are deprecated, but can still be used with the current release, to ease transition. For more information, see the docs on Forms using FormAlchemy.