Brief notes on user-centred evaluation of open data portals

This weekend, an interesting e-mail on the Open Spending mailing list from Angelica Peralta Ramos at the Desarrollando Latin American hack day put out a call for ideas around evaluating government data portals. What sort of factors should feature in evaluating an official government statistics site, or an open data portal?
Evaluating and rating a portal is a different matter from assessing a whole Open Government Data Initiative, but as with OGD Initiatives, it’s hard to reduce any analysis to a single dimension. Here’s my brief notes on some of the considerations that could go into constructing some user-centred metrics to evaluate a portal:
It could be useful to start an assessment from asking ‘Does this provide users with all they need to use the data to do X’ where X could be ‘directly find the fact they wanted’, or ‘visualise the data in their own way’, or ‘support more efficient and effective work in the public sector’ or ‘innovate and build on top of the data’.
Each of those have slightly different criteria for what makes a good data site (or for what is important to them), and a good site should either be clear about who it is serving, or should make a reasonable attempt to serve them all.
For example:
- To directly find the fact you want – a PDF or Excel file may be better (shock horror!) than a CSV or other data dump if it provides a layout that helps a human find facts without having to fire up an analysis tool
- To visualise the data – not only the format of the data matters, but also it’s structure – and that might involve looking inside a sample of datasets. Excel files with headings all over the place and non-standard coding of fields can be just as tricky to deal with as a well-structured table in a word file, for example.
- To support more efficient/effective work with the data – it might be important to have information about it’s provenance alongside it, or to have contact details attached to a dataset so that you can get in touch with the data owner to talk about it more.
- To innovate and build on top of the data – good machine readable data + a clear open licence displayed alongside the datasets is likely to be important. Links to shared source code / spaces to collaborate with other users of the data / detailed documentation and APIs are all useful things too – though very few data portals provide this right now.
Some of this can be seen feeding into the Desarrollando hack at where the top score goes to data accessible in a range of formats, not just in the most machine readable.

Leave a Reply

Open Data in Developing Countries

The focus of my work is currently on the Exploring the Emerging Impacts of Open Data in Developing Countries (ODDC) project with the Web Foundation.

MSc – Open Data & Democracy

RSS Recent Publications

  • An error has occurred, which probably means the feed is down. Try again later.