Join us on 28th September, 9:30–11am (BST/UTC+1), as we host Practical experiences of connecting government data; an online webinar which is part of DataConnect21. Tickets are up on our Eventbrite page so get them while they’re hot!

It’s …

A week of data-themed activities and events for people in the UK public sector. 27 Sept to 1 Oct 2021

… organised by the Data Standards Authority and the Government Data Quality Hub.

In this 90 minute session, hear from teams from the Office for National Statistics; the Environment Agency and Swirrl as they discuss

  • concrete problems ONS and EA are…

This article is by Robin Gower, of Swirrl and Infonomics

csvwr

We’re keen advocates of the CSV on the Web (CSVW) model for tabular data. Previous posts on the blog have explained why and how to publish CSVW data.

We’ve also been thinking about consuming CSVW. We believe the standard can be particularly useful for data analysts. The tabular metadata can help to reduce the amount of manual work needed to parse and prepare data before it can be used in analysis. You can use it to find tables, identify column names and cast values to the correct types.

To this end, we’re delighted to announce a new R package published…


We’re hosting a special Power of Data environmental and geospatial one hour webinar on June 16th. You can now sign up for tickets on our Eventbrite page and here’s a bit about our brilliant speakers and a brief outline of the talks they’ll be giving…

Abigail is Head of Innovation and Skills for the Geospatial Commission, Cabinet Office; a former Chair of the Association for Geographic Information and a fellow of the Royal Geographical Society.

Abigail’s presentation will be on Enabling Innovation to Unlock the Power of Location. In it, she’ll discuss the UK Geospatial strategy and share the approach…


This article is by Robin Gower, of Swirrl and Infonomics

We’ve been working on an R package for downloading and working with linked data: Linked Data Frames.

Image by Robin Gower

This tutorial shows you how you can use this package to download statistics from PublishMyData sites, along with rich descriptions of reference data for things like geographies and time intervals.

The library takes care of making SPARQL requests and interpreting the results, so you can get on with analysis and visualisation.

We’ve not submitted the package to R’s CRAN repository yet, so you’ll need to install the development version using devtools:

install.packages("devtools")
devtools::install_github("Swirrl/linked-data-frames")
library(ldf)

You then need to find a dataset using from…


This article is by Robin Gower, of Infonomics and Swirrl

Our last post explained the rationale behind CSVW. In this post we provide instructions for creating your own .​

We’ll work through an example. ​We’ll annotate a CSV dataset with CSVW metadata to accurately describe its contents.

​Let’s start with something simple.

​Data Mill North publishes a list of Grit Bins on behalf of Leeds City Council.

Data Mill North: Grit bin locations

​The first five lines of the CSV look like this:

Fortunately the page we got the file from tells us that the columns are:

  • location
  • easting
  • northing

We interpret “location” as an identifier for the spot where the bin is placed —…


This article is by Robin Gower, of Infonomics and Swirrl.

The CSV format has proven enormously popular in the open data world. Historically, faced by so much data being published in PDFs, the CSV format was a byword for machine-readability.

​It’s enduring popularity is no doubt helped by the fact that CSV files are near universally accessible. Practically everyone has access to a spreadsheet program — like Microsoft Excel — to read and write CSV files. Unlike Excel’s proprietary XLS format, the humble CSV is also easy to use programmatically — you don’t even really need to use a library.

​CSV does have some problems, however.

​Although the IETF published…


FT1000 logo

What a lovely way to start the week: we’ve been identified by the Financial Times and Statista as one of the 1000 fastest growing companies in Europe! As one of only 46 UK technology companies to make the cut this year, we’re thrilled to be included.

This is just a very quick thank you to all the people we’re working with in government who trust us to publish their data. Making data findable, understandable and useful is what we’re all about.

So thanks to the Scottish Government, the Ministry for Housing Communities and Local Government, the Government Statistical Service, the…


To make linked data publishing work it must be practical. Ongoing management of non-trivial collections of graph data by a team presents significant, real-world challenges around collaboration and change management. In this post I present a novel solution which has been implemented in Swirrl’s PublishMyData platform for the last 5 years . It’s a solution which provides collaborative real-time previewing and editing of any number of concurrent revisions, a publishing workflow, and authoring-by-API.

Even given the best tooling and technology, publishing accurate data is a demanding job. It is neither reasonable, nor practical, to demand authors get complicated data right…


Good design should hide complications. It will look simple and obvious but it takes a lot of work to boil down the hard stuff to get to that point. The COGS project, which we’re working on with ONS, includes a lot of complications. So this post is a short one about

  • The job pains COGS helps with
  • How we hide the complication so the user can use it
  • Why this is useful to government statisticians and analysts
COGS: A cracking idea

Before a data user can even start work, a lot of effort goes into getting data into a shape so that it can…


If you’ve wandered into this blog post expecting Jimmy Carr, then you may be disappointed. BUT if you’re the kind of person who’s interested in all things data, then here’s a quick rundown of stuff that happened with us at Swirrl in 2019 …

COGS (Connected Open Government Statistics) — not to be confused with DOGS (like this) Image Source

No, not clogs — COGS. In the quest for comparable and interoperable data, we’ve been working with the Office for National Statistics on the Connected Open Government Statistics (COGS) project, alongside statisticians at DIT, HMRC, MHCLG, DWP and others. In 2019, the project got a new name, the ONS have been scaling up their team of…

Sarah Roberts

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store