The last few years has seen a steady growth in appetite for open data. Much like open source software, open data tends to have a handful of faithful advocates. And despite new datasets becoming available daily, it can sometimes feel like a challenge just breaking into this global, yet relatively niche community.
We’re going to explore some simple ways in which you can get involved in using open data, from developing ideas and apps to contributing back to the knowledgepool.
What is open data?
In short, open data is information which is free to use, share and build upon. Typically, government and big business powers the open data market by providing non-personal data in a variety of formats (such as CSV). This information can then be used by creators in a number of ways, from simply harnessing it for knowledge to performing calculations and manipulations to essentially make something totally new.
Open data is, on the face of it, boring. We think of data as something that machines use, that sits in the background and is uninteresting to humans. But the application of data is what makes it interesting.
Open data must carry a licence which stipulates it is open and free to use. If it is not free to use by anyone for any purpose, it likely isn’t open in the formal sense.
How to find open data
Finding open data is simple with a quick Google search but picking the right data for your project can be a challenge. First, you must decide broadly what the goals of your project are. Then you can ask a few pertinent questions about the open data you seek:
- Roughly what size dataset do I need?
- Does the data need to be realtime? If not, how old can it be and remain useful?
- What formats do I need? Is a CSV or PDF file adequate?
- Do I need to use or build an API?
- Will I be contributing to this dataset?
- Does the data need any additional processing?
It is often helpful to sketch out the requirements of your project to help answer these questions. A good place to start is by creating a basic mind map on a sheet of paper to get a feel for the sort of information you will need (i.e. the column titles) and understand how they may relate to one another and the project as a whole. From this you will be able to work forward and answer most of the above questions.
How to develop open data ideas
One of the challenges with using open data can actually be found at the first hurdle: what to do with it?
Open data is, on the face of it, boring. We think of data as something that machines use, that sits in the background and is uninteresting to humans. But the application of data is what makes it interesting and useful.
A good place to start when coming up with open data ideas is to identify problems. In much the same way people start businesses, first they assess the market to establish a clear need, and then they develop the service or product idea around that need.
Love the open web as much as web do? Subscribe, and let’s be friends!
With open data we have a unique scenario because it is often released by governments in relation to communities and public services. Therefore, it is inherently fit for use by those who want to improve the world around them in some way. You will find lots of open data around transport, environment, education and health. But first, let’s ask a few questions to kick things off:
What common societal problems exist which I am sufficiently interested in tackling?
For example, a lot of data exists on foodbanks across the UK. If your interests pivot around poverty, food and politics for example, this might be an area you could explore. Furthermore, you could work with an organisation such as the ODI to develop data in this area if you wanted to help grow the volume of information available. A large part of your project may be contributing back to the open data community.
What form will this project take, and do I need help?
You might be using open data to create a static infographic or booklet. Or you might be using open data to produce an interactive web or mobile application. By deciding on the form your project will take you can quickly conjure up ideas for presentation and usage, and importantly, understand what help you may need from third parties. As before, sketch on paper your initial vision for presenting some random data, and new ideas will emerge.
What can I learn from existing projects?
It is helpful to see what has already been done. Not so that you can copy or change course, but to learn from real world products and teams and hopefully build on their progress.
Some open data projects are hobbyist or works in progress because often they do not produce profit, and some are funded. Don’t be afraid to contact developers who are using open data to find out more about what they have learnt. Many organisations openly publish their findings anyway, with such examples to be found here, here and here.
Generate as many ideas as possible
Keep in mind that ideas alone are worth nothing, but still track every idea you have. Don’t get too wedded to a single idea. Remember that ideas can appear at any time, so it’s important to be able to note them down wherever you are.
With open data we have a unique scenario because it is often released by governments in relation to communities and public services. Therefore, it is inherently fit for use by those who want to improve the world around them in some way.
You’ll find that most ideas are valueless and can be discarded, but some may form the foundation for a new idea, and then a new idea on top of that. When an idea develops beyond an embryonic stage into something you feel might be workable, think about how it may satisfy your other criteria (does it genuinely interest you? Do you need outside help?).
Making it happen!
Once you have a clear idea of the way in which you wish to pursue an open data project, you’re going to need at least a few materials:
- A brief, to explain the project goals
- A technical document, if this is a software project
- A roadmap, to tentatively commit to future developments
- A team, if you need outside resource
- A marketing plan, so that people can find out about your work
These processes would be found in most creative endeavours. It is important to keep in mind that each step does not need to be exhaustive; you just need to demonstrate to yourself that you have thought about it. As alluded to previously, many open data projects are side projects and therefore, arguably, you do not need to go through the same rigorous planning that a startup business might. Nevertheless, let’s take a look at each in some more detail:
Your brief and any technical documents do not need to be heavy. In fact, if you can explain the concept behind your project in only a few short sentences, this makes it easier for others (and future-you) to understand. Keep it simple, stupid.
Again, your roadmap document doesn’t need to be exhaustive or set in stone. Software projects usually start with an MVP (minimum viable product), so your app might launch with a single feature which you can change and enhance down the line. Be mindful of adding new ideas before you’ve confirmed its core reason for being.
Finding the right people
When collaborating on a new project it is important to work with people with whom you can communicate clearly, and trust. At DiP we put together a job board to help connect like-minded individuals who want to work on projects for the social good, but attending local design and digital events to meet others in your area could be very fruitful.
Planning for success
You need consider ways in which people will hear about your project. Sometimes, a simple blog post may suffice, especially if this is a side project with the central goal of learning. If you are more serious about your work and wish to turn a profit, however, you will need to treat this part of the process like any other business practice. It is beyond the scope of this article to help you write a marketing plan, and a quick Google search will produce plenty of results, but even a vague, bulleted-list may suffice for smaller projects.
Expanding the landscape
Interest is steadily growing in open data, as demonstrated at the beginning of this article. When information is liberated it can be used by developers to create new tools which may not have happened if the data was not readily available, or if there were a profit requirement attached to the project from the outset. This makes open data particularly important when we consider its social value. By making use of open data, contributing to the community and requesting new datasets, you can help expand this landscape and put social value at the heart of what we do.