Data Layer

Once we know why we are sharing data, the next step is to work out what data we need to share in order to address that specific use case. Sometimes we have a tendency to collect as much data as possible, but following the principle of data minimisation we should clearly define what data we need. Our organisation can then identify where to find that data in our internal systems.

Once we have identified the data, we can share it. However this can be complicated because different organisations use different words to describe the same data. One organisation might use the term "last name", while another organisation uses the term "family name"; or different cultures might have different naming conventions which do not fit together comfortably.

In order to make sharing easier, we need to have a "translation layer" for our data. We developed two data standards - one for deduplication and one for referrals - and a guidance note explaining how to use them. While these standards came out of our pilot projects, we believe that they can be useful for other projects in the humanitarian sector. You can download the standards at these links:

Data Standards for Deduplication and Referrals

These Data Standards attempt to harmonise and systematise the data required for collective action in humanitarian consortia. They were developed to support two use cases: Deduplication and Referrals. The Standards are not a policy document. They do not dictate which data humanitarian organisations should collect for their own internal use. The Standards are an operational tool. They are intended to be incorporated into collective data management processes which require interoperability. Download the Data Standards here.

The word “standard” can be used in different ways. In the humanitarian sector it is usually used to refer to a minimum set of criteria that an organisation should meet - one example is the Sphere Standards. However the word “standard” can also be used to refer to a technical specification of how data should be processed in order to ensure interoperability between different systems.

This might sound quite abstract, but these are very practical tools. We incorporated them into our technical platform to create that “translation layer”. This enables participating organisations to create templates for uploading their data to the platform without needing to reformat it - so organisations can continue to use their own terminology and formats, but still share their data.

Standards in any sector are always a work in progress - they need to be revised on a regular basis to take account of changing circumstances. These Data Standards should be understood as part of such a process, and not as a final product. This means that you can update or adapt them for your own work - but please apply them in conjunction with good practice and existing principles around data management.

We encourage anybody in the humanitarian community to adapt, build upon and distribute the standards freely under a Creative Commons Attribution-ShareAlike 4.0 International (https://creativecommons.org/licenses/by-sa/4.0/) - even for commercial use. The only condition is that you give credit to the CCD Network and license any modified material that you produce under identical terms.