By breathing new life into dormant data, we can see the future

By breathing new life into dormant data, we can see the future

Uber disrupted taxis; Tinder disrupted dating. It’s about time we disrupted food. Because something clearly isn’t working. If a fridge can tell me exactly when a stick of celery is about to reach its expiry date, why is agriculture – something we’ve been doing for 10,000 years – still so imprecise? Why do millions of farmers still look at the sky and pray for rain?

For me the problem isn’t too little rain, or even too much.

The problem is C:/.

By that I mean, the C-drives of the world. All those hard disks; offline repositories of knowledge.

They contain information that we can use now – today – to make farming more precise, to make it less risky. That’s because for decades, researchers have logged data from hundreds of thousands of farms; weather stations have tracked the minutiae of the climate all around the world. Much of that information is stored on C:/

And on C:/ it usually stays.

Researchers often don’t realise the value of sharing it.

They don’t realise it’s part of a greater whole.

They don’t realise we can use it to see the future.

I’m not just waxing poetic; we’ve already done it, with rice farmers in Colombia.

Back in 2013, the country’s rice growers’ federation provided us with datasets stretching back years, all held on C-drives. Then a team of analysts at CIAT screened them for patterns.

What they found was worth a million dollars. Or USD$3.6 million, to be precise.

Analysing the data enabled them to make one striking prediction: the rice was going to fail in the country’s Córdoba region, due to drought. It would spell disaster for producers there.

We ran the analysis again; the same prediction came back.

We called up the the rice growers’ federation. They advised the farmers not to plant.

Sure enough, the rains failed.

Had the farmers planted as usual, their rice wouldn’t have germinated. They would have lost their investments in seeds, fertiliser and labour, worth about USD$3.6 million.

But by breathing new life into dormant data, we were able to see the future, and the farmers were able to take evasive action. Delaying planting was a successful pre-emptive strike against the vagaries of the climate. The following season those same farmers produced good harvests, just as the data predicted.

Seeing the future used to be the domain of mystics and crackpots. But by bringing together multiple sources of data and analysing it, we can create a crystal ball effect. It’s just one example of the hidden power of C:/.

The challenge is moving from the anecdotal to the systematic – something the CGIAR Platform for Big Data in Agriculture aims help with. Through the Platform and its growing number of partners, we want to turn data sharing into a global movement. We want to encourage universities, research organisations, and NGOs everywhere to treat their datasets as global public goods, and share them so that number crunchers can mine them for trends, anomalies and clues about the future.

We can learn a lot from open databases like AgTrials, which pools the results of thousands of crop trials, and standardises the way the data is logged. The system enables researchers to learn from trials that have already taken place in other parts of the world, and avoid repeating experiments – saving them time and money.

We need more of this kind of work to help to free C:/.

Because ultimately, the promise is huge: it could help accelerate the pace of research, drive increases in food production and incomes, and improve the resilience of smallholders to challenges like climate change. It could mean that in the future they won’t look the sky for rain – instead they’ll look to the cloud.


Dr Andy Jarvis is Director of the Decision and Policy Analysis (DAPA) Research Area at the International Center for Tropical Agriculture (CIAT), and a self-confessed big data evangelist.

The CGIAR Platform for Big Data in Agriculture is jointly led by CIAT and the International Food Policy Research Institute (IFPRI). It is supported by CGIAR Fund donors.

CIAT will host the CGIAR Platform for Big Data in Agriculture Convention, at its headquarters in Palmira, Colombia from 19-22 September 2017. You can register here

Join the revolution: #BigDataInAg