Big data knows where you’ve been and who your friends are. It knows what you like and what makes you angry. It can predict what you’ll buy, where you’ll be the victim of crime and when you’ll have a heart attack. Big data knows you better than you know yourself, or so it claims. But how well do you know big data?

What is data? What makes it big? And is it only size that matters? From science to smart cities, business to politics, self-quantification to the Internet of Things, big data has been described as the fuel of the next industrial revolution, and as a modern oracle. It’s winning elections, revolutionising scientific research, and transforming how businesses interact with their customers. And it’s just getting started. Now is the time to decide how we want to use the power of big data. We already benefit from aggregating small improvements, saving time, money and energy through more efficient use of what we have. But we could be more ambitious, and aim to do more with more instead of the same with less. Big data could think bigger. In other ways, though, it’s already too big for its boots: however big the dataset, however powerful the analysis, big data has blind spots. It may be great for population-wide patterns, but it’s not so good at why an individual person might choose to do one thing or another.

This raises the question: why are we so keen to put our faith in big data? Does that say less about what the technology can really achieve than about our lack of trust in one another, or in ourselves?

Timandra Harkness

journalist, writer & broadcaster; presenter, Futureproofing and other BBC Radio 4 programmes; author, Big Data: does size matter?

Zulfikar Abbany

senior science and technology journalist, Deutsche Welle

Will Moy

director, Full Fact

Dr Alex Powlesland

principal scientist, Immunocore Ltd