Big data is a big deal, make no mistake about it, but it’s probably not as big a deal as it’s going to be eventually when we really figure out how to make good use of it. For now, we have this muddled middle where we understand the value of the data, but most organizations and governments don’t know how to use that data to its full potential.
In a recent issue of The Economist, two separate articles illustrated the state of big data today. In the first, Out of the box, it talks about the promise of open data, one that to this point remains unfulfilled. The article recognized open data could drive not only greater transparency, but act as a platform for business creation as it has with companies like Zillow based on publicly available real estate data and Garmin based on GPS technology.
According to the article, there are 200,000 datasets from 170 sources posted on Data.gov, the US government’s data sharing website. All of this data has not only made it possible for startups to blossom and flourish, it has also made information available to the public where citizens and watchdog groups can review it and root out corruption. Yet for all the good things listed in the article, the authors admit we are just at the edge of understanding what this data can do, and we need even more publicly available data to help it truly reach its potential.
In the same issue of The Economist, there is another big data article, Does Deutschland do digital?, about the changing nature of the manufacturing sector, and whether Germany can keep up. Trends driven by sensors and the Internet of Things are forcing these dyed-in-the-wool German manufacturers to transform into software and data companies where the data being generated by sensors could be more valuable than the machines themselves.
These two articles, which are separated by just one page in the print issue, in their own way show the confusion we are feeling about big data today. It’s like something right on the edge of our consciousness that we know is important, but we can’t quite put our finger on how to make the best use of it.
Data Doesn’t Live In A Vacuum
Perhaps it’s because we put so much faith into technology to solve our problems. We have been led to believe big data is going to help businesses make smarter and more informed decisions. In healthcare, it will help our doctors and medical professionals make better diagnoses and find the most appropriate treatments. In sports, it will help our favorite teams pick the best players. In government, it will open up information and lead us to the transparency promised land where no corrupt government official can hide. And it will help root out those people who are planning to do us harm.
As we learned in the recent horrific attacks in Paris, sometimes it doesn’t matter how much information we collect. As Natasha Lomas wrote in a recent TechCrunch article following the attacks, maybe because it’s not just about data. Maybe it’s about humans working with that data:
Another hard political truth is that effective counter terrorism policy requires spending money on physical, on-the-ground resources — putting more agents on the ground, within local communities, where they can gain trust and gather intelligence.
I think Lomas is onto something here. We tend to believe that technology is the answer when it’s just a tool in the hands of humans. As anyone who has watched an episode of Law and Order can attest, you can have the DNA evidence at the scene of the crime, but the detectives still have to do the hard work of finding the person connected to that bit of data. The data is useless without the human element to connect the dots.
The same is true for mass government surveillance. You can vacuum up all the data you want, but it’s hard to know what it all means without some good old-fashioned shoe-leather detecting to help figure it out.
This holds whether we are talking about preventing terrorist attacks or finding out how best to compete with the company next door. Data is neutral. It can’t do something for you unless you know how to make the connections. Computers can help you collect and process it faster, but in spite of what we are led to believe, we are still far from the point where the computers can extract meaning for us.
Humans And Machines Working Together
In their book, The Second Machine Age, MIT professors Andrew McAfee and Erik Brynjolfsson tell a story about the IBM computer Deep Blue that beat the best human chess player in the world under tournament conditions in 1997. In 2005 a group of amateur players working in conjunction with three computers was able to beat the machine that beat the world’s best chess players.
This shows that using technology as a tool, humans can enhance our own capabilities and make the best use of computers and the technology we have created. If you doubt this, consider that the United States Department of Defense has concluded pretty much the same thing, that the most effective strategy moving forward is one which teams troops with machines.
While the rate of technological change is increasing ever more rapidly, we still tend to overestimate just how advanced we are. Clearly, we still need humans to help make sense of the data we are collecting.
If we can learn to do that effectively, it could ultimately help us to be smarter, safer and healthier. We just have to remember that it’s a means to an end, not the end itself.
Comment