Paul Maritz, Zimbabwean-born alumnus of UKZN and UCT, is a titan of the IT world. As de facto number three at Microsoft behind Bill Gates and Steve Ballmer for 1986 to 2000, “he was responsible for essentially all (Microsoft’s) desktop and server software”, according to his Wikipedia entry. Now he is CEO of Pivotal, a spinoff from his previous company, VMWare, and his head is in the Cloud.
The Cloud is the globe-spanning archipelago of server farms operated by the likes of Amazon, Google, Facebook and, yes,America’s National Security Agency, where Big Data lives and is sifted. Big Data is the exponentially swelling cosmos of digitized information saved by and about the owners of the estimated 10 billion devices of all kinds now connected in one way or another to the Internet.
How big is Big Data? Just storing all those ones and noughts on the public Cloud’s servers, according to Hewlett-Parkard’s ad for its new Moonshot system, uses twice as much electricity as the entire United Kingdom.
Back to Maritz. CRN, a tech-focused online news service, ran an exclusive interview with him in April. Headline: “Paul Maritz’s Plan to take over Big Data.” Money quote: “We think at the end of the day, Big Data is not just about analytics. It is about data-centric applications. It is about driving some experience to a customer and causing them to do things in real time.”
Silicon Valley can degrade a person’s ability to make themselves understood outside the priesthood, but here’s what Maritz was trying to say: Pivotal’s aim is to use Big Bata not merely to help its clients understand what customers want but to go one step further and directly pull their strings.
Here’s how this might work. You walk into a shop. Your phone tells a programme in the Cloud which shop it is. This triggers an algorithm which looks at all the spoor you have left in cyberspace and within milliseconds sends you a message with product, price and sales pitch exquisitely calibrated to result in your leaving money in the store.
As behavioural science grows more sophisticated, Cloud-based databases fatten geometrically with bytes about our habits, circles, tastes, movements and finances, and the capacity to see predictive patterns in the data becomes ever more prodigious, the greater the power at the disposal of every kind of hidden persuader.
Many uses are and will be benign, of course. Through the Square Kilometre Array radio telescope programme, South Africa is involved, with IBM and the Netherlands Institute for Radio Astronomy (ASTRON), in developing the next generation but two or three of Big Data collection and analysis. When operational a decade from now , the SKA’s 3000 plus receivers will gather 14 exabytes (billion gigabtyes) of data every day.
“You can think of Big Data as a parallel universe to the world of people, places, things and their interrelationships,” IBM says in a fact sheet on the SKA. “All of this data creates the potential for people to understand the environment around us with a depth and clarity that was simply not possible before. It’s a new natural resource that’s available to be mined.”
Edward Snowden, the Booz Allen Hamilton IT technician wanted for sharing secrets about the NSA’s data mining , may think himself a martyr to the cause of transparency. Actually, the leaks came as no surprise to readers of James Bamford’s books, The Puzzle Palace, Body of Secrets and the Shadow Factory, or his recent article in Wired magazine describing the NSA’s latest data centre, now nearing completion near in Bluffdale, Utah.
“Stored in near-bottomless databases”, Bamford writes, “ will be all forms of communication, including the complete contents of private emails, cell phone calls, and Google searches, as well as all sorts of personal data trails — parking receipts, travel itineraries, bookstore purchases, and other digital ‘pocket litter.’ “
The one useful thing Snowden may have done is to make people think more carefully about Big Data and its uses, not just by the security services, but by all powerful interests, private sector or government, along with the trade-offs, some more Faustian than others, involved in each instance.
If you live in an advanced economy and are not a complete hermit or deliberately hiding, you shed useable data about yourself around the clock in ways that even George Orwell would have found hard to imagine. What matters is what’s done with the data. For now most Americans seem content to pay for Google’s services with the coin of their privacy. There might be less complaint about the NSA’s intrusiveness if it translated shorter airport security queues.