Federal Government Reportedly Vastly Expands Big Data Spying, Includes Innocent Citizens

After fierce internal controversy, the White House has reportedly authorized a vast expansion of spying capabilities, including the ability to investigate innocent citizens and mine previously separated databases.

“This is a sea change in the way that the government interacts with the general public,” said chief privacy officer of the Department of Homeland Security Mary Ellen Callahan, whose concerns were steamrolled, according to an investigatory report by The Wall Street Journal. One senior official called the expanded powers “breathtaking” in scope.

In part prompted by the frightening near success of the Christmas Day underwear bomber, President Obama demanded more sophisticated resources to prevent future terrorist attacks. “This was not a failure to collect or share intelligence,” said the president’s chief counterterrorism adviser, John Brennan, in January 2010. “It was a failure to connect and integrate and understand the intelligence we had.”

Prior to the updated guidelines, the National Counterterrorism Center (NCTC) maintained the Terrorist Identities Datamart Environment database (TIDE), a digital warehouse of half a million terror suspects and their friends and family. Under new rules, the NCTC now has access to many other government databases so long as it is “reasonably believed” to contain “terrorism information.”

The NCTC can now copy whole datastores on information, such as flight records, the names of Americans hosting foreign exchange students, and many others. The Federal Privacy Act of 1974 sought to stifle indiscriminate sharing of datasets on Americans, but the law contains a skyscraper-size loophole that exempts an agency from the rules if they notify the Federal Register. “All you have to do is publish a notice in the Federal Register and you can do whatever you want,” security consultant Robert Gellman told the Journal.

A supplementary blog post to the report notes a few key differences between an updated 2008 memo from the Bush Administration and the 2012 guidelines:

  • Dropping the requirement to remove innocent U.S. people: In 2008, the NCTC was to remove U.S. individuals “not reasonably believed to be terrorism information.” Now, they can keep tabs on U.S. persons for up to five years.
  • “Pattern-based queries”: Previously, analysts were prohibited from conducting certain sophisticated matching queries that “are not based on known terrorism datapoints,” explains the Journal. Now, its explicitly allowed.
  • Added oversight: 2012 guideliens added “periodic reviews” to review egregious violations and whether keeping some information “remains appropriate.”
  • Sharing information with foreign governments: 2012 added guidelines for data sharing with “any appropriate entity.”

Read the full report here.