Biz & IT —

An overview of the NSA’s domestic spying program

A recent report gives a great overview of the NSA's domestic spying program. …

In Wednesday's Wall Street Journal, Siobhan Gorman pulled together the disparate threads of reporting on what's known of the NSA's secret domestic spy program, and combined them with some of her own reporting to confirm, once again, that the NSA's program is another incarnation of the Pentagon's erstwhile Total Information Awareness program. Gorman also describes how Carnivore, the SWIFT database snooping program, and basically every other "Big Brother" database and data snooping program that the executive branch has developed over the past two administrations* feed information into the NSA's TIA-like system, which then looks for suspicious patterns in the data.

Gorman's article provides a great overview of how these programs fit together in the architecture of the modern, post-9/11 surveillance state, and it's required reading because it comes at a critical time in our national debate about privacy and the limits of executive power. However, if you've been following this topic closely then you know that most of the information in the article has been public since 2006.

In this post, I'm going to walk back through some of the previous reporting on the topic, both my own work and that of others, and offer corrections and adjustments where necessary based on the WSJ piece. My hope is that readers and reporters who are so inclined can dig through the details and links and follow up on any leads that others may have missed.

(*Note: Infamous codenames like "Carnivore" and ECHELON first cropped up in Bill Clinton's second term, and I covered them when Ars launched in mid-1998. In terms of the presidential orders he signed and the programs that were inaugurated on his watch, Clinton laid some of the groundwork for the Bush administration's pre- and post-9/11 surveillance-related lawbreaking. Or, perhaps a more accurate metaphor is that he blazed a trail that the Bush gang then paved over and turned into a six-lane highway.)

A look back at the role of the TIA in the NSA's surveillance activities

Back in December of 2005, when the NSA warrantless wiretapping story story broke in the New York Times, I took a close look at what was then known about the program and suggested that the NSA's program probably shared some technological DNA with the short-lived (2002-2003) Total Information Awareness program. In the years since TIA first appeared under the Pentagon's roof, the program has moved from agency to agency in the Executive branch, as Congress catches wind of each new incarnation of it and shuts it down only to see it reemerge again with a different acronym on a different department's budget.

In April of 2006, the MIT Technology Review published a piece by Mark Williams that moved the story forward by fleshing out the relationship between TIA and the NSA's domestic spying program. Williams reported that elements of TIA had indeed been moved from the Department of Defense to the NSA, and he suggested that this technology was almost certainly in use as part of the domestic spying program that the New York Times had uncovered.

One month later, a very important article on the role of "transactional information"—a term that originally referred to the phone company's call logs but has since been stretched to fit a widening array of communication types—appeared in USA Today. The article made clear that this "communication metadata" was the real target of the NSA's vast data collection efforts. Also that May, Wired's Ryan Singel released critical technical documents that had been sealed under court order, showing some of the nuts and bolts of how the NSA snoops Internet traffic on AT&T's backbone.

In terms of my own understanding of the NSA's program, the USA Today article made clear that my initial assessment of the NYT's piece had missed the mark on an important and central point: the new surveillance technology at the heart of the TSA's warrantless wiretapping program was not, as I had conjectured, an automated voice recognition system that sampled calls looking for "hits," and then escalated of-interest calls to higher levels of scrutiny and, ultimately, to a human monitor. This call monitoring is almost surely going on somewhere in the intelligence pipeline, but the core of the NSA's program really is the aggregation and analysis of communications metadata.

Based on the USA Today piece and on a number of other sources, I suggested in "TIA (aka Topsail) unveiled: the real scope of the NSA's domestic spying program" that "the original revelations about the NSA's SIGINT vacuum were just the tip of the iceberg," and that "it appears there's probably more that we've yet to see. Much more."

I then put the pieces together and asked the following rhetorical question: "Now, does anyone seriously think that the NSA is not collecting transactional data (at a minimum) for Web, email, FTP and other IP-based communications, and/or that they're not tying all of this data to individual users?"

Gorman's WSJ piece provides sourced confirmation that the NSA is doing exactly what I and others suspected they were doing, i.e., they're collecting e-mail headers, Web surfing histories, cell phone call logs, and every other trace of the digital and analog connections that we make to the world, and they're synthesizing this into complete informational portraits of individuals.

Network effects

In my "TIA unveiled" piece, I appear to have overstated the scope of the NSA's profiling by suggesting that the agency is building such informational pictures of everyone in the US. But Gorman's article provides an important correction by suggesting that the TIA driftnet works in a much more focused fashion.

According to Gorman, counter-terrorism officials must seed the system with leads—like the name or phone number of an individual with suspected terrorist ties. The system then begins monitoring the aforementioned types of transactional data in order to build an informational profile. The system also works outward through the individual's social network by turning its information vacuum on everyone that that person contacts, and then on their contacts in turn, in an ever-expanding web of surveillance. This way, the system can build of profiles of individuals and groups, and monitor their interactions for suspicious activity.

The fact that the driftnet is seeded by first giving it a single target—a target that is ostensibly drawn from some type of human-generated intelligence—makes it less of a lost cause than a massive, nationwide driftnet would be, but only marginally less so, depending on how far out in the suspect's social network the surveillance extends. As I explained in this article on why the NSA's program is a bad idea from a national security perspective, the main problem with these driftnet or "dragnet" systems is that the rate of false positives is typically so high that they produce an overwhelming flood of bogus leads that tie up law enforcement resources.

Even if the more targeted driftnet approach described by Gorman does result in fewer false positives, the constitutional, privacy, and oversight questions still remain. Let's hope that we, the people, eventually get a shot at answering those questions for ourselves.

Channel Ars Technica