Last week the Defense Science Board (DSB) released a report calling for reforms in the way the U.S. government monitors the development and proliferation of nuclear weapons programs around the world. The Board found there are “gaps” in the U.S. intelligence community’s “global nuclear monitoring” capabilities. It argued that closing these gaps “should be a national priority.” The report recommended adopting “new tools for monitoring,” including looking at “open and commercial sources” with “big data analysis.”
Searching through vast amounts of openly published information with computers is not a new idea. Some of the U.S. intelligence community’s tried and true sources and methods were depicted and subtly critiqued in the opening scenes of Three Days of the Condor: a 1975 spy thriller. Robert Redford plays a CIA analyst who works for a front called the American Literary Historical Society. As Redford explains later in the film, “We read everything that’s published in the world. I look for leaks. I look for new ideas. We read adventures and novels and journals. Who would invent a job like that?” In a classic case of what is old is new, the DSB, it seems, wants to reinvent it.
In the opening scene of the movie a young Chinese-American analyst reminds her colleagues of the computer’s limitations. Then Redford questions her about the different meanings of the Chinese character 天. She explains it means “heaven.” Redford asks, “That’s it, nothing else?” She responds, “Well it can mean, ‘the best’, ‘tops’ sometimes, why?” A few moments later Redford presses, “Are you sure about this ideogram?” “Look at this face,” she asks, “Could I be wrong about an ideogram?” With a smile and lingering doubt Redford notes, “It’s a great face, but it has never been to China.”
That is a very important point, and one completely lost, it seems, on the Defense Science Board.
In its report, the Board also turned to China for an example of sources and methods. It cited a discredited study of China’s nuclear forces, funded by the U.S. Defense Threat Reduction Agency (DTRA), as a “proof of concept” for the Condor-like, old-as-new reform it suggests in its report. The “crowd-sourced” study of China’s nuclear arsenal was conducted by undergraduate students at Georgetown University, who assembled a large collection of video, images and text related to China’s nuclear arsenal using keyword searches on the Internet. The DSB admitted there are questions about “the veracity” of the DTRA-funded Georgetown study. But it mistakenly sees data collection as the problem. The report criticized the “untrained” students for selecting “a semi-fictionalized Chinese TV series” as one of their intelligence sources, as if that were why the study was discredited.
The Board missed the point. The data this “crowd” of “untrained” students collected was not the problem. The problem was what their teacher, a retired U.S. intelligence professional, tried to do with it.
All data can be interesting and useful, which is why the NSA, Google and Facebook collect so much of it. But data is only useful if you know what to do with it. A work of fiction about China’s nuclear arsenal could provide a treasure trove of information in the hands of an analyst who understands the language, the culture and the content. The “gaps” in the U.S. intelligence community’s ability to carefully monitor China’s nuclear program are not in its capability to collect data, but in its ability to understand what it collects. UCS published more than a few studies documenting U.S. intelligence community failures in translating and interpreting open source Chinese language publications containing information about China’s nuclear weapons and space programs.
In some cases, this problem is partly due to a lack of careful scholarship, which requires an awareness of the limitations of the available knowledge, an understanding of how far one can go in drawing conclusions from it, and a willingness to ask tough questions about those conclusions to see how well they hold up to scrutiny.
But these gaps and failures are also the product of an institutionalized deficiency in the way the U.S. government recruits and trains intelligence analysts. In order to acquire the language and cultural skills needed to perform meaningful analysis and interpretation of Chinese cultural products, from news reports to television shows, the prospective analyst must spend a considerable amount of time in the target culture. That allows one to understand, for example, the nature and credibility of different newspapers, institutions, etc. But the U.S. intelligence community, for understandable reasons, does not want to hire people who spent too much time in China, who know too many Chinese people, or who seem otherwise attached to the place. Such people are considered a security risk, or biased. I referred to this deficiency in an earlier post on ICD 630: a recent directive from President Obama’s Director of National Intelligence aimed at closing some of the same intelligence gaps identified by the DSB.
Rather than “crowd sourcing” data collection to feed NSA-like computer programs, the U.S. intelligence community should look for ways to leverage the considerable language and cultural expertise that resides outside of government to help it monitor open source data it lacks the skills to use properly.
A cheap, quick and effective place to start would be restoring public access to the database of foreign news reports managed by the CIA’s Open Source Center. Restoring public access would allow non-government affiliated researchers with robust language and cultural skills to monitor the database for mistakes in translation. They could also provide feedback to the CIA on the nature and quality of the sources it is selecting for translation. This type of “crowd-sourcing” could improve the quality of the information the CIA makes available to government consumers of this often-used source of open intelligence data. Unfortunately, the Obama administration seems to be moving in the opposite direction, strengthening prohibitions on access to the database, which I could use without restrictions when I was an undergraduate student back in 1975.