United States General Accounting Office
GAO
Report to the Chairman, Committee on
Armed Services, U.S. Senate
December 2000
EXPORT CONTROLS
System for Controlling
Exports of High
Performance
Computing Is
Ineffective
GAO-01-10
Contents
Letter
Appendixes
3
Appendix I:
Possible Ways of Safeguarding U.S. National Security
Interests Related to Computer Exports
28
Appendix II: Additional Information on Computer Clustering
Technology and MTOPS
38
Appendix III: Identified Applications of National Security Importance
42
Appendix IV: Comments From the Department of Commerce
53
Abbreviations
DOD
ISTAC
MTOPS
Page 1
Department of Defense
Information Systems Technical Advisory Committee
millions of theoretical operations per second
GAO-01-10 Export Controls
Page 2
GAO-01-10 Export Controls
United States General Accounting Office
Washington, D.C. 20548
Leter
December 18, 2000
The Honorable John Warner
Chairman, Committee on Armed Services
United States Senate
Dear Mr. Chairman:
Because high performance computing is an important enabling technology
for military purposes, the U.S. government controls the export of high
performance computers to sensitive destinations, such as Russia and
China, based on foreign policy and national security concerns. A high
performance computer1 has both civilian and military applications,2
operates at or above a defined performance threshold, and requires an
export license to particular destinations, according to the Commerce
Department, the agency responsible for licensing dual-use items. U.S.
policy with respect to the export of sensitive technology, including
computers, is to seek a balance between the U.S. economic interest in
promoting exports and its national security interests in both maintaining a
military advantage over potential adversaries and denying the spread of
technologies used in developing weapons of mass destruction.
The President has periodically changed the performance thresholds above
which licenses are required based on technological advances. The National
Defense Authorization Act of 1998 requires the President to provide a
report to Congress justifying each change for exports of high performance
computers to certain sensitive countries. It requires the report, at a
minimum, to (1) address the extent to which high performance computers
with capabilities between the established level and the new proposed level
of performance are available from other countries, (2) address all potential
uses of military significance to which high performance computers at the
1
High performance computers are regulated based on their composite theoretical
performance as measured in millions of theoretical operations per second (MTOPS).
2
An application is a program or group of programs designed for end users. Software can be
divided into two general classes: systems software and applications software. Systems
software consists of low-level programs that interact with the computer at a very basic level,
including operating systems, compilers, and utilities for managing computer resources. In
contrast, applications software (also called end-user programs) includes database
programs, word processors, and spreadsheets. Figuratively speaking, applications software
sits on top of systems software because it is unable to run without the operating system and
system utilities.
Page 3
GAO-01-10 Export Controls
new levels could be applied, and (3) assess the impact of such uses on U.S.
national security interests. The President announced in August 2000 that
the control threshold above which computers exported to countries like
Russia and China would need a license would be 28,000 millions of
theoretical operations per second (MTOPS) effective immediately.
Rapid technological advances in computing power have fueled concerns
over the continued effectiveness of current computer export controls. You
raised concerns about whether the computer export licensing process
continues to protect U.S. national security interests. As agreed with your
office, this report discusses (1) the effectiveness of current export controls
in preventing countries of concern from obtaining high performance
computing capabilities that can be used for military applications and
(2) our evaluation of the President’s justifications for changing the
computer control thresholds. You also asked us to identify possible ways of
protecting U.S. national security interests related to exports of high
performance computers. We obtained the views of government and private
experts on this matter, and they are presented in appendix I of this report.
Page 4
GAO-01-10 Export Controls
Results in Brief
The current export control system for high performance computers, which
focuses on controlling individual machines, is ineffective because it cannot
prevent countries of concern from linking or clustering many lower
performance uncontrolled computers to collectively perform at higher
levels than current export controls allow. The Department of Commerce
controls the export of individual high performance computers using
MTOPS as its control measure. It currently requires licenses for export of
computers with performance levels above 28,000 MTOPS for countries of
concern, but lower performance uncontrolled computers have been
clustered together from readily available components to obtain computing
capabilities up to 70,000 MTOPS. However, even if control thresholds were
raised to reflect this performance level, U.S. government computer experts
and computer industry representatives agreed that using MTOPS to
calculate computer performance and set control thresholds is an outdated
and invalid means for determining whether individual high performance
computers should be licensed for export. They said that the measure does
not account for new designs of individual processors or for clustering
computers together to achieve high performance computing levels. A
Commerce Department technical advisory committee3 has examined three
alternatives to the current measure for establishing the licensing threshold,
but none of the alternatives would solve the problems posed by clustering
uncontrolled computer components to achieve higher computing levels. In
addition, the Commerce Department has not assessed the implications of
these technological advances on the effectiveness of the current licensing
process to limit the access of countries of concern to high performance
computing.
3
Technical advisory committees, such as the committee on Information Systems Technical
Advisory Committee, advise the Department of Commerce on the technical parameters for
export controls applicable to dual-use commodities and technology and on the
administration of those controls. The committees comprise representatives from industry
and government who represent diverse points of view on the concerns of the exporting
community. Industry representatives are selected from firms producing a broad range of
goods, technologies, and software presently controlled for national security, foreign policy,
nonproliferation, and short supply reasons; or that are proposed for such controls, and are
balanced to the extent possible among large and small firms.
Page 5
GAO-01-10 Export Controls
The justifications to change computer export controls in February and
August 2000, which the President provided in two reports to the Congress,
are inadequate for several reasons. First, the reports address the
availability of individual computers to other countries, but they do not
address the fact that the performance capabilities of clustered systems that
can be acquired by other countries have exceeded the new control
thresholds. Second, the reports address only selected computer uses of
military significance to which high performance computers could be
applied instead of all such uses at the new control thresholds, as required
by the 1998 National Defense Authorization Act. Finally, the President’s
reports make no reference to how specific identified uses at the new
control threshold would affect U.S. national security. Instead, the
President’s reports state that (1) pursuit of particular national security
applications would require more than computer hardware; (2) there is little
danger of most countries usefully employing high performance computers
for military uses; and (3) it is important for the United States to retain a
technological advantage in the design, development, and production of
microprocessors and computers. The reports’ sections dealing with
military uses of high performance computers and how such uses could
effect U.S. national security were based on information from a 1998 study4
used by the Defense Department about military computer uses. The study’s
authors concluded that neither they, nor the U.S. government, had
sufficient information to assess which countries could productively use
high performance computers to pursue particular applications.
Appendix I discusses the three alternative measures of individual computer
performance that the Department of Commerce’s Technical Advisory
Committee has examined. The appendix also discusses nine other ideas
that have been raised at different times by various computer experts as
possible alternatives to either the current MTOPS measure or the overall
control system. However, neither Commerce nor others have fully assessed
the feasibility of these ideas. We did not assess the feasibility of
implementing any of the ideas but describe their characteristics and
limitations when identified by the experts that we consulted. As
technological advances in high performance computing make it more
difficult to maintain the U.S. lead in military capabilities by denying
4
Seymour E. Goodman, Peter Wolcott, and Patrick Homer, High-performance Computing,
National Security Applications, and Export Control Policy at the Close of the 20th Century
(Palo Alto, CA: Stanford University, Center for International Security and Arms Control,
1998).
Page 6
GAO-01-10 Export Controls
advanced technology transfers to countries of concern, it may become
necessary to explore other options to maintain the U.S. lead in military
technology. As a step in this direction, the National Defense Authorization
Act for fiscal year 2000 requires (1) an annual report on transfers of
militarily sensitive technology to countries and entities of concern and
(2) an assessment by the Secretary of Defense on the cumulative impact of
U.S.-granted licenses for exports of technologies and technical
information. The report prepared under the act must include information
on countermeasures that may be necessary to overcome the use of such
technologies and technical information. It is not required, however, to
assess the cumulative impact of exports of nonlicensed computers, such as
those that can be clustered. The report is scheduled for completion in late
2000.
We are recommending that the Secretary of Commerce, in consultation
with the Secretaries of Defense, Energy, and State, convene a panel of
experts to conduct a comprehensive assessment and report to the
Congress on ways of addressing the shortcomings of computer export
controls, including, but not limited to, the ideas noted in this report. We are
also recommending that the Secretary of Defense determine what U.S.
countermeasures are necessary, if any, to respond to enhancements of the
military or proliferation capabilities of countries of concern derived from
both licensed and nonlicensed high performance computing.
Agencies commenting on a draft of this report generally agreed with our
findings and conclusions, and the Department of Energy agreed with our
recommendation to convene a panel of experts to conduct a
comprehensive assessment and report to the Congress on ways of
addressing the shortcomings of computer export controls. The
Departments of Commerce and Defense disagreed with the need to
implement our recommendations because they said they are already
engaged in interagency reviews of similar issues. When asked for
documentation on how interagency mechanisms are pursuing the points
covered in our recommendations, the agencies provided none. As a result,
we believe our recommendations are still valid and necessary to protect
U.S. national security interests. Since Commerce and Defense disagreed
with our recommendations, we have added two Matters for Congressional
Consideration. The Congress may wish to institute a requirement that the
Secretary of Commerce, in consultation with the Secretaries of Defense,
Energy, and State, convene a panel of experts to conduct a comprehensive
assessment of and report to the Congress on possible ways of addressing
the shortcomings associated with controlling individual high performance
Page 7
GAO-01-10 Export Controls
computers, including, but not limited to, the ideas noted in this report. The
Congress may also wish to consider instituting a requirement that the
Secretary of Defense determine and report on what U.S. countermeasures
are necessary, if any, to respond to computing-related enhancements of the
military or proliferation capabilities of countries of concern, as we have
recommended in this report.
Background
The use of high performance computing technology is reducing the costs
and time required for systems analysis, design, development, test, and
deployment of military systems and is improving the integration and
effectiveness of complex weapons systems, according to the Department
of Defense (DOD). As DOD continues to reform and reengineer its
acquisition processes, high performance computing assets are being used
to reduce the number and cost of building expensive prototypes. High
performance computing, a critical enabling technology, is now a key
ingredient to the successful implementation of many major DOD
acquisition programs, according to DOD. It is for these reasons that the
United States attempts to limit the extent of high performance computing
capabilities obtainable by countries of concern.
High performance computers and related components (for example,
processors) are controlled under the Export Administration Act and the
implementing Export Administration Regulations.5 The act authorizes the
Commerce Department to require firms to obtain licenses for the export of
sensitive items that may pose a national security or foreign policy concern.
The Departments of State, Energy, and Defense assist Commerce, which
administers the act, by reviewing export applications and supporting
Commerce in its reviews of export control policy.
Since 1993, the President has revised U.S. export control requirements for
high performance computers five times, including the revisions announced
in August 2000. The export control policy revision implemented in January
1996 removed license requirements for most exports of computers that
could perform at or below a level of 2,000 MTOPS—an increase from 1,500
MTOPS.6 The 1996 policy revision also organized countries into four
5
50 U.S.C. App. section 2401 and following and 15 C.F.R. section 730 and following.
6
Export Controls: Information on the Decision to Revise High Performance Computer
Controls (GAO/NSIAD-98-196, Sept. 16, 1998).
Page 8
GAO-01-10 Export Controls
computer “tiers,” with each tier after tier 1 representing a successively
higher level of concern related to U.S. national security interests.
• Tier 1. The policy placed no license requirements on tier 1 countries,
primarily those in Western Europe and Japan.
• Tier 2. Exports of computers above 10,000 MTOPS to tier 2 countries in
Asia, Africa, Latin America, and Central and Eastern Europe continued
to require licenses.
• Tier 3. A dual-control system was established for tier 3 countries.
• Tier 4. Exports of high performance computers to tier 4 countries were
essentially prohibited.
In 1996 there were 50 tier 3 countries, including China, Russia, India, and
Israel. Since then, the number of tier 3 countries has increased to 52. In
January 1996, the threshold for exporting high performance computers
without a license to tier 3 countries was set at 7,000 MTOPS for civilian end
users, while exports of computers for potential military end users at and
above 2,000 MTOPS required a license. Exports of high performance
computers with performance capabilities above 7,000 MTOPS to civilian
end users in all tier 3 countries required a license. High performance
computer exports to countries in tier 4 (for example, Iran, Iraq, and Libya)
were essentially prohibited because of national security and foreign policy
concerns about these countries.
The Fiscal Year 1998 National Defense Authorization Act7 requires the
President to report to the Congress justifications for changes in controls
for computer exports to certain sensitive countries. The act further
requires exporters to notify the Commerce Department of any proposed
high performance computer exports to countries that pose a concern for
military or proliferation reasons to determine if these exports need a
license. 8 The act also provides that if any designated agency9 raises a
written objection to the proposed export within 10 days after Commerce
receives the notification of the export, the export will require a license.
7
P.L. 105-85 sec. 1211, Nov. 1997.
8
Advance notification to Commerce of exporter’s intent to export high performance
computers will rise to 28,000 MTOPS, effective in February 2001, from the current threshold
of 12,500 MTOPS.
9
Designated agencies for this purpose are the Departments of Defense, Energy, and State.
Page 9
GAO-01-10 Export Controls
On July 27, 1999, February 15, 2000, and August 30, 2000,10 following
announced changes to the export control levels for high performance
computers, the President submitted a report to the Congress, as required.
According to the President’s announcements, these changes were needed
because of the extraordinarily rapid rate of technological change in the
computer industry.
In the August 2000 announcement, the President described plans to change
the controls on the exports of high performance computers by increasing
the computing performance level at which export licenses would be
required.11 These changes were as follows:
• Tier 1. No changes.
• Tier 2. The licensing level was raised to 45,000 MTOPS, effective
immediately.
• Tier 3. The two-level system for tier 3 countries was eliminated, and the
licensing level for the tier was raised to 28,000 MTOPS, effective
immediately.
• Tier 4. No changes.
Export Control System
for High Performance
Computers Is Not
Effective
The current system of controlling the export of individual machines is
ineffective in limiting countries of concern from obtaining high
performance computing capabilities for military applications. In addition,
information obtained from the Departments of Commerce, Defense, and
Energy and computer industry representatives shows that, given advances
in technology, using MTOPS to establish export control thresholds is
outdated and no longer a valid means for controlling computing
capabilities and that an alternative is needed to replace it. MTOPS is an
outdated measure because of its limited ability to estimate the performance
of new designs of processors and to account for clustering of individual
computers to achieve high performance computing capability.
10
Summary of Findings With Respect to Criteria Set Forth in Subsection 1211 (d) of the
National Defense Authorization Act for Fiscal Year 1998, attachment to letter sent to
Congress (Washington, D.C.: The White House, July 27, 1999, Feb. 15, 2000, and Aug. 30,
2000).
11
Executive branch agencies involved with reviewing and recommending changes to
computer export controls are the Departments of Commerce, Defense, State, and Energy
and the National Security Council.
Page 10
GAO-01-10 Export Controls
Computer Advances in
Clustered Systems
Compromise the
Effectiveness of Current
Controls
Over the past several years, technological advances in the ability to cluster
lower performance computers have resulted in an increasing capability
worldwide to achieve high performance computer levels. As a result,
current U.S. export controls cannot prevent countries of concern from
obtaining computing capabilities that can be used for military applications
at levels much higher than those at which computers are currently
controlled by the U.S. government.
Commerce’s Information Systems Technology Advisory Committee, which
advises and assists the Secretary of Commerce, reported in May 1999—and
the Lawrence Livermore National Laboratory confirmed in February
2000—that clustered computer systems composed of readily available
components have been built with a performance level up to about 70,000
MTOPS. As a result, the known performance levels of clustered systems
exceed even the latest export control thresholds set by the U.S.
government in August 2000. Figures 1 and 2 show examples of clustered
computer systems composed of readily available components, such as
personal computers and monitors, located at Clemson University and the
Nehru Centre for Advanced Scientific Research, respectively.
Page 11
GAO-01-10 Export Controls
Figure 1: A Clustered Computer System
Source: Grendel Beowulf Workstation, Clemson University.
Page 12
GAO-01-10 Export Controls
Figure 2: A Clustered Computer System in a Tier 3 Country
Source: Jawaharlal Nehru Centre for Advanced Scientific Research, India.
According to a February 2000 analysis prepared by the Lawrence
Livermore National Laboratory, “Ignoring the fact that low-performance
computers can be clustered to achieve a higher level of processing leads to
a false sense of security.” The analysis said that it is inconsistent to control
the export of individual high performance computers when the
performance capabilities of clustered computers exceed the current and
proposed control levels.12 Officials of the Los Alamos National Laboratory
said that expertise necessary to build clustered systems is available
worldwide, including in countries such as China and India, and that
laboratory officials communicate with people building these systems and
solving the same types of technical computing problems as the national
laboratories. While discussing the relatively lower level of expertise
necessary to put a clustered computer system together, the officials
stressed that it is more difficult to operate clustered systems than to build
12
The National Security Implications of Decontrolling Export to Tier III Countries of High
Performance Computers between 2,000 and 40,000 MTOPS. (February 2000)
Page 13
GAO-01-10 Export Controls
them. For example, support for computer functions requires having an
efficient schedule for running hundreds of separate problems and the input
and output of data among a computer, its local disks, networks, and
archival storage. Without vendor-supplied software to automate these
functions on a system clustered together from readily available
components, everything must be done manually, making computing a
labor-intensive operation.
One computer facility in India advertises that it has developed a new
computer system architecture with clusters of workstations with
commercially available interconnect technologies. The facility is on the
Commerce Department’s “Entities List,” which identifies foreign end users
that have been determined to present an unacceptable risk of diversion to
developing weapons of mass destruction or the missiles used to deliver
those weapons. Publishing this list puts exporters on notice that any
products sold to these end users may present concerns and will require a
license from Commerce.
MTOPS Is No Longer
Adequate as a Measure of
Performance
Analyses and information from U.S. government officials13 and computer
industry representatives show that MTOPS is no longer a valid measure of
individual computer performance and needs to be replaced. During the
course of our review, these officials and representatives said that MTOPS is
outdated and can no longer adequately account for the performance
capabilities available from today’s computers. As a result, the continued
use of the MTOPS measure would cause systems with the same
performance capabilities to be treated differently under export control
regulations, according to Commerce’s Information Systems Technical
Advisory Committee.
The MTOPS measure is unreliable today because it is heavily, if not
exclusively, dependent on the clock rate of a microprocessor, measured in
megahertz, when several other factors should be considered. It does not
account for certain factors, such as memory retrieval times,
interconnection methods, and internal bus speeds. (A bus is a collection of
wires through which data is transmitted from one part of a computer to
another.) An internal bus connects all the internal computer components to
the central processing unit and main memory. The size of a bus is important
13
U.S. government officials include the Deputy Under Secretary of Defense (Science and
Technology) and officials of the Commerce and Energy Departments.
Page 14
GAO-01-10 Export Controls
because it determines how much data can be transmitted at one time. A
fast bus allows data to be transferred faster, which makes applications run
faster. In networking, a bus is a central cable that connects all devices on a
local area network. In addition, MTOPS is inaccurate if incorrectly applied
to computing elements organized in certain ways, such as computer
clusters. Under DOD auspices, U.S. government officials, computer
industry representatives, and academics met in December 1997 to
determine whether the use of MTOPS is sufficient to rate the relative
performance of current and future computer systems. The group
determined that there were variances of a factor of 2 in the performance of
delivered systems relative to the MTOPS-calculated performance because
of the wide range of architectures in use, but it concluded that continuing
rapid changes in technology might result in yet larger variances in the near
future.
President’s
Justification for
Changing Computer
Export Control
Thresholds Was
Inadequate
The National Defense Authorization Act of 1998 requires the President to
provide a report to the Congress on three factors when proposing a
modification of the export control thresholds for high performance
computers. First, the report is to address the availability of high
performance computers from other countries. Second, it is to address all
potential computer uses of military significance at the new control
thresholds. Third, the report is to assess the impact of such uses on U.S.
national security interests. In response to the act’s requirements, the
February and August 2000 reports presented information on the availability
of individual computers from other countries but did not recognize such
countries’ capabilities to cluster computers to obtain computing
performance beyond the new thresholds. Furthermore, the August report
addressed only 22 of 172 known military uses of high performance
computers up to the new control threshold. Finally, the report made no
reference to the national security impact of specific identified uses at the
new control thresholds. Instead, the report stated that (1) pursuit of
particular national security applications would require more than computer
hardware; (2) there was little danger of most tier 2 and 3 countries
diverting high performance computers from other purposes and usefully
employing them for military uses; and (3) it was important for the United
States to retain a technological advantage in the design, development, and
production of microprocessors and computers. The reports relied for their
information about national security impacts of military computer uses on a
study that stated that it could not assess which countries of concern could
use computers for particular military applications because the U.S.
government did not have information to make such an assessment.
Page 15
GAO-01-10 Export Controls
Clustering Not Factored
Into New Thresholds in
Considering Computing
Performance Availability in
Other Countries
The President’s reports were required to address the extent to which high
performance computers with capabilities between the established level and
the new proposed level of performance are available from other countries.
In response to the requirement, the reports to the Congress justified the
new control thresholds based on industry-provided information indicating
the MTOPS performance of mass market processors that the industry
expected would be produced within the next 6 months. The reports stated
that such processors would be uncontrollable worldwide. While the
administration discussed computer clustering as contributing to such
availability, it did not factor computer clustering into its control threshold
changes made during 2000, nor did it indicate that with clustered systems
other countries could obtain performance capabilities that exceed the new
control thresholds. If they had presented this fact in the reports, it would
have revealed that the computer control system was ineffective.
Although the President’s February and August 2000 reports discussed
advances in clustering, they did not indicate that performance capabilities
of clustered systems have exceeded the new control thresholds. The
President’s reports noted that advanced network technologies, particularly
in parallel processing known as “distributed computing,” are freely
available. Cost considerations and advances in computer technology in
both the United States and overseas have created a favorable environment
for high performance distributed computing using readily available
hardware and software, according to the reports. The reports added that
relatively low-cost high performance computer systems typically consist of
large clusters of commercially available workstations or personal
computers that are linked by interconnection hardware and high-speed
communication software. As a result, the affordability and widespread
availability of these computer products permit foreign end users to
configure these commercially available products into high performance
computer systems. The President’s reports did not state that high
performance computing up to about 70,000 MTOPS is attainable.
While computer clustering was not factored into the control threshold
changes made during 2000, the President’s August 2000 report justified the
new control thresholds based on industry information indicating the
MTOPS performance of mass market processors to be produced within the
next 6 months that Commerce officials believed would be uncontrollable.
As of August 2000, the fastest mass-marketed microprocessor had a peak
performance measure between 3,000 and 4,000 MTOPS. By the fall 2000, a
new 6,100 MTOPS microprocessor was expected to be available, and a
four-microprocessor computer based on this new microprocessor was
Page 16
GAO-01-10 Export Controls
expected to have a performance level of about 24,000 MTOPS. The August
report further stated that the next generation of computers with eight
microprocessors and a performance of 26,000 MTOPS, is expected to be
widely available in early 2001. Consequently, for end users in tier 3
countries, thresholds were set at 28,000 MTOPS, slightly higher than the
expected availability of the 26,000 MTOPS machine. The report stated that
computer manufacturers in the United States and in foreign countries
continue to produce computers that are smaller, cheaper, and easier to
install and maintain but are more powerful than ever before. This trend is
due in large part to rapid advances in microprocessor technology,
according to the report. Computer companies projected that their chips
and systems would be produced in the tens of thousands per month,
depending on the specific processor.
All Potential Computer Uses
Not Addressed
The National Defense Authorization Act of 1998 requires that the
President’s report to the Congress address all potential uses of military
significance to which high performance computers at the new levels could
be applied. However, the President’s reports to the Congress during 2000
did not address all such computer applications, even though this
information was available from the 1998 Department of Defense- and
Commerce-sponsored study that was used as the basis for these sections of
the reports. DOD officials from the Defense Threat Reduction Agency,
which is responsible for sections of the reports on military uses of high
performance computers, told us that they took their information from the
1998 study because the agency did not have the resources to initiate an
independent review of this and related issues. The officials chose to report
on 3 of 10 categories of applications identified by the 1998 study.
Page 17
GAO-01-10 Export Controls
We identified a total of 22 computer applications mentioned in the
President’s report: 17 applications that can be operated up to a
performance level of 21,000 MTOPS and 5 applications that can be
operated at nonspecified levels between 1,000 and 115,000 MTOPS.14 The
1998 Defense- and Commerce-sponsored study, however, specifically
identified 172 military applications that were run on computers up to
28,000 MTOPS.15 Of these military applications, 47 were run on computers
between 20,000 and 28,000 MTOPS (the previous tier 3 computer control
thresholds for civilian end users and newly announced tier 3 computer
control thresholds for all end users). Appendix IV lists the applications of
national security importance identified for the 1998 Defense- and
Commerce-sponsored study.
Information Provided on
National Security Impacts
Was Inadequate
The National Defense Authorization Act of 1998 requires that the
President’s report to the Congress assess the impact on U.S. national
security interests of potential uses of military significance to which high
performance computers at the new levels could be applied. The President’s
reports, however, did not specifically discuss the impacts on U.S. national
security interests for any of the national security applications that the
report identified, as required by the 1998 National Defense Authorization
Act. The reports made general statements that most countries cannot
effectively use high performance computers and that there are therefore no
national security impacts on the United States. This 1998 study, however,
stated that the U.S. government provided insufficient information to assess
this issue.
Although the President’s August report identified only 22 of the 172 military
applications that were run on computers up to 28,000 MTOPS, the report
did not discuss the national security impacts on the United States of
Russia, China, or other countries obtaining high performance computing up
14
The President’s reports do not clearly distinguish computer applications by name and
performance levels. As a result, it is impossible in all cases to determine from the reports
whether mentioned applications are distinct or identical.
15
According to the 1998 Department of Defense- and Commerce-sponsored study, the
absence of a particular type of problem at some performance level should not be interpreted
as a statement that no version of that application can be solved at that performance level.
MTOPS are only rough indicators of the performance level required for a particular kind of
application. That a given problem was run on a machine with a particular MTOPS level does
not mean that all systems with a greater MTOPS level can solve the problem or that all
systems with a lower MTOPS level cannot.
Page 18
GAO-01-10 Export Controls
to the new control thresholds, even for those applications identified. The
President’s reports also did not identify which countries of concern could
gain the hardware capability to conduct militarily useful applications at the
new control thresholds. For example, the reports stated that Russia and
China have demonstrated that they have the expertise necessary to use
high performance computers for particular national security applications,
such as developing submarines, advanced aircraft, composite materials, or
a variety of other devices. The reports also did not discuss other countries
that can use high performance computing for military applications. India,
for example, which has organizations that we identified as both using high
performance computing for military purposes and being listed on the
Commerce Department’s “Entities List” of end users of proliferation
concern, was not mentioned in the President’s reports.
The reports stated that most tier 2 and 3 countries have little or no
experience in a host of national security applications and there is little
danger of these countries diverting high performance computers from
other pressing needs and usefully employing them to develop military
items. According to DOD officials responsible for the sections of the report
on military uses of high performance computers, the 1998 Defense- and
Commerce-sponsored study was the basis for these statements. The 1998
study, however, indicated that it did not have sufficient information to
assess the impact of militarily significant applications on U.S. national
security interests. The study stated that
“A critical question, which we have been unable to pursue satisfactorily in this study, is
which countries are able to productively use [high performance computing] to pursue which
applications? We have requested such information from the U.S. national security
community, but have received few answers. It does not appear that the U.S. government is
effectively gathering such intelligence in a systematic fashion. More specifically, the U.S.
government does not appear to have as good an understanding of individual end use
organizations of concern as is needed by the export control regime.”
The President’s reports did not state which countries can use high
performance computers for particular military applications or which
countries cannot, and the analysis that was absent in the 1998 Defense- and
Commerce-sponsored study and an earlier 1995 study has not been done.
As we reported in 1998,16 the principal author of the Defense- and
Commerce-sponsored study and DOD officials told us that no assessment
had been done in 1995 to determine how national security would be
16
Export Controls (GAO/NSIAD-98-196, Sept. 16, 1998).
Page 19
GAO-01-10 Export Controls
impacted if high performance computers were exported to particular
countries of concern and what military advantages such countries could
achieve. The 1998 study reiterated this position. We recommended in 1998
that the Secretary of Defense do such an evaluation, specifically addressing
(1) how and at what performance levels countries of concern use high
performance computers for military modernization and proliferation
activities, (2) the threat of such uses to U.S. national security interests, and
(3) the extent to which such high performance computers are controllable.
This recommendation has not been implemented. Although DOD stated
that the interagency process had considered these factors in the 1995
review of computer export controls and would consider them in any future
review, it provided no evidence that this has occurred.
An annual report on transfers of militarily sensitive technology to countries
and entities of concern, required under section 1402 of the National
Defense Authorization Act for fiscal year 2000, might partially address the
question of the national security impact of high performance computer
uses of military significance. It is to include an assessment by the Secretary
of Defense, in consultation with the Joint Chiefs of Staff and the Director of
Central Intelligence, on the cumulative impact of U.S.-granted licenses for
exports of technologies and technical information to these countries and
entities. Thus, the study would be expected to include licensed high
performance computers as part of the assessment. This report is also
required to include information on countermeasures that may be necessary
to overcome the use of such technologies and technical information.
However, it is not required to include an assessment of the cumulative
impact of nonlicensed computer exports, such as those that could be
clustered. The report is scheduled for completion by the end of 2000.
Conclusions
The current export control system for computers, as implemented by the
Department of Commerce and which focuses on controlling the export of
individual computers, is not effective in limiting countries of concern from
obtaining high performance computing capabilities. As a result of advances
in clustering technology, countries of concern can obtain computing
capabilities above current U.S. export control levels necessary for military
applications. It is also now widely recognized within the Departments of
Commerce, Defense, and Energy and the computer industry that the use of
MTOPS as the means of determining export control thresholds is outdated,
no longer a valid means for controlling computing capabilities, and needs
to be replaced. In light of these now well established developments, the
process needs to be reexamined, and potential alternative ways to
Page 20
GAO-01-10 Export Controls
safeguard U.S. national security interests related to high performance
computer exports need to be explored.
In addition, the President’s reports justifying computer control changes
continue to be inadequate. It is a particularly important omission for the
U.S. government to not assess the national security impacts of how
countries of concern can use high performance computers at successively
higher performance levels for military purposes. Although we have
highlighted this issue in the past, the administration has presented no
further assessment. As advances in high performance computing make it
more difficult to maintain the U.S. technological lead in military
capabilities by denying advanced technology transfers to countries of
concern, it may become necessary to explore other options to maintain the
U.S. military technology lead. For example, the National Defense
Authorization Act for fiscal year 2000 required the Secretary of Defense to
assess the cumulative impact of U.S. export licenses—including high
performance computing licenses—and possible countermeasures that may
be necessary to overcome the use of such technologies and technical
information. However, it did not require an assessment of the cumulative
impact of exporting nonlicensed computers that could be clustered, or of
potential countermeasures to such an impact.
Recommendations for
Executive Action
Since the current export control system for high performance computers
cannot prevent countries of concern from obtaining high performance
computing capabilities, we recommend that the Secretary of Commerce, in
consultation with the Secretaries of Defense, Energy, and State, convene a
panel of experts to comprehensively assess and report to the Congress on
possible ways of addressing the shortcomings of computer export controls,
including, but not limited to the ideas noted in this report. For example, the
Commerce Department’s National Institute of Standards and Technology,
which researches computer systems’ performance and promotes the
effective evaluation and efficient use of advanced computers, might
participate in the panel because it designs evaluations that economically
and reliably characterize high performance computer designs. This
assessment should report on the costs and benefits of each proposed idea,
including its technical feasibility.
In addition, the report required by the National Defense Authorization Act
of 2000 concerning countermeasures that may be necessary to overcome
the use of sensitive technologies and technical information exported to
countries of concern is not required to include an assessment of the
Page 21
GAO-01-10 Export Controls
cumulative impact of exports of nonlicensed computers, such as those that
could be clustered. Therefore, we recommend that the Secretary of
Defense determine what U.S. countermeasures are necessary, if any, to
respond to computing-related enhancements of the military or proliferation
capabilities of countries of concern.
Agency Comments and
Our Evaluation
The Department of Commerce provided written comments (see app. IV) on
a draft of this report. The Department of Energy’s Acting Chief Operating
Officer for the National Nuclear Security Administration and the Principal
Deputy Under Secretary of Defense for Policy provided oral comments.
The Department of Energy concurred with the report’s findings and
conclusions and our recommendation concerning the convening of a panel
of experts and said that the Department looks forward to participating with
the Secretaries of Commerce, Defense, and State in such a panel to conduct
a comprehensive assessment of possible ways of addressing the
shortcomings of computer export controls. The Departments of Commerce
and Defense generally agreed with the report’s findings and conclusions
relating to the problems we identified with clustering technology and the
continuing use of MTOPS as a measure for controlling individual high
performance computers. However, they disagreed with the need to
convene an expert panel or to determine what countermeasures are
necessary to respond to enhancements of military capabilities of countries
of concern which such countries may have gained through the use of high
performance computing.
The Departments of Commerce and Defense said that they disagreed with
the need to implement our recommendation that the Secretary of
Commerce and other secretaries convene a panel of experts to
comprehensively assess possible ways of addressing the shortcomings of
computer export controls because they said the administration and experts
are already studying similar issues. Commerce stated that it would be
counterproductive and unnecessary to convene a panel of experts, since
the administration was already examining this issue through existing
interagency mechanisms involving DOD, Energy, and State. Commerce also
said that the Center for Strategic and International Studies has a broad
group of experts currently studying the same issues as encompassed in the
recommendation. DOD said that the administration has been consulting
with experts to formulate an approach for controlling computing
capabilities and is assessing various alternatives—including “some”
identified in our draft report. Subsequent to receiving these comments, we
asked both Departments to describe in more detail and document what
Page 22
GAO-01-10 Export Controls
comprehensive study was being conducted. Neither Department provided
any additional information to show that there was an interagency study
being conducted as we are recommending. Although we sought
information on steps being taken that would address the issues covered by
our recommendations, neither the Commerce Department nor Department
of Defense provided any evidence that an interagency mechanism is
systematically examining alternative ways to protect U.S. national security
interests related to the shortcomings of computer export controls—beyond
the narrowly focused effort to seek alternatives to the MTOPS measure for
controlling individual machines.
Regarding the study by the Center for Strategic and International Studies,
we note that it is tasked to “develop the framework for a new effective . . .
agreement that would regulate certain militarily useful goods and
technologies on a multilateral basis,” not to examine the costs, benefits,
and technical feasibility of each possible way identified in this report of
addressing the shortcomings associated with controlling individual high
performance computers.
The Departments of Commerce and Defense also disagreed with the need
to implement our recommendation that the Secretary of Defense determine
what U.S. countermeasures are necessary to respond to enhancements of
the military capabilities of countries of concern from both licensed and
nonlicensed high performance computing. Commerce stated that the best
countermeasure is to ensure that the U.S. military and defense industries
continue to have access to the computer technology needed to maintain the
U.S. military advantage. DOD stated that the Department continuously
assesses the military capabilities of potential adversaries and countries of
concern to identify threats to U.S. forces and described the assessment
process. While we agree that maintaining the U.S. technological advantage
in computing is important, we note that neither Commerce nor Defense
specified how or whether DOD (1) has assessed and identified any threats
posed by high performance computing to U.S. national security interests or
(2) identified and implemented countermeasures to such threats.
Furthermore, ongoing studies tasked to assess the need for
countermeasures related to U.S. exports of sensitive technologies are not
reviewing threats posed by nonlicensed clustered computer systems, a
factor that we believe is critical to assess.
DOD disagreed with our conclusion that the President’s February and
August 2000 reports to the Congress were inadequate with regard to
required national security assessments accompanying changes to computer
Page 23
GAO-01-10 Export Controls
control levels. DOD stated that the President’s reports have consistently
noted that most activities associated with various aspects of military
capabilities benefit from some computing capabilities. It further stated that
the President’s reports to the Congress have provided examples of national
security applications to illustrate that the range of military applications that
(1) benefit from high performance computing is “almost ubiquitous” and
(2) can be performed on almost any computer is extensive. Nonetheless, as
this report and our prior reports have pointed out, we continue to believe
that the President’s reports did not adequately provide the information that
the law requires. The President’s reports did not address all potential uses
of military significance to which high performance computers at the new
levels could be applied—even though the administration possessed
information on such uses in the 1998 Defense- and Commerce-sponsored
study (part of which we have reprinted in app. III)—and did not assess the
impact of such uses on U.S. national security interests.
Matters for
Congressional
Consideration
Since the Departments of Commerce and Defense disagreed with
recommendations which we believe are still valid and needed, the
Congress may wish to institute a requirement that the Secretary of
Commerce, in consultation with the Secretaries of Defense, Energy, and
State, convene a panel of experts to conduct a comprehensive assessment
of and report to the Congress on possible ways of addressing the
shortcomings associated with controlling individual high performance
computers, including, but not limited to, the ideas noted in this report.
In addition, to address the issue of countermeasures that may be necessary
to overcome the use of sensitive technologies and technical information
exported to countries of concern, the Congress may also wish to consider
instituting a requirement that the Secretary of Defense determine and
report on what U.S. countermeasures are necessary, if any, to respond to
computing-related enhancements of the military or proliferation
capabilities of countries of concern, as we have recommended in this
report.
Scope and
Methodology
To assess the effectiveness of current export controls in preventing
countries of concern from obtaining high performance computing
capabilities that can be used for military applications, we reviewed studies
and briefing slides by U.S. government agencies and computer industry
technical specialists that described technological computing advances and
Page 24
GAO-01-10 Export Controls
assessed the MTOPS measure. Such studies included assessments by the
Lawrence Livermore and Sandia National Laboratories on possible
alternatives to current computer controls. We interviewed officials from
the Departments of Defense, Commerce, State, and Energy; the Central
Intelligence Agency; and the three major national weapons laboratories,
Lawrence Livermore, Los Alamos, and Sandia. We interviewed officials
from major computer manufacturers, Compaq, Hewlett Packard, IBM, and
SUN Microsystems, as well as the computer scientist responsible for the
Top 500 List of the most advanced high performance computers in the
world. In addition, we observed meetings of Commerce Department’s
Information Systems Technology Advisory Committee.
To evaluate the President’s justifications for changing the computer control
thresholds, we reviewed the President’s February 2000 and August 2000
reports to the Congress justifying changes to computer control thresholds
and a White House fact sheet detailing the changes made in August 2000.
We reviewed Commerce Department analyses and computer industry
information regarding projected production schedules and technical
performance ratings for processors scheduled to be marketed in the next
6 to 12 months. In addition, we reviewed studies and briefing slides by U.S.
government agencies and computer industry technical specialists that
described technological computing advances and assessed the MTOPS
measure. We interviewed officials from the Departments of Defense,
Commerce, State, and Energy; the Central Intelligence Agency; and the
three major national weapons laboratories, Lawrence Livermore, Los
Alamos, and Sandia. We interviewed officials from major computer
manufacturers, Compaq, Hewlett Packard, IBM, and SUN Microsystems, as
well as the computer scientist responsible for the Top 500 List of the most
advanced high performance computers in the world. In addition, we
observed meetings of the Commerce Department’s Information Systems
Technology Advisory Committee.
We conducted our review from February through November 2000 in
accordance with generally accepted government auditing standards.
Unless you publicly announce the contents of this report earlier, we plan no
further distribution of this report until 15 days from its issue date. At that
time, we will send copies of this report to other appropriate congressional
committees; the Honorable William S. Cohen, Secretary of Defense; the
Honorable Madeleine K. Albright, Secretary of State; the Honorable
Norman T. Mineta, Secretary of Commerce; and the Honorable William
Page 25
GAO-01-10 Export Controls
Richardson, Secretary of Energy. Copies will also be made available to
others upon request.
If you or your staff have any questions concerning this report, please call
me at (202) 512-4128 or Mr. Rhodes at (202) 512-6412. Key contributors to
this assignment were Claude Adrien, Jeffrey D. Phillips, F. James Shafer,
and Hai Tran.
Harold J. Johnson
Director, International Affairs
and Trade
Keith A. Rhodes
Chief Technologist
Page 26
GAO-01-10 Export Controls
Page 27
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
Appendx
ies
Appendx
Ii
To identify possible alternative ways to safeguard U.S. national security
interests related to computer exports, we reviewed analyses and other
documentation prepared by the two Department of Energy national
laboratories addressing this question, Lawrence Livermore and Sandia. We
also interviewed officials from the Departments of Defense, Commerce,
State, and Energy—including Los Alamos, Lawrence Livermore, and Sandia
National Laboratories; the Central Intelligence Agency; and officials from
major computer manufacturers, Compaq, Hewlett Packard, IBM, and SUN
Microsystems.
We identified 12 ideas for addressing shortcomings to the current export
control system. These ideas have not been comprehensively evaluated by
the U.S. government. The Department of Commerce has considered the
first three ideas we describe in this appendix, but none of the three
addresses the export control problem created by clustering technology. We
also present nine additional ideas identified through our research and
discussions with experts. We did not assess the feasibility of implementing
any of the ideas but rather describe their characteristics and limitations
when identified by the experts. It is important to note that these ideas have
not been assessed for their feasibility to replace the current export control
mechanism; moreover, most of them do not address the challenge created
by advances in clustering technology.
Alternatives to Using
MTOPS Considered by
the CommerceSponsored Technical
Advisory Committee
The three ideas that the Commerce-sponsored Information Systems
Technical Advisory Committee (ISTAC) reviewed are (1) counting
processors, (2) measuring power dissipation, and (3) indexing control
thresholds to a common benchmark.
Counting Processors
Instead of Using MTOPS
Using this approach, computers would be controlled by counting the
number of processors in each computing system. For example, a license
might be required if a computer to be exported to tier 3 countries contained
eight or more processors, while computers with fewer processors could be
exported without a license. Presumably, the precise number of processors
for each tier would be determined as the details of such a proposal were
finalized.
Page 28
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
Computer industry representatives and ISTAC members stated that
counting processors would be simpler than using the MTOPS measure.
Under current controls, computer chip manufacturers must calculate the
MTOPS rating for each processor they manufacture, and computer
companies must calculate the MTOPS rating for each of the computer
systems they produce. Department of Energy and ISTAC officials also
noted that by counting processors, they could better adapt to changes in
computer technology because advances in processors used in a system
would not result in the need to change the export controls. On the other
hand, ISTAC members said that counting processors is no more accurate
an indicator than MTOPS for indicating a level of computer performance.
Furthermore, despite the intended simplicity of the approach, the
Committee members noted that there is no consensus within the computer
community on the definition of a processor. They also said that exceptions
to the counting procedure would need to be determined for high
performance single processors and for processors and computers with
unique designs. For example, some processors today include advanced
graphics or memory capabilities, but others do not and therefore have
different performance capabilities.
Measuring Power
Dissipation
Power dissipation is a measure of watts per MTOPS that is currently used
in Japan to determine the environmental effects of computers. The
measure produces a ratio of the watts per MTOPS to the size of the box
housing the microprocessor. The May 2000 ISTAC paper stated that this
alternative was briefly studied but rejected because of its inability to
accurately measure the performance capabilities of a system. Two systems
with the same performance capability might have different ratios because
each is housed in a different sized box with different power supplies and
different disk drives, according to the Committee paper. As a result, the
system measured to have a lower ratio would be controlled less stringently
than the smaller system.
Indexing Control
Thresholds to a Common
Benchmark
Computer export control levels could be set by using a common
performance benchmark. The May 2000 ISTAC paper on alternative control
measures1 also identified one benchmark, Linpack, that it called an
accepted industry standard for performance measurements. The paper
1
Alternative Control Parameter for High Performance Computers.
Page 29
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
noted, however, that changing the operating system on a computer would
change the measure and that unless the exporter knows what operating
system the end user has, the measure would be impossible to accurately
gauge. Nonetheless, the paper suggested continued study of the
benchmark, which might have more relevance to higher performance
computers (at 100,000 MTOPS) in the future. We observed that the
Commerce Department’s National Institute of Standards and Technology
does research in the area of computer systems performance, promotes the
effective evaluation and efficient use of advanced computers, and designs
coherent evaluations that economically and reliably characterize high
performance computer designs.
Some computer industry representatives said that the industry would never
achieve consensus on which benchmarks to use. Furthermore, they said
that once any specific benchmark was established, computer designers
could circumvent the intent of the benchmark.
Other Alternatives to
Replace MTOPS Drawn
From Studies and
Discussions With
Computer Experts
In addition to the three alternatives to replace MTOPS considered by the
Commerce-sponsored advisory committee, the following two were
identified from studies and discussions with government and industry
experts.
Page 30
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
Indexing Control
Thresholds
Sandia National Laboratory, computer industry representatives, and a
computer expert from academia have presented the idea of indexing the
control thresholds through the use of various criteria, including (1) actual
sales or market data or (2) a list of the 500 most powerful computers,
which is maintained by the University of Tennessee and the University of
Mannheim.2
Actual Sales
According to an analysis prepared by Sandia National Laboratory, one way
of implementing a sales-based index would be for the Secretary of
Commerce, in consultation with other agencies, to determine every
6 months or after a petition by industry, which U.S.-origin microprocessors
have entered into mass production during the preceding 180 days. The
Secretary would then have 30 days to make the determination. Computers
incorporating such microprocessors could then be controlled by requiring
export licenses based on a multiple of the performance of a single
mass-produced microprocessor. Under this indexing option, U.S. computer
companies would have to first establish a threshold level of sales within the
United States before they are exported, according to Sandia National
Laboratory. The threshold might initially be 100, 500, or more commercial
sales within the United States and maybe tier 1 countries to establish that
the product is indeed a broadly applicable, commercial product. Once the
company demonstrates that it has reached the threshold through actual
sales, it would no longer be prohibited from exporting such computers to
tier 2 and tier 3 countries, although documentation of end use and controls
could be maintained for tier 3 countries, according to Sandia National
Laboratory.
Experts we consulted raised concerns about whether the sales or
market-based data would be reliable and a U.S. government official told us
that this option was flawed because it would not be based on national
security interests. Finally, some computer industry representatives
questioned whether this control could be effectively implemented because
2
To provide a better basis for statistics on high performance computers, the University of
Tennessee and the University of Mannheim maintain a list of sites that have the 500 most
powerful computer systems installed in the world. The Top 500 List has been updated twice
a year since June 1993 with the help of high performance computer experts, computational
scientists, manufacturers, and the Internet community who responded to a questionnaire.
The producers of the Top 500 List have also used parts of statistical lists published by others
for different purposes.
Page 31
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
of a perceived government inability to keep up with market and/or
technological changes.
Top 500 List
Another proposal suggested by some experts we consulted was to apply
export controls to computers based on the list of the world’s top 500 most
powerful computers maintained by the University of Tennessee and the
University of Mannheim. For example, licenses could be required for
computers whose capabilities qualified them for inclusion on the list, at
least at the 500th rank. Alternatively, the threshold could be set for
computers surpassing the 250th, 100th, or any other ranking on the list.
Proponents of this idea point out that the list is based on a transparent
measure: a publicly available, testable benchmark. Unlike the current
control system, this list would not be subject to sudden and significant
changes in license thresholds due to marketplace changes in just one or
two chips. The controls would therefore be tied much more closely to
technological realities.
Other Ideas for
Safeguarding U.S.
National Security
Interests
In addition to ideas to replace MTOPS discussed above, the following seven
ideas were identified from studies and discussions with government and
industry experts as other ways to protect U.S. national security interests
relative to high performance computer exports.
Tagging and Remote
Monitoring
Tagging and remote monitoring is an idea that has been discussed in export
control literature for several years. It is achieved by attaching a monitoring
system to the item that is to be exported. The active system would both
monitor the object tagged and communicate that information to the United
States. Elements of a remote monitoring system are data acquisition,
communications, and a command center.
U.S. government officials3 and computer industry representatives pointed
out several limitations of tagging and remote monitoring. Both procedures
require the cooperation of the end user of the computer system. In
addition, the American computer industry may resist having its products
3
These officials were from the Lawrence Livermore, Los Alamos , and Sandia National
Laboratories and the Department of Energy.
Page 32
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
tagged, given that this practice might discourage legitimate foreign
customers from buying American computers because of privacy concerns,
according to the officials. They also noted that a remote monitoring system
for exported computers might not be feasible because of the resources that
might be required to monitor computer operations and send data to the
United States. Also, even remote monitoring of computer operations
cannot distinguish how the computer is being used and for what
applications, according to the officials. Some officials cautioned that any
type of tagging or remote monitoring to which the end user has
unsupervised access could be tampered with without the U.S. government’s
knowledge.
Assessing End-User
Attainable Performance
This idea is simply a modification of the current licensing process for
determining when export control thresholds apply. Under this approach,
the licensing threshold would be applied according to the potential
capabilities of a computer system, rather than the actual performance level
at the time of sale, as under the current system. One option would be to
retain the MTOPS measure and adjust the control level to account for the
maximum processing capability of the system instead of setting the
threshold based on the capabilities of the system as configured at the time
of shipment. For example, under the current system, a computer that had
four microprocessor slots but was shipped with only one microprocessor
would likely be exported without a license. However, the end user could
buy three additional uncontrolled microprocessors and increase the rated
performance. Considering the end user’s attainable performance, the
system would be rated at its full four-processor potential and likely require
a license to better reflect the true capabilities acquired in the purchase.
This standard could be applied in any performance-based system, whether
using MTOPS or another measure.
Proponents of end-user attainable performance as a measure argue that
controls based on maximum capabilities prohibit the buyer from increasing
the capabilities of the purchased system without the U.S. government’s
knowledge. One criticism of this approach is that clustering can still
obviate the control because users can link computers together after
delivery without detection by the U.S. government. This control could also
be rendered less effective or ineffective by advancing technology that
allows users to upgrade their computers with next generation chips,
making the capabilities of the systems shipped difficult or impossible to
track. In addition, some critics stated that this control is not replicable in a
consistent fashion and raised questions about how to choose the control
Page 33
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
levels and use them. Finally, some doubted the acceptability of this option
within the computer community.
Raising Export Control
Thresholds to the Level
Obtained by Clustering
Under this option, the government could set the control levels to match the
computational power that can be effectively provided by computer clusters
composed of readily available components. This approach to setting export
control levels is based on the notion that effective parallel computers can
today be built using readily available commercial computers and
networking technology with performance between 50,000 to 70,000
MTOPS. Periodic revision of the export control levels could reflect the
combined effects of advances in central processor unit technology and the
increasing capabilities of adjunct technologies in communications and
networking to achieve very high levels of computing power by linking
together larger numbers of computers.
Using this approach, today’s large inconsistency between the levels at
which high performance computers are controlled and the performance
levels achievable by commercially available parallel clusters would be
eliminated, according to the Lawrence Livermore National Laboratory
analysis.4 The analysis stated that the true state of U.S. exposure to
changing national security threats would be acknowledged and addressed.
Appropriately responding to the implications of the actual availability of
high performance computing would require extensive review and
discussion in the national security community. Some critics of this
approach stated that this control is not replicable in a consistent fashion
and raised questions about how to choose the control levels and use them.
4
The National Security Implications of Decontrolling Export to Tier III Countries of High
Performance Computers between 2,000 and 40,000 MTOPS (Feb. 2000).
Page 34
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
Controlling Software
Applications
This option involves denying countries of concern key software
applications used in high performance computing through the use of
security classifications and/or export controls. According to the Director of
DOD’s High Performance Computing Modernization Program,5 one concept
under discussion is the development of approaches to ensure that software
is used as intended and authorized. He said that research would be needed
to develop encryption technology to restrict access for source and
executable applications. The intent of this concept is to develop means that
will prevent militarily relevant applications from operating on unauthorized
machines. The official stated that technology advances can provide
improved protections for militarily relevant applications. According to the
DOD Deputy Under Secretary (Science & Technology), militarily significant
software might be made more tamper-proof, that is, able to operate on only
one type of machine or to self-destruct if tampered with. The official said
that DOD is just beginning to examine this option.
The prevalent view among government and computer industry officials
with whom we spoke is that the applications software that can be
controlled is already protected by security classification, export controls,
and proprietary rights and that commercial dual-use software cannot be
controlled. A Lawrence Livermore National Laboratory analysis pointed
out that some software used in clusters is freely available on the Internet.
Operating systems such as Linux can also be downloaded free from the
Web. Even operating systems like Windows NT or Windows 2000 are sold in
such quantities that controls are viewed as impractical and unenforceable.
Research and academic institutions whose culture is characterized by the
free exchange of information and software are notable users and
developers of high performance computing technology. Thus, it is difficult
to find any part of the software application market that lends itself to
controls because distribution is easy and applications of national interest
are already tightly held by their owners, either through security
classification or efforts to protect a commercial interest. In contrast, the
Director of DOD’s High Performance Computing Modernization Program
stated that dual-use applications are being controlled to a limited extent.
5
The DOD High Performance Computing Modernization Program provides advanced
hardware, computing tools, and training to DOD researchers utilizing the latest technology
to aid their mission in support of the warfighter. The program seeks to modernize the total
high performance computational capability of DOD Science and Technology, Development
Test and Evaluation, and Ballistic Missile Defense Organization to a level comparable to that
available in the foremost civilian and other government agency research and development
environments.
Page 35
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
He noted that several commercially available applications cannot be run
without receiving an annually provided key from the vendor.
Controlling Technology
Used for Interconnection
The idea of controlling technology used to link computers together is based
on the control of the number and capability of commercial readily available
parallel clusters that can be created by controlling networking hardware
such as switches, interface cards, and related equipment.
However, the hardware used in high-speed, commercial, ready-made
networking is already widely available and relatively inexpensive
throughout the world, according to U.S. government officials6 and
computer industry representatives. They believe that establishing controls
at this point would be extremely difficult, if not impossible. To be effective,
this approach would depend on networking technology originating solely in
the United States or countries with binding agreements to enforce U.S.
export control objectives, according to U.S. government officials and
computer industry representatives. However, manufacturing capability for
interconnecting technology is available worldwide, and with the growth in
the cluster market, it will be considerably more difficult to restrict supply.
Industry representatives consider only some of the equipment related to
this technology controllable at the higher end of the market because of its
proprietary nature and the continued importance of vendor support.
Controlling Computer
Systems Based on
Bandwidth
According to the Director of DOD’s High Performance Computing
Modernization Program, a concept under discussion would develop a
methodology to control computer systems by some measure of processorto-main-memory bandwidth7 and potentially the number of processors in
each system. The methodology would need to distinguish between
commodity systems and the traditional class of supercomputers
characterized by specialized processors. Such computer systems typically
have a single processor and require a high level of bandwidth, such as those
manufactured by Cray, NEC, and Fujitsu. Another official from DOD’s High
Performance Computing Modernization Program said that these computers
still possess capabilities, capacities, and characteristics that should be
6
Officials were from the Lawrence Livermore, Los Alamos, and Sandia National
Laboratories and the Departments of Commerce, Defense, and Energy.
7
Bandwidth is the amount of data that can be transmitted in a fixed amount of time.
Page 36
GAO-01-10 Export Controls
Appendix I
Possible Ways of Safeguarding U.S. National
Security Interests Related to Computer
Exports
denied to countries of concern. The DOD official identified as a concern
those computers that can solve militarily significant problems that cannot
be broken down and operated on clustered systems. Computer industry
experts placed the total market for these computers in the range of 300 per
year. The Department did not provide any additional information about this
option because it was still under review.
Implementing
Countermeasures to
Military Advantages Gained
by Countries of Concern
From More Advanced
Computer Exports
Using this option, DOD would design countermeasures to deal with the
implications of wider computer availability. One purpose of export controls
is to maintain the U.S. technological lead in military capabilities by denying
transfers of advanced technology to countries of concern. As technological
advances in high performance computing make this purpose more difficult
for export controls to achieve, it may become necessary to explore other
options to maintain the U.S. technology lead. As a step in this direction,
section 1402 of the National Defense Authorization Act for Fiscal Year 2000
requires an annual report on transfers of militarily sensitive technology to
countries and entities of concern. It is to include an assessment by the
Secretary of Defense, in consultation with the Joint Chiefs of Staff and the
Director of Central Intelligence, on the cumulative impact of U.S.-granted
licenses for exports of technologies and technical information. This report
is required to include information on countermeasures that may be
necessary to overcome the use of such technologies and technical
information. It is not required to include an assessment of the cumulative
impact of nonlicensed computers, such as those that could be clustered.
The report is scheduled for completion in late 2000.
Page 37
GAO-01-10 Export Controls
Appendix II
Additional Information on Computer
Clustering Technology and MTOPS
Appendx
iI
In the past few years, broadly recognized technological advances in the
ability to cluster together low performance computers have resulted in an
increasing capability worldwide to achieve high performance computer
levels. As a result, the known performance levels of clustered systems
exceed the export control thresholds set by the U.S. government. In
addition, U.S. government officials1 and computer industry representatives
have discussed the shortcomings of the MTOPS measure for export control
purposes and now agree that MTOPS is no longer a valid measure of
computer performance and needs to be replaced.
Computer Advances in
Clustered Systems
Technological advances in computing that would threaten the effectiveness
of export controls have been expected for several years. For example, a
1994 U.S. government study2 warned that technological computing
advances would seriously impair the ability of U.S. export controls to
protect computer technology by the turn of the century. A Defense- and
Commerce-commissioned team in 19953 reviewed the computer industry’s
technological advances in parallel processing and concluded that advances
in clustering contribute to the uncontrollability of high performance
computing worldwide and inevitably reduce the effectiveness of U.S.
export controls. One of the technology trends of concern at that time
included other countries’ ability to link individual computers to achieve
higher performance levels. We reported 2 years ago that trends in high
performance computing technology development might pose security and
export control challenges and recommended further study to determine
their implications for national security and export controls.4
According to a Lawrence Livermore National Laboratory analysis dated
February 2000, in determining the usefulness of clusters made with readily
available commercial components, the following four aspects of computing
must be considered:
1
U.S. government officials include the Deputy Under Secretary of Defense (Science and
Technology) and officials of the Commerce and Energy Departments.
2
The unclassified title of this classified report is High Performance Computing Technology:
Implications for Development of Weapons of Mass Destruction.
3
High-Performance Computing Export Control Policy in the 1990s (Palo Alto, CA: Stanford
University, Center for International Security and Arms Control, 1995).
4
Export Controls (GAO/NSIAD-98-196, Sept. 16, 1998).
Page 38
GAO-01-10 Export Controls
Appendix II
Additional Information on Computer
Clustering Technology and MTOPS
• Performance, flexibility, low cost, and the ease of integration, taken
together, make the development of effective parallel computers using
commercial readily available networking hardware dramatically easier
than it was just 3 years ago, according to Lawrence Livermore National
Laboratory officials. Commercial clusters of readily available
components perform up to about a third as well as today’s best
distributed memory parallel computers (stand-alone high performance
computers), while the cost of clustering adds only 10 to 30 percent to
the cost of the computers that are clustered.
• The system software necessary to integrate a cluster of computers into
an effective computing system with a correspondingly higher capability
than its individual constituent processors is now widely available.
• Programming a commercial cluster of readily available components is
no different from programming most other commercial parallel
processing computers because the system software is the same on both.
Computer science departments in universities worldwide teach
concepts to support the development of parallel applications for
commercial, readily available clusters. Other U.S. government and
computer industry specialists, however, stated that there might be some
number of computer applications that cannot be adapted for clustered
parallel processing. To the extent that such applications are militarily
significant and must be operated on individual computers, maintaining
threshold controls on these machines could still be effective.
• Support for computer functions means having an efficient schedule for
running hundreds of separate problems and the input and output of data
among a computer, its local disks, networks, and archival storage.
Without vendor-supplied software to automate these functions,
everything must be done manually, making production computing a
labor-intensive operation. While U.S. practice is to pay computer
vendors to supply this support, foreign countries such as India and
China, where skilled labor is plentiful and low cost, may find providing
this support much less of a problem.
Other U.S. government and computer industry specialists also stated that
there might be some computer applications that cannot be adapted for
clustered parallel processing. The President’s August 30, 2000, report
indicated that some small number of computer applications still requires
traditional computer systems and that the lack of hardware may be a
barrier to solving certain kinds of problems. The President’s report did not
identify these applications. However, computer applications that have
problems which cannot be broken up to be solved in parallel and
applications associated with battlefield operations that rely on rapid
Page 39
GAO-01-10 Export Controls
Appendix II
Additional Information on Computer
Clustering Technology and MTOPS
solutions might not practically be run on clustered computer systems,
according to ISTAC members and a DOD official.
Shortcomings of
MTOPS as a Measure
of Performance
U.S. government officials5 and computer industry representatives agreed
that MTOPS is no longer a valid measure of computer performance and
needs to be replaced. In 1991, the Commerce Department began using
MTOPS as a measure to decide when an export license would be required
for high performance computers. But because of the diverse computer
designs that have evolved since then, the increasing performance level of
commercial microprocessors, and the capability to connect local and wide
area networks, it became prudent to reexamine the suitability of MTOPS as
a measure, according to the Institute for Defense Analyses. As a result,
under DOD auspices, U.S. government officials, computer industry
representatives, and academics met in December 1997 to determine
whether the use of MTOPS is sufficient to rate the relative performance of
current and future computer systems. They determined the following:
• The use of MTOPS was still an effective means at that time for
determining export controls for a single computing element. The
attendees stated that modest refinements could be made to the MTOPS
measure for systems comprising aggregate computing elements.
• There could be variances up to a factor of 2 in the actual performance of
delivered systems relative to the MTOPS-calculated performance
because of the wide range of designs in use. Continuing rapid changes in
technology might result in yet larger variances in the near future.
• Any export control measure should be reevaluated every 2 years
because of the rapid changes in computer designs.
By 2000, however, it had become clear to both government and industry
computer experts that MTOPS does not account for new processor
designs, particularly those dependent on the capability to exchange
information internally at high speeds. Representatives of three computer
companies said that MTOPS was an archaic measure not designed with
new chip designs in mind and should be replaced. The MTOPS measure
today is unreliable because it does not account for certain factors, such as
memory access times, interconnection methods, and internal bus speeds.
As a result, two microprocessors with the same megahertz could have
5
U.S. government officials include the Deputy Under Secretary of Defense (Science and
Technology) and officials of the Commerce and Energy Departments.
Page 40
GAO-01-10 Export Controls
Appendix II
Additional Information on Computer
Clustering Technology and MTOPS
different MTOPS ratings if the additional factors were considered. The
MTOPS indicator also would be inaccurate if it were applied incorrectly for
computing elements organized in certain ways, such as in clusters of
individual computers. It is not accurate to take the MTOPS measure of one
clustered machine and multiply it by the total number of computing
processors of the cluster because key characteristics, such as the time
taken to distribute tasks and exchange intermediate results, have not been
taken into account.
Page 41
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Appendx
Ii
This appendix lists the applications of national security importance
identified for the 1998 Defense- and Commerce-sponsored study, which
was the basis for information on computer applications in the President’s
reports in February and August 2000. The list is reprinted in its entirety as it
appeared in the 1998 study.
Page 42
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 43
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 44
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 45
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 46
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 47
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 48
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 49
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 50
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Page 51
GAO-01-10 Export Controls
Appendix III
Identified Applications of National Security
Importance
Source: High-Performance Computing, National Security Applications, and Export Control Policy at the
Close of the 20th Century, Seymour E. Goodman, Peter Wolcott, and Patrick Homer, (Palo Alto, CA:
Stanford University, Center for International Security and Arms Control, 1998).
Page 52
GAO-01-10 Export Controls
Appendix IV
Comments From the Department of
Commerce
Appendx
i
IV
Note: GAO comments
supplementing those in the
report text appear at the end
of this appendix.
See comment 1.
See comment 2.
Page 53
GAO-01-10 Export Controls
Appendix IV
Comments From the Department of
Commerce
Page 54
GAO-01-10 Export Controls
Appendix IV
Comments From the Department of
Commerce
The following are GAO’s comments on the Department of Commerce’s
letter dated November 16, 2000.
GAO Comments
1. Our report stated that the computer licensing level for tier 3 countries
was raised to 28,000 MTOPS, effective immediately. Commerce’s letter
refers to the 28,000 MTOPS threshold at which advanced notifications
to Commerce of computer exports would be required. According to the
President’s August 30, 2000, letter transmitting his report justifying
changes to computer export control thresholds, “the new level above
which an individual license will be required for exports to tier 3
countries is 28,000 MTOPS. The aforementioned licensing adjustments
will take place immediately.” Furthermore, an October 13, 2000, rule
amending the Export Administration Regulations stated that the upper
licensing level for computer tier 3 countries was raised from 20,000 to
28,000 MTOPS. In comparison, the President reported in the same
August 30, 2000, letter transmitting his report that he was establishing a
new notification level of 28,000 MTOPS which cannot take effect until
180 days after the Congress receives the President’s Report, that is,
February 2001. Section 1211 (a) of the National Defense Authorization
Act for Fiscal Year 1998 (P. L. 105-85) requires exporters to notify the
Commerce Department of any proposed high performance computer
exports to countries that pose a concern for military or proliferation
reasons to determine if these exports need a license. Also, the
October 13, 2000, rule amending the Export Administration Regulations
stated that the level for advance notification of high performance
computer exports to tier 3 countries was raised from 12,500 MTOPS to
28,000 MTOPS, effective on February 26, 2001.
2. We have clarified the sentences related to the 1998 Defense- and
Commerce-sponsored study in response to Commerce’s suggestions by
adding language to stress that “DOD officials responsible for the
sections of the report on military uses of high performance computers
told us that they relied on the 1998 Department of Defense- and
Commerce-sponsored study for support in the President’s reports.”
(711480)
Leter
Page 55
GAO-01-10 Export Controls
Ordering Information
The first copy of each GAO report is free. Additional copies of
reports are $2 each. A check or money order should be made out to
the Superintendent of Documents. VISA and MasterCard credit
cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.
Orders by mail:
U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013
Orders by visiting:
Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC
Orders by phone:
(202) 512-6000
fax: (202) 512-6061
TDD (202) 512-2537
Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.
Orders by Internet:
For information on how to access GAO reports on the Internet,
send an e-mail message with “info” in the body to:
info@www.gao.gov
or visit GAO’s World Wide Web home page at:
http://www.gao.gov
To Report Fraud,
Waste, or Abuse in
Federal Programs
Contact one:
• Web site: http://www.gao.gov/fraudnet/fraudnet.htm
• e-mail: fraudnet@gao.gov
• 1-800-424-5454 (automated answering system)
United States
General Accounting Office
Washington, D.C. 20548-0001
Official Business
Penalty for Private Use $300
Address Correction Requested
Bulk Rate
Postage & Fees Paid
GAO
Permit No. GI00