Conference Session B5
Paper #2197
CLAYTRONICS: A NEW INTERACTIVE MEDIUM
Michael Poporad (mgp13@pitt.edu), Adam Flemming (ajf70@pitt.edu)
Abstract – Claytronics, which is programmable matter,
will bridge the gap between the intangible world of
computer generation and the physical world we live in
through its use as an interactive medium. Claytronics
uses ensembles of microscopic programmable robots
called catoms to create physical entities in the same
manner that atoms and molecules make up matter.
Claytronics will enable users to create a fully responsive
duplicate of their physical surroundings in another
location. This Claytronic technology spawns a new
information medium called “pario”, which is Latin for “I
Create” [1]. Pario will revolutionize the world of
communication enabling objects thousands of miles apart
to interact on a physical level. Researchers are in the
process of manipulating principles currently applied to
modular robots in order to apply them to these
microscopic ensembles of robots, which will become the
backbone of Claytronics. In this paper, the concept of
Claytronics will be explored through four major areas:
general background information, possibilities of using
this technology in the world of communication, the ethics
of using this new technology, and the societal significance
once this technology has been fully developed.
Claytronics. These catoms are identical to each other in
every way, a necessary feature for the project to work
successfully. This ensures each particle can be fungible
for its neighbor, never limiting the extent to which the
matter can be manipulated. This will allow for
constructions of any size and orientation. The exact
mechanism of action is still being developed, and is in the
infantile stages of development. Seth Goldstein, the father
of Claytronics, envisions the catoms to be on a
microscopic scale, but they are roughly 44 mm in size in
their present state [2]. (Programmable Matter)
HISTORY
Since the early days of computers in which units with
limited functionality, in relation to today’s standards,
required entire rooms to house their components,
manufacturers have made steady progress in micronizing
computer components, increasing their efficiency and
potential for computational applications. This
development in nanoscale manufacturing has allowed us
as a society to create small scale computer devices with
abilities that far exceeded our expectations from even as
little as ten years ago. Even though this simultaneous
reduction in size and increase in efficiency have enabled
us to create revolutionary devices, the main focus has
always been to simply increase computational memory
and logic ability. The last 50 years have offered gigantic
leaps and bounds in the capabilities of technology like the
central processing unit and the hard drive, but what Seth
Goldstein envisions in his Claytronics project is a system
that has no unique piece. He has named this theory on
computer processing the “Ensemble Principle”, which he
officially defines by stating that: “a robot module should
include only enough functionality to contribute to the
ensemble’s desired functionality” [2]. (Programmable
Matter) Researchers like Goldstein interested in this
concept are currently applying developments previously
made for CPU based computers to nanoscale
manufacturing in order to create microelectromechanical
systems made of identical modules. The aim is to be able
to “inexpensively produce millimeter-scale units that
integrate computing, sensing, actuation, and locomotion
mechanisms. A collection of such units can be viewed as
a form of programmable matter” [2]. (Programmable
Matter) This idea of programmable matter is
revolutionary in concept, utilizing computer processing in
numbers to accomplish more than any specialized unit
could. Until now, robots, and computers in general, have
had serious limitations in capability due to the limitations
Key Words – Catoms, Claytronics, Modular Robots,
Nanotechnology, Pario, Programmable matter,
Smart dust.
BACKGROUND
Claytronics is the use of millions of programmable robots
the size of small particles in order to create a solid
material that is capable of being manipulated by a
computer program. Through the use of electromagnetic
fields to power each individual particle called Claytronic
atoms or “catoms”, and the use of a wireless receiver in
order to communicate with the base computer, the catoms
will be controlled through a computer GUI that acts as the
command center, much like a computer tower controls the
output to the speakers and monitor. This is fitting,
because with the advent of Claytronics, a whole new form
of media has to be recognized, pario [1].(Catom and
Eve) Just as computers allow us to hear audio from a
speaker or see video displayed on a monitor, this
command center will use catoms to create tangible
representations of data that the command center processes.
However, the crucial difference between speakers and
catoms is that speakers have specific pieces with special
functions, but that’s the opposite of the intent of
Twelfth Annual Freshman Conference
1
February 13, 2012
Conference Session B5
Paper #2197
of the parts that comprised them. A computer could never
merely gain more available memory, or expand its hard
drive; any change in the system would require whole new
pieces, and even then, those new pieces had limitations of
their own. It is because of these limitations on the CPU
system that programmable matter is the next step in the
progression of computer technology.
catoms, will have to meet in order to achieve their full
potential as a “programmable matter”. Specifically, four
criteria points must be met regarding the anatomy and
characteristics of the catoms that will make up this
“matter”. Firstly, each catom should be a self-contained
unit, having all of the necessary on board communication
equipment and sensors for functioning. Secondly, the
control system for the catom should, for the most part, be
self-contained as well, requiring little to no external
computation. Thirdly, if a catom should form a bond to
another catom, the bond should not require static power in
order to be maintained, otherwise a large draw of power
would always be present on a system of millions of
catoms staying connected. Finally, there can be no
moving parts inside of the catom, because the point of
catoms is to become the moving parts required
[3].(Anatomy based organization)
Based on these four critical requirements, two catom
prototypes have been proposed; a cylindrical model and a
spherical model. Fig. 3 is a diagram of the spherical
catom model, which uses electrostatic forces to generate
torque in order to manipulate orientation.
PRIMITIVE MODULAR ROBOTIC SYSTEMS
Before Claytronics was conceived a similar idea existed
in what is called modular robotics. The idea behind
modular robots is the cooperation of multiple robotic units
to form one cohesive unit. Although this may sounds
exactly the same as Claytronics it is different in that each
unit is not relatively small and there are only a limited
number cooperating as opposed to thousands or even
millions in Claytronics. Fig. 1 is an illustration of the
anatomy of single unit of the ATRON self-reconfigurable
robotic system. A group of up to one hundred of these
modules can self-reconfigure into a variety of formations
and perform specific tasks. However the motor driven
actuation and anatomical features such as the male and
female connectors make this unit a poor candidate for a
model of the small scale Claytronic atom. While
technically able to connect to several other modules
simultaneously, at the scale required to mass-produce
millions of modules cost effectively, the actuators in
charge of things like movement and connection simply
are not feasible [3].(Anatomy based Organization)
Fig. 3. The electrostatic catom model we use for our analysis assumes
insulated plates positioned near the surfaces of spherical modules. When
charged, the plates generate a torque around the point of contact
(Anatomy Based Organization).
Unfortunately, this spherical setup is entirely theoretical
at the moment. While drawings and designs exist for this
system, no physical models or mock-ups have been
created. While it may not be in practice, this system has
several benefits it would be able to offer if utilized in a
multi-million-catom unit, such as full three-dimensional
rotation and a sub dermal latch system without magnets or
hooks to get in the way. The spherical geometry makes
creating shapes out of units of catoms much easier, and
trying to display different colors with the outer skin
would be much more difficult any kind of latch system
breaching the surface of the silicon skin. The only
problem with this design is our current technology just
isn’t there yet, and the smooth rolling electromagnetic
spheres are too advanced without further research and
development.
The cylindrical model of the catom also uses electrostatic
forces for actuation. Large-scale cylindrical catom
prototypes are pictured below in Fig. 3.
Fig. 1. A single ATRON module: on the top hemisphere the two
male connectors are extended, on the bottom hemisphere they
are contracted.
ANATOMY OF THE CATOM
In Chrisansen’s “Anatomy-based organization of
morphology and control in self-reconfigurable modular
robot”, she goes into great, in-depth detail of the
requirements that Claytronics, and more specifically
Twelfth Annual Freshman Conference
2
February 13, 2012
Conference Session B5
Paper #2197
tiny catoms researches had to rethink current methods of
motion planning currently applied to modular robots.
Using current methods if one of the catoms fails then the
entire ensemble would not reach its desired orientation.
Researchers are implementing a method they call hole
motion in order to orchestrate large-scale movement. Hole
motion can be thought of as someone putting air into a
balloon. Instead of programming these catoms to organize
themselves in a hierarchical order to achieve a desired
orientation they will be programmed to create negative
space within the ensemble, which will ultimately result in
the desired orientation. For those familiar with biology it
is comparable to the way a unicellular organism engulfs
its food. Fig. 4 is a diagram of this hole motion process.
This ensures that even if one of the catoms malfunctions
the ensemble will still be able to assemble in the
appropriate way [2]. (Programmable matter)
Fig. 3. Claytronic atom prototypes. Each 44mm-diameter catom
is equipped with 24 electromagnets arranged in a pair of stacked
rings (Programmable matter).
A series of rings comprised of electromagnets
encompasses the catom cylinder, and provides 360-degree
point of contact for other catoms to latch to. While the
sphere provided attachment points from any point on its
surface, the cylinder only has those rings of magnets on
its side to latch to. While this certainly makes crafting
shapes much harder with the catoms, the design is much
more straightforward, making the cylindrical system a
much friendlier platform for creating working prototypes.
At the moment, the catoms require constant power to
continue latching together no matter which system would
be used, and the power drain from static power would be
devastating from a system of just thousands, let alone the
millions Goldstein envisions. That’s not all, the size at
which the catoms are currently operating, the 44 mm scale,
makes placing batteries for operation feasible, and some
of the energy requirements such as dispensing energy to
move versus gravity would scale down alongside a
decrease in size of the catom. Other factors, such as the
draw of the computational and communications systems,
would see an almost imperceptible difference due to the
new length of wiring, thereby remaining practically the
same. This means that there is a point where the size and
mass of battery required to operate the catom would far
exceed the size and mass of the catom itself, rendering the
possibility of an onboard power unit to be zero. Using a
connection through docking ports to transfer energy
between the catoms would lead to complicated
reconstruction and shape changes. This was the problem
with the previously mentioned ATRONs. Goldstein is not
definitive on exactly how they’re going to work around
this obstacle, but he does write: “we’re developing
methods for routing energy from an external source to all
catoms in an ensemble. For example, an ensemble could
tap an environmental power source, such as a special table
with positive and negative electrodes, and route that
power internally using catom-to- catom connections” [2].
(Programmable Matter)
Fig. 4. Hole Motion. Edges can (a) Contract by consuming a
hole, or (b) expand by creating a hole, purely under local control.
(Programmable matter)
The above motion describes the process for the future of
Claytronics, when ensembles are made of thousands of
catoms and are at a much smaller size than they are at
present. For now, researchers are looking into octagonal
chain arrangements to design the two-dimensional motion
paths for groups of catoms. These octagonal chains start
perfectly round, and have the ability of rotating around
each other to expand or contract. While Fig. 5 is using
servos fitted with gears to illustrate the point, the catoms
current magnetic rings share the same capabilities.
ORGANIZING A LARGE POPULATION
In order to control a population of potentially millions of
Twelfth Annual Freshman Conference
3
February 13, 2012
Conference Session B5
Paper #2197
working simulation platform that is sufficiently featurerich to write initial prototype applications for Claytronics”
[4].(Simulating Multi million) The user of the simulator
inputs the parameters of the experiment he or she is
conducting or, in other words, the orientation that he or
she aims to achieve, and the world state or the starting
position of the catoms. Fig. 6 gives an idea as to the level
of control researchers have over the environment they’re
working in. Using DPRSim, researchers have the
capability to test run assembly algorithms, instructional
code, or hardware without ever coming in contact with a
single catom.
(Simulating Multi million)
Fig. 5. One instance of a collective actuation system, consisting
of two octagonal cells. Note that the physical prototype (bottom)
includes four additional modules on each end to simplify the
servomotor mounting arrangements.
CLAYTRONIC SIMULATORS
In order to utilize large-scale modular-robotic systems as
in the case of Claytronics software to control these large
ensembles needs to be developed. Two big challenges that
researches face in completing this task are: one these
catoms do not yet exist in their final refined form to work
with directly, and second Claytronic functionality requires
unique algorithms which cannot be tested on more
primitive modular robotic entities [4]. (Simulating Multi
million)
The solution to these problems is developing a
computational simulator for Claytronic models where
programmers can test various algorithms and parameters.
Developing an adequate simulator for this purpose is a
difficult task in itself because Claytronic models require
the processing of information for millions of individual
catoms simultaneously.
Fig. 6. Screenshot of the original DPRSim simulator for
Claytronics. It incorporates simulation of distributed code
execution, physics, visualization, a world builder, and
interactive debugging support.
Fig. 7 gives an excellent idea as to how highly DPRSim
operates, as that is a replica of Carnegie Mellon’s GatesHillman Complex made from over 1.6 million catoms.
DPRSIM
Due to how distant the realization of Claytronics’ full
potential is, researchers needed to become creative in
order to begin testing theories and ideas they had about
multi-million-catom systems. The question is, without any
physical hardware to work with, how can one create a
system full of millions of catoms to be manipulated? The
obvious answer is to leave the physical hardware out of it
and use a computer simulation, thus DPRSim was born.
DPRSim was the first simulator developed that is
powerful enough to conduct computational Claytronic
experiments. “It is an integrated system that includes code
execution, physics, interactive visualization, debugging
support, and a GUI-based world builder to construct
experiment scenarios. Although intended to be scalable,
the primary goal of its development was to create a
Twelfth Annual Freshman Conference
Fig. 7. DPRSim simulation of a block made from over 1.6
million catoms reassembling itself into an exact replica of
Carnegie Mellon’s Gates-Hillman Complex.
4
February 13, 2012
Conference Session B5
Paper #2197
Since DPRSim a number of more effective simulators
have been developed to conduct computational Claytronic
experiments. Although these simulators vary in
computational power and the algorithms used to construct
the physics, they all serve the same four main purposes:
Bach instruments, and your business partners in Hong
Kong want you to experience the new acoustic feel for the
latest trumpet design. Unfortunately, they can’t ship it to
you for weeks. This is where Claytronics comes into play,
making the need to physically ship a trumpet across the
world disappear. They send a signal, and suddenly the
cube of catoms on the desk begin to take shape, and
suddenly a functioning replica has formed on the desk
where the cube once was, the process shown in Fig. 8.
1) Checking algorithms used when catoms to
interact with other neighboring catoms
2) Computing magnetic forces between catoms
3) Colliding catoms together
4) Computing and tracking catom motion
during disassembly/reassembly process
(Simulating Multi Mil)
Fig. 8. A DPRSim of over 160,000 catoms, turning into a
trumpet.
APPLICATIONS
Education would also see a huge benefit to incorporating
Claytronics into the curriculum, replacing all textbooks
with a cube of catoms. As informative as a paragraph on
the valves of the heart is, an exact replica of the human
heart that can be taken apart, studied, and reconstructed
would provide infinitely more experience to a medical
student. Finally, the world of emergency response would
see the greatest benefit of all through the integrating of
Claytronics into their system. When somebody calls 911,
an ambulance is promptly dispatched, and the police
officer can stay on the line to gather important
information and provide instructions on what to do while
waiting for the rescue team to arrive. However, if
Claytronics were to be involved, then the dispatch officer
would possess not only sound, but visual and physical
information on the problem, and would be able to provide
even more insight as to proper procedure, and perhaps
even send medical supplies in the form of catoms.
Sometimes, patients who are already at the hospital
require a transfer in order to receive care from a better
surgeon, or a specialist is brought into the hospital in dire
circumstances, and while waiting for all of the pieces to
fall in place, the patient’s health is deteriorating. In a
world where Claytronics are the common reality, a doctor
in Wisconsin can be performing heart surgery on a model
of a heart of a patient in New Hampshire, while
Claytronics mimic his moves, interpreting the catoms
movement on the doctor’s side, and transferring that
movement into action on the patient’s side.
Claytronics, as previously stated, has given rise to a
whole new form of media, pario. Just as the telephone
revolutionized the world of communication at its
inception, and television followed suit, so shall
Claytronics change how we as people connect over all
over the globe. As it is, we can already both see and hear
people on the other side of the world, but imagine a world
where we can hand them the latest mock-up of the
product we’re selling them, or a detailed model of the
house they’re looking to rent. Pario is going to bring these
imaginations, and more, to reality, and it’s going to use
Claytronics to do it. The nature of the catom system is to
have the ability to become anything that can be
programmed into a computer, which entails everything
from a raven to a writing desk, and everything in between.
As it stands today, the project is too far away from
anything remotely resembling shape construction, and the
catoms are far to large to feasibly manipulate into any sort
of ensemble. However, when the technology reaches a
point where it is not only physically possible but also cost
effective to produce several million catoms, the benefits
of such technology could be reaped in any category of life,
not just communication, but communication would be one
of the more impacted facets of the human condition.
Communication is a very broad term, and to narrow it
down, there are a few types of communication in which
Claytronics would flourish, from the mundane to the
fantastic, such as business, education, and emergency
response. In the business world, meetings are pivotal to
the success of companies, otherwise, confusion sets in and
people get lost in the chaos. Sometimes, a video
conference or a phone call is enough to keep everyone on
board, but other times, face to face meetings are required
to look over some new model or examine in fine detail a
new product. With Claytronics, those face to face
meetings are no longer necessary due to the ability for a
set of catoms to assemble itself into that model or product
even though the person receiving the product is several
hundred miles away. Imagine sitting in a boardroom at
Twelfth Annual Freshman Conference
ETHICAL ISSUES
Because Claytronics is still in its very early stages
consideration of ethical issues really takes the back seat
on researcher’s priority lists. Before ethical issues can
really be evaluated we must first discover what the
potential of Claytronics really is. However based on the
current progress of research and the proposed future
5
February 13, 2012
Conference Session B5
Paper #2197
applications we have come up with potential ethical issues
regarding Claytronics. The first major issue we foresaw
was safety. Unlike holographic representations of
physical objects, Claytronic models will have mass and be
able to interact with their environment. Let’s say that two
people are beginning a pario session and while the catoms
are assembling something goes wrong with the signal
being sent to them. Similar to the way cancer cells divide
uncontrollably, these catoms could go haywire and begin
forming objects with no regard for their surrounds causing
them to displace whatever real objects are in their path.
This could cause serious danger for anyone in the
immediate vicinity of the Claytronic model.
Another major ethical issue regarding
Claytronics is the same issue that arose with automated
manufacturing. Pario enabled by Claytronics could slow
the growth of the job market in skilled trades. The reason
being that since pario will enable someone to essentially
be in two places at once, or at least have two functional
physical representations in multiple places, the demand
for multiple skilled professionals who perform the same
task will decrease. Competition in professional realms
will become much higher because the “territory” pario
will allow them to practice over will be much greater than
what is currently available. Why have ten different
averagely skilled doctors at ten different hospitals when
you can have one extremely skilled doctor cover the same
number of hospitals within the course of a day?
Its all about cost. While today the cost of
having millions, billions even, of catoms would be
outlandish if not downright unholy, that does not mean
that future research and development could bring such a
staggering ensemble to fruition. At some point in the
future, catoms may be so cheap to manufacture that they
become commonplace, as is Goldstein’s goal. When this
happens, what becomes of every day items’ inherent
values? While this issue may seem frivolous on the
surface, there should be real concern for what happens to
pricing when any item requiring human effort to construct
can now simply be crafted from catoms. It may seem
drastic to think this way, but value in society is
determined through pros and cons, cost and benefit; when
the cost to manufacture becomes more than the benefit of
being convenient, no one is going to manufacture goods
to his or her own detriment.
Claytronics. However, speculation on the uses of
Claytronics is just that, speculation; the science just is not
there yet, and the research is years or even decades away
from achieving a working prototype.
REFERENCES
None
ADDITIONAL RESOURCES
[1] F. Markus. (2009, July). “Catom & Eve: Behold The
Genesis Of Pario, Claytronics, and Synthetic reality.”
Motor Trend. [Online]. Available:
http://go.galegroup.com/ps/i.do?id=GALE%7CA2020263
87&v=2.1&u=upitt_main&it=r&p=AONE7sw=w
[2] S.C. Goldstein, J.D. Campbell, T.C. Mowry.
“Programmable Matter.” Computer Magazine, Vol 38,
Issue 6, pp. 99-101.
[3] D.J. Christansen, J. Campbell, K. Stoy. (2010, June
13th). “Anatomy-based organization of morphology and
control in self-reconfigurable modular robot.” Neural
Computing & Applications. Volume 19, Number 6.
[Online]. Available:
http://www.springerlink.com/content/d1n77053335u5114/
[4] M. Ashley-Rollman, P. Padmanabhan, M. Goodstein.
(2011, May 9th). “Simulating Multi-Million-Robot
Ensembles.” 2011 IEEE International Conference on
Robotics and Automation. [Online]. Available:
http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber
=5979807
[5]Michael P. Ashley-Rollman, Jason D. Campbell,
Michael de Rosa, Stanislav Funiak, Seth C. Goldstein,
James F. Hoburg, et al. “Beyond audio and video: using
claytronics to enable pario.” AI Magazine, Vol 30, Issue
2. (Summer 2009) p29.
REALISTIC FUTURE EXPECTATIONS
The future of Claytronics will work its way into society
once the science has reached the working stages of
prototyping. Realistically, almost every facet of life is
going to be affected in some way or another. As discussed,
communication is one of those areas, and any area where
cheap custom reproductions or replicas would come in
handy would see dramatic benefits from the world of
Twelfth Annual Freshman Conference
[6]S. Upson. (2011, April 16th). “Go Reconfigure.” New
Scientist. Volume 210, Issue 2808. [Online]. Available:
http://web.ebscohost.com/ehost/detail?sid=915092160b17-46c1-b338-
6
February 13, 2012
Conference Session B5
Paper #2197
8f30f85475e7%40sessionmgr11&vid=1&hid=9&bdata=J
nNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#anchor=AN00610
22146-3&db=aph&AN=61022146
[7]J. Campbell, P. Pillai. “Collective Actuation.” The
International Journal of Robotics Research 2008, Vol 27,
pp 299-314. DOI: 10.1177/0278364907085561.
http://ijr.sagepub.com/content/27/3-4/299
[8]B. Agarwal. (2010, March). “Inguinal hernia repairChallenges beyond zero recurrence.” Saudi Journal of
Gastroenterology. [Online]. Available:
http://go.galegroup.com/ps/aboutJournal.do?pubDate=120
10010&actionString=DO_DISPLAY_ABOUT_PAGE&i
nPS=true&prodId=AONE&userGroupName=upitt_main
&searchType=&docId=Gale%7C5PYG>
[9]K. Smith, S.C. Golstein (2011, April 12 th).
“Programmable Matter: Applications for Gastrointestinal
Endoscopy and Surgery.” Official Journal Of The AGA
Institute. Volume 140, Issue 7. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/S001650
8511004707
[10]Y. Yen. (2007, May). “Forget Nanotech. Think
Claytronics.” Business 2.0. Volume 8, Issue 4. [Online].
Available:
http://web.ebscohost.com/ehost/detail?sid=c2688e600ccd-42d6-948bf80730a7e562%40sessionmgr15&vid=1&hid=9&bdata=J
nNpdGU9ZWhvc3QtbGl2ZQ%3d%3d#anchor=AN00250
10584-4&db=bth&AN=25010584
ACKNOWLEDGMENTS
We would like to thank Seth Goldstein of Carnegie
Mellon for introducing us to the topic of Claytronics.
Twelfth Annual Freshman Conference
7
February 13, 2012