Claytronics—Objects of Imagination

A dozen people walk into a boardroom. They settle into their seats waiting for a meeting to begin, and before long the conversation turns to the subject of weather. Bob from Phoenix motions to the rain pelting the windows of the boardroom then shakes his bag and suit coat as if to keep his belongings dry. Several chuckle at his joke. "I should have brought an umbrella," says Bob.

Although the storm is raging outside, Bob, Mary from St. Louis, Carol from Baltimore, and everyone else in the group, is completely dry—and not one of them brought an umbrella today for the meeting in this rainy city. In fact, they didn't need one. Why not? Because Bob is physically in Phoenix, Mary in St. Louis, Carol in Baltimore—each person is in a city where today's forecast is clear skies.

Unquestionably, this scenario shows that what is coming in the future is very different from today's business meetings. When a dozen men and women walk into a meeting of tomorrow, it may seem completely ordinary that there is only a convincing model of each person in the room—sort of a clone that each has sent in their place. And this same technology isn't limited to meetings by proxy. Any object or thing could have a copy made of it. What's more, practically any model of a person or thing could become bigger or smaller. For instance, a veteran heart surgeon, whose hands aren't as steady at age 60, could create a model heart that would grow to a more workable size. The surgeon would then "operate" on the model, and each movement of the scalpel would transmit to a robot inside a patient's actual heart, a robot that would perform the life saving procedure with a more delicate touch.

Affecting the business world, health care, and the culture of nearly everything we know, this technology is centered on the control and manipulation of three-dimensional objects—in other words, a programmable form of matter. One instance of this technology is known as Claytronics; the 3D objects that are manipulated, tiny robots, are known as catoms (Claytronic atoms).

Rest assured—after digesting this new vernacular of the future, it will work like the best of science fiction.

Video of an actual person—from not one, but all points of view—streams into a distant room where the data is fed into an "ensemble" of catoms, the building blocks of any model. The ensemble stirs into action as scores of these tiny robots, each the size of a grain of sand, morph harmoniously into a three-dimensional copy of the same person whose video image tells them what to look like, be it Mary, Carol, or the new guy from accounting. The skin tone, the voice inflection, the actual lighting and shadows on whomever the catoms are modeled after—any physical trait that convinces the brain it's looking at a human being—would be rendered in the 3D model, even the weather conditions of the person's locale. This means that Bob could be out in the Phoenix sun, and the Claytronic model of him in the meeting would be lit up with the suns rays (even in the rainy city!).

Undeniably, this vision requires great imagination and creativity. Add a great deal of optimism to the mix and that describes Seth Goldstein, associate professor of computer science at Carnegie Mellon, and Todd Mowry, Director of the Intel Research Lab-Pittsburgh, whose shared brainchild is the two-year-old Claytronics project.

"There are all kinds of challenges: friction, energy transfer, heat management," says Goldstein. "There are all kinds of things that are going to be hard, but you can look at it and think: This really looks feasible. We can have millimeter scale catoms in a couple of years."
The vision of a future business meeting appeals to Mowry, also an associate professor of computer science at Carnegie Mellon. He says he was inspired to start the Claytronics project with Goldstein for the sake of creating "all the benefits of face-to-face interaction without traveling anywhere."

How exactly these interactions will unfold, however, is cause for some designers of the Claytronics project to stop and think. Imagining again the scenario of a dozen people walking into a meeting, what if, in fact, there was only one who wasn't formed out of catoms—indeed, was the only person in the room who was literally there? How strange would it be for this person to conduct business with human look-alikes? And how about watching a co-worker take shape out of two million micro robots?

"It would be really weird to be in a room that's oozing around and changing shape," says Seth Goldstein, "Would we get used to it? It seems like it would be weird if a co-worker was melting and reforming."

His 30-year vision is to see the shape composed by catoms—called a claytron—look convincing enough to appear human. To determine if a clayton makes the cut, Goldstein will use a standard similar in spirit to the Turing test, a protocol designed by British mathematician, Alan Turing, to gauge the humanness of a machine.

He says, "You won't be able to tell by sitting over there, whether or not I'm sitting next to you or sitting somewhere else and being rendered here in Claytronics. The particles are going to be small enough that we can get the hair, and they'll be robust enough that they can move around dynamically in 3D."

The idea is that when shaking a claytron's hand it would both appear human and be convincing enough to "feel" human. Since there is a finite size to what we can feel, if claytrons can be made that small, Goldstein adds, "with the proper movement it should feel like anything, fur or cement, or anything."

But the mind-blowing features of Claytronics aren't restricted to just making duplicates of a person or an object that feels real. Todd Mowry has pondered over the dilemma of making a cell phone keypad, "big enough to do any real work," and has determined that a claytronic cell phone that doubles as a laptop could fit the bill. "Because a device must be a certain size," says Mowry, "your cell phone could grow to the size of a laptop where you could work on a project. And then once you're done, it would shrink back into the cell phone, which fits inside your pocket." And with a wave of a wand, a new concept in consumerism is born: resizable electronics.

Mowry adds, "Audio and video are great, but we don't actually live in a two-dimensional world of sound and moving images. We live in a 3D world. I think our imaginations are really limited by the constraints of audio and video. We can do so much more if we can start manipulating physical things."
 

Current Limitations

But before anyone holds proxy meetings, drops in for a surprise birthday party, or displays a replica of a Ming dynasty vase in their living room, Carnegie Mellon researchers, in collaboration with the Intel Research Lab-Pittsburgh, have several key issues to resolve in controlling the behavior of each catom.

Produced under the mantra of "scale up in numbers and down in size," a catom will require a bevy of features, especially since an estimated two million micron-scale robots will be required to make a claytron appear convincingly human. A CPU, a power store, a network device, video output devices, sensors, a means to move around, a means of sticking to other catoms—each of these features must successfully be implemented to ensure that Claytronics emerges someday from the lab. 

For the time being, however, Goldstein stresses, "We're really at the nitty grittry here." He says they are working to "write a program that gets them to assemble into a cube, let alone a dynamic human." Before catoms coalesce into a claytron, Goldstein and others "want to get four catoms to roll on top of each other to look like a pyramid."

But some discoveries have come much sooner than expected—like the solution for how to move individual catoms around. Achieving their coordination, says Goldstein, will not include the use of magnets, as was once thought, because each magnet would make a robot too heavy. Instead, "our hunch is that we will have to switch to electrostatics." Using this approach, researchers have built large prototype catoms filled with helium, moving them around with static electricity. "It's kind of counterintuitive to experiment with larger models, but if we fill them with helium, they have properties that are closer to the tiny catoms," explains Goldstein.


Sharing Information Across Disciplines

"In some ways this project is a matter of community building instead of creating new science," says Goldstein, who describes the Claytronics group of researchers as highly multi-disciplinary.

For instance, "There are people in ECE [Electrical and Computer Engineering] who are working on power transfer and adhesion, people in the Computer Science Department working on things as diverse as networking and programming theory, and people in robotics working on manipulation and grasping," says Goldstein. Including the participation of Intel in research and training students at Carnegie Mellon, there is indeed a mixed crew involved with this project.

"One of the goals of this Claytronics project was to have this 20-year, 'Wow that's a really cool vision,' that allows people to do their research—research they would want to do anyway—but frame it in this thing that really gets people excited," says Goldstein. He says it's been enjoyable to participate with "people from completely different areas who really care about what they're doing."

Research done in one field will influence the research done in another. Specific details about catoms, like adhesiveness, affect many other aspects of the project, for instance. "It matters when you're programming what kind of strength they'll have," he says. "And vice-versa. The programming method affects the kinds of hardware mechanisms needed."

Besides ensuring the progress of Claytronics through interdisciplinary means, another component, says Goldstein, is to keep the community stretched with intriguing challenges along the way. Discoveries made at each step will have a practical application to countless other uses that might only be nominally related to the goals of Claytronics. Goldstein speculates, "if we can program a million catoms in simulation, and work out how to debug them, that's going to really help people looking to program 64 different things simultaneously."

As major breakthroughs arise at the very edge of discovery, researchers expect to yield derivative inventions that have a more immediate application to the world, inventions like consumer products—cordless power tools, sports medicine—spun off from the Apollo program in the 1960s.

Ever the visionary, Goldstein adds, "I suspect there are going to be a lot of things like this that will have an impact now even though we're working on this broad 20-year vision."


What the Future Will Bring

Seth Goldstein suspects he will "be unhappy" if, five years from now, Claytronic technology hasn't progressed to the stage where a couple hundred of these units of the same size as today haven't started to work together to form shapes, albeit limited ones. He says they may be expensive, slow, and only work in the lab, but if someone draws a circle on a screen, the catoms should form into the shape.

"I think that's a real reasonable goal," he adds, only to follow with, "I should say that most of the principal investigators don't believe that much can be done in five years, and they're probably right. But I'm an optimist."

As dreamers will, Goldstein and Mowry hold this challenge beyond the reach of their team, urging them to strive for the grand vision of Claytronics—to the day when we're living the impossible: a daughter receives a pony made of catoms, a sofa one night turns into a love seat the next, a flawless Van Gogh hangs on the wall of every home in America, doctors make high-tech house calls to patients living on the opposite side of the country…and, and, and?


--John Worlton