Pittsburgh Supercomputing Center Celebrates 25 Years of Cutting-Edge Technology

The Pittsburgh Supercomputing Center (PSC) started from an idea over lunch in 1984.

Carnegie Mellon Physics Professor Michael Levine knew that the National Science Foundation was soliciting proposals for something that didn’t then exist: supercomputing centers to serve U.S. science and engineering research.

“Why don’t we submit a proposal?” he said to his research collaborator and friend across the table.

“Why would they want to give a supercomputer to us?” asked University of Pittsburgh Physics Professor Ralph Roskies.

“Who else should they give it to?”

The two physicists soon teamed with Jim Kasdorf, then director of supercomputing at Westinghouse, now PSC director of special projects, and more than a few sleepless nights later, in the spring of 1985, their proposal flew by special courier to Washington — on the day of the deadline.  After many suspense-filled months, on Jan. 17, 1986, the National Science Board approved funding for the PSC, with Levine and Roskies as co-directors.

Twenty-five years later, PSC is a nationally leading center in providing the best possible computing environment for scientists and engineers doing unclassified research.

Researchers who use PSC resources — numbering in the thousands each year — include government and industry scientists as well as universities.

“We’re a service organization,” Levine said. “We buy supercomputers and make them available along with the expertise required to use them productively.”

Partnership with Westinghouse was a key to PSC’s winning proposal. While administrative offices are at 300 S. Craig St. (having moved from the Mellon Institute Building in 2006), PSC’s supercomputers are at Westinghouse Energy Center in Monroeville and are linked to Oakland by high-speed fiber. Rather than building from scratch a facility for high-performance scientific computers, PSC has been able to leverage Westinghouse’s infrastructure. Westinghouse in 1984 was running two Cray systems, and their experience and credibility reassured skeptics that the Pittsburgh team could operate supercomputers productively for the national research community.

Perhaps even more important to PSC’s proposal and to its staying power as a world-class research center is the partnership between CMU and Pitt.

Forefront of Innovation

Beginning with Levine and Roskies’ lunchtime brainstorm, PSC was one of the first major research efforts — there have been others since — to receive official backing from both CMU and Pitt.

“Affiliation with two great universities,” Levine said, “has certainly helped in attracting talent.”

Currently numbering about 90 scientists and computing and communications professionals, PSC’s staff, Levine said, takes a backseat to no one as an assemblage of skill and experience in supercomputing.

It should be no surprise that PSC participated in breathtaking technological change. It’s built into the idea of a supercomputer, which — by common definition — means the most powerful scientific computer available at a given time, and times change rapidly. The useful life of a supercomputer, Roskies noted, has been three to four years.

“In cost effectiveness, the evolution from 1986 to now is nearly 100,000-fold,” he said. “Our first machine was a CRAY X-MP that cost $18 million dollars. A 2011 laptop is more powerful.”

To bring home the point about cost-effectiveness, he analogizes to cars. A car today of similar quality to one that cost $30,000 in 1986 would be 30 cents. “You wouldn’t pay to park it.”

Beginning with that CRAY X-MP, PSC has implemented 13 systems for research use, most of them “serial number one” — first of their kind released by the vendor. PSC has a reputation for taking the plunge with new technologies and quickly transforming them into productive research tools.

“The advantage,” Levine said, “is the payoff to the scientific community. If you get new machines into use early in their cycle, you can use them longer. We’re experienced and skilled at this shakedown process with new systems. It allows us to have more influence with the vendors for the course of development of the system.”

Latest Acquisition

PSC’s current lead system is Blacklight (an SGI Altix UV1000), acquired in 2010 with a $2.8 million award from the NSF. Featuring 512 eight-core processors (4,096 cores) with 32 terabytes (a trillion bytes) of memory, Blacklight is partitioned into two 16-terabyte systems — the two largest “shared-memory” systems in the world, a significant advantage for many kinds of scientific applications. Shared memory, as opposed to distributed memory, means all of a system’s memory can be accessed from all of its processors, making it — relatively speaking — easy to program and use, and useful for applications that require large amounts of data to be resident in memory.

Early this year, Blacklight became available for research as part of the TeraGrid, the NSF cyberinfrastructure program that links resources at 11 computational centers across the country.

Through funding from the National Institutes of Health, PSC’s biomedical group, the National Resource for Biomedical Supercomputing (NRBSC), carries out a research program of its own and provides outreach and training in supercomputing for biomedical research. With a $2.7 million “grand opportunities” grant from NIH in 2009, NRBSC for the first time made a novel system — named Anton — available to biomedical researchers nationally.

Built and donated to NRBSC for a year by D. E. Shaw Research in New York City, Anton is designed to do only molecular dynamics — a method to simulate the structure and motion of proteins and other biomolecules — and is the world’s most effective system for doing this.

Collaboration and Outreach

Among other ways the PSC serves western Pennsylvania is its networking group that operates and maintains 3ROX (Three Rivers Optical Exchange), a high-speed network hub that connects CMU, Pitt, Penn State, and other universities and public-school districts to research and education networks, such as Internet2 and National LambdaRail, that link universities, corporations and research agencies nationally.

In 2010, 3ROX joined with a coalition of Pennsylvania organizations to form the Pennsylvania Research and Education Network (PennREN), and PSC staff led the PennREN effort that won $100 million in federal stimulus money to build a broadband network across Pennsylvania.

PSC also has an active program in educational outreach. Through CAST (Computation and Science for Teachers), PSC staff members have trained many southwest Pennsylvania science and math high-school teachers in easy-to-use modeling and simulation tools that bring “cool” technology into the classroom. With CMIST (Computational Modules in Science Teaching), NRBSC provides multi-disciplinary teaching materials — lecture slides, animations and lesson plans — as ready-to-use online modules and DVDs. The initial module, “Molecular Transport in Cells,” presents principles of osmosis and diffusion. A second module, “Big Numbers in Small Spaces: Simulating Atoms, Molecules, and Brownian Motion,” premiered in 2009, and a third, “Enzyme Structure and Function,” came out last year.

Another PSC outreach program, BEST (Better Educators of Science for Tomorrow), introduces high-school teachers to a bioinformatics curriculum. BEST provides ready-to-use lesson plans for single-subject trained educators to teach bioinformatics and has become a course in several southwest Pennsyl-vania high schools.

In 2010, the PSC introduced SAFE-Net, funded by an NSF grant for Cyber Safety Awareness. In this program, PSC provides learning materials that address cyber threats, measures of protection, and questions of cyber ethics that arise from social networking and other uses of the Internet.

Signs of Success

To Levine and Roskies, the measure of success that counts most is scientific accomplishment. PSC resources have enabled thousands of published papers.

Along with many CMU projects, the PSC has contributed to heart modeling that led to a practical prosthetic valve, protein simulations that were cited in
the 2003 Nobel Prize for Chemistry, and storm forecast modeling that for the first time successfully predicted precise location and structure of a severe thunderstorm six hours in advance.

Industrial applications include beverage can modeling by ALCOA and quantum simulation of photochromic technology for sunglasses for PPG Industries.

During the H1N1 outbreak, epidemiological modeling at the PSC supported decision makers in Allegheny County and in Washington. Recently, PSC scientists co-authored a paper in “Nature,” the prestigious international science journal, that for the first time presented a wiring diagram for a portion of the brain.

“We’re pleased and lucky,” Roskies said, “to be, in a sense, voyeurs — to be able to see wonderful and important scientific accomplishments made possible and happening because the technology is progressing at an astounding pace. To be a computational scientist is to live in very interesting times.”

Mike Levine (left) & Ralph Roskies, PSC co-directors