Supercomputing Gets A New Superhero

Just five years ago, European scientists needing supercomputer time faced a fragmented and frustrating system. A scientist in one country might have access to just that country’s supercomputer, which might well be obsolescent or obsolete.

As a result, many European scientists could not pursue cutting-edge research in computationally demanding areas, such as cosmology, climatology, or nuclear fusion.

Europe’s supercomputing landscape is completely different today, thanks to DEISA (Distributed European Infrastructure for Supercomputing Applications), an EU-funded project that has linked a dozen of the world’s fastest supercomputers into one smoothly functioning transcontinental grid.

“Cross-national communication was extremely difficult, and the rules and behaviour for accessing each supercomputer were all different,” says Hermann Lederer, in charge of DEISA external relations. “But now, through DEISA, a scientist in Italy can access a new supercomputer, say, in the Netherlands, and in addition have a more comfortable way of addressing it.”

DEISA is one of just two top-level supercomputing grids in the world, the other being TERAGRID, in the USA. “In terms of functioning infrastructures, there is not yet a third supercomputing network at the same level as TERAGRID and DEISA,” says Lederer.

Building the infrastructure

The first step towards DEISA involved linking participating computer centres via a dedicated high-speed fibre-optic network. DEISA’s links can flash 10 gigabits of data per second from one centre to another.

In addition, DEISA’s grid was designed to route data in several streams at the same time, multiplying the network’s speed.

As an added bonus, the dedicated grid provides a secure computing environment. “All DEISA supercomputers are trusted hosts,” says Lederer. Security is important since supercomputers can be used for illicit purposes, such as designing nuclear weapons.

The high-speed dedicated network allowed DEISA’s IT specialists to design and build a unified global file system for users logging in from any participating centre.

A user’s log-in activates a personal data directory. That allows instant and automatic access to all of the user’s data wherever they may be.

“It can happen that you output your data from a job running on one computer into your data directory, which physically is in another country,” says Lederer. “This also allows scientists to collaborate easily – they’ll find their data at the same place whether they’re in Italy, the UK or Finland.”

DEISA’s designers were determined to unify supercomputing not only physically, but also in terms of the ease and consistency with which researchers could access and use the system.

The goal, says Lederer, was to provide users with the same interface and computing environment for all DEISA supercomputers, whether they are remotely accessing DEISA from one of the cooperating supercomputing centres or from their own laptop.

Making that happen required a great deal of hard work and fine-tuning, but the result – the DEISA Common Production Environment – has proven its usefulness.

“When a scientist is looking at whatever computer, he gets the same appearance,” says Lederer. “He doesn’t need to ask, ‘Where are my data? Where are the compilers? Where are the applications I need? How can I access them?’”

Fomenting world-class research

It did not take long for DEISA to prove its worth to scientists across Europe. Starting in 2005, DEISA called for research proposals under the DEISA Extreme Computing Initiative (DECI). The response came from leading researchers and research groups in fusion, cosmology, the life sciences, materials science and engineering, and more recently, climatology.

“DECI has become a regular institution at DEISA,” says Lederer. The most recent call, in May of 2008, produced 66 proposals from 15 European countries. The projects that were accepted have already started running.

DEISA has recently taken on a challenging new role. Under a new European initiative, called virtual community support, DEISA is offering the services not just to individual investigators or research groups, but to entire research communities.

The aim is not just to provide computational resources to existing communities, but to stimulate the formation of new cooperative communities of researchers. The European Commission hopes that coordinated access to DEISA can help convince competing research groups of the benefits of collaboration.

Currently, DEISA is working closely with Europe’s best-organised research community, which focuses on nuclear fusion. Fusion theorists are collaborating with DEISA developers to design and run new and more realistic simulations.

An emerging community focuses on understanding the genetics of infectious diseases and their treatment. DEISA is now working closely with that network, spearheaded by the European life science project VIROLAB.

In December 2008, DEISA published a call for communities that are in the process of coming together to submit expressions of interest. “That’s our contribution,” says Lederer, “to start providing support for existing science communities in Europe and to help form new ones.”

Above and beyond the research DEISA has empowered and hopes to stimulate, its most important contribution may well be the extent to which it has prepared the ground for the future development of Europe’s supercomputing capabilities.

Since DEISA started, just four and a half years ago, the aggregated peak computing power it can offer has multiplied by a factor of 300, from 30 teraflops (30 thousand billion floating point operations per second) to over a petaflop (a million billion operations per second).

“From three to five petaflop supercomputers will be installed in Europe by 2010,” says Lederer. “The infrastructure for that has already been designed, built and installed by DEISA. All the services are in place. It’s ready to go.”

DEISA has received funding from the Sixth and Seventh Framework Programmes for research.