Featured Research

from universities, journals, and other organizations

San Diego Supercomputer Center Reaches Data Transfer Milestone

Date:
November 29, 2002
Source:
San Diego Supercomputer Center
Summary:
Movies and popular music are moving from tape to disk, but tape is still on a roll at the San Diego Supercomputer Center (SDSC) at the University of California, San Diego. The center’s huge, updated tape storage system has illustrated its effectiveness by transferring data at 828 megabytes per second. "Nobody in academia has such a fast data archive system," said Phil Andrews, program director for High-End Computing at SDSC.

Movies and popular music are moving from tape to disk, but tape is still on a roll at the San Diego Supercomputer Center (SDSC) at the University of California, San Diego. The center’s huge, updated tape storage system has illustrated its effectiveness by transferring data at 828 megabytes per second. "Nobody in academia has such a fast data archive system," said Phil Andrews, program director for High-End Computing at SDSC.

Related Articles


Supercomputers are ranked by their raw computational speed, but rapid movement of increasingly large data sets has become vital to researchers in scientific disciplines from anatomy and astronomy, to climatology and particle physics. Data sets in those fields have grown over the past decade, by multiples of 1,000, from megabytes (millions of bytes) to gigabytes, to terabytes. Some data sets are now approaching or exceeding one petabyte (one quadrillion bytes).

Even though tape costs a fraction of disk, supercomputer centers have been investing heavily in more expensive disk storage because its data-transfer rates are higher, by a factor of at least 10, than what had been the best tape systems. At the same time, however, supercomputer centers are struggling to pay for the disk capacity required to keep up with the explosive growth of scientific data.

"While we are expanding our disk storage capacity, we’re also adding new higher-density tapes, faster tape drives and more of them, and other technology in a highly tuned, extremely capable data management system," said Andrews. "It’s all these pieces working together that has allowed us to reach a new milestone in data-transfer speed."

New technology at SDSC involved in the tape-to-disk data transfers includes tape drives from StorageTekฎ (Storage Technology Corp.), switches from Brocade Communications Systems, fibre channel adapters from QLogic, and end-to-end systems and technology from Sun Microsystems. The combined effect the new technology has been a reduction–from days to hours–in the transfer of multi-terabyte data sets from SDSC’s tape-storage system to its IBM Blue Horizon supercomputer, which has a peak speed of 1.7 teraflops (trillion floating point operations per second).

"Our supercomputer center is dealing with more multi-terabyte data sets–such as the 10-terabyte Digital Sky astronomy project–and very fast transfers will allow astronomers to make discoveries faster," said Andrews. "With tape, we have achieved a data-transfer rate that very few people realized was feasible."

No industry association keeps track of data-transfer speed records. "As far as the use of tape drives or multiple tape drives is concerned, the data transfer rates of most academic groups have been well under 100 megabytes per second," said John Marshal, program manager at StorageTek. "The groups that have achieved data transfer rates in the range of 800 megabytes per second have not used tape: they have done it with much larger investments in disk drives, switches, and routers."

SDSC is replacing its 20-gigabyte-capacity tape cartridges with 200-gigabyte native capacity tape cartridges manipulated robotically in five silos with a total storage capacity of six petabytes. The center also installed 24 StorageTek T9940B tape drives, each of which can transfer data to and from tape at roughly 30-megabytes-per-second throughput without data compression. (Compression of data increases its rate of transmission.) SDSC has measured over 60 megabytes-per-second peak speed with the new tape drives with roughly twofold data compression. (The actual measured data-transfer ratedepends on the degree to which data is compressed, the type of software used, and the capacity of the associated hardware involved in the transfer.)

As part of its data-management system, SDSC has also installed Sun Microsystems’ SAM-FS Advanced Storage Management and QFS High Performance Shared File System software to provide maximum scalability, performance, and throughput for the most data-intensive applications. SDSC has also installed Sun’s High Performance Computing Storage Area Network (SAN) storage solution. Such SANs allow multiple computers, using a range of operating systems, to share data seamlessly. The disk capacity attached to SDSC’s SAN will be increased from 50 terabytes to 500 terabytes by 2003.

SDSC’s recent and planned improvements will enable the center to deliver the unprecedented data-management infrastructure required by the National Science Foundation’s $88 million TeraGrid project. The TeraGrid will be deployed in 2003 as the world's largest, fastest, distributed infrastructure for open scientific research. When completed, the TeraGrid will include more than 20 teraflops of distributed computing power, facilities dedicated to managing and storing nearly one petabyte of disk storage and more than six petabytes of tape capacity, high-resolution visualization environments, and toolkits for grid computing. The components of the TeraGrid will be tightly integrated and connected through a network that will operate at 40 gigabits per second–the world’s fastest research network.

The TeraGrid infrastructure will be distributed among five sites: SDSC, the lead site for the National Partnership for Advanced Computational Infrastructure; the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign; the Center for Advanced Computing Research at the California Institute of Technology; Argonne National Laboratory; and the Pittsburgh Supercomputing Center.

A Second Data-Transfer Achievement

In a second data-transfer feat demonstrated Nov. 21 at the Supercomputing 2002 (SC2002) conference at the Baltimore Convention Center, SDSC computer scientist Bryan Banister transferred data from disk at the supercomputer center in La Jolla, CA, to disk at the convention center at 721 megabytes per second.

Banister performed the disk-to-disk data transfer as a demonstration of what will soon be routine–movement of large data sets from SDSC to the other TeraGrid sites over a high-speed network. The cross-country, disk-to-disk transfer made use of a Qwest Communication 10-gigabit-per-second fiber-optic backbone. It also involved Nishan Systems storage switches, Force10 Networks switches, and Juniper Networks routers.

SC2002, this year’s edition of the world’s largest annual high-performance networking and computing conference, is designed to demonstrate supercomputing technology and the breakthrough science it makes possible.


Story Source:

The above story is based on materials provided by San Diego Supercomputer Center. Note: Materials may be edited for content and length.


Cite This Page:

San Diego Supercomputer Center. "San Diego Supercomputer Center Reaches Data Transfer Milestone." ScienceDaily. ScienceDaily, 29 November 2002. <www.sciencedaily.com/releases/2002/11/021126203210.htm>.
San Diego Supercomputer Center. (2002, November 29). San Diego Supercomputer Center Reaches Data Transfer Milestone. ScienceDaily. Retrieved December 18, 2014 from www.sciencedaily.com/releases/2002/11/021126203210.htm
San Diego Supercomputer Center. "San Diego Supercomputer Center Reaches Data Transfer Milestone." ScienceDaily. www.sciencedaily.com/releases/2002/11/021126203210.htm (accessed December 18, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Thursday, December 18, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Jaguar Unveils 360ฐ Virtual Windshield Making Car Pillars Appear Transparent

Jaguar Unveils 360ฐ Virtual Windshield Making Car Pillars Appear Transparent

Buzz60 (Dec. 17, 2014) — Jaguar unveils a virtual 360 degree windshield that may be the most futuristic automotive development yet. Jen Markham explains. Video provided by Buzz60
Powered by NewsLook.com
BlackBerry Launches Classic Smartphone

BlackBerry Launches Classic Smartphone

AP (Dec. 17, 2014) — BlackBerry is returning to its roots with a new smartphone called the Classic, featuring a traditional keyboard at a time when rival Apple and Android phones - and most smartphone customers - have embraced touch screens. (Dec. 17) Video provided by AP
Powered by NewsLook.com
The Future of Work, Skills & Careers in a Digital World-Dr. Tracy Wilen

The Future of Work, Skills & Careers in a Digital World-Dr. Tracy Wilen

Working Mother (Dec. 16, 2014) — 2014 Worklife Congress Video provided by Working Mother
Powered by NewsLook.com
Tech Companies Make Holiday Shopping Easier Than Ever

Tech Companies Make Holiday Shopping Easier Than Ever

Newsy (Dec. 16, 2014) — Innovative new services allow consumers to shop with their smartphones, split bills and even haggle. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:

Strange & Offbeat Stories

 

Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins