Featured Research

from universities, journals, and other organizations

San Diego Supercomputer Center Reaches Data Transfer Milestone

Date:
November 29, 2002
Source:
San Diego Supercomputer Center
Summary:
Movies and popular music are moving from tape to disk, but tape is still on a roll at the San Diego Supercomputer Center (SDSC) at the University of California, San Diego. The center’s huge, updated tape storage system has illustrated its effectiveness by transferring data at 828 megabytes per second. "Nobody in academia has such a fast data archive system," said Phil Andrews, program director for High-End Computing at SDSC.

Movies and popular music are moving from tape to disk, but tape is still on a roll at the San Diego Supercomputer Center (SDSC) at the University of California, San Diego. The center’s huge, updated tape storage system has illustrated its effectiveness by transferring data at 828 megabytes per second. "Nobody in academia has such a fast data archive system," said Phil Andrews, program director for High-End Computing at SDSC.

Supercomputers are ranked by their raw computational speed, but rapid movement of increasingly large data sets has become vital to researchers in scientific disciplines from anatomy and astronomy, to climatology and particle physics. Data sets in those fields have grown over the past decade, by multiples of 1,000, from megabytes (millions of bytes) to gigabytes, to terabytes. Some data sets are now approaching or exceeding one petabyte (one quadrillion bytes).

Even though tape costs a fraction of disk, supercomputer centers have been investing heavily in more expensive disk storage because its data-transfer rates are higher, by a factor of at least 10, than what had been the best tape systems. At the same time, however, supercomputer centers are struggling to pay for the disk capacity required to keep up with the explosive growth of scientific data.

"While we are expanding our disk storage capacity, we’re also adding new higher-density tapes, faster tape drives and more of them, and other technology in a highly tuned, extremely capable data management system," said Andrews. "It’s all these pieces working together that has allowed us to reach a new milestone in data-transfer speed."

New technology at SDSC involved in the tape-to-disk data transfers includes tape drives from StorageTekฎ (Storage Technology Corp.), switches from Brocade Communications Systems, fibre channel adapters from QLogic, and end-to-end systems and technology from Sun Microsystems. The combined effect the new technology has been a reduction–from days to hours–in the transfer of multi-terabyte data sets from SDSC’s tape-storage system to its IBM Blue Horizon supercomputer, which has a peak speed of 1.7 teraflops (trillion floating point operations per second).

"Our supercomputer center is dealing with more multi-terabyte data sets–such as the 10-terabyte Digital Sky astronomy project–and very fast transfers will allow astronomers to make discoveries faster," said Andrews. "With tape, we have achieved a data-transfer rate that very few people realized was feasible."

No industry association keeps track of data-transfer speed records. "As far as the use of tape drives or multiple tape drives is concerned, the data transfer rates of most academic groups have been well under 100 megabytes per second," said John Marshal, program manager at StorageTek. "The groups that have achieved data transfer rates in the range of 800 megabytes per second have not used tape: they have done it with much larger investments in disk drives, switches, and routers."

SDSC is replacing its 20-gigabyte-capacity tape cartridges with 200-gigabyte native capacity tape cartridges manipulated robotically in five silos with a total storage capacity of six petabytes. The center also installed 24 StorageTek T9940B tape drives, each of which can transfer data to and from tape at roughly 30-megabytes-per-second throughput without data compression. (Compression of data increases its rate of transmission.) SDSC has measured over 60 megabytes-per-second peak speed with the new tape drives with roughly twofold data compression. (The actual measured data-transfer ratedepends on the degree to which data is compressed, the type of software used, and the capacity of the associated hardware involved in the transfer.)

As part of its data-management system, SDSC has also installed Sun Microsystems’ SAM-FS Advanced Storage Management and QFS High Performance Shared File System software to provide maximum scalability, performance, and throughput for the most data-intensive applications. SDSC has also installed Sun’s High Performance Computing Storage Area Network (SAN) storage solution. Such SANs allow multiple computers, using a range of operating systems, to share data seamlessly. The disk capacity attached to SDSC’s SAN will be increased from 50 terabytes to 500 terabytes by 2003.

SDSC’s recent and planned improvements will enable the center to deliver the unprecedented data-management infrastructure required by the National Science Foundation’s $88 million TeraGrid project. The TeraGrid will be deployed in 2003 as the world's largest, fastest, distributed infrastructure for open scientific research. When completed, the TeraGrid will include more than 20 teraflops of distributed computing power, facilities dedicated to managing and storing nearly one petabyte of disk storage and more than six petabytes of tape capacity, high-resolution visualization environments, and toolkits for grid computing. The components of the TeraGrid will be tightly integrated and connected through a network that will operate at 40 gigabits per second–the world’s fastest research network.

The TeraGrid infrastructure will be distributed among five sites: SDSC, the lead site for the National Partnership for Advanced Computational Infrastructure; the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign; the Center for Advanced Computing Research at the California Institute of Technology; Argonne National Laboratory; and the Pittsburgh Supercomputing Center.

A Second Data-Transfer Achievement

In a second data-transfer feat demonstrated Nov. 21 at the Supercomputing 2002 (SC2002) conference at the Baltimore Convention Center, SDSC computer scientist Bryan Banister transferred data from disk at the supercomputer center in La Jolla, CA, to disk at the convention center at 721 megabytes per second.

Banister performed the disk-to-disk data transfer as a demonstration of what will soon be routine–movement of large data sets from SDSC to the other TeraGrid sites over a high-speed network. The cross-country, disk-to-disk transfer made use of a Qwest Communication 10-gigabit-per-second fiber-optic backbone. It also involved Nishan Systems storage switches, Force10 Networks switches, and Juniper Networks routers.

SC2002, this year’s edition of the world’s largest annual high-performance networking and computing conference, is designed to demonstrate supercomputing technology and the breakthrough science it makes possible.


Story Source:

The above story is based on materials provided by San Diego Supercomputer Center. Note: Materials may be edited for content and length.


Cite This Page:

San Diego Supercomputer Center. "San Diego Supercomputer Center Reaches Data Transfer Milestone." ScienceDaily. ScienceDaily, 29 November 2002. <www.sciencedaily.com/releases/2002/11/021126203210.htm>.
San Diego Supercomputer Center. (2002, November 29). San Diego Supercomputer Center Reaches Data Transfer Milestone. ScienceDaily. Retrieved October 20, 2014 from www.sciencedaily.com/releases/2002/11/021126203210.htm
San Diego Supercomputer Center. "San Diego Supercomputer Center Reaches Data Transfer Milestone." ScienceDaily. www.sciencedaily.com/releases/2002/11/021126203210.htm (accessed October 20, 2014).

Share This



More Computers & Math News

Monday, October 20, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Facebook Says The DEA's Fake Accounts Go Too Far

Facebook Says The DEA's Fake Accounts Go Too Far

Newsy (Oct. 19, 2014) — Facebook says the DEA violated its Terms of Service and that such impersonations damage the integrity of the site. Video provided by Newsy
Powered by NewsLook.com
Court Ruling Means Kids' Online Activity Could Be On Parents

Court Ruling Means Kids' Online Activity Could Be On Parents

Newsy (Oct. 17, 2014) — In a ruling attorneys for both sides agreed was a first of its kind, a Georgia appeals court said parents can be held liable for what kids put online. Video provided by Newsy
Powered by NewsLook.com
For Google, Even A $16.5 Billion Earnings Report Is A Miss

For Google, Even A $16.5 Billion Earnings Report Is A Miss

Newsy (Oct. 17, 2014) — Analysts were expecting more, but Google’s ad growth slowed on the quarter and the company is spending more of its money. Video provided by Newsy
Powered by NewsLook.com
Obama Signs Cybersecurity Order, Wants Safer Payments

Obama Signs Cybersecurity Order, Wants Safer Payments

Reuters - US Online Video (Oct. 17, 2014) — President Barack Obama announces details of a new executive order designed to make federal payments safer following recent massive data breaches. Rough Cut (no reporter narration). Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:

Strange & Offbeat Stories

 

Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins