Featured Research

from universities, journals, and other organizations

Updated software uses combination testing to catch bugs fast and easy

Date:
November 15, 2010
Source:
National Institute of Standards and Technology (NIST)
Summary:
Researchers at the National Institute of Standards and Technology have released an updated version of a computer system testing tool that can cut costs by more efficiently finding flaws. A tutorial on using the tool accompanies the new release.

NIST's software for testing computer systems -- ACTS -- takes advantage of research that shows that virtually all software failures appear to be caused by six or fewer interactions.
Credit: NIST

Researchers at the National Institute of Standards and Technology (NIST) have released an updated version of a computer system testing tool that can cut costs by more efficiently finding flaws. A tutorial on using the tool accompanies the new release.

Catching software "bugs" before a program is released enhances computer security because hackers often exploit these flaws to introduce malware, including viruses, to disrupt or take control of computer systems. But it's difficult. A widely cited 2002 study prepared for NIST* reported that even though 50 percent of software development budgets go to testing, flaws in software still cost the U.S. economy $59.5 billion annually.

Exhaustive checking of all possible combinations of input actions that could cause software failure is not practical, explained NIST's Raghu Kacker, because of the huge number of possibilities, but it's also not necessary. Based on studies of software crashes in applications, including medical devices and Web browsers, NIST's Rick Kuhn and other researchers determined that between 70 and 95 percent of software failures are triggered by only two variables interacting and practically 100 percent of software failures are triggered by no more than six. "Testing every combination up to six variables can be as good as exhaustive testing," said Kacker.

Working with researcher Jeff Yu Lei and his students from the University of Texas at Arlington, NIST designed Advanced Combinatorial Testing System (ACTS), a freely distributed software tool to generate plans for efficiently testing combinations of two to six interacting variables. The method goes beyond the commonly used "pairwise" approach to software testing, which tests combinations of two variables, so it can detect more obscure flaws.

Kuhn describes the process "as packing as many combinations into a set of tests as efficiently as we know how." For example, testing all possible interactions for a product with 34 on and off switches would require 17 billion tests. Using ACTS, all three-way interactions can be evaluated using only 33 tests and all six-way combinations with just 522 tests, instead of 17 billion.

The first version of ACTS was released in 2008. Since then, it has been distributed at no cost to 465 organizations and individuals in industry, academia and government. "About half of our users are in IT, but other heavy users are in the financial, defense and telecommunications sectors," said Kuhn. In August, NIST and Lockheed Martin initiated a Cooperative Research and Development Agreement to study the application of ACTS in the company's large and complex software applications. The two groups will jointly publish the results.

NIST released the latest update of ACTS in October. The new version includes an improved user interface and a better method of specifying relationships between parameters for testing. This can eliminate the problem, for example, of spending time on tests for invalid combinations, such as using Internet Explorer on a Linux system.

Just released is a new tutorial, Practical Combinatorial Testing, that introduces key concepts and methods along with explaining the use of software tools for generating combinatorial tests. Cost and other practical considerations are addressed. The tutorial is designed to be accessible to undergraduate students in computer science or engineering and includes extensive references. NIST Special Publication 800-142 is available at http://csrc.nist.gov/groups/SNS/acts/documents/SP800-142-101006.pdf

* Research Triangle Institute, The Economic Impacts of Inadequate Infrastructure for Software Testing, NIST Planning Report 02-3, May 2002.


Story Source:

The above story is based on materials provided by National Institute of Standards and Technology (NIST). Note: Materials may be edited for content and length.


Cite This Page:

National Institute of Standards and Technology (NIST). "Updated software uses combination testing to catch bugs fast and easy." ScienceDaily. ScienceDaily, 15 November 2010. <www.sciencedaily.com/releases/2010/11/101110113042.htm>.
National Institute of Standards and Technology (NIST). (2010, November 15). Updated software uses combination testing to catch bugs fast and easy. ScienceDaily. Retrieved April 16, 2014 from www.sciencedaily.com/releases/2010/11/101110113042.htm
National Institute of Standards and Technology (NIST). "Updated software uses combination testing to catch bugs fast and easy." ScienceDaily. www.sciencedaily.com/releases/2010/11/101110113042.htm (accessed April 16, 2014).

Share This



More Computers & Math News

Wednesday, April 16, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

German Researchers Crack Samsung's Fingerprint Scanner

German Researchers Crack Samsung's Fingerprint Scanner

Newsy (Apr. 16, 2014) German researchers have used a fake fingerprint made from glue to bypass the fingerprint security system on Samsung's new Galaxy S5 smartphone. Video provided by Newsy
Powered by NewsLook.com
Twitter, Apple Social Data Purchases Likely to Spur More Mergers and Acquisitions

Twitter, Apple Social Data Purchases Likely to Spur More Mergers and Acquisitions

TheStreet (Apr. 16, 2014) The social media data space is likely to see more mergers and acquisitions following Twitter Inc.'s acquisition of tweet analyzer Gnip Inc. on Tuesday and Apples Inc.'s purchase of Topsy Labs Inc. back in December. One firm in particular, the U.K.'s DataSift Inc., could be on the list of potential buyers. Among other social media startups that could be ripe for picking is Banjo, whose mobile app provides aggregated content by topic and location. Banjo could also be a good fit for Twitter. Video provided by TheStreet
Powered by NewsLook.com
Bitcoin Exchange Mt. Gox to Liquidate After Rebuilding Rejected

Bitcoin Exchange Mt. Gox to Liquidate After Rebuilding Rejected

TheStreet (Apr. 16, 2014) Bitcoin exchange Mt. Gox has agreed to liquidate after a Japanese court rejected its plans to rebuild, according to a report by the Wall Street Journal. Mt. Gox filed for bankruptcy protection in February after announcing about 850,000 bitcoins, worth around $454 million at today's rates, may have been stolen by hackers. It has since recovered 200,000 of the missing bitcoins. The court put Mt. Gox's assets under a provisional administrator's control until bankruptcy proceedings begin. Video provided by TheStreet
Powered by NewsLook.com
BlackBerry: The Crash That Launched 1,000 Startups

BlackBerry: The Crash That Launched 1,000 Startups

Reuters - Business Video Online (Apr. 16, 2014) Tech startups in BlackBerry's hometown of Waterloo, Ontario, are tapping talent from the struggling smartphone company and filling the void left in the region by its meltdown. Reuters correspondent Euan Rocha visits the region that could become Canada's Silicon Valley. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins