Featured Research

from universities, journals, and other organizations

Listen, watch, read: Computers search for meaning

Date:
November 16, 2009
Source:
ICT Results
Summary:
Researchers have created the first integrated semantic search platform that integrates text, video and audio. The system can 'watch' films, 'listen' to audio and 'read' text to find relevant responses to semantic search terms. At last, computers are able to look for meaning in our multimedia searches.

European researchers have created the first integrated semantic search platform that integrates text, video and audio. The system can 'watch' films, 'listen' to audio and 'read' text to find relevant responses to semantic search terms. At last, computers are able to look for meaning in our multimedia searches.

There is a phenomenal amount of content out there on the internet, but therein lies a problem. Sure, text content can be skimmed or glanced, but audiovisual content has to be viewed in linear time. It is very complex to search inside a film or audio recording for relevant information.

But European researchers in the MESH project have developed an integrated platform which they say, for the first time, can combine semantic search -- or search by the meaning of the words -- and a host of associated tools to deliver more relevant information, from a wide variety of sources that can be accessed from an individual user.

The platform can search annotated files from any type of media -- photographs, videos, sound recordings, text, document scans -- using a host of techniques including optical character recognition, automated speech recognition and automatic annotation of movies and photographs that track salient concepts.

Technology shift

This represents an emerging paradigm shift in search technology.

Here is why. Right now, text in computing is defined by a series of numbers, most commonly the Unicode standard. Each number signifies a particular letter, and computers can scan these codes very quickly. So when you enter a search term, the machine has no idea what those letters signify. It simply looks for the pattern -- it has no inkling of the concept behind the pattern.

But in semantic search, every bit of information is defined by potentially dozens of meaningful concepts. When a copywriter invoices for his or her work, for example, the date could be defined in terms of calendar, invoice, billing period, and so on. All these definitions for one piece of information are called 'metadata', or information about information.

Collections of agreed metadata terms for a particular field or task, like medicine or accounting, are called ontologies.

So the computer not only searches for the term, it searches for related metadata that defines types of information in specific ways. In reality, the computer still does not 'understand' a concept in its semantic search -- it continues to look for patterns of letters. But because the concepts behind the search terms are included, it can return results based on concepts as well as text patterns.

Imminent domains

These technologies are becoming common in particular knowledge domains, and more are emerging every day, but most relate to the concepts behind text-based documents. The MESH platform sought to use semantic search for every type of media.

On the way, it created some cutting-edge technology. "Our automatic annotation for video, for example, is state of the art," explains Pedro Concejero, coordinator of the MESH project.

"The annotation system is capable of identifying the general scene setting, such as whether a video is a studio shot or a shot recorded on location. With adequate training, it can also detect (within some error margins) the general topic of the video, such as a scene about an earthquake or a flood. It can also find a number of salient objects within the scene, such as persons or fire, but cannot yet identify consistently objects with great variations in shape or aspect."

One of the major challenges of the project was a product of its own success: It annotated too much information!

"This is good -- it is what we wanted the system to do -- but the quantity of data was vast, too much to handle, so we had to find ways to cut down on the amount of metadata," Concejero tells ICT Results.

Manual override

So the project developed a manual annotation tool that can, with a little training, be used by non-technical people. "It is a very powerful, very advanced professional program. There are other manual annotation tools available commercially, but we have developed a strong and user-friendly program that could probably compete very successfully with what is currently available."

For the project, the platform was developed to search video news sources relating to civil unrest and street violence, and natural disasters like earthquakes, forest fires and floods.

"We had to focus the demonstrator because there is a lot of work involved in developing ontologies for specific news topics. You would need to develop a very detailed ontology for politics, or crime and so on. We have designed the system so that it can accept ontologies from elsewhere, but for the demonstrator we reserved our work to these two domains," says Concejero.

The beginning of the end?

The technology will not be challenging the industry leading search engines any time soon. This project does not necessarily mark the end of the type of keyword-based search that we use every day.

But it could well be the beginning of the end, and in the meantime the work of the MESH project will find a happy home in a number of stand-alone commercial applications and work will, in one way or another, continue to develop new applications.

The MESH integrated project received funding from the ICT strand of the EU's Sixth Framework Programme for research.


Story Source:

The above story is based on materials provided by ICT Results. Note: Materials may be edited for content and length.


Cite This Page:

ICT Results. "Listen, watch, read: Computers search for meaning." ScienceDaily. ScienceDaily, 16 November 2009. <www.sciencedaily.com/releases/2009/11/091111120759.htm>.
ICT Results. (2009, November 16). Listen, watch, read: Computers search for meaning. ScienceDaily. Retrieved April 19, 2014 from www.sciencedaily.com/releases/2009/11/091111120759.htm
ICT Results. "Listen, watch, read: Computers search for meaning." ScienceDaily. www.sciencedaily.com/releases/2009/11/091111120759.htm (accessed April 19, 2014).

Share This



More Computers & Math News

Saturday, April 19, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Nintendo Changed Gaming World, but Its Future Uncertain: Upstone

Nintendo Changed Gaming World, but Its Future Uncertain: Upstone

AFP (Apr. 19, 2014) The Nintendo Game Boy celebrates its 25th anniversary Monday and game expert Stephen Upstone says the console can be credited with creating a trend towards handheld gaming devices. Duration: 01:21 Video provided by AFP
Powered by NewsLook.com
Nearly Two Weeks On, The Internet Copes With Heartbleed

Nearly Two Weeks On, The Internet Copes With Heartbleed

Newsy (Apr. 19, 2014) The Internet is taking important steps in patching the vulnerabilities Heartbleed highlighted, but those preventive measures carry their own costs. Video provided by Newsy
Powered by NewsLook.com
Facebook To Share Nearby Friends Data With Advertisers

Facebook To Share Nearby Friends Data With Advertisers

Newsy (Apr. 19, 2014) A Facebook spokesperson has confirmed the company will use GPS data from the new Nearby Friends feature for advertising sometime in the future. Video provided by Newsy
Powered by NewsLook.com
Man Claims He Found Loch Ness Monster With... Apple Maps?

Man Claims He Found Loch Ness Monster With... Apple Maps?

Newsy (Apr. 18, 2014) Andy Dixon showed the Daily Mail a screenshot of what he believes to be the mythical beast swimming just below the lake's surface. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins