Featured Research

from universities, journals, and other organizations

Crow or raven? New birdsnap app can help

Date:
May 28, 2014
Source:
Columbia University School of Engineering and Applied Science
Summary:
Using computer vision and machine learning techniques, researchers have developed Birdsnap, a free new iPhone app that's an electronic field guide featuring 500 of the most common North American bird species. The app enables users to identify bird species through uploaded photos, and accompanies a comprehensive website.

Screenshot of Birdsnap wheel of bird species.
Credit: Image courtesy of Columbia University School of Engineering and Applied Science

Researchers at Columbia Engineering, led by Computer Science Professor Peter Belhumeur, have taken bird-watching to a new level. Using computer vision and machine learning techniques, they have developed Birdsnap, a new iPhone app that is an electronic field guide featuring 500 of the most common North American bird species. The free app, which enables users to identify bird species through uploaded photos, accompanies a visually beautiful, comprehensive website that includes some 50,000 images. Birdsnap, which also features birdcalls for each species, offers users numerous ways to organize species -- alphabetically, by their relationship in the Tree of Life, and by the frequency with which they are sighted at a particular place and season. The researchers, who collaborated with colleagues at the University of Maryland, are presenting their work at the IEEE Conference on Computer Vision and Pattern Recognition in Columbus, OH, June 24 to 27.

Related Articles


"Our goal is to use computer vision and artificial intelligence to create a digital field guide that will help people learn to recognize birds," says Belhumeur, who launched Leafsnap, a similar electronic field guide for trees, with colleagues two years ago. "We've been able to take an incredible collection of data -- thousands of photos of birds -- and use technology to organize the data in a useful and fun way."

Belhumeur and his colleague, Computer Science Professor David Jacobs of the University of Maryland, realized that many of the techniques they have developed for face recognition, in work spanning more than a decade, could also be applied to automatic species identification. State-of-the-art face recognition algorithms rely on methods that find correspondences between comparable parts of different faces, so that, for example, a nose is compared to a nose, and an eye to an eye. Birdsnap works the same way, detecting the parts of a bird so that it can examine the visual similarity of its comparable parts (each species is labeled through the location of 17 parts). It automatically discovers visually similar species and makes visual suggestions for how they can be distinguished.

"Categorization is one of the fundamental problems of computer vision," says Thomas Berg, a Columbia Engineering computer science PhD candidate who works closely with Belhumeur. "Recently, there's been a lot of progress in fine-grained visual categorization, the recognition of -- and distinguishing between -- categories that look very similar. What's really exciting about Birdsnap is that not only does it do well at identifying species, but it can also identify which parts of the bird the algorithm uses to identify each species. Birdsnap then automatically annotates images of the bird to show these distinctive parts -- birders call them 'field marks' -- so the user can learn what to look for."

The team designed what they call "part-based one-vs-one features," or POOFs, each of which classifies birds of just two species, based on a small part of the body of the bird. The system builds hundreds of POOFs for each pair of species, each based on a different part of the bird, and chooses the parts used by the most accurate POOFs as field marks. Birdsnap also uses POOFs for identification of uploaded images.

The team also took advantage of the fact that modern cameras, especially those on phones, embed the date and location in their images and used that information to improve classification accuracy. Not only did they come up with a fully automatic method to teach users how to identify visually similar species, but they also designed a system that can pinpoint which birds are arriving, departing, or migrating. "You can ID birds in the U.S. wherever you are at any time of year," Berg notes.

The Leafsnap app, which involved costly time and resources spent in collecting and photographing thousands of leaves, took almost 10 years to develop and now has more than a million users. Belhumeur got Birdsnap going in about six months, thanks to the proliferation of online data sources and advances in computer vision and mobile computing. Photos were downloaded from the Internet, with species labels confirmed by workers on Amazon Mechanical Turk, who also labeled the parts. Descriptions were sourced through Wikipedia. The maps were based on data from eBird, a joint venture of Cornell University's Lab of Ornithology and the National Audubon Society, and BirdLife, an international network of conservation groups.

Belhumeur hopes next to work with Columbia Engineering colleagues on adding the ability to recognize bird songs, bringing audio and visual recognition together. He also wants to create "smart" binoculars that use this artificial intelligence technology to identify and tag species within the field of view.

"Biological domains -- whether trees, dogs, or birds -- where taxonomy dictates a clear set of subcategories, are wonderfully well-suited to the problem of fine-grained visual categorization," Belhumeur observes. "With all the advances in computer vision and information collection, it's an exciting time to be immersed in visual recognition and big data."

This research was funded by the National Science Foundation, the Gordon and Betty Moore Foundation, and the Office of Naval Research.

App: https://itunes.apple.com/app/birdsnap/id880461148

Website: http://birdsnap.com/


Story Source:

The above story is based on materials provided by Columbia University School of Engineering and Applied Science. Note: Materials may be edited for content and length.


Cite This Page:

Columbia University School of Engineering and Applied Science. "Crow or raven? New birdsnap app can help." ScienceDaily. ScienceDaily, 28 May 2014. <www.sciencedaily.com/releases/2014/05/140528163635.htm>.
Columbia University School of Engineering and Applied Science. (2014, May 28). Crow or raven? New birdsnap app can help. ScienceDaily. Retrieved December 22, 2014 from www.sciencedaily.com/releases/2014/05/140528163635.htm
Columbia University School of Engineering and Applied Science. "Crow or raven? New birdsnap app can help." ScienceDaily. www.sciencedaily.com/releases/2014/05/140528163635.htm (accessed December 22, 2014).

Share This


More From ScienceDaily



More Computers & Math News

Monday, December 22, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Can Bitcoin Survive 2015?

Can Bitcoin Survive 2015?

Newsy (Dec. 22, 2014) Bitcoin's stock has tumbled significantly this year, but more companies now accept it, leading supporters and critics alike to weigh in on its future. Video provided by Newsy
Powered by NewsLook.com
Touch-Free Smart Phone Empowers Mobility-Impaired

Touch-Free Smart Phone Empowers Mobility-Impaired

Reuters - Innovations Video Online (Dec. 21, 2014) A touch-free phone developed in Israel enables the mobility-impaired to operate smart phones with just a movement of the head. Suzannah Butcher reports. Video provided by Reuters
Powered by NewsLook.com
Building Google Into Cars

Building Google Into Cars

Reuters - Business Video Online (Dec. 19, 2014) Google's next Android version could become the standard that'll power your vehicle's entertainment and navigation features, Reuters has learned. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com
After Sony Hack, What's Next?

After Sony Hack, What's Next?

Reuters - US Online Video (Dec. 19, 2014) The hacking attack on Sony Pictures has U.S. government officials weighing their response to the cyber-attack. Linda So reports. Video provided by Reuters
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Space & Time

Matter & Energy

Computers & Math

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins