|
|
 |
 |
 |
 |
   |
'KEY WORDS' FOR GRAPHICS?Ever use one of the Web's search engines? Then you know how frustrating it is when, instead of zeroing in on what you want, they bring back a bunch of irrelevant Web pages. Imagine, then, how much worse things could be when you're dealing with material that, unlike a Web page, is not mainly text and cannot be easily cataloged according to key words. A photograph doesn't have key words, and computers can't recognize what humans see--the setting, facial expressions, or mood of a photo, for instance. But programmers are working furiously on the problem, so that within a few years even consumers will be able to scan multimedia archives for pictures and sound clips to drop into their computers. Until now, the standard approach has been to hire people to look at pictures--or listen to recordings--and write captions that describe their key attributes. But that's expensive--as much as $25 per item--and it's hard to note every possible attribute. For example, if you were seeking images that showed ''fatigue,'' a computer would skip a shot of a Depression-era bread line--unless its caption included the word ''fatigue.'' Says Dragutin Petkovic, manager of visual media management at IBM's Almaden Research Center near San Francisco: ''Key words are too brittle.'' One new approach is to try to infer new meanings from old captions. Software from Oracle Corp. and SRA International Inc. uses deep linguistic knowledge and artificial intelligence techniques to analyze natural language queries (below). Cycorp Inc. in Austin, Tex., uses thousands of commonsense rules. Say a caption reads ''Soldier holding a gun to a woman's head.'' The Cycorp software knows that guns kill people and that people threatened by guns feel fear. Thus, it would retrieve that photo if asked to find ''someone in danger.'' VISUAL ''RHYMES.'' The latest advance is software that directly indexes images themselves. IBM calls its version Query By Image Content. Virage Inc., a San Mateo (Calif.) startup, sells VIR Image Engine. Both products measure how much of each color an image shows and identify textures and prominent shapes. The computer can then use that data to compare one image with another. Tell it to find pictures resembling this one of a white rose, and it will retrieve more flowers. But it might also fetch a white poodle on a green bedspread, based on the matching color pattern. IBM's Petkovic says one can avoid such ''visual rhyming'' by first using key words to limit the subject of the search. The potential applications for this software are diverse. By indexing key frames, such as scene changes, it can help search video footage. And the Los Angeles County Sheriff's Dept. is using IBM's package to track urban gangs' tattoos--something you're not likely to find on the Web just yet.
By John W. Verity in New York
|
Updated June 14, 1997 by bwwebmaster
Copyright 1996, by The McGraw-Hill Companies Inc. All rights reserved.
Terms of Use