search technologies promise to put an end frustrating
searches that don''t quite satisfy user requirements. By
an age of information overflow, information retrieval
or search is increasingly being acknowledged as a critical
issue, and the focus on search technologies is stronger
now than at any time before. Today, search is one of the
most challenging problems faced by individuals, business
organisations and government and related organisations.
transformation that is occurring in the area of ''search''
is indeed radical. As computers become more pervasive
in our daily lives, increasing fluidity or more human-like
interactions, will be sought with these machines.
information is a need not just for businesses, but also
at an individual level. This is particularly true today
as increasing number of personal tasks be it buying
a product or getting directions or phone numbers
begin to go online. A basic search box, with no further
interaction than the feeding of a keyword, is no longer
enough to serve anybody''s purpose.
large number of search enhancement tools are now talking
of context. "Not so long ago, content was king. In
2006, context will reign," so says Susan Feldman,
vice president, content technologies research, IDC. Context
is just a term, however, and its varied usage refers to
a diverse range of ideas from domain-specific search
engines to personalisation. At a general level, however,
search engines have to address a specific problem
the fact that people are never going to ask the right
questions, or, given the limitations of search technologies,
the right answers are never going to be thrown up.
For an analyst, such as Susan Feldman, the advent of search
technology issues is part of a larger, ongoing development
in the last decade. Discussing the issue in an article,
Search: the quiet revolution, Feldman says, "Search
technology has been around for more than four decades.
But only in the past ten years, as the worldwide web has
become an integral part of the technology landscape, has
it occupied a prominent place in our work and personal
lives. And only in the past four years has search finally
become a hot and lucrative area of technology development.
Why the delay?" Feldman''s reasoning, as enumerated
below, would show that some kind of a historical conjunction
may now have been reached, allowing search technologies
to emerge at the forefront of technology related issues.
to Feldman, to be truly effective, search technologies
require computing power to sort through massive amounts
of text or data. This has now been achieved, she says,
even with our desktop machines. With computing power no
longer a problem, she points out, sophisticated language
analysis and complex matching algorithms are now coming
into play in aid of search technologies.
added spur, in the advancement of such technologies, has
been the discovery by companies that lost information
puts them at risk for non-compliance. Negative issues
such as these are not the only reason why information
access is a critical issue for firms today. The fact is
that search technologies are also turning out to be a
positive aid in product development and decision making
as Feldman points out, our professional and personal lives
have been transformed in the past decade at an individual
level. Increasingly, we are looking for tools that will
allow us to sustain both professional and personal tasks
- from scheduling meetings with a client, to buying movie
tickets for the family.
as a historical conjunction of demand, technology and
computing power may now have come to pass, Feldman says
that the kind of access to information that individuals
and organisations require may still be some distance away.
What is lacking, she says, is a deep understanding of
information interactions, and how to automate them
Language is the basis of all our interactions, be it with
the computer or in the cyberspace, and language, as Feldman
points out, is "complex and ambiguous." This
is where frustrations with the sophistication of "search"
technology, as they exist, begin to be experienced by
individuals and organisations alike. This is also the
point at which a company such as Autonomy Corporation
now enters the picture.
last month, content management and enterprise search specialist,
Autonomy Corporation Plc, announced its half year results
up to 2006, and reported a trebling in sales and a 351
per cent jump in operating profits to $27.9 million. A
company release quoted Autonomy founder and chief executive,
Dr Michael Lynch, as saying "
during the past
year unstructured information issues had gone "prime
says that the last few years have seen an explosive growth
in the use of unstructured information, which definition
would include documents, emails, telephone conversations
and multimedia. According to Autonomy, unstructured
information has traditionally been difficult for computers
to understand and use. Interestingly, it says that more
than 85 per cent of all information inside an enterprise
is now unstructured.
course, unstructured information issues are what
Autonomy specialises in, a space that it defines as Meaning-Based
Computing (MBC). According to Autonomy, MBC enables
computers to understand the relationships
that exist between disparate pieces of information
and perform sophisticated analysis operations with real
business value, automatically and in real-time."
The disparate pieces of information that MBC can
act on are things like email and documents, as well as
PDFs, voice over IP and other types of content.
Lynch, explains, information in the IT world is divided
into two distinct groups: structured material that
goes in relational databases, and all the unstructured
material that doesn''t fit into IT infrastructures very
well. It is this unstructured material that is
now exploding in terms of usage.
to Autonomy, MBC solves the problem of accessing unstructured
information by applying a new breed of applications "
not only uncovers, but also makes sense of the 85 per
cent of enterprise information that remains hidden to
all other technologies including keyword search engines
and relational databases." Autonomy says MBC would
also be an active aid in tracking illegal activity.
these disparate pieces of information in a meaningful
way is perhaps what Feldman is referring to when she speaks
about information interactions.
According to Lynch, unlike structured data, which allows
IT to automate, the problem with unstructured data is
that it has not proved to be amenable to automation -
so far. The aim of MBC, Lynch says, is to enable companies
to attempt a similar process with unstructured information.
to Lynch, a variety of technologies go into MBC, from
speech recognition to text understanding, creating platforms
that take the search and interactivity process yet another
step ahead. In an interview with James Murray (ITWeek,
21st Aug, 2006) Lynch refers to this application of
technologies as "a broad church."
an interesting analogy that Lynch uses, for the origins,
of what he refers to as MBC, have a quaint connection
to churches, vicars and some very interesting English
history. A 18th century English country vicar and mathematician,
Thomas Bayes, set about trying to prove the possibility
that God exists and arrived at a theorem that discussed
the mathematical probability of things.
says that MBC traces its origins to Bayes''s enterprising
work and also builds on the work of Claude Shannon, ''the
Father of Information Theory'', whose Principles of
Information enable identification of the patterns
that naturally occur in text. These two sets of research,
says Lynch, lie at the heart of MBC. The underlying pattern-recognition
algorithm, derived from Bayes'' formulations, enables computers
to comprehend context, generalize from words to an idea,
and according to Lynch, grasp the root concepts beneath
the play of syntax..
for now, Lynch says, "
we are just at the beginning
of this movement, but in a few years time you will see
unstructured information used and processed all over the
Lynch confesses, MBC is dominated by enterprise search,
but the technologies being developed to aid enterprises
will soon begin to have a wider use. One such radical
technology is implistic query. Instead of pausing
one''s work and seeking the help of a search engine with
a query, implistic technology would not only read
what is on the screen at any time, be it an email or a
web page, but with the press of one key understand what
is on the screen and summon up related information. Hyperlinking
is another interesting technology, which would provide
links to internal and external information based on whatever
one is working on. Smart or Active folders would do filing
by themselves allowing, for instance, all documents related
to Autonomy to be filed under that company''s name. This,
says Lynch, is the direction in which search technology
is broadly headed.
as MBC evolves, Autonomy says that more than 16,000 blue-chip
corporations and government agencies are now accessing
the pattern matching algorithms in its products to extract
meaning from unstructured information. These range from
the US Department of Homeland Security, which is using
MBC across 21 agencies to monitor suspected terrorist
groups, the Ford Motor Company which is using the company''s
applications to transform the text, audio and video files
in its research libraries into meaningful reference material
in order to speed up work on new projects, as well as
a financial giant like Zurich Financial Services, which
is using MBC applications to prioritise research emanating
from more than 500 of its sources, for its risk managers.
Deere, Sprint, Whirlpool, Saab Ericsson, LexisNexis China
and RioTinto are the major new clients that Autonomy sold
its software to in the second quarter. The company has
also signed new business with multiple government, defence
and intelligence agencies around the globe, including
the US, the UK, the Netherlands, France, Italy and Singapore.
new additions join a roster of existing clients that includes
BAE Systems, Boeing, Daimler Chrysler, Shell, AOL, BBC,
Reuters, Hutchison 3G, Ericsson, T-Mobile, Philips, Coca
Cola, Kraft Foods, Nestle, Lloyds TSB, GlaxoSmithKline,
KPMG, Citigroup, ABN AMRO, Deutsche Bank, Nomura and the
US Securities and Exchange Commission.
to Feldman, Search technologies, such as text mining,
text analytics, categorisation, speech analysis, and translation
will eventually be embedded in a majority of people-facing
applications, such as cell phones, cars, gas pumps, home
entertainment centres, call centres and transit systems.
feels that as the Search needs of people and organisations
develop, and get more complex, more than a search engine
what perhaps really is the need of the times is a true
information discovery platform, incorporating all search
related technologies. Given the pace at which IT technologies
develop, that may occur sooner than one may think.