Search

CMS Watch says enterprise search vendors are opening up

Ron Miller — Tue, 25 Nov 2008 10:00:22 -0500

In a new CMS Watch report on enterprise search, the content management consulting firm points out that search is opening up in the enterprise. CMS Watch Principal Theresa Regli says, enterprise search was once a mysterious black box where indexing went on behind the scenes, out of the view of the enterprise IT staff. Now, enterprise search vendors are joining other enterprise applications in providing more transparency through web interfaces that let administrators access back-end processes.

"Historically search has often been seen as 'magic'--and vendors encouraged secretiveness around their indexing processes and algorithms," Regli said. "But that's changed quite a bit, and everyone from Endeca to the notoriously secretive Autonomy now has web interfaces to control and modify the infrastructure and indexing approaches, and even to fine-tune details of what used to be known as a 'black box.'"

The Search & Information Access Report 2009 looks at 20 enterprise search products. Among the findings in this year's report is evidence that vendors are providing easier set up and maintenance, and they are including more functionality out of the box instead of requiring customers to deal with long, custom development cycles. As we move into 2009, customers are going to be looking for products that solve their search issues for a good price and with the least amount of pain. This guide could be a starting point for you to help you make your enterprise search purchase decisions.

For more information:
- see the press release

Related Article:
AIIM study: Enterprise search needs work

Google unveils Search Wiki

Ron Miller — Mon, 24 Nov 2008 17:22:16 -0500

Google came out with an interesting concept last week when it released the Google Search Wiki. It enables you to take the standard set of results and move the ones you like best to the top of the results lists. What's more you can leave notes about why you like or dislike a site, and the notes are then available to the public. To be sure, there is room for lots of manipulation on the open web, but this could prove quite useful in an enterprise search environment where the universe of results is going to be much smaller.

Some search vendors like Oracle's Secure Enterprise Search product include the ability for IT to set certain results at the top of the list, but this could put the power to place search results in the hands of the end user, which is very powerful. Picture being able to put your favorite presentation at the top of a set of search results.

Further, you could attach a note to it, giving you the ability to let other folks know why you like it, and why it makes a good template for other customers in the same vertical market. It's important to understand that the notes are public, but the way you manipulate the results is only available in your personal set of results. Whether this can work at all on the open web is still open to question, but it should be interesting to see what happens if Google makes this available in its enterprise search products.

For more information:
- see this article

Related Articles:
Google news from FierceContentManagement

Google Site Search upgrade includes on-demand indexing

Ron Miller — Wed, 19 Nov 2008 08:48:45 -0500

Google announced last week that they were upgrading the Google Site Search product to include on-demand indexing. In previous versions of the product, when you added new content to the site, you had to wait for Google's algorithm to kick in and re-index, a situation many customers were not happy with. The newest version enables you to click the "Index Now" button whenever you add new content, giving the site owner complete control over the indexing process. Article

One on One with Nitin Mangtani of Google Search Appliance

Ron Miller — Tue, 04 Nov 2008 14:28:20 -0500

Nitin Mangtani is the lead product manager for the Google Search Appliance (GSA). In this role, Nitin focuses on defining enterprise search strategy, establishing the product road map and defining new releases for the GSA. There have been several pieces, published in this space, on the difficulty of enterprise search and what happens when you move consumer-style Google search to the enterprise.

Recently, Mangtani surprised a few people when he published an SAP-sponsored editorial on Forbes.com in which he basically defended Google enterprise search against critics. It was a bold move. It prompted me to write to him and ask some questions of my own about his reason for publishing the Forbes piece and his rebuttal to Google Search Appliance critics.

FCM: You gave a spirited defense of your product in Forbes. What prompted you to react in that forum in that fashion?

NM: Our words are a reflection of our commitment to enterprise search. At Google, we are constantly innovating to ensure that our search solutions for the enterprise are as powerful and easy to use as Google.com. It's important to us that everyone--from CIOs to IT admins to employees--hear how we're taking our online search expertise, and tailoring that experience to meet the specific needs of businesses large and small.

FCM: Google's PageRank is the frequently touted "secret sauce" for the web-based search experience. Yet the majority of content in enterprise is not web-native content. How does Google adjust the search relevancy strategy for a non-hyperlinked body of content?

NM: Google has made a name for serving the most relevant results. In the workplace, the more relevant your results, the easier and faster you can get your job done. Speed and relevance are the twin pillars of a successful search, and are our primary focus in delivering search solutions for enterprise customers.

It's important to understand that PageRank is just one of the many signals we use to determine relevancy for Google.com. Similarly, in the enterprise, there's no one signal we rely on to deliver results. The Google Search Appliance leverages the technology tested daily by millions of users at Google.com, but also incorporates algorithms and models that we develop exclusively to suit the challenges of an enterprise search environment.

To meet these challenges, we have some of the best and brightest engineers in the world working on enterprise search. Our dedicated team is made up of experts in information retrieval, putting the power of PhDs and years of experience to work for our customers. If there's any secret to our strategy, it's simply that we don't let our recipe for search get cold. Our secret sauce is a team committed to innovation, and delivering the truly useful results that allow businesses to focus on business goals.

FCM: Along the same lines, I think Google loses its advantage inside the enterprise where there are fewer likely correct choices. As one analyst puts it, it's far easier to find a correct document among many correct answers, more difficult when there are only one or two correct ones. How does Google compensate for this difference?

NM: We find that end users in the enterprise often need to perform both functions, whether it's finding a specific document, or more general information discovery. Our goal is to make sure they can tackle both of these tasks with ease.

In both instances, relevance is key--and our ability to deliver relevant results is unmatched. One way we achieve relevance is through the personalization of search results. With the Google Search Appliance, administrators can adjust search results for different user groups, based on department or function. This means engineers see code and design documents at the top of their search results, while employees in marketing department see market analysis and white papers at the top of search results for the same query.

For the employee who doesn't know exactly what they're looking for, we also dynamically cluster results, which allows users to drill down on information through automatically generated categories. This allows employees to pull information from related areas and projects, often leading to the discovery new information they didn't know existed.

And for those looking for information residing in enterprise applications, like SAP or an employee directory, we provide an Enterprise OneBox that pulls information directly from across a business' enterprise applications. That way employees looking for the Q4 sales report or the status of a purchase order can find it all from the convenience of their search box.

Providing the best search possible isn't just about returning results. It's about understanding the nature of the search query and the searcher. Our search solutions are designed to take both into account, and to get better over time by learning the types of results that are most useful to you.

FCM: Carl Frappaolo of AIIM has said that what makes enterprise search so difficult is what he calls the "digital landfill" of information, data that is spread out across repositories in the enterprise. How does Google enterprise search get at this type of information locked away in a variety of repositories?

NM: This is one area where our experience online has taught us an important lesson. Whether searching online at home, or behind the firewall at work, employees want to have all their results accessible through one search box.

We call this universal search. In the same way we pull together web pages, news, blogs, financial quotes and more online, we've designed our enterprise search to pull results from across business infrastructure, including intranets, databases, portal servers, fileshares and content management systems, like Documentum, FileNet, OpenText, Livelink and Microsoft Sharepoint. We provide native connectors that allow businesses to index data from their various repositories, and our built-in Content Connector Framework gives businesses the flexibility to build the custom connections they may need.

Pulling all this information together certainly isn't easy, but its our job to ensure that from a user's perspective, everything they need is at their fingertips. Our universal search ensures that employees can find information, no matter where or how it's stored, while our expertise delivering fast, relevant results ensures they can act on the information they uncover.

FCM: In my view, based on my conversations with industry experts over the years, the Google Appliance is an excellent choice for a department, but not so great for an enterprise-wide search tool that crosses departments, repositories and data types. Why do you think I'm wrong about this conclusion?

NM: I think the best way to answer this is with the response we've gotten from our customers. We have more than 20,000 customers using our search solutions, encompassing a range of business sizes and types, and many of these are large enterprises and Fortune 500 companies.

Kimberly Clark, a Fortune 500 company known for brands such as Huggies and Kleenex, is a fantastic example of a large company faced with precisely the dilemma you outlined. They had millions of documents across a range of departments and repositories, and they needed a solution that could bring it all together. With the Google Search Appliance, employees can now search more than 22 million documents from across their company intranet, as well as web applications, homegrown document management systems, filer servers, and their public internet site. And best of all, they were able to install the search appliance quickly, and with minimal training for employees who were already used to the Google.com search experience.

Honeywell International, a leader in the tech manufacturing industry, is another great example of a large enterprise using the Google Search Appliance with great success. Honeywell needed a solution that could power both their internal search, as well as the external search on their public website. The Google Search Appliance addressed both of these needs, and was optimized and serving results within a day.

The range of customers we serve, again, speaks to the flexibility of our search solutions for the enterprise. Whether a small business with a rapidly growing number of files to manage, or a large business with content buried across databases and departments, our search solutions can scale and be tailored to meet a range of different needs from across the business community.

FCM: Users are always saying they want the Google consumer experience in the enterprise, but you don't seem to have the same success there you have on the web, why do you think that is?

NM: Our corporate motto at Google is "focus on the user and all else will follow". In the case of enterprise search, users are demanding the Google experience they know, trust and depend on. We can't think of anyone in a better position to provide this experience than us.

We hear from many of the businesses we work with how glad they are to have a solution at work that is as easy to use as Google.com. Our solution saves time and money by being easy to set up and maintain, and saves businesses the worry of needing to provide extensive training to employees who already understand and trust the experience they get with Google.

Success has many forms. If our success is judged by adoption, then we stand by the 20,000 customers using our search solutions. If it's judged by commitment to innovation, then we stand by our continued deployment of new products and improvements. And if it's judged by end user experience, then we absolutely stand by the power and simplicity of the search solution we provide to our customers. Search for us is never a solved problem, and in our minds, the best is yet to come.

Microsoft's Fast in trouble over questionable accounting practices

Ron Miller — Mon, 20 Oct 2008 12:03:54 -0400

Last week, Reuters reported that Norwegian police raided the headquarters of Fast Search & Transfer, an enterprise search company purchased by Microsoft earlier this year for $1.2 billion, for alleged accounting fraud. The irregularities reportedly happened prior to the purchase, but according to an August EContent magazine article, "the problems were no secret, even then." The charges involve announcing revenue when it didn't really have the business to support the claim, according to police quoted in the Reuters' article. Article

SPOTLIGHT ON...MarkLogic releases MarkLogic Server 4.0

Ron Miller — Fri, 10 Oct 2008 17:18:25 -0400

MarkLogic, the XML search vendor, released MarkLogic Server 4.0. Key new features include geospatial support to build location-based applications that search and analyze content enhanced with geospatial markup, and support for 18 different types of entities including credit card numbers, longitude/latitude and email addresses. The new version also introduces alerts so that users receive a notice whenever content matching the query is updated. This saves the user from having to manually conduct the search on an ongoing basis. Press Release

For more information:
- see the Geospatial demo and the Co-occurrence demo

Enterprise search is not a simple matter

Ron Miller — Tue, 23 Sep 2008 10:35:48 -0400

If you work in an enterprise, chances are you've heard the complaint: "Why can't our search work more like Google?" Employees, spoiled by the Google search experience on the web quite naturally think it should be simple enough to extend to the enterprise. Unfortunately, as Dan Woods points out in his Jargon Spy column on Forbes.com this week, it's not easy to extend this type of search to the enterprise for a number of reasons related to the differences between the open web and a company content repository.

In fact, AIIM released a study last spring that proved that employees want a consumer experience, but the enterprise is a different animal. In an article in EContent called "Enterprise Search Still Lacking", Carl Frappaolo, VP at AIIM stated that the enterprise has to deal with what he calls a digital landfill of information. With so many content sources to cross, it's not easy to pinpoint the document you need. Woods points out that Google is built on a page rank algorithm and most companies don't have a link-based system.

There are a myriad of other issues, too, including the fact that, as Woods points out, you don't always know what you're looking for. One way to solve this issue is to use targeted tagging, but in a large enterprise this is a hard system to keep under control. There are, in fact, no simple answers to enterprise search. If there were, somebody would be making a lot of money.

For more information:
- read Wilson's column
- check out the EContent article
- view AIIM's findability study

Web CMS could be to blame in UAL stock plunge story

Ron Miller — Fri, 12 Sep 2008 13:16:31 -0400

The web was abuzz this week with news about how a Chicago Tribune story from 2002 was released as current news and caused the price of UAL stock to go tumbling. My immediate thought was that this was a web CMS glitch. Somehow, somebody released a story with the wrong date, which then was picked up by Google News and the financial wires services, and then took on a life of its own as can only happen on the wild, wild web. In fact, my colleague Tony Byrne who runs the respected CMS Watch initially thought the same thing, but then back-tracked in a later post, blaming the Google News Bot instead. I'm not ready to be so forgiving. You still have to ask why the story was posted in the first place and why it didn't have a clear date line.

Rule No. 1 for organizations placing content on a website is to display a clear date so readers can see how current it is. The web CMS should do that as a matter of course. Google sent out a crawler as it always does, and apparently it simply picked up the story in a "current news" section of the Florida Sun Sentinel. According to Google, the crawler, which is programmed to check for a date, only found a current one and picked up the story. We may never know how or why it was served up on the CMS in the first place, but one thing is certain: it certainly caused a large ruckus and had a very real effect on UAL's stock prices (at least for the short term). There are probably lessons aplenty here for everyone, but for folks who are designing or running your web CMS; for goodness sakes, post a date.

For more:
- check out this eWeek article
- read this blog from Duo Consulting
- read this New York Times blog

Newspapers can survive as portals, content service providers

Ron Miller — Wed, 30 Jul 2008 10:55:05 -0400

The last few years have not been kind to newspapers. Everywhere you turn, there is word of layoffs and bleak financial news, but Joe Lichtman offers a survival strategy in his latest FastForward blog; it involves a two-pronged digital approach that could be applied to practically any big city paper. First, Lichtman prescribes transforming a newspaper's website into a go-to portal for all things related to the city where the paper publishes, instead of just an online version of the daily paper. That might mean including links to competitor's content, as Boston.com has done with its local search tool. Sure, you may lose eyeballs for a moment, but if you are the go-to destination for all-things Boston, those eyes will return soon enough.

His other strategy involves opening up the publication as a content service, which allows other sites to access your publication's content. He cites the New York Times' recent announcement about developing an API to provide an easy way for other sites to publish and incorporate Times content. The New York Times last week also announced an agreement with LinkedIn to deliver custom content on the business and technology section based on an individual's LinkedIn profile. Lichtman points out that it's entirely possible for newspapers to use both approaches, and since the Times and the Boston Globe are owned by the same parent company, it's likely they will share ideas with one another moving forward. Regardless, Lichtman's advice provides a way for newspapers to survive online, and that's got to look pretty good to desperate publishers right about now.

For more information:
- read Lichtman's blog entry

You think you've got content issues? Google index hits 1 trillion pages

Ron Miller — Sun, 27 Jul 2008 17:05:43 -0400

Most large enterprises boast a decent-sized content repository, and if your company deals in high-volume transaction processing like check handling, it reaches another level. Still, no matter how high your volume may be, chances are you aren't dealing with the mega volume that Google has to worry about. Google reached a true milestone last week when it indexed its one trillionth page; a mind-boggling number that shows just what the company is up against when indexing the open web. That Google's indexing capabilities can scale to such heights is very impressive, but what really is amazing about this number is that it doesn't end there; instead, it just keeps growing (some say infinitely) into the future. So stop complaining about your company's content problem. Nothing is impossible, even a trillion pages.

To learn more about this milestone:
- check out Google's blog entry