One on one with Paul Doscher, CEO LucidWorks

Big data, cloud changing enterprise search

LucidWorks (formerly Lucid Imagination) CEO Paul Doscher has more than 30 years experience in the enterprise software business. Prior to joining LucidWorks, he served as CEO at enterprise search vendor Exalead from 2008-2011. We asked Doscher about the changing state of the enterprise search business.

FierceContentManagement: As we deal with increasing amounts of data beyond pure office documents, how has enterprise search adapted?

Paul Doscher: Enterprise search has adapted dramatically over the past years. Many people think of search as executed through the use of a simple user interface and predicated on keyword matching. Such search results are static and lack intelligence. These types of search solutions remain relevant and appealing to certain enterprises. And yet, with the rapid explosion of text messaging, emails, video, digital recordings and smartphones, the amount of data that companies need to access and understand has grown monumentally and continues to mount. We see enterprise search extending well beyond the basic functionality to deliver results to business users that are meaningful and actionable. Key to accomplishing this is the interfacing with, or inclusion of, business intelligence and analytics functionality.

FCM: How do you index data outside the firewall such as content on cloud services?

PD: LucidWorks Search is able to access cloud services and data managed by cloud solutions once that solution has opened a port enabling us access.

FCM: What impact is big data having on enterprise search?

PD: Enterprise search has become an important enabling technology for big data.  A key benefit promised by big data is the opportunity to find new insights about customers, operations, events and such that up until now has been buried within large unstructured data files and stores. The only technology that is capable of extracting these new insights is search (also known as enterprise search). Some enterprise search products have evolved their feature sets to provide key value store capabilities. When placed alongside of Hadoop, these search systems provide a very high speed, almost real-time retrieval system for data stored in Hadoop File Systems. Further enterprise search is able to provide retrieval across more than one data store in a single query, supporting a much broader view of the information.

FCM: What are the relationships between analytics and search?

PD: Search is the way information is extracted from multi-structured data stores--how the data is retrieved. Analytics is the process of assessing the information retrieved via search. An important step that is missing in the question is Discovery. Search and Discovery combined is an iterative process where results of a search become the input for the next search. Machine learning plays a role in this process by learning what an individual is searching for and making recommendations to broaden the search to provide insights on the data that would otherwise have been missed. These insights are the bases for informed data-driven decisions.

FCM: What role does search play in customizing a website visitor's experience and helping companies understand their customers better?

PD: Capturing a prospective customer's search activity (click-stream) in real-time enables ecommerce companies to understand what that prospect is interested in at that moment. This knowledge allows them to push product ads to the customer in real-time. The technology and architecture of real-time ad placement is highly complex and requires many cooperating technologies, but it all begins with search. Enterprise search companies that offer a feature called faceting allow products to be grouped logically within a side navigations to allow customers to find related products quicker.  Relevance allows the system to push specific products up in the search results based on what the customer is searching for at any given time.