You think you know what a record is?
Guest post by Kon Leong, President and CEO at ZL Technologies
We've all heard the proverb about things that look and quack like a duck, but sometimes things aren't quite what they seem in the world of documents and business records.
In today's world of records and information management, or RIM, this predicament is more evident than ever as we grapple to comprehend the information hatched by the digital age. With the proliferation of electronically-produced material, sometimes a duck just isn't a duck anymore. Or, more succinctly, it has become much harder to discern which documents are relevant for RIM.
Yet, for a long time, the traditional hard copy methodologies have remained our mainstay for records management. It was easier to determine what needed to be retained as a "record" and what was simply a byproduct of the business process. If it looked like a record, it probably was. We made a habit of counting and lining up our proverbial ducks. It was easy to determine what should be filed away because "important" information might be called upon in the future. Businesses mostly came up with their own policies and stored hard copies in basements and offsite. The ducks were all in a row, at least temporarily.
But trouble began to surface in the early 1980's when computer-generated documents and word processing triggered a proliferation in potential record-worthy materials. Enterprise Content Management solutions were developed to save the day, and provided digital depositories for specific categories of data.
Then, the Internet changed everything. With widespread adoption by the mid-1990s, there was a surge in data, such as email, which cascaded down and threatened to overwhelm ECM solutions. Vendors and users were caught off-guard by this sudden tsunami of data and devised temporary shortcuts to circumvent scaling. Human judgment was used to compensate for lack of scalability by having end-users manually select and tag records. Problem solved ... but not really.
The inevitable byproduct of the human work-around was that human capital was utilized, with all its inadequacies. The cost of HR rose, and the decision to use humans for highly-repetitive tasks came with inherently human mistakes such as bias, inconsistency, insufficient training and apathy.
By the late 2000s, it became increasingly clear that ECM's shortcuts of using end-user participation for categorization were destined to collapse under their own weight, for multiple reasons:
- Data volume: The amount of data spawned by email alone had grown to two or three magnitudes of order larger than anything ECM solutions had ever been built to handle.
- Variety of data types: The assortment of data formats had expanded to a much larger array, including not just files and email, but also IMs, social media and others.
- Changing Scope of RIM: The basic definition of a "relevant" record came into question, with the debut of "non-records," which fell outside definitions of established business records, but which were, nonetheless, important to business.
- Raised stakes: Regulatory agencies such as the SEC enacted retention requirements for managing digital content. Then, in 2006, the Federal Rules of Civil Procedure gave further evidentiary weight to electronically-stored information by issuing guidelines on electronic discovery. The combined legal and regulatory assault expanded the depth and breadth of digital records keeping.
- Ad-hoc silos: These problems spawned attempts to create temporary, ad hoc solutions, which in turn created data "silos." The rise of silos caused--and still cause--intractable difficulties, for simple reasons. Silos proliferate duplicates. Silos search inconsistently since each silo employs its own search engine. Silos have mismatched retention since each silo has unique retention capabilities. Combined, these problems result in the loss of data control. Silos currently constitute the biggest obstacle to digital content management.
Today, industry gurus are starting to opine on how to move forward. Even stalwarts such as the National Archives and Records Administration have started to question the ECM system of crutches and work-arounds, especially end-user classification. More and more are joining the growing chorus of voices that recommend taking end-users out of the records management business.
The changes have just begun ... and promise to continue.
Technology advances may change the nature of the challenges ahead. As natural language processing and text analytics improve, the potential of unstructured data is attracting attention from top management, especially for strategic and competitive advantage. And yet, paradoxically, concerns about legal exposure still prompt many to purge data that they consider "unnecessary." Exposure is minimized, but so is potential for insight. The question of whether we're throwing the baby out with the bathwater remains. Strategic value of unstructured data is just beginning to be explored, but effective management today is the key to extracting value further down the road. The changes triggered by a new generation of analytics could subsequently trigger another wave of changes in managing future content.
Figuring out the right approach now is critical. With a long-term problem at hand, a misstep today can mean a costly re-direction in the days ahead.
In summary, we live in a world where knowing a duck when we see one isn't as easy as it used to be. Since emails and various non-traditional methods of digital communication have gained use in the workplace and are still increasing exponentially, suddenly a duck isn't a duck anymore. Now, geese and swans can be ducks too, and trying to individually sort them looks increasingly like a wild goose chase. The argument over what constitutes a true "record" is a dated concept, as regulatory and legal needs deem a far wider swath of information as relevant. The true challenge is holistically organizing all digital communication in a way that is accessible, useful and sustainable for multiple stakeholders within an organization. They include the departments for IT, Compliance, Legal and Records; operating business units; and top management. To design a long-term enterprise content management solution, the voices of these stakeholders have to be heard.
Perhaps the question isn't whether it's a duck, but rather what we should be doing with the quacks.
Kon Leong is responsible for managing all aspects of the business--including strategy, finance, sales and marketing--at ZL Technologies. Earlier, Kon was co-founder and president of GigaLabs, a vendor of high speed networking switches. Prior to that, he was First Vice President of Mergers and Acquisitions at Deutsche Bank. He was also at the General Motors Treasurer's Office in New York City, where he managed GM's venture capital investments in high tech.