Friday, May 7, 2010

Mapping your Records Retention Schedule

OK, so you had a consultant meticulously categorize and detail every type of record your company has and how long to keep them. The consultant looked at all of the paper documents for the passed 20 years which have been stored at Iron Mountain. This you thought is the source of all records, but did the consultant cross reference these “best practice” categories/file plan with the actual ECM repository(s) in use today? Did the RRS get vetted by a representative sample of business groups from the company? Is the RRS in a form that can be easily modified during the years it might take to implement it and the constant tweaks made to it on an ongoing basis?

Let’s assume that some of these questions were considered and that you have a reasonably well put together RRS. Now let’s look at the RRS to figure out how to apply it to an ECM application. Mapping the RRS columns of information may not be that obvious at first glance. The column headers will be generic like “record class name” and “customer record class code”. Which one should be what the Records Officer sees and what should the Users of the ECM see? The “code” is usually distinct thus it should be used as the File Plan number and Record Series Indicator. The File Plan corresponds to the folder or classification and the Record Series Indicator corresponds to the retention schedule object.

When you see the columns “Event” and “Period” these correspond to the status that triggers the status date and how long to keep the content in that state. For example, if the period states ACT+INA 6, this means keep the content if it’s active and dispose of it when the content is inactive for 6 years. The event could be when the content was created, or if the content is superceded. The schedule is attached to the Record Series Indicator and might have multiple schedules per series for ACT, INA, ARC, CLO, etc.

The “description” section might have extra information which could be useful for classification terms or record series types. It should also be added to the configuration to help describe both the File Plan and the Record Series Indicator.

Do you have content types and lifecycle states?
Doc type, status, and status date are essential to configuring your RRS. These may be obvious based on how the ECM object model was designed or how workflows push content through their states, however it assumes that all your content is processed this way and most important that it is consistently processed this way. A tall order for even the best designed repositories.

User Adoption
As you map and configure your RRS keep User adoption in the back of your mind. Think about seamless use for tagging records. Should all content be a record or just some designated by the Records Officer? My opinion is that all content needs to be disposed of eventually so all content is a record. There are different levels of records importance to the company and these will be retained, monitored, and secured accordingly.

Wednesday, April 28, 2010

ECM Acquisitions: the Paradigm is Shifting

To hear that Open Text bought Burntsand, a professional services company with expertise in MS SharePoint (MS Gold Certified) and EMC Documentum (Signature Partner), the paradigm is shifting... What is the strategy behind this type of acquisition? Take out the legs of SharePoint proliferation? Or, partner with the competition now to ease the eventual shift to lighter weight, federated repositories with agile, modular services of functionality of the future? Or, is this a Canadian acquisition among friends? It’s just strange at first glance.

As the Facebook-like mashup wars continue, this may be one of many interesting acquisitions. How can a traditional ECM repository layer be repurposed into a service layer that underlies SharePoint as a frontend? A fully integrated (not EMC’s SharePoint web parts, or OT’s CLM or Wingspan’s Docway) backend, maybe a “faster” performing CMIS? Hey all the pundits said 2010’s the year of CMIS…

One obvious synergy is the need for OT to integrate its whole suite of applications and solutions with SharePoint or get fully cut out of the picture in the next five years. EMC does this sound familiar? Did you hear the shock and awe salvo being launched when SP 2007 was released? Can you see the rockets heading toward Documentum as SP 2010 is getting hyped?

As these huge suites of software get split and repurposed, ECM as we know it will evolve into Enterprise Content Services, ECS. Content will remain as king, it’s the “Management” that will be replaced by agile services which when mashed up will be greater than the sum of its individual purposes. Plug and play like iPhone apps. What will be the platform for this: SharePoint.

Monday, April 26, 2010

Some Tensions of ECM

Navigation vs. Search

Gerry McGovern says it well when he says that navigation is just as important as search: navigate first to a sub division of content, then search the index of that section. Some repositories have no rhyme or reason to their folder structures: governance and standards were nonexistent during the proliferation of folders and content. This happens when the software product was oversold and the budget for rollout planning and information architecture was pilfered for the sake of getting the ECM solution deployed and stable. The same groups who were negligent are now forming strategy groups to figure out how to reorganize the navigation and content, as well as fine tune search results. Sound familiar? The people responsible for the mess are now tasked with cleaning it up. Do they understand metadata standards? I hope so. Will they decide to reclassify existing content, migrate and tag content to a new repository, or design virtual folders? They’ll have to start somewhere.

Legal Preservation vs. Information Architecture

Understandably, the Legal department wants to limit the amount of metadata to as little as possible to reduce exposure to liability. The Records Manager wants to tag content according to preservation rules and schedules. The Users want to be able to find their content by typing in metadata values which they are accustomed using to describe their content. Doc Type, Area, Owned By, Modified Date, Reviewed By, etc. These are fundamental to all repositories, but Legal would prefer to cut them to the point of obscurity; least amount of description equals the least amount of risk.

Agile Business Process vs. IT Stability

Users, trying to keep current with their changing business processes, are constantly pressured to modify their workflows. Meanwhile, the IT department cannot keep up with all of the change requests in a timely fashion. This creates a constant battle of change management between the users and the supporters of the system. Support is typically under staffed to fully handle all of the necessary changes in an agile and stable way. Also, architects usually consider change management and support an after thought to the main design of the repository and integrations.

Friday, April 16, 2010

“Too Big to Fail” Resonates with ECM

The “Enterprise” part of ECM conjures up visions of comprehensive coverage of content, metadata, and economies of scale. Like our “too big to fail” financial institutions, some ECM systems are too large and cumbersome to deal with lack of metadata standards and content hoarding that exists in many large corporations. Can an enterprise afford to bail out a monolithic ECM system when it eventually fails? Sooner or later the system will need to go through a major upgrade or a migration to another vendor. Because the system is centralized, it will cost a lot of money and resources to plan and implement this transition.

Federating ECM
Let’s make a distinction here between having one monolithic repository from implementing agile repositories which are small and more focused on “Silos” of business unit functionality and “case” type processing. “Silos” of information have long been wrongly accused by vendor marketing of being the issue with inconsistent business process and integration. The real issues are lack of governance and enterprise architecture reviews of applications. This is why Sharepoint is infiltrating: it fills the need for business units to create their small silos and integrate them with overall standards of the enterprise. Of course the lack of governance and architecture with Sharepoint deployments will result in the same issues of “Silos” with a capital “S” as vendors have been successfully selling on.

Change Management
If change management is not considered up front during the design and rollout of the ECM system, the emphasis on standards of metadata and integration will be pushed out to later phases. As the ECM system fills with process and content without policing the expansion of content/metadata and process, the cycle of complexity of these “internal” silos of information continues. This comes to a head when the system bogs down or the centralized resources can not keep up with the business’s demands for new functionality.

The Cost Effective Way
Typically, IT will focus on hardware and software upgrades to make ECM faster or give it more capacity: add more machines to scale it, the biz will be happy with the speed. Hire offshore resources to build workflows. IT has to control costs by centralizing content management; enterprise architecture traditionally puts ECM in a storage box; metadata is controlled by silos of business. This is part of the overall issue with ECM. There’s a gulf of misunderstanding between IT and the business. Save money by consolidating, then blow tons of money in four years trying to upgrade or migrate. Does this make sense? “The cost effective way” is a myth with a short-term vision that ECM IT managers tend to by into.

Derivatives
I think of UIs and add-ons as the equivalent of financial instruments like derivatives. When a vendor introduces a new look and feel and different functionality, this is an attempt to create an illusion of a new application. But, in reality, the underlying information is still the same. A new UI cannot mask inefficient governance and lack of regulations. For example, as DCTM’s webtop was waning in popularity, EMC decided to create Taskspace. This UI was promoted as an easier way for a business power user to configure UIs to suit their process needs. However, Taskspace relies on sound information architecture and group/permission structures. If these aren’t implemented, a vendor will promote creating a whole new info architecture next to the exisiting ECM one. So now we have two different, competing metadata structures. Professional Services will promote the new arch as being for case management—leave the old one for regular content. Can you see where this is leading?

Default Swaps
In ECM, these are like assurances that previous failed attempts at governance and info architecture will be covered. If one project fails, we’ll buy the latest technology to swap it out on. Enterprise 2.0 will solve the current issues regarding lack of business agility and limited IT resources. Don’t think so. Application Default Swaps are common and only push the issues into the next cycle of gadgets.

Credit Ratings Companies
Lastly, let’s look at Gartner as the research companies that put ECM apps in their quandrants. They slice and dice the application’s functionality, technology stack, scalability, etc., but this obfuscates the issues underneath the hood of a company’s ailments. What good is the best enterprise search application without a consistent metadata and access control standard in place enterprise wide? Is any part of an ECM application research company’s revenue derived from the vendors? Indirectly? Hmmm.

Monday, April 5, 2010

eDiscovery Solution Drill Bits are Dull

After reading Reed Irvin’s 10 Steps to Litigation Readiness in the March/April Infonomics, I felt like I was glossed over with good intentions and a sense of sales urgency, but I was annoyed at the same time. When will Infonomics stop writing the steps and start dealing with real fundamental and complicated issues in enterprise content, process, and people? I know it’s easy to poke at articles like this which serve as an introduction litigation readiness and try to weave in experience as well. The issue I have is that it is written by a promoter of a certain product. The writing expands on the core functionality of the software and services which his company is trying to sell. It’s generic for the purposes of a “drill baby drill” mentality meant to sell services.

Here are my comments on the “ 10 Steps”:

First of all, where are some of the new prerequisites in this piece for effective enterprise litigation? Put an “effective” in front of all of these…
  • Metadata Repository
  • Enterprise Architecture
  • Enterprise Search/Index
  • Identity Management
  • Content Architecture Standards
  • Integration Services
  • Role based Retention Schedules
  • Enterprise Process Management

These requisites are essential to defending a lawsuit in court. Imagine investigating a crime, but not knowing how to drive or read a road map, you wouldn’t even know what type of crime occurred? Are the above not included because they haven’t been developed yet into the toolset? Or, because they are “too technical” for the readers? Or, not focused enough on litigation? I hope not. For example, eDiscovery products assume a great deal is working effectively with the above technology areas. Tell me one company that has IT running all of these initiative flawlessly…

Second, the big bucket, then smaller buckets approach is not addressed here because it negates a lot of the need to create an inventory, automate retentions and dispositions at a detailed level. If you don’t need to tag and control all of the insects in the jungle, then why would you need our tools? If you could start with a big bucket compliance policy and slowly identify and weed out the more sensitive content wouldn’t that make more sense? When you see an ant in your kitchen do you have to identify it to decide whether you are going to step on it?

Datamap: to make legal search easy, determine data source, figure out the who, what and why of content. This is all good, but needs a few more details of potential pitfalls to round it out. One huge issue is the legal department itself. Legal wants the map, but also has to make sure that vendor and the software being used is vetted out. This means that the process of getting approval to actually use the tools might take a lot longer than anticipated. Having an effective enterprise metadata repository, architecture and index/search in place would greatly benefit the datamap effort. And vice versa, the datamap would open the eyes of IT to the needs for enterprise level processes and frameworks.

In step 9, Reed mentions having a flexible process which is spot on, but there is no discussion of historical analysis of the data and process in order to develop a framework for change. It’s not enough to say there will be a periodic audit of the map and processes, you have to know what to expect based on the company’s cultural changes in the past, present, and future.

Step 2 is a good start but needs to be tested with scenarios. These scenarios are just as important as the policies. Using a Zachman framework may help with figuring out the details of the whens which are not covered here. In other words, what is the roadmap of this readiness? What are the different ways to roll this out? Also, access control policies and identity management are components of the policies which are usually left for the IT department to figure out, but should be detailed up front.

Getting buy in and executive support for Step 3 is right on. This is absolutely necessary and should be step one. However, the legal department as a sponsor may not have the clout needed to push all of the agenda through the organization, secondary buy in from governance groups would also be necessary. What would help tremendously is a previous litigation or a few drills that permeate through the whole company and highlight the need to get the house in order and more importantly save in huge costs associated with manual labor.

Step 4 and Step 7 are at odds with one another. On the one hand, know your users and on the other take them out of the process. Adding Step 6 (automating the process) here, what needs to happen is development of an automated process for preservation and disposition which is seamless to end users, but is very clearly understood and vetted by end users. One way of doing this is to create a pit stop mechanism which implements these policies at the integration level not the UI level. Also, if there’s an IT compliance department in your company then why not drop the process of applying retention there and add it to the other steps to adding storage to applications. A business partner within an enterprise has to justify adding storage and why not add it step to detail the retention schedule for the integration service?

Step 5, Build your team, should be a framework of roles not the actual team members which will form and disperse depending on the type of suit against the company. The in-house IT folks will need to shift priorities and delay other projects. This could create a wave of ramifications through the interrelated projects that are in progress.

The bottom line is that no one product suite or solution can possibly handle the myriad of issues involved with eDiscovery. The best case would be to develop scenario based approaches and frameworks with the knowledge that a hybrid solution will inevitably be developed. A comprehensive datamap could take years and is really an exercise in futility. The prerequisites listed above are necessary to get to the point where a map will make sense. The map will have to be a framework more than a blue print—the exact specifications will change based on the litigation at hand. Building integration services that regulate retention policies and control content processing will make dealing with the human intervention factor more straight forward and less of a burden on “repeatable” processes. Building and morphing a set of litigation scenarios all honing your processes to these will go a lot farther than buying one-size-fits-all software solutions. This may be where EMC’s Case Management and Microsoft’s Sharepoint projects collide into interesting hybrid (mashup) solutions.

Saturday, March 27, 2010

Why Financial Services and ECM is a Challenge

Most Financial Services companies large and small have really screwed up the ECM model. Years ago when they purchased Documentum, IT folks looked at the technology and installed the “docbase” server with HA and DR, and that was it. They are heavily into databases and the interrelationships of client IDs and Customer IDs, products and services, etc. IT looked around at the time and saw programmers, DBAs, and web tech guys and thought we’ll build our own frontends to Documentum, our permissions and rules for metadata are too complex, too fancy for the WDK. This was partly true as EMC Professional and Product Services never caught up to the specific requirements and functional needs of the finance industry as they did with life sciences.

Typical scenario at a Financial Company

The client applications director has been somewhat satisfied for the past four years with the way IT has maintained custom web applications integrating with Documentum on the back end. The director is getting more heat from above to look at cheaper, OOTB Documentum solutions. The IT department development (even outsourced to India) is getting too expensive. IT has been planning and executing an upgrade for the past year and a half. They are still at 5.3 (not supported) for the repository and are upgrading the website DFCs to 6.0. Why not 6.5? Because at the time of planning 6.0 was just coming out and the client development started. This is way too expensive and lethargic, but what are the alternatives?

BSAs with their head in the sand

The Business Systems Analyst Manager, with his MBA says we need to think about the big picture, but deal with the low hanging fruit now. In other words, there’s no bridge in communication between operations and IT. This is where a Deloitte is usually helpful in terms of getting the too sides to agree on long-term goals. I know consulting firms can be devastating, but in this case, all the MBAs need to get together and show off their power points. It’s not about just fixing one issue then the next. It’s about getting the CIO to wake up and figure out ECM should be handled like it’s ERP. There is usually a disconnect between the Client App Business Ops vision and the IT technical vision. This will not get fixed at the director level, as much as the C’s would like it.

IT is RESTful with their plans

Historically, the IT Internet/Extranet group has run the show. Choosing how to architect the ECM solution as a by product of their database and web application server architecture plans. Just figure out the storage of content and keep it forever. Here’s the argument: we already have custom applications for Documentum that work. All they care about is the “harvesting and production” side of content. They have no clue what the business is actually doing. How do we get IT to feel the pain of the business? A strong Client app director who cares about business process and content management would help. Get the IT managers out of the web services cloud and either do something enterprise wide with CMIS and other integrations or work with the OOTB products.

EMC Professional Services Where Are You?

The problem with EMC Documentum professional services is that it is not engaged fully during the pre sales activities of the sales engineers who are building POCs and demoing new software solutions. If they are they’re spread too thin and are not effective enough. They can’t keep up with actual work let alone consult on goals and vision. Most companies form their opinions on the can do all sales guys. The client is overwhelmed with products not solutions. They see scanning solutions, but not high end UIs which solve the complex financial relationship issues. A WDK or Taskspace form is a child next to the maturity of what IT has created for their highly customized UIs. EMC, what’s the solution here?

Sample Financial Solution Patterns

There are many solution patterns within the financial applications that EMC could solve with a product.

  1. Center a User Interface around Client and Customer Ids. The pre vs. post sales information gathering activities.
  2. Deal with the historical changes in attributes and their values. Financial services databases are always current and don't necessarily handle change management in content and metadata very well.
  3. Like the case management solution, figure out the foundational use cases of financial services (beyond scanning and OCR) and build product and solutions from them.

Monday, March 22, 2010

eDiscovery’s Promise Is Dependent on Your Enterprise’s Laziness

eDiscovery solutions are predicated on your company’s lack of cohesive and best information management practices. If you read any eDiscovery software solution’s proposal it will not mention how to fix the issues of inadequate metadata structures, lack of auditing, risks of pre-ediscovery spoliation. They will not tell you to get organized and clean house. They’ll say information governance is impossible to keep track of, that information growth is out of control. They’ll all be lawyers and MBAs and be able to pontificate extremely well about your issues, but do they really want you to help you where you really need it?

If I was selling software they was really a search tool and a legal CYA process would I be interested in telling how to create an Enterprise Metadata Repository? Do creditors want you to save money? Managing your content requires standards, conventions, governance, and compliance. MoReq is an EU set of standards for records management that basically outlines the keys to eDiscovery freedom.

Without the discipline and perseverance that is necessary to push these throughout your company you’ll be stuck with an eDiscovery application infiltrating your email, ECM repositories, file shares, etc. It’s a matter of how you want outsiders to carte blanch access to your information. If you know where everything is then you’ll be able to clearly define and execute the information discovery without a whole sale fishing expedition on all of your seemingly sensitive information. Most importantly you’ll know what needs to be disposed of!

So what’s it going to be a house of hoarders, or an extreme home makeover?