Composer and ACHells

Hosed by Composer again! This time trying to install Permission Set from Dev to Test Repositories:

Environments:

• Dev Repository name: LoserDev
• Dev Repository Domain User: LoserDev
• Test Repository name: LoserTest
• Test Repository Domain User: LoserTest

1. Created a Permission set in Dev, which set the ACL domain as LoserDev which corresponded to the LoserDev install parameter in Composer.
2. Installed it into LoserDev and everything unit tested fine.
3. Went to install the dar file to the LoserTest repository and got an error: “user LoserDev does not exist in Repository”.
4. ** Don’t be tempted to create the user in the repository. This will allow the dar to install, but will really confuse the UI with ACLs that kind of work, but not really.
5. Went back to the LoserDev Project and opened the LoserDev install parameter, typed in “dm_dbo” into the default value box, saved it, and created another dar.
6. Went to install the dar file to LoserTest: same error. What the?
7. Went back to LoserDev Composer project, check the LoserDev install parameter and the default value was blank. Hmmm.
8. Type the default value into the user parameter value of dm_dbo again and hit the enter key. Ahha! Saved it, created the dar and the install to LoserTest worked.

Bottom line: In Composer make sure you have an asterix * in the tab, to guarantee that your work is getting saved to the underlying xml file which is used to create the dar file.

Documentum Composer Wrestles with Lifecycles

Hosed by Composer again! Whoever thought Composer was ready for primetime with Lifecycle management was really in the clouds. Here’s what happened:

• I created a Composer Project (call it Poser) and a new lifecycle (let’s call it DOA) along with many other artifacts.
• I installed the Poser.
• DOA had issues with ACLs and was not working correctly
• I thought it would be better to create a delta dar for modifying just DOA
• I created Composer Project 2 (Hoser) and imported DOA from the repository
• I fixed DOA and installed Hoser.
• Now Webtop showed 2 DOA lifecycles.
• Naturally I deny that anything wrong is happening and I choose the wrong DOA, try it and get frustrated.
• I DQL, I look at ACLs, I search Powerlink, I download Doc App Builder 5.3sp65
• So I go back to Hoser, import the original DOA and click the “uninstaller” checkbox to uninstall both DOAs.
• I install the Hoser again.
• Now Webtop showed the original DOA still installed…What the? What got uninstalled?
• At this point I installed Documentum App Builder, created a docapp, imported the lifecycles and uninstalled them. I fixed the DOA and had no other issues. This is still the true work horse!

Bottom line: once you create a lifecycle and actions, stick with that project, don’t create new projects using the original artifacts.

Content Architecture using Memetics

Virus of the Mind: The New Science of the Meme by Richard Brodie, describes the foundation of memes as being distinctions, associations, and strategies. When applied to Enterprise Content Management there are some interesting comparisons. Advertisers know how to push our buttons to drive sales, just as content architectures know the best ways to describe and find content, or do they?

Distinctions

Example: It’s snowing, or our content is all on a share drive.

These are the ways to describe content that are particular to the business unit, the company, and the industry. This metadata is vital for survival of the content, in other words, can a User find it among thousands or millions of other pieces of content? What key information can be drawn out of the content file or its context to direct successful search result?

Associations

Example: Snow is dangerous when driving, or without metadata I get thousands of search results.

Relationships among content and its environment are key the understanding the thoughts (memes) behind the content. A taxonomy helps by categorizing a business unit’s way of thinking for its search or retention purposes. This taxonomy would have to fit into the enterprise as a whole. The issue here is to start at the level where the content is created and is useful to the local users, then expand the levels out in a way that doesn’t disturb the functional aspects of the original group. Too much of an imposition will get rejected or worse slowly ignored.

Rules and Regulations come in to play for controlling and focusing content for common delivery to people and interfaces outside of the company’s mindset.

Fuzzy vs. Absolute: Users want to be able to fill out metadata and find that exact content later. This means the content architecture has to balance the business unit’s requirements with the enterprise's.

Strategies

Example: If I have an all wheel drive I’ll make it through the snow, or with a taxonomy I can make sense of complex organizations.

Repetition: this is used to drill home the importance of certain ways of thinking (memes). For example, a naming convention will reinforce ways of thinking about content and its context (association meme).

Cognitive Dissonance: this is used to reward a User for taking the time to fill out metadata correctly. For example, filling out metadata and associations is rewarded with less change management, less hassle in the future when ways of organizing content changes.

Content Silos

However you want to attack the issues of content silos, they will always exist. The strategy memes of the business unit will always differ in meaning and scope from the enterprise. I’ve come to the conclusion that the best way to make importing and/or changing content the easiest on the User is the hardest to figure out in terms of scale and performance on the system. This means that finding the right balance of splitting up the system’s resources for each business unit weighing the content demands, the ability to find content, the access control, and the application of the latest rules and regulations. This balance when seen visually will make sense, but the challenge is to get agreement from all the parties involved, the governance. This is where strategy memes make inroads: they help far removed executives understand the long term benefits when seen from the past, present and future.

Why Test Scripts Suck

I’ll trade one motivated business user in for five IT testing professionals for making sure an application works as designed. Why? Because that business user has all of her day to day requirements, pain points and frustrations invested into the new application. On the other hand, the testing professional has to make sure the test script is executed flawlessly, that’s it, on to the next project.

Below is the typical best practice, software development mantra that project managers will promote. I’ve added some notes under each of them and few new mantras.

The software implements the required functions.

Have the requirements been aloud to change during the project? Flexibility is key to the perceived success of any project. If concessions can not be made without huge push back and it’s a pain to change requirements from the business’s perspective, the project should be stopped and re-evaluated for its purposes (and management). This is very pronounced with large projects.

Added to normal Project Manager’s software development lifecycle list:

Prototyping

Prototyping functionality for business users to experience (see) what’s been talked about and promised. If a third party or internal team cannot commit to show their application during development for fear of “giving a bad impression” or “scaring” the business then there are trust and lack of communication issues going on that need to be dealt with now rather than during the full User Acceptance Testing.

The software has passed unit testing.

Make sure the developers know the architecture of the application as a whole, its requirements and the importance of the unit testing and integration into the larger application or service. If one developer is slacking the project is at risk of failure. Project managers should have a good idea of who can perform and who needs help at this point in the project timeline.

Added to normal Project Manager’s software development lifecycle list:

Code Reviews

A junior developer cannot possibly know all of the ins and outs of the application if they are focused on coding specific components or services. All code, at least initially, should be reviewed by senior developers and architects to assure efficiency and scalability.

The software source code has been checked into the repository.

This can be a pain in the ass if the project is small, however necessary if you are developing with others and integrating into a larger repository. This also is a good check for senior developers to do quick code reviews.

The software has been compiled into the current build (for compiled systems) and deployed into the appropriate test environment.

Without proper safe guards, one developer’s code could break a whole series of other test scripts. Smoke testing is highly advised before fully committing the code.

The team has developed appropriate test scripts that exercise the software's capabilities.

These scripts are usually end to end tests that are few clueless testers, not irrational business users, who change their minds, cancel, go to lunch, upload their whole hard drive, etc.

The software has passed integration and system testing, and been deployed into the user acceptance test environment.

Many large projects are desperate for true testing environments are usually skimp on resources for them. This poses an issue when the new build is supposed to be deployed and fully functional in a test environment that has kludges.

Added to normal Project Manager’s software development lifecycle list:

Performance and Scalability Testing

Normal third party developers comment during the development phase of the project that they wonder if this will scale or how it will perform under a load. These developers talk to the project manager and usually the discussion ends there. If this is brought up at this time in the project with no time allotted to it then forget about it. Also, why would the third party be motivated to do this when this is a typical reason that they get called back in to do more business?

The users have had an opportunity to use and respond to the software, and their change requests have been acknowledged and implemented where appropriate.

Again, this is important, but there should be no surprises at this time. The users should have had their requirements changed and prototyped and testing by this time.

The software has been documented in accordance with whatever standards your project follows.

Have you ever seen a test script written for documentation accuracy?

If documentation is not ongoing during the whole project this document will be worthless. I have not worked on a project where the design document is perfect after being signed off on. During development and fixing, the design doc needs to be corrected, changed, or expanded on.

Also, the deployment and knowledge transfer documentation should be complete and tested.

What the F’ Happened to the Customer’s Vision?

It seems every new technology or architecture or new way of looking at the complexities of content is like building a new platform on quicksand. It eventually sinks below the surface and then a new “genius” comes along with a solution that gets sold to our “shock and awe” addicted users.

The customer used to always be right, now they are sold what’s “right”. What is sold to the customer is pretty and “easy-to-use” technology which is over their heads. They become reliant on experts to build the solution and to come up with language that makes the Manager/Director look good to his superiors.

Once ECM is in place, the users look at it and inevitably want their old system back. After a while they become more comfortable to the new ways of doing things. Then they want continuous improvement. By the time this happens a new version is released, new bugs cause the experts to come in and fix them. The continuous improvement requirements get scaled down by technology issues of performance and content growth. The IT department thinks they own the system. The Business Units get frustrated with IT. Yada, yada, yada.

At this point “shadow IT” starts its cycle again. In the late 90’s it was websites popping up every where as intranets via easy to use, inexpensive website publishing tools. Now it’s Sharepoint portals. These portals are what the customers want. They want messy rooms (unstructured content spaces) where they can play with content and ideas, not technology. Metadata, security, taxonomy, workflow, lifecycles, retention, etc. need to be worked into these “messy rooms”, periodically cleaning them up, organizing the useful content and throwing away the building blocks. EMC’s CenterStage, like Sharepoint, is trying to fulfill the need for users to produce, edit (collaborate) content while the systems handles structuring and storing behind the scenes.

This introduces the big gray area of ECM: the void between structure and unstructured content. Let’s say an invoice is structure content because it originated from a database and has a number. The problem is that this invoice was printed to paper, signed, scanned, and place back into a content repository. The number is still there, but the systems are different. Even though the systems are interoperable there is no source of record anymore. What is more important the financial aspect of the content or the actual scanned proof of purchase? It depends who you ask.

Solution Pattern for OOTB Webtop and Search

Scenario

• The client requires a multi-tiered custom object model with most of the attributes at the child level.
• There are 10 distinct types of content which share some common attributes, but have very specific attributes as well.
• The client want to search for attribute/value pairs across all of the children documents easily by selecting one parent object, typing in search criteria, and executing.
• All imports, checkins, new templates, and properties interfaces need to be tailored to the client’s specific requirements for conditional attribute population in specific order.
• All content types have some mutually exclusive required attributes that cannot be null.
• And of course, the client does not want a lot of Webtop customization.

Possible Solution Routes

• Traditional: Customize Webtop import, checkin, new doc, properties, and search pages.
• New: TBO with common attributes pushed to the parent object type, with limited WDK customization.

Design Road less Traveled By

We isolated the absolutely necessary WDK components that needed to be developed to satisfy display and functional requirements. There is some functional requirements work here to decide on which attributes are common and have the most impact finding content in the repository. Searching across all child object types was a critical requirement so we focused the design of the object model toward shared or common attributes. The design of the object model looked like this:

Parent (common_attr 1, common_attr2)

Child 1(common_attr 1) --- Child2(common_attr 1, common_attr2) --- Childx(common_attrx)…

Every time a document is saved, the common attributes of that child doc type is replicated to the common attributes of its parent. Besides display customization of search results and grid columns, there is no need to customize the query builder of the search page. The query performs better because the attributes are at the parent level, there are no database table unions happening behind the scenes during the execution of the search.

Developing the TBO

The main purpose of the TBO is to override the Save and Checkin methods that are triggered during the use of Webtop. This TBO basically gets the common attributes of the content being saved or checked in and sets their values to the parent attributes. This TBO is deployed to override the parent object type.

Developing the Webtop Components

The customizations are to the object and doc lists. The “documentum.webtop.webcomponent.objectlist” and “doclist” components have the common attributes added to them in order to show the same view of the attribute columns for browsing folder contents and search results. This could have been a preference setting, however there was a bug with the particular version at the time in terms of sorting the columns so we had to customize these. The “search60” component was also changed to show only the parent object type and its children object types in the search.

Max Session: Obscure Documentum server.ini key saves the day!

Environment
Windows D6.5 installation using Webtop, some WDK customizations, and a TBO for major customizations. The TBO has to create a superuser session to do some work.

Issue
Max sessions are reached on the application server after a few hours of use even though the “Active” sessions are far below the configured threshold. In other words, the Tomcat server is counting “Active” and “Inactive” sessions in it determine of “max” sessions.

Keep in mind that there is a lot of customization in the TBO and this required creating and releasing a session manager for a superuser during each save of the document. This is what is building up the "Inactive" sessions. 

Patch Solution Given
The initial solution was to jack the max sessions up to 100,000. This caused the Tomcat service to die once a week or so, basically maxing out the memory allocation to the process.

Max Sessions Investigation
I opened the Documentum Content Server Administration Guide and searched for “max” or “session”.

I listed the variables in the session being maxed out

• Application versions
• Property files
• Custom code

I looked into implicit and explicit sessions

Hint: For testing superuser in TBOs, I used a different superuser than the install owner account.


Application Server

• Measured by Active and Inactive Sessions
• dfc.properties Key/Value Pair: dfc.session.max_count = 1000 (default)
• DQL: execute show_sessions
• DA: Administration > User Management > Sessions (All)
• web.xml: HTTP session timeout is set in the \app server\ web.xml (default is 30 min): 30
• Hint: To find leaks set dfc.diagnostics.resources.enable = true (default is false)

Content Server

• Measured by Active Sessions
• server.ini Key/Value Pair
• concurrent_sessions = 100 (default is 100, max is 1024). These sessions are “Active” sessions from the content server’s perspective
• history_sessions = (how many timed out sessions show in list_sessions)
• history_cutoff = (default is 240 minutes)
• client_session_timeout: default is 5 min
• check_user_interval: frequency in seconds which the CS checks the login status for changes.
• Default is 0, meaning it checks only and connection time.
• login_ticket_timeout: length of time a ticket is valid, default is 5min
• DQL: execute list_sessions
  
  

Final Solution

I added “history_cutoff = 5” in the Content Server’s server.ini file.The “history_cutoff” key controls the longevity of the inactive sessions. The default value of this key is 240 minutes (4hrs). This would explain why only on occasionally the max session is hit.

My testing has shown that if you set the “history_cutoff” key to a value much smaller like 5 to 30 minutes, that this allows for the inactive sessions to clear reasonably soon, so as not to fill the max sessions of the Tomcat server.

To test this I set the following:

Set up WDK Automated Test Framework to run the same tasks over and over again to build up Active and Inactive sessions.

Set up the Content and Tomcat servers with these base line settings:
server.ini file: concurrent_sessions = 20
dfc.properties in the webtop/WEB-INF/classes: dfc.session.max_count = 30

Result: The Tomcat server fails when the total of Active and Inactive sessions exceeds 30.

Then set up the Content and Tomact servers with these settings:
Settings with history_cutoff changed
server.ini file: history_cutoff = 5 and concurrent_sessions = 20
dfc.properties in the webtop/WEB-INF/classes: dfc.session.max_count = 30

Result: The Tomcat server fails only if the number of active sessions exceeds 20, thus relieving it of the inactive session burden.