WP installation and configuration: This can be painful.

First of all there’s very little detail in the installation manual on how to actually configure Webpublisher to work correctly. For example, here’s how you first set up a template:

1. To import your first Template file, first you have to know what an example template file should be, then you have to know that to import you must create a new “category” whatever that’s supposed to be, go into that new category and then import. Patched together functionality or what?
2. To edit your first Template, you need a rules file. So you go to the “Rules” area and create one. You go back to the Template, check it in (because it was checked out when you to edit it the first time), View/Associations and click on Rules, and Add the new Rule. Is this intuitive enough yet?

Second, once you’ve installed the WP, you have to install and configure Site Cache Services (SCS). Configuring the website publishing using Documentum Administrator is relatively straight forward, however, when it comes to figuring out what happens during a SCS publish, you’re kind of on your own. One thing to check is the SCS configuration in WP. To do this, click on “Web Cabinets”, select the cabinet that you are publishing, and click on “View” and select “Web Cabinet Overview”. This should show you the publishing configurations that were configured earlier using DA. Here’s what happens during a Site Cache run:
1. Either the SCS job executes or WP promotes a doc to Staging or Active which invokes the dm_webcache_publish method.
2. SCS service reconciles modified files with the configuration and the source SCS tracking table.
3. A properties.xml file is generated and sent over to the target SCS server.
4. Content is transferred via http or https to the target server.
5. On the target, when all of the files have arrived, target SCS processes database inserts and deletes if there’s a database, it expands zip_html files, it copies the files from the temp directory into the live website, and much more.

File extensions: htm vs. html
The first time you site cache the html renditions you’ll notice that the files are published with the “htm” extension. To get the files to publish with an “html” extension, you’ll have to make changes to the webcache.ini file located on the content server in the “/Documentum/dba/config” folder.

exclude_formats=xml
use_docbase_formats=true

You’ll also have to change the extension of the html format using Documentum Administrator. It took me a day of part-time rummaging through the forums to piece together this configuration fix.

There will be times when you want to throw up your hands and give into a beginner WP class, but by investigating this issues that arise first by browsing the forums at EDN and Bluefish and other site, you’ll learn as you go and more importantly remember what you did when you need to configure WP again.

WebPublisher Business Requirements: Who Are We Writing These For?

If someone tells you that Business Requirements are not that important to your role as a developer, ask them what happens if you need to make a change to them? Get involved in this phase if you can because BRs set the tone for the whole project, and you need to meet and get to know all the team players. Most developers get pigeon-holed into roles that suggest that they just do what you tell them to do; follow the specs… It might feel comfortable to do this, but you’ll always feel frustrated during the process, especially if you need to push back on the BRs.

Know Webpublisher
As you sit through the Discovery phase (if you get invited) you’ll gather incite into the issues that really matter for the project and who will be looking for what functionality. What happens during the business requirements writing phase is that everyone becomes an “expert” in Webpublisher. The requirements reflect what the business thinks WP can do OOB, not necessarily what it’s potential is. To be fair, the main objective is to use WP OOB as much as possible, but also to find the middle ground in terms of configuration vs. customization.

Consultants Who Know Everything
If there are outside consultants, especially those newly graduated from our fine Ivy League schools, they will commandeer the discussions about requirements and do breakout sessions to fine tune their own understanding of the requirements and their skills in Excel. Business school graduates are fond of Excel and can achieve incredible feats in expanding BRs to new levels of complexity. These are well thought out, but are usually overkill for OOB WP functionality. Make sure the consultants know WP functionality inside and out before they start writing BRs. The more the requirements grow in size and complexity the less likely that the business users will read all the requirements and sign off on a document they can completely grasp.

Think about the whole application
It’s easy to get bogged down by concentrating too much on the details. For example, there were many times when we drilled down on the requirements only to realize that it was close enough to OOB functionality that we didn’t need to write it all up. By “whole application” I mean:
- What’s the end to end flow of content through the system?
- Who will access the content and when?
- What level of support can be sustained?
- How much training will be needed?
- Do you have enough resources to satisfy all the requirements?

Whiteboard the end to end flow
One absolutely necessary step in the requirements process is to draw out the end to end flow of content so that everyone is on the same page. This flow diagram will serve as a milestone of the group’s understanding of the application. It will be altered and republished, but the core flow should not change. This flow should have a combination of technical and visual elements. It should be of use for the business side as well as the IT side. Everyone needs a visual representation of the application and now is the time to create it.

There will be times when an issue can not be resolved by the people within the team. This is the time when you spin off pros and cons of the possible solutions to the problem. This is also the most frustrating time of the project, when egos are flying and a lot of time is wasted. A few types of issues I’m talking about are:
- Configuration vs. customization
- Integration vs. manual data entry

Configuration vs. Customization
It’s easy to get excited about developing new functionality to WP mainly because WP has many foibles that are usually corrected during the first release of the project. However, you have to weigh customizations with wanting a smooth upgrade path. One of the WP projects I was a part of was so heavily customized that management waited until the version was no longer supported and then had to plan the upgrade to which was scheduled to talk over a year to accomplish. This type of customizing indulgence is usually avoidable if the right balance of Configuration and Customization is achieved.

The typical pros and cons of these issues involve:
- Cost
- Length of time to develop
- Testing
- Functional worth
- Priority
- Impact on upgrading

Integrations
There are all sorts of integrations that you can do with Documentum, the one I’d like to talk about is one with Documentum Compliance Manager (I’ll carry this example throughout this series on WP). The pharmaceutical company I worked for wanted to be able to run certain regulated web content to DCM. A content flow diagram was created and a prototype was developed during the requirement writing phase mainly because no one knew for sure if integrating WP with DCM was even possible. By architecting the solution with a TBO we were able to integrate with DCM without disrupting any of DCM’s functionality.

“Nice to have” vs. an absolute must
When you have the business users in the room they will say everything they bring up is necessary. But, when pressed, they’ll start to admit that some functionality is “nice to have”. WP offers many features that can be configured to get close to most requirements around publishing to a website. I’ve migrated from a custom Documentum web application to WP and the biggest complaint was the amount of versioning that goes on in WP. They also had very complex ideas on how to organize the content. These ideas carried over into WP as categories. WP categories are versatile and powerful when used in the right way, but, in our case, we went way overboard. We had over 10k nodes, multiplied by 10 locales. This was and continues to be a nightmare.

Two thorns of WP publishing
There are two issues that plaque WP: Security and Navigation. Customizations gravitate toward these two monsters. Why hasn’t EMC purchased or built a web server / app server yet? Sitecache is nice, it’s decoupled, it scales. However what happens when the content gets to the website? The coupling of the decoupled security and navigation is a sink hole for bad performance and enormous support issues.

Discovery: do they really know what they want?

As a Developer, depending on the size of your company and how enlightened the management, you’ll most likely get involved with the Discovery phase of the project. If you can try to influence who’s on the Discovery team by suggesting other resources which you trust will “do the right thing” try to do it before the first meeting.

Don’t solve the issues: listen
The first few meetings during this phase are spent listening to the Project Manager(s) go over what they believe are the questions that should be asked and the corresponding answers from the Business Manager(s). Developers: don’t try to solve these complaints in the meeting, just listen and answer questions when asked. Be careful not to ask detailed questions at this point, as the discussion will dive way too deep into the nitty gritty and could get religious. The managers need to hear themselves talk. If you create too many disruptions at this point, they’ll write you off as a pain.

Know OOB functionality cold
I’ve been in situations where I knew the functionality being asked for what hard to implement, but wasn’t 100% sure and couldn’t back up my gut feeling with proof. If I had known WebPublisher OOB functionality better I would have been more effective in arguing my point. One way I dealt with not knowing the answer to a technical question was to suggest that I prototype the functionality and get back to the team with an answer the next day. This allowed me to get proof (or not) of my gut feeling and work through possible solutions without getting shutdown immediately. This also saves lots of time and effort in the long run by exploring the ramifications of a small decision which could balloon out the project (80x20 rule).

Where does the content come from?
The content of the website project was already designed and developed, but was being hosted on a tradition web hosting site. The website content was FTP’d to the site and manually published. The users wanted “everything we have now” and more. The “everything we have now” was a black hole which needed to be dealt with asap. What can you do with a statement like that without going through a huge process of requirements gathering? What we came up with was punch list of what WP does and presented it to the business.

Lumpers and Splitters
On the project team, there will be people who like to drill down on every piece of functionality, not only to understand it but to drive others crazy. They like to split issues into smaller pieces and understand them better that way. Others like to lump issues together and hand off the large understanding to those who understand the software (WP) better. During each meeting the lumpers and splitters will go at it. If you can catch this happening before it gets too heated, call for another meeting to hash out the particulars. It’s important to keep the focus on manageable chunks of the project.

Who’s poking holes in the boat?
During these meetings, at first it was hard to figure out who was aligning with whom in terms of making the hard decisions about what functionality was in and what was out. Over the course of a few meetings, it became clear and it also was obvious who was fully invested in the project and who didn’t care. The folks who didn’t care reacted by saying things like:
- “Let’s cross that bridge when we get there”
- “We don’t have any expertise in that area”
- “We don’t have the time or money to do things the right way”
- “Let’s get a consultant in to look at this, I know someone…”
- “What are the risks involved with this decision?”

Sense of Urgency
Remember that managers like to drive the project to their own schedules. During the discovery phase, if you don’t push back when necessary, you’ll be setting the stage for long hours down the project lifecycle line. Managers want to push the project along no matter what issues arise. Let’s say you’re trying to decide whether to use WP’s import screen the way it is or to customize it. This kind of exercise requires time and attention to detail. If something is missed, you’ll pay for it later.

Your homework
As the high-level, prioritized issues are ironed out you’ll get a better picture of the scope of what’s being asked for. During the priority step, you should fill in a matrix that you make up on your own. Columns that I’ve used in the past are Requirement, Priority, Scope, Effort, Custom/Config, Cost:

- Requirement: This is the specific functional piece of functionality that is being asked for by the business.
- Priority: The weight given to the requirement by the business.
- Scope: All the WebPublisher, technical and project-related ingredients of the requirement.
- Effort: Gut feeling on how much time it will take to accomplish.
- Custom/Config: Are there any customizations? Is it really all configuration? Try to find any holes that are not being discussed
- Cost: Any potential sink holes that will be expensive.

Example:
Requirement: Automate website approval process
Priority: 1
Scope: Design workflow template, notification design, prototype, user testing, rework, application migration from dev, to test, to integrated test, to training, then to production.
Effort: Moderate
Custom/Config: Mostly configuration, possible customization: deal with aborted workflows.
Cost: Developer training, consultants

Cognitive Dissonance
Take note of Business folks and members of your team who do not admit when they are wrong or have made a mistake. These are the people who will sometimes unwittingly take the project of course and will need to be reigned-in when they do so. They are also the people who pull some of the team aside and have “off the record” discussions about the project. They occasionally need ego boost by show boating an experience in the past that they think is important: “the way I did that at was to implement with .

The following are some paths of discovery decisions and where it took WP customizations:

Security
- Implemented RSA Cleartrust
- Had to add 2 servers per environment: 10 in total
- Had to hire an RSA consultant for initial installations
- Spent numerous cycles with basic documentation and endless trial and error troubleshooting.
- Customized the login, denied, and logout pages
- Held up the project as authentication became the critical path during the initial testing phase

Multiple Locales
- Used Categories to manage multiple languages for the website navigation.
- Created 10 sitecache configurations, times 5 for each environment.
- Redesigned the web cabinet structure to accommodate the new content submission groups.
- Trained publishers how to add translations and publish them.
- Created a custom application to integrate with outside translation vendors.
- Dumped huge amounts of money on developing an automated way to process the sitecache target content and metadata.

Automated Navigation
- Designed a simple way to publish navigation changes based on folders and linked content.
- Developed a TBO and customized WP to prototype the functionality.
- After the User Demo, we rewrote the design and developed a much more complex solution.
- This went on for a few more cycles until we had to stop and move on to other aspects of the project to meet our schedule.
- The business said they wouldn’t make any changes to the site folders for a year. After the site went live using WP, within a month they were changing the navigation.

Discovery is a term used for exploring what the business would like to have. What it really means is finding out what is realistic to have now and what is going to be released at a later date. There will always be disappointed members of the team who will never be satisfied and other who will feel over worked. Hopefully you’ll have a balance of these types with the more motivated types who enjoy what they do.

Documentum In the Trenches: WP

I have been working with EMC Documentum's Webpublisher long enough to know a few things about the got yas during the phases of a website project.

1. There is no guide that provides a soup to nuts view into how a website publishing project is designed, developed, and implemented.
2. Webpublisher will have issues, it's just a reality.
3. As with most websites the business requirements will focus mainly on the look and feel of the site, not the mechanics of how you get the content on the site.
4. If you like to follow project schedules closely then you better pad in time because you're going to need it.

With these four observations in mind, I'd like to take you on a ride down Webpublisher memory lane with a project that I recently completed (no projects are ever really done, who's kidding who?). The structure of my remembrances will following the usual project lifecycles:

1. Discovery: do they really know what they want?
2. Business requirements: who are we writing the requirements for?
3. WP installation and configuration: This can be painful.
4. Prototyping and proof of concept: This is fun!
5. Specs: We'll write these if we have to.
6. WP Development: Try not to Thrash.
7. Deployment: That wasn't in the spec!
8. Training: That was easy; until they actually start to use it.
9. Stabilization: Is anyone using this?
10. Support: Who's doing it?