Updated Documentum Tools and Applications

I had a little free time this week and made some updates to tools and applications I use frequently.  These updates represent ideas sent to me by users or were necessary to address additional situations I encountered in the field.  I hope they are helpful for you too.

QuikDIE v1.5 – This application exports content and metadata from a Documentum repository based upon a query.  Content is exported in its native format, and metadata is exported as XML.  Updates include:

  • detects and eliminates content “parked” on BOCS server
  • “dmi_” object types are ignored
  • implemented DCTMBasics JAR

For additional information regarding QuikDIE (including important info in the comments section), search for “QuikDIE“.

DeepExport v1.1 – This application exports CONTENT ONLY from a folder in a Documentum repository.  Content is exported in its native format and the folder hierarchy in the repository is replicated on the export drive.  Updates include:

  • detects and eliminates objects parked on BOCS server
  • updated to use DCTMBasics v1.3
  • refactored to remove Utils class
  • added password encryption/decryption

For additional information regarding DeepExport, search for “DeepExport“.

DCTMBasics v1.3 – This JAR file contains a set of helpful methods for interfacing with Documentum via the DFC.  It contains implementations of methods discussed in “The Basics” series.  Updates include:

  • getDFCVersion() – returns DFC version of your client
  • checkDFCVersion() – compare the current DFC version with a minimum version
  • createDocbasePath() – create a folder hierarchy in the repository
  • encryptPassword() – encrypt a password using the RegistryPasswordUtils class (used to encrypt dm_bof_registry user)
  • decryptPassword() – decrypt a password encrypted with the encrypt() method
  • runDQLQueryReturnSingleValue() – run a query that returns a single value, for example:  select count(*) from …
  • isSysObject() – tests it an IDfPersistentObject is a sys_object or not
  • createLogFile() – create a log file and return it to the caller as a PrintWriter
  • other misc updates, changes, corrections, and refactoring

For additional information regarding “The Basics” series, search for “The Basics“.

 

 

 

Documentum Bulk Export Tool v1.4

I just updated the Documentum bulk export tool, QuikDIE, to version 1.4.  You can download it here.  More info about the tool can be found in  previous posts here and here.  Be sure to read the comments.

Updates in this version include:

  • Compiled using Java 1.7
  • updated JDOM library to 2.0.
  • check for supported DFC version (only 6.5 and later)
  • export a_antecedent_id so version tree can be reconstructed
  • updated batch file to use dctm.jar (Hopefully this will eliminate the java.lang.NoClassDefFoundError: org/apache/commons/lang/StringEscapeUtils error some have reported.  This class is found in the commons-lang-2.4.jar.)

Check it out and let me know what you think.

 

 

 

Documentum Bulk Export Tool v1.3

I updated the QuikDIE Documentum Export Tool to v1.3.  You can download it here.

This update includes the following changes and additions:

  • Added r_creator_name, r_creation_date, r_modifier, and r_modify_date attributes to the standard output set.
  • Converted object_name, subject, title and all custom attribute values to XML-safe.
  • Updated the dmRecordSet JAR to v1.2 and included it in the /lib folder.
  • Fixed bug that considered any object not starting with “dm_” to be custom, instead of ignoring all objects in OMIT_OBJ_PREFIXES array.
  • Included source project in archive.
  • Minor cosmetic tweaks.

The first two enhancements were suggested by Malcolm MacArthur (thanks!).  You can see that discussion thread here.

DRX 2.3 Released

Brian Y. and I are pleased to announce the release of DRX (Documentum Repository eXplorer) version 2.3.

DRX is a powerful exploration, auditing, and documentation tool that allows Documentum administrators and developers to quickly document vital information  about their repositories and environments. DRX provides a simple UI to  facilitate logging into the Documentum repository, configuring and running  specific reporting modules, and viewing the resultant reports. DRX reports are  generated in HTML for easy viewing and XML for more complex processing.

DRX is distributed as an EXE file that contains all of the Java code,  requisite libraries, JVM and the DFC required to run the program. Its compact  distribution and simple installation (simply unzip it!) make it easy to run from  a thumb drive, or move from one computer to another.

Here are the highlighting and new features in version 2.3:

  • Added additional stats for WF (terminated and halted)
  • Separated SCS from WebPub reporting (still in same module)
  • Added additional product/feature checks in the EnvInfo Module
  • Distribute config/dfc.keystore file to avoid endless hangs in certain conditions
  • Tweaked queries for custom object types to eliminate stock BOF classes
  • Corrected bug in role query
  • Corrected bug in TCS detection
  • Add “top user” queries to users and groups module
  • Expanded role reporting to match group reporting
  • General code cleanup and standardization
  • Added drx.exclude_types property to config/dfc.properties. These entries will be object types excluded from the Custom Types and Hierarchy report.
  • DRX no longer overwrites the config/dfc.properties file (required for drx.exclude_types).
  • Added Webtop Presets module (user must copy dfc.globalregisty.* entries from the Content Server dfc.properties to the DRX confg/dfc.properties file.

DRX v2.3 is available for download here:  http://sourceforge.net/projects/drxproj/files/

If you haven’t tried DRX yet, I encourage you to download it and give it a try. For a brief explanation of DRX, see here .  And, as always, if you have any feedback or feature requests, please log them here.  Enjoy.

Qit 3.1 – New Documentum Open Source Tool

If you aren’t using Qit yet, you should be.  Qit is an awesome alternative to repoInt from Belgian company Qtree. Qit provides developers and admins easy access to the tools they need to manage and troubleshoot Documentum repositories.  For example:

  • Tree navigation of cabinets/folders as well as administrative nodes (e.g., users, ACLs, jobs),
  • DQL editor,
  • API editor,
  • Properties editor for any object selected in results panel, and
  • Search.

I have found it to be a solid and indispensable tool.  Check out the screen shot!

Qit-UI

In addition, Qit is  extendable with custom actions.  Just for fun, I wrote an extension to produce an SHA-256 hash for a selected object’s content (see image below).  It was super simple.  Code here.

QitSHA

Qit is built on the Eclipse framework and is available for Windows, Linux, and Mac.  Check it out, www.qtree.be/qit!

Documentum Bulk Export Tool

For some time now I’ve had this idea rattling around in my head that there should be a simple way to export content and metadata from the Content Server in a manner that allows for quick access to both the exported content and the metadata. Drag & Drop or Export from Webtop only gets you the content files. Dump & Load is a major pain, inconsistent, and too opaque. DAR files can also be problematic and too opaque.  I wanted a way to quickly export the content in its native format and the relevant metadata as XML. These pairs of files would then contain everything I needed to know about the content. I could visually verify them on the file system; I could open the content with a double-click; I could easily examine the metadata — or change it; I could easily archive it to CD or a ZIP file; or I could import it into another CMS.  (I think Records Manager does something like this when you disposition files to NARA as XML.)  Anyway, I finally converted the idea to code. Here is the Java/DFC export tool if you are interested.

To run it, configure the export.properties file to contain the necessary login credentials for your repository, and a query to gather the files you want to export. In addition, specify a location for the exported content on your file system. Here is my sample export.properties file.

export.query=select * from dm_sysobject where folder('/Temp',descend)
export.user=dmadmin
export.password=dmadmin
export.repo=repo1
export.path=c:/Temp/Export
export.log=c:/Temp/Export/export.log

Notes:

  • After the export tool runs once it will encrypt the password so it doesn’t sit around unprotected.
  • The query must return r_object_id.

Once configured, run the export.bat batch file. The batch file does its best to create the correct class path, but you might have to help it if your Documentum environment is not installed in the default manner.

The export tool will export all of the objects it gathers from running the configured query. Each content object is exported to a file name that is its r_object_id (an easy way to avoid any file name collisions), followed by the DOS extension for its content type. The accompanying metadata is similarly named, followed with .metadata.xml. All files are exported to the path you specify in the export.properties file.   Note that no folder structures are created to mirror the structure in the repository. That is, the export directory will contain a “flat” file structure (that’s why avoiding file name collisions was important). Right now, the export tool can handle dm_sysobjects, dm_folders, virtual documents, and content-less objects. Since the export tool will export whatever the query returns, you can export versions by specifying ‘(all)‘ in the query, and subfolder structures by using the FOLDER(*,'descend') function.

Here is an example .metadata.xml file.

export_xml

Notes:

  • This particular file is a virtual document, and it’s child nodes are listed in the <vd_children> element.
  • By default, the export tool exports metadata values for:  object_name, object_type, title, subject, acl_domain, acl_name, owner, version and a_content_type) (if applicable).
  • If the type is a custom type (like this example is), values for all custom metadata properties will also be exported (note sr_cat_no, sr_lot_no, and sr_expire above).

If the export tool encounters an object that appears to be a custom type, it will export a .typedef.xml file that contains definitions for all its custom attributes.

Here is an example of a .typedef.xml file.

type_xml

It’s not perfect, but it works for my purposes.  Perhaps one day I will write an import tool that will read the XML and content files and import them into another repository (e.g., think simple migration tool). However, I suspect that the import process will be a lot more complicated than the export process. Keep an eye here, it might show up one day.

DRX v2.2 Released

Brian Y. and I are pleased to announce the release of DRX (Documentum Repository eXplorer) version 2.2.  We made this release only a “point” update (i.e., from 2.1.1 –> 2.2), even though it contains some exciting new features — not the least of which is its ability to run against D7 repositories.  This release of DRX is compatible with Documentum 5.3 – 7.0 Content Servers.

Here is a quick list of additions, enhancements, and features included in this release of DRX:

  • recompiled to work with D7 repositories;
  • updated JRE to 1.6_43;
  • expanded the list of Documentum housekeeping reports included in jobs report;
  • include list of users who have superuser or admin privileges in user report;
  • added list of top 20 ACLs according to the number of objects to which they are applied. (This helps gauge ACL coverage.);
  • include JMS, BOCS, and ACS in configuration report;
  • added count of inbox items to user report;
  • general improvements, corrections, and tweaks.

DRX has enjoyed over 1,300 downloads in the past year.  If you haven’t tried DRX yet, I encourage you to download it and give it a try. For a brief explanation of DRX, see here .  And, as always, if you have any feedback or feature requests, please log them here.  Enjoy.

HTA Monitor for InputAccel and Documentum

I recently did some work for a client who had a fairly large Documentum and Captiva installation which spanned numerous servers and environments (16 servers x 3 environments). One of the challenges they faced when troubleshooting a problem was simply determining if all of the required processes (services) on each server were running. They were not using any network monitoring tools like Hyperic, or Nagios, or Reveille.

My solution was to put together a quick batch file that used some standard DOS commands to ping each host, and where possible, solicit some information about the processes it was running. This solution was OK, but required the use of a super user’s password in the clear, and didn’t really provide the information that was needed.

My next attempt at this solution had me writing a VBScript using WMI Scripting API objects to solicit information from the servers. This worked pretty well but suffered from two major drawbacks: 1) the super user’s password was still used in the clear; 2) the server list was so long, the results scrolled off the screen.

My latest approach to this solution is to use HTA – an HTML Application. HTA is a Microsoft program (mshta.exe) which uses HTML for the UI, and VBScript for program logic. HTA programs look and act like web sites, but HTAs execute locally without the constraints of the IE security model. In fact, HTA applications run as fully trusted applications, just like Microsoft Word. Obviously, this can be both good and bad. Fortunately for me, this was a good thing and allowed me to quickly port my VBScript to a framework that provided a simple and familiar UI construct, and didn’t make me jump through unnecessary hoops to execute administrative functions on the desktop.

Another plus for HTA is that the code is not compiled and can be easily viewed/modified in any text editor. This makes it easy to change the list hosts and processes monitored; making the application portable to other environments.

Here is a screen shot of the IACheck HTA application.

Notice that the user’s password is protected from view, and the process can be configured to loop, in this case, every 10 minutes. The looping feature allows the application to be launched on the console or admin’s desktop and continuously monitor an environment.

Though originally created to monitor InputAccel and Documentum processes, the application can be easily changed to meet your needs.  If you are interested, the source code for the application can be downloaded here. Simply change the names of the hosts and processes you would like to monitor and double-click the file to run it. I have included a list of Documentum and Captiva processes in the source code so you don’t have to look them up.

UPDATE:  Newer version of the monitor here.

DRX v2.1 Released

Brian Yasaki and I released DRX (Documentum Reporitory eXplorer) v2.1 this past week on SourceForge.  http://drxproj.sourceforge.net.  This is the first update to the software since it was released to the open source community a year ago, and we are really pleased with it.  The update includes a few bug fixes, a few tweaks and enhancements, a new module for the Audit Trail, and the ability to cancel the tool mid-run.  I encourage you to give it a try.

For those of you unfamiliar with DRX, here is a brief introduction:

DRX is a powerful exploration, auditing, and documentation tool that allows Documentum administrators and developers to quickly document vital information about their repositories and environments. DRX provides a simple UI to facilitate logging into the Documentum repository, configuring and running specific reporting modules, and viewing the resultant reports. DRX reports are generated in HTML for easy viewing and XML for more complex processing.

DRX is distributed as an EXE file that contains all of the code, requisite libraries, JVM and the DFC required to run the program. Its compact distribution and simple installation (simply unzip it!) make it easy to run from a memory stick or move from one computer to another.

We have found DRX to be very helpful for:

  • Documenting the final delivery of a system;
  • Creating As-Built documentation for a system;
  • Comparing different environments (e.g., DEV and QA, QA and PROD);
  • Exploring a new or unknown environment;
  • Auditing a Documentum installation;
  • Creating an inventory of installed components, document types, users, etc.

DRX v2.1 offers the following reporting modules:

  • Environment – report on the Connection Broker, server configuration, product licensing, the OS and Java.
  • Configuration – report on server, repository, client, full-text server, LDAP and session configuration objects, as well as server configuration files.
  • Custom Types and Hierarchy – describe all custom types and the entire object hierarchy. The report does not detail objects with names beginning with: ‘dm_’, ‘d2_’, ‘c6_’, or ‘bpm_’. However, these objects are included in the hierarchy report.
  • Maintenance Jobs – include the most recent maintenance job reports: State of the Docbase, Consistency Checker, Update Stats, DMClean, and DMFilescan.
  • ACLs – describe all regular ACLs, Template ACLs, and Public ACLs, including extended permissions.
  • Cabinets and Folders – depict the entire cabinet and folder hierarchy of the repository including folder sizes and object counts. This report can take a very long time to run if your cabinet – folder structure is deep.
  • Users and Groups – report all active, inactive and locked users as well as all groups, roles and aliases. All users, groups and roles include explicit membership listings.
  • Lifecycles – describe all document lifecycles.
  • Jobs and Methods – describe all Documentum and custom jobs, methods and procedures.
  • Workflows – describe all workflow templates, instances, and activities.
  • File System – report on Documentum’s use of the file system including storage locations, directory sizes, and file counts. This report must be run from the
    Content Server to report on the actual use of the file system.
  • Registered Tables – describe all database tables registered with the Content Server and produce SQL scripts to recreate the tables if necessary. The report
    excludes registered tables with names like: ‘dm_%’, ‘cya_%’, ‘wcm_%’, ‘%_r’, ‘%_s’, and ‘adm_turbo_size’.
  • Document Templates – list and describe all document templates found in the /Templates cabinet.
  • TaskSpace – report on all TaskSpace applications and components found in the repository.
  • DocApps and DARs – report all DocApps and DARs installed in the repository.
  • BOF – describe all TBOs, SBOs and Aspects found in the repository.
  • Retention Policy Services – describe all retention policies installed in the repository.
  • Physical Records Manager – reports on physical records management configuration, and lists label templates.
  • Records Manager – describes record management configuration and lists cabinets and folders under formal records management.
  • Web Publisher – describe the WebPub configuration including templates, channels, and site caching sources/targets.
  • Audit Trail – report the size of the audit trail, the number of entries grouped by event type, and the events that are registered for auditing.

Check it out and let us know what you think.  See a module that is missing?  Join the DRX Project Team.

DRX Is Now Open Source

I am pleased to announce that the Documentum Repository eXplorer (DRX) is now an open source project hosted at SourceForge.  Flatirons Solutions has generously released this internal R&D project to the open source Documentum community.  The project is managed by Scott Roth and Brian Yasaki, the tool’s original developers.

DRX is a Java/Swing application that utilizes DQL and the DFC to explore a variety of Documentum repository configurations and customizations.  The executable contains everything necessary to run DRX — no additional Java libraries are necessary to install.  The executable even contains its own JVM and a copy of the DFC to increase its portability.  The application can easily be run from a memory stick!   Users not running the application on Windows can execute the non-Swing  interface using Ant and still generate the same output.

To download the application, visit the project’s home page at Source Forge:  http://www.sourceforge.net/projects/drxproj.  Users interested in adding to or extending the reporting mechanisms of DRX can join the SourceForge project.

Current DRX modules include:

  • Environment – report on the Connection Broker, server configuration, product licensing, the OS and Java.
  • Configuration – report on server, repository, client, full-text server, LDAP and session configuration objects, as well as server configuration files.
  • Custom Types and Hierarchy – describe all custom types and the entire object hierarchy.
  • Maintenance Jobs – include the most recent maintenance job reports:  State of the Docbase, Consistency Checker, Update  Stats, DMCleam and DMFilescan.
  • ACLs – describe all regular ACLs, Template ACLs, and Public ACLs, including extended permissions.
  • Cabinets and Folders – depict the entire cabinet and folder hierarchy of the repository including folder sizes and object counts.
  • Users and Groups – report all active, inactive and locked users as well as all groups, roles and aliases. All users, groups and roles include explicit membership listings.
  • Lifecycles – describe all document lifecycles.
  • Jobs and Methods – describe all Documentum and custom jobs, methods and procedures.
  • Workflows – describe all workflow templates, instances and activities.
  • File System – report on Documentum’s use of the file system including storage locations, directory sizes, and file counts.
  • Registered Tables – describe all database tables registered with the Content Server and produce SQL scripts to recreate the tables if necessary.
  • Document Templates – list and describe all document templates found in the /Templates cabinet.
  • TaskSpace – report on all TaskSpace applications and components found in the repository.
  • DocApps and DARs – report all DocApps and DARs installed in the repository.
  • BOF – describe all TBOs, SBOs and Aspects found in the repository.
  • Retention Policy Services – describe all retention policies installed in the repository.
  • Physical Records Manager –report on physical records management configurations
  • Records Manager – describe records management configurations
  • Web Publisher – report all Web Publisher configurations and templates.
%d bloggers like this: