Moving Source Code to GitHub

I am slowly moving source code for my tools, utilities, and applications over to

Links to All of My Source Code

Here are links to all of the source code and projects I have discussed on my blog.

Don’t forget about all of my articles on the Publications page and the cool tools on the Tools page.

D2 v4.5 Inbox Widget — Part 2

In my previous post, I provided an overview of my D2 Inbox widget design.  In this post I will discuss the JavaScript code required to maintain the freshness of the login ticket.  The JavaScript and OpenAjaxHub for the Inbox widget work essentially the same as they did for the D2 DQL Editor widget, with the exceptions discussed below.

As with the D2 DQL Editor widget, a ticket is generated when the JSP page initially loads.  This ticket is consumed by the query that immediately updates the page with the list of notifications in the user’s Inbox.  When a user clicks the subject of a notification to view its details, another query is generated to retrieve those details from the Docbase.  This means another ticket is required to login to the Docbase and run the query, which means the JavaScript must publish another D2_ACTION_DM_TICKET_GENERATE message.  Recall that the callback function for the message subscription event is onNewTicket().  In the D2 DQL Editor widget, this function immediately called the servlet to run the query.  With the D2 Inbox widget, a set of states are defined (INIT, UPDATE, DETAILS) that determine how the onNewTicket() responds once it has received the new ticket.  See the code snippet below.

function onNewTicket(event, oMessage) {
  ticket = oMessage.get("ticket");

  if (inBoxState === INIT) {
    inBoxEvent = "";

  if (inBoxState === UPDATE) {
    inBoxEvent = "";

  if (inBoxState === DETAILS) {
    inBoxEvent = "";

The INIT state is set when the page first loads.  The result of this state is that the page of Inbox notifications is built.  The UPDATE state is set upon the closing of the Inbox Notification Details pop-up window.  When the pop-up closes, the Inbox notifications list is refreshed — one could have been deleted from the Details page.  The DETAILS state is set when a user clicks a link to view the details of a notification.  When the onNewTicket() function identifies a DETAILS state, it calls doShowItemDetail() to display the notification’s details.  More of the JavaScript is displayed below.


The updateInbox() function calls the D2QueryInboxServlet servlet to run the query to display the Inbox notifications.  It inserts the username, ticket, and Docbase name into a URL string and uses AJAX to call the servlet. Here is the most interesting snippet of that code.

var  xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");

// do AJAX
xmlhttp.onreadystatechange = function () {
  if (xmlhttp.readyState === 4 && xmlhttp.status === 200) {
    document.getElementById("InboxTable").innerHTML = xmlhttp.responseText;
};"POST", "D2InboxQueryServlet", true);
xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
xmlhttp.send("user=" + user + "&docbase=" + docbase + "&ticket=" + ticket);


The showItemDetail() function is called when a user clicks on the subject of an Inbox notification.  The URL built for each notification on the page, calls this function and passes the r_object_id of the corresponding dmi_queue_item.  You can see the function sets the inBoxState variable to DETAILS and requests a new ticket.  Once the new ticket is received by the onNewTicket() function, the doShowItemDetail() method is called to present the pop-up.

function showItemDetail(objId) {
  objectId = objId;
  inBoxState = DETAILS;


The doShowItemDetail() function pops open the window containing the details for the selected Inbox item.  Clicking the link opens the InboxDetail.jsp page, which uses Java and AJAX.

function doShowItemDetail() {
  var url = "InboxDetail.jsp?itemId=" + objectId + "&user=" + user + "&docbase=" + docbase + "&ticket=" + ticket;, 'newwindow', config = 'height=400,width=500, toolbar=no, menubar=no, scrollbars=yes, resizable=no,location=no, 
    directories=no,   status=no');
  objectId = "";


This function is called when the Details pop-up window closes.  It sets the inBoxState to UPDATE and requests a new ticket.  When a new ticket is received, the list of Inbox notifications is refreshed.

function on_DetailsClose() {
  inBoxState = UPDATE;

In summary, this isn’t a very complicated widget.  The primary difference between this widget and the D2 Query Editor widget is the necessity to refresh the tickets and to react differently depending upon the inBoxState state variable.  I hope you find this widget useful and use it as a foundation for building your own D2 external widgets.

You can download the D2 Inbox WAR file here.

D2 v4.5 Inbox Widget — Part 1

Building on the success of the D2 DQL Editor external widget, I embarked on building a D2 external widget to fill another gap in D2 capability:  the Inbox.  My D2 Inbox external widget mimics the Webtop Inbox node for notifications (see figure).  This widget used the same framework I established in the D2 DQL Editor widget, but required some additions to the JavaScript code to handle some unique login ticket issues.

Let’s walk through the design before we jump into the code; please refer to the figure below.

D2InboxDesignWidget Initialization (green line)

The Inbox widget is initialized when the D2 tab containing it is activated.  Upon activation, D2 loads D2Inbox.jsp, which fires off a series of JavaScript methods and AJAX calls.  First, JavaScript parses the URL query string to retrieve the user name and Docbase name passed in by D2 (this is configured in D2-Config).  Next, the OpenAjaxHub is instantiated and registered to listen for the D2_EVENT_DM_TICKET_GENERATED message, and immediately requests a login ticket.  Once the ticket arrives, an AJAX call is made to the D2InboxQueryServlet to build the Inbox page.

Inbox Message Details (red line)

When an Inbox notification is clicked, its details are displayed in a pop-up window (InboxDetail.jsp).  A new ticket is requested from the OpenAjaxHub and then passed to an AJAX call to login to the Docbase and retrieve the message details.   Note a new ticket is required to login because the previous ticket was used to generate the Inbox page, and is now invalid.  More about tickets and ticket management later.  The D2InboxDetailQueryServlet builds the pop-up window containing the message details.

Delete Inbox Message (blue line)

From the message detail pop-up, the user can delete the message from the Inbox.  The delete is performed by the D2InboxDeleteQueueItemServlet, and upon its completion, closes the pop-up window and refreshes the Inbox.

Close Inbox Message Details (brown line)

Closing the Inbox message details pop-up simply returns the user to the Inbox page (no refresh).

In the next post I will discuss the details of managing tickets and configuring the widget in D2-Config.  You can download the D2 Inbox WAR file here.

D2 v4.5 DQL Editor Widget – Part 4

In the last post of this series I will discuss how to install and configure the D2 DQL Editor external widget. To recap:

  • Part 1 – overview of widget design,
  • Part 2 – discussion of JavaScript and OpenAjaxHub implementation,
  • Part 3 – discussion of Java servlet to run query and format results.

Installation and configuration of the D2 DQL Editor widget occurs in three easy steps:

  1. First, install the D2DQL.war file on your application server (I only tested with Tomcat). The WAR should contain all of the necessary DFC, dmRecordSet, and DCTMBasics JARs (in /WEB-INF/lib), in addition to the DQLQueryServlet.class class file, D2DQLEditor.jsp JSP file, and DQL.css style sheet.  The directory structure should look like this:
    • ../webapps/D2DQL
      • D2DQLEditor.jsp
      • /META-INF
      • /resources
        • DQL.css
        • D2-OAH.js
        • OpenAjaxManagedHub-all.js
      • /WEB-INF
        • /lib
          • aspectjrt.jar
          • certFIPS.jar
          • commons-lang-2.4.jar
          • DCTMBasics.jar
          • dfc.jar
          • dmRecordSet.jar
          • jsafeFIPS.jar
          • log4j.jar
        • /classes/com/dm_misc/D2
          • DQLQueryServlet.class
  2. In D2-Config, configure a new widget using the settings below (you may need to adjust the URL for your environment).  See the EMC Documentum D2 v4.5 Administration Guide for details on configuring new widgets D2 configurations.
    • Name:  D2DQL
    • Label and Description:  D2DQL
    • Widget Type: ExternalWidget
    • Widget URL:  http://localhost:8080/D2DQL/D2DQLEditor.jsp?user=$USER&docbase=$DOCBASE
    • Bi-directional Communications:  checked
    • Communication Channels:  D2_ACTION_DM_TICKET_GENERATE


  1. Configure the D2DQL widget on a D2 workspace and configure it in the Configuration Matrix appropriately.

Login to D2, open your workspace, and run a query.


You can download the WAR file and all of the source code for the D2 DQL Editor here.

I hope this series of blog posts on building the D2 DQL Editor have been valuable to you.  I find the widget itself useful and the experience of developing it incredibly valuable.  I hope to build additional D2 external widgets using this model in the future.

Leave me a comment, I’d be happy to hear your thoughts.

D2 v4.5 DQL Editor Widget – Part 3

In the last two posts I introduced you to my D2 DQL Editor external widget. In Part 1, I discussed the general workings of the widget. In Part 2, I discussed the JavaScript and OpenAjaxHub code required to request and receive login tickets via D2’s bi-directional communication channels. In this post I will briefly discuss the Java servlet that the widget calls to execute the DQL query and format the results.

There isn’t anything surprising in the Java servlet code. Instead of showing all of the servlet code, I will just highlight some areas that are noteworthy.

  • The servlet extends HttpServlet; nothing special.
  • Note the use of the DCTMBasics to login and run the query
  • Note the use of dmRecordSet classes to simplify the processing of the query results. The use of this class provides several nice capabilities:
    • The number of rows returned by the query can be determined from the collection without having to run a second query with the count(*) in the selection criteria.
      // get record set
      dmRecordSet rs = new dmRecordSet(col);
      // if results do this
      if (rs.getRowcount() > 0)
         output.append("<h3>Rows returned: " + rs.getRowCount() + "</h3>");
    • You can easily print the column names returned in the collection without knowing them ahead of time by iterating over the dmRecordSet.getColumnDefs() ArrayList.
      ArrayList&lt;IDfAttr&gt; cols = rs.getColumnDefs();
      // print col names as headers
      output.append("<th>Row No.</th>");
      for (IDfAttr a : cols) {
        output.append("<th>" + a.getName() + "</th>");
    • Likewise, you can easily iterate over the entire collection by using the column name ArrayList to retrieve the value of each column from each row.
      while (rs.hasNext()) {
        IDfTypedObject tObj = rs.getNextRow();
        output.append("<td>" + (rs.getCurrentRowNumber() + 1) + ".</td>");
        for (IDfAttr a : cols) {
          output.append("<td>" + tObj.getString(a.getName()) + "</td>");

You can download the WAR file and all of the source code for the D2 DQL Editor here.

D2 v4.5 DQL Editor Widget – Part 2

In the last post I introduced you to my external D2 DQL Editor widget.  In this post I will discuss the JavaScript used to invoke the OpenAjaxHub and to use D2’s bi-directional communication channels.

The JSP file containing the JavaScript is named D2DQLEditor.jsp.  The HTML portion of the file is very simple and contains code to display the edit box for the DQL, the buttons to run the query and clear the editor box, and a place holder for the query results in the form of a <div> tag.  Note the onload='init();' event attribute in the <body> tag.  The init() JavaScript method initializes the OpenAjaxHub when the page is loaded.

<body onload='init()'>
  <h3>DQL Editor</h3>
  <textarea id="dqlEditor" rows="8" cols="60"></textarea>
  <button onclick="runDQL();">Run DQL</button>
  <button onclick="clearEditor();">Clear Editor</button>
  <div id="dqlResults"></div>

I lifted the code that implements and invokes the OpenAjxHub from the GetTicket example in the D2 Widget Samples from Momentum 2014.  The D2 v4.5 Developers’ Guide was also helpful. Follow the logic:


The init() method calls the parseQueryString() method to parse the user and docbase name out of the URL (nothing special there). It then instantiates an OpenAjaxHub and connects the hub to D2. The OpenAjaxHub.connectHub() method takes two callback functions as parameters: one to execute once the connection has been made (connectCompleted()), the other when the widget is activated (onActiveWidget()). The callback for the activated widget is not used.

function init() {
  d2OpenAjaxHub = new D2OpenAjaxHub();
  d2OpenAjaxHub.connectHub(connectCompleted, onActiveWidget);

Once the hub is connected, the connectCompleted() function is called. If the connection was successful, it calls subscribeToNewTicket() to subscribe the widget to the D2_EVENT_DM_TICKET_GENERATED message.

function connectCompleted(hubClient, success, error) {
  if (success) {
  } else {
  alert("Hub not connected. " & error);

function onActiveWidget(bActiveFlag) {


The subscribeToNewTicket() function simply subscribes the widget to the D2_EVENT_DM_TICKET_GENERATED message and passes onNewTicket() as the callback function for the subscription. This means that whenever a D2_EVENT_DM_TICKET_GENERATED message is received for this widget, the onNewTicket() function handles it.

The reciprocal of the subscribeToNewTicket() function, the publishNewTicketRequest() sends a message to D2 requesting a new login ticket be generated. These two functions (subscribeToNewTicket(), publishNewTicketRequest()) implement the bi-directional communication with D2.

function subscribeToNewTicket() {
  d2OpenAjaxHub.subscribeToChannel("D2_EVENT_DM_TICKET_GENERATED", onNewTicket, false);

function publishNewTicketRequest() {
  messageToSend = new OpenAjaxMessage();
  d2OpenAjaxHub.sendMessage("D2_ACTION_DM_TICKET_GENERATE", messageToSend);


onNewTicket() is the callback function for the subscribe event. When the message is recieved, the ticket is extracted from the message object and saved as a JavaScript variable. submitDQL() is then called to extract the query from the HTML form and pass it to the DQLQueryServlet for processing.

function onNewTicket(event, oMessage) {
  ticket = oMessage.get("ticket");

That is all of the JavaScript concerned with the OpenAjaxHub and communicating with D2. The remainder of the JavaScript code handles submitting the DQL query to the servlet and handling the results. This JavaScript to instantiate and use the OpenAjaxHub is pretty generic and can be re-purposed for building other external, bi-directional D2 widgets.


The submitDQL() function extracts the DQL query syntax from the HTML textarea control, combines it with the user name and docbase name parsed from the URL, and the the ticket received by the onNewTicket() callback function, and submits it to the DQLQueryServlet using a little AJAX.

function submitDQL() {
  var dql = document.getElementById("dqlEditor").value;
  dql = encodeURI(dql);
  if (dql !== "") {
    var xmlhttp;
    if (window.XMLHttpRequest) {
      // code for IE7+, Firefox, Chrome, Opera, Safari
      xmlhttp = new XMLHttpRequest();
    } else {
      // code for IE6, IE5
      xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");

    // setup AJAX callback
    xmlhttp.onreadystatechange = function () {
      if (xmlhttp.readyState === 4 && xmlhttp.status === 200) {
        // set the query results on the div tag place holder
        document.getElementById("dqlResults").innerHTML = xmlhttp.responseText;

    // call servlet"POST", "DQLQueryServlet", true);
    xmlhttp.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
    xmlhttp.send("user=" + user + "&docbase=" + docbase + "&dql=" + dql + "&ticket=" + ticket);
    document.getElementById("dqlResults").innerHTML = "Submitting query for execution...";
  } else {
    document.getElementById("dqlResults").innerHTML = "DQL statement cannot be blank";


The runDQL() function is called when the Run DQL button is clicked on the widget. Because a new ticket is required for every query, this function simply requests a new ticket. Remember that the onNewTicket() callback function of the subscription to the D2_EVENT_DM_TICKET_GENERATED message calls the submitDQL() function, which runs the query.

function runDQL() {

In the next post I will discuss the Java and DFC code used in the servlet to process the DQL query passed from the widget, and to return the results. You can download the D2 Configuration, the WAR file, and all of the source code for the D2 DQL Editor here.

D2 v4.5 DQL Editor Widget – Part 1

One function of Webtop that I really miss in D2 is the DQL Editor.  So, as an exercise to learn how to create external D2 widgets that use bi-directional communication, I decided to build one.  Recall that the last D2 widget I build generated a bar code for the selected object in the Doclist widget, but did not employ bi-directional communication with D2 or the Docbase.

The D2 DQL Editor looks and functions similarly to the DQL Editor in Webtop (see figure).  And, as it turned out, was not too difficult to build, once I understood the OpenAjaxHub and its callback structure.  The rest of the widget is implemented as a JSP page with a Java servlet behind it.

DQL Editor

D2 loads external widgets into iFrames.  As part of the configuration of this widget, D2 passes the name of the current user and Docbase in the URL.  JavaScript in the widget parses this information out of the URL when it loads and saves it for use later.  The JSP then instantiate the OpenAjaxHub and subscribes to the D2_EVENT_DM_TICKET_GENERATED message.  When a DQL query is run, the widget publishes a request for a login ticket on the OpenAjaxHub.  Once the ticket is received, the query, the ticket, the user, and the Docbase name are passed to the servlet to be run.

In the following three posts (Parts 2, 3, and 4) I will discuss in more detail the various components of this external D2 widget.

You can download the WAR file and all of the source code for the D2 DQL Editor here.

Barcode Widget for D2

Now that I have D2 v4.5 installed and working, I have started to explore writing External Widgets.  As my first experiment, I created this little barcode generator.  As far as External D2 Widgets go, this one is super simple, but it was an instructive exercise.  So, what does the barcode generator do?  When you click on an object in the Document List or Repository Browser widget, the selected object’s r_object_id and object_name are transmitted to the Barcode widget.  The Barcode widget sends the r_object_id to an Internet service that produces the barcode as a GIF.

Here’s how I put it all together:

  •  First, I created a single HTML/JavaScript page that parses the r_object_id and object_name out of the URL parameters.  The r_object_id is inserted into a new URL that calls the barcode generator; the object_name is simply displayed with the barcode (see source code below).
<!DOCTYPE html>
<!-- Barcode D2 External widget             -->
<!-- (c) 2015 MSRoth - -->

        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        <meta http-equiv="Cache-Control" content="no-cache, no-store, must-revalidate" />
        <meta http-equiv="Pragma" content="no-cache" />
        <meta http-equiv="Expires" content="0" />
<p align='center'>

<!-- parse url parameters -->
function getUrlVars() {
  var vars = {};
  var parts = window.location.href.replace(/[?&]+([^=&]+)=([^&]*)/gi,
  function(m,key,value) {
      vars[key] = value;
  return vars;

<!-- get the r_object_id and object_name passed in on URL -->
var id = getUrlVars()["id"];
var name = getUrlVars()["name"];

<!-- make sure that an object was selected -->
if (id == null || id == "") {
  document.write("Please select an object.");

  <!-- determine object type selected -->
} else {
  if (id.lastIndexOf("0b") == 0) {
      document.write("Object type: 'dm_folder' &nbsp;&nbsp; Name: '" + name + "'");
  } else if (id.lastIndexOf("0c") == 0) {
      document.write("Object type: 'dm_cabinet' &nbsp;&nbsp; Name: '" + name + "'");
  } else if (id.lastIndexOf("09") == 0) {
      document.write("Object type: 'dm_document' &nbsp;&nbsp; Name: '" + name + "'");
  } else if (id.lastIndexOf("08") == 0) {
      document.write("Object type: 'dm_sysobject' &nbsp;&nbsp; Name: '" + name + "'");
  } else {
  <!-- generate bar code -->
  document.write("<br/><br/><img src='" + id + "&dpi=96&imagetype=gif&rotation=0&color=&bgcolor=&fontcolor=&quiet=0&qunit=mm' alt='" + id + "' />");
  document.write("<br/><center>Barcode by <a href='' target='_blank'>TEC-IT.COM</a></center>");
  • Next, I put this file in my Tomcat’s /webapps/ROOT directory so it could be served as a web page.  This page can be easily tested using a URL like h ttp://localhost/barcode.html?id=1234&name=test.
  • In D2-Config, I created a new widget named Barcode of type ExternalWidget.  For the URL I provided h ttp://localhost:8080/Barcode.html?id=$value(r_object_id)&name=$value(object_name).  The $value(r_object_id) tells D2 to substitute the selected object’s r_object_id in the URL.  The same is true for the object’s object_name value.
  • Then I selected D2_EVENT_SELECT_OBJECT in the Focus Events and Communication Channel panes to make sure the widget listened for the correct messages to update itself (see image below).


  • The final step was to include the widget in a workspace and configure it for visibility in the D2 configuration matrix.  The result is below.


Like I said, this External Widget is super simple.  Now, to create an External Widget that uses bi-directional communication to communicate with D2 and the repository…

Similarity Index Aspect v2

I recently tinkered with the solution contained in my EMC Knowledge Sharing Article, Finding Similar Documents Without Using a Full Text Index. With a simple addition to the Aspect code, I managed to eliminate one of the solution’s main shortcomings: that it only worked for crtext content.  The solution to this problem came in the form of Apache Tika.  (Thanks to Lee Grayson for introducing me to Tika.)  Tika is a text-extraction engine originally incubated in the Lucene project.  With the simple addition of the code below, the SIAspect class (see the article for full context) is now able to extract and compare content regardless of format.  This small addition has increased SI’s effectiveness hugely; I only wish I had discovered Tika before submitting of the article.

//, Line 129
// *** \/\/\/ changed in v2 \/\/\/ ***
	// string1 = cx.ByteArrayInputStreamToString(content);  // "old" method

	// use Tika to extract text from any content
	try {
		Tika tika = new Tika();
		format = tika.detect(content);
		// if tika detects the content is a binary stream, just convert the stream
		// to a string.  Sometimes text files, RTF and PDF files that have
		// extended ASCII chars will cause Tika to treat the file as a binary file
		// and not extract the content.
		if (format.contains("octet-stream") ||
				format.contains("image") ||
				format.contains("audio") ||
				format.contains("video")) {
			string1 = cx.ByteArrayInputStreamToString(content);
		} else {
			// use Tika to extract content
			string1 = tika.parseToString(content);
	        string1 = string1.replaceAll("\\s+", " ");
	} catch (Exception e) {
		// if Tika causes an error, just convert the stream to a string
		string1 = cx.ByteArrayInputStreamToString(content);

// *** ^^^ changed in v2 ^^^ ***

To test the improvement, I modified the test code to use the JODConverter to generate a variety of document formats from my test corpus.  The new project can be downloaded here.


%d bloggers like this: