BP Labs: Unraveling the Toolkit Dependency Problem

Linkedin process

One of the great advantages BP3 brings to our customer base is our BP Labs team, which is constantly investing in software and methods to support our business and our customers’ processes.  The Neches suite for process analysis and governance is one evolving example, and in this post we’re going to show how it helps address just one specific challenge that many IBM BPM customers will face.

BP Labs has used the Neches process analysis and governance tooling to examine IBM BPM solutions.  Neches allows us to quickly identify areas of the solution that are of considerable concern when examining the adherence to best practices as well as potential maintenance issues.  In many cases real-world solutions for complex processes wind up pulling in a large number of toolkits to meet their functional needs.

Recently we have seen customers that have solutions that cannot be successfully exported from their environment – it turns out that many of these are due to problems in the underlying toolkit dependencies.  A problem that IBM BPM does not yet properly identify for you at development time.

 

Problem Statement

Toolkits allow you to bundle up parts of a BPM solution for use across multiple Process Apps.  This is frequently used for things like common integrations, or Business Objects that exist in 2 separate Process Apps.  When used well, toolkits are a very powerful tool for re-use and governance of edits. However, misuse of any good feature can quickly get you into trouble.

To understand the problem with Toolkit dependencies, it is important to understand how a toolkit dependency works.  When you add a dependency to a Process App (or Toolkit) the dependency entry does not point to the toolkit overall but to a specific snapshot of that toolkit.  This is good in that it ensures the code you are calling is a static known quantity, even if new versions of the toolkit are created later.  However for heavily linked toolkits it can cause a problem.

The problem is that you can easily wind up with a case where the solution has multiple snapshots of the same toolkit included in the solution.  Lets say I have a toolkit called “BP3 Common Integrations.”  The Process App includes snapshot “v15″ of this toolkit.  The PA also includes another Toolkit called “Extra Special Integrations”.  That Toolkit has a dependency on v12 of “BP3 Common Integrations”.  This means that BP3 Common Integrations is in the solution twice.  This is not a good situation to be in for several reasons including –

  1. Your export will be much larger than it likely needs to be.
  2. A service that changed between the 2 snapshots will yield different results depending on the context of the caller.
  3. Determining the source of a defect can be very difficult.
  4. Installation and run-time migration can be degraded.

Bigger Problems

IBM BPM allows you to create toolkits with circular references.  While this is a questionable practice, the Process Designer does not stop you from doing it.  This can happen as a direct reference – A depends on B and B depends on A, or as an indirect reference A depends on B. B depends on C.  C depends on A.  When I see this in a solution it is the type of thing you want to immediately make sure is really necessary before giving it a pass.  In most cases, the 2 toolkits really need to be a single merged toolkit.

Where this can get very bad is when the toolkit dependencies are no longer a “circle” but rather a “spiral”.  That is the Toolkit snapshots are, somewhere along the chain, out of phase.  So in our A->B->C->A example the actual snapshots are something more like –

  • Toolkit A snapshot 1.2.0 depends on Toolkit B snapshot 1.2.0
  • Toolkit B snapshot 1.2.0 depends on Toolkit C snapshot 1.1.0
  • Toolkit C snapshot 1.1.0 depends on Toolkit A snapshot 1.1.0
  • Toolkit A snapshot 1.1.0 depends on Toolkit B snapshot 1.1.0
  • …..

You can see the spiral starting.  When C depends on an older version of A, then we start to have significant dependency problems.

Solutions

IBM BPM does not provide an Out-of-the-Box tool to detect this problem today.  It will detect different version of the System Data toolkit and warn you about only that specific problem.  I have been told that BPM Advanced can detect the above problem and alert you, but if you are using BPM Standard and the process designer, you likely have this problem and do not even know it.

BP3 is working with our customers to create the governance steps to look for these sort of problems before deploying to production.  Currently this is done with our internal Neches tools.  We are also have some tactical toolkits that can detect this situation in your BPM Process Center so that you can monitor your own development team.  If you are interested in these tools, please contact our labs group – support@bp-3.com.   If you are interested, Pre-register for Neches Analysis today   It will be generally available in June.

LI-NechesAnalysis