I’m not sure there are exactly seven forms of BPM, but Tom Baeyens writes a thoughtful article here that lays out 7 use cases and how they are addressed by JBoss jBPM.

Before we even get to the 7 forms, Tom has an insight that is key to understanding why BPM systems are a bit “different”:

A key capability of BPM systems is that processes steps can be wait
states. For example in the business trip process above, nodes ‘manager
evaluation’ and ‘ticket purchase’ are human tasks. When the execution
of the process arrives in those nodes, the system executing the process
should wait till the assigned user completes the task.

From a software technical point of view that capability is a big deal. As the
alternative is a bunch of methods that are linked by HTTP requests,
Message Driven Beans (MDB), database triggers, task forms, etc. Even
when using the most applicable architectural components available in
Java today, it is still very easy to end up in a bunch of
unmaintainable hooks and eyes. Using an overall business process makes
it a lot easier to see and maintain the overall execution flow, even
from a software technical perspective.

I couldn’t agree more with this, though it is a sort of “under the hood” consideration.  It is part of why well-designed BPMS packages can scale really well with respect to the number of process instances that might be in a resting state (waiting for something to happen).  Any BPMS solution that is holding all these instances in memory is going to be at a distinct disadvantage because for most of the time a process instance is alive, it is in a waiting state, waiting for an activity to execute, for a human to act, etc.  and not actively processing work.

Setting aside scale, its just hard work to write the code that surrounds such wait-states correctly and you don’t want to have to write this code every time you confront such a problem - rather, you want to use a BPMS that seamlessly represents these wait-states in a diagram, giving the developer the ability to focus on the functionality of the process steps or the flow of the process, and not get caught up on the internals.

A couple additional thoughts on the seven forms (use cases) he outlines…

For Use Case 5, Visual Programming, he lists a few benefits:

With visual programming, we will target developers that do not yet have
the full skillset to develop in Java. They can graphically specify the
activities and draw transitions to indicate the control flow. Instead
of typing Java code, just put blocks like ‘perform hql query’,
‘generate pdf’, ’send email’, ‘read file’, ‘parse xml’, ’send msg to
esb’. Instead of writing Java, they will be composing software programs
by linking activity boxes in the diagram and filling in the
configuration details.

While simplifying activities like stringing together queries and PDF generation without writing that code may make this operation more accessible to developers without these Java skills, I would contend a different benefit:  Things that should be easy (or rote) become easy (or rote) and therefore free the mind to focus on the bigger business problem.  This is akin to the value I receive from a BPM system hiding the details of implementation of parallelism from me.  While I may have the skills to write this code, it is tedious to write, and expensive to get it right.  So you don’t want to roll this kind of code in a one-off fashion - you want to write it once and re-use it liberally to get benefits of scale.  To the extent that a visual environment frees the developer to focus on the important abstractions of the process, it is a big win.  (Note: As soon as I read Use Case 6 it precisely points out my thoughts about threaded programming, by pointing out that you can use a BPM system as a thread control language).

What really intrigues me is whether Tom and the JBoss jBPM team can create the necessary interfaces and utilities around jBPM to allow the creation of the Domain Specific Languages that can be implemented in jBPM (e.g. BPMN).  While Tom apparently doesn’t see much value in the use of BPMN for executable models, my experience differs (and, since his technical approach doesn’t preclude BPMN for executable models in the future, the disagreement is more theoretical).  Perhaps we can change Tom’s mind about this!  While I think it is possible (and even useful) to create BPMN models that are not intended for execution, I believe a good technical implementation can still be accomplished with BPMN (plus technical details captured in context of the process).  And this technical implementation can still be understood by the Business.

I think the problem isn’t BPMN… its a lack of imagination about how to capture business and technical details in different layers so that the user of these authoring tools can show or hide the “layers” that are interesting to them.   For clarity, I think “technical layer” could include adding additional BPMN elements to the diagram, not just scripting and code and variable input/output details.  And I think tailoring the visibility of details in the diagram is a key component in making a single model multi-purpose in audience.

I’ll definitely have to keep my eye out for jBPM 4…

Kevin Stollznow wrote a good article on process modeling. He makes a couple of key points that I would agree with, slightly rephrased…

• There is no right or wrong way to model a process.  I would say rather, just that “right or wrong” isn’t the primary question.  The primary question is whether the process is effective.  This is essentially the point Kevin makes, but I would posit that it is actually possible to have a “wrong” process model :)
• He takes a casual swipe at BPMN as a modeling and the idea that commenting on modeling a process must necessarily include a diatribe about the “right” way to use BPMN to do it, or cover some minutiae of the specification.  If you’re an avid reader of BPM blogs like I am, this especially hits home with recent discussions about BPMN compliance and roundtripping with BPEL and whether models should even be executable! :)
• He cites Occam’s Razor (though, I think it actually hypothesizes that the simplest explanation is also the most likely explanation). The point is, simplicity is generally preferred over complexity, when both represent the problem adequately.
• Kevin makes a good point about focusing on the audience or intended use of the model (a model for training purposes may be quite different than the executable model of the same process).

One thing I really like about the article is that Kevin correctly points out that even modeling a process is, itself, a process.  And it is a process that we can improve upon over time…

There has been a flurry of discussion around compliance for BPMN specifications, particularly on Bruce Silver’s blog, and with additional commentary from Ismael’s blog.

Bruce makes some great points about why BPMN is catching on in the marketplace.  It is (relatively) intuitive, it represents most of the really important ideas that need to be expressed in the Business Processes, and I’ll add another thought:  you can actually draw it on the whiteboard pretty accurately.  Some people think that is silly, but I think it matters that you can sketch it out pretty faithfully on a whiteboard or piece of paper as well as in software.  A great many people still think most freely on whiteboard/paper rather than on a computer screen.

Bruce also makes a great point about BPMN to BPEL mappings:

“A key reason for that is it is graph-oriented like a flowchart, not block-oriented like a structured program, or more to the point, like BPEL.  For example, you can loop back to some previous step with a BPMN gateway, but BPEL places severe restrictions on that.  What that means is you can draw BPMN diagrams that cannot be mapped to BPEL, or at least to “roundtrippable” BPEL that preserves the BPMN view after even minimal tweaking in the BPEL environment.”

Personally, I don’t even want “roundtrippable” I want “native” - meaning, a representation specifically designed to store BPMN.  Whether it is “custom” XML or “standards-based” XML, I want it to be targeted so that roundtripping simply isn’t an issue - the diagram is the xml/data and the xml/data is the diagram.  Otherwise, how can the business ever be sure that they’re BPMN diagram is being accurately executed?  If the representation isn’t roundtrippable or native, then it undermines faith that the process works as designed.  One of the key problems with all the previous software systems I’ve used and built prior to BPM is the separation between requirements and the finished solution (or, put another way, between the business and IT delivery).

Ismael of Intalio gives this response in his blog:

“Do not worry about round-tripping between BPMN and BPEL
Round-tripping is a futile and wasteful exercise when attempted between two languages or notations that have a significant semantic gap between them, and so is the case for BPMN and any process execution language optimized for computers (like BPEL). The same is true between UML and Java by the way. Round-tripping should be set as a goal only between two languages or notations that are essentially representing the same thing, such as BPMN and its future serialization format for example. What this means is that there must exist a way to fully serialize BPMN diagrams on one hand, and a way to graphically display serialized BPMN on the other. And all this should work in a deterministic and predictable way, uniformly across tools.”

I find the assertion that round-tripping is irrelevant to be a bit startling.  The problem is, if the translation from BPMN to BPEL representation is wrong, we’ll have a hard time detecting that.  And then how would it help with compliance? I can’t take the BPEL representation and load it in another BPMN tool and see the model… so we haven’t achieved the interchange that Bruce Silver is looking for, nor proven compliance…

Personally, I’d start with the following:

1. can i MODEL the specification model (ie, can i draw it in this tooling with standard components).  By this definition of compliance, a visio stencil may do the trick…
2. can the target environment EXECUTE the model.  the compliance model can describe the expected behavior of the model especially illuminating areas that might be up for debate (presently) but which should be standard (near future) interpretation.  Execution should be separated from modeling - from a compliance perspective - because BPMN does not exist solely for execution purposes.

Next, additional models could be provided that describe “illegal” models. If you model these in a compliant environment, it should, in some fashion, warn you that the model is not correct (either by throwing an informative error when you run/test the model, or by showing validation errors at design time).

Granted, testing compliance gets easier once BPMN 2 style storage spec gets released, but til then, there’s no reason we can’t have a representative model - it will just be harder to confirm compliance because we’ll have to actually do the work to model something.

Bruce points out that without the data storage spec, he’s not satisfied with the results - and I can’t blame him - but I think defining what would be a representative model to exercise the specification for a given tool is something that has to be done regardless of the storage medium.  And in the meantime, even the visual representation of such a model could help customers and consultants evaluate BPMN tooling appropriately for compliance.  Currently there’s just too much distance between here and there if we have to have a portable data schema before we can have a model (or set of models) that exhibits all the features of compliance.

Recently there has been quite an interesting discussion around Pi Calculus vs. Petri-Nets, and between BPEL and other implementations of BPMN.  The real discussion thread (consolidated place to read) is here.  The start of this discussion appears to be Ismael’s post on BPEL/Pi-Calculus vs. BPMN engines that use Petri-Nets.  This is pretty esoteric stuff to the average person, but it has sparked quite the debate.

You can find Ismael’s original post here.

If you’re technical, and you’re curious about what’s going on inside the black box BPMS you’re using, this is a great debate to read through.  The basic point made by Arthur and others is that BPEL is an implementation detail, and that BPMN is the most important standard because it represents the process, and as we all should know, representation affects how we think about process.  Arthur’s basic point is not that BPEL is deficient, but merely that it is like arguing about whether we should compile to Java bytecode or C-sharp byte code.  Or native machine code… do we really care as long as the code runs well on our target platform? :)

The victim in the debate is often “petri-nets” as compared to “pi-calculus”. Ismael makes the argument that BPEL is superior to the other approaches out there, partly because it is standards-based, and partly because it employs pi-calculus and that that is superior to petri nets for parallel processes. A great debate over whether BPEL really implements pi-calculus ensues, missing that the key point Arthur was making is that either a pi-calculus model or a petri-net model could represent parallel processing of BPMN models well. My personal experience reflects this as well, as I’ve participated in projects that leveraged both technologies and they both work… moreover, they don’t appear to be entirely mutually exclusive.

Lost in all of this debate is the B in BPMN:  The Business.  The Business doesn’t care as much about the specific technical differences of these approaches under the hood, so long as they execute the process faithfully.  Personally I just want to use the best tool for the job.  I’m going to “write” processes in BPMN.  I’m going to execute them and deal with differences in interpretation of the spec when BPMN is married up to execution engines, regardless of whether they are BPEL or other technologies (and the fact that it goes to BPEL doesn’t get me away from interpretation issues, because the issue is how is BPMN “interpreted” into BPEL just as much as it is an issue of interpreting BPMN into some other exeuction form).

The case for modeling without thought of execution…

I recently came across a blog entry from IDS Scheer on their Aris BPM Blog. Thanks to Sandy Kemsley for pointing me to it from her blog. Upon first read of the article by Sebastian Stein, I was struck by the difference in perspective between those who implement processes and those who model them.

For those who model (Modelers), the Model is the chief output and goal. Having a Model that will survive the test of time is the goal. You can see that bias throughout the post. In fact, the core philosophy is embodied right here:

“A business process model, depicted in one of the popular notations like BPMN or EPC, should not contain any technical details. If the underlying IT infrastructure or implementation technology changes, the business process model should remain stable. Your warning bells should ring if you have to change your business process just because you changed the implementation technology used.”

The two key points:

1. No technical details
2. stable with respect to technology changes

Something Overlooked by a Model-only Perspective…

But there are some problems with this… First, all the BPMN/BPMS tools that I have worked with support layering of processes. This layering allows the user to create a model that reflects Business sensibilities at the top layer, and if needed, several layers down in detail. So, if your need is to model something without “any technical details” you are not prevented from doing so in the BPMN-oriented tools that I’ve used.

Second, when you get to a certain level of detail, the process design should be informed by Technology. How so? It is important to understand if a transition is a manual or an automated one. Is it a non-value-added manual step? Then generally we want to automate it, or ideally remove it. Value-added manual step? Then generally we want to optimize around its constraints, but automation won’t be the goal. However, we may want to use technology to reduce errors, to improve time-to-execute, etc. In the posting, Sebastian doesn’t go into detail as to what he considers a “technical detail”, but it does beg the question: what is too technical? How about input and output data from a step in the process? These are critical process design considerations (if you know that a piece of data is required as an input, but you’re not sure where it comes from, you have a problem to resolve in your process design. And those inputs and outputs help define the “contract” of an activity or subprocess (or even of the entire process).

Third, Modeling tools today make it exceedingly easy to change a Model to adapt to Process changes. While it seems like a good idea to have a Model that is “stable” with respect to technology changes - the fact is, business processes change faster and more often than the technologies and systems that support them. The real problem isn’t keeping the Process consistent across technology changes - the problem is that the underlying technology may not be flexible enough to adapt to the new process model! At the least, the technology layer is often not agile enough to do so at a sufficiently affordable price and on a sufficiently short timeline (unless of course, that process technology layer is a good BPMS).

Fourth, the resilience that one needs, truly, is with respect to performance data. Performance data analysis is what will drive my process improvement activities, or identifying a process operating outside control limits. I need to be able to compare the performance of my process now to the performance of my process next year, to the performance of the process last year… If my process changes dramatically, how do I do that? Note: I’m not saying the technology changed. The process changed. So what I need is a way to track data that will make sense even in the face of relatively substantial changes in my process. BPMS tools can provide this facility, either baked in or via smart modeling practices, by taking snapshots of data at key milestones in the process that are not likely to change, semantically, even while the syntax (specific steps) of the process may change. To this end, even though the order entry portion of the process may change dramatically, you can still track information around the # of orders in, the value of those orders, the time it takes to process them, etc. even though the order entry process may go from highly manual to highly automated to web-self-service (or may yet encompass all three).

How do we Sum it up?

So the argument is that a modeling-only tool buys you a benefit (stability against technical change) that you don’t need, while not providing a benefit (technical agility with respect to business process changes) that you do need… yet still doesn’t address the key stability need -that of the measured process performance data. Moreover, the integration from most modeling tools to an actual functioning BPMS is, for the most part, non-existent from a practical perspective. Even when that integration exists, it is usually lacking process execution sensibilities in the model. There is a difference between drawing a model that represents the business needs and drawing one that can NOT be executed because of ambiguities and inconsistencies. For the best integrations I’ve seen so far, the products and the integration are all written by one vendor. (I’m definitely interested in seeing examples of this kind of tooling and integration and I’d be happy to write up reviews for such)

I’ve actually written an import to a BPMS suite using an Aris model as a starting point - and its hard!  There is a ton of non-relevant data in the export - positioning information, for example - and other information you need is difficult to lay hands on (roles/ownership).  To be fair, this wasn’t a BPMN diagram in Aris, but it WAS a diagram of a process, in a very unstructured environment.  It wasn’t any easier than parsing it out of Visio vdx files.  My recommendation, is that if you are given a process modeled in a modeling only tool - your first instinct should be to redraw that process in your execution modeling environment rather than try to import it (unless the importer ships with your product, in which case, give it a try!).  You’ll be surprised how fast you can recreate the model in your execution environment.

Now what?  Does an Execution-Oriented Model still make sense?

Okay. Given the arguments Sebastian presents, it seems he is suggesting that if you don’t know what product you will use to implement, you should use Aris to model your process (in fairness, if you don’t know what execution environment you will use, paper, visio, and Aris are all good options). And that, because it is “agnostic” with respect to the implementation tool you use, there is some derived benefit (this is really the point I disagree with). However, if you are going to build your solution in a completely different toolset, and you accept my premise that exports out of Aris (and other modeling tools) into execution BPMS suites leave a great deal to be desired, then you come to an interesting crossroads. Is he suggesting that once given an Aris model we should just write BPEL xml or some Java code to implement the process? or that we should then use a BPMN-oriented modeling suite to re-model and then implement the process?

In our experience, just “writing code” to codify a process in a modeling tool is a mistake. For one, how can the business determine if you have faithfully reproduced the process in your code? Extensive usability / UAT testing might reveal an answer, but it is a very expensive way to find out, and it only happens after all the code is written - and any mistakes will be very expensive to fix at this point because they could be simple mistakes or they could be conception or foundational mistakes. An Agile development process can help, but many organizations have trouble carrying off this approach with traditional software tools. If the technical team uses a BPMN execution environment (a BPMS) to build that process, then the business will be able to see the process in BPMN, a language (drawing) that they can understand, and understand the semantics thereof. By visually inspecting the design, the business can eliminate the greatest proportion of future defects at the earliest part of the design phase. And the technical team will implement each portion of the process in context of the business process at that point. And that is critical for providing useful business context to the technical team at the time they most need it.

Which Model is the Master?

And finally, now that your Process is implemented in an execution-oriented BPMS, as well as modeled in your modeling-only environment… which Model is the “Master”? Of course, you can make either answer work.  But let’s be clear about the choice you make :

Option 1:  The Model as drawn by the business in the modeling tool is the master.  it does NOT reflect what is actually happening in the business, or within IT, but it does show what the business was hoping the process would look like when the project started.  (Optionally, it may have even been revised and updated at the end to reflect some of the changes that implementation and testing revealed needed to be made).

Option 2:  The Model that works as agreed to by IT and the Business, drawn and executed in the BPMS environment.  This is the model that was actually tested by business users in UAT, by Unit Testing in IT, and system testing in IT.  This is the model that is actually running your business process in production, and it reflects reality.

Is it important that your original Model is resilient to technology change in this context?  Is it relevant that your model doesn’t have any technical details in it?

Or does it seem to be more interesting that there is now a BPMN model that represents what actually runs in your business every day, that can be measured and analyzed over time.  Does it matter that this BPMS is resilient to back-end technology changes (activities provide abstraction to what type of integration, and each integration can provide abstraction as to what specific systems are being tapped)?  Does it matter that this BPMS can support relatively rapid changes in process to adapt to your real business?  Does it matter that you can map the data you are tracking to your Model, to generate heat maps and highlight problem areas?

Well, you can guess where our heads are at.  Modeling is important, but Execution makes it relevant to the bottom line, and makes the Model itself more valuable.  If you want help turning your models into reality, we can help.

Overview and What’s New

We had a chance to see Lombardi’s latest Blueprint release at Driven last week.  We’ve used Blueprint before in earlier versions, but after revisiting it this time around, there are definite, significant improvements.  I thought we’d share our early experiences with Blueprint here, and specifically our reaction to some of the new features added.  What’s new? Visio import and a good export/import into Teamworks are big pluses. Word export is another new feature, and of course PowerPoint generation has been there from virtually Day 1.  Archiving and a revamped project screen were also introduced.

The New Stuff

Visio importing has long been the “holy grail” for process modeling tools.  If I had a dollar for every time someone asked me if a particular BPM product could import Visio models directly I would be rich!  However, Visio import into a process execution environment isn’t always all its cracked up to be.  Visio diagrams tend to be quite unstructured, whereas BPMN is very structured, and executable BPMN is even more structured in form.  Moreover, Visio models don’t have enough information attached to them to be immediately executable.  It is possible to run into issues of “who owns this model” once you import the Visio (the business may have the expectation that they can keep making modifications and “reimport” into Teamworks, for example).  At some point, the implementors must take over the model and own it to produce something executable.  I’ve been working on some models for OMG certification and I thought they would be a fun (albeit simple) set of examples to import into Blueprint for a test drive.  Blueprint imports these easily and accurately.  I went back to the archives and tried importing some really awful process diagrams circa 2004.  The results weren’t pretty (the original wasn’t pretty), but Blueprint imported the models nicely (a visio diagram with 10 tabs and one process per tab).  Going to the Diagram View I was able to sort out the diagram into swimlanes and go from there.  Interestingly, when I imported a diagram WITH swimlanes defined, Blueprint created those swimlanes and participants for me.

Word export works great, but didn’t work so well on my Mac.  On Windows it produces a professional document that is a good appendix or a good starting point for the doc.  It also eliminates the need to maintain two separate documentation formats - you can use Blueprint as your source and export when you need a Word doc snapshot.  Moreover, you can “snapshot” the process in Blueprint as well, so that if you need to reproduce the Word export later, you can…

The diagramming is much improved, from process mapping to actual BPMN
diagramming.  The new clickable controls are more intuitive (though I had to unlearn some previous notions because I’ve been using Blueprint since the early Betas).  As before, quick double-click on any item takes you to a pretty robust editing interface for adding participants, owners, experts, inputs, outputs, problems, documentation…  Its the Process Diagram (BPMN) view that has really changed over time.  Clicking on any line allows adding a new activity, gateway, subprocess or event.  Blueprint successfully removes the need to obsess over placement of each item in the diagram. It tends to organize things in neat right-angles, taking care of horizontal and vertical placement within a lane (you pick the lane though). 

Project Details Page - This page is so different as to feel like a new page.  You can export powerpoint, word, excel process data, and BPDM all from a simple project page.  From this page you can also move a process to a different project, archive it, or copy it.  The value of these things is more obvious after you import ten processes from Visio!

The Tried and True

The PowerPoint generation is a nice whiz bang feature (only available in the paid-for version).  It lays out the first section nicely: goals, followed by process mapping, followed by opportunities. I’d like to see it turn out a process-diagram view as well, but that may be a little tricky to squeeze into a small space.  The appendix is aptly named for its content, which is a blow-by-blow layout of participants, inputs, outputs, and problems with each step of the process.  I found the color scheme in this section a little jarring compared to the relatively slick look of the rest of the powerpoint.  All-in-all, its a great timesaver.  It ran so fast for me that I thought it hadn’t actually done anything the first time I ran it, ad I can pull together a reasonably professional presentation out of it using the generated slides as a starting point, and putting my own slides referencing ROI or Business Case up front, along with some concluding slides about the opportunities presented and how to tackle them.

BPMN Diagramming.  I actually like the diagramming portion better than
Teamworks!  And collaborating on the same process isn’t just possible,
its actually cool.  You almost look for an excuse to try to be logged
into the same process at the same time so you can try to step on each
other.  Blueprint handles all the conflicting edits really well.  I’m
impressed!  And I understand a good deal of the diagramming techniques
in Blueprint may be adapted for Teamworks in Teamworks 7, which is an
exciting development for Business Process implementors. 

The handling of Revision History is a clear advantage over Visio or any other tool I’ve worked with.  As far as I can tell, every change I make is tracked as a revision, such that I can recover lost work if I accidentally delete something important by restoring from those automated snapshots.  But I can also create my own Snapshot and name it, a point in time to be easily remembered going forward.  This takes the risk out of collaborative authoring, and out of revisions in general, because you don’t have to be worried about version history.  Cycling between versions is simple and fast, so finding the right one isn’t too stressful.

Room for Improvement?

The product has a clean interface and a good look overall.  But there are a few parts I’m not too excited about (yet) that mostly have to do with teeing up projects.  First, when prioritizing problems, you can select severity (low medium high) and frequency (low medium high) but you don’t have the ability to track how observable the problem is.  A problem that is very difficult to detect is much more serious than a problem that is easy to detect, and implies a different type of solution approach.  It might seem too complicated for some, but no reason you can’t ignore the third column by leaving it always at “low” if the notion of detection/observing is too complicated.

Second, the overall analysis screen leaves something to be desired.  It wasn’t immediately obvious to me that the impact scores I was looking at were the impacts of solving one problem across the set of higher level business goals I had established.  Once that was clear, I found myself wondering how these impact scores were derived.  To the layman it doesn’t appear that there is any structural relationship between a timing problem and my goals, but if I give them both high weight then the impact score will be higher against that goal…. So it isn’t immediately clear to me if the idea is to look at this page and then revisit your scores, or if it is really supposed to guide me in determining which parts of the process to look at.  But as long as the correlation is a black box, I don’t trust it.

Third, a process as we know has multiple dimensions to it; it’s not just the activities and relationships. Blueprint would benefit from adding Lean Flow data elements like WIP, Cycle Time, Throughput Time, Takt Time, Lead Time, and Number of Operators to each activity and to the process as a whole and have these values actually calculated, not just string values (perhaps an advanced user view or analysis view). Blueprint would then be much more effective in prioritizing and triaging process improvement, i.e. It would close the loop on the qualitative aspects (Severity, Frequency, and the missing Detection) and now provide quantitative elements as well such as time and volume. Again, this would really address the need to represent the many dimensions of business processes.

But for the primary purpose of capturing process maps and diagrams and capturing living documentation about those processes, Blueprint looks like a great tool.  Its come a long way in one year, and I look forward to seeing what else is coming down the pipeline. The progress so far is encouraging for what is to come.