IBM has released a new update to BlueworksLive, on December 17th.  We had a preview just two days before it went live to discuss some of the thought behind the features. What interests me isn’t just the outcome but the thought and direction behind it.  Once again the specific features seem “small” but have interesting consequences and implications.

Starting with the shorter topics first:

The Word Export is much more pleasing to the eye than previous versions.  Having the graphics of severity and the diagram itself exported are a big help to the overall readability of the document.

The expand-all/collapse-all functionality in the Process Diagram is also convenient – especially when prepping to export a large diagram.

The BPMN export API works as advertised.  This is an important step to allow people to use BlueworksLive without feeling locked in.  After all, in a cloud “rental” model, one of the big fears is that your data is residing on someone else’s servers.  IBM needed to provide a clean way to get at that data and make it portable.  Not to mention, this lets customers apply some of their more standard SDLC to their requirements production in BlueworksLive.

First, there was quite a bit of attention given the Decision Discovery feature added to BlueworksLive.  I’d heard that this was coming, but I was picturing it as something that would be added to the automation features of BlueworksLive – I should have realized that the “Discovery” in the name implied that it would be part of the modeling (“Blueprinting”) part of the product.

The premise is that you set up a few Considerations (one or more).  The combination of these considerations is like a truth table.  However, BlueworksLive also lets you provide more than one conclusion – which is nice.  When modeling, we can label the column headers smartly, allowing the contents of each cell to be concise and simple (Yes/No, >$500/<$500, etc.).  Finally, we can label the conclusions well- “Adjustment Required”.  If we have more than one conclusion, it gets its own column to keep ideas separate.

A couple of surprising perks:  you can reorder columns and rows with a simple drag-and-drop.  Look, this makes sense given the point of the tool – flexible discovery of decisions.  But this is the kind of fit-and-finish often missing in enterprise software.

I also appreciated that they thought through why the cells should be free-form rather than constrained to integers or strings or a particular data type. The goal is to leave discovery unconstrained.  Plenty of time for constraints when you move into modeling for execution (had this been targeted at execution, you can bet there would have been tight treatment of data types).

Like David Brakoniecki, I think BlueworksLive is showing that it will live up to its promise as a BPM discovery tool.  Not because it does everything it needs to do today, but because IBM have shown that they’ll keep turning the screws until they get there.  His take on the impact of tiny changes at this point in the maturity of the product:

Now, at the push of a button, the process documentation and process diagram can be exported into a single word document. Basically, this document becomes the high-level scope of any potential BPM deployment or process improvement initiative. All of the great power of Blueworks around social collaboration and process discovery now can painless produce a document to playback to the client or business teams for review and iterative improvement.

SaaS products really emphasize the benefit of incremental improvement.

Peter Schooff of ebizQ asks: “How big of a role does BPMN play in today’s projects?” And the responses were interesting to me.  Most of them took the line that BPMN isn’t that important, or that they don’t typically use it.  That someone who fails to understand BPMN will fail to understand the process, just as surely as you might get false agreement in a process by being vague in its description.  My response:

We use BPMN all the time in our projects, but we do a lot of process implementation projects, and it is a good fit for the tooling we use.  Craig makes a good point about someone not understanding BPMN precisely – but at least BPMN has a precise definition – unlike the usual whiteboard drawings.

I like to get on the whiteboard and build out the diagram and explain and talk as we do it.  By seeing and hearing the build-out, there’s much less chance of confusion about the interpretation.  And when you’re done, the diagram is actually accurate for someone who reads it “after-the-fact” if they know BPMN.

Having said all that, BPMN is a means to an end.  It isn’t the goal, it is a tool.  There are other tools and in the right time-place each one is useful.

Whenever there is a standard there’s inevitably a bit of back-lash against the standard from experts – almost as if it is a badge of honor to buck the industry standard.  I say this knowing full well that I’ve done it before myself!  But with experience comes a little wisdom and perspective and I don’t hand out badges of honor for either obeying doctrine or bucking it.  The badges of honor come for delivering great results.

BPMN isn’t perfect.  In fact, to misquote Churchill, it is the worst form of process modeling that has been tried… except for all the others that have been tried from time to time.

Bruce Silver reports that IBM is following through on its promises with respect to BPMN 2.0 in its next release, IBM BPM 7.5.1, which ships this week.

Not that IBM is covering EVERY corner of BPMN 2.0, but it is a significant advance – in that they are surprisingly supporting import and export of BPMN 2.0 XML from the Process Designer:

In a recent post, I talked about what “BPMN 2.0 support” really means, in both non-executable and executable model contexts.  It’s not primarily about the notation, although a few shapes and symbols – notably non-interrupting events and event subprocesses – are new in BPMN 2.0. BPMN 2.0 support is really about the XML serialization, the ability to export the process model according to the XSD and rules of the spec, and ideally import from the XML as well.  IBM BPM 7.5.1 can do both.

[...]

The important thing, though, is not just the palette of shapes but the fact that Process Designer supports export and import of the BPMN 2.0 standard XML format.  (Oracle BPM 11g has had the BPMN 2.0 shapes for a year and a half and still cannot do that.)  I haven’t seen the XML yet but I believe that the export includes data objects, data inputs and outputs, data association mappings (assignment), and other details of executable BPMN 2.0. At least I hope it does.

A few new palette items have been added as well, though as Bruce notes, we’re still missing explicit representation of message flows.  I happen to agree with Bruce that this could improve the readability of IBM BPM models. And knowing how things are implemented underneath, I believe I’m qualified to say there aren’t really any technical barriers to having this “transparently” implement message flows, except to update some of the assumptions that go into the process canvas.

Great news, and great recap from Bruce.

John Reynolds wrote a post recently about Interdependent tasks, and the resulting complexity. John takes a simple example, the vacation approval process, and then points out what makes the difference between a cute model and a real implementation:

Sam can’t really (in good conscience) make a Decision about any Vacation Request in isolation.  Only one Employee can be absent at any one time, so every Decision that Sam makes potentially effects all of the Pending Requests.  To be fair to everyone, Sam needs to take into account all of the Pending Vacation Requests before rendering a Decision on any of them.

And further:

Examples like these are what makes implementing “real world” processes hard.  Processes seldom execute in a vacuum, and work done within one instance often influences other instances.  Participants often have to consider multiple Tasks together, rather than performing each task in isolation.

He’s right, of course.  This is why a demonstration of a BPM solution can look easy, but the actual implementation actually takes real work and thought.

John purposely held back from suggesting a clever BPMN modeling solution or other trick of the trade to give us something to think about. I’ll give you my thoughts on how to approach the problem.   But in a general sense, this falls back into a general process pattern:

• a process model that does a decent job of representing one process “instance”.
• another process that manages the set of all processes
• yet another process that is the maintenance and improvement of the process definition and the management of process instances collectively.

What John is describing is a variation on the second level of process.  It already goes without saying that we need to manage a set of vacation requests collectively.  The extra wrinkle is that at a step in the approval process, the process should present context to the user, that likely includes:

• All pending and approved vacation requests for other team members
• Possibly other pending and approved vacation requests for people on other teams
• Remaining vacation days for this person
• Remaining days in the year in which to use those days
• Time since last vacation

All of this information gives the Approver context in which to make the decision.  The individual process’ execution flow hasn’t gotten more complicated. But the implementation details of that Approve step got more interesting.  Luckily, the information above will be pretty easy to provide if your BPMS had reasonable tracking and introspection capabilities.  So if we simply invest some trust in our user, and supply them with the information they need to make a decision, we’ll get really good outcomes with minimum implementation headaches.

Sometimes the really interesting bits in a process implementation aren’t in the BPMN.  As John points out, we shouldn’t assume that every detail should be captured in a BPMN model.

I’m with Stephen White on this one:

The process above is not an extremely complex or hard to understand BPMN Process, but it is a perfectly valid one. BPMN was fully intended to support such modeling. So what is the problem? Why are there complaints about BPMN?

Look, BPMN has more complexity than it needs to have for many uses, and less complexity than it needs for other uses, but I’m with him on this one.

Is it too complex to use for simpler models? No.

Do these simpler processes require too complex a BPMN model to be understandable? No, they do not.

To me, the focus on complexity is a bit off.  The issue is expressiveness.  I don’t think that BPMN is too complex for the expressiveness it provides.  What makes one tool better than another has a lot to do with its expressiveness.

Stephen has it right when he says “The burden, then, is not on BPMN as a specification, it is on companies that develop process modeling tools or methodologies to help their users effectively create business processes – using the BPMN language.”  (Of course, it would help if the spec writers don’t get too carried away, and actually take a break for a while to see how BPMN2 shakes out before starting on a new version).

He has it right because often these simpler conceptual models evolve into more complex executable models (or just more detailed models), and it is reasonable that the same people who understood the simpler model want to be able to see and grasp how the more complex versions relate to the original simple model.  As we’ve commented before, most business users can get by on just a fraction of the BPMN language.

XKCD often hits the target squarely, and they’ve done so again with this little gem that everyone in the BPM / BPMN space can appreciate:

Sure, maybe there were only 3-4 BPM standards going to 5 when BPMN was introduced… but now there’s BPMN 2… and one gets the feeling that there will be more!

But Gosling has a fantastic repartee that even more closely nails it:

Tell me the BPMN 2 specification doesn’t feel like that…!

Anatoly has posted another process pattern: the Incoming Processor Pattern.

It is a good pattern – and actually forms the basis for a lot of variants of that pattern (in a sense, it is the base case of a wide variety of patterns).

Essentially, Anatoly describes a process that begins with a message start event listener:

Beyond that, Anatoly points out there is a challenge with respect to having more than one instance of this process:

But here is the question: how would the message from client John Smith find its way into specific process instance associated with him? Let’s not forget that there are about thousand of instances behind the single pool “Credit card issuance” shown in the diagram. [...]

But when a message is sent into the middle of a process (i.e. to the intermediate event) one must specify the process instance ID. Moreover, the process instance ID is the internal BPMS data so we can’t expect that a message from an external entity (from collapsed pool on a BPMN diagram) would contain it explicitly. In our case it’s just client’s words “hello, I want my card.”

He even diagrams this “incoming processor”:

I found his description of this pattern interesting, because the BPMS that I use most often (Lombardi / IBM BPM) wouldn’t require the incoming processor pattern for this use case.  The typical case for BPMS’ is that the event listeners can hear events targeted at their process instance id.  But Lombardi took a different tact that IBM has preserved – events “correlate” on any piece of process instance information – process instance id is certainly one option, but so is a name, or a debit card number.  And the logic to find and message the right process instance is built into the engine.

Not only that- a message my correlate with more than one process instance -automatically- based on the various correlations.

So this is a case where the BPM engine allows the models to be both more expressive and more concise, and to leave the incoming processor logic as a black box that “just works.” But if your BPMS doesn’t do this for you, or if you need to do additional pre-processing logic before the process instance is triggered, this pattern will do the trick.

Once again, a great, thought-provoking post from Anatoly’s blog.

Anatoly often posts the best examples and cautionary tales in BPMN2. In the latest post, he derides the limited usability of Swim Lanes in BPMN 2 – And he has a point.

On the one hand, embedded subprocesses can’t have swim lanes (the best way to think about these is simply a set of collapsed activities, for notational convenience).

On the other hand, “Reusable subprocesses introduce additional complexity because unlike embedded they are executed in a separate data context”.

Anatoly’s conclusion is that overusing reusable subprocesses is bad practice, because of this overhead.  I conclude differently:  the BPMS should minimize or optimize this overhead – and the business process designer should be able to ignore it on a robust BPMS.  To make a coding analogy:  we should be talking about the difference between an embedded block of code and a function call.  Yes there is overhead, but it should be managed by the BPMS transparently.

In fact, some BPMS authoring environments don’t even allow for the embedded subprocess- treating it as just a special case of the reusable subprocess who’s primary difference is that it doesn’t happen to be re-used. I don’t think we should optimize around the shortcomings of a particular BPMS too much, in terms of our general BPMS modeling advice.  However, I’ll concede that once you’ve chosen your BPMS, you might as well optimize somewhat around its capabilities as they become known to you, and model accordingly.

Anatoly has posted an example regarding orchestration and collaboration, and he has two particularly good pieces of advice at the end, though the whole post is worth reading:

Rule 1. If your attempts to model a process are unsuccessful because some significant aspects are missed repeatedly then stop and think – maybe in fact it’s not a single process but two or more?

Rule 2. Do not overuse collaboration – stay within the orchestration as long as possible. Never introduce collaboration into a diagram just because you’ve recently learned how to do that.

Both of these are excellent.  Rule #1 is exactly how I approach splitting something into multiple processes.  Rule #2 is just a good rule in general for any piece of functionality you’re not overly familiar with, or any advanced functionality.  Start with the basics, and only go to the more advanced features when you clearly need to.

Score one for the “BPMN Zealots“, as Bruce Silver reports:

Today Software AG announced a tight integration between ARIS, its leading Business Process Analysis suite, and webMethods, its SOA-based BPM Suite.  The integration features roundtripping and continuous synchronization between business-oriented and developer-oriented models in those tools.  The medium of interchange is BPMN 2.0 XML.

EPC was previously ARIS focus, but as Bruce Silver says:

…and Software AG now recommends that for new modeling projects where the intent is to automate in webMethods, ARIS users model in BPMN 2.0 from the start. This move is really heartening to me, and highlights the key new feature of BPMN 2.0, which is interchange between modeling tools.

Glad to see ARIS and Software AG getting on the BPMN bandwagon.  Interestingly, they ran an April Fool’s joke about ditching EPC…Sometimes the April Fool’s jokes hit a little close to home.

Good review of Bruce’s training:

There are few people who have this depth of BPMN knowledge, and Bruce is the only one who I know who is doing this as a professional trainer: his is the only BPMN course that I recommend to my clients. He needs to work out a few bumps in how the online course works, but in general, I thought this was a great course, perfect for a business analyst who is already doing some process modeling but doesn’t know any BPMN, but also informative for those of us with some prior knowledge of BPMN.

Sandy’s review is quite an endorsement of Bruce’s training.  Of course, any online training environment is a bit of a challenge compared to on-site or in-person delivery.  Sandy’s isn’t the only endorsement he’s had – many BPM product vendors have also implicitly or explicitly endorsed his training over the years.

Bruce Silver is still carrying the torch for BPMN2 interchange – thank goodness someone is- and has yet another update to explain the ins and outs of “valid” models.

That means an interoperability validation tool needs to test, in addition to the normal semantic rules of BPMN, rules that relate the semantic and graphics elements.

For example, the rule in question here might be something like this:  In a diagram (i.e. page of the model), if a process is represented by a pool then all elements of that process on the page must be enclosed within the pool shape.

It is going to be a long road to get there but Bruce is, at least, making progress and uncovering some of the semantic (as well as syntactic) issues.

And since it took me a while to publish this post, we have an update from Bruce regarding BonitaSoft:

I have run across 5 BPMS vendors interested in my BPMN-I work: Activiti, BonitaSoft, Oracle, SAP, and IBM.  Of the five, BonitaSoft is so far the most successful in actually implementing BPMN 2.0-based model interchange.  Not only that, they are the only one so far that has implemented any of my suggestions for conforming to the xsd and BPMN-I.

I’ve said it before, I’ll say it again: when it comes to interchange, I think open-source offers the best alternatives.  Activiti was probably first occupied by providing an upgrade path to folks running JBPM3 and 4, rather than from other BPMN2 tools (very few of which yet export proper BPMN2 XML).

I’d agree with Bruce’s assessment that so far, BonitaSoft does the best job importing someone else’s BPMN2. In a project last summer we exported JBPM4 to BPMN2 (via an xsl transform) and then loaded that into BonitaSoft, and while we ran into a few issues, we could only get BonitaSoft and Oracle to import BPMN2 at the time. Unfortunately for Oracle we had to add quite a bit of custom Oracle decoration to the XML to get the diagram to show up decently.  (BonitaSoft has an auto-layout feature that helped).

(note: another update from Bruce)

Bruce Silver has been teaching the Process Modeling with BPMN class for years in cooperation with the BPM Institute, but this year:

If you were thinking of taking my 2-day class Process Modeling with BPMN at a BPM Institute event in Chicago, San Francisco, DC, or New York… well think again.  Rather than pay me for the classes I did last year, [...] Brainstorm/ BPMInstitute have just now swapped in their own instructor and are trying to pass it off as the same class I have been giving for the past four years.

Bruce Silver is well-known for his writing, reviews, advocacy, and training in BPM circles.  In particular, he is sought after for his training classes.  This strikes me as poor form by the BPM Institute.  Clearly Bruce isn’t pleased.

The good news?  You can get Bruce Silver’s training, directly from Bruce, online in April.  Details here.

Architects are discovering BPMN, and Bruce Silver has explained how business architecture relates to BPMN or BPM more broadly:

When business process architecture enumerates processes that are not really processes, or activities that are not really activities, usually the problem is that what is enumerated are actually functions or capabilities, not specific actions (or sequences of actions) performed repeatedly in the course of business, with each instance having a well-defined start and end.  Even when labeled VERB-NOUN to suggest an action, they may be things performed continuously rather than repeatedly. Activity names starting with “Manage”, “Monitor”, or “Maintain” typically fall into this category.

I’m glad to hear that Bruce observes that architects are increasingly discovering BPMN – we’re seeing the same thing at BP3, though we’re not as involved in BPMN training.  Good to see that market validation.  I think it is also a symptom of more organizations looking to extend BPM implementations beyond the project.

I was surprised to find this piece on ebizQ, covering the “emerging star of business process modeling“, BPMN 2.0.  BPM and BPMN get a lot of coverage in the forums on ebizQ, but don’t as often show up as topics for articles (at least, that hit my radar screen).   Of particular note is Jon Siegel’s concise separation of three key business process concepts: conversation, orchestration, and choreography:

The underlying idea is that “a set of processes form choreography if they communicate and there is no overarching process in charge of it,” explains Jon Siegel, OMG’s vice president for technology transfer. He says BPMN is fantastic at modeling, but the addition of choreography represents a major step forward in version 2.0. He says choreography was inserted belatedly because BPMN was first designed and came into widespread use a time when there wasn’t as much Internet-based commerce and choreographic processes were unusual, if they happened at all.

In contrast, Siegel says, “orchestration” is the BPMN word for conduct of a process internal to a business entity. “These days, a substantial fraction of e-commerce is choreography-based rather than conversation-based or orchestration-based,” he notes. “So the language had to be upgraded to take this into account, particularly for business users to help them to model their systems.”

I don’t think I’ve seen such a concise description, if you just take the first sentence of the first paragraph, and then the second paragraph, that neatly puts each type of interaction in its place.

March has arrived, as well as Activiti 5.3:

• Added BPMN multi instance (==foreach) support
• Added BPMN intermediate timer catch event
• Added business rule task with Drools integration
• Improved Spring integrations: added possibility to limit visibility of beans and also exposed spring beans in scripts
• Added administrator console to manage users and groups
• Added automatic DB type discovery
• Various bug fixes

Not bad for a point-release.  Activiti seems to be adding BPMN support pretty quickly  – important to eliminate those “but it doesn’t do … ” objections early on.  And, to set a precedent of just making progress on extending BPMN support. Impressive so far!

Bruce has been writing about BPMN2 model interchange for some time, and I’m a fan of the work he’s doing in this regard, and the light he’s shining on lack of vendor effort. Here’s his latest take on the status of things:

Last summer I posted on the challenge of achieving process model interchange via the BPMN 2.0 standard.  In the half year since then, vendor progress toward that goal has been about zero.  It seems that vendors, in particular the ones that drove the standard, don’t really care about this most fundamental user expectation of any standard.  Ah well, no surprise there…  But in the past couple weeks, some encouraging developments.  Activiti and BonitaSoft – both are open source startups with a BPMN 2.0-based BPMS – have begun to tout BPMN 2.0 import and export.  Neither one supports even the Descriptive subclass of the spec (what I call Level 1 in the training), but  both vendors are full speed ahead at expanding the capabilities of their process engine.

I’ve always felt (even when I worked at Lombardi) that interchange would be best served in the open source market – no single vendor has much impetus to do it – and invariably there would be bugs that the vendor doesn’t view as high priority (hey, the lombardi to oracle transformation exhibits some obscure bug, not sure whether lombardi isn’t exporting right or oracle isn’t importing right… think either vendor ever wants to fix that?  But in the open source world – even if the sources and targets are NOT open source, if the “interchange” (the spec) and the “transform” (the code that does the work) are open source, then (at least) developers who experience issues can actually attempt to fix them (even if those fixes are a temporary hardcode or hack).

Bruce goes on to comment on IBM:

It will be interesting to see if IBM takes Lombardi Edition in a BPMN 2.0 direction; I’m not sure Phil Gilbert is a believer in its value.  If not, when Activiti and BonitaSoft finish the Common Executable subclass of BPMN 2.0, the BPMS marketplace could get very interesting.

I think Phil Gilbert’s issue with BPMN2 was that it got lost in the weeds (my interpretation based on reading his blog posts at the time). For example, every BPMN2 xml I’ve seen so far has several vendor-specific extensions (which are allowed by the spec, but likely meaningless to other tools until someone writes the adapters). As BPMN2 was getting started he was a fan and wanted it to succeed, and he drove lombardi to be one of the early adopters of a native BPMN 1.0 engine (not that it covered 100% of the spec- but there was no lossy transformation to some other format to interfere with the interpretation of your BPMN – Model preserving, to use Keith Swenson’s terminology).  Certainly, BPMN2 implementations to-date have failed Phil’s test of providing tools so good that no one needs to bother to read the XML behind the model…

I don’t have any inside knowledge of IBM’s stance on BPMN2 with respect to their products, but I, too, will be interested to see what shakes out at Impact. If they don’t make progress it might be interesting to write a BPMN2 exporter or importer. But it is a fair amount of work to do as an outsider.  I can see why IBM might not view interchange as a high priority – again, a good argument for an open source implementation of interchange.

As if to underscore my feeling that open source will pave the way, Activiti releases this video of an import of BPMN2 from another editor into Activiti:

 


Pretty nice demo. I do wonder if the import/export functionality would be better off as a standalone interchange open source project…

Anatoly’s blog is one of my favorites, and his post on Star Wars only reinforces it. And yes, you can model just about anything:

Bruce Silver has periodic updates on his training offerings, and when we pick up on them I’ll reference them here, because I think his courses offer a good opportunity for people to get a better sense of BPM. The latest update is a bit of an F.A.Q. for BPMN Training, and includes information about certification, standards, etc.:

I provide BPMN training through a variety of channels.  Right now, the only one that offers certification is BPMessentials, which is a joint venture of Bruce Silver Associates and itp commerce, maker of the Process Modeler for Visio BPMN tool.  The standard BPMessentials online training (wtc.01), private on-site 2-day classes, and the 2-day public classes offered four times a year through Brainstorm/BPMInstitute all include certification.  An abridged BPMessentials Level 1-only class (wtc.l1.01) and classes based on Microsoft Visio Premium 2010 do not come with certification.

Alberto Manuel’s excellent blog takes on the “year in review” post before anyone else this year!

He’s clearly a fan of the emergence of the ACM meme, and not a fan of the Social BPM meme.  He takes issue with BPMN2′s complexity:

BPMN 2.0: Well the new standard finally hit the cloud doesn’t it? I hate BPMN for the following reasons:
- It takes more time to draw a process with BPMN that other process notation language;

- Never provided 100% compliance with BPEL and XPDL, providing data loss when importing the never ending, complicated, difficult to understand flowcharts (read my previous post on Antiamba about this).

Anyway despite all the clutter provided it’s becoming a standard, because BPM community never cared about how processes should be mapped.

Well, I don’t fault BPMN2 for not complying with BPEL – that argument has been fought and won – BPEL doesn’t represent everything BPMN can represent without losing data/information – that’s a deficiency of BPEL’s representation more than BPMN’s.

I wouldn’t impugn the intent of the BPM community to having “never cared” about how processes should be mapped.  Bruce Silver, for example, is part of the BPM/BPMN community and he clearly cares (and has the classes and writings to prove it! ).

I’m also more optimistic about the PKBoK effort than he is (although, the name seems horribly circular and redundant…. ).   These types of efforts take a long time, however, so it isn’t like anyone should be waiting on the output of PKBoK to get started on their BPM initiatives!

I’m not quite ready to look back on 2010 and sum it up.  That’ll have to wait til  a bit closer to end of year.