Process for the Enterprise » BPMN 2.0

IBM Fulfilling BPMN 2.0 Promises?

Scott Francis — Wed, 16 Nov 2011 06:39:50 +0000

Bruce Silver reports that IBM is following through on its promises with respect to BPMN 2.0 in its next release, IBM BPM 7.5.1, which ships this week.

Not that IBM is covering EVERY corner of BPMN 2.0, but it is a significant advance – in that they are surprisingly supporting import and export of BPMN 2.0 XML from the Process Designer:

In a recent post, I talked about what “BPMN 2.0 support” really means, in both non-executable and executable model contexts. It’s not primarily about the notation, although a few shapes and symbols – notably non-interrupting events and event subprocesses – are new in BPMN 2.0. BPMN 2.0 support is really about the XML serialization, the ability to export the process model according to the XSD and rules of the spec, and ideally import from the XML as well. IBM BPM 7.5.1 can do both.

[...]

The important thing, though, is not just the palette of shapes but the fact that Process Designer supports export and import of the BPMN 2.0 standard XML format. (Oracle BPM 11g has had the BPMN 2.0 shapes for a year and a half and still cannot do that.) I haven’t seen the XML yet but I believe that the export includes data objects, data inputs and outputs, data association mappings (assignment), and other details of executable BPMN 2.0. At least I hope it does.

A few new palette items have been added as well, though as Bruce notes, we’re still missing explicit representation of message flows. I happen to agree with Bruce that this could improve the readability of IBM BPM models. And knowing how things are implemented underneath, I believe I’m qualified to say there aren’t really any technical barriers to having this “transparently” implement message flows, except to update some of the assumptions that go into the process canvas.

Great news, and great recap from Bruce.

Carrying the BPMN Interchange Torch

Scott Francis — Fri, 15 Apr 2011 18:31:27 +0000

Bruce Silver is still carrying the torch for BPMN2 interchange – thank goodness someone is- and has yet another update to explain the ins and outs of “valid” models.

That means an interoperability validation tool needs to test, in addition to the normal semantic rules of BPMN, rules that relate the semantic and graphics elements.

For example, the rule in question here might be something like this: In a diagram (i.e. page of the model), if a process is represented by a pool then all elements of that process on the page must be enclosed within the pool shape.

It is going to be a long road to get there but Bruce is, at least, making progress and uncovering some of the semantic (as well as syntactic) issues.

And since it took me a while to publish this post, we have an update from Bruce regarding BonitaSoft:

I have run across 5 BPMS vendors interested in my BPMN-I work: Activiti, BonitaSoft, Oracle, SAP, and IBM. Of the five, BonitaSoft is so far the most successful in actually implementing BPMN 2.0-based model interchange. Not only that, they are the only one so far that has implemented any of my suggestions for conforming to the xsd and BPMN-I.

I’ve said it before, I’ll say it again: when it comes to interchange, I think open-source offers the best alternatives. Activiti was probably first occupied by providing an upgrade path to folks running JBPM3 and 4, rather than from other BPMN2 tools (very few of which yet export proper BPMN2 XML).

I’d agree with Bruce’s assessment that so far, BonitaSoft does the best job importing someone else’s BPMN2. In a project last summer we exported JBPM4 to BPMN2 (via an xsl transform) and then loaded that into BonitaSoft, and while we ran into a few issues, we could only get BonitaSoft and Oracle to import BPMN2 at the time. Unfortunately for Oracle we had to add quite a bit of custom Oracle decoration to the XML to get the diagram to show up decently. (BonitaSoft has an auto-layout feature that helped).

(note: another update from Bruce)

ebizQ Revisits BPMN 2.0

Scott Francis — Fri, 04 Mar 2011 17:43:49 +0000

I was surprised to find this piece on ebizQ, covering the “emerging star of business process modeling“, BPMN 2.0. BPM and BPMN get a lot of coverage in the forums on ebizQ, but don’t as often show up as topics for articles (at least, that hit my radar screen). Of particular note is Jon Siegel’s concise separation of three key business process concepts: conversation, orchestration, and choreography:

The underlying idea is that “a set of processes form choreography if they communicate and there is no overarching process in charge of it,” explains Jon Siegel, OMG’s vice president for technology transfer. He says BPMN is fantastic at modeling, but the addition of choreography represents a major step forward in version 2.0. He says choreography was inserted belatedly because BPMN was first designed and came into widespread use a time when there wasn’t as much Internet-based commerce and choreographic processes were unusual, if they happened at all.

In contrast, Siegel says, “orchestration” is the BPMN word for conduct of a process internal to a business entity. “These days, a substantial fraction of e-commerce is choreography-based rather than conversation-based or orchestration-based,” he notes. “So the language had to be upgraded to take this into account, particularly for business users to help them to model their systems.”

I don’t think I’ve seen such a concise description, if you just take the first sentence of the first paragraph, and then the second paragraph, that neatly puts each type of interaction in its place.

Bruce Silver’s BPMN2 Interchange Update

Scott Francis — Mon, 28 Feb 2011 14:55:07 +0000

Bruce has been writing about BPMN2 model interchange for some time, and I’m a fan of the work he’s doing in this regard, and the light he’s shining on lack of vendor effort. Here’s his latest take on the status of things:

Last summer I posted on the challenge of achieving process model interchange via the BPMN 2.0 standard. In the half year since then, vendor progress toward that goal has been about zero. It seems that vendors, in particular the ones that drove the standard, don’t really care about this most fundamental user expectation of any standard. Ah well, no surprise there… But in the past couple weeks, some encouraging developments. Activiti and BonitaSoft – both are open source startups with a BPMN 2.0-based BPMS – have begun to tout BPMN 2.0 import and export. Neither one supports even the Descriptive subclass of the spec (what I call Level 1 in the training), but both vendors are full speed ahead at expanding the capabilities of their process engine.

I’ve always felt (even when I worked at Lombardi) that interchange would be best served in the open source market – no single vendor has much impetus to do it – and invariably there would be bugs that the vendor doesn’t view as high priority (hey, the lombardi to oracle transformation exhibits some obscure bug, not sure whether lombardi isn’t exporting right or oracle isn’t importing right… think either vendor ever wants to fix that? But in the open source world – even if the sources and targets are NOT open source, if the “interchange” (the spec) and the “transform” (the code that does the work) are open source, then (at least) developers who experience issues can actually attempt to fix them (even if those fixes are a temporary hardcode or hack).

Bruce goes on to comment on IBM:

It will be interesting to see if IBM takes Lombardi Edition in a BPMN 2.0 direction; I’m not sure Phil Gilbert is a believer in its value. If not, when Activiti and BonitaSoft finish the Common Executable subclass of BPMN 2.0, the BPMS marketplace could get very interesting.

I think Phil Gilbert’s issue with BPMN2 was that it got lost in the weeds (my interpretation based on reading his blog posts at the time). For example, every BPMN2 xml I’ve seen so far has several vendor-specific extensions (which are allowed by the spec, but likely meaningless to other tools until someone writes the adapters). As BPMN2 was getting started he was a fan and wanted it to succeed, and he drove lombardi to be one of the early adopters of a native BPMN 1.0 engine (not that it covered 100% of the spec- but there was no lossy transformation to some other format to interfere with the interpretation of your BPMN – Model preserving, to use Keith Swenson’s terminology). Certainly, BPMN2 implementations to-date have failed Phil’s test of providing tools so good that no one needs to bother to read the XML behind the model…

I don’t have any inside knowledge of IBM’s stance on BPMN2 with respect to their products, but I, too, will be interested to see what shakes out at Impact. If they don’t make progress it might be interesting to write a BPMN2 exporter or importer. But it is a fair amount of work to do as an outsider. I can see why IBM might not view interchange as a high priority – again, a good argument for an open source implementation of interchange.

As if to underscore my feeling that open source will pave the way, Activiti releases this video of an import of BPMN2 from another editor into Activiti:

Pretty nice demo. I do wonder if the import/export functionality would be better off as a standalone interchange open source project…

Anatoly on Signal Events

Scott Francis — Thu, 02 Dec 2010 19:35:51 +0000

If you don’t know much about Signal events in BPMN2, read Anatoly’s blog post on the subject:

In order to make the diagram work we must limit the signal propagation somehow. How it can be done?

The first thing that comes into my mind is an attribute that would limit signal broadcasting by the current process instance boundaries. Yet there is no such attribute in the standard. Under BPMN 1.x one may say that it’s implementation issue not covered by the standard. But BPMN 2.0 fully specify the process metamodel. Let’s look at page 281 of OMG document dated June 2010: signal has a single attribute – its name. Therefore, a signal will be transmitted to all process instances.

If the signal has only name then let’s use what we have. The diagram above may work if we could change signal name dynamically i.e. during the process execution. If we could name the signal “Process 999 Concept Ready” instead of “Concept Ready” then everything will be fine. But it’s a dirty hack and it’s hard to count on it. BPMS engines allow to change certain things during the execution (e.g. timer settings) but unlikely the name.

As Anatoly points out, the signaling examples given in several highly regarded books are fine, but when you think about how to make them work for the n>1 case, the logic breaks down.

It turns out that Lombardi’s BPM product (Now IBM’s Websphere Lombardi Edition) does not distinguish between Message flow and Signal events (the distinction between the two has only artificial merit). All events include an information payload (which can be arbitrarily simple or complex). The receivers control what they listen to by identifying a correlation key. The correlation key has to be something in the information payload, and as you can imagine, often one element of the payload is specifically designed to be the correlation key for listeners.

So sometimes implementations get the design right in a way that the spec-writers don’t – because of what I would call over-design. Let’s face it, writing specifications like BPMN2 goes against the principle of lean development (producing the minimum viable set of features).

So I find it interesting to see some of the challenges Anatoly has faced by facing in some cases a more pure representation of the BPMN 2 specification.

63 Types of Events

Scott Francis — Tue, 30 Nov 2010 18:58:19 +0000

Paul Vincent on BPMN 2 events (all 63 types of them) :

OK, “63 types of event” needs a bit of explanation (and justification). These consist of:

13 main event types: untyped, message, timer, escalation, conditional, link, error, cancel, compensation, signal, multiple, parallel multiple, and terminate

… across 8 situations classified by location in a process:

Start: top-level, event sub-process interrupting, and event sub-process non-interrupting

Intermediate: catching, boundary interrupting, boundary non-interrupting, and throwing

End

… but with some situations not requiring certain types of event (e.g. there is no “start cancel process” event) leaving58 63 event types defined [*1].Presumably BPMN tools will let the user specifiy the main event type and associate the correct symbol from the context in most cases, leaving us to consider just the 13 main event types.

Paul is right: 63 is too many. But I think he also hints at the right answer: the tooling should be able to pick the right symbol based on the context of the detail of the event (or conversely, show the author the right context if they pick a specific event). But there’s no reason for anyone to know 63 event types off the top of their head. In fact, they really don’t need to distinguish between all of the 13 “main” event types in most cases.

It is up to the BPMS providers to make the authoring experience powerful and yet simple enough to be manageable and useful. If the notation has gotten away from us a bit, then software can help reduce it back down to something manageable. One interesting aspect is that many of these event types are not currently supported by BPMS vendors- so depending on which tool you’re using you can cross off a lot of the potential event types! That isn’t exactly what I mean by “simplifying” but it is one way to get there, I suppose.

BPMN = Death to your Process?

Scott Francis — Thu, 09 Sep 2010 14:50:51 +0000

Antiamba’s Ligurio says that “BPMN can bring death to your process“:

The problem lies that BPMN is so complex to implement, that people made some workarounds, simplifying process maps. If a business process needs to be mapped (some don’t like ad-hoc) it should be done in a way everybody understands it, and it can be exchanged using multiple platforms that support process notation. Once it does not happen, there is a high risk of losing process data, because you can’t use things like intermediate message, or timer, that was represented in the process repository you need to move. Thus, you are sending your process data to oblivion (and BPEL, if used, also).

If process map is an asset, imagine re-training people that work in call centrer executing business processes by the book, telling that the decision gateway changed to something different, because process maps were updated to other tool. This has an impact in the way people perform tasks and in customer perception how the company execute. If process managers don’t have this concern they are very far from process execution where everything happens.

I can definitely relate to his criticisms of the tools around BPMN. We have a problem in our business right now. Almost every BPM software package supports BPMN. But what does “support” mean? It usually means, the BPMS uses a subset of BPMN notation standard. Anything produced would be BPMN compliant. However, it does not mean that ALL BPMN notation is available. Therefore, despite two tools speaking a “common language” there may not be a 1-to-1 translation of a BPMN model in one tool to a BPMN model in another tool.

The definition of BPMN 2 will help. By defining a storage format, it gives vendors a more concrete target to hit in terms of export/import. And it will make missing BPMN icons feel more like a bug than an annoyance.

But Ligurio takes the criticism to BPMN, rather than, primarily, the vendors. Vendors have got to get their act together and embrace BPMN 2 more fully. He blames BPMN for allowing you to model things so many different ways – but in a modeling world absent BPMN, this problem is worse, not better.

Earlier in his post, Ligurio criticizes Blueprint for its Visio importer being less-than-perfect. However, keep in mind that Blueprint is importing native Visio XML. This is not some standard BPMN XML that Blueprint is importing – because Visio doesn’t produce it, despite using a BPMN stencil. He seems disappointed with the idea of manually choosing that one kind of gateway icon maps to a split, or a decision gateway, rather than Blueprint just guessing. Keeping in mind that Visio can label anything a split or gateway, regardless of what it truly is, this seems to be asking a bit much. It isn’t as if you have to choose the mapping for every occurrence of an icon – just once per type.

The more accurate criticism is that Blueprint doesn’t (yet) support things like complex gateways and attached events (and pools vs. lanes). I know the Blueprint folks are trying to avoid these “advanced” use cases, but if they want Blueprint to stay relevant as users get more advanced or import valid BPMN diagrams from other tools, they simply have to add this functionality or risk becoming irrelevant. Having said that, Ligurio’s example diagram has a sequence flow crossing pool boundaries, which looks like a no-no according to the BPMN spec (they often get over-used in tools where they are supported).

He runs into similar issues with ARIS and ITP Commerce. Bottom line: he has a 25-page process. It will clearly get mangled in import routines today, and require lots of manual work to clean up.

My advice: for now, stick with one modeling environment. Choose one that starts with modeling and lets you seamlessly add execution details. If you use a “modeling only” environment it should stay at a very high level to work out concepts and disagreements – not to model execution-level details.

BPMN2 model portability, along with the diagram interchange format, will help these problems greatly. But the commercial vendors have to step up their game to support a much larger subset of the BPMN specification if they want customers to consider them compliant.

Meanwhile, it isn’t BPMN killing your process. It is BPMN exposing the problems in process definition and communication that were always there – but going unnoticed. They’re now coming to light, and vendors have an opportunity to really address these issues.

Universal Translators, Open Source, and BPMN 2

Scott Francis — Tue, 07 Sep 2010 14:27:49 +0000

Bruce Silver has recently posted his latest efforts to perform BPMN model interchange. It has been a recurring theme on his blog, BPMN 2.0 and its potential for interchange, as well as XPDL’s capacity for interchange. Recently Bruce went on vacation and spent his time in a rather dubious way (just kidding, Bruce!):

On my summer vacation I’ve been thinking a lot about the XML side of BPMN. While we usually think of BPMN as a diagramming standard, it is also – in principle – a model interchange standard, an XML format than can be exported from tool A and imported into tool B. BPMN 2.0, XPDL 2.1 (for BPMN 1.2), and XPDL 2.2 (for BPMN 2.0) all purport to deliver this. In reality, however, BPMN model interchange faces serious – some would say insurmountable – hurdles. I have been working on a number of tools to overcome these obstacles.

So the holy grail of model portability isn’t fully realized yet. But BPMN 2.0 does represent progress. He goes on to outline what has to happen to enable model portability:

1. An explicitly enumerated set of interchangeable model elements and attributes. The full BPMN 2.0 schema is too open-ended for unrestricted interchange. Fortunately, we now have such an enumerated list in the Descriptive and Analytic process modeling conformance classes in the BPMN 2.0 spec.
2. Modeling tools that unambiguously support all the elements and attributes in those conformance classes, meaning the mapping of diagram shapes and labels to the standard is unambiguous. We have a number of such tools today. My work has focused on two of them: Process Modeler for Visio by itp commerce, which supports both BPMN 1.2 and 2.0, and native Visio 2010 Premium, which is just BPMN 1.2.

I agree that getting a complete set of BPMN 2.0 elements explicitly enumerated is going to be difficult. And getting the modeling tools to support all the elements and attributes, not to mention exporting them as XML, is too much to expect from our modeling tool software partners. I believe the only way we’re going to get this kind of interchange is with an open-source project – because open source projects excel at eventually covering wide specifications as people volunteer to fill in the gaps. Interchange of data, and support for a wide specification, are both well-suited to open source projects – if they have the right framework in place that makes it clear how a newcomer can add to it (by, for example, adding a new exchange adapter). I can envision a pretty interesting model interchange utility that would read a variant of BPMN2 produced by some tool, and perform a little magic on it to produce “clean” BPMN 2. Then a second adapter that would produce BPMN 2 that a target BPMS tool expects. The author (say, Bruce) doesn’t have to solve all the possible interchange combinations, just provide a framework and an example, and see what other interchange adapters people will add to it.

That’s the beauty of open source – if you build the framework, and make it “accessible” to developers… then if there is a need for something, it will be built. If not, it won’t. Not too much wasted effort.

I See Business Professionals… Using BPMN

Scott Francis — Fri, 03 Sep 2010 05:11:47 +0000

So Jim Sinur really opened a can of worms the other day with his missive on BPMN, literally calling for it to burn baby burn – nothing like a gentle start like that to initiate a moderate discussion of the finer points of BPMN. I couldn’t help but respond both within his blog as well as on our own blog. I feel like Jim is letting the business off the hook – as he puts it – they don’t care about process, and they’re too busy making money to worry about process. I think this is a cop out. There is a comment thread on Jim’s blog that I’d recommend reading for the follow up discussion, and the original “burn baby burn” statement got walked back somewhat.

But the debate didn’t stay contained there. Keith Swenson chimed in, taking advantage of the opportunity to pile on BPMN. I can’t accept the black-and-white approach he is taking to the discussion, and so of course we had a bit of back-and-forth about whether BPMN is appropriate for no one in the business (his contention) or at least some people (my contention). I was challenged to name people within the business who read or write BPMN, which was quite easy to do, because this is the kind of stuff we do every day for work. I think the comment thread on his blog, and on Jim’s, or incredibly telling.

But there was also a great post from Neil Ward-Dutton on the subject, that captures my perspective perfectly:

Or – in other words perhaps – surely it’s not too much to ask non-IT participants in BPM initiatives to take a little time to learn some fairly straightforward modelling technniques?

From our case study work here I think what Scott is saying leads to a sensible, middle-ground answer – which is, that the applicability of BPMN depends on a number of factors; saying that BPMN (especially BPMN 2.0) either is, or is not, suitable for “the business” is too simplistic and black/white. It’s like saying Cloud Computing is the future of IT. Firstly it supposes that we have to talk about BPMN as an all-or-nothing proposition; secondly it supposes that “the business” is some kind of homogeneous organisation with one set of skills, experiences and inclinations.

I literally could not have said this better myself. He goes on to make another important point I agree with:

At the same time, though, there’s significant evidence to suggest that a core subset of BPMN symbols are absolutely usable by business analysts with experience in high-level analysis and design and provide great results in terms of delivering a common language across multi-disciplinary teams. I’ve come across many BAs who know and use (aspects of) BPMN as part of their armoury. They’re not “IT people” – they have business backgrounds and they work in line-of-business departments.

Great read from Neil.

In the comments on this one, Keith takes a nice shot at my assertion that understanding just a few BPMN shapes will allow you to read someone else’s thoughts on a process, or to communicate your own basic processes to others:

Also funny is the comment that learning six (or 7) shapes means that you understand the non-trivial interactions between those shapes at run time without needing the programmer’s insight into how systems function. That would be a little like saying that learning 26 letters makes you a Shakespeare, or able to read all western European languages. (But I must avoid use of similes since this apparently is sometimes confusing.) BPMN certainly is useful is some situations, it simply isn’t useful in all situations.

For the record, I don’t find Keith’s “similes” confusing at all. I find them inaccurate, misleading, and misrepresentative. And when we turn the analogy on its head, I think that proves how pointless they are. In practice, when people read Shakespeare they’re usually in school and get help from cliff’s notes, teachers, and fellow students. Not unlike those working with business processes and BPMN … and other tools (six sigma, lean, value stream, etc. ). Once again, I’ll point out that analogies are illustrative, they simply don’t constitute proof or refutation.

Jakob Freund of Camunda commented on Keith’s blog and summed up a reasonable reader’s interpretation of both Jim’s post and Keith’s post:

I think the main problem is that in both blog posts (Jim and yours) this very important distinction between “all” business professionals and “business (process) analysts” was not made. Analysts are not programmers but very often part of a business department, therefore a subset of “business professionals”. To throw all “business professionals” in one pot judging there skills in working with BPMN (or whatever) makes a good headline, but does not say anything useful.

Furthermore, there has not been made any distinction between “creating” and “reading” BPMN diagrams, and between the extremely different grades of complexity a process diagram can bear (please excuse my bad English).

But those are exactly the parameters you always have to look at when judging modeling approaches (no matter whether they are control flow – based, grids, prosa or what ever).

I guess it just comes down to this: BPMN is quite useful. It is even useful to people most of us would consider as “business professionals”. But there are other quite useful tools in our business process management space, and there’s no reason not to use each one when appropriate. I also recommend as practical reading, this post on practical application of BPMN by Jakob on his own Camunda blog. I liked how he closed his last comment:

cheers from my customer’s office in Germany (currently introducing BPMN in a 80k-people company, and huh, it works for Business people, but it’s bloody hard work to make that happen ).

Similarly, as I was writing on the same comment thread, I was about to head in to visit my customer, which also uses BPMN to communicate broad requirements between business stakeholders and IT. Regardless of what the theory says, the practical reality is our customers’ businesses are using this stuff.

August 2010 Blueprint Update

Scott Francis — Sat, 14 Aug 2010 19:55:42 +0000

IBM has put out the latest BPM Blueprint update today. It isn’t the most exciting update they’ve ever made to Blueprint, but of course I’d rather see incremental and frequent updates than one big overhaul for a service like this. The August release seems to focus on adding avatars (aka pictures), and has a security model tweak, and a much-needed update to the Word export.

I’m still looking for more support for expert features (not necessarily their target audience yet, but I think the time has come to include more expert functionality in a tool that also appeals to more general collaboration). Part of the reason I think this support is needed is so that users can import or export BPMN2 XML. Not that the users will care that it is XML, but this is the modeling format that they’re going to expect BPM tooling to support.