Every so often, someone makes the argument that essentially nothing has changed in the world of BPM.  Actually, this isn’t unique to BPM – it is a common refrain across all kinds of software categories.

And it is tempting to buy into this, when you realize how durable a writeup like the history of BPM can be (this isn’t a bad writeup, by the way). But one has to remember that history *should* be a bit durable with the passage of time. Jon Pyke recently opined that no one in BPM has anything new to offer.  But the substance of his post is that the marketing isn’t differentiated, and the product positioning isn’t differentiated, and further, that the people who work at these companies don’t know what differentiates them.

Having worked on both sides of the coin (on the software side and the consulting side), I have to disagree with some of Jon’s conclusions because the input data is more limited than he’s realized.

For example:  of course the marketing messages are commoditized among BPM vendors, as is product positioning.  Sadly, it is perfectly easy to copy someone else’s marketing and positioning – it takes minutes, hours, or days at most to do so.  I think the tragedy in BPM is that all the vendors seem willing to implement “checkbox” versions of every feature that analysts care about, rather than go deep with the product in areas that produce real value for customers.  Unfortunately, vendors have been largely rewarded for such behavior with good product comparison reviews and chart placement – but their customers have not been similarly rewarded by this kind of investment.

There are real differences in these products, but it requires a much more in-depth understanding of the products to appreciate it.  Can we be surprised that customers have a hard time achieving this level of product knowledge during the evaluation process?  I used to work with someone who was always telling me how “nothing had changed” in the last 5 years in BPM – at a time when, 5 years before, there was no BPMN – and at that time, BPMN was the de facto modeling standard for BPMS offerings.

There has been quite a bit of innovation in the last 10 years in BPM, but some of the best ideas didn’t get enough investment to go from interesting to indispensable, and some of the best ideas really were commoditized – picked up by all the pure play vendors (and later, but the stack vendors).  I could argue that nothing much has changed since 1994, when I wrote a sales process application that leveraged Lotus Notes VIP to replicate sales data and manage workflow between Sales, Sales Engineering, Manufacturing, and R&D.  But that would be a bit disingenuous. I could write that solution in 1994, but I didn’t have a way to communicate the process to the business (BPMN), that accurately reflected the implementation so we could make sure we had it right.  And I didn’t have a standard data representation for analyzing the process data (for business process improvement).  I certainly couldn’t handle in a trivial way a process that required parallelism the way I can with executable BPMN models.

Jon says:

That’s why, and I’ve said this many times before, BPM is far too important a topic to leave in the hands of product vendors – this is a Business thing – the clue is in the name BUSINESS PROCESS MANAGEMENT. [...] They will talk about the presentation layer, the SOA integration support, analytics, modeling and what have you.

This is, unfortunately, true of a great many people in the BPM ecosystem.  However, it doesn’t sound like a few of the pure plays that have been acquired (take a gander at Phil Gilbert’s blog for several essays on the subject of business taking control back from IT), and it doesn’t sound like some of the new vendors (ActionBase as one example).  Whether these vendors are well-represented at Gartner conferences is another question entirely, but it is ironic that it is typically the small vendor that is more focused on business value. One of Jon’s last points:

Every vendor believes they are unique but the fact of the matter is many of the software metaphors used in these products were defined by pioneering workflow vendors such as Filenet, Staffware, Plexus and Wang

Well, honestly, software metaphors are meant to be re-used, so I’m not sure that that, in and of itself, is a point of criticism.  For example, some of the metaphors in older integration technologies like CORBA and DCOM are embodied in SOAP and WSDL – but not many would argue that web services weren’t a step forward. And if BPM functionality is truly commoditized – is that bad for the customer and the industry if it becomes more standard and more cheaply available?

I know it is tempting to look at the glass as half-empty, but with so many BPM vendors performing so well in a challenging environment, its hard not to look at the glass as half-full. Call me an optimist.

Sandy Kemsley has a few good updates on these topics.

In the first, she releases a review on IBM’s BlueWorks online community for BPM.  Some of the interesting tidbits:

• IBM BlueWorks uses Flash.  Interestingly, Lombardi started with a flash interface (and it was a very slick prototype) and scrapped it for GWT/Ajax.  Why?  Because Flash was just not stable enough to support what they wanted to do (even in the early stages), and they could see that they were going to run out of “room to run” with Flash, whereas in GWT they felt the sky was the limit in terms of layout and functionality over time.  Quote from her blog: “The process designer is Flash-based, and it only took me about 5 minutes to crash it; luckily, it saved as I worked, so I didn’t lose any work.”
• She gives pretty good marks to the content they included, which might form the basis or significant contribution to a CoE.

Speaking of BPMN modelers in the cloud… Sandy followed up with a good post about why locating your hosting services in different locales matters (a lot) to customers.  Although I can point anecdotally to data points (companies) that don’t have an issue with the location of servers (unless it affects performance), I can also attest that quite a few customers in other geographies *do* have an issue with hosting location.

Hopefully as these services mature they can offer more options for their customers.  Certainly IBM has the global reach to put its cloud / community offerings in as many geographies as it needs to to be sufficient for its customers.

A research paper was recently published, which purports to research improving the traversal of large hierarchical process repositories.  After a good introduction and background to the topic, the authors quickly jump to IDS Scheer and IBM’s Websphere as examples of BPM tools.  And next, the authors jump to the conclusion that 3D modeling will solve the problem of complex modeling in BPM.

However, the evidence presented is incredibly unconvincing.  3D modeling may hold some promise for BPM, but the examples presented don’t look more understandable to me – they look less understandable. It seems like, conceptually, 3D might take advantage of our innate spatial relationship capabilities and render complex things more understandable… but that isn’t what comes across from the examples in this paper.

Much more powerful than “3D” is the concept of containment, and representing containment. Containment is a form of abstraction that allows me to talk about the behavior of a black box, but then shine a light inside the black box and you can see additional details at a finer granularity, only if and when you choose to.  Containment is the key abstraction needed for understanding large models, and 3D just looks like a fun science project by comparison.  Abstraction by containment is a big part of how we process geography, maps, and directions… and process flows are a geography (road / map) of sorts.

Meanwhile, we’ll have to keep waiting for the silver bullet for making large complex models more understandable or accessible!

Pretty interesting update from Joram Barrez on jBPM – looks like it now supports BPMN2.0. (or, more accurately, it will come January 1st).

Its a pretty interesting look under the hood of one of the top open-source solutions in the BPM space.  The BPMN2 implementation uses the jBoss PVM (process virtual machine) to execute the BPMN, rather than transforming BPMN into BPEL  or JPDL or some other XML format.

jBPM also gives the developer a good Java programmatic interface into the BPM engine implementation (and, the processes it runs).

This sort of development effort does make me wonder if, eventually, we’ll have a fairly standardized open-source BPMN engine with commercial products that incorporate it in different ways (picture Apache or jBoss as examples, or the WebKit in the browser world).  Hard to say if jBPM (or key pieces like PVM) will turn out to be that open-source engine of choice, but at least from the blog posts it seems like they’re taking an intelligent and focused approach.

Its hard to keep this argument buried, as Bruce Silver demonstrates in yet another post on this subject, reacting to yet another response from the BPEL crowd.  I was going to respond directly in his post here, but for some reason I couldn’t comment on his site today.

Bruce makes the killer arguments:  only a subset of BPMN is “isomorphic” to BPEL.  Quite simply: BPMN allows the author to represent process flows that BPEL cannot represent accurately in a model-preserving fashion.

There was an argument put forth that wouldn’t these limitations be true of proprietary execution implementations as well?  And the answer is – perhaps – if those execution implementations don’t account for tricky stuff like interleaving (my experience is, that they do).  The difference (mainly) is that these “proprietary” execution implementations speak BPMN natively and were designed from the ground up to support the use cases defined by BPMN. The same cannot be said for BPEL and the engines which implement primarily BPEL. An engine could do both – but I don’t think the right answer is translating BPMN to BPEL to get there.

And we couldn’t agree more.

In this post, Bruce rips into a post from ActiveVOS which claims that BPMN->BPEL is simpler than using BPMN.  ActiveVos’ CTO Michael Rowley points out that because of BPMN’s focus on communicating between people (I believe that he means, sharing process definitions so that two people can come to consensus) is also a weakness of the standard – because this people-centric view is bound to cause it to evolve and change over time.  Mr Rowley has it backwards – the point of “code” isn’t to communicate to a machine to get it to do something!  The real purpose of code is to expose an interface into the machine that is understandable by humans – note: the audience is for humanity, the audience is not the machine.  Otherwise we’d have to write machine code (hex or binary) to write programs.  Or assembler.  Languages make the machine instructions and behaviors more accessible to humans, and that is what they are there for.

Also, the most costly thing about code is that so few of us really understand what it does – even really well-written code – making maintenance over time very expensive.  If BPMN is understandable by a greater number of people, that makes it more valuable than BPEL, which as an XML standard is understandable by relatively few people.  The fact that this expressiveness is challenging to the engineering teams who have to turn this into process execution is beside the point – after all, Rational Software did it for UML, there is no reason that BPMS vendors can’t do it for BPMN.  In fact, we can see that BPMS vendors are meeting this standard, despite the fact that they are each proceeding at a different pace on this trajectory.

Bruce’s post is worth a read, but my favorite points are these:

However, for process modelers, and even for executable process designers, there’s no way BPEL execution makes BPMN modeling “simpler.”  That’s because the subset of BPMN supported with BPEL execution excludes the very features that make BPMN attractive to business-oriented modelers in the first place: things like freeform looping back to a previous step in the flow.  BPEL is inherently block oriented, like a computer program, while BPMN is inherently graph oriented, like a flowchart.

Right. From the modeling point of view I don’t care what the engine is as long as it faithfully implements everything I’m modeling without losing any fidelity.

And Bruce wraps up with:

If BPEL were adequate to execute processes the way business wants to model them, it would have become the BPM runtime standard.  It hasn’t.

Well said, Bruce.

Rick Geneva has a post on the event-based-gateway, offering to demystify it.  Admittedly, this is a feature of BPMN that needs demystifying.  Having worked on the BPM certification for OMG (OCEB), I’m familiar with the event-based gateway.  And I have to say it is one of the uglier creations of an otherwise pretty good notation.

The key source of confusion for users of the event-based gateway is that the events being evaluated by the decision gateway are “drawn” on to the model after the split.  That’s not consistent with the way our brains tend to simulate the running of these models with tokens moving along paths.  And it isn’t consistent with normal predicate logic where the consequence is described after the condition rather than before the condition.

Having said that, Rick does a good job explaining it.  I’ll just add this:  far better (if you can) to design your process to receive a single event that can be interpreted, based on the data it delivers, to cause you to proceed down one path or another.  It is easier to read, just as easy for the BPM engines to process, and it goes back to normal condition-then-action ordering… (and usually yields a simpler diagram).

Over the weekend, Lombardi pushed out their August, 2009 Blueprint update.  This release continues Lombardi’s track record of pushing releases out every 6-12 weeks with significant improvements (and yet, without so many changes that current users get lost).

In the current revision, additional export options were added.  I was able to easily export my processes in an Excel summary format, as XPDL, and as BPMN 2.  The hardest part about doing this was just finding/remembering where the export feature was hidden within Blueprint (hint: instead of just opening your process, open a “project” and then you’ll get a view that lists each of the processes and gives you export options for each).  The help and/or forums will benefit from an update to guide users to this very useful page.

I always assumed that Lombardi would make it easier to import than to export from Blueprint, and admittedly the import features were rolled out first (from Visio and Teamworks for example).  But the export / publication features have caught up – to Powerpoint, Word, Excel, XPDL, BPMN2.  The last two representing the kind of structured export I wasn’t confident that Blueprint would support because those are also opportunities for other BPM tools to pick up the models for execution.  Clearly Lombardi feels confident that their end-to-end user experience and tooling will cause customers to use Blueprint and Teamworks in combination rather than Blueprint and other tools.  Lombardi claims this is the first shipping implementation of the BPMN 2 specification.

Earlier this year, I wondered out loud about the future of BPMN 2.0 as an exchange format given that Lombardi and a couple of other hold-outs had finally adopted XPDL 2.1 as a supported exchange format.  Lombardi reassured me that they still fully intended to support BPMN 2.0, and I recently had a conversation with Signavio (another vendor which supports XPDL -look for more on this in another post), who also stated their preference for BPMN 2.0 as an exchange format.

Full-text search is another feature that was added to Blueprint.  Not sure what technology they use behind the scenes but it seemed to do the trick for my searches.  The what’s new feed has been updated as well, but those and other refinements are a little more subtle – the kind of things you might not explicitly notice as being different, but you’ll appreciate being able to find things just a bit easier, in general (and *thankyou* for the longer process names – mine always seem to be a bit verbose, like my blog posts).

Bruce Silver has posted another update on model portability.  This is related to the previous discussion regarding XPDL, Lombardi Blueprint support, and model portability.

In this round, Bruce has time to really dive into the couple of aspects of the import that were not working, and tries to address them through some XSL judo.  Judging by the end-product screenshots he’s posted, he did a pretty good job at that.  The main issues were around losing lanes and XY coordinate mapping.

Bruce was generous enough to not only share the narrative of his efforts, but to share the end-product XSL as well (link available on his blog posting).  I think it shows (a) how close we are to real BPMN-level portability, (b) the fact that products still have a ways to go to support it properly (really, I have to write XSL to convert the models!?), and (c) how much harder accurate portable execution models would be given that these tools have different ideas about how steps in the process should be executed…

Thanks again Bruce!

UPDATE:  Bruce has an update on the model portability issues based on Diagram Interchange (DI) and BPMN 2.0.  He points out that some of the decisions made for supporting diagram interchange make it impractical to implement, despite being technically possible.  As usual, he provides good insight into the standards process for BPMN, and exposes some of the warts in the outcomes – hopefully it will result in some remedies in minor revisions to the specification.

Max J Pucher writes a pretty thorough rebuttal of the idea that BPEL matters.  He starts by pointing out that the defenders of BPEL often malign those who question BPEL’s utility.  Mr. Pucher has quite a few controversial posts and discussions in the BPM space, and this post is no exception. I can summarize his article in a few key points:

• BPEL is not widely supported.  There are over 200 BPM vendors, and less than 10% support BPEL, and by Mr. Pucher’s reckoning, their support is partial at best.
• Avoiding “vendor lock-in” is not a valid goal of BPEL – because there is still vendor lock-in with BPEL implementing products.  I happen to agree that it is a bit naive to think that software you buy from a software vendor won’t entail a certain amount of vendor lock-in.  Transferring execution details from one body of software to another and expecting complete faithful reproduction is non-trivial.  Just think about Java Virtual Machines – without a robust specification test to validate against, these JVM’s would never approach parity.  Even so, software vendors usually only warranty their products against particular JVMs and even JVM versions. He quite rightly points out that the combinatorics on database, OS, JVM, appserver, rules, security, etc. make it nearly impossible that an implementation can merely be lifted from one BPEL vendor and run inside another BPEL vendor’s platform.
• Since BPEL is XML that we hope never to lay eyes on, why is it the representation that we hope to preserve?  Aren’t the diagrams more useful to preserve?
• BPEL doesn’t address key business requirements – such as monitoring the execution of the process for important business metrics.  And the business shouldn’t really care how the process is executed so much as those metrics… So is BPEL missing the point?

His conclusion:

Conclusion: When BPEL and even BPMN are looked at from a pragmatic business need perspective the only acceptable benefit is that products that use them have similar functionality and people designing processes will find it easier to switch products.[...]

Well, this is a significant benefit – but it is not what the vendors of BPEL vendors will claim – “model portability” and vendor independence.  As usual, interesting stuff from Mr. Pucher, and a little different perspective on the BPM space.

John Reynolds’ has a good post over on his blog on Programming Language Sociology and he draws connections between his work in the C++, Java, and BPMS communities.

I have slightly different biases… My early experiences with C++ were using standard template libraries to generate “typed” lists of objects for a class on C++ in college.  At the same time, I was taking a class on NextSTEP programming (CS193d) which employed objective-C.  I discovered the elegance of having all objects respond to standard messages, and I experienced first-hand the “problems” that C++ purported to address with heavy-handed solutions.  Many of those “problems” weren’t really high-occurrence defects for me, as a programmer – paying the price for a solution to a problem I didn’t have!  (as an example, take the strongly-typed list container…) So I was never a huge fan of C++, but unfortunately (for me), it won the battle over objective-C.  (The silver lining, of course, is that Apple’s platforms for both the Mac and the iPhone make heavy use of objective-C, so I can at least enjoy a renaissance of objective-C in the mobile world)

I considered Java as a language a step up from C++.  I think it better-addresses modern programming problems – machine portability, connecting to remote objects and services, memory management, etc. It isn’t that there aren’t great applications for C++, there are, but Java better addressed the next set of challenges.

You could consider BPMN a “language” that addresses the current (for John Reynolds and I, and for many in the IT space) problem space of business process definition and implementation.  As John says, part of what makes the BPM tools exciting is that they are good tools for speaking the language that solves business process problems.  The tooling has a long way to go to meet our expectations of what it will do for us in the future, but it has come far enough that there is no longer any excuse for putting off BPM implementations, waiting for a better mousetrap.

BPMN Method and Style, a new book by Bruce Silver.  Its going up on Amazon sometime in June Its already up on Amazon, and has some great endorsements already (check out the link for details).  Looking forward to giving it a good read myself!  Congratulations Bruce, in getting this new book out there!

Jim Sinur of Gartner poses this question in a blog post the other day.

Actually, he phrased it as “Is XPDL 2.1 on the Edge of Becoming a Dominant Process Standard”.  I think the answer is a “no (not yet).”

(this may just be because my definition of where the edge is, or what dominant means, is different than Mr. Sinur’s, or it may be because we disagree on some of the background material – either way, here’s my take)

Some arguments in favor of XPDL are cited by Mr. Sinur, and by others (Keith Swensen in a previous post as well), and by WfMC.  The primary argument in favor is the portability of the activity models.  And, a pretty inclusive vendor list that supports XPDL.

I’m not against XPDL at all.  I won’t even mind if it becomes the dominant process standard. But here are the arguments working against it for now:

1. First, keep in mind that these portable activity models are stripped of their execution attributes/elements – those elements are not, according to Mr. Sinur, part of the portability of XPDL 2.1.
2. Second, the processes that are ported are also devoid of the implementation of human activities (and typically, system activities), which are presumed to be implemented outside the BPMN model.  (See John Reynolds thoughtful post on this subject here)
3. When I look at the “implementations” of XPDL on the list above, I found a few problems with it:  the references aren’t dated (and may, therefore be out of date or obsolete), and several of the implementations are for tiny pieces of the software or companies being mentioned, and don’t represent full support of XPDL.
4. According to Gartner, the two leading BPM vendors are Pega and Lombardi.  I looked through that list and I couldn’t find either of them on the XPDL bandwagon.  If the two leading vendors in the space don’t find it necessary to support the standard, then I’m doubting that it becomes the dominant standard.

All that said, XPDL could still become the preferred format for model exchange if BPMN 2.0 doesn’t fit the bill.  Or Gartner could ratchet up the pressure on the vendors that haven’t supported XPDL so far to begin doing so – by weighting that support higher on evaluation criteria for the next Magic Quadrant (though that is quite a ways off, it probably gives these vendors no excuses if they haven’t delivered it by then).  Currently the vendors not on the XPDL list seem to be waiting on the BPMN 2.0 to be released.  If that spec doesn’t live up to the billing, I can imagine vendors supporting XPDL for model interchange as a backup plan.  But I don’t see those vendors investing in two solutions for interchange if they don’t have to.  And I can’t see calling a format a “dominant” format if the leading vendors aren’t fully supporting it.

I think the portability of XPDL, and the compliance tests they’ve put together, are a big step forward for the BPM community.  But that conclusion is regardless of whether XPDL in fact, dominates over BPMN, or vice versa.  Either way, the BPM community wins.  Because I’m sure some of the enterprising XPDL vendors will come up with a mapping from BPMN2 to XPDL 2.1… (These XML formats do lend themselves to being translated after all…)

Keith has updated the community with a Model Portability Landmark:  WfMC’s announcement of a BPMN Model portability test.

This is a great step forward, and Keith does a great job summing up the results.  Unfortunately, from my perspective, the portability standard uses XPDL, rather than the as-yet-to-be-approved BPMN 2.0.  I have nothing against XPDL, I just wish some of the vendors I work with supported it!  Without that support, I’m tempted to rain on the parade of this announcement of portability.  For example, missing from this list on WfMC’s website are Lombardi, Savvion, Appian, Intalio, Pega.  Well, Pega and Lombardi (arguably market leaders based on Analyst reports) were recently placed prominently in Gartner’s Magic Quadrant for BPM, and they don’t support XPDL.

However, there is a silver lining for those of us who don’t use products that support XPDL.  Whether BPMN portability can be accomplished or not is not longer a theory.  It is a fact.  It can be done, and it has been done.  Now the question is whether the BPMN-2.0 supporters will get to the altar of portability sooner than later… one can hope!

I’ve read a couple of articles recently related to business process visualization.  I think this topic is particularly interesting because most people think they have what they need once they buy a BI tool or a good enterprise reporting tool (and there are several that are quite good).

But the thing that makes process visualization so interesting is the context of the process in which the data is generated.  John Reynolds’ Thoughtful Programmer blog has a good post on the topic of Visualization.  In it, he makes a few great points that should resonate with anyone pursuing BPM excellence.  The dream that everyone working in a business will not just understand the activities for which they are responsible, but the role of those activities in the business processes they impact.  It has been shown that feedback loops improve performance by focusing on how individual performance affects overall results.

John also hits on another favorite of mine:  the idea that the same model should support views/layers/aspects that allow different users to understand the model and interact with it ways that are different, but compatible (and enabling).  He also references Phil Gilbert’s dream of BPM on every desk in an enterprise- which makes sense when you think how most processes are executed today – via email/paper/fax – and every single desk in an organization is involved in its business processes… they just don’t have the right tools to support the processes they execute (at least, in many cases!).

So, I think John does a great job of making the case for why visualization matters.  But I think if one hasn’t read his other posts, you might miss some of the context:  that the reason this visibility matters is precisely because of the process context.  It is the lack of context that makes using existing BI and reporting tools challenging.  You can get the data, but does your IT staff know when (in the process) to collect the data? Can the business articulate this need to IT?  Often these decisions are driven as much by convenience as by design because no one has the right answer with respect to the process.

Another issue to discuss with respect to visualization is the lack of standards.  While some vendors use very easily read/used/manipulated data formats for their process information, they are still considered to be “proprietary” formats by customers and the industry (proof that the definition of proprietary continues to evolve.  Once upon a time, just having your data accessible in a published relational database schema was considered being “open” rather than “proprietary”!  Now, if you don’t adhere to some industry standard schema you are closed or proprietary… even if such a schema doesn’t exist yet… )

This brings me to an article on The Business Process Analytics Format (BPAF).  Good read, and a good reminder that there are folks out there trying to put standardized formulations around the Business Process Data that feeds into analytics.  I’m encouraged that the format appears to be consistent/compatible with the tools I’m familiar with, but of course some of the terminology could use a BPMN overhaul (or at least a primer for those who are more familiar with other BPM* standards).

For example, there is a focus on the Business Process Audit Format XML… Which has a defined Business Process Analytics Format Event. In isolation this is a good naming convention, but I think it would help everyone see the relevance if the standard also related this Audit event to where it might occur in a BPMN-defined diagram – is it a standalone event in that diagram, or is it an artifact of other items in the diagrams – and if so, which ones?

Even without this context in terminology, BPAF might turn out to be important.  After all, if you can standardize on what a measurable “event” looks like, then you can actually build analytics that are cross-vendor and consume BPAF compatible events.  Eventually you  could have standard analytic packages… But of course the goal isn’t to have standard packages… the goal is for people who run businesses (and manage processes), to be able to visualize, comprehend, manage, and improve their processes.  That makes visualization central, if not the heart, of what will make BPM relevant to every desktop in the enterprise, even if it is dependent on delivery of executable processes to have full process context in that visualization.

Keith has put out a series of articles (possibly more to come) that lay down a great marker for how to evaluate Modeling strategies and how these approaches compare (strengths and weaknesses of each, across several dimensions).

It started with this post, which set the table for further discussion.

Following that were posts on Analytics, Round-tripping, Performance, and Simulation.

I recommend reading them all if you want a better understanding of some of the BPMN – BPEL debates (without even mentioning those terms), and understanding implicit tradeoffs in a product that the vendor may not (or may not be able to) articulate explicitly.

The interpreted vs. compiled debate has been going on for a long time now.  Keith Swenson brings up a version of it in his Go Flow blog in this post on Model Strategy and Performance.  I think at any given point in time, the compilers have a very good argument because they can find situations where a higher degree of performance is required.  Even in the BPM world, if I can imagine a process that executes 10,000 times an hour, you can just imagine that same process at Wal-mart to picture that process running 10,000,000 times an hour!

However, the right design decisions for software are not judged by a single point in time, these decisions are judged over time… and over time, the interpreters have a lot working in their favor:

1. Over time, the interpreters have time to make their interpretations smarter.  Just-in-time compiling of java is one example… finding better algorithms (in terms of big-O notation) is another approach…
2. Over time, hardware gets faster.  my interpreted code will run some percentage faster each year.  Although compiled code will also increase in speed, the real time-differential on any given problem will get smaller (and eventually, small enough that I won’t care).  Examples of this:  graphical user interfaces, once to slow to even ponder using, now we have all kinds of bells-and-whistles – even though commandline interfaces are still faster!  Java is another example…
3. Over time, hardware gets cheaper, especially if you are looking at per FLOP or per operation.  As a result, even if I have to buy more hardware, in a few years I can buy twice the cpu’s at roughly half the cost… and the CPUs are faster too (or, utilize multiple cores to get faster).
4. Developer time and maintenance time, as costs, usually outweigh the cost of additional hardware expense to provision a system.  When I was in college, a professor demonstrated that the human can still write better (faster) code than a compiler.  He had experts at the university (and named them) write performance-optimal solutions for a matrix fill routine.  The lisp program was the simplest.  But the slowest.  The C routine was 3x faster than the Lisp version.  But then our professor wrote an assembly code solution that was yet 1/3 faster than the C version.  He did the loop-unrolling by hand, for example.  Well, why don’t we just write all our important apps in assembly?  Because it would be incredibly inefficient use of a scarce resource: namely, our software developers (and, in the case of BPM solutions, our business analysts)!

Given those reasons (and others), my general thesis on interpretation vs. compiling is:  first, come up with the best/right representation from the perspective of the authors and consumers of that format (the developers typically, and in the case of BPM, the business analysts, process owners, and process developers).  If performance is a problem, invest in addressing the bottlenecks or areas with the most yield first.  If necessary, figure out how to compile or just-in-time-compile the model for performance.  But doing this performance-enhancement work is a lot easier to do on the “right representation” than it is to try to bake in this kind of performance up front and still end up with the “right representation” for users…

The short story is, transforming your model may yield performance benefits in the short term, but if those benefits come at the cost of a good representation of your process, then over time you’ll lose to those who invested in the right representation and gave other technologies the time to catch them up on the performance front.

A long-standing debate in the BPM world has centered around whether or not to preserve the model.  It sounds like an esoteric topic, and Keith Swenson’s post: Model Strategy: Preserving vs. Transforming, takes a pretty clinical angle on this debate, and I think lays out the two camps and their respective arguments pretty well.

I like his definition of the common ground components:

• business process enactment: the business process itself has a dynamic component as it moves from the beginning to the end of handling a single case. The process definition does not normally change here, only the process instance or context that records the state of a particular case changes.
• business process lifecycle: these are the changes that a business process goes through from initial concept, to modeling, to integration, and finally to deployment into an enactment environment.
• business process improvement: this is the change to a business process that occurs over time through repeated use of the business process lifecycle followed by analysis of how well that version of the business process worked. This is the TQM or continual improvement aspect of BPM.

The two camps then divide themselves into Model Transformers, and Model Preservers.  The Transformer camp, as Keith argues, has roots in long traditions and proven success in software engineering (and related engineering tasks).  For example, 4G tools, UML, and even VLSI chip design applications are good examples – where the picture translates into code through transformation to a new form.  However, the analogy to these prior tools and content areas isn’t perfect.  For one thing, the author of the “picture” was likely the same person who was the consumer of the picture – the engineer (or developer).  It might not literally be the same person, but the same kind of audience.

Keith gives a good example of how transformation might apply to a Business Process model:

Coming back to a BPM example: the business analyst might put an activity describing a document review in the business view. This would be transformed to: a activity which sends email informing the person; an activity to check the document out of a repository; an activity to send a reminder message if the response is getting late, and an activity to wait for the response to come back. The programmer might extend this to include notation to handle exceptions and other timeouts that the business person did not include. Later this might be transformed again in order to get the executable code.

This is a pretty good example, though in my experience the business usually thinks of things like email, repository checkout, and reminder messages… However, the programmer very well might have exceptions to handle and other technical details that don’t direclty concern the business person creating the original document.  I think the key thing to note here though, is that to the model transformers, it is okay, and even expected, that by the time the technical folks are done adding their fit-and-finish and details to the model, that it will be unrecognizable to the business (Keith has a great image of simple model transformed to complicated model transformed to XML… surely a format the business won’t relish).  There is great discussion in this camp about something that is a hard problem – the “roundtripping” problem.  Because people can model things that are hard to round-trip.  And they can edit the transformed content, which is not guaranteed to have exactly the same freedoms and restrictions as the modeling environment.  So you have what I would call a “lossy” transformation.  It is not the same as compiling to bytecode and executing, because the expectations is that a developer or engineer will EDIT the “bytecode”… Anyone who has decompiled optimized java bytecode has seen that the compiler makes changes, and it no longer looks like the original code.  If you also edited that bytecode, the resemblance to the original would be even more tenuous.

Suffice to say, Round-Tripping is a really-hard-problem for the folks in the Transformation camp.  The best argument I’ve heard from this camp is that you should *not* round-trip, but instead, treat the transformation as a one-way journey.  I think that would be fine, so long as the developer cannot make further changes to the result of the transform. (this is, by the way, what Ismael Ghalimi recommends)

The second camp, the Model Preservers, believe the model is the model is the model.  While developers or engineers can ADD to the model, they must do so in a way that preserves the model.  So, integrations, exceptions, etc. can be layered onto that model, but they are layered in context- and without losing the original context of the model.  The concept of Round-Tripping is trivial – the visual and storage and execution are expected to be 100% in-synch, and any difference would be treated as a defect to be fixed, and all three are designed with BPMN as the starting and ending point…

So why is preserving the model important?  Because if you embark on the transformation approach, the model that the business creates eventually becomes a piece of paper or image on the screen that isn’t truly relevant to what is running in production.  In traditional software development there might be a very important phase where the business validates that the software developed meets “the spec” as represented in a generally large Word document… But this step can be eliminated if you do model preservation – instead, the business is testing whether the BUSINESS captured the process effectively and correctly, and whether the technical bells and whistles adequately support the process, and no longer testing adherence to a 6-month old specification of requirements.

The power of being able to execute the model the business understands is huge.  The power of being able to tie back the analytics to that model (trivially even) is also huge.

However, software vendors in either camp still have to execute well on their designs and software development to get good results… And neither approach is easy.  But then, that’s why we buy the software! (or the support contract)

I ran across this blog entry titled BPMN for People and Robots by Anatoly Belychook thanks to Sandy Kemsley, and it (and the commenting below it) is a great read on how different tools that “implement BPMN” can very quite widely in how well they are suited to a common understanding by business and IT.  Anatoly takes a very simple process and demonstrates how several different tools represent that process.

I think the most telling argument Anatoly makes, and one with which I firmly agree, is:

From my point of view, the possibility for IT and business to talk the same language is not a feature but the classifying attribute of BPMS; if this “magic” is absent then other features don’t count.

And I think this is the unifying principle for those who believe BPMN is, to this point, the best “language” for achieving this.  However, the products that support both modeling and execution need to provide good fidelity between the BPMN model and the execution of the model in order for this to work well.

Another flurry of updates in the BPMN vs. BPEL discussion: Bruce fires a shot across the bow here, and then Keith Swenson makes a response in his own blog, but there is also a pretty good discussion below Bruce’s post in the comment section.

I think it is either a sign of progress or weariness that the debate has gotten a bit more reasonable, in that no one is arguing BPEL over BPMN or BPMN over BPEL, there is simply a discussion of how to allow the two to live harmoniously.

Bruce makes key points I agree with:  we can’t make the success of “BPM” dependent on agreement on definitions – its very hard to get a consensus on the definition for something as vaguely termed as “Business Process Management.”  Moreover, there is no reason to believe that different organizations can’t make these decisions DIFFERENTLY based on their own unique culture, personnel, skills, etc.  In one organization, the business may have members that are technical enough to model valid BPMN diagrams.  In another organization, those skills may go wanting, and the modeling will get done by IT.  In yet another, outsourcing the modeling and developing those models in JAD/RAD sessions may be the right approach.  Does anyone really expect something as broad as BPM to be a one-size-fits-all methodology?

Keith raises the point that BPEL isn’t enough to represent all the interesting process use-cases, and he’s right.  It doesn’t mean that BPMN and BPEL can’t play nice together, but it does mean that a “native” BPMN xml representation, independent of BPEL, makes sense.

Finally, Bruce raises the concern that this portability of models, and fidelity between diagram and XML representation, would likely be a good thing for any BPEL vendor, but the existing BPMN-native vendors might not get on board.  I think if the BPMN-native (my term for BPMS vendors who diagram BPMN processes and then execute those without losing fidelity, without translating into some other representation that doesn’t match 1:1 with BPMN) vendors won’t play ball with this portability, then it begs for an opensource project to help provide the import/export.  You’ll have one common storage (BPMN 2.0 presumably), and you’ll have a set of vendors with (potentially) proprietary storage.  That cries out for individuals to create the import/export mechanisms from each of these formats (based on which ones they know) as part of an open source project, if the vendors won’t do it on their own.

I’m still not sure it commoditizes the runtime however.  It commodotizes the storage definition… but one still has to write the code that will execute these models, and there will be room to compete on several fronts (performance, resource utilization, correctness, platform support, architectural compatibility, etc).  And hopefully the competition will move toward value-added features on top of these models and run-times… Because there is so much more to BPM in our future than just running these diagrams – and yet some of those future states may be hard to get to if we can’t make the basic blocking and tackling easier.