Sandy captured some notes from the Lombardi conference call here, and Dennis captured additional notes here.  They both do a good job capturing the gist of the call -maybe we’ll get invited one of these days :)

One thing Dennis points out from the call is fewer pilots, just diving in and doing it.  This makes sense in a rough economy, as the pilot process is expensive.  A long selection process is expensive.  If you can get the right product and skip some of the expense of a pilot, so much the better.  I’ve heard similar points made by other folks in the business lately.

Sandy points out that while she believes BPM is well positioned in a tough market, that she thinks the real spending will be on projects with customers who have already swallowed the capex to buy software.  We’ve definitely observed this phenomenon in the market - expanding on old solutions, building new ones on existing infrastructure.  Its good to be a BPM consultant in this environment, and really see the yield that customers can get on this process improvement investment.

I’m looking forward to seeing if some of the other vendors in the space will do these calls as well, and see if the trend of good news is across the market or localized with just a couple of the vendors.  I think the big news was continued growth of license revenue, and the fact that they are profitable for 2008.  Given the funding environment right now, it seems like A Very Good Thing to be profitable.

Ok, back to work on processes now…

What are the basic metrics needed? There are five, with a “really nice to have” sixth metric. With these few metrics you can learn a great deal about a process, and don’t sweat how accurate or precise they are up front. That is something through additional work you figure out but getting even rough numbers up front are enough to get some real progress going. With emphasis- having these with a 95% or better confidence level is not a pre-condition to move on. Here is the definition of the 6:

1) Cycle Time - what is the mean or median total time for an activity, or step in a process to be executed? could be data entry, call handle time, etc.

2) Lead Time - What is the total mean or median time, defined by the process owner or company, that is required to meet the customer demand? The term backlog comes to mind, and the motivation for companies having SLA’s for their vendors (or customers). This is the fundamental measurement of your commitment to deliver to a customer (or the commitment you expect as a customer).

3) Throughput Time - What is the total mean or median time from the absolute beginning to the absolute end of a process. The summation of Cycle Times. Key metric to use for the notion of continuous improvement.

4) “Nice to Have” Takt Time- The time the customer expects a given process output to be delivered to them.

5) Work In Progress (WIP) - This is the volume of transactions or widgets in a process. An invoice, A report, a car, whatever it is your process is built to deliver.

6) Number of Operators - Number of process consumers participating in the process to create and deliver the good or service. Claim Specialists, HR Specialists, etc.

With these few metrics you can start to get a picture of how the process performs. There are other metrics like good versus bad process outputs (such as number of defects, think bad invoices, bad orders, etc) but up front these will really get things moving along. For instance, you will see as you look horizontally what steps quite possibly should be left alone versus the one or two that you need to remediate to promote flow. Processes which execute faster have few defects by and large. So how do I start getting these?

Value Stream Mapping is almost always the defacto starting point to then bridge metrics against processes and their activities. Let’s look at an example, this does not use standard Lean notation. This VSM was made with Lombardi’s Blueprint product. What is important is not the iconography but rather being able to see the big picture and relative detail in one view.

Here you can see the value stream representing the core end-to-end process with a target of completing the process in 72 hours. It shows the highest level (1) and goes to the next level down. This is the equivalent of using sticky notes. I could add meta information behind the scenes describing problems in each of these process areas, but what it doesn’t tell me is where am I going to get the biggest bang in terms of improvement. Now let’s look at the same image with the basic metrics assigned:

Now, with the metrics associated I can get a much better understanding of where to focus. If the target is to deliver the tickets to the customer in 72 hours, but we can see that our Throughput Time is 162 hours, where do we need to focus? Conventional thinking is we should improve each area and that the sum total of improvements will make everything better. However, that is not in fact how it works. We will just waste scarce resources doing that. If we drop the listing process from 2 hrs to 1 hr it is equivalent of a rock in the Grand Canyon. We need to get yield! So look at the Order Fullfillment, if we drop it from CT 120 hrs/LT 40 hrs to something similar to the others, we just created a ton of velocity. With these metrics attached, and with the process decomposition we were able to channel those scarce resources to the weakest link in the chain- Order Fullfillment. As a result, if we carve off 100 cumulative hours and reduce the lead time coming in to this process area we are much closer, if not dead-on, to hitting the takt time of 72 hours.

Could we have improved those other areas? Sure and it’s most likely without these basic metrics we would have tackeled them (possibly at a great expense). Being intellectually incurious about basic data is a key driver behind failing to move the needle in the business; just flat missing the mark because project selection was poorly executed.

Again, its not critical to be dead-on accurate up front, there are techniques to help shore up what measurement variation is really doing to the overall management of the process. The point is understanding that processes have multiple dimensions to it. The picture is just one important part of it. You need the metrics to help focus in on what will make a meaningful difference, and the obvious ROI.

Hope this helps, if you have questions or feedback, please leave a comment or catch me at lgibbs@bp-3.com

Meanwhile, if you have an iPhone, feel free to check out our blog in its new iPhone mode.  I was pretty impressed by the folks at BraveNewCode-  the WPTouch theme/plugin is pretty ingenious and it doesn’t interfere with the normal browsing experience.  I wish all the blogs had this kind of theme for reading from the iPhone.

Incidentally, if you’re wondering how our team feels about the iPhones, we love ‘em.  Being able to read websites or html email on your phone is actually useful.  Typing is marginally harder after using a blackberry for 4-5 years, but getting easier.  And some of the apps we’ve found are pretty entertaining (when you travel a lot for work, the iPhone apps for Yelp! and Urban Spoon are invaluable for finding a quick foodie fix… and the directions/mapping are great too).

Before we even get to the 7 forms, Tom has an insight that is key to understanding why BPM systems are a bit “different”:

A key capability of BPM systems is that processes steps can be wait
states. For example in the business trip process above, nodes ‘manager
evaluation’ and ‘ticket purchase’ are human tasks. When the execution
of the process arrives in those nodes, the system executing the process
should wait till the assigned user completes the task.

From a software technical point of view that capability is a big deal. As the
alternative is a bunch of methods that are linked by HTTP requests,
Message Driven Beans (MDB), database triggers, task forms, etc. Even
when using the most applicable architectural components available in
Java today, it is still very easy to end up in a bunch of
unmaintainable hooks and eyes. Using an overall business process makes
it a lot easier to see and maintain the overall execution flow, even
from a software technical perspective.

I couldn’t agree more with this, though it is a sort of “under the hood” consideration.  It is part of why well-designed BPMS packages can scale really well with respect to the number of process instances that might be in a resting state (waiting for something to happen).  Any BPMS solution that is holding all these instances in memory is going to be at a distinct disadvantage because for most of the time a process instance is alive, it is in a waiting state, waiting for an activity to execute, for a human to act, etc.  and not actively processing work.

Setting aside scale, its just hard work to write the code that surrounds such wait-states correctly and you don’t want to have to write this code every time you confront such a problem - rather, you want to use a BPMS that seamlessly represents these wait-states in a diagram, giving the developer the ability to focus on the functionality of the process steps or the flow of the process, and not get caught up on the internals.

A couple additional thoughts on the seven forms (use cases) he outlines…

For Use Case 5, Visual Programming, he lists a few benefits:

With visual programming, we will target developers that do not yet have
the full skillset to develop in Java. They can graphically specify the
activities and draw transitions to indicate the control flow. Instead
of typing Java code, just put blocks like ‘perform hql query’,
‘generate pdf’, ’send email’, ‘read file’, ‘parse xml’, ’send msg to
esb’. Instead of writing Java, they will be composing software programs
by linking activity boxes in the diagram and filling in the
configuration details.

While simplifying activities like stringing together queries and PDF generation without writing that code may make this operation more accessible to developers without these Java skills, I would contend a different benefit:  Things that should be easy (or rote) become easy (or rote) and therefore free the mind to focus on the bigger business problem.  This is akin to the value I receive from a BPM system hiding the details of implementation of parallelism from me.  While I may have the skills to write this code, it is tedious to write, and expensive to get it right.  So you don’t want to roll this kind of code in a one-off fashion - you want to write it once and re-use it liberally to get benefits of scale.  To the extent that a visual environment frees the developer to focus on the important abstractions of the process, it is a big win.  (Note: As soon as I read Use Case 6 it precisely points out my thoughts about threaded programming, by pointing out that you can use a BPM system as a thread control language).

What really intrigues me is whether Tom and the JBoss jBPM team can create the necessary interfaces and utilities around jBPM to allow the creation of the Domain Specific Languages that can be implemented in jBPM (e.g. BPMN).  While Tom apparently doesn’t see much value in the use of BPMN for executable models, my experience differs (and, since his technical approach doesn’t preclude BPMN for executable models in the future, the disagreement is more theoretical).  Perhaps we can change Tom’s mind about this!  While I think it is possible (and even useful) to create BPMN models that are not intended for execution, I believe a good technical implementation can still be accomplished with BPMN (plus technical details captured in context of the process).  And this technical implementation can still be understood by the Business.

I think the problem isn’t BPMN… its a lack of imagination about how to capture business and technical details in different layers so that the user of these authoring tools can show or hide the “layers” that are interesting to them.   For clarity, I think “technical layer” could include adding additional BPMN elements to the diagram, not just scripting and code and variable input/output details.  And I think tailoring the visibility of details in the diagram is a key component in making a single model multi-purpose in audience.

I’ll definitely have to keep my eye out for jBPM 4…

• A Question:  “Is this project a good fit for BPM?”
• An Answer:  “This isn’t a good fit for BPM.”

First, let’s talk about the question, and let’s think about it in terms of process rather than project.  BPM is good for processes.  Not just some processes, but processes generally.  The real question that should be answered is “What is the process?” Define the process in terms of what it does for the business.  Don’t define the process in terms of what it does with an incoming system event, or in terms of what a single user community does with some data.  Define the process in terms of how you:

• service your customers
• fulfill your demand
• respond to exceptions or challenges
• meet service level agreements
• create delighted customers and renewable business
• recognize revenue

(These are just a few ideas… )

If you can re-orient your thinking from the “project” or the “application” to really thinking about the process, you won’t question whether it is a good fit for BPM, that will be obvious.  BPM is a good fit for processes, and if you want to manage your processes, you need some BPM!

The answer/statement: “This isn’t a good fit for BPM.”  When I hear someone say that, the following ideas run through my head:

1. They haven’t found the process that their project is intended to serve, or
2. The BPMS they have doesn’t lend itself to a clean representation and implementation of their process, or
3. They just shined the light on their process and discovered all the ugly parts that were hidden in the boiler room.  Now they are feeling like it is barely a process at all, just a jumbled pile of exceptions connected by spaghetti.

To the first point:  Find your process.   Broaden your scope if you must, to consider what happens before your “project” starts (almost every project has inputs - follow the inputs - where do they come from? how did they get created?  who / what is responsible for that?  how do they impact my project/process?), and consider what happens after (who receives my outputs, where do they go and what impacts do my outputs have on their project/process/applications/users).

To the second point:  Just because your BPMS doesn’t represent the problem well doesn’t make it not a process, or not a good fit for BPM.  Let’s distinguish between a software package’s shortcomings and a shortcoming of the concepts embodied by BPM.  The software part is still evolving and improving.  In the meantime, there are going to be gaps.  These can be resolved or patched without abandoning the benefits of BPM.

To the third point:  If you haven’t been managing your projects and applications as key processes that support your business, of course there is a scary boiler room down there in the basement.  Take the time to extract process out of that mess, or else find a way to leverage the boiler room as a black box, with a nice tidy interface (think, electrical wall sockets) that let’s you design higher level process around this one area.  Come back and clean up the boiler room when you can show that by having a better result out of that black box will yield benefits (after all, with BPM, you’ll be able to measure the inputs and measurable outputs and timing and count data… and infer what could be improved without even knowing the inner workings)…

So, when I hear these things, I know we have a chance to help our customers realize an opportunity to look at their problem or project differently and yield the benefits of a BPM approach.  We’ve seen lots of data entry applications that lose sight of the fact that they are interacting with and altering the state of a real process with customer value.  Its really gratifying to see the value we can extract out of this change in perspective, and our CEO, Lance Gibbs, is one of the best in the business at this kind of out-of-the-box thinking (I’ve had the good fortune to be in the room with him on several occasions when such eureka moments have happened).  If you’re using BPM and you think you’ve got a bum deal on your process, take a step back and see if you can’t find the process there, the real process, which your project is meant to service…

• There is no right or wrong way to model a process.  I would say rather, just that “right or wrong” isn’t the primary question.  The primary question is whether the process is effective.  This is essentially the point Kevin makes, but I would posit that it is actually possible to have a “wrong” process model :)
• He takes a casual swipe at BPMN as a modeling and the idea that commenting on modeling a process must necessarily include a diatribe about the “right” way to use BPMN to do it, or cover some minutiae of the specification.  If you’re an avid reader of BPM blogs like I am, this especially hits home with recent discussions about BPMN compliance and roundtripping with BPEL and whether models should even be executable! :)
• He cites Occam’s Razor (though, I think it actually hypothesizes that the simplest explanation is also the most likely explanation). The point is, simplicity is generally preferred over complexity, when both represent the problem adequately.
• Kevin makes a good point about focusing on the audience or intended use of the model (a model for training purposes may be quite different than the executable model of the same process).

One thing I really like about the article is that Kevin correctly points out that even modeling a process is, itself, a process.  And it is a process that we can improve upon over time…

I’ve deployed several projects that integrated a BPM platform with a rules product, and its just easy to do via webservices or an API call in all but the most extreme cases.  Anyway, enough from me, here’s a link to Sandy’s post.

Disclaimer:  I used to work for a company that did a lot of work in the configuration space, which has a pretty big overlap with rules.  We did heuristic search, constraint satisfaction, resource allocation and pooling, spatial constraints, containment, and we even did massive rule systems that were super fast.  Intellectually it was a very interesting field because you take really hard problems (in some cases, problems that you could demonstrate were NP Hard problems) and finding “reasonably optimal” solutions in a very finite amount of time.  As I said, intellectually very stimulating.  In other cases, it was coming up with very creative ways to use simple rule-based systems to compute very user-friendly user-interfaces in millisecond time against very large rule bases.  But one thing I learned for sure: thinking about the world as a set of configuration logic or rules is a different way of thinking, and it just isn’t intended for the average Bear.  This is why I don’t see representing “everything as rules” as being a terribly useful way of approaching the world when it comes to involving your organization in the process of business process management or improvement.  I consider myself a reformed rules guy, and now, tongue-in-cheek, I see everything as a process!

Update: On a (somewhat) related note, a somewhat humorous post from Jim Sinur on how rules might have initiated the real-estate meltdown.

Bruce makes some great points about why BPMN is catching on in the marketplace.  It is (relatively) intuitive, it represents most of the really important ideas that need to be expressed in the Business Processes, and I’ll add another thought:  you can actually draw it on the whiteboard pretty accurately.  Some people think that is silly, but I think it matters that you can sketch it out pretty faithfully on a whiteboard or piece of paper as well as in software.  A great many people still think most freely on whiteboard/paper rather than on a computer screen.

Bruce also makes a great point about BPMN to BPEL mappings:

“A key reason for that is it is graph-oriented like a flowchart, not block-oriented like a structured program, or more to the point, like BPEL.  For example, you can loop back to some previous step with a BPMN gateway, but BPEL places severe restrictions on that.  What that means is you can draw BPMN diagrams that cannot be mapped to BPEL, or at least to “roundtrippable” BPEL that preserves the BPMN view after even minimal tweaking in the BPEL environment.”

Personally, I don’t even want “roundtrippable” I want “native” - meaning, a representation specifically designed to store BPMN.  Whether it is “custom” XML or “standards-based” XML, I want it to be targeted so that roundtripping simply isn’t an issue - the diagram is the xml/data and the xml/data is the diagram.  Otherwise, how can the business ever be sure that they’re BPMN diagram is being accurately executed?  If the representation isn’t roundtrippable or native, then it undermines faith that the process works as designed.  One of the key problems with all the previous software systems I’ve used and built prior to BPM is the separation between requirements and the finished solution (or, put another way, between the business and IT delivery).

Ismael of Intalio gives this response in his blog:

“Do not worry about round-tripping between BPMN and BPEL
Round-tripping is a futile and wasteful exercise when attempted between two languages or notations that have a significant semantic gap between them, and so is the case for BPMN and any process execution language optimized for computers (like BPEL). The same is true between UML and Java by the way. Round-tripping should be set as a goal only between two languages or notations that are essentially representing the same thing, such as BPMN and its future serialization format for example. What this means is that there must exist a way to fully serialize BPMN diagrams on one hand, and a way to graphically display serialized BPMN on the other. And all this should work in a deterministic and predictable way, uniformly across tools.”

I find the assertion that round-tripping is irrelevant to be a bit startling.  The problem is, if the translation from BPMN to BPEL representation is wrong, we’ll have a hard time detecting that.  And then how would it help with compliance? I can’t take the BPEL representation and load it in another BPMN tool and see the model… so we haven’t achieved the interchange that Bruce Silver is looking for, nor proven compliance…

Personally, I’d start with the following:

1. can i MODEL the specification model (ie, can i draw it in this tooling with standard components).  By this definition of compliance, a visio stencil may do the trick…
2. can the target environment EXECUTE the model.  the compliance model can describe the expected behavior of the model especially illuminating areas that might be up for debate (presently) but which should be standard (near future) interpretation.  Execution should be separated from modeling - from a compliance perspective - because BPMN does not exist solely for execution purposes.

Next, additional models could be provided that describe “illegal” models. If you model these in a compliant environment, it should, in some fashion, warn you that the model is not correct (either by throwing an informative error when you run/test the model, or by showing validation errors at design time).

Granted, testing compliance gets easier once BPMN 2 style storage spec gets released, but til then, there’s no reason we can’t have a representative model - it will just be harder to confirm compliance because we’ll have to actually do the work to model something.

Bruce points out that without the data storage spec, he’s not satisfied with the results - and I can’t blame him - but I think defining what would be a representative model to exercise the specification for a given tool is something that has to be done regardless of the storage medium.  And in the meantime, even the visual representation of such a model could help customers and consultants evaluate BPMN tooling appropriately for compliance.  Currently there’s just too much distance between here and there if we have to have a portable data schema before we can have a model (or set of models) that exhibits all the features of compliance.

You can find Ismael’s original post here.

If you’re technical, and you’re curious about what’s going on inside the black box BPMS you’re using, this is a great debate to read through.  The basic point made by Arthur and others is that BPEL is an implementation detail, and that BPMN is the most important standard because it represents the process, and as we all should know, representation affects how we think about process.  Arthur’s basic point is not that BPEL is deficient, but merely that it is like arguing about whether we should compile to Java bytecode or C-sharp byte code.  Or native machine code… do we really care as long as the code runs well on our target platform? :)

The victim in the debate is often “petri-nets” as compared to “pi-calculus”. Ismael makes the argument that BPEL is superior to the other approaches out there, partly because it is standards-based, and partly because it employs pi-calculus and that that is superior to petri nets for parallel processes. A great debate over whether BPEL really implements pi-calculus ensues, missing that the key point Arthur was making is that either a pi-calculus model or a petri-net model could represent parallel processing of BPMN models well. My personal experience reflects this as well, as I’ve participated in projects that leveraged both technologies and they both work… moreover, they don’t appear to be entirely mutually exclusive.

Lost in all of this debate is the B in BPMN:  The Business.  The Business doesn’t care as much about the specific technical differences of these approaches under the hood, so long as they execute the process faithfully.  Personally I just want to use the best tool for the job.  I’m going to “write” processes in BPMN.  I’m going to execute them and deal with differences in interpretation of the spec when BPMN is married up to execution engines, regardless of whether they are BPEL or other technologies (and the fact that it goes to BPEL doesn’t get me away from interpretation issues, because the issue is how is BPMN “interpreted” into BPEL just as much as it is an issue of interpreting BPMN into some other exeuction form).

The project starts, you know, the one that’s larger than it should be, a seemingly invisible scope boundary, involves business units that don’t even know their included; yeah, that one. The group meetings start with good intentions, the process is kept in the middle for a while. But then, the conversations about how the interface looks, how fast will it run, what kind of system integrations are “we” going to pull off, begin. And suddenly, everyone loses focus of the business process. Suddenly, the “IT guys/gals” are writing software and the business analysts are trying to keep up with the requirements. The group meetings including the business sponsors give way to technical meetings with whiteboard discussions about how the BPM tool will be bent, prodded and tapped into to accomplish the “tricks” at hand. At the end of the day (month, year) you’re left with perhaps a slick solution; but where’s the process? Is there a useful process artifact for which the business sponsors can consume? Is the business unit ready to assume ownership and ongoing evaluation of the business process solution delivered? What about real process metrics, no, not the custom reports that were glued together on top of the custom database wired into the solution; but the real process metrics that could/should be gleaned from an adequate process model? These are the things that get lost when the process discipline and process centric visions are abandoned.

What to do?

If I had all the answers I suppose I’d write a book instead of this blog, but I do think we can start with the Project Manager role. This role is usually rooted in or reports to the IT department. As such, they’re often more focused on time lines, budget, and meeting requirements. Understandably so, but if this role were also responsible for an effective transition of ownership to the business sponsors/process owners, perhaps we could take some measures to keep the process centric view intact.

From the start of the project, the business sponsors should be accountable for delivering a series of statements regarding the metrics they wish to obtain from the proposed solution. In addition, they should be required to agree upon the manner in which the solution will be deemed a success or failure; this often tied to the metrics acquired. The Project Manager should keep this “project contract” or “decree” if you will in focus at each meeting. When scope is concerned, if the item in question does not lend itself in support of the metrics and measurement for success, then it should not be considered (at least for the first release, table suggestions and “sugar coating” for subsequent releases) . After all, an important goal of BPM is to deliver timely solutions with an opportunity for rapid change; that too must not be lost in the vision.

Many project teams are conducting routine “play backs” with their sponsors, but all too often these merely include screen shots or execution of interface screens and/or reports. Again, the process should run front and center. The process owners should know exactly what their process looks like before it’s delivered. They should be acutely aware of the swim lanes that exist, the roles associated with each, the activity steps defined. These are the things process owners manage, not an interface screen.

Upon delivery, the Project Manager should conduct routine meetings with the process owners and hold them accountable for the metrics they stated were necessary. The same measurements for success should be evaluated again and again. The Project Manager must evaluate his/her own success against the process owner’s ability to own the process, to understand the process, to measure the process, and ultimately to be empowered to improve upon the process.