I think this post by Activiti‘s Tom Baeyens reveals a blind-spot in the folks behind open source BPM tooling.

To be clear: it isn’t a bad post, and I agree with his conclusions!

Which are, summarized:

• “hosting traditional BPM engine on the cloud is a big technical challenge with a relative low value for professional consumers”
• The data manipulations required by BPM’s automated steps are too complicated to expect professional consumers to design in a webpage (the example given was getting contents from a spreadsheet into a pdf document).
• “On the other hand, the trend to Advanced Case Management (ACM) really fits well into the cloud.”
• “Dynamic management of tasks without a predefined flow matches perfect with the professional consumer needs and capabilities.” (Author’s note: whether you call it ACM or BPM, we understand what Tom is getting at here. )

So if I agree with his conclusions… what’s the problem?  Just that commercial companies have come to these same conclusions a few years earlier – and open source BPM projects have been more focused on building the engines than on exactly what the deployment architectures will be – and what the implications on product direction would be if you change those deployment choices.  So, it has been a blind spot – but that isn’t the end of the world. BPM in the cloud is still in its very early stages.  Even “ACM” in the cloud is in its infancy. Activiti (and others) have time to address the blind spot, and maybe something new and interesting will come out of new entrants to that combination of cloud and BPM/ACM. I’m looking forward to see what Tom and the team working on Activiti come up with.

Tom’s writeup also confirms another conclusion I’ve long held about “ACM” as software implementation- it just isn’t as technically difficult to produce as a BPM platform (throw out all that integration stuff for example, and any notion of structure).  That isn’t some kind of badge of honor to be “more difficult” – it just means it may not be very defensible for a software company to build around, and it tends to look like a toy to customers, rather than a serious enterprise product that stands on its own.

Activiti strikes again with the release of 5.4.  They have had a drumbeat of releases since the original GA shipped (and even before, but who’s counting), and they’re at it yet again.

I’m once again humbled by the great information available in the user guide.  It only took a few minutes to download and install Activiti 5.4 – but those interested should bear in mind the installation is really targeted at a developer-persona – you have to run scripts to install etc.  Certainly no worse than many commercial products, and in fact, quite a bit easier than most.

There are a few interesting and useful new features:

• Improvements to Activiti Cycle round-trip
• Support for more “case management” functionality – dynamic comments, attachments, dates, etc.
• Looks like you can scan an IMAP account for tasks (often a useful way to detect process starts or status updates).

In related news, Tijs Rademakers joins the Activiti team at Alfresco.  He’s leading the development of the Eclipse Designer, obviously a key element in the framework. I don’t know Tijs personally but he has a great (and relevant) track record and is working on a book about Activiti.

Very interesting post from Tom Baeyens on Activiti’s approach to ad-hoc processes and how that plays into their overall BPM strategy – as well as how it relates to BPMN2:

The first step that we’re adding now to Activiti is a snappy environment in which that kind of collaboration is supported. Apart from the tasks that are created by process instances, you’ll be able to create tasks dynamically on the fly. It will be possible to involve people with these tasks, have discussions and associate any kind of content like plain URL’s, Alfresco docs, Google docs etc to the task (aka case). Furthermore, it will be possible to create sub tasks dynamically. Here’s one of our early mockups:

I think it is smart for Activiti not to *just* focus on the engine, but also think about collaboration and other features that weren’t as strong in the previous generation of BPM tooling. The interesting tidbit in Tom’s email (to me) was this:

In some cases, this might grow to become pretty complex processes. At that point, Activiti KickStart modeling might be too limited as that is targeted at non technical people. Because Activiti is based on BPMN 2.0, it will be possible to move these organically grown processes into full BPMN 2.0 based modeling tools.

This reveals that even with KickStart and their tackling of ad-hoc, they intend to represent the outcome as a BPMN2 model (XML), and therefore it is a model that can be “uplifted” to a more structured or managed process as it matures or when it becomes valuable enough to manage in that way. If they can pull this off gracefully it sets a high bar for other BPM products.

I’ll share some more thoughts on “unstructured” knowledge work in the next post…

This isn’t a review of Activiti 5, per se – I haven’t had time to play with it enough to render a review.  This is a bit like my previous post on the subject, where I had time to download, install, and build an example.  Just like last time, I was happy with how easy the process was to download, install, and run on my Mac OSX laptop.

Tom Baeyens announced Activiti 5′s GA, and I have to hand it to the team for meeting their aggressive goal of releasing before the end of 2010!

Moreover, Joram announced the introduction of a feature to “generate” the BPMN diagrams (based on the diagram interchange specification), as of Activiti 5.1.

It is quite an impressive feat of community building and momentum – including the Activiti Cycle module that I’ve been reading about on Camunda’s blog posts. Clearly Activiti has raised the bar for BPM on open source, liberal license, open development community.  While Activiti has a way to go to reach the same level of maturity as some other BPM offerings, it already has a lot to offer for someone who wants to both develop a BPM solution and contribute to a fairly interesting entrant in the BPM space.

Lots of news on the Activiti front lately.

First, Tom Baeyens has a list of what industry experts are saying about Activiti.  I was even mentioned in this summary – a sure way to get a mention in our blog ;)

Next, Tom announces that Beta 1 is released (Nice to be out of alpha!).  It includes a release of Activity Cycle, contributed by Camunda.  Pretty good stuff.

All of this is followed by a re-org of the wiki, and the announcement of the first iPhone App for Activiti.  I’ll just say I think the iPhone app follows the obvious path – I’d like to see something a bit more… creative… but you have to start somewhere, right?  I’d like to see something a bit more dynamic… maybe I’ll have to write an iPhone App though before I criticize someone else’s efforts.

Seems as though progress on Activiti is going well.  Congrats to the various contributors -

If I know Tom Baeyens, he’s pretty happy with this blog post announcing the collaboration of Activiti and Camunda on “Activiti Cycle”.  Previously, Camunda had announced Camunda Fox, a set of tools to help accelerate using open source software for BPM, while pursuing business-IT alignment.  Cycle has been proposed as the name for the collaborative authoring of Activiti processes, and Camunda has now become the lead developer on that feature set.  This sounds like a win-win for both organizations.

Surprised to run across this post the other day on the BlueWorks blog.  Not sure exactly what prompted the timing of it, but it is a good introduction to BPMN and why it exists.

Meanwhile, Tom Baeyens has his first Activiti presentation available via slideshare and on his blog.  Activiti is the new BPMN 2 open source project he is working on at Alfresco.

Tom Baeyens writes:

We’re very committed to BPMN 2.0. In fact, we’re aiming to build the #1 BPMN 2.0 process engine and deliver the full BPM Suite components, all available as open source.

Glad to hear it – I’m sure the world of BPMN and Open Source BPM will be the better for it.

With all the hubbub about Activity, I have to admit I found Mark Little’s post about jBPM underwhelming.  After four rambling paragraphs about the historical ties between jBPM and the rest of jBoss (much like the pre-amble you hear right before someone says “…buttttttt…..” ), Mark finally gets to the point.  By this time, I was waiting for the axe to fall.

But of course things don’t always run according to plan and 6 years after joining, Tom decided to leave for pastures new. That left us with a new jBPM lead, Alejandro, who has been a skilled engineer working on the project for many years. It also coincided with the release of SOA-P 5.0 and a need to continue the workflow rethink that had been going on for quite a while. Naturally the departure of any project lead may cause worries in the community and with customers, but where jBPM is concerned they are unfounded: jBPM has been core to all of our BPM and SOA efforts for years and that will remain the case, whoever is the project lead.

Oh wait. That wasn’t the point yet. Ah, here it is, two paragraphs later:

However, it seems that there remain concerns around jBPM 4. Will it be productized? Will it be supported (really the same as the first question)? Will we put any engineering effort into it? It is fair to say that there was expectation that jBPM 4 would appear in a version of the SOA Platform as a replacement for jBPM 3. With the changes to the jBPM project team and the next steps in unifying our efforts in this area across projects, it is not going to happen.

Mark reassures the jBPM faithful that they will continue to invest in jBPM 4 and release new versions.  But RedHat isn’t going to offer paid support for it, and yet they don’t have a plan for jBPM 5 yet.  There’s a pretty tough thread on the discussion forums. You can’t help but feel for people who’ve advocated for jBPM4 who now feel abandoned because they or their customers cannot purchase jBPM4 support contracts.

I think the interesting part is that Mark doesn’t seem to understand what all the fuss is about.  The comment thread on his blog is an interesting read. Mark’s first response to a very tough criticism (from Mark Roy) of the handling of the transition is:

Mark, I’m surprised and disappointed that you feel that this transition has been handled poorly. The team needed to think things through before making many public statements and that’s the right thing to do. I don’t think the community or our customers would thank us for making rash statements that we then retracted only a few days later. And let’s be clear: it’s only been 3 weeks since Tom left and in the software business that’s hardly an eternity. [...]

Well, I think that the open source community expects to be involved in the discussions. They feel an ownership of the project, because they’ve invested in it. They want to be able to make their case *before* a final decision has been made behind closed doors, not after.  Managing communities (open source or otherwise) is tough.  Mark and his team have their work cut out to rebuild trust in the jBPM community.  I’m not an expert on the jBPM community or the politics within open source communities in general, but I thought this kind of support angst only happened with commercial software!

When I worked for a BPM software vendor, I often told people that I wasn’t too worried about open source software because BPM is a different animal than many other software categories-  user experience is critical;  the problem-space is wide, rather than narrow; and it requires seamless coordination of many different activities, rather than just a few activities.  Add to that, the standards and specifications hadn’t firmed up yet, as they had for databases.

However, the environment is evolving:

• BPMN 2.0 provides what might be the first standard that is complete enough for open source software to use as a basis for competing with the commercial vendors
• User Interface software development tools have improved.  It is easier to produce visually appealing applications that run inside a browser than it ever was before (thanks to HTML 5, AJAX, and several libraries built to leverage both).
• The cloud makes potential deployment of complicated software components easier to manage.

This sets the stage for a new round of open source software projects.  Tom Baeyens and Joram Barrez (formerly of the jBPM team) have joined Alfresco and are building a new BPM solution with a tight integration to ECM (Alfresco) in mind. They’re calling the new effort “Activiti”, just announced yesterday.  The alpha is already up on their website and downloadable.

Sandy Kemsley was briefed last week and has a very good summary written up.

I also had the pleasure of a briefing – quite flattered that Tom and company thought of us as well.  There are a couple of developments with Activiti that I already find refreshing:

1. They’re using a version of Signavio’s modeler.  Gero Decker’s team produces one of the better web-based BPMN 2 compliant modeling tools out there.
2. The installation process was pretty painless – even on my Mac – for a developer.  You need Java and Ant to run the install and build the demo, but clearly it would be a small effort to create a consumer-usable installer if that was warranted.  Since the install is really for the server components, I don’t know if an end-user consumable install is really necessary. Authors would just point their browser at the server.  Did I mention that the whole stack ran just fine, natively, on my Mac as well as a Windows VM?
3. The documentation is already pretty comprehensive, and gets down to no-nonsense details (not true for many commercial products).
4. There’s already a REST API.  I like the early emphasis on interfaces, APIs, and framework – because that effort will be rewarded by making it easier for contributors to be productive.

There’s still a focus on embedding the engine inside other software – and on more liberal licensing terms (Apache license) that should make it easier for software companies to adopt it.  And I think the market is ripe for an open source BPM platform that leverages standard underlying technologies and is built from the beginning to allow for cloud-based deployment.  I think they’re off to a good start to create something that is really developer-friendly.

So what are the concerns?

1. It isn’t clear (to me, yet) what the long-term involvement / direction of Signavio is – after all, Signavio has a for-profit business of hosting modeling in the cloud.  But at some point I think a true open source project will want to be able to improve on the modeler as well as other components at the engine level.
2. There’s a lot of work yet to do. Right now much of the engine isn’t yet implemented.  This is actually apparent from the documentation – and I love that the team had the courage to expose what they have, and what they don’t have ready at this point.
3. Earlier I mentioned developer-friendly… what about user-friendly??

Well, as to the last concern, I’ll give my 2 cents.  I think for a commercial product, user-friendly is one of the out-of-the-gate concerns. For an open source project, developer-friendly has to come first – because if it is developer-friendly, then additional developers can join the team and build the user-friendly layers.  If you start out with something relatively user-friendly but not developer friendly, it is as though you were “charging” your developers more to work on your software – because they have to give up more of their time before they can produce something useful.

Already two of us at bp3 have downloaded the alpha, installed it and run it.  We may end up investing some time in the project ourselves.  Clearly Alfresco and the other companies involved in the project are putting a lot of energy into this effort, and more PR behind it than I felt jBPM ever got as part of RedHat.  They’re off to a good start, with ambitious goals – and as they say, dream no little dreams…

Lest there be any doubt that BPM skills are in demand, Tom Baeyens has pointed out that SimplyHired stats claims the average jBPM salary in CA is US$114,000.  Not bad.  Lombardi BPM shows similar numbers.  It supports what we’ve been saying all along – BPM skills are in demand, and it should be a great career for the next decade (or two, or more).  Its just hard to see “process” going away as an important concept in business. So what the SimplyHired stats tell us is that the technical side of BPM is in demand – regardless of the tool set, there is a mismatch between supply and demand right now that is going to take time to fill – and knowing the technical side of the coin is only half of it – the other half being the business process side of things.  I can tell you from experience that not all great technical people have an interest in business process, and not all of them can make the adjustment to focusing on the process over the technology.

In the meantime, your best bet to get up to speed is to get a job that will let you learn on the job, attend training, etc.  There really aren’t any formal education programs at universities that are widely recognized (although there are a small number of universities that have a small number of opportunities to learn about business processes).  There are classes you can take from the software vendors or the likes of Bruce Silver, and there are certification programs-  but no sense paying for certification until you know what you’re doing.  Once you get started, then try to take advantage of the many resources on the net, and resources like bpmCamp.

Tom continues to update the world with jBPM updates – in this case, using jBPM 4 and Drools to blend process and rules. His updates definitely play to the technical audience rather than the business – but I don’t find that too surprising in the open source world.  From a technical perspective, it is certainly interesting.  Proof that these memes seem to emerge on their own : Bruce Silver has also recently posted on rules and BPM (part 2 of a previous effort).

At some point I look forward to digging into jBPM more thoroughly, and now that it supports BPMN 2, I’m more inclined to make the time, its starting to get interesting for the kinds of problems we look at.  However, I still fear that it is just a bit too technical in terms of what it requires of process authors still.

A previous update confirms that jBPM now supports BPMN 2.0, as of version 4.0.  This is a niche I think open source can help fill – potentially fully implementing a spec that probably won’t be fully implemented by any commercial software vendor.  (Filling out the corners is just the kind of academic exercise that seems to get tackled by *someone* within an open source effort)

Tom’s blog is one of my favorite to keep track of, partly because he takes a fairly pure software engineering approach to the business problem of BPM.  It sounds ironic perhaps, that I would find this interesting, given that our focus at BP3 is all about business process improvement, and *business* results. However, there is something refreshing about Tom’s approach to BPM, as embodied in the jBPM project.  There is also something pure about trying to provide tooling for a problem, without worrying about certain commercial interests that affect the behavior of enterprise software vendors.

I still think the commercial vendors are providing an overall user experience that is closer to what the business (and IT) wants out of BPM for truly business-facing processes.  However, efforts like jBPM have a real chance at becoming the de-facto plumbing behind commercial products, by providing (as Tom would put it) a “bottom up” set of foundational building blocks that build up to what you need for a BPM solution. Will jBPM be the chief open source answer to BPM the way that Apache was for web servers?  Too early to tell, and there are other efforts – but if jBPM lives up to its billing, and its core team of contributors keep at it, it seems plausible.

I’ve been thinking about what would happen if someone took a very BPMN centric point of view and tried to overlay it on jBPM – could they describe all of the behaviors in jPDL?  Would the Process Virtual Machine (PVM) have all the necessary primitives to support the BPMN aspects?  Could the end-result be tied into an existing modeling framework/toolset?

Here’s the slideshow Tom gave at a recent BeJUG meeting: