I guess if we can have Technical Debt, and Process Debt, why not Management Debt?

Ben Horowitz’ post on Management Debt is a good read, but one thing that separates it from technical debt or process debt is that it seems to my own naivete that it almost never pays to pick up the management debt if you can avoid it, as described by Ben:

Like technical debt, management debt is incurred when you make an expedient, short-term management decision with an expensive, long-term consequence. Also like technical debt, the trade-off sometimes makes sense, but often does not. More importantly, if you incur the management debt without accounting for it, then you will eventually go management bankrupt.

He gives three tough examples – two in a box, overcompensation, and no feedback loop.

In the end, it comes down to leadership even more than management.  In each of his scenarios, good leadership cuts through the problems or is willing to pay the debt now rather than later.

Dave Brakoniecki writes about Technical Debt:

The fundamental metaphor underlying technical debt is easily understood in technology circles but nearly incomprehensible outside of them. Unless you understand in detail the technical trade-off you have no appreciation for how you are ‘borrowing’ which makes it a trust issue between the business and IT teams. If this trust exists, then you don’t need the metaphor to make the problem explicit. If the trust doesn’t exist, then the metaphor won’t help you.

A counterpoint to this – I recently saw a talk by HomeAway’s CEO, Brian Sharples (at Capital Factory).  I also had a chance to talk to one of their lead developers (a friend of mine from way back).  When I asked my friend how it was going these days, he said (paraphrased) “we’re paying down a ton of technical debt right now… and when I look back, and ask if we should have done anything different, the answer is no.  I think we made the right call.”  Interesting. Thanks to Eric Ries, I knew what he meant by Technical Debt.  I also knew what he meant about paying it down (HomeAway has been acquiring companies, and paying it down in this context means they are consolidating these companies onto common software systems instead of maintaining fragmented systems… )

In their terminology, technical debt is a cost that has to be paid in the future.  It can be estimated in terms of work effort.  But consider a variable interest rate – you have the same problem. You estimate costs, but you don’t know for sure what the future cost will be.

Finally, I actually heard the CEO refer in his talk to paying down technical debt.  He had the same meaning the developers had.  What this told me is that he’s invested in the whole company, not just the “business part” of the company.  Those techies are his people too, not just an IT arm to be offshored later.  And they explicitly understood they were postponing a cost (integration) into the future, a future in which they could afford to pay it down (IPO money helps with that).

Dave goes on to say:

The reason the metaphor doesn’t hold is that the unit of borrowing can’t be quantified. Technical debt is an attempt to use the well-understood concept of financial debt to explain a software development problem but everyone (business and technical) understands what happens when you borrow money and the cost of that decision. It’s explicit.

If you borrow $1, well then the amount you owe is $1. People have mortgages and credit cards and use them to make decisions to borrow money so both everyone in an organization can be reasonably expected to understand the basic concepts of financial debt.

The same is not true of technical debt. The unit of measurement is entirely a trust issue between people. Most technical debt conversations are really the engineers saying to the business stakeholders: If you make me do it this way and in this timeframe, it will cost you more in terms of effort to support per month.

But, unlike the financial debt, the interest may not materialize. On your mortgage, you know exactly how much interest you will pay. It’s not a random number.

Getting a handle on the cost (future cost) of technical debt is hard, but it can be done.    You usually have an ongoing cost that you can think of as interest (the cost of maintaining two systems, for example, or supporting two databases when you only need one).  When you remove one of the two systems, it may have a lot of accumulated debt-  which is now all retired debt.  Presumably its functionality was either no longer needed, or replaced by implementing (and paying for) another system.

As to quantifying – this all comes down to estimation and convention.  A proxy is just lines of code (each line of code incurs some maintenance cost, in theory).  Another proxy is to put work estimates against it and cost those.

One might expect me to come down against metaphors – I’m not a fan of using them as proof in arguments or debates.  They’re better used to explain yourself or your thinking, than they are as “proof”.  But in this case, technical debt is a decent short-hand for work-effort we need to plan for in the future.  It avoids people thinking they’re avoiding cost when they take shortcuts – they’re really just spreading that cost out over a longer period of time (for perhaps a much higher overall cost).  Similarly, it assigns cost to gold-plating the software implementation (more lines of code = more debt).

Pretty interesting perspectives on the concept… for more about applying this concept to processes, check out these posts.

There is one question one should always ask in software, and in particular when designing a BPM solution:

How do you do that now?

In Jason Cohen’s blog, his framing is “What did they do before you came along?” – which is the way I would phrase the question to one of my colleagues at BP3 if they were describing a BPM solution to me without explaining the current approach to the process first. But when I’m speaking with customers, the framing I ususlaly use is “How do you do that now?” And you really do get interesting answers.

Jason’s post mostly discusses how to apply this question to the expert users and niche users within a market segment – the folks who can become real evangelists for your product if you make their lives better – and who can give you a beachhead into selling to the masses.  But the point of understanding how things work today is critical in BPM:

1. Often BPM implementations are asked to do things that are nice-to-haves – not directly related to the business process nor to business value.  For example, “This process needs single sign-on.”  The appropriate response: “How do users sign in today?” If they use single-sign-on today, then you probably have a user expectation to either manage or live up to.  If they don’t – do they have SSO for other applications, or is this the first application with SSO?  (You don’t want to be first).
2. Often BPM projects are asked to implement an existing process exactly – with the technology platform being the only change.  But this misses the chance for really leveraging software to provide process benefit.  By asking how they did this before BPM came along, you can tell whether they’re just papering over an existing process, or whether they’re really re-thinking things.
3. If an integration can’t be completed on schedule or under budget, a BPM practitioner might legitimately ask “well, how do they get this integration done today?” If it is manual, or swivel chair, then continuing that approach at least is no worse than status quo, and can buy some time (via Process Debt) for the right solution to be built.

Just a few examples…

There have been a lot of posts about Google’s “broken” search function lately.  Oh, if only I had a business as broken as Google’s!  But, quite seriously, there has been an erosion of the value proposition.  Not because the information we want isn’t out there, and indexed by Google.  But because it is buried in an avalanche of less relevant and less authoritative information.

So, when someone used to approach me with a general technical question – the process went something like this:

Me:  “Did you Google that?”

Them: “uh, no.”

Me (typing into google search for the answer)… : “there ya go.”

However, these days I increasingly find myself going to more targeted sources – wikipedia, for example.  And if the subject isn’t technical in nature (product-related, lets say) – then I’m heading to twitter, facebook, or other channels to get the opinions of peers.  Of course, you can still use Google and constrain the search to particular domains or sites – but many Google users simply don’t do this or don’t know that they can.

The “Just Google It” process is broken.

If this isn’t an example of how Process Debt can sneak up on you, what is?!

Keith Swenson’s article on the fallacy of flexibility makes a good case for lean software development and the Lean Startup:

This article is about software design, and makes the case that flexibility for flexibility sake should never be your goal.  There is a very delicate balance between design and implementation in order to provide both usability and capability when it comes to software.  Flexibility is often held up as a axiom, but flexibility should be provided only to the extend that it is actually needed by the end user.

I’d put it slightly different: flexibility should be provided to the extent that it is either less work, or actually providing value to the end user.  In that order.  If the absence of doing something yields flexibility, that’s good.  But if you design your software to solve hypothetical situations that have yet to manifest, you run a real risk of overdesign (and over-engineering).

Keith goes on to document a set of rules to keep software simple and maintainable, including removing outdated functionality.  You could sum these up as minimizing technical debt (or in BPM, process debt). The less debt you incur, the more maintainable (and less costly) your code base will be going forward.  The more debt you retire, the better, generally speaking (but, there are trade-offs to retiring debt).

The key insight:

What I am trying to say here is that our normal intuition about the value of designing up front is wrong most of the time when it comes to software because it is not like a physical production process.

So true.  Great read, I recommend anyone technical read the whole article.  If you’re not technical, you might want to read it anyway – it will give you insight into how much the flexibility ideas of your technical staff are costing you.

The first go-round on Process Debt got quite a few reads and private emails and comments that motivated me to keep thinking about his topic and how to further clarify process debt so that we can use this concept to help manage our process development efforts.  It also motivated me to run a small session at bpmCamp on Creating and Retiring Process Debt.

We had a good discussion during the presentation and lots of examples from recent history to draw on in this room.  A couple of folks from the same company were in the room and as a result they tell me that they now have a common, useful vocabulary for describing trade-off process decisions which are made *all* the time in BPM groups.

Presentation embedded here, but also some rough notes below for those who prefer reading text…

(Incidentally, I used a tool called “Prezi” for this which I find pretty useful for organizing thoughts about a topic that requires a lot of context or gets past a certain number of slides… Something about the spatial relationship of the elements helps maintain context in the discussion.)

First, let’s agree that there is such a thing as Technical Debt, as defined by smarter folks than I. But also, that we can repackage it just a bit to more closely align with Process terminology and concerns.

Why we incur Technical Debt (borrowing heavily from McConnell):

1. Time to market – we want to get our process built quickly so that we can start to reap the ROI quickly – and time is money.  If a process can save you $1M per month, each month of additional development effectively costs you $1M.  Time to market matters.
2. Preservation of budget (for startups, preservation of capital)- We have a limited budget, and we want to make sure we spend it on the things that give us the most leverage for getting ROI.  These are likely the same things that will help justify budget for additional improvements going forward.
3. Delaying development expense.  As a system nears end of life, it takes a very high degree of return to justify anything but expedient fixes -because when a system is retired, its technical debt no longer matters and isn’t paid within that system – it is being paid by whatever is replacing it.

What are the types of Technical Debt that we can be concerned with:

1. Unintentional Debt – debt incurred due to poor technical choices, but not with forethought.
2. Intentional Debt – debt incurred for strategic reasons, with explicit trade-offs.
   1. Short-term debt:  sacrifices made to get a specific release out by a certain date.
   2. Long-term debt:  typically architectural decisions (assuming only one database platform, for example) that can be good trade-offs measured in years.

And then we have “Product Design” debt – which I believe can occur in process projects where functionality is added over time until the originally simple screens have become cumbersome and unwieldy.  This isn’t quite the same as Process Debt – but process projects can become afflicted with it.

Finally, we have Process Debt itself.  I think we can follow on the Technical Debt framework and build on it:

1. Unintentional debt
   1. Process Shift – this happens when performance of the process degrades over time.
   2. Requirements Shift – this happens when the external realities change but your process fails to adjust adequately.
   3. Squeezing the balloon – the local process gets optimized at the (greater) expense of other parts of the organization (or customers or vendors)
2. Intentional debt
   1. Short-term trade-offs to get a release out – scope removed, etc.
   2. Experiments to find out which possible solution is the right one
   3. Manual work-arounds in place of system integration (SOA team can’t give you your webservice in time? will a manual work around work for the first few weeks or months?)
   4. Sub-optimal integrations (batch instead of real-time, etc. )
   5. Assumptions around geography or localization or product line support.

Tracking Debt:

1. Enter tasks into work tracking or defect tracking software to track both the items that need to be worked on and the effort required to retire the debt incurred.
2. Measure the process for squeeze-the-balloon effects (can be difficult if the other parts of the balloon are not measured).
3. Teach the team not to take short-cuts that “aren’t worth entering compensating tasks in the software tracking tool” – if they’re not worth it then the short cuts aren’t worth it either.
4. Another suggestion (from James Shore) is to measure Source Lines of Code (SLOC) as an approximation for technical debt. It isn’t an exact measure, but since lines of code will affect cost of maintenance, it is a reasonable proxy.

The starting point is that you have to have a system for tracking your work and work backlog – and from there you can mature the way you think about Process Debt based on what works for your organization.

I’ve been reconnecting with some concepts from the startup world, such as “technical debt” – well-defined in Wikipedia, but also better covered by this article, among others.  Essentially, Technical Debt is the future cost you incur by taking short-cuts (knowingly or unknowingly) today.  Of course, we explicitly choose sub-optimal solutions in the short term in order to have a shot at a better solution later (thus the axiom “Perfect is the Enemy of Good”).  But we also incur technical debt when we inadvertently introduce defects or poor design decisions into production software.

In this The Concept of Product Design Debt is introduced (to me) by Andrew Chen (@andrew_chen).  Although the concepts of Technical Debt and Product Design Debt are not targeted explicitly at BPM projects, the concepts apply particularly well to iterative development projects, and to environments with a lot of change (in requirements and/or technical direction).  As a result, it seems that they apply particularly well to BPM.

Andrew walks through the example of Amazon’s website tabs that would grow seemingly without limit, before being refactored to something more manageable later (paying down the debt).  If you’re truly running an iterative, agile BPM endeavor, the following passage may ring true for you:

The interesting part is when you get a couple months into your product cycle. You often end up with lots of half-done experiments lying around, an infrastructure that isn’t built to scale, and a mishmash of code that needs to be refactored. Most engineers know that in this kind of a case, the best practice is NOT to rewrite your code, but rather refactor it continually and take down the so-called “Technical debt” so that it’s always under control.

However, there’s the other side of the coin, which is the product design. After you’ve added a ton of new features and stuck them all on the homepage, you create Product Design debt. The Amazon tabs at the top are a great example of this – you have a design philosophy built around tabs, you scale it as far as you can, and then you have to refactor your design.

He gives some great examples of how product design debt is introduced into production sites – but it isn’t hard to imagine in the world of process, where we’re often looking to alter our process designs to accommodate changing business requirements.

The analogous question in BPM circles is whether the experiments and changes implemented in your process have created a process debt that needs to be paid down through refactoring.  In large part, I think you could describe the process investments of the last 20 years as having created great improvements in efficiencies, but they’ve also introduced process debt over time – processes have become more complicated than necessary, manual shortcuts have been introduced “temporarily” that became permanent, and IT systems were built to support re-engineering that couldn’t support ongoing change. In each of these cases, BPM can help surface this “process debt” and offer tools for carving it down to size.  The goal isn’t necessarily “zero debt” – but the goal is to keep it from getting out of hand.

Even within BPM projects we can accrue process debt.  This is independent of the technical debt we may have accrued that is purely technical – we’re not talking about running two queries where one would do – we’re talking about the process itself.  Decisions about the process itself may be intentionally short of “right” in the short term, in order to get incremental improvement out the door.  Or the design may have unintended consequences – creating a bottleneck where one wasn’t expected (squeezing the balloon, so to speak) – or fixing errors in one area and introducing errors in another.

I think there is yet another way we accrue process debt – and that is when our organizations fail to adapt to a changing business climate.  If you rolled out a process two years ago and haven’t made any tweaks in the meantime, I believe you have acquired process debt – a steady, growing gap between what your software and processes are designed to handle, and what the reality of current business conditions requires.  After all, new products and services enter the market, new competitors enter, old competitors exit, and customer needs change over time.  In a typical organization, the organization will wait until the process debt becomes quite large, and then address it with a big process improvement project, or a new software roll-out (new CRM system anyone?).  It isn’t that this decision to invest is wrong – in fact without a BPM Suite, this may be the most rational way to deal with process debt.  But an organization focused on process needs to pay down this process debt along the way – and the best way to do it is with a robust BPMS and a team focused on BPM for the long haul, that can manage this process debt and keep it from getting out of reach.

A BPM team that intends to really build capability in their organization for process improvement needs to account for the concept of process debt, and figure out how to manage it over time (and even pay it down from time to time).

Of course, some would probably argue that there’s no such thing as “debt” – that you are simply leaving yourself with additional work to do in a future iteration.  But the argument for applying the concepts of technical debt, product design debt, and perhaps process debt – is that the “iteration” notion doesn’t properly help you capture the cost of decisions you are making now.  We’re not talking about implementing 3/4 of something – we’re talking about implementing something that we know will cause a problem (as described above), and choosing to do it that way because we’ve measured that cost against other benefits which outweigh it.  A classic example is when an integration isn’t available in time for go-live of a process – but we expect it to be in the future.  In the short term we can choose to take on that work with manual intervention (or a suboptimal technical solution).  But we’re incurring a future commitment to switch to that “correct” integration.  It just makes sense to proactively manage these commitments as “debt” to be paid off.

So.  I’ve made the argument in favor of including the Process Debt concept in long-running BPM operations.  Are there good counter-arguments to be made?  Are there problems with managing process debt the way we would manage technical debt?