Although it’s not a formal part of the BPM research programme I set out in advance at the end of last year, in the past few weeks I’ve been drawn into looking in some detail at the emerging process mining space.
Process mining has been an active academic research space for some years (and eminent BPM research leader Prof Wil van der Aalst leads a team at TU/e which has been instrumental in launching commercial projects as well as advancing research in the area).
The Process Mining manifesto, published late last year, is a really good overview of the area and call to action – and one thing it makes clear is that the scope of process mining technology is much broader than process discovery (which is the area that most commentary has focused on so far, with much discussion conflating the two concepts).
According to the manifesto, there are three applications of process mining – discovery, conformance checking and enhancement:
- Discovery is about taking event logs and analysing them to produce models of work.
- Conformance is about comparing existing models of work with evidence from event logs to discover any operational 'gaps' between prescribed or recommended practices and actual work in the field.
- Enhancement is about using data from event logs to enrich the information provided by static models – perhaps by overlaying performance information, for example; or even using historical event information to predict the performance of work currently in progress and suggest ways to optimise it 'on the fly'.
What's particularly interesting to me, based on my reading of the manifesto at least, is that the authors (or at least some of them) appear to propose that process mining in its broadest context provides the foundation for a different kind of process management system from the kind many people are familiar with today – one that’s 'passive' rather than 'active'.
This 'passive system' is not like today’s BPMSs, which manage processes and the execution of work using those processes through a core co-ordinating application that orchestrates the flow of work between people and systems.
Rather, through ongoing and continuous mining of event logs 'in the background', not directly connected to the systems that people use to get work done, such a system would work by detecting the shadows that work casts onto existing IT systems; tracking those shadows in the context of models (discovered or purposely created); and then using that analysis to drive a) management insights into opportunities for improvement and b) operational insights into optimal execution of work.
As the manifesto itself points out, the engineering and research foundations are already in place to make a system like this possible today. Such a system would have the potential to deliver many of the benefits that today’s BPM projects can deliver, but without interposing a new application layer that risks disrupting relationships that people have with their existing working habits and IT systems.
Still, though, I think it’s going to take a few years before such systems gain significant mainstream traction in industry. Why? Because a lot of the practical detail of implementing such a system in industry would require new tools to be built, and the big vendor money is currently being poured into ongoing marketing and improvement of today’s generation of BPMSs; and there are no vendors of any significant size that could release such a platform in the near future without confusing the hell out of its prospect and customer base.
I think we will see systems like this start to be deployed in the coming years, particularly in scenarios where 'unstructured' knowledge work is at the heart of the business domain under consideration – but that doesn’t mean process mining is a dead-end: far from it.
I really think we’ll see a lot of real-world deployment of process mining’s discovery application, and quite soon (in the coming months). Why? Because in this context, process mining techniques and technologies help to address an immediate pain point that an established community of industry practitioners have. Specifically, how to quickly discern the actual state of work in a given area of a business to provide a reliable foundation for analysis of improvement opportunities.
I’ll be looking a little more at this 'discovery' aspect of process mining in a forthcoming post. In the meantime, I’d love to get your thoughts on this larger question – it’s still an emerging area and I know I am a long way from having all the answers!