IT-Analysis.com
IT-Analysis.com Logo
Enterprise SME Business Issues Technology Services Channels
Module Header
Philip HowardBloor IM Blog
Philip Howard
8th February - Bribery
Nigel StanleyBloor Security Blog
Nigel Stanley
8th February - Conficker grounds police checks
David NorfolkThe Norfolk Punt
David Norfolk
3rd February - What's wrong with "security"
Laurie McCabeLaurie McCabe
Laurie McCabe
2nd February - What is Total Cost of Ownership, and Why Should You Care?
Philip HowardBloor IM Blog
Philip Howard
2nd February - Calpont finally comes to market
Module Header
Q. What features do you want to see on this site?
 
Blogs > The Norfolk Punt
The MDM Tarpit
David Norfolk By: David Norfolk, Practice Leader - Development, Bloor Research
Published: 17th August 2009
Copyright Bloor Research © 2009
Logo for Bloor Research

As I look over my Governance landscape I find that I struggle to see how MDM (Master Data Management) really fits into it. "Master Data" isn't something new, as far as I can see, it's just data. And I was brought up to analyse data as part of building an automated system, as part of understanding the (business) problem—before coming up with a solution, which included managed data. Solutions to problems you don't fully understand are usually high risk.

So I don't see a need for "master data", there's just "data". "Good governance" says that it should be maintained, logically, in one place even if it is physically duplicated (but under automated control, not just ad-hoc duplicated as people feel like it) wherever it is needed, for performance reasons. There is a customer entity, and it probably has a customer number and domain-specific names—but everyone should be using the same customer number and the domain specific names should refer back to the single entity. "Obviously" the same applies to the product catalogue and so on—although with things that are "obvious" there is often a small devil hiding in the detail.

However, in my opinion, if you have an MDM project, this is prima facie evidence of poor governance in the past. If you couldn't manage your data properly before, once the initial euphoria of buying your shiny new MDM software has worn off, won't you make the same mess of the "master data" that you previously made of your original data? After all, a fool with a tool is still a fool—what has changed?.

Which all means I am a little puzzled by the popularity of MDM—why make a public fuss over admitting poor governance of your data and development processes? But I think I understand now—MDM isn't a "solution" in the accepted sense, a good practice way of doing things; it should be thought of as an "antipattern"—a temporary or expedient fix for bad practice which includes a documented path back to good practice.

An antipattern is the documentation of a superficially attractive but dysfunctional situation (a tarpit you can't easily escape from), together with a description of how you got into this dysfunctional situation in the first place and a description of how you can, if you are prepared to put in some discipline and effort, get out of it and return to good practice.

So, how do you get into the MDM tarpit? Losing control of development is a good way. I remember a scripting guru saying that development was much faster and more efficient if you gave every developer their own database. I think I can see how this might work (cloned databases from a single data model, with strong change management for changes to the metadata) but I bet this attitude really results in a confusion of (at first) slightly different but overlapping domain-specific data structures. Of course, an even easier way into the tarpit is to forget about data analysis and even proper databases altogether: after all, who needs them when web services" happily lets you mix apples and oranges as long as they fit into the same length field?

But there is one excusable reason for finding yourself in the MDM tarpit—consolidating autonomous parts of your organisation; or taking over or merging with another company with a complete set of systems overlapping your own, will put you there quite neatly. Well, I say "excusable", but it does occur to me that even here a bit of enterprise architecture modelling and up front planning might have helped you to avoid the tarpit.

Enough; what you want to read about here is something positive, about how to get out of the tarpit. It is well-known (irony alert) that doing things wrong and plastering on a quick fix is always more cost effective than doing things right in the first place (besides, doing things right is boring and uncreative). Luckily, a host of software automation solutions for MDM are available to help you—and you won't need me to tell you about them.

Unfortunately, as you won't be surprised to learn, I believe that these software solutions are necessary, but not sufficient. You need to address the cultural and management issues which got you into the tarpit as well—you need to introduce "good governance" to ensure that you don't fall back into the tarpit.

So, when you're out, it's going to be important to carry on with MDM religiously, before you fall back into the tar? Wrong! Once you get out it's important to start to manage your data properly at the metadata level—all data, not just your "master data". After all, what is fairly static "master data" to you may be "transactional data", to me, in my application.

Once your master data is under control you should expand your "master" data management to include all data that is important and shareable—which is all of your data (storing lots of unimportant data that no-one much wants to use is pretty poor governance, in my opinion). If your MDM tools help you to do this, all well and good—but don't build an MDM silo. Look at and re-evaluate the system architecture tools and data modelling tools that would have helped you avoid the expensive detour into the MDM tarpit in the first place—if you'd used them properly, in the context of the delivery of a business service.

Reader Comments

We are no longer accepting comments against this item. We suggest contacting the author directly.

18th August 2009: 'Frank Johnson' said:

Gosh, it's too bad the rest of us don't have Mr. Norfolk's awesome powers of prophecy.

If only we could have predicted the future back in the 80s when we were making good faith efforts to re-engineer processes and develop new applications. If only we'd visited a fortune teller in the 90s before we built siloed data warehouses with the best of intentions to serve the immediate requirements of our internal and external clients.

Then all these problems with data governance and master data management could have been avoided!

(Any advice on the winner of the World Cup? We could bet the full monty and clean up!)

Reply to Frank Johnson?

18th August 2009: 'David Norfolk' said:

Well, there's often a good excuse for doing things wrong - usually, it's poor management. But doing things wrong is still doing things wrong.

Im the late 70s and 80s I was working in DBA on systems that weren't building an MDM tarpit, ad far as I could see.

Nothing too big, the Australian Dept of Health pharmaceutical benefits system and Bank of America automated banking.

I can't claim the credit - too junior and our management supported and resourced DBA and backed us when we warned that people were trying to build a tarpit!

I've no idea if either place built the MDM tarpit later; if they did, it would be because management took its eye off that particular ball and/or allowed itself to be bullied by programmers who didn't understand data analysis!

But no great powers of prophecy were needed.

Reply to David Norfolk?

18th August 2009: 'Frank Johnson' said:

Mr. Norfolk:

I appreciate that you achieved a state of non-MDM tarpit-ness at the two engagements you mentioned. However the very size of those engagements -- the Australian Dept of Health pharmaceutical benefits system and Bank of America automated banking – simply proves my point.

No doubt it was difficult, in enterprises of that size and projects of that scope, to get your arms around all the data sources and applications involved, and to get the political buy-in needed from all levels of management. However, in organizations that large you also had the resources to keep the projects on track and get the work done. Once you had the buy-in and access to the data, you had the means to make it happen.

Meanwhile, over the next two decades, out here in the rest of the non-Global 2000 world, small and mid-size businesses were lucky simply to have one or two people on board who would even think of something as cutting-edge as a data warehouse or BPM applications, much less build them. It was enough that these businesses could just get these applications up and running in answer to specific business needs; they had nowhere near the human or financial IT resources you would have had at Australia DoH or Bank of America to project a strategic MDM vision across the entire enterprise.

I’m sure the chief medical officer at Addenbrooke's or Boston General has grand visions for multi-year, cross-departmental, multi-disciplinary research projects to cure the world’s worst diseases. And he or she has the resources to carry them out. Meanwhile, the country doctors at regional hospitals and charity clinics in the real world are doing all they can to keep their patients healthy and alive -- with limited means and under very trying conditions.

Your attitude is like that CMO of a major research hospital criticizing a country doctor for cracking open a patient’s chest to massage his heart back to life and leaving an ugly scar that someone else has to fix. Does that country doctor wish she’d had the time and resources to do things nice and neat the first time? Of course. But don’t blame her for at least saving the patients life in an urgent situation.

Reply to Frank Johnson?

19th August 2009: 'David Norfolk' said:

Well, fair enough - although even if SMEs have fewer resources, they also have smaller problems to deal with.

I can describe the issue and I think it is a real issue, even in an SME. If you find my implied suggestion impracticable in your environment, I suggest you don't use it....

Reply to David Norfolk?

15th September 2009: 'Jon Ayre (The Enterprising Architect)' said:

I both agree and disagree. I agree that the creation of an MDM actvity in its own right is not productive. Development of the desired information model coupled with a pragmatic alignment to what already exists, identifying within that the preferred master source(s) and introducing mechanisms to propagate and/or expose information as appropriate to the circumstances is all part of good Enterprise Architecture. When tackled as an independent activity the result IMO is that those performing the MDM activity make the data the driving force and do not balance this against other conflicting constraints.

On the other hand, I do not believe that the solution is simply good architecture coupled with good governance. Of course, these play a part, but in anything but the smallest of organisations we live in the real world of COTS/bespoke/legacy/add-on mish-mashes and do not have the luxury of starting from scratch. Thus MDM does exist in a sense but not as an activity or intiative in its own right, but more as an integral part of what we do.

Regards
Jon Ayre
The Enterprising Architect

Reply to Jon Ayre (The Enterprising Architect)?

15th September 2009: 'David Norfolk' said:

Yes, a good comment, and I admit that my original post was a trifle one-sided. But in a world where "everyone" is telling me how great MDM is, I thought a little positive discrimination for the other side was needed :-)

Of course "in anything but the smallest of organisations we live in the real world of COTS/bespoke/legacy/add-on mish-mashes and do not have the luxury of starting from scratch" is absolutely true. But I also believe that compromises have consequences for the business - and that abstractions can be a means of providing a framework for evolving an existing situation towards where it should be, without the crippling overheads of starting from scratch.

Providing that is, people can separate abstractions and compromised physical realities and map between them - and providing that they aren't tryng to work in disconnected IT and business silos...

Reply to David Norfolk?

4th October 2009: 'william' said:

MDM is a pattern of architecture. MDM can be built using a federated set of DBMS and an integration engine. It can be a full system, all in one, integrated.
I think, people tend to use MDM for two main reasons:
- IT crisis: systems are too complex to manage and business rules applied to data are hidden. So when making an Information System overhaul, then, you tend to avoid to make the same mistakes twice. Separate data (MDM) and business rules (BRMS) and try to build your process/apps/service around it.
- Value of data: Since data stewardship is a complex discipline, people tend to think that having everything in one place is simpler to manage.

From my point of view, I think that MDM is good if you follow a model driven approach. You create your UML model and then all the rest is generated (data model, GUI, data services, data lifecyce management).

One tool that is very near to my architect dream is Orchestra Network EBX platform. Nevertheless, I consider the level of abstraction since too low.

Reply to william?

6th October 2009: 'David Norfolk' (Author) said:

Yes that all seems to make sense to me. My only issue is that it sounds awfully similar to the way we were supposed to architect/design systems before "MDM" came along!

I'm all in favour of more abstraction and a model driven approach - done properly, abstractions and models are smaller and more comprehensible than the real world and help provide real governance for automated systems/services

Reply to David Norfolk?

7th October 2009: 'william' said:

Well in fact MDM are managing quite specific data. Most of them are used by several systems and are not changing very often. So we should avoid copying them everywhere without any master/slave vision.
Some systems also magically appeared during M&G or when having a new SaaS product to support. So in that case, it was more an integration problem. Again, data mismatch can cause significant loss for the company.
MDM data are also specific because we should be able to manage their evolution and may be keep tracks of the changes.

Reply to william?

Advertisement



Published by: IT Analysis Communications Ltd.
T: +44 (0)190 888 0760 | F: +44 (0)190 888 0761
Email: