Technology Data Management
Business Issues Channels Enterprise Services SME Technology
Module Header
Craig WentworthMWD Advisors
Craig Wentworth
16th April - Egnyte the blue touchpaper...
Louella FernandesLouella Fernandes
Louella Fernandes
11th April - Managed Print Services: Are SMBs Ready?
Louella FernandesLouella Fernandes
Louella Fernandes
11th April - The Managed Print Services (MPS) Opportunity for SMBs
Simon HollowayThe Holloway Angle
Simon Holloway
11th April - Intellinote - capture anything!
David NorfolkThe Norfolk Punt
David Norfolk
11th April - On the road to Morocco


Party Animal Enters the Data Quality Market
Andy Hayler By: Andy Hayler, CEO, The Information Difference
Published: 17th April 2009
Copyright The Information Difference © 2009
Logo for The Information Difference

In a crowded software market it is difficult to make your marketing message stand out from the crowd, so I was intrigued to see a new data quality start-up with a genuinely eye-catching press release: "DataQualityFirst Makes Sense of Madoff Account Data". It turns out that during the various legal proceedings underway regarding the Madoff Ponzi scheme, a US judge placed a file of Madoff's unfortunate investors in the public domain, apparently in order to help claimants realise that they could register for a potential claim against the now bankrupt Madoff organisation.

The newspapers naturally had their prurient fun identifying famous names of betrayed investors, but DataQualityFirst, a Massachusetts-based start-up, uses the file as a case study for its new software. The Madoff file is quite instructive as it is just the typical kind of poorly structured file which we have all encountered in the world of real-life data projects. The file has no metadata, just six columns, with over 13,000 records. The PartyQualityInsight (PQI) product was used to detect "parties" (such as investors, custodians, intermediaries, estates) from the raw data and, after just four simple rules were defined (e.g. "first name found is the likely account holder") produced a neat analysis of the data. Most of the records had more than one "party" so first PQI figured out there were a total of more than 23,000 "party instances" to work with. Then PQI automatically reduced that down to 13,593 unique people and businesses and showed where there are relationships between them. A few other points surfaced, such as the clustering of Madoff customers at certain addresses, as presumably these were clients recommending the Madoff scheme to their neighbours.

The point underlying the demonstration is not the specifics of this case but how quickly the software could profile the data and make sense of the structure inherent in it. While many profiling tools could do something similar, I liked the very intuitive "data health" type scorecard display that the software has, which makes it is easy to see at a glance the general state of the data. The company makes the important but often-missed distinction about understanding that a person or company can have many roles. I can be a customer of a company and also a supplier to it—i.e. be different "parties".

The company founders have a solid background with Vality, one of the pioneers of data quality, and have taken the unusual approach of basing their application on top of the existing and well proven Vality tool set (now owned by IBM and sold as Quality Stage). This technology has considerable history so is well proven and has quite powerful functionally, but typically requires a fair bit of set up and configuration to apply in practice. By taking advantage of this existing technology PartyQualityInsight, while limiting its market somewhat to the IBM world, at least does not have to worry about justifying whether its underlying technology works.

It is early days for the company but the idea of using the Madoff dataset as a case study was a very inventive piece of PR. If their software turns out to be as innovative as their marketing then they should do well.


Published by: IT Analysis Communications Ltd.
T: +44 (0)190 888 0760 | F: +44 (0)190 888 0761