Methods or process for determining scorecards, determining weightings and thresholds

[Former Member]

Hi All,

I was wondering if there were any available best practice or applied methods for determining which rules should form part of a scorecard and what the weightings (dimension weightings %) and thresholds should be (eg yellow = 5, red = <5 green > 9).

I have applied my own logic where I am saying different dimensions could have different weightings, depending on the number of rules and importance of the dimension. eg if you have 1 rule for timeliness you woludn not want this to weight at 25% of the domain score.

Have there been any ways you have broken up the data, into categories and then determined what weightings and thresholds should be applied?

Kind Regards,

Adam

[adrian_storen]

Adam,

I have not seen any so please post if you come across something useful.  In terms of Scorecards, you could break down information by subject areas (eg finance, logistics, payroll, etc) or more granular for different circumstances.  It will depend on the users (managers) requirements for how they want to report DQ.

In my view, thresholds should be set high (ie above 95% for yellow and higher for green).  If you think about your data having 1 million+ rows, 90% leaves 100,000 incorrect rows which is a lot.  I am not sure you could say this is 'good' and deserving of yellow or green.  When there are thousands of records, a lower threshold may suffice but ideally you want minimal exceptions in most cases, especially when it is regulatory/ legislative compliance driving the requirement.

When there is a completeness metric, this would be driven by rules and there should be a high value but could see this having lower importance whether fields are completed.  For example, how many missing customer phone numbers would allow users to know whether the data is 'good' (green) to rely on it?  What if a marketing campaign is being planned?  Often there would be a requirement, but I would lean towards higher weightings/ thresholds for some DQ dimensions (uniqueness, timeliness, accuracy) and less for others (completeness).  The answer is - it depends.

I agree different dimensions should have different weightings.  The default in the CMC is for low/ high thresholds for rules rather than by DQ dimension, so tailor it for your needs.  The 1 timeliness rule could result in a penalty payment to a customer if an action is not completed on time, or extra manual work, or a safety breach, or could result in a single team KPI being missed.  The weighting will be driven by the importance more than arbitrary weighting so it is hard to define rules for weighting.

Hope this helps.

regards

Adrian