DATA AUDITING AND CLEANSING PROCESS

This comprises four steps:

  1. Data auditing. Data is audited using statistical and database methods to detect anomalies and contradictions against known data facts.
  2. Workflow specification. Following the audit, this is the step that sets out the definition of the defects and anomalies to be detected and corrected. It is important to complete this stage thoroughly to make the next stage as efficient as possible.
  3. Workflow execution is the stage that cleans the data by detecting and removing the anomalies and recording the results. With large sets of data, it can be both costly and time-consuming to complete this stage, emphasising the need to get the second stage right.
  4. Post-processing and controlling. After executing the cleansing stage the results are inspected to verify correctness. Any data not processed automatically should be done so manually. The results are also used to define the next data cleansing cycle, beginning with the audit.

This process is usually carried out using specialist software designed to conduct the audit, define the defects, cleanse them, and report the outcome.

Data quality culture

The best way to ensure good quality data is to have a ‘data quality culture’ that is initiated by the leadership team. It’s not just a matter of implementing validation checks on input screens, because no matter how good they are, they can be avoided by users.

Here are some of the things that organisations with good data quality usually do:

  • Make a commitment to having a data quality culture at high level, with executive responsibility, resources, and budget;
  • Invest in improving manual data entry through real-time data validation, employee training, and incentivisation;
  • Change how processes work and improve systems integration to remove keying or re-keying of data or manual handling of data files;
  • Continually measure and improve data quality.

Data Protection

The data held must comply with local data protection legislation regarding the legitimacy of holding the data, and how they are gathered, updated, and made available to the data subjects.

With many consumers living mostly digital lives, data protection regulation is increasing to keep pace with the breadth and depth of data being created. Compliance with local and global data protection best practice is increasingly a corporate capability, and governed accordingly. Consumers increasingly believe their data belongs to them, and not the bank, which is merely a steward, so customers will demand that the bank takes better care of customer data, and not just from a data protection and cyber-crime point of view. Many will punish banks who don’t take care of data by moving to competitors who do.

Back to