Big Data With Dangerous Weaknesses: Many Companies Neglect Data Quality

Outdated and incomplete data can reduce the validity of analyses, cause additional work and become a security problem why data quality is so essential for companies.

Companies collect more and more data to evaluate it and derive essential insights from it. However, highly developed data analysis tools, which are increasingly also based on artificial intelligence, are of little use if the data quality is incorrect. Companies that rely heavily on data for their business models and processes need clear guidelines. Namely, when data can be maintained and, if necessary, deleted. There are corresponding policies in almost every company. The problem in practice, however, is often the implementation.

Businesses are collecting more and more diverse data and using more and more channels to interact with their customers. This often results in fragmented data silos that can only be broken open and centralised with great effort. Against this background, there is a great danger that inaccurate, incomplete, and outdated data sets will be created, which will reduce the meaningfulness and topicality of the insights gained in analyses.

Data Quality: Guidelines For Handling Data

The complexity has increased from a legal point of view – keyword General Data Protection Regulation (GDPR). This applies in particular to the question of the deletion of data. According to a study by Blancco, a solution provider for data maintenance, 96 percent of the more than 1,800 companies surveyed worldwide have guidelines for handling and deleting data. However, most companies fail to convey these regulations to their employees comprehensively. In Germany, this applies to half of all participants in the survey. With noticeable effects on the operational handling of information collected by the company.

Sensitive Data Can Quickly Fall Into The Wrong Hands

Against this background, in many companies, there is a feeling of false security in data handling. Especially when it comes to deleting them. As the study further shows, this often involves the purely physical destruction of data carriers or deletion or formatting processes. However, formatted hard drives are comparatively easy to recover, so sensitive data can potentially fall into the wrong hands.

A similar risk exists when employees leave the company or when obsolete laptops, desktop computers, hard drives, or server hardware are earmarked for disposal. According to the study, around half of all old devices are disposed of by third parties outside the company’s direct sphere of influence. Suppose the devices are stored for a long time before they are deleted, or there is insufficient documentation of which data was securely deleted. In that case, companies can quickly find themselves in need of explanations.

Anchoring Data Competence In The Company

The formulation of guidelines for compliance and data protection, as well as for handling and deleting data, is not enough. Companies that work with data and their analysis also need clear personal responsibilities that go beyond the pure text of the regulations. Namely for the areas of data competence and data security, for example, in the form of a CDO (Chief Data/Digital Officer). He is responsible for implementing the relevant guidelines, promoting their compliance and implementation, demands the necessary processes, and communicating them.

Secure Erasing Of Data

This is how data can be deleted securely and in compliance with legal compliance:

• First of all, the primary conditions have to be defined. This includes defining standards regarding availability, use, data quality, access, security, and protection.
• Responsibilities for implementing the guidelines and monitoring compliance must be assigned.
• The times for cleaning and deleting personal data and retention periods should be specified in the framework conditions. The legal requirements (such as the GDPR) must also be considered.
• The previously defined guidelines must be communicated company-wide, and all employees must be aware of the data quality issue.
• The data erasure policy should cover all IT assets – including smartphones, tablets, PCs, servers, and the virtual infrastructure.
• Devices with sensitive data mustn’t leave the company or the data centre environment – this is particularly true given that hybrid work models can no longer separate the private and business use of end devices.
• Even with old appliances, those responsible should ensure that their data remains within the respective company’s IT infrastructure sphere of influence. For example, recycling or donation. In these cases, data should be deleted from the devices on-site, and a corresponding certificate should prove the cleanup.
• If an external provider takes over the disposal of old appliances, they should create a complete chain of evidence detailing how the goods have been handled since they were picked up. In this case, it is advisable to have a data destruction certificate issued for each device.
• At the end of its useful life, companies should preferably dispose of appliances within 24 hours.

The Path To High Data Quality Is A Continuous Process

Oliver Rozić, Vice President of Product Engineering at Sage, explains: “Deleting data is an ongoing learning process for everyone. The secure and legally compliant deletion of data no longer only affects IT or data managers but many departments and employees. Therefore, in addition to the relevant measures, regular training and internal feedback loops are important to ensure that the guidelines are correctly implemented in all departments and among all employees, co-workers, freelancers, and partners. Like digitization, the path to clean and well-maintained data is a continuous process that cannot be implemented overnight. But it is important to start today.”