Anonymization and masking of data

Friday afternoon. SAP administrator finishes refreshing test system after copy from production. Monday morning, the project team is scheduled to begin testing. Consultants, testers and ABAP developers get access. Some of them need broad permissions, because the end-to-end process needs to be tested quickly. However, there is more than just “test" data in the system. There’s also customer data, suppliers, discounts, account numbers, personnel information and user e-mail addresses. Production has been copied. Risks, too.

This is not a description of a specific incident. It’s a situation that many IT departments are familiar with.

The test system must be accessible. It must allow testers, consultants and developers to test processes on near real-time data and volumes. Without this, it is difficult to reliably test new features, patches, reports, interfaces or migration scenarios.

But testable does not mean neutral. And it doesn’t mean: safe by definition.

The test environment may contain data that every user does not see in the production system: employee names, identification numbers (Tax ID/PESEL), physical and e-mail addresses, salaries, customer and supplier data, bank account numbers, pricing terms, discounts, sales volumes, material specifications, SAP users’ production logins and passwords.

If such a system is more widely available than production, the risk surface grows. Sometimes all it takes is a mistake, overly broad permissions, a file exported “for a moment," or access given to a person who was only supposed to test a piece of the process.

The production system is usually heavily guarded. Access is restricted, roles are better controlled, and every change goes through strict procedures. A mistake in production can stop a business process.

The test system works differently. It is supposed to enable project work. It is supposed to be flexible. Sometimes a user needs to be given broader permissions so as not to block testing. Sometimes an outside consultant enters the system. Sometimes a tester from another team. Sometimes an ABAP developer who needs to check an extension on data stored in SAP tables.

This is sometimes operationally justified. But it begs the question: should a person with such access really be able to see full production data?

Usually no.

That’s why securing a test system should not just be about managing permissions. Permissions are important. However, they do not solve the whole problem. The second layer of protection is working on data that has been previously anonymized, masked or selectively deleted.

Test data anonymization is often associated with RODO. Rightly so, but names, addresses, employee numbers, salaries, contact information, customer and supplier data are only part of the issue.

We also have information that is not always personal data, but nevertheless has great business value. Discounts given to customers. Discounts negotiated with suppliers. Sales channels. Material data. Production specifications. Selected controlling data, for example, profitability data of the CO-PA module.

For IT, these are specific objects and table fields that, when copied from production, can appear in the test. Each data area may require a different approach. Not everything needs to be deleted. Not everything needs to be anonymized in the same way. First you need to determine what is truly sensitive. Then decide what should be deleted, what should be masked, and what can be left unchanged.

It seems that the simplest solution is to remove sensitive data from the test system.

Sometimes this is a good solution. If you don’t need the data for testing, you can delete it. The problem starts when you delete too much. The process stops working. The report has nothing to show. The integration test ends with an error because related records are missing. The environment is safer, but less useful.

Therefore, simply deleting data is often not enough.

A better approach is to combine several techniques. Some of the data can be removed. Some can be masked. Part can be anonymized. Some to replace with values from a dictionary. Some to simplify or aggregate.

Example: the first and last name can be replaced with another name. The company name can be replaced with another name from the dictionary. Bank account number can be changed to a pseudo-IBAN, which meets the formal requirements, but is not a real account. The tax number can be replaced with a value that is technically correct, but unrelated to the real entity. The full date can be simplified by leaving the year and a technical date, such as January 1, if the field requires a full format.

You can also use masks like “First name 0001" and “Last name 0001", interfere with selected values, aggregate records or perform random substitutions. The goal is one: the data is to stop pointing to actual people, companies, accounts, discounts or specifications, but still allow you to check the process.

Well-designed data masking is not about randomly blurring fields. It’s about working on the structure of the system.

If we anonymize business partner data, it is not enough to change one name in one place. Business partner can appear in many tables and documents. It has links to sales, finance, logistics and reporting. The substitution must be consistent.

In the test system, instead of the actual counterparty, a different partner extension number, a different company name, a different address, a different bank account and other values in sensitive fields may appear. The record still exists. The process can still work. However, the test user does not see the data of the real business partner.

The same is true for financial, HR or sales data. If a company is testing reports, JPK, KSeF, sales or financial processes, the data must maintain the right structure. They don’t always have to be true. They need to be logical from the point of view of the test.

This is a fine line. If we change too much or in the wrong place, the test is no longer reliable. If we change too little, the data is still too close to production.

Therefore, the extent of anonymization must be determined consciously. The most critical data is worth anonymizing or deleting. For moderately critical data, you can choose a specific range. Non-critical data does not need to be changed just because it is technically possible.

All for One Test Data Anonymizer is a tool for anonymizing and selectively removing sensitive data from production systems in SAP test systems. The product helps protect business-critical data and mitigate risks associated with RODO on personal data.

The scenario is simple. First, a copy of the production system is created. Then the All for One Test Data Anonymizer is installed on the test system. The solution is configured under a set range of data and rules. Once the process is running, the test system contains either anonymized, masked or selectively deleted data.

This is not a one-time manual operation on the tables. Once the solution is set up, the process can be used again for subsequent refreshes of the test system. In many companies, such a cycle is repeated on a regular basis, so it’s not about “one cleanup" of data, but a repeatable process.

After implementation and initial deployment, subsequent anonymization is usually performed by SAP administrators on the client side. The solution is installed in the SAP system using ABAP transport orders. The scope of anonymization is defined during the workshop, then configured and verified as part of the test.

The first step in anonymizing data is to determine the scope. It is worth starting with the most critical data.

Then rules are defined. They can apply to data selection, value mapping, how to swap fields or delete a certain range of information. The tool uses technical anonymization objects assigned to modules. They include tables, links between them and prepared data transformation mechanisms. Importantly, the tool has a set of pre-configured data anonymization objects, which significantly shortens and simplifies the implementation. Once established, the configuration is saved for reuse.

It is not always worth starting with full anonymization of the entire system.

A more sensible starting point may be a selected area (Proof of Concept): business partner data, employee data, financial and discount data, or that part of the process where people outside the organization have access to the test system.

Such an approach allows you to quickly anonymize the most sensitive data and think calmly about expanding the scope of anonymization, masking or deleting data in the future.

The result of the implementation is an environment in which data has been either anonymized, masked or selectively deleted according to an agreed scope.

The benefit is concrete: less risk of working on real data outside of production – mitigating the risks associated with RODO and leakage of company-critical data.

If there are concerns in your company about whether your business data is sufficiently protected in an SAP test environment, it’s worth asking yourself a few simple questions:

• Is the SAP test system being developed as a production copy?
• Does it get data from customers, suppliers, employees or SAP users?
• Do the tests involve people from outside the organization?
• Do broad entitlements happen in a test environment, if only temporarily?
• Do we test processes on financial, HR, sales or discount data?

If the answer to several of these questions is “yes," then anonymization of test data is necessary to work securely with the SAP test system.

The conversation about anonymizing SAP test data is best started by defining the scope. Here, the predefined ready-made anonymization areas provided by All for One can be helpful. As a next step, it is worth considering whether there is data that is not in the list of “ready" areas that should be removed, masked or anonymized. Based on the scope, you can quickly price the project using the All for One Test Data Anonymizer tool. It is a good idea to start with a PoC for the most critical area.

The standard implementation schedule is 1-2 months – from the scope confirmation workshop to the first launch.

Krzysztof Siwiec, Director of SAP Transformation Center