Data Conversions: Best Practices for a Smooth Transition

You just acquired a new company. How do you successfully fold their data into your back-office system? Or maybe you just purchased a new back-office system. How do you move the data from your old system to the new? You are about to embark on a data conversion — the process of transferring data from one system to another. A clean data conversion is key to a smooth implementation process, but thorough preparation and an organized process is paramount to an efficient and clean outcome. 

Data Availability 

The first step to preparing a data conversion plan is finding out what data is available. During this discovery phase, the goal is to locate all available information so you can make an informed decision about what data to convert and how. Is data being kept in a database, or is it maintained manually on index cards or in paper files? Manually kept data equates to a manual conversion, but most often, at least some portion of the information is in a digital format. 

Look for files that contain primary customer data — perhaps the most important data file, including things like name, address, balance, billing information, credit information and customer notes. This is the minimum information needed to service the account, and sometimes it is the single file that is chosen to convert. Also, look for transactions, tank information, service information, equipment information or other data that reflects changes made to the account. 

Manual vs. Automatic Data Conversion

After the data has been located, the most basic conversion decision is whether to perform a manual or an automatic conversion. Even if there is useful data in a system, if the number of accounts is below a certain threshold, some companies will opt to do a manual conversion. That number is often 1,000 accounts, but this needs to be weighed against the manual labor needed to input those accounts by hand, as well as the potential for human error during the input process. 

With automatic conversions, data can be exported and imported automatically. The first step is extraction, and often the software vendor for the old or new system can assist. This can happen through direct access to the old database, a dump of the data into files or by running reports from the old system. Whatever method is used, the goal is to successfully pull the data out for the next step. 

Data Forensics 

Once the data is extracted, it’s time to look closer to better understand what is available and determine the quality of the data. This phase requires some forensics to determine what the fields truly mean. Even when an acquisition uses the same back-office system as the purchaser, the fields can be used differently, so it’s best not to make any assumptions. 

Melissa Spahr of Dead River Company, a fuel provider in Northern New England, has been part of multiple acquisitions over the years. “We actually go on-site to the company and go through their software, looking at each field and what their definition is. You’d be surprised at some of the answers that you get.” 

In addition, data integrity is a concern. Often, systems have been used for years, and while the current staff does things one way, there may still be remnants of old methods in the data. For example, creditworthiness may be indicated by credit codes today, but it may have previously been stored in a text field, and some accounts may still hold it there. Also, older systems may not have enforced business rules as well as modern ones, so the data could contain many inconsistencies. All of this needs to be investigated and understood in order to set up a clean mapping to the new system. 

Map the Data 

Now that the data is understood, it’s time to set up the mapping from the old to the new system. The key to this stage is making sure you are ready for it. “A deep understanding of the data in the old system helps limit assumptions in the mapping and leads to a cleaner conversion,” shared Daniel Legault, conversion team leader at ADD Systems, a propane software provider. 

For some fields, mapping will be straightforward, like customer name, but some may be more complex or even missing and will require logic or calculations to pull across correctly into the new database. For example, important information saved in a text field may need to be parsed and then mapped to a new field in the new system. Other fields, like k-factor, are sometimes kept manually outside a system, so calculations on the last few deliveries can be used to determine and load a k-factor in the new system. The mapping process can handle these needs as well. 

Also, it’s important to think about the transition for customers. You can likely expect people to call for customer support with their old account number for a while, so plan ahead by setting up their old account numbers as an alternate on the new system. It’s a simple step, but it will make for a better customer experience, especially if they are leery of the transition. 

In addition, the mapping process is an opportunity to clean up data. With clear business rules in mind, data that is outside set parameters can be caught and edited or removed. For example, invalid customer types or incorrect coding can be fixed with the import. Additionally, if sensitive data like credit card numbers were kept in a text field, that data can be stripped out and stored properly. 

Test, Test & Test Again 

Once your data map is created, it will be used to convert the data and load it into a test database for review. That means it’s time for testing, and it’s important to remain deeply engaged during this stage of the conversion. It is an opportunity to pinpoint data misinterpretations, anomalies and other issues, and most importantly, make corrections before going live. 

This is an excellent opportunity to investigate the data at a detailed level on individual accounts and in a broad way via aggregate reports. Spot-check customer records for accuracy and consistency. Confirm that account balances are the same in both the old and new systems. Compare the number of automatic accounts and budget accounts by product and compare aging reports in both systems. 

Inevitably, this step will uncover issues that need correcting, and you could have multiple rounds of corrections and testing. The more thorough the testing, the fewer rounds you will need and the quicker the path to live data. 

“Having your three Ts crossed — testing, testing and testing — and not leaving any stones unturned in the verification phase will make your live conversion and transition to the new system a smoother process each and every time,” shared Legault. 

It’s best not to rush this phase, since recognizing a data integrity issue after a few days of live processing can be disruptive and complex to fix. 

Going Live! 

With the data conversion approved, the final phase is to pull the data into the live database. At this point, you can make a clean cut-over to the new system, or you can run the two systems in parallel for a period of time for added confirmation that things are set up properly. No matter which method you choose, it’s important to communicate it clearly. “When making the transition from the old to the new system, it’s important to ensure that you have a clear, concise cut-off date and that users are aware of it, stop entering information in the old system and begin using the new system,” Spahr shared. 

Adding Efficiency 

Once the data conversion is done and the new system is live, you can celebrate a job well done. When moving to a new back-office software solution, the process might be a one-and-done situation, and you won’t have to think about an additional conversion for a number of years. However, if you are a company in acquisition mode, data conversions may be a regular occurrence, and efficiency becomes paramount. Developing a clear process and procedure helps streamline future conversions. 

As you become more of an expert on conversions, there are tools that can empower your company to do more of the work without the help of a software vendor. For example, there are application programming interfaces (APIs) that allow the user to import or export the data themselves. These tools can be used for large or small conversions, granting access to specific files and fields. These tools do not eliminate the need to dedicate time and effort to understanding the data and testing, but they can cut down on the people involved in the conversion. Oftentimes, these APIs are used with smaller acquisitions, ones that would have been done manually and required hours of people’s time for data entry. Access to the API means that all that manual entry can be eliminated. 

Take Your Time to Get It Right 

“Conversions can be quick, long, simple or complex, but they always have many moving parts with many people involved,” shared Legault. It’s most important not to rush. Be thorough with the discovery phase and avoid making assumptions. Take full advantage of testing opportunities to ensure that your final conversion is as clean as possible. A data conversion is more than just a necessary part of the process of moving to a new software solution or merging an acquisition into your system — it is also an opportunity to cleanse and improve your data integrity to lead to a more efficient operation in the future.

For other resources on best technological practices, take a look at these other articles from BPN.