06 Feb Lessons I’ve learnt in Data Migration
As we all start to settle in to another busy year I think it’s a good time to look back and reflect on the lessons we’ve learnt in the past to make sure we don’t make the same mistakes again.
When I look back at the data migration projects I’ve done in the past, and after speaking to people from different businesses, I found they all encounter similar issues regardless of their industry or data set.
Here are some key lessons I’ve learnt through various data migration projects.
- Engage the business as early as possible. For most data migration projects I’ve worked, getting the right business resources has always been a struggle. Many subject matter experts are already over committed on several projects and priorities. Business commitment is crucial to the success of a data migration project. Not having enough resources will delay your data migration project and put the project at risk. The best way is get the business involved early, this would save a lot of rework down the line. A full time resource from each business sector is highly recommended during the data migration process.
- Assess your data quality. Data migration should not be a health check for your data. It is important that before even thinking about data migration to assess your data quality. So many times I’ve seen projects get delayed due to errors in loading or incomplete data. Most of the errors are caused by bad quality data. Data cleansing and data quality analysis needs to be complete before data migration starts.
- Allow enough time for data migration. On many projects I’ve worked on, the importance of data migration has been overlooked. The data migration schedule has been compressed in order to allow enough time for testing and other deliverables. The data migration team is always under pressure to perform the loads as soon as possible, and as many records as possible. In many instances steps are skipped, the quality of work is likely to be compromised, and so the outcomes are poor.
- Reconciliation is important. Most people think a data migration project is pretty much complete once all data have been loaded. Reconciliation is an important step of the data migration process and should be done as soon as the loading is complete. Adequate time needs to be allocated for reconciliation and business sign off. This refers back to my first point, getting business commitment early to allow no conflicts in time and/or resources.
The above points are a summary of the main lessons I’ve learnt during my past data migration experiences. Thinking back on my personal experience I recall when one project was delayed due to not having the right people at the right time. Data quality was ignored in order to meet project deadlines. The end result of that caused severe business disruptions because incorrect data was loaded. As you can imagine, the effort required to rectify the data issues was massive. Prevention would have been better than the cure, but the project did not learn this lesson until it was too late. What I learnt from that project was that business commitment and in time reconciliation was not only vital for the success of a data migration project, but also to every system implementation project.
As you prepare for your next data migration project, I would urge you to keep my four points in mind. Do they sound familiar? While most people don’t find data migration projects enjoyable, there are lessons we can all take on to make the journey less painful.
Perhaps you have other lessons you want to share. I’d love to hear your thoughts, you can contact me via email or comment on LinkedIn (my contact details can be found at the bottom of this blog).
George Bernard Shaw said “Success does not consist in never making mistakes but in never making the same one a second time.” Remember those mistakes you made in the past? What have you learnt from them?
Jenny Zhu is a Data Consultant for Seed Analytics, an SAP Services Partner, and global provider of solutions for SAP Data Governance and SAP Data Migration.
You can contact Jenny via e-mail (jenny dot zhu at seedanalyticsgroup dot com), find her on linkedin (au.linkedin.com/pub/angel-jenny-zhu/11/490/659/), and follow him on Twitter using @jzhu01