Unlike simple data transfers between systems, data migration is a significantly more complex process. It involves extracting data from the source system, transforming it into a format compatible with the new target system, and then loading or transferring it, ensuring it’s ready for analysis. This comprehensive procedure, referred to as Extraction, Transformation, and Loading (ETL), is fundamental. We handle diverse source data types, including databases like RDBMS/NoSQL, file formats such as CSV/JSON/XML, SaaS applications, and application events.
Define, Gather, Assess, Plan: The objective(s) of the migration will be defined and the necessary data gathered and assessed; it’s location, format, complexity, quality and quantity. A plan will be formulated including timelines, risks and challenges.
Extract, Transform & Load: The data is extracted from the source using appropriate tools or scripts, it is transformed or converted into the format required, then loaded into the data warehouse awaiting analytical processing.
Data Cleansing: The data is profiled; its format, characteristics, relationships, and dependencies, then cleansed by addressing inconsistencies, errors, or duplications.
Validation & Testing: Data validation ensures the migrated data meets quality and integrity requirements, followed by functional, performance & user acceptance testing.
Data Mapping: The data migration is mapped, with a clear pathway between the source / raw data and the target system; the Data Warehouse.
Monitor, Audit & Maintain: The data is monitored to verify the successful migration and is accessible as planned in the target system.
© 2011~2024 Redland Information Systems. Company Number 7520551 VAT Registration Number: 107109253
Illustrations by Catalyst Labs’s Stuff
