Forum Discussion

Kausthub's avatar
Kausthub
Copper Contributor
Apr 21, 2023

Migration of Data from Azure table storage to Azure cosmos DB Table API

Hi,

 

I have a requirement to migrate tables from Azure table storage to Azure cosmos DB Table API. What is the most efficient way to do this? I have got about around 30 tables to migrate with at least 50Millions records in each table.

I tried doing it with Azure data factory but had to face a lot of issues with Data Type in migration.

 

CosmosDB Table API, Table Storage, Azure data Factory, Azure Synapse

  • azharamir13's avatar
    azharamir13
    Brass Contributor
    May 14, 2023
    There is no good or bad way , you to need evaluate the options, here are some comapre which might help the decision

    Azure Data Factory:
    Advantages: Fully managed cloud-based data integration service, can handle complex data transformations and ETL scenarios, supports incremental data loading.
    Disadvantages: Requires some experience with Azure Data Factory and may require additional configuration.

    Azure Storage Explorer:
    Advantages: Easy-to-use graphical user interface, free and cross-platform, supports exporting data to CSV files, which can be easily imported into Azure Cosmos DB Table API.
    Disadvantages: May not be as efficient as other methods for large-scale data migrations, requires manual export/import steps.

    Azure PowerShell:
    Advantages: Powerful command-line tool that can be used to automate the migration process, supports exporting data to JSON files, which can be easily imported into Azure Cosmos DB Table API.
    Disadvantages: Requires some experience with PowerShell and may require additional configuration.

    Azure Cosmos DB Data Migration tool:
    Advantages: Easy-to-use graphical user interface, supports continuous migrations, can handle large-scale data migrations with minimal downtime.
    Disadvantages: May not be as efficient as other methods for small-scale data migrations, may require additional configuration.

Resources