Azure SQL 数据库：Transfer/Migrate Table 从一个 Azure SQL 数据库到另一个 Azure SQL 数据库使用复制 Activity

Question

我正在尝试将 table 从数据库 A 迁移到同一 Azure SQL 服务器中的数据库 B。我使用 Azure 数据工厂成功地将数据从 Azure blob 存储迁移到 Azure 数据库，但我似乎不明白如何修改我的 ADF 代码以跨数据库迁移数据。

我已经学会了在 This Whosebug Link 的帮助下迁移数据，但我正在寻找迁移 table 使用复制 Activity。

我的 Azure Blob 到 Azure SQL 的管道如下。请建议将导致数据从一个 Azure SQL 数据库迁移到另一个数据库的修改。这是完整的数据工厂代码。我正在寻找一种解决方法或至少一些可以指导我的资源。提前致谢。

Azure SQL 链接服务

{
"name": "AzureSqlLinkedService",
"properties": {
    "description": "",
    "hubName": "dalete_hub",
    "type": "AzureSqlDatabase",
    "typeProperties": {
        "connectionString": "Data Source=tcp:server.database.windows.net,1433;Initial Catalog=DB;Integrated Security=False;User ID=login@server.database.windows.net;Password=**********;Connect Timeout=30;Encrypt=True"
    }
}

}

Azure 存储链接服务

{
"name": "AzureStorageLinkedService",
"properties": {
    "description": "",
    "hubName": "dalete_hub",
    "type": "AzureStorage",
    "typeProperties": {
        "connectionString": "DefaultEndpointsProtocol=https;AccountName=newstorageaccount;AccountKey=**********"
    }
}

}

输入数据集

   {
    "name": "InputDataset",
    "properties": {
        "structure": [
            {
                "name": "Region",
                "type": "String"
            },
            {
                "name": "Sales",
"type": "String"
            }
        ],
        "published": false,
        "type": "AzureBlob",
        "linkedServiceName": "AzureStorageLinkedService",
        "typeProperties": {
            "fileName": "data.txt",
            "folderPath": "adfpoc/",
            "format": {"type": "TextFormat",
                "columnDelimiter": ","
            }
        },
        "availability": {
            "frequency": "Hour",
            "interval": 1
        },
        "external": true,
        "policy": {}
    }
}

输出数据集

    {
    "name": "OutputDataset",
    "properties": {
        "structure": [
            {
                "name": "Region",
                "type": "String"
            },
            {
                "name": "Sales",
                "type": "String"
            }
        ],
        "published": false,
        "type": "AzureSqlTable",
        "linkedServiceName": "AzureSqlLinkedService",
        "typeProperties": {
            "tableName": "data"
        },
        "availability": {
            "frequency": "Hour",
            "interval": 1
        }
    }
}

ADFPipeline

  {
    "name": "ADFTutorialPipeline",
    "properties": {
        "description": "Copy data from a blob to Azure SQL table",
        "activities": [
            {
                "type": "Copy",
                "typeProperties": {
                    "source": {
                        "type": "BlobSource" },
                    "sink": {
                        "type": "SqlSink",
                        "writeBatchSize": 10000,
                        "writeBatchTimeout": "60.00:00:00"
                    }
                },
                "inputs": [
                    {
                        "name": "InputDataset"}
                ],
                "outputs": [
                    {
                        "name": "OutputDataset"
                    }
                ],
                "policy": {
                    "timeout": "01:00:00",
                    "concurrency": 1,
                    "executionPriorityOrder": "NewestFirst"
                },
                "scheduler": {
                    "frequency": "Hour",
                    "interval": 1
                },
                "name": "CopyFromBlobToSQL"
            }
        ],
        "start": "2019-03-11T00:00:00Z",
        "end": "2019-03-12T00:00:00Z",
        "isPaused": false,
        "hubName": "dalete_hub",
        "pipelineMode": "Scheduled"
    }
}

Answer 1

从 sql 数据库移动到另一个数据库与您已经完成的操作类似，但这次源将是 sql table，就像您之前的接收器一样（或输出）。

所以你应该做的是为新数据库创建一个新的链接服务，然后创建一个数据集作为输入（格式与你之前的输出相同，但更改链接服务名称以便它使用新的链接服务）。

最后，创建管道并正确配置输入和输出数据集。

希望对您有所帮助！

Answer 2

我尝试了与您相同的操作，并在数据工厂中成功迁移了我的 table。

例如，我在数据库dbleon中有一个table table1，我想用复制[=]将table1迁移到另一个数据库dbleon1 40=].

我在 dbmeon1 中创建了一个新的 table table1，它与 dbleon 中的 table1 具有相同的架构。

这是我的 ADF 代码：

{
    "name": "CopyPipeline_0oh",
    "properties": {
        "activities": [
            {
                "name": "Copy_0oh",
                "type": "Copy",
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [
                    {
                        "name": "Source",
                        "value": "[dbo].[table1]"
                    },
                    {
                        "name": "Destination",
                        "value": "[dbo].[table1]"
                    }
                ],
                "typeProperties": {
                    "source": {
                        "type": "SqlSource"
                    },
                    "sink": {
                        "type": "SqlSink",
                        "writeBatchSize": 10000
                    },
                    "enableStaging": false,
                    "translator": {
                        "type": "TabularTranslator",
                        "columnMappings": {
                            "id": "id",
                            "name": "name"
                        }
                    }
                },
                "inputs": [
                    {
                        "referenceName": "SourceDataset_0oh",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "DestinationDataset_0oh",
                        "type": "DatasetReference"
                    }
                ]
            }
        ]
    },
    "type": "Microsoft.DataFactory/factories/pipelines"
}

源数据集：

{
    "name": "SourceDataset_0oh",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureSqlDatabase1",
            "type": "LinkedServiceReference"
        },
        "type": "AzureSqlTable",
        "structure": [
            {
                "name": "id",
                "type": "Int32"
            },
            {
                "name": "name",
                "type": "String"
            }
        ],
        "typeProperties": {
            "tableName": "[dbo].[table1]"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

目标数据集：

{
    "name": "DestinationDataset_0oh",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureSqlDatabase2",
            "type": "LinkedServiceReference"
        },
        "type": "AzureSqlTable",
        "structure": [
            {
                "name": "id",
                "type": "Int32",
                "precision": 10
            },
            {
                "name": "name",
                "type": "String"
            }
        ],
        "typeProperties": {
            "tableName": "[dbo].[table1]"
        }
    },
    "type": "Microsoft.DataFactory/factories/datasets"
}

希望对您有所帮助。

Azure SQL 数据库：Transfer/Migrate Table 从一个 Azure SQL 数据库到另一个 Azure SQL 数据库使用复制 Activity

Azure SQL Database: Transfer/Migrate Table from one Azure SQL Database to Another Azure SQL Database using Copy Activity

azure

azure-storage

azure-data-factory

azure-sql-database