Transfering .bak file from sftp_server to blob storage and storing as "PAGE BLOB", using Azure Data Factory

Question

Hi,

Im trying to move a ~3GB .bak file from an ftp_server to azure blob storage using sftp. I want the transfer to happen automatically once a day, as a new .bak file with the same name is placed at the FTP_server every night.

I have looked into logical apps, azure functions and Azure Data Factory right now.

I have already managed to do the transfer with Azure Data Factory!! BUT, the file is stored as a Block Blob in the container, and it HAS TO BE a Page Blob in order for the virtual machine SQL server should be able to restore a database from it.

THEREFORE, how can i tell Azure DF to store the file as page blob?? here is my current pipeline script:

{
    "name": "pipeline1copy",
    "properties": {
        "activities": [
            {
                "name": "Copy data1",
                "type": "Copy",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "source": {
                        "type": "BinarySource",
                        "storeSettings": {
                            "type": "SftpReadSettings",
                            "recursive": true,
                            "disableChunking": false
                        },
                        "formatSettings": {
                            "type": "BinaryReadSettings"
                        }
                    },
                    "sink": {
                        "type": "BinarySink",
                        "storeSettings": {
                            "type": "AzureBlobStorageWriteSettings",
                            "pageBlob": true
                        }
                    },
                    "enableStaging": false
                },
                "inputs": [
                    {
                        "referenceName": "Binary1",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "Binary2Sink",
                        "type": "DatasetReference"
                    }
                ]
            }
        ],
        "annotations": []
    }
}

Also, the json for the blob sink is like this:

{
    "name": "Binary2Sink",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureBlobStorage1backupexcelinegeneralp",
            "type": "LinkedServiceReference"
        },
        "annotations": [],
        "type": "Binary",
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "fileName": "XXXX.bak",
                "container": "backup"
            },
            "blobType": "PageBlob"
        }
    }
}

none of the blobType parameters work.

Thanks in advance!

Answer 1

Hello @August Asheim Birkeland ,

Welcome to the MS Q&A platform.

Please correct me if my understanding is wrong. You are using copy activity to copy .bak files from FTP server(source) to Azure blob storage(sink) and you wanted to copy the files as "Page blob"

Azure blob storage connector supports copying blobs from block, append, or page blobs and copying data to only block blobs.

So you can't copy them as "page blobs"

Please see the below documentation link for your reference.

https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage?tabs=data-factory

Please let us know if you have any further questions.