Transfering .bak file from sftp_server to blob storage and storing as "PAGE BLOB", using Azure Data Factory

August Asheim Birkeland 20 Reputation points
2023-02-01T13:23:50.89+00:00

Hi,

Im trying to move a ~3GB .bak file from an ftp_server to azure blob storage using sftp. I want the transfer to happen automatically once a day, as a new .bak file with the same name is placed at the FTP_server every night.

I have looked into logical apps, azure functions and Azure Data Factory right now.

I have already managed to do the transfer with Azure Data Factory!! BUT, the file is stored as a Block Blob in the container, and it HAS TO BE a Page Blob in order for the virtual machine SQL server should be able to restore a database from it.

THEREFORE, how can i tell Azure DF to store the file as page blob?? here is my current pipeline script:

{
    "name": "pipeline1copy",
    "properties": {
        "activities": [
            {
                "name": "Copy data1",
                "type": "Copy",
                "dependsOn": [],
                "policy": {
                    "timeout": "0.12:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "source": {
                        "type": "BinarySource",
                        "storeSettings": {
                            "type": "SftpReadSettings",
                            "recursive": true,
                            "disableChunking": false
                        },
                        "formatSettings": {
                            "type": "BinaryReadSettings"
                        }
                    },
                    "sink": {
                        "type": "BinarySink",
                        "storeSettings": {
                            "type": "AzureBlobStorageWriteSettings",
                            "pageBlob": true
                        }
                    },
                    "enableStaging": false
                },
                "inputs": [
                    {
                        "referenceName": "Binary1",
                        "type": "DatasetReference"
                    }
                ],
                "outputs": [
                    {
                        "referenceName": "Binary2Sink",
                        "type": "DatasetReference"
                    }
                ]
            }
        ],
        "annotations": []
    }
}

Also, the json for the blob sink is like this:

{
    "name": "Binary2Sink",
    "properties": {
        "linkedServiceName": {
            "referenceName": "AzureBlobStorage1backupexcelinegeneralp",
            "type": "LinkedServiceReference"
        },
        "annotations": [],
        "type": "Binary",
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "fileName": "XXXX.bak",
                "container": "backup"
            },
            "blobType": "PageBlob"
        }
    }
}

none of the blobType parameters work.

Thanks in advance!

Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
2,286 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
9,151 questions
{count} votes

1 answer

Sort by: Most helpful
  1. BhargavaGunnam-MSFT 24,276 Reputation points Microsoft Employee
    2023-02-08T17:26:20.23+00:00

    Hello @August Asheim Birkeland ,

    Welcome to the MS Q&A platform.

    Please correct me if my understanding is wrong. You are using copy activity to copy .bak files from FTP server(source) to Azure blob storage(sink) and you wanted to copy the files as "Page blob"

    Azure blob storage connector supports copying blobs from block, append, or page blobs and copying data to only block blobs.

    So you can't copy them as "page blobs"

    Please see the below documentation link for your reference.

    https://learn.microsoft.com/en-us/azure/data-factory/connector-azure-blob-storage?tabs=data-factory

    User's image

    Please let us know if you have any further questions.

    0 comments No comments