Bewerken

Delen via

Copy job definition

  • Artikel

This article provides a breakdown of the definition structure for Copy job items.

Definition parts

This table lists the definition parts.

Definition part path type Required Description
copyjob-content.json ContentDetails (JSON) true Describes properties and settings of the item like source and destination
.platform PlatformDetails (JSON) false Describes common details of the item

ContentDetails

Describes content of payload

Name Type Description
properties CopyJobProperties CopyJob properties.
activities CopyJobActivity[] List of CopyJob activities.

Description for CopyJobProperties contents

Describes the fields used to construct the properties

Name Type Required Description
jobMode String (Enum) true Describes the job mode with two possible values: Batch and CDC(Incremental).
source CopyJobConnection false Describes the source.
destination CopyJobConnection false Describes the destination.
policy CopyJobPolicy false Describes the policy for the copy job like timeout duration.

Description for CopyJobConnection contents

Name Type Required Description
type String true Describes the type of data source or destination.
connectionSettings CopyJobConnectionSettings true Describes the connection settings for the endpoint.

Description for CopyJobConnectionSettings contents

Describes the fields for Connection Settings. Here, depending on the type, either typeProperties or externalReferences or both need to be mentioned.

Name Type Required Description
type String true Describes the type of connection.
typeProperties CopyJobConnectionTypeProperties false Describes the properties for the connection.
externalReferences ExternalReference false Describes the external reference for the connection.

Description for CopyJobConnectionTypeProperties contents

Name Type Required Description
schema String false Specifies the schema.
database String false Name of the database.
workspaceId String (Guid) false Specifies the Id for the workspace in which the connected item exists.
artifactId String (Guid) false Specifies the Id for the connected item.
rootFolder String false Specifies the root folder.
endpoint String false Specifies the endpoint.

Description for ExternalReferences contents

Name Type Required Description
connection String (Guid) true Specifies the Id of the connection.

Description of CopyJobPolicy contents

Name Type Required Description
timeout String true Specifies the timeout duration. Follows the pattern: ((\\d+)\\.)?(\\d\\d):(60\|(\[0-5\]\[0-9\])):(60\|(\[0-5\]\[0-9\])) .

Description for CopyJobActivity contents

Describes the fields used to construct the properties

Name Type Required Description
properties CopyJobActivityProperties true Describes the activity properties.
id String (Guid) false The activity id.

Description for CopyJobActivityProperties contents

Describes the fields used to construct the properties

Name Type Required Description
source CopyJobActivitySource true Describes the source settings.
destination CopyJobActivityDestination true Describes the destination settings.
translator CopyJobActivityTranslator true Describes the translator settings.
typeConversionSettings CopyJobActivityMappingTypeConversionSettings true Describes the type conversion settings.
enableStaging Boolean false Specifies whether staging is enabled or not.

Description for CopyJobActivitySource contents

Name Type Required Description
exportSettings CopyJobExportSettings false Describes the export settings.
datasetSettings CopyJobDatasetSettings false Describes the dataset settings.
changeDataSettings CopyJobChangeDataSettings false Describes the change data settings.
storeSettings CopyJobStoreSettings false Describes the store settings.
formatSettings CopyJobFormatSettings false Describes the format settings.
noTruncation Boolean false Specifies if truncation is allowed or not.

Description for CopyJobActivityDestination contents

Name Type Required Description
tableOption String false Specifies the option for table related action.
upsertSettings SqlUpsertSettings false Describes the upsert settings.
partitionOption String false Specifies the partition mechanism for writing to destination.
partitionNameList String[] false Specifies the list of keys/columns to be used for partition mechanism.
writeBehaviour String false Specifies the write behvaiour like Append or Overwrite.
importSettings CopyJobImportSettings false Describes the import settings.
datasetSettings CopyJobDatasetSettings false Describes the dataset settings.
storeSettings CopyJobStoreSettings false Describes the store settings.
formatSettings CopyJobFormatSettings false Describes the format settings.
noTruncation Boolean false Specifies if truncation is allowed or not.

Description for CopyJobExportSettings contents

Name Type Required Description
type String false Describes the type of export settings.

Description for CopyJobDatasetSettings contents

Name Type Required Description
Schema String false Describes the schema.
table String false Specifies the table name.
Location CopyJobDatasetSettingsLocation false Describes the location.
Compression CopyJobDatasetSettingsCompressionSettings false Describes the compression settings.
columnDelimiter String false Specifies the column delimiter.
rowDelimiter String false Specifies the row delimiter.
escapeChar String false Specifies the escape character.
firstRowAsHeader Boolean false Specifies if the first row will be used as header or not.
quoteChar String false Specifies the quote character.
encodingName String false Specifies the encoding.
compressionCodec String false Specifies the compression codec.
compressionLevel String false Specifies the compression level.
objectApiName String false Specifies the object api name.
reportId String false Specifies the report Id in case of Salesforce type connection.

Description for CopyJobDatasetSettingsLocation contents

Name Type Required Description
type String false Describes the type of location.
fileName String false Specifies the file Name.
folderPath String false The Folder Path.
container String false Container name.
fileSystem String false Name of file system.
bucketName String false Bucket Name.

Description for CopyJobDatasetSettingsCompressionSettings contents

Name Type Required Description
type String false Describes the compression type.
level String false Decribes the compression level.

Description for CopyJobChangeDataSettings contents

Name Type Required Description
ReadMethod String false Specifies the read method.
Columns CopyJobChangeDataColumn[] false Specifies the list of change data columns.

Description for CopyJobChangeDataColumn contents

Name Type Required Description
name String false Specifies the name of the column.
type String false Specifies the type of the column.
physicalType String false Specifies the physical type of the column.
length String false Specifies the length of the column.
scale Integer false Specifies the scale of the column.
precision Integer false Specifies the precision of the column.

Description for CopyJobStoreSettings contents

Name Type Required Description
type String false Specifies the type of store setting.
recursive Boolean false Specifies whether it is recursive.
prefix String false Specifies the prefix string to be used.
WildcardFolderPath String false Specifies the wildcard folder path.
wildcardFileName String false Specifies the wildcard file name.
fileListPath String false Specifies the file list path.
modifiedDatetimeStart String false Specifies the start of the modified datetime range.
modifiedDatetimeEnd String false Specifies the end of the modified datetime range.
enablePartitionDiscovery Boolean false Specifies whether partition discovery is enabled.
copyBehavior String false Specifies the copy behavior.

Description for CopyJobFormatSettings contents

Name Type Required Description
type String false Specifies the type of format setting.
fileExtension String false Specifies the file extension.
quoteAllText Boolean false Specifies whether to quote all text.
enableVertiParquet Boolean false Specifies whether VertiParquet is enabled.

Description for CopyJobSqlUpsertSettings contents

Name Type Required Description
useTempDB Boolean false Specifies whether to use temp db for upsert interim table.
interimSchemaName String false Schema name for interim table.
keys String[] false Key column names for unique row identification.

Description for CopyJobImportSettings contents

Name Type Required Description
type String false Describes the type of import settings.

Description for CopyJobActivityTranslator contents

Name Type Required Description
type String false Specifies the type of the translator.
mappings CopyJobActivityMapping[] false Specifies the list of activity mappings.

Description for CopyJobActivityMapping contents

Name Type Required Description
source CopyJobActivityMappingColumn false Specifies the source column mapping.
destination CopyJobActivityMappingColumn false Specifies the destination column mapping.

Description for CopyJobActivityMappingColumn contents

Name Type Required Description
name String false Specifies the name of the column.
type String false Specifies the type of the column.
physicalType String false Specifies the physical type of the column.
length String (Integer) false Specifies the length of the column.
scale Integer false Specifies the scale of the column.
precision Integer false Specifies the precision of the column.

Description for CopyJobActivityMappingTypeConversionSettings contents

Describes the fields used to construct the properties

Name Type Required Description
typeConversion CopyJobActivityMappingTypeConversion true Describes the type conversion properties.

Description for CopyJobActivityMappingTypeConversion contents

Describes the fields used to construct the properties

Name Type Required Description
allowDataTruncation Boolean false Specifies whether data truncation is allowed.
treatBooleanAsNumber Boolean false Specifies whether to treat boolean as a number.

ContentDetails Example 1

{ 
  "properties": { 
    "jobMode": "Batch" 
  }, 
  "activities": [] 
}

ContentDetails Example 2

{
  "properties": { 
    "jobMode": "Batch", 
    "source": { 
      "type": "LakehouseTable", 
      "connectionSettings": { 
        "type": "Lakehouse", 
        "typeProperties": { 
          "workspaceId": "00000000-0000-0000-0000-000000000000", 
          "artifactId": "aaaaaaaa-6666-7777-8888-bbbbbbbbbbbb", 
          "rootFolder": "Tables" 
        } 
      } 
    }, 
    "destination": { 
      "type": "LakehouseTable", 
      "connectionSettings": { 
        "type": "Lakehouse", 
        "typeProperties": { 
          "workspaceId": "00000000-0000-0000-0000-000000000000", 
          "artifactId": "aaaaaaaa-0000-1111-2222-bbbbbbbbbbbb", 
          "rootFolder": "Tables" 
        } 
      } 
    }, 
    "policy": { 
      "timeout": "0.12:00:00" 
    } 
  }, 
  "activities": [ 
    { 
      "id": "eeeeeeee-4444-5555-6666-ffffffffffff", 
      "properties": { 
        "source": { 
          "datasetSettings": { 
            "table": "publicholidays", 
            "firstRowAsHeader": true 
          } 
        }, 
        "destination": { 
          "writeBehavior": "Append", 
          "datasetSettings": { 
            "table": "publicholidays", 
            "firstRowAsHeader": false 
          } 
        }, 
        "translator": { 
          "type": "TabularTranslator" 
        }, 
        "typeConversionSettings": { 
          "typeConversion": { 
            "allowDataTruncation": true, 
            "treatBooleanAsNumber": false 
          } 
        } 
      } 
    } 
  ] 
}