Passing an object as a parameter to a Data Factory pipeline

Question

Hi Team,

I'm trying to use dynamic mapping in a pipeline Copy activity. I want to read the mapping from a SQL database using a Lookup, then send it to the pipeline.

I've created the pipeline and a parameter of type Object, and it works when I manually paste in the string below (Working String) as a value for the for the parameter.

If I then create another pipeline with an activity to call this pipeline and paste in the same string for the parameter value, it doesn't work. When I examine the sub-pipeline's input, I can see that all the double-quotes have been excaped with a backslash, which then doesn't seem to parse correctly. I get the same results if I try and generate my string programatically via a Lookup, either as an acitivty in the same pipeline, or from a calling pipeline.

I've also tried generating the string with | instead of ", then in the mapping section of my Copy activity, using a replace to turn them all back to ", but this doesn't work either.

Is there a workaround?

Thanks

Damian.

Working String
{ "type": "TabularTranslator", "mappings": [{ "source": { "path": "['ExternalID']" }, "sink": { "name": "ExternalID", "type": "String" } }, { "source": { "path": "['Name']" }, "sink": { "name": "Name", "type": "String" } }, { "source": { "path": "['Description']" }, "sink": { "name": "Description", "type": "String" } }, { "source": { "path": "['SessionType']" }, "sink": { "name": "SessionType", "type": "String" } }, { "source": { "path": "['StartDateTime']" }, "sink": { "name": "StartDateTime", "type": "DateTime" } }, { "source": { "path": "['EndDateTime']" }, "sink": { "name": "EndDateTime", "type": "DateTime" } }, { "source": { "path": "['ExpectedMark']" }, "sink": { "name": "ExpectedMark", "type": "String" } }, { "source": { "path": "['Organiser']" }, "sink": { "name": "Organiser", "type": "String" } }, { "source": { "path": "$['OrganisationID']" }, "sink": { "name": "OrganisationID", "type": "String" } }, { "source": { "path": "['Tags']" }, "sink": { "name": "Tags", "type": "String" } }], "collectionReference": "$['value']", "mapComplexValuesToString": true }

Answer 1

Hi @Williams, Damian (Barrachd) ,

I looked at it closely and observed that the when output from lookup is passed it is not able to treat JSON correctly. I have used below casting to the parameter in mapping section that enabled copy activity to read the value correctly as shown below.

@json(string(pipeline().parameters.mapping))  

Below is GIF showing the details

Please suggest if this addresses the requirement, let us know for further queries and we will be glad to assist.

---------------

• Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer. Here is how.
• Want a reminder to come back and check responses? Here is how to subscribe to a notification.

Answer 2

Hi Can you please let me know your source and sink setting of copy activity here.
Also can I pass append variable as copy activity source.?

Answer 3

Source and Sink were both SQL Server databases, with parameterised table names. I haven't tried using a variable as the source, but it should work if you put parameters on the Linked Service / Dataset and populate them using the variable.