Datafactory Multi Relative URL expressions

Axell 66 Reputation points
2021-12-01T11:33:25.11+00:00

Hi!
I user RestApi action to load the data into DB.
As i have several Relative URLs with subfolders i create Datasets for all subfolders.
Can we create multi Relative URL for example on my screen?
I try to use Concat for it, but it doesn't work. I think it should be a loop for going through all positions.
Base URL: example.com/api
Relative
/moon/1
/moon/2
/moon/3
/sun/1(2,3,4,5)
/planet/1(,2,3,4,5
)

@markus.bohland@hotmail.de ('/moon/', string(dataset().URL_moon_page))
where URL_moon_page is array: [1,2,3,4,5,6]

154145-9.jpg

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
0 comments No comments
{count} votes

Answer accepted by question author
  1. ShaikMaheer-MSFT 38,556 Reputation points Microsoft Employee Moderator
    2021-12-02T15:58:13.677+00:00

    Hi @Axell ,

    Thank you for posting query in Microsoft Q&A Platform.

    You should parameterize REST dataset to dynamically pass your relative URLs.

    Kindly check below example, where I tried explain step by step implementation of your requirement.

    Step1: Have parameter in pipeline which is going to take all your relative URLs as array. While running pipeline you need to supply array of your relative URLs into this parameter.
    You should create a array of your relative URLs in below format to pass it in to this parameter while running pipeline.
    ["/moon/1","/sun/1","/planet/1"]
    154525-image.png

    Step2: ForEach activity to loop through over your "relativeURLs" array parameter items.
    154536-image.png

    Step3: Parameterize your REST dataset.
    154537-image.png

    Step4: Use Copy activity inside ForEach activity. Under Source tab select dataset created in step3.
    154514-image.png

    Hope this will help. Please let us know if any further queries.

    ---------------------

    Please consider hitting Accept Answer button. Accepted answers helps community as well. Thank you.


1 additional answer

Sort by: Most helpful
  1. Nasreen Akter 10,891 Reputation points Volunteer Moderator
    2021-12-01T15:51:07.503+00:00

    Hi @Axell ,

    In my understanding, I don't think you can pass a LIST in the ReplativeURL. You will need a ForLoop activity for this. What you can do, a) in the Dataset --> Parameters--> Add a new parameter e.g., testParam. b) then in the RelativeURL --> add dynamic content --> @dataset().testParam. c) Now pass the parameters from your pipeline when doing the loop. You can use Wildcard in your RelativeURL. Hope this will help. Thanks!


Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.