[Logic app] Download a file from from a website with xpath

Nguyen Thi VIet 31

Hi, I am creating a logic app to automatically download a file from a website.
However, the problem is the file changes every month like the following picture.

So I am thinking of using the following xpath to download the file and upload it into sharepoint.
/html/body/div/div/div4/div2/div/ul1/li2/a

I am currently using the following structure.

Can you please show me a way to download the file using xpath. Thank you in advance.

Nguyen Thi VIet 31 Reputation points

2022-02-15T07:28:17.87+00:00

@Mike Urnun @tbgangav-MSFT @MayankBargali-MSFT @Andrew Blumhardt @wwwbitcoincom-1919 @Orlando @craig raine @Michael Koerner @Andreas Baumgarten @Scott Azure RTOS @Azure Apprentice @Kevin Azure @AzureUser-9588 can you please answer this question.

1 answer

MayankBargali-MSFT 69,946 Reputation points

2022-02-17T03:43:18.793+00:00
@Nguyen Thi VIet Logic app does have expression function and you can refer to this document for more details.
We do have the XPath function that can help you with the use case but as XPath only accepts the XML string so you need to make sure that you are passing the XML string only.

If you view the source code of any page then you can see in most of the pages there might be <!doctype html> which will make your string as invalid XML when you will try to parse the string to XML using xml function.

Therefore you need to make sure that you are passing the string without <!doctype html> and you can use split method to split the string and pass the right value.
For your reference, I have defined the html body to my string variable and using xml and XPath math I am getting the array value. This is only for reference you can modify it as per your end.

Once you have the href URL you can leverage http connector to get the content and create the file in sharepoint.

Expression:

xpath(xml(variables('html')),'/html/body/a/@href')

Workflow:

Output:
Please sign in to rate this answer.
Nguyen Thi VIet 31 Reputation points

2022-02-17T07:27:28.973+00:00

@MayankBargali-MSFT in my case, a new file is uploaded to the site each month. That means with your method, I need to update the xml string every month. Is that correct?
If so, can you tell me another way to get the file?

MayankBargali-MSFT 69,946 Reputation points

2022-02-17T07:37:27.373+00:00

@Nguyen Thi VIet That is only for reference where I have hard-coded my page content. To get the page content you need to first call http connector. In the below workflow I am calling GET method on my URL i.e. google.com and then splitting so I can get the well formatted XML. In case if it is not well formatted content then you need to perform more operation to make the right XML content.

Workflow:

Note: If the rendering of page changes (need element added, tags changed, call that you are using to traverse change) then you need to update your xpath according to the new format.

In case if it didn't help you as there can the string that is returned couldn't be converted to the right format then you can either leverage the inline code to write your own code to traverse through the HTML content or create Azure function to offload this functionality. With whatever language you are comfortable with that is supported in the azure function you can write the code to get your desired output.

Nguyen Thi VIet 31 Reputation points

2022-02-17T09:21:00.433+00:00

@MayankBargali-MSFT thank you very much. The link I use it indeed not well formatted. You said I need to do more operations. Can you show me those operations?
And can you show me what to do after getting the xpath. Thank you very much for your help?

Nguyen Thi VIet 31 Reputation points

2022-02-21T03:22:57.377+00:00

@MayankBargali-MSFT sorry to bother you. Can you please show me what to do after I get the xpath? Thank you very much

Nguyen Thi VIet 31 Reputation points

2022-02-21T08:30:37.377+00:00

@MayankBargali-MSFT you said that I need to "leverage the inline code to write your own code to traverse through the HTML content or create Azure function to offload this functionality". Can you show me how to do it?

Can you also show me what to do next after getting the xpat?

I am really new to this and do have any clue. Please answer question.

MayankBargali-MSFT 69,946 Reputation points

2022-02-22T10:55:01.827+00:00

@Nguyen Thi VIet It would not be possible to share what would be the action needed as there are different control actions that you can leverage in your workflow without having the sample body which you are expecting from your webpage. Further I will not suggest to implement for complex object as it may need multiple action.

The inline code would not help as DOMParser is not available and you cannot use require(). The best option would be offload it to azure function in any of the supported languages. To call function app from logic app you can refer to this document.

Sharing third party article where you can use HtmlAgilityPack.NetCore nuget package to perform the operation on DOM object. Please let me know in which language you are comfortable with so I research and share the reference article which can help you with your use case.

Nguyen Thi VIet 31 Reputation points

2022-02-22T11:29:37.563+00:00

@MayankBargali-MSFT Thank you very much for answering. I am currently using Python. Can you show me some articles relating to this?
Sign in to comment

Share via

[Logic app] Download a file from from a website with xpath

1 answer