Azure Data Factory Zendesk Connector Missing Tickets

Tomasz Swinarski 0

Hello,

We are trying to integrate Zendesk data into our Azure SQL DB using Azure Data Factory.

I am using Zendesk connector available in ADF to load tickets (Mapping Data Flow).

Tickets are loaded to sink but to my surprise not all of them - just fraction of all of the tickets available in this particular Zendesk instance.

The earliest creation date month of the tickets loaded to sink is July 2022 but of course there are thousands of tickets created in Zendesk before that. Moreover July 2022 and August 2022 tickets are mostly missing as well for example for 16th August 2022 there are more than 100 hundred tickets created in zendesk but only one has been loaded to sink using ADF... Tickets created in January 2023 have all been loaded fine.

I would be grateful for any help you can provide.

ShaikMaheer-MSFT 37,971 Reputation points Microsoft Employee

2023-01-12T09:04:28.13+00:00

Hi Tomasz Swinarski,

Thank you for posting query in Microsoft Q&A Platform.

Which entity you are querying here? I hope its tickets entity, right? Could you please manually check in that entity, are we really have data in it?
Also, if any configurations or implementation you did in dataflow, which is filtering data in anyway? Kindly share implementation details along with screenshots to understand better. Thank you.
Tomasz Swinarski 0 Reputation points

2023-01-12T10:57:40.4733333+00:00

Hello,

Exactly tickets entity. I have searched for the tickets directly in Zendesk using creation date:

It returned tickets from April 2020 up until today in total around 150 000 tickets whereas using zendesk connector in mapping data flow (without any filters/transformations) I got around 16 000.

Flow is very simple:

Source:

Sink:

There is nothing set up additionally just Zendesk dataset using Zendesk linked service.
ShaikMaheer-MSFT 37,971 Reputation points Microsoft Employee

2023-01-13T07:01:40.9966667+00:00

Hi Tomasz Swinarski,

Hope you selected, tickets under source options. If possible, could you please share your activity runid or pipeline runid? so that I can try to check more with internal team on this if they have any clue. Will update if any updates. Thank you.
Tomasz Swinarski 0 Reputation points

2023-01-16T11:21:31.0766667+00:00

Hello @ShaikMaheer-MSFT

Yes I have selected it. I have re-run pipeline here is pipeline runid XXXXXXXXXXXXX(masked it)

It looks like older tickets are being cut out for some reason - there are even fewer ticket now loaded from July/August 2022. Total number of tickets stays around 16k.
ShaikMaheer-MSFT 37,971 Reputation points Microsoft Employee

2023-01-17T09:18:17.1666667+00:00

Hi Tomasz Swinarski,

I checked internally. This needs more deeper investigation to understand why its happening, I would encourage you to have support ticket for this. So that support team can help you by doing deep investigation.
Kindly check below link for creating support ticket.

[https://azure.microsoft.com/en-us/support/create-ticket
Tomasz Swinarski 0 Reputation points

2023-01-18T12:00:07.59+00:00

Hello @ShaikMaheer-MSFT

Thanks for taking your time. We have raised support ticket internally.

BR

Tomek