Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
How to use the continuationToken from Rest API of Table Storage in web activity while using Azure Data Factory?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 11%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHG%3C/text%3E%3C/svg%3E)
Hemanath Goud Kandula
1
Reputation point
I recently used the web activity to get the list of table names from table storage using Rest API.
I didn't get the list of tables due to exceeding the limit of having more than 1000 tables.
I found one alternative solution to one of the suitable questions in my case which is the below steps. I tried to implement these steps but didn't get any luck.
Step 1 : Initialize a variable ContinuationToken as blank**
Step 2 :** Perform Copy activity with the ContinuationToken value (with merge behavior)**
Step 3 :** Perform Web activity
Step 4 : Set the variable ContinuationToken with continuation token obtained from the Web activity.
Step 5 : Repeat steps 2-4 using the Until activity - until the continuation token is not returned.
Can someone help me implement the above steps with gifs or images so that I can implement them in my case?
Azure Data Factory
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 38%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EQ%3C/text%3E%3C/svg%3E)
QuantumCache 20,366 Reputation points Moderator2024-02-12T20:06:47.8233333+00:00 Hemanath Goud Kandula I hope you have already seen this document: Query timeout and pagination
I didn't get the list of tables due to exceeding the limit of having more than 1000 tables.
Could you please add more info on the above statement? Are you seeing any errors? please add sample images for your pipeline on this question, so that we understand any errors and provide better suggestions on your use case.
A query against the Table service may return a maximum of 1,000 items at one time and may execute for a maximum of five seconds. If the result set contains more than 1,000 items, if the query did not complete within five seconds, or if the query crosses the partition boundary, the response includes headers which provide the developer with continuation tokens to use in order to resume the query at the next item in the result set.
-
Hemanath Goud Kandula 1 Reputation point
2024-02-13T04:21:35.1266667+00:00 Hi Satish, I don't see any errors while executing the pipeline. In the response, I have x-ms-continuation-NextTableName value and I don't exactly how to use it and get the remaining table names with continuation token.
-
QuantumCache 20,366 Reputation points Moderator
2024-02-14T02:50:08.6733333+00:00 Hello @Hemanath Goud Kandula Please make use of the
NextTableNameDid you try the Param in subsequent WEB Activity calls?The Header value will not work for this, as we have to pass the continuation token as a param. First Web Activity i got the 1000 tables list. Then in the second Web activity i got the next 1000 tables list. Hope this is helpful
-
QuantumCache 20,366 Reputation points Moderator
2024-02-21T04:17:03.9266667+00:00 Hello @Hemanath Goud Kandula Just checking if we are still connected on this discussion? Please let us know if you need to add more info so that we better assist you! If the Response is helpful, then please click "Accept Answer" . So that we can close this thread.
Sign in to comment
Sign in to answer