Table Storage Dynamic lookup query from ADF

Max T 21 Reputation points
2021-08-31T14:19:04.907+00:00

Hello everybody.

I'm trying to setup an ADF Pipeline that 'explodes' data from an Azure Table Storage to a file system creating csv files with dynamic names based on the Partition Key value of the table.

I have on the left side a list of devices taken from a SQL Azure DB (Now they're 16 but in real-life will be thousands) and for each device I need to create every day a CSV.

I've configured a copy pipeline like this
127974-image.png

and set a TableStorage Source inside the FOR EACH cycle
127870-image.png

and everything seems to work well except the fact that the filter expression of the TABLESTORAGE provide a strange error:

The function 'length' expects its parameter to be an array or a string. The provided value is of type 'Object'.

127983-image.png

I've tried several ways to format the input string, all using or not the Dynamic Content to parse it but I always obtain the same error.

PartitionKey eq '{@item().deviceserial}' and Timestamp gt datetime'2021-02-01T00:00:00' and Timestamp le datetime'2021-02-02T00:00:00'
or
PartitionKey eq @item().deviceserial and Timestamp gt datetime'2021-02-01T00:00:00' and Timestamp le datetime'2021-02-02T00:00:00'

but nothing changes.

Where am I wrong ?
Thanks
Max

Azure Table Storage
Azure Table Storage
An Azure service that stores structured NoSQL data in the cloud.
171 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,990 questions
{count} votes

Accepted answer
  1. ShaikMaheer-MSFT 38,466 Reputation points Microsoft Employee
    2021-09-08T17:26:02.533+00:00

    Hi @Max T ,

    Thank you for posting query on Microsoft Q&A and thank you for sharing your pipeline json.

    I see small errors in your pipeline json.

    It seems you are passing @activity('lkDeviceList').output to items field in ForEach activity. You should pass @activity('lkDeviceList').output.value there.

    Could you please update your configurations accordingly and see if that helps.

    Hope this will help. Please let us know if any further queries. Thank you.

    -------------------

    • Please don't forget to click on Accept Answer button 130286-image.png whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
    1 person found this answer helpful.

0 additional answers

Sort by: Most helpful

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.