Hi @Glasier ,
Thank you for reaching out to us with your query.
As I understand that you are encountering a size limit error in Azure Data Factory (ADF) when transferring larger data using Azure Functions, please let me know if that is not.
Unfortunately, this is a hard limit on Web activity where the response size should be less than or equal to 4MB. Plese refer this documentation.
Azure Resource Manager has certain restrictions on the number of API calls you can make in a given period. Be sure to keep your call rate within these API limits. Additionally, the API schema used by a validation policy cannot exceed 4 MB in size. If the schema is larger than this, validation policies will encounter errors during runtime. For more details, please refer to this documentation: API Management policies to validate requests and responses.
And as a workaround you may try using ForEach Activity. Maybe you need to use paging query for your rest api/endpoint and return a limited number of the data each time. Then query your data in loop until the number of return data is lower than threshold value. Please refer to this source: Web activity throws overlimit error when calling rest api
Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.
Hope this helps. Do let us know if you any further queries.
If this answers your query, do click
Accept Answer and
Yes for was this answer helpful. And, if you have any further query do let us know.