Hello @Michael Giusto ,
Thanks for the question and using MS Q&A platform.
As I understand the overall goal is to have a process that:
- Determines what files on SFTP need to be processed
- Copies from SFTP to Blob storage
- inserts a record into table storage confirming upload state, and other metadata
- Copies from Blob storage to Azure SQL
- updates the earlier record to confirm the data is in SQL
Currently you are using both Logic App and Data Factory. I am confused why you are using both services, wouldn't it be easier to put everything in one or the other?
it's becoming cumbersome creating a new Logic-App for every new file I need to work with.
This would be solved by step 0. I need to know more how you choose files. Is it by file modified date on the SFTP or something else?
Please do let me if you have any queries.
- Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
- Want a reminder to come back and check responses? Here is how to subscribe to a notification