Hi @Pragiri Greetings! Welcome to Microsoft Q&A forum. Thank you for posting this question here.
With the current connectors and actions available in Logic App, there is no direct approach to parse the CSV file. However, you can manipulate the data from the CSV file and break it into line arrays and process them as JSON data using the approach provided in the following similar question
You can then connect to SQL database and execute SQL queries using the steps provided in the article Connect to an SQL database from workflows in Azure Logic Apps
I would recommend using an Azure function such as a Azure blob triggered function which provides more capabilities on parsing the CSV file and connecting to SQL database directly.
Let us know if you have any additional questions on need further assistance.
If the response helped, please do click Accept Answer and Yes for the answer provided. Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.