Using ASP.NET Ccore MVC to extract data from csv file

Question

Hey Guys,
I'm looking for some suggestions on the most efficient ways to extract data(based on search criterias) from a very large CSV file, using ASP.NET Core MVC.

Thanks....

Answer 1

Hi

If your dataset is large, then you can try streaming the data to the user. Even if your dataset is larger, the streaming uses the same memory.
You can get the bytes of the Excel spreadsheet, or simply convert your data to a CSV, and then set the HTTP Response Type and stream it to the user.

sample:

byte[] reportDoc = GetExportExcel();  
context.Response.ContentType = "application/vnd.ms-excel";  
  
//set the content disposition header to force download  
context.Response.AddHeader("Content-Disposition", "attachment;filename=" +  
                    "Export.xls");  
  
//write the file content byte array  
context.Response.BinaryWrite(reportDoc);  

Answer 2

Hi @donny ,

First, to extract data from csv file, you can refer to the following code to upload the csv file to the file folder, then extract the data.

and the view page like this:

You can view the page source from here: 217601-sourcecode.txt

I fear that with the file being so large, over, 1 million rows, reading row by row, can make the application very slow.

Second, if the extract data action spend too much time, you could consider create a background service to handle this action. Upload the csv file to the folder fist, then tell the background service to extract the data from the file.

Refer to this article: Background tasks with hosted services in ASP.NET Core

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.

Best regards,
Dillion