Taking too much time to Load all files from the folder in ASP.NET Web page.

Question

I am loading the files from the server folder in a web page.

But, the head ache is, it's taking too much time to load all the files from the specific folder.

Hardly it will be taking 30 to 45 mins to load all the files.

End user can't wait till load all the files to take these much time.

pls. give us the suggestion to load all the files within short span of time.

Below is the code

protected void Page_Load(object sender, EventArgs e)
        {

            string path = @"\\SVRUS1XX\USA\MPS\TEK\STD REMOTE\1.INITIAL\">FIRST TAKE";
            RenderFolderContents(path);

        }

        public void RenderFolderContents(string FolderPath)
        {
            FileList.DataSource = GetfilesAndFolder(FolderPath);
            FileList.DataBind();
        }

        private string[] GetfilesAndFolder(string path)
        {
            return Directory.GetFiles(path, "*.*", SearchOption.AllDirectories);
        }

        protected void FileLinkButton_Click(object sender, EventArgs e)
        {
            string path = ((LinkButton)sender).Text;
            Response.ContentType = "application/octet-stream";
            Response.AddHeader("Content-Disposition", string.Format("attachment; filename={0}", Path.GetFileName(path)));
            Response.TransmitFile(path);

            Response.End();
        }

Answer 1

As noted, you mentioned about about 30 folders, and 6 files inside? That should not take any time at all.

However, keep in mind that you can (and should) populate the child folders "on demand". That way, response times should be instant.

I don't really have a extensive folder with files in it to test. However, if I right click on my source code folder with many projects?

(I even surprised me!!!)

I see this:

So, we have 36,000 files, and 6,000+ folders.

however, if we populate say a web based treeview on demand?

I have this markup:

            <asp:Label ID="lblFolder" runat="server" Text="" 
                style="float:left">
            </asp:Label>
            <asp:Label ID="lblLoadTime" runat="server" Text="" 
                style="float:left;margin-left:40px">
            </asp:Label>
            <br />


            <asp:TreeView ID="TreeView1" runat="server" ImageSet="XPFileExplorer"
                NodeIndent="15" OnTreeNodeExpanded="TreeView1_TreeNodeExpanded">
                <HoverNodeStyle Font-Underline="True" ForeColor="#6666AA" />
                <NodeStyle Font-Names="Tahoma" Font-Size="8pt" ForeColor="Black"
                    HorizontalPadding="2px" NodeSpacing="0px" VerticalPadding="2px" />
                <ParentNodeStyle Font-Bold="False" />
                <SelectedNodeStyle BackColor="#B5B5B5" Font-Underline="False"
                    HorizontalPadding="0px" VerticalPadding="0px" />
            </asp:TreeView>

So, I display the starting folder, and also the time in seconds to load these folders.

And I'm on a laptop - nothing fancy or fast.

Note the time to load is WELL under 1/10th of a second!!!

So, I get/see this:

Note how MOST of the time we can't even get a clock tick (milliseconds) to display in above. In other words, even being a web page, response time is 100% absolute instant - I mean instant!!!

the reason of course is I don't load nor populate the files UNTILL the user clicks on the treeview folder. So, that means we only ever loading a "limited" set of files.

So, the code for above looks like this:

        string sRoot = @"C:\Users\Kalla\source\repos";
        public class MyFolder
        {
            public FileInfo[] MyFiles;
            public DirectoryInfo[] MyFolders;
        }
        protected void Page_Load(object sender, EventArgs e)
        {
            if (!IsPostBack)
            {
                DateTime TimeStart = DateTime.Now;
                lblFolder.Text = sRoot;
                MyFolder MyFiles = GetFiles(sRoot);
                LoadTreeFiles(sRoot, MyFiles, "", null);

                DateTime TimeDone = DateTime.Now;
                lblLoadTime.Text =
                    $"Time to load files = {(TimeDone - TimeStart).TotalSeconds.ToString()}";
            }
        }

        public MyFolder GetFiles(string sRoot)

        {
            DirectoryInfo MyDir = new DirectoryInfo(sRoot); 
            MyFolder cResultFolder= new MyFolder();

            cResultFolder.MyFolders = MyDir.GetDirectories("*.*", SearchOption.TopDirectoryOnly);
            cResultFolder.MyFiles = MyDir.GetFiles("*.*", SearchOption.TopDirectoryOnly);

            return cResultFolder;
        }

And 2 more routines.

        public void LoadTreeFiles(string fRootL, 
                                MyFolder lParent, 
                                string sParentID,
                                TreeNode tTreeNode)
        {
            // get all folders (if any)
            foreach (DirectoryInfo sFolder in lParent.MyFolders)
            {
                TreeNode child = new TreeNode();
                child.Text = sFolder.Name;
                child.Value = sFolder.FullName;
                child.Expanded = false;
                child.PopulateOnDemand = true;
                child.ShowCheckBox = false;

                if (sParentID == "")
                    TreeView1.Nodes.Add(child);
                else
                    tTreeNode.ChildNodes.Add(child);
            }
            // get all files(if any)
            foreach (FileInfo sFile in lParent.MyFiles)
            {
                TreeNode childF = new TreeNode();
                childF.Value = sFile.Name;
                childF.Value = sFile.FullName;
                childF.Expanded = false;
                childF.ShowCheckBox = true;
                childF.PopulateOnDemand = false;
                if (sParentID == "")
                    TreeView1.Nodes.Add(childF);
                else
                    tTreeNode.ChildNodes.Add(childF);
            }
        }

        protected void TreeView1_TreeNodeExpanded(object sender, TreeNodeEventArgs e)
        {
            TreeNode child = e.Node;
            MyFolder dtChild = GetFiles(child.Value);

            // DateTime TimeStart = DateTime.Now;
            LoadTreeFiles(child.Value, dtChild, child.Text, child);

            //DateTime TimeDone = DateTime.Now;
            //lblLoadTime.Text =
            //    $"Time to load files = {(TimeDone - TimeStart).TotalSeconds.ToString()}";

        }

I should also note that this code is NOT recursive, and in fact this is not really a recursive problem or task. You would ONLY need a recursive routine if you wanted a expand all button - but as noted, I don't recommend that approach anyway.

Now, as noted, you still NOT really mentioned how many files you are dealing with. (you noted 30 folders - but that not sufficient information here).

However, as above shows, even with 36,000 files, I can't even get the time span to register 0.01 (1/1000th) of a second anyway!

I suspect that some kind of VPN or other issue is at play here for your routines, but using the above approach that populates the treeview "on demand", then we never really loading many files/nodes into the treeview anyway.

with the correct approach, your code and response time(s) should be WELL under one second with little effort.

Answer 2

Hi @Gani_tpt,

I think you can use EnumerateFiles instead, as the docs say: EnumerateFiles can be more efficient when you're dealing with many files and directories.

EnumerateFiles example

Is it possible to download the selected child node files (Not folder) from the treeview...?

It is possible. You can set the TreeView property ShowCheckBoxes to "Leaf" and then get the filename and filelength.

<asp:TreeView ID="TreeView1" runat="server" ImageSet="XPFileExplorer" NodeIndent="15"
        ShowCheckBoxes="Leaf">

 protected void Save(object sender, EventArgs e)
        {
            string fileName = "";
            string fileLength = "";
            foreach (TreeNode node in TreeView1.CheckedNodes)
            {

                FileInfo file = new FileInfo(node.Value);
                fileName = file.Name;
                fileLength = file.Length.ToString();
            }
            var mimeType = MimeMapping.GetMimeMapping(fileName);
            Response.Clear();
            Response.AddHeader("Content-Disposition", "attachment; filename=" + fileName);
            Response.AddHeader("Content-Length", fileLength);
            Response.ContentType = mimeType;
            Response.Flush();
            Response.WriteFile(fileName);
            Response.End();

        }

Best regards,
Lan Huang

---

If the answer is the right solution, please click "Accept Answer" and kindly upvote it. If you have extra questions about this answer, please click "Comment".
Note: Please follow the steps in our documentation to enable e-mail notifications if you want to receive the related email notification for this thread.