c# - Improve the performance for enumerating files and folders using .NET

Question

I have a base directory that contains several thousand folders. Inside of these folders there can be between 1 and 20 subfolders that contains between 1 and 10 files. I'd like to delete all files that are over 60 days old. I was using the code below to get the list of files that I would have to delete:

DirectoryInfo dirInfo = new DirectoryInfo(myBaseDirectory);
FileInfo[] oldFiles = 
  dirInfo.GetFiles("*.*", SearchOption.AllDirectories)
    .Where(t=>t.CreationTime < DateTime.Now.AddDays(-60)).ToArray();

But I let this run for about 30 minutes and it still hasn't finished. I'm curious if anyone can see anyway that I could potentially improve the performance of the above line or if there is a different way I should be approaching this entirely for better performance? Suggestions?

Answer 1

This is (probably) as good as it's going to get:

DateTime sixtyLess = DateTime.Now.AddDays(-60);
DirectoryInfo dirInfo = new DirectoryInfo(myBaseDirectory);
FileInfo[] oldFiles = 
    dirInfo.EnumerateFiles("*.*", SearchOption.AllDirectories)
           .AsParallel()
           .Where(fi => fi.CreationTime < sixtyLess).ToArray();

Changes:

• Made the the 60 days less DateTime constant, and therefore less CPU load.
• Used EnumerateFiles.
• Made the query parallel.

Should run in a smaller amount of time (not sure how much smaller).

Here is another solution which might be faster or slower than the first, it depends on the data:

DateTime sixtyLess = DateTime.Now.AddDays(-60);
DirectoryInfo dirInfo = new DirectoryInfo(myBaseDirectory);
FileInfo[] oldFiles = 
     dirInfo.EnumerateDirectories()
            .AsParallel()
            .SelectMany(di => di.EnumerateFiles("*.*", SearchOption.AllDirectories)
                                .Where(fi => fi.CreationTime < sixtyLess))
            .ToArray();

Here it moves the parallelism to the main folder enumeration. Most of the changes from above apply too.