Using LINQPad , PLINQ to grep for files

I use linqpad as my primary dev tool for iterative code development. Today I had to search the source code tree on my hard disk for certain keywords that were present in database. There is big impedance mismatch between database and the rest of the world. I would have to fetch the data from DB and had to use either powershell or cmd to look for them. This is fine if I have to look for one keyword the issue if I have few of them then it becomes a bigger issue.  I remember looking at a cool example from PFX about using parallel grep. Here is the code I modified it to include optional parameters and return a type


public class GrepResult
 {
 public string File {get;set;}
 public int Line {get;set;}
 public string Text {get;set;}
 }
 public static class Extension
 {
 public static IEnumerable<GrepResult> Grep(string regexString, IEnumerable<string> wildcards,bool ignoreCase = true, bool recursive = false)
 {
 var regex = new ThreadLocal<Regex>(() =>
 new Regex(regexString, RegexOptions.Compiled | (ignoreCase ? RegexOptions.IgnoreCase : RegexOptions.None)));

 var files = from wc in wildcards
 let dirName = Path.GetDirectoryName(wc)
 let fileName = Path.GetFileName(wc)
                from file in Directory.EnumerateFiles(
                String.IsNullOrWhiteSpace(dirName) ? "." : dirName,
                String.IsNullOrWhiteSpace(fileName) ? "*.*" : fileName,
                recursive ? SearchOption.AllDirectories : SearchOption.TopDirectoryOnly)
                select file;

 var matches = from file in files.AsParallel(). AsOrdered().WithMergeOptions(ParallelMergeOptions.NotBuffered)
               from line in File.ReadLines(file).Zip(Enumerable.Range(1, int.MaxValue), (s, i) => new { Num = i, Text = s, File = file })
               where regex.Value.IsMatch(line.Text)
               select line;
 foreach (var line in matches)
 {
      yield return new GrepResult() {File = line.File, Line = line.Num,Text = line.Text} ;
 }
 }
 }

I would compile this into a dll and add reference to linqpad.  And here is the code that actually using it


static void Main()
 {
 Environment.SetEnvironmentVariable("LinqQueries",@"c:\Users\naveen\Documents\LINQPad Queries\*.*");
 var linqPath = Environment.GetEnvironmentVariable("LinqQueries");
 var x = from i in ConfigEntries
 select i.Name;
 x.Select (y => Extension.Grep(y.Name,new [] {linqPath})).Dump();
 }

I usually set environment variables (I am setting the env variable just for demonstration) for my projects so that I don’t have type the path manually. In the above code I am getting the values from database and searching those keywords within my hard disk. I could have use the Findstr method via Util class within LINQPad , but the PLINQ  implementation can use all my quad cores. I search for code very often and this is going to be handy.

2 thoughts on “Using LINQPad , PLINQ to grep for files

  1. This is a very interesting idea. I am trying to make the code work but “ConfigEntries” is undefined. Could you expand on that please. Thanks for this example. This is just what I was looking for.

  2. Here is my working code:
    static void Main()
    {
    Environment.SetEnvironmentVariable(“LinqQueries”,@”c:\Users\username\Documents\LINQPad Queries\*.linq”);
    var linqPath = Environment.GetEnvironmentVariable(“LinqQueries”);

    // initialize ConfigEntries to be a collection of FileInfo objects
    IEnumerable ConfigEntries = new DirectoryInfo(@”C:\Logs”).GetFiles(“*.log”);

    var x = from i in ConfigEntries
    select i.Name;

    x.Select (y => GrepResult.Extension.Grep(“logEntryText”, new [] { linqPath }, true )).Dump();
    }

    Thanks,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s