Large Collection(s) of RegEx objects, is this OK?

I'm writing a small application that parses data from a server as it is received, and have around 500 regular expressions that, when matched, perform various tasks. What would be the best way to keep up with all of these Regex's, keeping the application as fast as possible

I created a test application which looked something like this:

     int y = 0;
     for (int x = 1; x < 2000; x++)
     {

          {
               y++;
          }
          xRegex = null;
     }
     Console.WriteLine(string.format("Created and matched {0} regexes.", y));

This test ran very fast, however, would keeping 500 regex objects (probably a custom regex-derived class actually) in a set of collections be OK for most systems, or is this too much It will probably equate to something like 15 or so collections each containing 40 or so regex-derived objects. This way, when certain objects are disabled in the client I could completely skip parsing of a certain collection - ie. one collection may only contain debugging patterns, and wont need to be parsed if debugging mode is off.

Any advice would be appreciated, thanks in advance!
Brandon

(Moderator: Thread moved to the Regular Expression Forum)

.NET Development17

If the regex's don't change, you can store them in a collection. While requiring more memory, this will be faster than creating new ones all the time.

If speed is a concern, you can also use the RegexOptions.Compiled.

Thanks, I just wasn't sure if having around 500 regex's in memory would be considered too much.

Also, all of these regular expressions will be loaded at startup via either deserialization or simply reading in from a text file, will this limit me as to which method I can use (on-the-fly or compiled) I'm not quite up to par on all of this Regex stuff in .NET yet, but it seems like loading the expressions into memory during runtime will sort of negate the RegexOptions.Compiled benefits, is this accurate

Thanks in advance as always,
Brandon

No, the regex's will be compiled at runtime. .NET is able to create new code on the fly and execute it (this is called Reflection.Emit, if you want to know more).

The regex pattern gets parsed once, and a new class is created that's specialized to this regex. Of course, this takes some time, so you have to weigh between fast loading or fast execution.

Just check how long it takes to load 500 regex with the compiled option. You might want to show a splash screen during that time, just to give the user some feedback...

Ahh, I get it! I'll certainly go read up on Reflection a bit more. Thanks again!

Also, you may want to try Microsoft's CLR Profiler if you really want to know how much memory is allocated for those regex objects.

No, the regex's will be compiled at runtime. .NET is able to create new code on the fly and execute it (this is called Reflection.Emit, if you want to know more).

The regex pattern gets parsed once, and a new class is created that's specialized to this regex. Of course, this takes some time, so you have to weigh between fast loading or fast execution.

Just check how long it takes to load 500 regex with the compiled option. You might want to show a splash screen during that time, just to give the user some feedback...

One can also precompile the regex's into an assembly so the initial load problem is alleviated. Check out the section 3.13. Building a Regular Expression Library in the CodeProject's book chapter preview Microsoft Visual C# .NET 2003 Developer's Cookbook showing Chapter 3: Strings and Regular Expressions.