c# InvalidProgramException  
Author Message
evilkid





PostPosted: Visual C# Language, c# InvalidProgramException Top

Hi,

I am auto-generating large (but simple) blocks of C# code, compiling to a dll then loading the dll via reflection. This seems to work fine for smallish test cases, but when generating larger programs otherwise similar code decides to throw the InvalidProgramException.

The c# compiler version is 8.00.50727.42 from the .net framework sdk v2.0.50727. Is there an updated compiler I can try If not is there an appropriate place to report the bug with an example

Cheers.

Andy.


Visual C#3  
 
 
TaylorMichaelL





PostPosted: Visual C# Language, c# InvalidProgramException Top

You can post the code that is causing the problems here and we'll take a look at it. If there is simply too much code to post then you can ZIP it up and post it to one of the public, free hosting sites that allow temporary storage of files. Ideally though nobody wants to look through a bunch of code they don't know so it is best to try and narrow down the problem area first.

I myself have generated 1000s of lines of code in order to enable theming in ASP.NET v1.1 and MS themselves use the CodeDom to generate theming and many other features in v2.0. Therefore the problem is unlikely to be in the relative size of the code. It is more likely that there is something not quite right with your program. Unfortunately the CodeDom can be unforgiving.

Michael Taylor - 12/5/06


 
 
evilkid





PostPosted: Visual C# Language, c# InvalidProgramException Top

Hi Michael,

Thanks for your reply.

First up a bit more detail. I am not using CodeDOM - I've written my own routines to generate the code instead. The main application is written in F# and it is into this that the dll is loaded and used.

Here is a short example of the generated code (which works). In bigger examples the functions get longer but the general structure stays the same.

using System;

using HDFS_Int = System.UInt32;
using HDFS_Int64 = System.UInt64;

namespace HDFS_CSim_test_sim_array {

public enum HDFS_PortType { Input, Wire, Output }

public class HDFS_Port {

public string name;
public int bits;
public int data;
public HDFS_PortType type;

public HDFS_Port(string _name, int _data, int _bits, HDFS_PortType _type) {
name = _name;
bits = _bits;
data = _data;
type = _type;
}

}

public class HDFS_Simulator_test_sim_array {

public HDFS_Int [] mem;
public HDFS_Int mem_size;
public HDFS_Port [] ports;
public HDFS_Int num_ports;

public void Reset() {
HDFS_Int i;
}

public HDFS_Simulator_test_sim_array() {
HDFS_Int i;
mem_size = 3;
num_ports = 2;
mem = new HDFS_Int [mem_size];
ports = new HDFS_Port [num_ports];
ports[0] = new HDFS_Port("i11", 1, 11, HDFS_PortType.Input);
ports[1] = new HDFS_Port("o0", 2, 6, HDFS_PortType.Output);
for (i=0; i<mem_size; i++) memIdea = 0;
Reset();
}

public HDFS_Port Port(string name) {
HDFS_Int i;
for (i=0; i<num_ports; i++)
if (name == portsIdea.name) return portsIdea;
return null;
}

public void Cycle() {

HDFS_Int temp;
HDFS_Int64 temp64;

// Inputs

// wire (11)
mem[1] &= 0x000007ff;

// Scheduled logic

// select (6:1)
mem[0] = mem[1] >> 1;
mem[0] &= 0x0000003f;

// Outputs

// wire (6)
mem[2] = mem[0];
mem[2] &= 0x0000003f;

// Registers (calculate next value)


// Registers (set next value)

}

}
}


With a long example (65K lines, mem_size=17155, num_ports=2400) it crashes when the Cycle() function is called. To ensure this isnt an F# or reflection issue I have tried using the code with the following C# program and get the same error:

using System;
using System.Collections.Generic;
using HDFS_CSim_test_sim_array;

class HdfsTest {
static void Main() {
HDFS_Simulator_test_sim_array sim = new HDFS_Simulator_test_sim_array();
sim.Cycle();
}
}

Unhandled Exception: System.InvalidProgramException: Common Language Runtime detected an invalid program.
at HDFS_CSim_test_sim_array.HDFS_Simulator_test_sim_array.Cycle()
at HdfsTest.Main()

I have also tried the code in Visual Studio Express (I generally build at the command line) and the same problem occurs. I cant even seem to get the de**** to step into the Cycle() function.

Cheers,

Andy

 
 
TaylorMichaelL





PostPosted: Visual C# Language, c# InvalidProgramException Top

If you are not using the CodeDOM then how are you generating the code Are you creating C# code that is then compiled through the compiler or are you generating IL directly. If you aren't use Emit or CodeDOM to generate code then is there any reason why not If you are generating an assembly then try loading it into Reflector or MSILDASM to see if it can be read. If it can't then the IL is wrong.

Michael Taylor - 12/5/06


 
 
evilkid





PostPosted: Visual C# Language, c# InvalidProgramException Top

I am generating a c# file directly then using csc.exe to compile the file. I'm not using CodeDOM because the eventual target is unmanaged c-code to be compiled with cl or gcc. I'm experimenting with some different options to try and get the best possible performance from these simulations.

Anyway, I've tried Reflector and it crashes when I try to look at the cycle() function. ildasm takes a while but does eventually show all the instructions.

As an experiment I tried generating a bunch of different size simulations and recording the size of the Cycle() function from ildasm.

These dont work:

IL_13e5d5
IL_c50e0
IL_b9bda
IL_a037c
IL_a13fd
IL_9f273

These do work:

IL_9eda5
IL_9b950
IL_81fad
IL_5abd2
IL_41257

This does seem to suggest that something or other is dependant on the size of a function.

Cheers,

Andy


 
 
TaylorMichaelL





PostPosted: Visual C# Language, c# InvalidProgramException Top

Interesting. There are limits to the length of most everything. AFAIK the C# compiler does not specify a limit for the size of a function. It is also not identified in the specification. I would say it is at least 1K if not 4K but I'm not sure. Either way if the function is too long then the compiler would generate an error. How long is your function exactly Do you use things like gotos

Personally I'd recommend breaking up the large function into smaller functions. Here's my rationale but please note that it is not back by scientific research just experience. The larger a function the longer it takes to compile and the more memory it'll consume. Furthermore in .NET the larger the function the longer JIT takes. Now breaking up the large function will reduce all the above including the runtime memory footprint, at least initially. The parts do not always sum up to equal the whole. It is just like threading. By breaking a function up you actually increase the JIT time, potentially the memory usage and runtime cost however it is spread out over a longer period of time so the implied behavior is different.

Breaking up an arbitrary function can be challenging especially if you use a lot of local variables. I use to write compilers for a living and I ran into the same problem you have. I use to convert standardized languages to C++ so they could then be compiled down to various Windows CE platforms. Since CE had tight memory constraints I had to break up functions that the user wrote. As the code generator generated lines of code I kept track of the "memory" used. When the function crossed the threshold I would create a new, hidden function to store the remainder of the code. The original function would then call the hidden function. This chaining could go on indefinitely and hid the limitation from the user. Hence a user's 1000 line function might have ended up being two 400 line functions and one 200 line function. By only creating additional functions as needed performance didn't suffer in the general case of small functions.

The hardest part of this whole process was getting the local variables from one function to the next. To solve that problem I used what was effectively an array of variable definitions. This array (along with a few support variables) was passed to the hidden functions. This kept performance up while still retaining the local variables across functions. I also had to support debugging and single stepping so part of the information that was passed was debugging information that our de**** needed to map generated code back to the original source.

As for determining the size to break the function I ended up basing it on the target platform. In my situation each target platform had a compiler setting block associated with it. This block provided the compiler with some settings needed by the target platform, including the maximum length of a function. You might need to do a similar thing because GCC might restrict the size of a function to something different than CL.

Michael Taylor - 12/6/06


 
 
Eric Cosky





PostPosted: Visual C# Language, c# InvalidProgramException Top

I know this is an old thread but this issue just hit me with a code generator I'm working on this is the first thread I've found on the subject. I'm pretty sure I've hit some soft limit on how many instructions a function can have; the code I have now is throwing a stack overflow exception when run in the de****, but when run outside the de**** will consume about 1.5 gig of RAM to JIT the function. It's very simple code, just a bunch of arrays being built with static/hardcoded data produced by another tool. About 15k lines of array assignment code in total, where each assigment has a few nested new statements in the object being assigned. Certainly nothing that would suggest it would require that much memory to JIT.

It would be nice if the JIT were able to deal with large functions more robustly, perhaps produce a warning or something when a "reasonable" function length is exceeded. Prior to this, I had no idea that function length was any kind of issue for the JIT whatsoever or I never would have done it this way. On a related note, it would sure be nice to have static array initializers that behaved like real static arrays instead of translating into a bunch of Add() calls. I had hoped the C#3.0 member initializer constructor form (ie var myobj = new mytype() { member = something, member2 = something }; ) would produce true static arrays but no luck there. Seems like there is no way at all with C# to create true static data (like C++) instead it creates IL that sets the data up. If I've overlooked something that allows this I'd sure like to hear about it.

Thanks