New Post: Memory leak (/issue), Please Help !

evolvedmicrobe wrote:

gc() should print out memory usage after each call, call it at the start and start of the loop and verify that the memory is being cleared. You can also do memory profiling in R via, http://stackoverflow.com/questions/7856306/monitor-memory-usage-in-r.

You seem to be holding two copies of the memory, one in C#/.NET the other in R, this alone will double the memory. It would be useful to know where the OOM error occurs, and how much memory is being used when it is run in R versus just as a .NET loop.

.

Thanks for the tip on using gc() to see how much memory R is using at the time. I'll experiment with that.

I did a similar experiment to the one you suggest, but just by removing all the R (R dot net) stuff out, but leaving my loops and ADO and SQL queries the same, and ran that. Ran fine, problem really appears to be not clearing the R memory. (despite all my gc() calls).

Here is my bugbear:

The bloody thing runs out of memory on the 9000th iteration. Where I want R data completely cleared after each one! i.e: I should have virtually no more memory used on the first iteration as the millionth!

The "clear all R memory" stuff (since restarting rEngine is forbidden) I am trying to do by this -

                    rEngine.Evaluate("rm(m)");
                    rEngine.Evaluate("rm(SCORE_FIELD)");
                    rEngine.Evaluate("rm(trans_model_dataframe)");
                    rEngine.Evaluate("rm(model_dataframe)");
                    rEngine.Evaluate("rm(m.RSQ)");
                    rEngine.Evaluate("rm(mNames)");
                    rEngine.Evaluate("rm(mCoef)");
                    rEngine.Evaluate("gc()");

                    rEngine.Evaluate("rm( list = ls( all = TRUE ) )");
                    rEngine.Evaluate("gc()");                           
                    

And also,

                GC.Collect();
                rEngine.ForceGarbageCollection();

As you can see in the code above. But no luck as yet.