This is unlikely to be an R.NET issue, even if this happen to occur when run from R.NET.
It certainly is machine dependent (the sample code I used to reproduce on a 32 bit laptop works fine on a 64 bit machine, both 32 and 64 bit R)
You should look at the function
If you have a 32 bits OS and/or limited RAM (you do not provide information), this is the likely cause. You may or may not be able to overcome your issue using R settings; you might even need to upgrade your hardware.
It certainly is machine dependent (the sample code I used to reproduce on a 32 bit laptop works fine on a 64 bit machine, both 32 and 64 bit R)
You should look at the function
memory.limit, and search e.g. in StackOverflow for questions with a similar error messages; there are plenty of them.If you have a 32 bits OS and/or limited RAM (you do not provide information), this is the likely cause. You may or may not be able to overcome your issue using R settings; you might even need to upgrade your hardware.