New Post: Architectural evolutions of R.NET for multiple concurrent R session

There is a demand for handling multiple R session in conjunction with R.NET, and improved features to run it from ASP.NET.

What this discussion is about:

• Confirm high level use cases for R.NET requiring multiple sessions of R
• Gather technical and architectural insight into where to evolve R.NET
  

In your replies, please:

• post only short summaries of your high level use cases, with link to more details if need be.
• Technical and architectural proposals can be more substantial; conciseness remains valued.
  

Known high level use cases

• Web app multi-user client via ASP.NET
• Running on cloud computing e.g. R.NET on Azure
• Cluster computing with MPI
  

Constraints

• The R interpreter itself is not safe enough for most multi-threading operations. This is not something R.NET can do anything about except limited workarounds.
  

Scenarios

• .NET application with single-session R embedded. This is what R.NET currently excels at, and the fastest way to interop with R in terms of MB/sec.
• .NET app + R.NET_client --> <some channel> --> R.NET_server + R
  <some channel> can be Inter Process Communication (IPC), WCF, TCP/IP etc. MPI is possible and has been done, though in practice used only in "pure" scientific computing so far as I know.
  

Related work

• RProvider.Server, part of (RProvider)[https://github.com/BlueMountainCapital/FSharpRProvider]. The protocol used is Inter Process Communication (IPC). I think this is a sound starting point to evolve into a component of R.NET.
• RServeCli is the equivalent of the layer R.NET_client above. The server is R+RServe, no .NET involved. The channel protocol is limited to TCP/IP with binary serialisation.
• RStudio and its handling of R sessions. This is however probably not something that can be reused, for licensing reasons.
  

Other general technical considerations for R.NET

• R.NET SymbolicExpression is inheriting from SafeHandle. The disposal of the objects is (usually) happening during the finalization of these objects. This is inherently an issue since this is done on a separate thread, clashing with R limitations. It is worked around with a mutex locking, but this remains problematic. A revision of the R data representation in memory intersects with what is needed to expose R data structures across multiple processes.
• rClr relies increasingly on R.NET for more sophisticated data interop.
  

Looking forward to your ideas.
Cheers,
J-M