R Coupling

Dinamica EGO 4 provides some sophisticated ways of using R as part of a model:

Exchanging Data between R and Dinamica

Dinamica EGO 4 changes the game when it comes to exchanging data with R. Now Dinamica can use a specialized set of functors to send and receive data to and from R. R can use an special package (called “Dinamica”) to send and receive data to and from Dinamica. This allows both tools to run simultaneously and none of them needs be in charge of controlling the simulation. Those specialized functors and package functions not only take care of data exchange between both tools, but also ensure that the calls are kept synchronized by blocking a receiver when the corresponding data is not yet available or the sender has not yet finished baking all the information.

Under the hood, the data exchange works by creating a “shared session” between an instance of Dinamica and an instance of a R interpreter. Each “shared session” creates a special entity used to store all pending messages exchanged between the applications. The messages can be divided in two different groups with short messages being exchanged via a shared queue and long messages also being placed in temporary storage while the transfer is being processed. Typically, numbers and strings are handled as short messages while lookup tables and tables are handled as long messages.

A “shared session” uses an unique name as identifier. It is possible to have more than one instance of Dinamica and R talking to each other, but each pair of instances must use an unique identifier, called an “active session name” to identifier their corresponding sessions and isolate their communication from the other pairs. The “active session name” can be configured in the “Tools → Options | Integration | Communication Session” options. The session name is also visible in the status bar. Double clicking that label is another way of editing the corresponding definition.

Dinamica EGO is responsible for creating the communication session when the application starts, and it happens automatically when the application starts. Functors related to sending and receiving data will always use the current session to exchange information. On the other hand, R does not have to create a session for talking to Dinamica, and in fact it does not have the tools for that, but it has to explicitly connect to the “shared session” created by Dinamica. This should be done by calling Dinamica::openSession(). See the help about external communication for additional information about how to set up a session.

When sending and receiving data between the applications, ordering is important. Dinamica and R execute their respective functors and command in a specified order, R according to how the sequence of command is written in the script, and Dinamica EGO according to the functor dependencies in the model. The user has to make sure that the execution of a functor or command “Receive” is matched by a preceding execution of a functor or command “Send” in the other application.

The following figure shows a pseudo-example where a Dinamica model and a R script are exchanging data.

Calculate R Expression

Another way of coupling Dinamica and R is by using the functor Calculate R Expression. This functor allows an R script to be executed as part of a Dinamica model. This is usually the preferred way of coupling Dinamica and R. By default, Dinamica uses an Enhancement Plugin containing a custom R interpreter. This interpreter comes with several pre-installed packages that can be used to define the R script.

Data from Dinamica model can be passed using hooks just like it would be done with any functor from the Calculate family, including Calculate Lookup Table, Calculate Map etc. Inside the R script, the data is accessible via binding variables such as v1, s1, t1 etc.

Returning data from the R script back to the Dinamica model is done using the commands such as outputDouble(), outputTable(), outputString() etc. All results returned by the script are bundled in a special type called Struct. Struct's are basically collections of unrelated data where each one of their components can be accessed using a corresponding key. The use of the Struct as a data return mechanism ensures that, despite the Calculate R Expression having only one output port, the R script can return as many results as necessary. Struct's can be disassembled in the Dinamica model using the “Extract Struct” family of functors.

The following figure shows a model where R is being used to calculate the mean of both columns of a Lookup Table. The R script returns its results in Struct two real value as fields, named “meanForKeys” and “meanForValues”. A couple of functors Extract Struct Number is used to extract the value of the fields from the resulting Struct.

The next figure shows a more complicated example where the R script is being used to list all files from a given folder with a particular filename extension. The particular example shows how to return a table as a result. Tables are always passed and returned to and from R scripts as data frames. The documentation of the Calculate R Expression explains some of the pitfalls of dealing with data frames to pass and receive data, specially when a table contains strings.

In the example, the single table represented the list of files is returned as a field called “files” in the resulting Struct produced by the Calculate R Expression.

Some very simple examples showing how to use an R script as part of a Dinamica EGO model are provided as part of the dataset distributed with the Dinamica EGO application. Check the content of Dataset\Examples\various_applications\calculate r expression.

Selecting the R Interpreter to Use

When using the Calculate R Expression, Dinamica automatically invokes and stops the selected R interpreter to evaluate the expression as necessary. The user can control which R interpreter will be used by selecting appropriate options in “Tools → Options | Integration | Calculate R Expression / R Integration”. It is possible to choose between the default R interpreter provided with the Enhancement Plugin or an alternative R installation installed independently.

Any R interpreter can be used as an alternative (and external) R installation provided that a compatible RTools and the "Dinamica" package are installed. The biggest advantage of using a alternative version of R is that the user will be free to install and import any packages he wants. When using the default R interpreter from the Enhancement Plugin, the user is limited to the packages that come pre-installed with the platform.

For more information about how to set an external R interpreter, see the documentation about the Calculate R Expression and external communication.

Legacy Ways of Exchanging Data Between Tools

The previous ways how Dinamica EGO and an external tool, such as R, could be linked together are still valid and can be used when the new ways do not suffice.

Legacy way of calling R from Dinamica EGO

Previously, calling R from Dinamica was only possible using the Run External Process functor. In this scenario, Dinamica was simply running R as any other external tools, without knowing anything about the specificity of the process being executed. All data transfer had to be done in an ad hoc manner by saving the data that was going to be passed to R to the storage in the Dinamica model, and loading that data again from the disc in the R script. Sending the results from R to Dinamica would require the opposite steps, with the R script being responsible for saving the data to storage and the Dinamica EGO model being responsible for loading the data again.

Preserving a state across invocations of the same R scripts would require R to save and reload the relevant parts of its model for each iteration, since the Run External Process would be simply spawning and terminating an instance of the R interpreter every time step.

Legacy way of calling Dinamica EGO from R

Calling Dinamica from R would be very similar, except that now Dinamica would be responsible for preserving its simulation state and R would be in charge of invoking Dinamica by using a system() call or a similar mechanism. R would also be required to run the command line version of Dinamica instead of running the graphical interface.

Exchanging data would work in a similar way with R saving the data to disc to pass the data to Dinamica with Dinamica loading the data again as part of its own model. As expected, sending the result back would required the opposite.