Differences

This shows you the differences between two versions of the page.

--- calculate_r_expression [2017/03/11 05:12]
hermann [Notes]
+++ calculate_r_expression [2026/08/28 03:14] (current)
hermann Sync from local documentation review
@@ Line 3: / Line 3: @@
 ===== Description =====
-The functor 'Calculate R Expression' allows Dinamica EGO to call R externally and process the script outputs as if they were part of Dinamica itself.
+This is a container functor that calls R externally with a user-defined expression. Like the other calculator functors, data is connected through hook functors placed inside its block.
 ===== Inputs =====
 ^ Name  ^ Type  ^ Description  ^
-| Expression  | [[String Type]]  | The expression that will run on R.  |
+| Expression  | [[Code Type]]  | The expression to run on R. Written directly as a Code constant using its own raw string syntax — see [[#writing_the_expression_in_ego_script|Writing the expression in EGO Script]] below.  |
-| Treat Warning As Errors  | [[Boolean Value Type]]  | Warnings on R script will be considered errors.  |
+| Treat Warning As Errors  | [[Boolean Value Type]]  | If true, warnings raised by the R script are treated as errors.  |
 ===== Optional Inputs =====
@@ Line 15: / Line 15: @@
 None.
-===== Output =====
+===== Outputs =====
 ^ Name  ^ Type  ^ Description  ^
-| result  | [[Struct Type]]  | A struct containing the output values generated by the expression.  |
+| Result  | [[Struct Type]]  | Struct containing the output values generated by the expression, one entry per call to an output function in the R script.  |
 ===== Group =====
@@ Line 26: / Line 26: @@
 ===== Notes =====
-There are two ways to setup the 'Calculate R Expression' functor:
+==== Expression inputs ====
-=== Dinamica EGO Enhancement Plugin ===
+Data is passed into the expression through hook functors placed inside the container's block — the same verbose-form hook mechanism used by the calculator functors. Hooks can be added using the Create a hook button on the functor bar, or by dragging them in individually.
-Simply download and install the [[plugins_4|Dinamica EGO Enhancement Plugin]]. It contains everything you need to run R scripts inside Dinamica EGO.
+  * Tables and lookup tables → [[Number Table]] → available in R as t1, t2, …, t100
+  * Scalar values → [[Number Value]] → available as v1, v2, …, v100
+  * Strings → [[Number String]] → available as s1, s2, …, s100
-=== Local R Installation ===
+Maps cannot be connected — this functor has no cell context. There is no shorthand notation; see [[Calculate Functors|Calculate Functors — Complete Operator Documentation]] for the general hook mechanism and syntax.
-Additionally, users can specify and use their own local R installation for execution. The pre-requisites are:
+=== Tables and lookup tables ===
-  * R needs to be at least at the 3.3.1 version.
+Lookup tables and tables require extra care, since each is transferred to R using a different representation.
-  * ''Rscript.exe'' must be valid and located at the ''<installation folder>/bin'' sub-folder.
+A lookup table is transferred as a list with two columns, Key and Value. Each column is accessed with the $ operator:
-  * [[external communication|Dinamica package]] for R must be installed and at the latest version.
+<code rsplus>
+# Access the third key and its corresponding value in lookup table t1
+patchId <- t1$Key[3];
+patchArea <- t1$Value[3];
+</code>
-To specify your custom R installation, go to Tools -> Options -> Integration tab and select //Use alternative R installation for Calculate R Expression//:
+A table is transferred as a [[https://en.wikibooks.org/wiki/R_Programming/Working_with_data_frames|DataFrame]], with each column likewise accessed using $. Two conventions apply to tables in either direction:
-{{:dinamica_options_use_alternative_r_install.png?nolink|}}
+  * Key columns are marked by an asterisk (*) appended to their column name — this is the same convention used throughout Dinamica EGO's table representation. A key column can be Real or String, the same as any other column.
+  * Each column's type is inferred from its data, the same rule that applies to any table: a numeric vector produces a Real column, a character vector produces a String column. R automatically converts string columns to Factors inside a data.frame, and a Factor is neither of those — Dinamica requires plain Character Vectors to infer String correctly. Always build tables with ''stringsAsFactors = FALSE'' to prevent this conversion.
-==== Usage ====
+For further detail on the underlying table representation, see [[External Communication]].
-The functor is available under the //Integration// tab at the library.
+==== Expression outputs ====
-=== R Script Inputs ===
+Values are returned to Dinamica by calling one of the following functions from the R script. Every call requires an identifier as its first parameter — the name Dinamica uses to place the value into the output struct — and the value itself, which can be constructed inline, as in the examples below, or supplied as a variable:
-To pass input parameters from Dinamica EGO to R, hooks must be placed to collect data. You can use the //Create a hook// button on the selected functor bar or drag the hooks individually. The valid hook types are:
+^ Function  ^ Output type  ^ Notes  ^ Example  ^
+| outputDouble()  | Real  | Accepts any numeric value.  | outputDouble("myDouble", 3.14)  |
+| outputNumberVector()  | Tuple  | Accepts any collection of numbers.  | outputNumberVector("myTuple", c(1:10))  |
+| outputString()  | String  | Accepts any string value.  | outputString("myString", "This is a string")  |
+| outputLookupTable()  | [[Lookup Table Type|Lookup Table]]  | Requires two number vectors of equal length — one for the keys, one for the values. Lookup tables are always Real-typed on both sides; there is no String option.  | outputLookupTable("myLUT", c(1:10), c(1:10) * 10)  |
+| outputTable()  | Table  | Requires a table built with the [[https://www.r-tutor.com/r-introduction/data-frame|data.frame]] function, using ''stringsAsFactors = FALSE'' as described above. Its optional second parameter (default 1) sets how many leading columns, from the left, are key columns.  | outputTable("myTable", data.frame(State = c("Massachusetts", "Massachusetts"), City = c("Boston", "Chelsea"), Population = c(667137, 39398), stringsAsFactors = FALSE), 2)  |
-  * [[Number Table]]
+==== Retrieving outputs ====
-  * [[Number Value]]
+Calculate R Expression returns a single [[Struct Type]] value (via its Result output port) containing every value passed to an output*() function. To retrieve individual values from that struct, use the corresponding functor from the Integration group:
-  * [[Number String]]
+^ Functor  ^ Retrieves  ^
+| [[Extract Struct Number]]  | A value passed to outputDouble()  |
+| [[Extract Struct Tuple]]  | A value passed to outputNumberVector()  |
+| [[Extract Struct String]]  | A value passed to outputString()  |
+| [[Extract Struct Lookup Table]]  | A value passed to outputLookupTable()  |
+| [[Extract Struct Table]]  | A value passed to outputTable()  |
-To access the passed inputs on your R script, follow Dinamica EGO naming conventions and use ''t[0-99]'', ''v[0-99]'', ''s[0-99]'' for each type respectively. For example:
+Each functor takes two inputs: the Struct returned by Calculate R Expression, and the name of the entry to extract as a string constant.
-<code rsplus>
+==== Installing packages ====
-# Will store the 5th key from the passed Lookup Table in the variable 'testing'
-testing <- t1$Keys[ 5 ];
-# Store the second passed String.
+Packages are installed by calling dinamicaPackage("packageName") from within the expression — one call per package. Unlike [[Calculate Python Expression]], there is no separate input port for listing packages; dinamicaPackage() is the only mechanism available.
-secondString <- s2;
-# Store the 10th passed Value.
+dinamicaPackage("packageName") also acts as R's library() call: when the package name matches the name of the module to load, calling it both installs the package (if not already present) and loads it, in a single call.
-tempNumber <- v10;
+<code rsplus>
+dinamicaPackage("moments");
 </code>
-=== R Script Outputs ===
+==== Setup ====
-To output data back to Dinamica EGO, the user must call a set of functions designed for this functionality:
+There are two ways to run R scripts from Calculate R Expression — though only the local installation is available on Linux.
-^ Function name  ^ Output Data Type  ^ Usage Example  ^
+=== Dinamica EGO Enhancement Plugin ===
-| outputDouble()  | Real  | outputDouble( "myDouble", 3.14 )  |
-| outputNumberVector()  | Tuple  | outputNumberVector( "myTuple", c(1:10) )  |
-| outputString()  | String  | outputString( "myString", "This is a string" )  |
-| outputLookupTable()  | LookupTable  | outputLookupTable( "myLUT", c(1:10), c(1:10) * 10 )  |
-| outputTable()  | Table  | outputTable( "myTable", anyTable, 4 )  |
-In the examples above, data was constructed inside the calling functions. You can also specify variables as parameters.
+Windows only. Download and install the [[Dinamica EGO Enhancement Plugin]]. It contains everything needed to run R scripts inside Dinamica EGO, with no further configuration.
-<note important>All output functions require an identifier as the first parameter, that's the name Dinamica uses for retrieving the correct data.</note>
+=== Local R installation ===
-Keep in mind:
+On Linux, this is the only option — Dinamica EGO always uses the R installation already present on the system. On Windows, it can be used as an alternative to the plugin. Either way, it requires:
-  * You can pass any kind of numeric value on the ''outputDouble'' function.
+  * R installed on the machine, with the Rscript executable (Rscript.exe on Windows) present in its bin sub-folder.
+  * The Dinamica package for R installed and at its latest version.
-  * Collections of numbers are valid number vector's.
+On Windows, this alternative installation is selected in the Dinamica EGO GUI by going to Tools → Options → Integration tab and enabling "Use alternative R installation for Calculate R Expression".
-  * A [[Lookup Table Type|Lookup Table]] requires //2// number vectors, one for the "Keys" and another for the "Values". Both vectors must have the same number of elements.
+Reports an error if this functor is used without either the Enhancement Plugin or a working local R installation configured.
-== Tables / Lookup Tables ==
+Raises an error if the number of Number Table hooks nested inside this container exceeds the capacity of the external communication message queue.
-[[Table Type|Tables]] or [[Lookup Table Type|Lookup Tables]] can be transferred to R using the [[Number Table]] functor. When dealing with each of these types extra caution must be taken:
+==== Examples ====
-== Lookup Tables ==
+The following examples use a consistent set of inputs: t1 is a lookup table of land cover patches mapping Key (patch identifier) to Value (patch area); v1 is a scalar minimum area threshold; and s1 is a string giving the name to use for a threshold-flag column.
-Lookup Tables are transferred to R as a List with two columns, "Keys" and "Values".
+Compute the mean and total area across all patches, and report progress to the Message Log:
-Each column can be individually accessed by using the ''$'' operator, for example:
+<code rsplus>
+patchMean <- mean(t1$Value);
+patchTotal <- sum(t1$Value);
+print(paste("Processed", length(t1$Value), "patches"));
-  * ''t1$Keys[ 1 ]'' will access the first key of the Lookup Table ''t1''.
+outputDouble("meanArea", patchMean);
+outputDouble("totalArea", patchTotal);
-  * ''t2$Values[ 2 ]'' will access the second value of the Lookup Table ''t2''
+</code>
-== Tables ==
+The print() call is visible in Dinamica EGO's Message Log, shown as a Result-level message. Log levels are ordered Unconditional, Error, Warning, Result, Info, Info2, Debug, Debug2 — messages printed from R are only shown when the Message Log level is set to Result or a more verbose level; at Unconditional, Error, or Warning they are suppressed.
-Tables are transferred to R as a [[https://en.wikibooks.org/wiki/R_Programming/Working_with_data_frames|DataFrame]] type variable. Each column can be individually accessed by using the ''$'' operator, just like Lookup Tables. Please check [[external_communication#table|this page]] for more information on how to deal with Tables. The rules for them in [[Calculate R Expression]] are the same as sending and receiving through [[External Communication]].
+Filter the lookup table down to patches whose area meets the threshold, and return the filtered lookup table:
-=== Example Scripts ===
+<code rsplus>
+aboveThreshold <- t1$Value >= v1;
+filteredKeys <- t1$Key[aboveThreshold];
+filteredValues <- t1$Value[aboveThreshold];
-  * Multiply each value of the passed Lookup Table by itself
+outputLookupTable("filteredPatches", filteredKeys, filteredValues);
+</code>
-{{ ::ex1_calculate_r.png?nolink |}}
+Build and return a table with one row per patch, including a column that flags whether each patch meets the threshold. The column's name is taken from the passed string s1 rather than being hard-coded:
-Expression:
 <code rsplus>
-for ( i in 1:length( t1$Values ) ) {
+patchFlags <- data.frame(
-  tempValue <- t1$Values[ i ];
+    PatchId = t1$Key,
-  t1$Values[ i ] = tempValue * tempValue;
+    Area = t1$Value,
-}
+    MeetsThreshold = t1$Value >= v1,
+    stringsAsFactors = FALSE
+);
+names(patchFlags)[3] <- s1;
-outputLookupTable( "poweredTable", t1$Keys, t1$Values );
+outputTable("patchSummary", patchFlags, 1);
 </code>
-  * Extract the mean of passed Lookup Table values
+Install the moments package and use it to compute the skewness of the patch area distribution -- a statistic not available in base R -- then flag patches whose area is a statistical outlier:
-{{ ::ex2_calculate_r.png?nolink |}}
-Expression:
 <code rsplus>
-tableMean <- mean( t1$Values );
+dinamicaPackage("moments");
-print( paste( "Mean is", tableMean ) );
-outputDouble( "mean", tableMean );
-</code>
-Noticed the ''print()'' statement? Dinamica EGO's Message Log will show output messages from the R script (as Result's).
+patchMean <- mean(t1$Value);
+patchStdDev <- sd(t1$Value);
+patchSkewness <- skewness(t1$Value);
-  * Plot passed Lookup Table to an image on the path specified by passed String
+isOutlier <- abs(t1$Value - patchMean) > 2 * patchStdDev;
+outlierPatches <- data.frame(
+    PatchId = t1$Key[isOutlier],
+    Area = t1$Value[isOutlier],
+    stringsAsFactors = FALSE
+);
-{{ ::ex3_calculate_r.png?nolink |}}
+outputDouble("meanArea", patchMean);
+outputDouble("stdDevArea", patchStdDev);
+outputDouble("skewnessArea", patchSkewness);
+outputTable("outlierPatches", outlierPatches, 1);
+</code>
-Expression:
+==== Writing the expression in EGO Script ====
-<code rsplus>
-outputFile  <- s1;
+Like [[Calculate Python Expression]], the Expression input is type [[Code Type]]. It can be filled in directly as a text constant, using Code Type's own raw string syntax -- the same ''$"<delimiter>( raw_characters )<delimiter>"'' form used by String constants. This is also the form the Dinamica EGO GUI's dedicated code editor generates when it writes the Expression port's value, so a hand-written script and one produced by the GUI take the same shape. See [[Code Type]] for the full grammar, including the base64 alternative form.
-print( paste( "Plotting to", outputFile ) );
-jpeg( outputFile, quality=100 );
+<code>
-plot( t1 );
+result := CalculateRExpression $"(
-dev.off();
+aboveThreshold <- t1$Value >= v1;
+outputLookupTable("filteredPatches", t1$Key[aboveThreshold], t1$Value[aboveThreshold]);
+)" .no {{
+    NumberTable landCoverAreas 1;
+    NumberValue minimumArea    1;
+}};
+filteredPatches := ExtractStructLookupTable result "filteredPatches";
 </code>
+Here .no is the value of Treat Warning As Errors.
 ===== Internal Name =====