Parallelism in Dinamica EGO

Parallelism, in computer science, is a technique that allows two or more computing tasks to execute at the same time. The central problem it solves is reducing data dependencies so that computations can run on independent computing units with minimal communication between them. It can even be worth doing the same computation twice on different units if that avoids a costly synchronization point.

On modern systems, parallelism is achieved in two different ways:

• By executing tasks in the same time slice, pausing inactive tasks to give an impression of concurrency (time-slicing).
• By having multiple processing units physically execute tasks simultaneously.

Since modern processor architectures are built from many cores, the latter approach strongly surpasses the former in both speed and ease of programming. Taking advantage of it means adapting software to account for concurrent access to data. Data is usually broken into independent slices that are processed on different computing units, but this becomes hard when the data is not easily separable, or when it depends on the result of many previous iterations. Software developers and architects have built a range of algorithms and techniques to minimize communication between computing units and reduce data dependency for exactly this reason.

Efficiently using the parallel capabilities of modern hardware is necessary to meet the demand for modeling complex dynamics with large datasets. At the same time, users are often reluctant to develop or adapt their models to take advantage of parallel environments, since that kind of development is typically complicated, time-consuming, and error-prone. Dinamica EGO's approach is to handle the parallelization itself, so a model author gets the benefit without having to reason about threads, cores, or scheduling directly.

Besides general-purpose processors, devices such as GPUs offer parallelism as well. A GPU can provide a much higher degree of concurrent operation than a traditional CPU, since it packs many more processing units onto the same device. Because those units are specialized, GPUs also require their own programming model and their own way of accessing data. In Dinamica EGO, GPU (and other accelerator) computation is exposed through OpenCL devices - see Accelerator Devices below.

Dinamica EGO's parallelism is coordinated by the Task Library, a global system for handling and scheduling work across the whole application. The Virtual Machine (VM) and Functor Library (FL) work together through it: models are broken into tasks according to both the capabilities of the machine and the semantics of the model - the rules the engine must follow, as prescribed by the dependency graph of the model itself (see Basic Data Flow for how that graph is built from a script). These tasks are distributed to the underlying computing units - processor cores and GPUs - and the workload is balanced using a work-stealing approach: idle workers take tasks from busier ones instead of sitting idle, which keeps utilization high without requiring the tasks to be evenly sized up front.

Every computing unit can manipulate and transform data in parallel - for example, different parts of the same image can be read and written simultaneously - and units can also communicate, synchronize, and exchange data with their siblings.

The Task Library lets Dinamica EGO use all available processing power on a machine, typically reducing model run times and leaving room to add more complexity to models without a proportional cost in execution time.

Dinamica EGO executes a model using a pool of workers, sized to the number of logical processors (CPU cores) available. This can be controlled two ways:

• In the GUI, via Tools → Options | Advanced | Automatically detect the number of processors available. See Number of Workers for the full description.
• On the command line, via the -processors=[processors] flag of Dinamica Console. Pass 0 for automatic detection.

Tip: Automatic detection sizes the worker pool to the number of available processors. Oversubscribing this - using more workers than physical processors - can still help when the work is heavily I/O-bound, since a worker blocked on I/O isn't using the CPU, and an extra worker can use that idle time productively.

Granularity controls how finely a job is split into parallel tasks. It can be set:

• Globally, on the command line, via the -granularity=[granularity] flag of Dinamica Console, which overrides the default granularity multiplier used to break jobs into parallel tasks.
• Locally, per container, via the Execution Policy functor's maximumGranularity and forceSpecifiedGranularity ports (see below).

A loop container's iterations can run simultaneously as long as three conditions all hold:

• No mux is used directly inside the loop.
• Nothing produced inside the loop is consumed by anything outside it.
• No functor inside the loop is used as a submodel's output port.

Without a mux, no iteration carries state forward to the next one, so iterations that share no connection have no dependency on each other and can be scheduled freely - possibly at the same time. Conversely, a mux can be added deliberately (MuxValue 0 0 is enough) to force sequential execution when ordering matters. See Basic Data Flow for the full explanation of how this works and why.

Accelerator (OpenCL) devices, typically GPUs, are a second computing target Dinamica EGO can schedule work onto, alongside regular processor workers. From Dinamica Console:

• -accelerators-policy=[accelerators-policy] - policy used to select accelerator devices. Values: NO_DEVICES=0, ALL_DEVICES=1, USER_CONFIG=2.
• -list-accelerators - show the available accelerator devices and exit, without running the script.

For per-container control inside a model, see the Execution Policy functor below. Debugging accelerator-related issues can be narrowed down with -log-level=6 -log-subsystem=2 (the OPENCL subsystem flag), also documented on the Dinamica Console page.

Execution Policy (registered internally as ExecutionPolicy; also recognized under its older alias ProcessorPolicy) is a Control container functor that executes its contents under a specified processor policy - it is the per-container equivalent of the -granularity and -accelerators-policy command-line flags.

It has no outputs.

Tip: Wrap a subgraph in an Execution Policy with computingTarget = .processorsOnly to sidestep accelerator-related execution errors on machines without a usable OpenCL device, without having to change the global -accelerators-policy for the whole run.

Beyond parallelizing across the cores and accelerators of a single machine, Dinamica EGO can also hand off a subgraph to run on a different machine over the network, through a Coordinator service. This is a separate, coarser-grained mechanism from the Task Library's work-stealing: instead of splitting one job across local workers, an entire container is submitted as a job to the Coordinator, which is responsible for placing it on a remote worker.

Run Remotely (RunRemotely) is a Control container functor that executes its contents through a network Coordinator.

Create Connection Info (CreateConnectionInfo, category Distributed) builds the ConnectionInfo value that Run Remotely's coordinatorConnectionInfo port expects.

Dinamica Console exposes fine-grained switches for turning parallelism on or off at different levels of the engine:

• -processors=[processors] - override the number of workers; 0 for automatic detection.
• -disable-parallel-functors - disable parallel processing of independent functors.
• -disable-parallel-steps - disable parallel processing of independent loop steps.
• -disable-parallel-map-load - disable loading maps in parallel.
• -granularity=[granularity] - override the default granularity multiplier used to break jobs into parallel tasks.
• -accelerators-policy=[accelerators-policy] - select which accelerator (OpenCL) devices are used.
• -list-accelerators - list available accelerator devices without running the script.

See Dinamica Console for the complete flag reference, including memory-allocation and logging options that interact with parallel execution.

• The progress bar shown in the lower-left corner of the interface during execution shows, on hover, what task each worker is currently performing, and keeps updating live.
• The Explorer panel's Attributes column indicates whether a loop can execute multiple steps simultaneously; hover the attribute icon for its tooltip. See GUI Tips.
• The Explorer's “Show model bottleneck only” icon narrows the view to the 10% of functors responsible for the largest total execution time - useful for finding what is limiting parallel throughput.
• Running Dinamica Console with -profile inside an interactive terminal opens a text-based UI with tabs for the message log, worker status, accelerator usage, and resource (memory) tracking.

• Basic Data Flow - the dependency-graph rules that determine what can run in parallel.
• Dinamica Console - full flag reference for controlling workers, granularity, and accelerators from the command line.
• Number of Workers and Memory Management - related GUI settings.
• Execution Policy, Run Remotely, Create Connection Info - the functors that control parallel and distributed execution directly inside a model.