(Compared to stable version 8.3.0)

The recent updates to the core system and the GUI (Graphical User Interface) significantly enhance both the overall performance and the user experience of the application.

Many core system updates focus on optimizing resource utilization, speeding up operations, and improving efficiency:

• Compilation warnings and errors have been fixed.
• Support for C++20 has been added.
• Scripts for configuring the build, compiling source files, and installing libraries and utilities have been added.
• Performance flags have been added to Windows builds in CMake.
• Memory errors caused by compiled expressions have been fixed.

• A memory leak on Distributed has been fixed.
• Unnecessary data copying has been reduced or eliminated in parallel constructs and during data transfer to viewers.
• Basic data types used for port data transfer have been refactored and optimized, eliminating the need for casting, improving construction and destruction efficiency, and simplifying inheritance relationships.
• The way data is copied during transfer between data holders has been changed to prevent multiple copies.

• The `CalcPatchLabelMap` execution has been massively optimized through a complete re-implementation, maintaining the same basic algorithm but improving execution time and performance.
• `std::map` has been replaced with `std::unordered_map` where possible.
• The `stringstream` use has been replaced with the `fmt` library for category name creation, leading to faster execution.
• `std::unordered_set` is now used instead of `std::set` when dealing with task graphs.
• `DFF::toString` implementations for float and double have been replaced with `fmt::format`, and direct stream writing using `fmt::format` is now utilized for faster operations.
• `LookupTable` functions have been optimized.
• `ExtractCategoricalMapAttributes` has been optimized by parallelizing the insertion of attributes into the table.
• Optimizations have been added for copying subtables as-is and moving column chunks instead of copying rows.
• A fast-double-parser is now used for parsing doubles when reading CSV tables.
• The `CalcHistogram` functor has been improved with better floating-point support and performance.
• The use of `gtl::btree_map` has been experimentally introduced to replace `std::map` in several core data structures, leveraging its potential for better cache-friendliness and performance in:
  • `LookupTable`.
  • `Categorization`.
  • `SparseRasterImage`.
  • `Properties`.

• Tasks added to task groups can now be dispatched immediately.
• `Extract Categorical Map Attributes` now computes mode, median, average/variance/stddev concurrently.
• Subtable merging can now utilize tasks, and table normalization from the UI uses tasks.

• An event indicating a port received new data was removed to prevent unnecessary performance degradation.
• Atomic normalization was removed from tables as it was unused.
• Searching for modules that hurt MSVC performance on Windows has been disabled.
• Optional `mimalloc` support has been added to Windows builds.
• Python scripts are now executed as independent sub-processes, which includes improvements like allowing `dinamica` python to detect termination requests and making Linux system processes use default environment variables.
• A custom `AtomicMutex/Lock` (SpinLock) was added to Table to allow locking/unlocking between different threads.

Both core and GUI updates contribute to a more intuitive, stable, and feature-rich user experience:

• Numerous bug fixes address issues such as:
  • Errors with compiled expressions and invalid data to `RunRemotely`.
  • Incorrect file path conversions.
  • Python assertions related to UTF-8 conversion.
  • Regression bugs.
  • Issues with parallel constructs.
  • Memory leaks in the GUI, especially in table editors/viewers, are prevented.
  • Crashes when visualizing complex image trees.
  • An inconsistency where a viewer could retain outdated content has been fixed.
  • A temporary workaround was implemented for a potential crash when joining subtables through the task system.
  • A code fix was required to prevent crashes when using `gtl::btree_map` with `SparseRasterImage`.
• Error messages have been improved for clarity and grammar.
• The application version has been updated to 8.4.0 and then to 8.5.0.
• A custom `AtomicMutex/Lock` (SpinLock) has been added to `Table` for inter-thread locking.

• New resampling methods (weighted-sum and root-mean-square) have been added to `TransformMapFunctor`.
• Functionality to verify coordinator availability is now exposed to Java and available in the GUI.
• Python script execution improvements include detecting termination requests and adding `dinamica`'s Python path to the process. An advanced optional input port for packages has been added to `CalculatePythonExpression`.
• The `GetRegionInfo` functor has been introduced for accessing region details.
• The ability to expand all submodels in the selected script has been implemented.
• An action to import the currently selected submodel has been added.
• New features include adding `getTemporaryPathFreeSpace` and allowing agent IDs and temporary space to requirement expressions for `RunRemotely`.
• Dynamic debugging support has been added to CMake.
• The application now allows tables to have any name format on their column titles.

• Updates to the update manager provide better selection between stable and development releases, clearer messages, and fixes for update notifications.
• A temporary blinking border has been added around editors and options dialogs where validation errors occur or settings are edited via shortcuts, drawing user attention.
• Information about ports is always displayed in the functor editor window, even if the port cannot be edited. The message displayed by port editors when a port cannot be edited has been clarified.
• The design of the “Previous-Next History” has been improved with consistent layout, adjusted panel sizes, and standardized borders.
• A filter to display only selected submodels has been added.
• The Explorer panel's rendering artifacts have been fixed.
• Tooltips have been added to UI elements like the `showSelectedItemsButton` and the option to control redundant carrier removal.
• Table loading now uses `LazyPresentationPanel` and existing indices to speed up display.
• The GUI is prevented from hanging when the log is filled with messages.

• Redundant carrier utilities have been centralized in the C++ codebase, with a new method to remove all redundant carriers from the script.
• A flag in `ScriptRunner` allows automatic removal of redundant carriers, with console support.
• An action to remove all redundant carriers from the script has been added to the GUI.
• The carrier removal process has been simplified by merging the redundancy check into a single dialog with clear options (Remove and Readjust, Just Remove, or Cancel).
• The default behavior for redundant carrier removal before model execution has been changed to `false` for safety, preventing unintended semantic alterations or visualization issues.
• Submodel pendencies are now ignored if all derived pendencies causing them are ignored.
• Selection is no longer cleared after submodel installation, and `CheckBoxSelectionModel` selection handling has been refactored for better clarity.

• Information about the functor corresponding to each running task is always shown in the execution summary tooltip.
• An option to force the generation of core dumps when the application crashes has been added.
• Improved Python error reporting functions.

• Formatting fixes have been applied throughout the code.
• The `isAutoUpdateSupported()` method was removed as auto-update is now supported across all operating systems.
• The function reporting if the application is 64 bits was removed, as the application is now always 64 bits.
• The `Test::runScript` function's documentation for the `optimizeScript` flag has been fixed.
• The GUI is now prevented from hanging when the log is constantly filled with messages.

• The update manager has been improved to better select between stable and development releases.
• Messages have been clarified within the update manager.
• The pending-update icon no longer loses its tooltip under certain circumstances.
• Minor issues related to update notifications showing on the system tray have been fixed.

• How functions from the ProxyFunctorUtils, CellTypeInfo, LogTagInfo, ScriptUtils, SubmodelUtils, NameUtils, and StringUtils namespaces are exported to Java has been changed to emulate classes with static members.
• Some Java exported functions were moved from the `SubmodelUtils` and `ScriptUtils` namespaces to the `Script` class in Java.
• Functions related to projection are now exported as static members of the `Projection` class in Java.
• Functions related to the comparison of versions as strings are exported as static members of the `Version` class in Java.
• Functionality to verify coordinator availability has been added and exposed to Java.
• The Java `dff.LogMessage` now uses `char[]` instead of `string` to speed up logging.

• A bug causing rendering artifacts in the Explorer panel has been fixed.
• Selection is no longer cleared after submodel installation.
• The handling of selection in `CheckBoxSelectionModel` has been refactored using a `TreeSelectionListener`.
• Validation logic from `attemptConnection` was added to `canReadjustConnections` to prevent suggesting invalid connection readjustments.
• A bug where closing the dialog still confirmed the action of expanding the submodel definition has been fixed.
• An inconsistency where a viewer could retain outdated content if its equivalent peer did not have data during the last model execution has been fixed.
• A bug that could trigger an assertion when visualizing a port while submodel expansion and redundant carrier removal were enabled simultaneously has been fixed.
• Grammar in error messages has been improved, replacing “can not” with “cannot”.

• A filter to display only selected submodels has been added.
• A temporary blinking border has been added around the editor when validation errors occur, or around relevant options in the OptionsDialog when settings are edited via shortcuts.
• A method was defined to clarify the behavior when a functor is highlighted as responsible for an error.
• Component placement has been fixed, removing unnecessary uses of `fill` and `weightx`.
• The presentation of information for the `STB_IMAGE_WRITE` library in the About window has been fixed.
• Information about a port is always displayed in the functor editor window, even if the port cannot be edited.
• The message displayed by port editors when a port cannot be edited has been clarified.
• The design of the “Previous-Next History” has been improved with consistent layout, adjusted panel sizes, standardized border thickness for selected/unselected items, and a JScrollPane that always shows a horizontal scrollbar.
• Tooltips have been added to UI elements, such as the `showSelectedItemsButton` and the option to control the removal of redundant carriers.

• Submodel pendencies are now treated as ignored when all derived pendencies causing them are ignored.
• An action to expand all submodels in the selected script has been implemented.
• An action to import the currently selected submodel has been added, and its name made more consistent.
• Inconsistencies in the definition of accelerator keys for Submodel menu items have been fixed.
• The order of “Sign In” and “Sign Out” items in the Submodel menu has been fixed.
• The carrier readjustment logic has been fixed by using `dff.ScriptUtils.isCarrierRemovable` and direct `createConnection` calls.
• An action to remove all redundant carriers from the script has been added, utilizing DFF redundant-carrier utilities.
• The interface refresh logic after connection adjustments has been improved.
• `reconnectAllFunctors()` was introduced to refresh interface connections after a removal.
• A constraint was added to suggest redundant carrier connection adjustments only when removing a single functor, with more complex cases handled by the dedicated utility.
• The carrier removal process has been simplified by merging the redundancy check into a single dialog with clear options: “Remove and Readjust”, “Just Remove”, or “Cancel”.
• The `removeRedundantCarriers` flag handling has been included in the interface.
• The default behavior for redundant carrier removal before model execution has been changed to `false` for safety, preventing unintended semantic alterations or visualization issues.
• The command-line flag controlling the removal of redundant carriers now enables the feature instead of disabling it.

• `LazyPresentationPanel` is now used for loading tables, with task-based normalization when needed.
• Pointers that could possibly leak memory on Table editors/viewers are now cleared.
• Displaying tables is faster by using existing indices instead of copying them again.

• An option to force the generation of core dumps when the application crashes has been added.
• Information about the functor corresponding to each running task is always shown in the execution summary tooltip.
• The Python environment reset now attempts to restart the application if Python libraries are in use.
• Previously missing details about the licensing options of certain libraries used by the application have been included.

• Compilation warnings and errors have been fixed, including specific fixes for GCC and C4100 warnings.
• Formatting has been applied throughout the code.
• The application version has been updated to 8.4.0 and later 8.5.0.
• The copyright year has been updated.
• The function reporting if the application is 64 bits was removed, as the application is now always 64 bits.
• The SWIG version has been updated to 4.3.0.
• C++20 support has been added.
• The `dinamica_dependencies_windows.zip` has been updated.
• Unused includes have been removed.
• `stringstream` usage has been replaced by the `fmt` library for category names, and `DFF::toString` float/double implementations now use `fmt::format` for speed. A regression caused by the `stringstream` removal was fixed.
• The use of `std::map` has been largely switched to `std::unordered_map` where possible, particularly for task graphs.
• Performance flags have been added to Windows builds on CMake.
• Mimalloc support has been added to Windows builds, with an optional CMake flag.
• Previously missing licensing details for certain libraries used by the application have been included.
• Grammar in error messages has been improved, replacing “can not” with “cannot”.

• Functions from ProxyFunctorUtils, CellTypeInfo, LogTagInfo, ScriptUtils, SubmodelUtils, NameUtils, and StringUtils namespaces are now exported to Java to emulate classes with static members.
• Some Java exported functions were moved from `SubmodelUtils` and `ScriptUtils` namespaces to the `Script` class in Java.
• Projection-related functions are now exported as static members of the `Projection` class in Java.
• Version comparison functions (for strings) are exported as static members of the `Version` class in Java.
• Functionality to verify coordinator availability has been added and exposed to Java.
• The Java `dff.LogMessage` now uses `char[]` instead of `string` to speed up logging.
• Normalization functions have been exported to Java.

• Massive optimization of `CalcPatchLabelMap` execution by completely re-implementing the functor for better performance, including a new flag for sequential labels. The test case for `CalcPatchLabelMap` was also changed to ensure identical results on computers with different core counts.
• Tasks added to task groups can now be dispatched immediately.
• `ExtractCategoricalMapAttributes` execution now has extra parallelism, with mode, median, average/variance/stddev computed concurrently. It was further optimized by parallelizing the insertion of attributes into the table.
• Displaying tables is faster by using existing indices instead of copying them again.
• `LookupTable` functions have been optimized.
• The event indicating that a port received new data was removed due to unnecessary performance degradation.
• The way data is copied when transferred between data holders has been changed to avoid performing the copy during `getData()`, preventing some copies from being performed more than once.
• Table optimizations are ongoing.
• Basic data types used for port data transfer have been refactored and optimized, eliminating the need for casting, improving construction/destruction efficiency, and simplifying inheritance.
• Functors (e.g., Muxes) now forward input data directly instead of creating new instances, reducing overhead.
• `std::map` has been replaced with `gtl::btree_map` in `LookupTable`, `Categorization`, `SparseRasterImage`, and `Properties` experimentally for performance.
• Table normalization from the UI now uses tasks.
• Table's existing indices are used for `getSortedIndicesImpl()`.
• `fast-double-parser` is now used for parsing doubles when reading CSV tables.
• Direct writing to stream using `fmt::format` is used instead of `DFF::toString` for faster output.
• The `CalcHistogram` functor has been improved for better floating-point support and performance.
• The search for modules that was hurting MSVC performance on Windows has been disabled.

• Compiled expressions no longer cause memory errors.
• An error message issued when passing invalid data to a `RunRemotely` was fixed.
• A bug where transferring a Folder to `RunRemotely` would trigger an assertion has been fixed, and test cases were added.
• Variable renaming was performed to prevent declaration hiding and fix C4456 warnings.
• The conversion of some filenames to string resulting in relative paths instead of full ones has been fixed.
• Patcher and Expander now use the random number generator from the current worker.
• A memory leak on Distributed (the context was never destroyed) has been fixed.
• Linux test problems were fixed.
• A fix was added to GDAL projections on the Linux setup script.
• A problem where Python could throw an assertion when strings couldn't be converted to UTF-8 has been fixed.
• A problem where tables with a single column could be read with one less row has been fixed.
• A bug where the workdir property for bundle scripts was not properly propagated to local submodels has been fixed.
• A compilation error related to `TableDescriptionColumn` not having a default constructor in Java binding generation has been fixed.
• `RunRemotely` now clears both the socket and context after running.
• Python not outputting directly to the console has been fixed, along with UTF-8 string support for outputs.
• Windows External process now identifies and prints UTF-8 outputs properly.
• Matrices are now written in scripts to ensure transitions are always sorted.
• Functor storage by tasks is now truly protected against concurrent writes by workers and reads by the graphical interface.
• A bug related to parallel constructs using `FOR_EACH_ELEMENT` (container being copied instead of using a pointer) has been fixed.
• System process management of pipes has been fixed, and `Calculate Python Expression` can now catch and report unhandled exceptions.
• A duplicate const in `calcDistanceMap2` function declaration has been fixed.
• Bugs in the subtable implementation have been fixed, including an optimization to move column chunks instead of copying rows.
• Unreadable tables that don't have a new line are now fixed by appending a `\n` character after reading from disk.
• A crash when visualizing images formed by a complex tree of images has been fixed.
• Invalid validation when retrieving Region Managers has been fixed.
• Compatibility with `vcpkg 2025.04.09` and `msvc 17.14` has been fixed.
• Forgotten Table locks have been fixed.
• Forgotten GTL dependencies were fixed.
• GDAL initialization was fixed to prevent throwing an exception for a check.
• A problem with a default value not specified on `Table::addColumn` has been fixed.
• Comments in `Type.h` were fixed.
• An inconsistency where a viewer could retain outdated content if its equivalent peer did not have data during the last model execution has been fixed.
• The behavior when transferring data to the viewers was changed to avoid the possibility of data being transferred twice. Map viewers were updated to ensure deferred maps can still be visualized.
• A bug where copying a temporary image would always incorrectly assume the first layer was being copied has been fixed.
• Properly defined copy constructors and assignment operators for maps ensure cloning instances of `MapData` and `CategoricalMapData` preserves their contents and behavior.
• The categorization assignment mechanism for region masks has been redesigned to avoid failures in edge cases.
• A bug where `TEST_CHECK_EQUAL()` would abort subsequent tests on failure has been fixed.
• Unnecessary parameter assignment to port “shapeAttributeName” in functor `RasterizeShape` was removed.
• Image validation that could incorrectly reject valid Lat/Long images covering Earth's poles has been removed.
• A bug causing models created before version 8.x to ignore the `keepNonRegionCells` flag when reading models using `RegionalizeCategoricalMap` has been fixed.
• The documentation for the `optimizeScript` flag in the `Test::runScript` function was fixed.
• A bug that could trigger an assertion when visualizing a port while submodel expansion and redundant carrier removal were enabled simultaneously has been fixed, with improved robustness when handling carriers.
• A forgotten include for `linux statvfs` has been added.
• A potential crash when joining subtables through the task system is being temporarily worked around.

• Tables now allow any name format on their column titles.
• Custom tabs have been added when editing the `AddTableColumn` input ports.
• Assertions and adjusted parameter order in `ExtractCategoricalMapAttributesFunctor` ensure consistency.
• `CalculatePython` port data fetching and `getInputPortValue2` documentation were improved.
• Initial work on merging subtables using sub-table indices.
• Optimization to copy subtables as-is if they exist.
• Tasks are used for subtable merging where possible, and a bug where inserted subtable rows would get the wrong line number in the table index has been fixed.
• A hash for Tuples of any size (using `boost::hash_combine`) was added.
• Subtable implementation bugs have been fixed, and an optimization to move column chunks instead of copying rows has been added.
• The table is normalized before obtaining its indices.
• The table writer normalizes the table before writing it.
• Table writing uses the table indices for the order of keys.
• A custom `AtomicMutex/Lock (SpinLock)` has been added to Table to allow locking/unlocking between different threads. These were renamed to `TaskMutex/TaskLock` with guards to ensure proper usage. Recursive support was removed from `TaskMutex`.
• GTL is used for the map implementation of Table.
• Large parts of Table were rewritten to use `*Impl` functions and lock just once.
• A formatter for `Tuple/TupleView` has been added.
• Tasks on the table constructor have been disabled until the `ScriptReader` lock is fixed.
• `StringUtils::trim` has been added and used for trimming table parser read strings.
• The construction of `Dinamica::None` was refactored to be explicit.
• `DFF::getDataValue()` was introduced to uniformly extract the internal value from a `BaseData` instance.
• Data converters (String to Code, Code to String and String to Date) that can be automatically defined were removed.
• Categorization has been added to all maps representing region masks.
• Refactoring of region mask definition ensures it uses the right values for categorization classes.

• Error reporting on tests has been fixed.
• Console arguments that print information (version, list-accelerators, list-functors) no longer require an additional argument.
• Threads corresponding to each worker have been assigned a description name.
• Additional information about the current step is now shown as progress for the cost calculation.
• Python scripts from the “Calculate Python Expression” are now executed as independent sub-processes.
• Dinamica Python can now detect termination requests and lock if pip is running.
• Python map tests and Dinamica module Python dependencies/embedded Python integration have been removed.
• Thread naming on Linux and Linux SystemProcess waiting times have been fixed.
• Linux system process now uses default environment variables if none are specified.
• Assertions have been placed on `zeromq` context termination.
• Scripts have been added to configure the build, compile source files, and install libraries and utilities all at once.
• Dinamica's Python path and subfolder Scripts have been added to the Python process.
• An `advanced optional input port` for packages to be installed using PIP has been added to `CalculatePythonExpression`, and its documentation improved.
• Python error reporting functions have been improved.
• An option to force the generation of core dumps when the application crashes has been added.
• Dynamic debugging support has been added to CMake.
• A script to compare test times with a reference has been added, reporting changes due to regression or improvement, with options for limits and filtering.
• The `Distributed/TooManyAgentsAndRunRemotely` test was removed from fast tests due to crashes.
• Options to filter the tests to be analyzed by execution time and to select which results to display were added.
• Initial implementation of run remotely requirements.
• `getTemporaryPathFreeSpace` has been added.
• `std::atomic` has been replaced by `std::atomic_flag` in distributed components.
• Agent IDs and tempspacegb have been added to requirement expressions.
• A timeout limit to `RunRemotely` has been added.
• Safety in `RegionManager` has been improved by refactoring code to correctly distinguish between pointers and references.
• The `GetRegionInfo` functor has been introduced for accessing region details, including direct access to the mask image.
• Test cases for functor `GetTableKeys` were improved.
• Test cases were added to test the behavior when running another real world model under different configurations.
• A forgotten include for `linux statvfs` has been added.
• The Python environment reset now attempts to restart the application if Python libraries are in use.
• `ProgressReporter` and related functions were refactored to be available for the Base library too.
• Error messages related to native expression compilation were updated.

• Carrier necessity logic has been refactored into a reusable utility method.
• The check for carrier necessity has been extended to support multiple output-port connections, removing the single-peer limitation.
• Redundant carrier utilities have been centralized in the C++ codebase, and a method to remove all redundant carriers from the script has been created with unit tests.
• The command-line flag controlling the removal of redundant carriers now enables the feature instead of disabling it.
• An action to remove all redundant carriers from the script was added.
• A flag in `ScriptRunner` allows automatic removal of redundant carriers when `expandSubmodel` is active, with console support.
• The interface refresh logic after connection adjustments has been improved.
• `reconnectAllFunctors()` was introduced to refresh interface connections after a removal.
• A constraint was added to suggest redundant carrier connection adjustments only when removing a single functor, with more complex cases handled by the dedicated utility.
• The carrier removal process has been simplified by merging the redundancy check into a single dialog with clear options: “Remove and Readjust”, “Just Remove”, or “Cancel”.
• The `removeRedundantCarriers` flag handling has been included in the interface.

(Generated using Google NotebookLM)