Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
weights_of_evidence_calibration [2026/08/10 03:01] hermann |
weights_of_evidence_calibration [2026/08/12 01:22] (current) hermann |
||
|---|---|---|---|
| Line 9: | Line 9: | ||
| Weights of Evidence is a Bayesian method, originally used in geology to map favorability for events such as mineralization, adapted here to relate a land-use transition to a set of predictive spatial variables. | Weights of Evidence is a Bayesian method, originally used in geology to map favorability for events such as mineralization, adapted here to relate a land-use transition to a set of predictive spatial variables. | ||
| - | For a transition from category i to category j, each class of a predictive variable (a category, or a range of a continuous gray-tone variable) is assigned two numbers: a positive weight (W+), reflecting how the transition's likelihood changes for cells that fall inside that class, and a negative weight (W-), reflecting the same for cells that fall outside it. | + | For a transition from category i to category j, each class of a predictive variable (a category, or a range of a continuous gray-tone variable) is assigned two numbers: a positive weight (W<sup>+</sup>), reflecting how the transition's likelihood changes for cells that fall inside that class, and a negative weight (W<sup>-</sup>), reflecting the same for cells that fall outside it. |
| [[calc_w._of_e._probability_map|Calc W. Of E. Probability Map]] combines these weights with a transition's baseline probability to produce a probability for each cell, based on which range or category the cell falls into for each variable. This is why a weight's sign has a consistent meaning: | [[calc_w._of_e._probability_map|Calc W. Of E. Probability Map]] combines these weights with a transition's baseline probability to produce a probability for each cell, based on which range or category the cell falls into for each variable. This is why a weight's sign has a consistent meaning: | ||
| Line 17: | Line 17: | ||
| * **Negative** -- lowers the probability below baseline. Cells in that range are less likely than average to make the transition -- the range repels it. | * **Negative** -- lowers the probability below baseline. Cells in that range are less likely than average to make the transition -- the range repels it. | ||
| - | **Contrast** (C = W+ - W-) measures a range's overall association with the transition: it is what determines whether the range meaningfully favors or repels the transition, or has close to no effect. This is the number the significance test applies to (see [[#determining_coefficients|Determining Coefficients]] below). A range with a Weight_Coefficient near zero and a non-significant Contrast are two ways of saying the same thing: the range isn't doing meaningful explanatory work for the transition. | + | **Contrast** (C = W<sup>+</sup> - W<sup>-</sup>) measures a range's overall association with the transition: it is what determines whether the range meaningfully favors or repels the transition, or has close to no effect. This is the number the significance test applies to (see [[#determining_coefficients|Determining Coefficients]] below). A range with a Weight_Coefficient near zero and a non-significant Contrast are two ways of saying the same thing: the range isn't doing meaningful explanatory work for the transition. |
| Every variable's weights combine together to produce a transition's overall probability at each cell, which is why the method requires its input variables to be spatially independent of one another (see [[#checking_variable_independence|Checking Variable Independence]] below). | Every variable's weights combine together to produce a transition's overall probability at each cell, which is why the method requires its input variables to be spatially independent of one another (see [[#checking_variable_independence|Checking Variable Independence]] below). | ||
| Line 41: | Line 41: | ||
| These four parameters apply only to continuous gray-tone variables. A variable marked Category in the Skeleton does not use them at all -- increment, the two deltas, and the tolerance angle have no meaning for a variable that is already classified. | These four parameters apply only to continuous gray-tone variables. A variable marked Category in the Skeleton does not use them at all -- increment, the two deltas, and the tolerance angle have no meaning for a variable that is already classified. | ||
| + | |||
| + | > **Note:** When written as an EGO Script literal, a continuous variable's parameters appear in this order: Increment, **Maximum** Delta, **Minimum** Delta, Tolerance Angle -- the two deltas are reversed from the order shown in the table above. See "Example: A Basic Calibration Model" below for the exact syntax. | ||
| ===== Determining Ranges for a Continuous Variable ===== | ===== Determining Ranges for a Continuous Variable ===== | ||
| Line 52: | Line 54: | ||
| Determine Weights Of Evidence Ranges finds range boundaries by watching how the transition's weight changes as it sweeps across a continuous variable's values, rather than requiring pre-set cut points. | Determine Weights Of Evidence Ranges finds range boundaries by watching how the transition's weight changes as it sweeps across a continuous variable's values, rather than requiring pre-set cut points. | ||
| - | Starting at the variable's minimum value, it grows a sequence of nested buffers outward in steps equal to the Increment: the first buffer covers up to one Increment past the minimum, the second covers up to two Increments past it, and so on, each one containing everything the previous one did plus a bit more. For each buffer, it counts how many cells fall inside it and how many of those cells actually made the transition, and from that pair derives the weight (W+) that buffer's edge would produce if used as a single cutoff. Sweeping through the whole sequence of buffers produces a curve: weight as a function of how far the buffer has grown. | + | Starting at the variable's minimum value, it grows a sequence of nested buffers outward in steps equal to the Increment: the first buffer covers up to one Increment past the minimum, the second covers up to two Increments past it, and so on, each one containing everything the previous one did plus a bit more. For each buffer, it counts how many cells fall inside it and how many of those cells actually made the transition, and from that pair derives the weight (W<sup>+</sup>) that buffer's edge would produce if used as a single cutoff. Sweeping through the whole sequence of buffers produces a curve: weight as a function of how far the buffer has grown. |
| Where this curve stays flat, the variable isn't discriminating anything at that point -- going a bit further out doesn't change the odds. Where the curve bends, that's where crossing that value of the variable starts to matter. So the range boundaries are wherever the curve changes direction, which is a shape-finding problem, not something that can be set analytically in advance. | Where this curve stays flat, the variable isn't discriminating anything at that point -- going a bit further out doesn't change the odds. Where the curve bends, that's where crossing that value of the variable starts to matter. So the range boundaries are wherever the curve changes direction, which is a shape-finding problem, not something that can be set analytically in advance. | ||
| Line 104: | Line 106: | ||
| * **Possible_Transitions** -- the number of cells that could have made this transition, given they started in the "from" category and fell inside this variable's range. What counts as "possible" is controlled by ''potentialTransitionCriteria'' (see below). | * **Possible_Transitions** -- the number of cells that could have made this transition, given they started in the "from" category and fell inside this variable's range. What counts as "possible" is controlled by ''potentialTransitionCriteria'' (see below). | ||
| * **Executed_Transitions** -- of those possible cells, how many actually made the transition during the observed period. | * **Executed_Transitions** -- of those possible cells, how many actually made the transition during the observed period. | ||
| - | * **Weight_Coefficient** -- the resulting weight (W+) for that range or category: its contribution to the transition's probability wherever it applies. | + | * **Weight_Coefficient** -- the resulting weight (W<sup>+</sup>) for that range or category: its contribution to the transition's probability wherever it applies. |
| - | * **Contrast** -- W+ minus W- for that range or category. Positive values favor the transition, negative values repel it, and values near zero mean the range has little effect. See [[#what_is_weights_of_evidence|What Is Weights of Evidence]] above for how this is derived. | + | * **Contrast** -- W<sup>+</sup> minus W<sup>-</sup> for that range or category. Positive values favor the transition, negative values repel it, and values near zero mean the range has little effect. See [[#what_is_weights_of_evidence|What Is Weights of Evidence]] above for how this is derived. |
| * **Significant** -- whether the Contrast passes the 95% confidence threshold (|C| > 1.96 S(C)). Rows marked "No" are exactly the ones the range-tuning loop above targets when adjusting the minimum delta. | * **Significant** -- whether the Contrast passes the 95% confidence threshold (|C| > 1.96 S(C)). Rows marked "No" are exactly the ones the range-tuning loop above targets when adjusting the minimum delta. | ||
| Line 120: | Line 122: | ||
| Weights of Evidence's one formal requirement is that the input variables be spatially independent -- if two variables are highly correlated, their combined contribution double-counts the same spatial signal. | Weights of Evidence's one formal requirement is that the input variables be spatially independent -- if two variables are highly correlated, their combined contribution double-counts the same spatial signal. | ||
| - | [[determine_weights_of_evidence_correlation|Determine Weights Of Evidence Correlation]] checks this pairwise, for every candidate variable at once, against a chosen transition. Required inputs: ''ranges'' (Weights) and ''transition'' (TransitionSet, editable). Output: ''report'' (Table), with pairwise statistics including Chi-square, Cramer's V, Contingency, Entropy, and Joint Information. | + | [[determine_weights_of_evidence_correlation|Determine Weights Of Evidence Correlation]] checks this pairwise, for every candidate variable at once, against a chosen transition. Required inputs: ''ranges'' (Weights -- the same output produced by [[#determining_ranges_for_a_continuous_variable|Determining Ranges for a Continuous Variable]] above) and ''transition'' (TransitionSet, editable). Output: ''report'' (Table), with pairwise statistics including Chi-square, Cramer's V, Contingency, Entropy, and Joint Information. |
| There is no fixed universal threshold for "too correlated" -- it's a judgment call based on the report values. When a pair of variables shows high correlation across these measures, drop one of them or combine both into a new derived variable, then re-run the range and coefficient calibration for the surviving variable set. | There is no fixed universal threshold for "too correlated" -- it's a judgment call based on the report values. When a pair of variables shows high correlation across these measures, drop one of them or combine both into a new derived variable, then re-run the range and coefficient calibration for the surviving variable set. | ||
| Line 126: | Line 128: | ||
| ===== Reusing and Saving Weights ===== | ===== Reusing and Saving Weights ===== | ||
| - | [[SaveWeights|Save Weights]] writes either the ranges or the final coefficients to a file, with the format inferred from the filename extension. For calibrations that vary across model steps, [[NumberWeights|Number Weights]] / [[NumberWeightsFilename|Number Weights Filename]] tag a Weights value or filename with a step number, and the corresponding [[SelectWeights|Select Weights]] / [[SelectWeightsFilename|Select Weights Filename]] container functors pick the right one back out by step during execution. | + | [[save_weights|Save Weights]] writes either the ranges or the final coefficients to a file, with the format inferred from the filename extension. For calibrations that vary across model steps, [[number_weights|Number Weights]] / [[number_weights_filename|Number Weights Filename]] tag a Weights value or filename with a step number, and the corresponding [[select_weights|Select Weights]] / [[select_weights_filename|Select Weights Filename]] container functors pick the right one back out by step during execution. |
| ==== Weights file format ==== | ==== Weights file format ==== | ||
| - | When [[SaveWeights|Save Weights]] writes a Weights object to CSV, it uses a five-column table: ''From'', ''To'', ''Variable'', ''Range_Lower_Limit'', ''Weight''. The upper limit of each range is not stored explicitly -- it is inferred as the next row's ''Range_Lower_Limit'' for the same transition and variable, with the final range in each group extending to the maximum representable value. | + | When [[save_weights|Save Weights]] writes a Weights object to CSV, it uses a five-column table: ''From'', ''To'', ''Variable'', ''Range_Lower_Limit'', ''Weight''. The upper limit of each range is not stored explicitly -- it is inferred as the next row's ''Range_Lower_Limit'' for the same transition and variable, with the final range in each group extending to the maximum representable value. |
| A six-column variant is also accepted when loading a Weights CSV, with an explicit ''Range_Upper_Limit'' column inserted between ''Range_Lower_Limit'' and ''Weight''. This form is useful for hand-edited or externally generated weight tables, but ''Save Weights'' itself only ever produces the five-column form. | A six-column variant is also accepted when loading a Weights CSV, with an explicit ''Range_Upper_Limit'' column inserted between ''Range_Lower_Limit'' and ''Weight''. This form is useful for hand-edited or externally generated weight tables, but ''Save Weights'' itself only ever produces the five-column form. | ||
| Line 150: | Line 152: | ||
| | ''probabilities'' | Output | Map | -- | The resulting spatial probability map. | | | ''probabilities'' | Output | Map | -- | The resulting spatial probability map. | | ||
| - | That probability map, together with a change matrix derived from the transition rates (via [[Calc Change Matrix|Calc Change Matrix]]), is what [[Patcher|Patcher]] and [[Expander|Expander]] consume to allocate the simulated changes across the landscape. In a multi-step simulation running inside a [[Repeat|Repeat]] loop, ''Calc W. Of E. Probability Map'' is typically recalculated every step against the updated landscape, since dynamic variables (such as distance to previously changed cells) change as the simulation progresses. | + | That probability map, together with a change matrix derived from the transition rates (via [[calc_change_matrix|Calc Change Matrix]]), is what [[patcher|Patcher]] and [[expander|Expander]] consume to allocate the simulated changes across the landscape. In a multi-step simulation running inside a [[repeat|Repeat]] loop, ''Calc W. Of E. Probability Map'' is typically recalculated every step against the updated landscape, since dynamic variables (such as distance to previously changed cells) change as the simulation progresses. |
| ===== Example: A Basic Calibration Model ===== | ===== Example: A Basic Calibration Model ===== | ||
| This example ties the pieces above together: one continuous variable, one categorical variable, a range calculation, a coefficient calculation, and saving both results. | This example ties the pieces above together: one continuous variable, one categorical variable, a range calculation, a coefficient calculation, and saving both results. | ||
| + | |||
| + | Each variable here is loaded as its own separate map, connected to its own [[name_map|Name Map]]. Every map, cube or not, is organized into named [[map_type|layers]] -- a single, non-cube map defaults to one layer named ''layer_0'', but the functor that produced the map can assign its own name instead: [[calc_distance_map|Calc Distance Map]], for instance, names its output ''distance_to_<category>'' rather than using the default. [[extract_map_layer|Extract Map Layer]]'s own editor lists a map's actual layer names, which is the place to check when in doubt, and it also pulls a specific layer out on its own if one is needed outside a Skeleton reference. It's also possible to assemble every predictive variable into a single multi-layer cube via [[create_cube_map|Create Cube Map]], the way Lesson 18's own example does -- but for a handful of variables that adds a layer of indirection without much benefit. A cube's usefulness shows up with many static variables: one [[name_map|Name Map]] alias then covers the whole set, and the model loads and connects one file instead of one per variable. Inside a cube, each layer's name comes from [[number_and_name_map|Number And Name Map]] rather than plain Name Map, since it's assigning both an index and a name to a single layer as it's assembled. | ||
| <code> | <code> | ||
| Line 169: | Line 173: | ||
| protectedAreas := LoadCategoricalMap "protected_areas.tif"; | protectedAreas := LoadCategoricalMap "protected_areas.tif"; | ||
| - | // The Skeleton: one entry per variable, written "alias/layerName". A map | + | // The Skeleton: one entry per variable, written "alias/layerName". The |
| - | // that does not explicitly define its own layer names uses the default | + | // alias is whatever name is given to that map's Name Map hook below -- |
| - | // "layer_0" for its first (here, only) layer. A continuous gray-tone | + | // this is how each container matches a Skeleton entry back to the actual |
| - | // variable is followed by four numbers, in this order: Increment, | + | // map connected to it. "layer_0" is only the default layer name for a |
| - | // Maximum Delta, Minimum Delta, Tolerance Angle -- see "The Weights of | + | // map that doesn't define its own -- check with Extract Map Layer's |
| - | // Evidence Skeleton" above. A category variable takes no numbers. | + | // editor if a map's actual layer name is in doubt. A continuous |
| + | // gray-tone variable is followed by four numbers, in this order: | ||
| + | // Increment, Maximum Delta, Minimum Delta, Tolerance Angle -- see "The | ||
| + | // Weights of Evidence Skeleton" above. A category variable takes no | ||
| + | // numbers. | ||
| skeleton := WeightsOfEvidenceSkeleton [ | skeleton := WeightsOfEvidenceSkeleton [ | ||
| 2->1 [ | 2->1 [ | ||
| Line 182: | Line 190: | ||
| ]; | ]; | ||
| - | // Step 1: determine ranges for the continuous variable(s) | + | // Step 1: determine ranges for the continuous variable(s). Name Map |
| + | // assigns each connected map the alias used above in the Skeleton. | ||
| ranges := DetermineWeightsOfEvidenceRanges { | ranges := DetermineWeightsOfEvidenceRanges { | ||
| initialLandscape = initialLandscape, | initialLandscape = initialLandscape, | ||
| Line 210: | Line 219: | ||
| </code> | </code> | ||
| - | The saved ''coefficients.dcf'' is the finished Weights object -- reload it with [[LoadWeights|Load Weights]] wherever [[calc_w._of_e._probability_map|Calc W. Of E. Probability Map]] needs it (see [[#using_the_weights_downstream|Using the Weights Downstream]]). The saved ''report.csv'' can be opened directly in a spreadsheet to review ''Contrast'' and ''Significant'' per range, without needing to scroll the message log. | + | The saved ''coefficients.dcf'' is the finished Weights object -- reload it with [[load_weights|Load Weights]] wherever [[calc_w._of_e._probability_map|Calc W. Of E. Probability Map]] needs it (see [[#using_the_weights_downstream|Using the Weights Downstream]]). The saved ''report.csv'' can be opened directly in a spreadsheet to review ''Contrast'' and ''Significant'' per range, without needing to scroll the message log. |
| ===== Tips ===== | ===== Tips ===== | ||
| * A model with more than one transition needs its own ranges and coefficients per transition; the range parameters from one transition can be copied as a starting point for another, but each must still be tuned independently. | * A model with more than one transition needs its own ranges and coefficients per transition; the range parameters from one transition can be copied as a starting point for another, but each must still be tuned independently. | ||
| - | * The [[Name Map|Name Map]] aliases used inside [[determine_weights_of_evidence_ranges|Determine Weights Of Evidence Ranges]] and [[determine_weights_of_evidence_coefficients|Determine Weights Of Evidence Coefficients]] must match exactly between the two containers -- this is how each container knows which named map corresponds to which variable in the Skeleton. | + | * The [[name_map|Name Map]] aliases used inside [[determine_weights_of_evidence_ranges|Determine Weights Of Evidence Ranges]] and [[determine_weights_of_evidence_coefficients|Determine Weights Of Evidence Coefficients]] must match exactly between the two containers -- this is how each container knows which named map corresponds to which variable in the Skeleton. |
| - | * To inspect or chart a report outside the message log, connect its ''report'' output to [[Save Table|Save Table]] and export to CSV. | + | * To inspect or chart a report outside the message log, connect its ''report'' output to [[save_table|Save Table]] and export to CSV. |