Runs Compare

<< Click to Display Table of Contents >>

Navigation:  Reference Manual > Toolsets, Tools, and Causal Tracing > Analysis Tools > Dataset Analysis Tools >

Runs Compare

Previous pageReturn to chapter overviewNext page

The Runs Compare tool compares the values of variables between the first and second loaded datasets.  If the values of variables are different, or if variables in one run are missing from the other run, this is reported.  If one or both datasets are not from a simulation (e.g., imported data), the compare proceeds with reduced detail.

The Runs Compare tool is most useful when the runs are from the same, or very similar models.  The order in which the runs are loaded can influence the order in which differences are reported, but will not change the content.  

Runs Compare Options

The Run Compare tool lets you select which type of variables you would like to compare, and the threshold of difference that is required before any differences are reported.

Show Differences for Type allows you to select the different types of variables to show differences for.  Note that Gaming is actually a subset of Auxiliary.  It is included as a separate category for comparison of Gaming simulations.

Thresholds before differences are displayed lets you specify these thresholds.  The difference must exceed the value for Relative times the original value and also exceed the value for Absolute before the difference is reported.  By default both of these are set to zero so that any difference is displayed.

Run Compare Output

If another runs comparison window is open when the Runs Compare tool is invoked, the old comparison is deleted.  The Runs Compare tool either creates a table window describing the differences, or reports that no differences were found.  

Run names

Comparing  logistic and rattus

Run metadata

Information about run logistic
Origin: (Mon Mar 02 11:51:46 2015) From LogisticTester.mdl
Command line: sim  logistic -I
Information about run rattus
Origin: (Mon Mar 02 11:51:46 2015) From LogisticTester.mdl
Command line: sim  rattus -I

Differences

VariableType        Difference        At Time        logistic        rattus        N Diffs        N NA Diffs        MAD        MAD%Mean        MA%D        MaxAD        MaxAD%M        MaxA%D        Mean RSSE        Slow RSSE        Fast RSSE        
rConstant        value                3.875        2.9000001        
xLevel        from time        1        0.038362499        0.02871        400        0        0.24059731        38.3        32.8        0.59847111        95.2        82.6        0.01654        0.005587        0.9731        

For constants, the tool shows the value in each simulation. For dynamic variables, it shows the following:

Type - the type of difference (value difference, missing variable, etc.)

Values - the first pair of different values, if any

At Time - the time at which the first difference occurs

N Diffs - total number of differences (including those below the tool's threshold settings)

N NA Diffs - instances where one series has a value and other is NA (where other statistics would be undefined)

MAD - mean absolute difference

MAD%Mean - mean absolute difference, divided by the mean of the absolute value of the defined points in both series

MA%D - mean percent difference (i.e. MAPE)

MaxAD - largest absolute difference

MaxAD%M - largest absolute difference, divided by the mean of the absolute value of the defined points in both series

MaxA%D - largest absolute percent difference

Mean/Slow/Fast RSSE - a heuristic decomposition (via wavelets) of differences into slow and fast components, useful for distinguishing substantial features from noise

 

Note that the statistics are computed over all differences, not just those that exceed the thresholds.

The table can be copied and pasted into a spreadsheet for easy sorting and filtering.

The  Runs Compare tool first tests the first loaded run against the second and reports differences by type in alphabetical order.  After doing this, it checks to see if the second loaded run has any variables of the selected type that are not in the first run, or are of a different type in the first run.  Thus after getting a number of messages about differences between the first and second run, you might also get some messages about the second and first run that are out of alphabetical order.