If you do not want to use the defaults you can set options in the Optimization Control dialog or directly in the Optimization Control file. In the file these options must precede the search specification parameters and take the form :KEYWORD=value.
As always, capitalization and spacing are ignored by Vensim. Options must precede the parameter declarations, but otherwise order is not meaningful. All options can be shortened to four characters. In addition, you can include comments inside of braces {}. If repeated or conflicting options are specified, only the last are used.
Comments can be placed in optimization control files using the :COMMENT keyword or {} This lets you put in remarks which are ignored by the system. The remark will be repeated in runname.out.
If you use the Optimization Control Dialog all of the keywords will be entered automatically. The keywords have (approximately) the same names as the fields in this dialog. They are reported in the order they appear in the dialog.
NOTE comments are not supported by the Optimization Control dialog and will be lost if you edit a file containing comments with this dialog.
:OPTIMIZER = POWELL {POWELL | OFF}
The default option is POWELL, which denotes a modified Powell search. When set to OFF, no optimization is done. This is useful when the MULTIPLE_START option is activated. For example, :MULTIPLE_START=GRID and :OPTIMIZER=OFF performs a grid search of increasing definition starting from the specified values. This can be used to test the global validity of an optimum or the sensitivity of the model to the search parameters outside normal operating ranges.
:SENSITIVITY=OFF { OFF | PAYOFF_VALUE{=4.0} |
PAYOFF_PERCENT{=10.0} |
PARAMETER_PERCENT{=10.0} |
ALL_CONSTANTS{=10.0}}
This option allows you to find out how sensitive the payoff is to changes in parameter values. The default setting is off, which does not invoke the sensitivity calculations. Otherwise, there are four modes of use: PAYOFF_VALUE, PAYOFF_PERCENT, PARAMETER_PERCENT, and ALL_CONSTANTS.
The sensitivity analysis done here is not part of the sensitivity analysis discussed earlier in this chapter though they are closely related conceptually. The terminology, unfortunately, overlaps somewhat and where clarity is required we will refer to optimizer sensitivity analysis.
PAYOFF_VALUE and PAYOFF_PERCENT make sense only at an optimum. When invoked, they run through the parameter list, and determine how much each individual parameter would have to be changed to decrease the payoff by the desired value or percent. Positive and negative value changes are calculated separately, since they are not normally symmetric.
PAYOFF_VALUE is useful when the payoff is a true log likelihood or properly weighted sum of squares. In this case, different changes in the absolute value of the payoff can be used to derive confidence bounds on parameters. For example, for a sum of properly weight squares using the default PAYOFF_VALUE of 4 (the default) results in 95% confidence intervals being reported for each parameter. If you are using Kalman Filtering, which computes the actual log likelihood, you should use a PAYOFF_VALUE of 2. (The properly weight sum of squares is 2 times the log likelihood. It can be shown the log likelihood tends to a Chi Squared distribution with one degree of freedom and this can be used to determine confidence bounds for a single parameter.)
PAYOFF_PERCENT is useful when the payoff cannot be interpreted as a log likelihood, as, for example, in the case of policy. The resulting intervals serve to identify which policy levers need to be keenly controlled, and which can be largely ignored.
PARAMETER_PERCENT looks at a percentage change in parameters and reports the percentage change in payoff. This is faster to execute than PAYOFF_PERCENT and gives similar information if the payoff surface is well behaved. Also, it can be executed even if the system is not at an optimum, and still give meaningful results. The numbers reported show the change in the magnitude of the payoff as a result of increasing and decreasing each search parameter by the specified percent. If the model has been optimized, both of these numbers will be negative.
ALL_CONSTANTS is similar to PARAMETER_PERCENT but considers all model Constants, not just search parameters. This is a comprehensive diagnostic tool that has little to do with optimization as such. Again, it shows the change in the payoff as a result of the given percentage increases and decreases in all Constants. Since most Constants are not search parameters, there is no reason to expect these changes to be negative. At completion, a file named sortsens.tab is created, which lists the information in sensitivity.tab in order of importance.
The optimizer sensitivity analysis can be invoked in conjunction with an optimization or MULTIPLE_START, in which case the sensitivity will be determined relative to the point with maximum payoff. If the OPTIMIZER and MULTIPLE_START control parameters are turned off, the sensitivity will be performed relative to the input evaluation point.
Optimizer sensitivity analysis will not be performed if you have clicked on Cancel and no optimization has been completed. If an optimization has been completed and a break requested, then the sensitivity analysis will be performed about the maximum. If the sensitivity analysis is wanted after a break, but not performed because of a break, set the options OPTIMIZER=OFF and MULTIPLE_START=OFF in runname.out, copy it to a new file and use it as the optimization control file. The sensitivity will be done about the best point the optimizer has reached.
The output from the sensitivity analysis will be placed in a file called sensitivity.tab. For SENSITIVITY set to PAYOFF_VALUE or PAYOFF_PERCENT, the file will look like:
:COM The base payoff is 123.22
:SENSITIVITY = PAYOFF_PERCENT = 10
:COMMENT * Means a boundary was reached.
.21 <= FRAC_REQ_REW = .332 <= 1.0 *
26.5 <= NORM_QUALITY = 27 <= 27.21
10* <= WORK_SPEED_BASE = 16 <= 29
The first line reports the payoff value at the starting point. The second line just reiterates the sensitivity settings. The third line is a comment reporting that * means that a boundary has been hit. (That is, the difference in payoff can be less than specified at this point but the constraints in the optimization control file forbid moving this parameter any further). The remaining values give the parameter names with their starting values, surrounded by their values at the sensitivity limits.
This file is in a form to be used with the optimizer and the vector graphing option. During the calculations of the sensitivity bounds, no intermediate calculations will be posted to the screen nor will any output go to trace, startpoint, or endpoint files. The fractional tolerance for sensitivity calculations is set at 1.0e-7 to ensure high accuracy of these results.
For SENSITIVITY set to PARAMETER_PERCENT and ALL_CONSTANTS, sensitivity.tab will look like:
:COM The base payoff is 3.000000
:SENSITIVITY = PARAMETER_PERCENT = 10.000000
:COM Parameters are changed by +- 10% if 0 by += 0.1
PARAMETER -0.1 0.1
FRAC_REQ_REW = .332 -0.607316 -1.889160
NORM_QUALITY = 27 -0.0375092 -0.060395
WORK_SPEED_BASE = 16 -1.238576 -1.569352
Again, the first two lines are the payoff at the starting point and the type of sensitivity analysis performed. The next two comments tell the definition of elasticity as reported and what happens in special cases. The sensitivity results are given in table form, where the parameter name and its base value are given on the left and the changes in payoff that result from changes in the parameter are given on the right.
If SENSITIVITY is set to ALL_CONSTANTS, then another file named sortsens.tab is created. This file is basically the same as sensitivity.tab, but the constants are sorted by decreasing impact so that it is easier to weed out unimportant constants. If the optimization is interrupted with a Cancel command, the sorted file will not be created, but sensitivity.tab will.
:MULTIPLE_START=OFF {OFF | RANDOM | GRID | VECTOR}
This option restarts optimization multiple times from different starting points or, if :OPTIMIZER=OFF, simulates the model for different values of the search parameters.
If MULTIPLE_START is set to RANDOM the optimizer will never stop unless you click on the Cancel button to interrupt it. GRID will stop, but probably not after a reasonable time since the potential number of simulations with GRID is 1024 to the power of the total number of search parameters. When the optimizer receives the Cancel command, it shuts down cleanly, writing out the best values found to the file runname.out. The files startpoint.tab, startpoint.dat, endpoint.tab, endpoint.dat, and vector.dat (if VECTOR is specified) might also be created. These files and the conditions under which they are created are discussed in the section "Optimization Output."
If MULTIPLE_START is OFF, then a single optimization is performed.
If MULTIPLE_START is RANDOM, starting points for new optimizations are picked randomly and uniformly over the range of each parameter. A different random series for selecting these starting points can be generated by changing the value of :SEED. The first point used for optimization is always the initial setting of the search parameters.
If MULTIPLE_START is GRID, starting points are computed over successively finer grids over the range of each parameter. This first grid computes the endpoints of the ranges, the next grid also computes the midpoints, and each successive grid divides the previous grid by 2. This continues for up to 1024 divisions (which could take years!) unless it is interrupted.
If MULTIPLE_START is VECTOR, starting points are computed over uniformly partitioned values of each parameter. Unlike GRID, VECTOR changes only one parameter at a time. VECTOR searches the first parameter from its minimum to maximum values, then the second, and so on. The partition is the maximum - minimum value over the setting of VECTOR_POINTS.
The ranges for parameters are specified as constraints on the parameters being optimized. If constraints are missing, the maximum and minimum available floating point numbers (roughly 1E34) are used as for GRID and VECTOR. These are not typically good choices unless you are just testing extreme conditions. A grid search on an unconstrained pattern would start at -1E34, then do 1E34, then do about 0.0, then do -1E16 and so on. The number of points it hits that are sensible is likely to be small in this circumstance.
For RANDOM, the following rules are used to compute search starting points:
· Upper and lower constraints present:
min <= parameter <= max
A number, X, will be chosen in the range [0,1) (including 0 but not including 1.0) . The starting value for the parameter will be:
parameter = min + X(max - min)
· Only the lower constraint present:
min <= parameter
A number, X, will be chosen in the range [0,1) (including 0 but not including 1.0). The starting value for the parameter will be:
parameter = min + 1/(1 - X)
· Only the upper constraint present:
parameter <= max
A number, X, will be chosen in the range [0,1) (including 0 but not including 1.0). The starting value for the parameter will be:
parameter = max - 1/(1 - X)
· Neither constraint present:
parameter
A number, X, will be chosen in the range (-1,1) exclusive. The starting value for the parameter will be:
parameter = 1/(1 - X) - 1/(1 + X)
Two files, startpoint.dat and startpoint.tab, will be created containing the starting values for the parameters and the payoffs at these starting values. If the OPTIMIZER=POWELL option is set, two additional files endpoint.dat and endpoint.tab will be created, showing the parameter values and payoffs that result from these starting conditions. These files contain the same information in different formats. The .dat files can be loaded into the environment using Dat2vdf for further review.
If MULTIPLE_START=VECTOR is used and OPTIMIZER=OFF, an additional file named vector.dat will be created listing the payoff as a function of each parameter value. Again, this can be loaded into the environment using Dat2vdf, although the graphs look odd if more than one search parameter is specified because of the incommensurate time axes. This vector.dat information is very useful for viewing separate cuts of the likelihood surface.
:RANDOM_NUMBER=LINEAR (LINEAR|SUBTRACTIVE)
This option applies only if :MULTIPLE_START=RANDOM is used. There are two random number generators which can be used with MULTIPLE_START: LINEAR and SUBTRACTIVE. The default generator is LINEAR.
NOTE Both of these techniques are based on routines given in Numerical Recipes in C, by Press, Flannery, Teukolsky, and Vetterling. If you are not certain that MULTIPLE_START is sampling the entire space, then switch to the other random number generator and try again.
If RANDOM_GENERATOR is LINEAR, three linear-congruent generators are used. The first one generates the most significant portion, the second generates the least significant portion, and the third randomly picks one from a list of previously generated numbers.
If RANDOM_GENERATOR is SUBTRACTIVE, the subtractive method of Knuth is used.
:SEED =1
This option is only applicable if MULTIPLE_START=RANDOM. This is a positive integer needed to initialize the random number generator for the optimizer. The default value is 1. Any other value will produce a different random sequence. (Simulation and optimization use the same algorithm for generating random numbers, but represent separate invocations. For each simulation, the random number generator is reset and regenerates the same values.)
:RESTART_MAX=0
This option is applicable when MULTIPLE_START is set to RANDOM or GRID It determines the maximum number of times the optimizer will restart. By default RANDOM and GRID will continue until they are interrupted. Specify a positive number to have them restart only that number of times.
:OUTPUT_LEVEL=1 {0 | 1 | 2 | 3 | 4 | 5 | 6}
This determines the amount of information the optimizer displays as it is running. This information will go to the screen and to the error file for the simulation. It takes a value between 1 and 6. Higher numbers give more information, with each higher level including all the information from the lower levels. The specific information follows:
0 Suppresses output. This is useful if MULTIPLE_START is turned on.
1 The starting point and final point of the optimization are given along with their payoffs.
2 Displays any point with its payoff which is an improvement over the best payoff calculated so far. The percent improvement is also displayed.
3 Displays the best point and payoff of every optimization, particularly useful with MWYS.
4 Displays best point and payoff of each iteration through the entire set of search vectors.
5 Displays the best point and payoff after each individual vector optimization.
6 Displays every point and payoff.
An asterisk (*) is printed in front of the parameters which have changed for the most recent simulation.
:TRACE=NO {NO{0} |YES{2} |3|4|5|6}
NO is a synonym for 0 and YES is a synonym for 2. This is valid only if MULTIPLE_START=OFF. This allows you to trace the optimization results. If trace is set to anything other than NO, then two files, trace.tab and trace.dat, will be created. The file trace.tab is a tab-limited file that lists parameter values and payoffs at different iterations. The file trace.dat lists the same information in a format that can be loaded into Vensim using Dat2vdf. The time axis for this file is simply an index that increases by one on each simulation. As optimization progresses, both of these files are filled in. Termination of the optimization session leaves them intact.
The value to which trace is set (2-6) determines how much information is included in the created files. See the :OUTPUT option for a description of these different output levels.
If any point had a payoff which caused an overflow or some other numerical error, then the special value NA is entered in trace.dat. These points will be ignored when loaded into Vensim.
:MAX_ITERATIONS=1000
This is the maximum number of times the optimizer will search through the list of parameters to avoid entering an infinite loop. This is a control parameter for POWELL optimization.
:PASS_LIMIT=2
This is the number of times the optimizer will run through an entire Powell search. At the beginning of each search the vector set is reinitialized to ensure that none of the directions have collapsed to zero. A value of 2 is usually good enough to deal with this and is therefore the default value. This is a control parameter for POWELL optimization.
:FRACTIONAL_TOLERANCE= 0.0003
This represents the fractional tolerance for a parameter value and is used to determine when to terminate an optimization. The search is refined until the value for each parameter is within the fractional tolerance times the search range (maximum value minus minimum value)of the parameter. If no search range is selected the fractional tolerance multiplies the larger of 1.0 and the absolute value of the parameter when the search starts. This option sets the fractional tolerance for all parameters. You can set individual parameters separately to override this option, using the FRAC=# notation discussed in "Specifying Search Parameters" earlier in this chapter.
If you specify a fractional or absolute tolerance for an individual parameter this overrides the global fractional tolerance setting. At the beginning of the optimization absolute tolerances are computed for all search parameters and these are used to determine both the initial step size (granularity) of the search and the termination conditions. . All tolerances are adjusted by the tolerance multiplier for the initial phase of the optimization. When searching in a direction that involved multiple parameters, the tolerance is determined as the geometric mean of the tolerances in the different directions.
Optimizations work best when the tolerances for the parameters being searched over are balanced. Essentially this means that changing any parameter by its tolerance will have approximately the same impact on the payoff. If this is not the case you may get early convergence, or Vensim may perform an excessive number of simulations to converge which takes longer. If you are conducting searches over a large number of parameters it is worth it to take the time to try do this balancing.
:TOLERANCE_MULTIPLIER = 21.0
The termination criteria for the search are adjusted for the early and coarse searches over the parameter space, so that these searches will terminate quickly and a more refined search can start from a good starting value. The tolerance multiplier allows you to adjust the balance between accuracy and completion time in early searches. On each iteration, the tolerances are multiplied by 1 + ((n - iter)/ n) * tolerance_multiplier, where n is the total number of search parameters and iter is the number of iterations through the search directions. When iter gets bigger than n, or if no improvement is found on an iteration, the tolerance multiplier is no longer used. This is a control parameter for POWELL optimization.
:ABSOLUTE_TOLERANCE =1.0
This sets the default value for an absolute tolerance. The absolute tolerance is used in determining when to terminate a search, but it is not applied to a parameter unless the ABS modifier is given for the parameter (as discussed in the previous section) and no value is specified. All absolute tolerances are multiplied by the :SCALE_ABSOLUTE and adjusted by the tolerance multiplier during optimization.
:SCALE_ABSOLUTE =1.0
Because of differing units of measurement, different parameters are likely to require different absolute tolerances. Once these different absolute tolerances have been set, they can be scaled up or down using the :SCALE_ABSOLUTE value. For example, if PARM_1 has an absolute tolerance of .3 and PARM_2 has an absolute tolerance of .4, then setting :SCALE_ABSOLUTE=2 is the same as giving PARM_1 an absolute tolerance of .6 and PARM_2 an absolute tolerance of .8.
SCALE ABSOLUTE does not apply to fractional tolerances. Generally, if you are specifying tolerances for individual parameters you will get the best performance by specifying absolute tolerances and scaling them all for early exploration (or if using multiple start).
:VECTOR_POINTS=25
This option only applies if the MULTIPLE_START=VECTOR option is set. This sets the number of points to search along the vector.