Friday, May 24, 2024

Getting Good Graphs From Woodstock

Introduction

I’m always surprised when I learn that clients do not use the Woodstock graphics section to produce on-screen graphs of their solutions. Frankly, I can’t conceive of doing an analysis without monitoring standing inventory, harvest levels, operable inventory and silvicultural activity areas. If nothing else, a graph will quickly indicate if you forgot to generate a new schedule after imposing an even-flow constraint or something similar. Seeing how inventory is changing can provide insights into why harvest levels don't seem to be what you expect.

That being said, the default color choices that Remsoft provides are at best, meh. People who know me know that I despise the “mustard” color, which to me looks more like baby poop. They also know that I don’t think much of 3D charts (which are hard to interpret), line charts with all sorts of markers, and graphs with too many outputs graphed simultaneously. There are good resources on how to display data, but Woodstock doesn’t enforce any of these rules. It is up to you to make something informative and visually pleasing.

About the Graphics Colors

Unfortunately, you cannot redefine Woodstock’s standard color codes to more pleasing shades. These are, RED, GREEN, BLUE, YELLOW, CYAN, MAGENTA, SCARLET, EMERALD, NAVY, MUSTARD, PEACH, ORANGE, BROWN, PURPLE, GREY, and BLACK. However, you can tell Woodstock that you want to use a custom color instead of one of the standard colors.

Right-click the graph and bring up the Properties dialog. Select the output you want to change and click the More (…) button. You can choose one of the standard Windows colors, or you can define your own by clicking in the rainbow box, or by specifying HSL percentages (hue, saturation, luminence) or RGB decimal values.

How to customize Woodstock graph colors

Once you’ve selected and saved your custom color, Woodstock will save your choice in the Graphics section using the _RGB() function, which uses decimal numbers instead of hex.

 *LINES
 OICE 1 1000 _RGB(203,24,29) _SOLID "Cedar "
 OIPW 1 1000 _RGB(107,174,214) _SOLID "Ponderosa & white pines"
 OIDF 1 1000 _RGB(35,139,69) _SOLID "Douglas-fir, larch, spruce"
 OIWW 1 1000 _RGB(204,204,204) _SOLID "Whitewoods "
 OILP 1 1000 _RGB(33,113,181) _SOLID "Lodgepole pine"

That is all well and good, but if you save a bunch of custom colors, how do you use them in your next modeling project? You could copy and paste from your graphics section to inherit the colors, but the better approach is to edit the Woodstock palette.ini file and save them.

The WKPalette.ini File

In your Remsoft installation folder (e.g., C:\Remsoft\Ver2023.02.01), you will find a text file called wkpalette.ini. It is just a text file with a bunch of colors defined by hexadecimal RGB values. For example, #000000 is the RGB code for black, where red = 00, green = 00 and blue = 00. White would be #FFFFFF, with all three colors set to the maximum 255 (FF in hex). All other colors are some combination of RGB values ranging from black (00) to white (FF). Greys are easy because they are all the same value for red, green and blue (i.e., #969696 is a mid grey).

If you have decided on colors previously, you can save them as a new palette in the wkpalette.ini. First, save the old file as wkpalette.old, just in case you muck things up. Then open the file in any text editor (Notepad, Woodstock)

[Palette:_Default]
#000000
#808080
#FF8000
#804000
#0000FF
#000080
#FFFF00
#C2C200
#8000FF
#800080
#C20000
#008000
#21A161
#FF80C2
#FFC080
3FF0000

There are multiple palettes defined in the file, which can be selected from the Options menu of the Graphics window. 


How to choose alternative palettes

The best source I know of for choosing pleasing but contrasting color combination in your maps is the ColorBrewer 2.0 website. Designed for GIS professionals, it has numerous options for displaying different kinds of data on choropleth maps. But the same concepts also apply to graphs. 

For a project I am currently working on, one of the team members is red-green color blind. After researching some websites on the subject, I selected a color palette that should allow anyone to discern differences in the data.

[Palette: ColorBlind]
#000000
#E69F00
#648FFF
#009E73
#F0E442
#0072B2
#D55E00
#CC79A7
#332288
#117733
#44AA99
#88CCEE
#DDCC77
#CC6677
#AA4499
#882255

Results

The colors in the new palette (right) are softer than the Remsoft defaults (left):



Alternative palettes: Remsoft Default (L) vs Custom Color Blind (R)

Summary

In addition to colors, you can also choose different fonts. The standard Tahoma fonts look good, but for those who like a fixed pitch font, Consolas is also nice. I use it as the text font in the Woodstock editor. I believe it displays better on screen than the default Courier New. When I am teaching new analysts, it is important to be able to distinguish spaces and underscores, and that's hard to do with Courier font. 

NOTE: If you change the palette file, you'll also have to change the palette setting in your Woodstock graphs if you want the graph colors to change. And remember, colors are just randomly chosen when you first define graphs. You still have to manually change them if you find the same colors repeated, etc. Have fun!

Interested in How to Better Present Results? Contact Me!

Model diagnostics are very important which is why I think using graphs is mandatory. With graphs, you can immediately see if constraints are being respected and if your outputs are calculating properly. I can lead a custom session for your staff on how to do QA and model testing right!

Wednesday, May 15, 2024

Errors of Omission and Commission

Introduction

Recently, I've been working on a modeling project with a lot of interesting silviculture. In addition to the usual suspects (clearcutting, commercial thinning), this model includes group selection, individual tree selection and variable retention harvesting. Devising the conceptual model for this project was challenging (something I'll talk about in another post), but keeping track of the many different yield tables was equally so. Specifically, it was a challenge to know that I was associating the correct yield table with each stand, AND that I wasn't inadvertently missing tables. So, in this post, I’m going to review errors of omission and commission and highlight a feature in the Woodstock editor that can be a big help.

Errors of Omission

You are committing an error of omission in your model when you reference a cost coefficient that doesn’t exist. For example, if you have stumpage prices by district, but you forgot to include one district in the YIELDS section, Woodstock assumes a value of 0 for prices in the missing district. Normally, that will just generate the familiar Warning 74 (yield not found, assume 0), despite the fact that it is a major problem.

Luckily, you can avoid errors of omission fairly easily by following a simple methodology based on this idea: never rely on Woodstock to assume 0 for you:

Define all yield sets with the same yield components (columns). If a product doesn’t exist in a stand, then print a column of zeros. Yes, Woodstock will complain (Warning 328) that you don’t need that column. You can mute that error code in the CONTROL section.

If you include non-productive areas in your model, assign them an explicit yield set as well. You just need a single row of zeros under each yield component heading for AGE or CP = 1.

If you do this, you should be able to check for missing yields using any of your defined yield components (e.g., ytmvt) using the following report:

*TARGET yield_check.dbf
ytmvt

Open the report file, and sort the YldNumber column in ascending order. If there are blanks in the YldNumber column, the development types shown do not have yield tables. Woodstock has assumed 0 for you. 



Excerpt of Yield_Check report

How does it work?

Internally, Woodstock maintains a list of YldNumbers (corresponding to each yield component) to which each development type points. If a development type does not point to a yield component, it triggers the warning 74.

By ensuring that every development type points to an instance of every yield component, there should be no gaps in the yield report. Yes, it does require redundant information that must be stored in memory, but the trade-off is knowing that you are not missing any real yield information.

The DevTypes you should be most concerned about are existing types that have never been treated. These ALL should have matching yield tables or else your initial inventory numbers will be off.

You may also have missing yields for specific timing choices of an action, even though all existing untreated stands are accounted for. This can arise when you have specified a general operability statement, but some conditions do not have yields associated with them. You can apply very specific DevType masks on multiple lines to account for every single stand, but this may not be worth the effort to account for them all. If the number of missing yield tables is small and only represents a small subset of some specific timing choices, all may be fine. When Woodstock evaluates those timing choices, the 0 coefficients associated with them should cause those timing choices to never be chosen.

Errors of Commission

You are committing an error of commission when you point to wrong information in your model. For example, this can occur if you associate the wrong price coefficient in your output definition, or if Woodstock matches an incorrect yield table with a development type. Unfortunately, errors of commission are much harder to find than errors of omission. There is no easy "Wrong_Yield" report you can use to identify them.

Avoiding these types of errors requires diligence and a methodical approach to model development. You should use the *GROUP keyword in your OUTPUTS section to develop all your outputs. Just like an outline in a Word document, you can use *GROUP to identify all the potential outputs you will need. Then, within each *GROUP, systematically work through all the variations on outputs that required. Copy and paste operations often result in overlooked edits that lead to errors of commission. Automation tools like Integrator or Python are better at ensuring that you are enumerating all the possible combinations with the right yield component.

CTRL-M - a handy tool

My favorite newer feature of the Woodstock editor is the CTRL-M (mask search) function. This feature works very much like the CTRL-F (Find) function but it isn’t literal like CTRL-F. For example, if you search a file for the string ‘CH00018 ? ? ? ? UT’, Woodstock will look only for strings where there is a single space between question marks. It will not find this string where there are two spaces between the question mark and UT : ‘CH00018 ? ? ? ? UT’. Unless you are very consistent in spacing, CTRL-F can be of limited utility if you’re trying to find a matching yield table for a particular development type.

If you open the CRTL-M dialog, you can enter a fully defined development type mask, with or without global and aggregate attributes. When you click Find, the Woodstock editor searches for masks that match, including global attributes. For example, suppose you input the following string into the CTRL-M dialog:

'FEE NONE NONE FC Y CSP CH00018 FE N WH I UT 250 ZR6024 1 LBC'. 

In your YIELDS section, CTRL-M will locate this yield set header:

*Y ? ? ? ? ? ? CH00018 ? ? ? ? UT ? ? ? ? 

and this one,

*Y ? ? ?      ? ? ?   CH00018 ? ?         ? ? UT ? ? ? ?

and also this one,

*Y ? ? ? ? ? ? agGRP8 ? ? ? ? ? ? ? ? ?

In the first instance, the mask search matches to global attributes (?) in all themes except TH7 and TH12. The second instance is exactly the same, but spacing does not matter, as it would in the Find function. With the third instance, the mask search also locates yield headers with valid aggregate attributes - the aggregate agGRP8 includes CH00018.

A Potential Use Case for CTRL-M

Let's assume that you’ve noticed you are getting incorrect volumes reported, and you’ve identified one of the development types that is yielding wrong values. Using CTRL-M, you start searching your YIELDS section for matching yields. You expect to find the correct yield in the bottom third of the file, but instead you find a match almost immediately. When you search again, you find the yield set you were expecting. You have a problem! Remember, Woodstock only searches for yield information until it finds a first match; later matches are ignored.

Now that you’ve found the problem yield table, the question is, why is it there? Did your yield generation process include old inventory information that produced the errant yield table? Was this just temporary code used earlier in the development process? Regardless of how it got there, it can be easily removed. However, if you have an automated process or you rely on contractors to provide the data, you need to make sure that future yield updates do not create the problem once again.

Is Your Model Driving You Crazy? Contact Me!

Finding errors of omission and commission can be difficult in a large complex model. If you’d like a new set of eyes to look over your model(s), give me a shout to schedule a model audit Afterwards, we can schedule an on-site training review with your staff to review recommendations for avoiding problems in the future.

If you have a topic that you'd like me to address in a future post, send me a message. I'm always more interested in helping with client issues than ones that I think might be of interest.

Why are MIP models difficult to solve (or not)?

Introduction I recently joined a conversation about why a mixed-integer programming (MIP) problem is so much harder to solve than a regular ...