Forecaster Grades for the March 8-9 Snow Event (Non-Event?)

Our standard disclaimer: The grades we provide are not scientific. However, they represent our best effort to assess the performance of Twin City weather outlets. Our grades are based on established grading criteria and we maintain forecast histories for each event. It might also be helpful to read our “Weather Watcher’s Bill of Rights” to get a sense of our perspective.

In recent memory, it’s hard to recall a snowstorm that was advertised so early and with such ferocity. It’s also hard to recall a storm that felt so short in comparison to initial expectations.

Last Tuesday evening (March 1), most weather outlets identified the possibility of a big storm. From Thursday to the first part of Friday, “storm hype” was in full force (see below). Shortly after that, forecasters began to sense that the storm was tracking to the southeast and not likely to make a direct hit on MSP. However, even after adjustments were made to the initial “whopper” forecast, the storm was still an underachiever in comparison to final forecasts. And while TMF does not specifically evaluate forecaster performance in locations outside the Twin Cities metro, it seems clear that “the big hit” never really hit anyone.

Prior to the grading section, we wanted to take a more detailed look back at the so-called hype. Several forecasters, particularly in the last few days, have seemed to distance themselves from the hype produced by “some other forecasters.” We reviewed the progressive forecasts and, depending on how you define hype and what you perceive to be a prediction vs. a “mention of possibility,” it seems that the “other forecasters” were more or less everyone. Here’s “the height of the hype” from each weather outlet.

WCCO: One model said 12-16

KSTP: Could spread 6-12 inches our way

Fox: 10-15 inches

KARE: Looking very much like the other big storms this winter

Strib: 13+ inches

MPR: This is a perfect track to dump heavy snow in MN, including the metro

In the end, the snow event of March 8-9 totalled exactly one inch of snow at the MSP airport.

Here is our forecaster review and grades:

MPR

MPR was very deliberate in the timing of its first issuance of snowfall estimates. In fact, MPR included 225 words on its Monday morning blog post devoted to discussion of their snowfall prediction policy (24-48 hours prior to an event). It spoke to readers in a clear, down-to-earth manner.

We were disappointed with the rather long time (three days) without at least a short update on the MPR blog during a time when prospects for a major storm were changing and new information was available.

Early on Sunday morning, MPR said there was an 80% chance of a plowable event that included the Twin Cities. Obviously, this did not transpire. That said, MPR was the first to mention a lower range of snow totals and also emphasized the low end of 1-4-inch prediction before the storm started. Our grade for MPR is a B-.

KARE

Conservatism was king for this storm. The less early hype given to the storm, the better the grade, in our minds. KARE’s policy of not providing snowfall accumulation estimates until 24 hours prior to a storm served them well this time around.

KARE made an exception to its extended outlook policy (which generally seems to be five days, except in the case of weekends) by mentioning the storm a week out. They commented on the similarity of the look of the as-of-yet-unformed storm to the big storms of this winter, thereby suggesting a big one might be on the way.

KARE’s first snow totals after the general “few inches” were for 3 to 5 inches. Their final 2-4 inch snowfall prediction for the metro (which we extrapolate to mean 3 inches for airport) was high and still a significant variance for this sized storm. We give KARE a grade of B-.

KSTP

KSTP was an early rider on the big-snow wagon with the prospect of a foot mentioned in a blog entry on Wednesday evening. However, their cause was helped out to a degree by their use of confidence ratings (something regular TMF readers know that we’re big on). On Thursday, they gave a 10 (on a 10-scale for confidence) to the possibility of “measureable snow” with this snow event (which happened, though a mini-ruler was sufficient). Their 7 on the same scale for a “plowable snow” reflected a strong conviction, yet not a certainty. However, just as surely as a .300 hitter in baseball gets a hit, their prediction was wrong. (Think about that one….)

Still, on Saturday morning, KSTP reported that “someone in the Midwest will get nailed.” We interpret “nailed” to be upwards of 6-12 inches in this context, that did not happen. KSTP’s final prediction of an average metrowide accumulation 3 inches was considerably high.

While KSTP’s use of social media and confidence ratings continues to set the standard, its forecast this time around was lacking in accuracy. We gave them a D+.

WCCO

WCCO’s effort on this storm brought attention to the use of “tossing out” model numbers. It’s our feeling that if a forecaster, particularly a television forecaster, voices possible snowfall amounts in any way – as an actual prediction or as a possible output from a model run – the attention span-challenged viewer is going to walk away remember one thing: the largest snowfall amount they heard. On Thursday evening, WCCO cited that one model spit out 12-16” and another 15.5”. Related to that, we got a kick out of the Saturday 6 p.m. weathercast when Mike Fairbourne refused to answer the anchor’s request to provide weather model information and snowfall projections. Fairbourne responded, in his typically gentle way, “I’m not tipping my hand. It’s still three days away!”

Like most other weather outlets, WCCO overrated the storm even after the storm’s likely movement to the southeast was widely accepted. They suggested the snowfall would be a plowable event as recently as Saturday night. Our grade for WCCO is D.

FOX

FOX was a little slow out of the gate on this one. Last Tuesday evening, it was the only station to not mention the possibility of snow in its extended forecast for Sunday through Tuesday (recall that in addition to the just-ended snow, several inches of snow also fell on Sunday). Yet they ended up riding the same roller coaster as everyone else, suggesting 10-15 inches of snow at one point on Friday. FOX’s grade is a D.

Star Tribune

The old saying, “You live by the sword, you die by the sword,” seems appropriate in this case. When you throw out 13”+ in writing and it actually happens, you’re a rock star. When it doesn’t, you become the object of considerable criticism.

Partly given the forecaster’s early and large “prediction” and partly because the Star Tribune chose to feature the weather blog’s brassy and flatline “13+ inches of snow next week” headline on its highly visible home page, an expectation level was set for an entire metropolitan area, if not a region. And while the Strib presentation was sprinkled with disclaimers such as “this is a forecast; forecasts are often wrong” it somehow got lost in the context of predictions that some Strib readers called outrageous.

Clearly, this was not a winning forecast, nor did it seem to be handled in a particularly responsible fashion from a journalistic point of view. Our grade for the Strib is an D-.

NWS

We track the NWS by simply noting the text forecast rather than getting into the official NWS discussion. However, based on the straight day-to-day forecasts, the NWS was steady. However, it also suffered from snow overprediction, forecasting 3-4 inches as of Monday afternoon. The NWS grade is a B.

Agree? Disagree? Share your thoughts.