We’ve based our always subjective grades on forecasts issued mid-morning on Saturday with the idea that that was about 12 hours prior to storm onset and were the forecasts people could best use to plan their day. Many forecasters raised their forecast snow totals later in the morning based on newer model information.
MPR: A- MPR essentially nailed the storm (save for blowing and drifting snow which never materialized).
KSTP: B On the surface, KSTP’s morning forecast snow totals were in the ballpark. However, we think the storm still didn’t unfold as they thought. Their forecast was for “significant blowing snow.” However, due to the fact that virtually all snow fell when temps were above freezing, the snow was heavy and wet, and blowing/drifting never materialized.
FOX: C- It’s difficult to grade FOX because their website did not provide any forecast info in the morning. All we had to go on was a tweet in the late morning with predictions of 3-5 inches (and Sunday winds of 20-40 with gusts approaching 60 mph) and an evening prediction of 1-3. FOX was long on the snow estimate and as of this writing midday Sunday, the highest gusts have been 45 mph.
KARE: B- KARE’s morning forecast included a snow range of 1-4 inches, a range that we felt was unreasonably wide. They tightened it to 2-3 inches by the evening newscast, which was ultimately a little higher than actual.
NWS: D This was a disappointing performance by the National Weather Service. Their use of a winter storm warning seemed questionable from the start, and ultimately proved to be overkill.
Strib: B- The Strib’s initial forecast of 1-3 inches was in the range. However, they swayed with the majority of forecasters and became overly ambitious based on late morning model information.
Accuweather: C Morning predictions of a coating to an inch were too low. Interestingly enough, Accuweather’s forecast as of 5 p.m. was for 1.9 inches, quite close to the final tally.
@NovakWeather: C This grade reflects a mixed performance. Novak was among the first forecasters that we came across who saw the possibility of a New Years Eve storm of significant magnitude earlier in the week, a time when most others were pointing to wind as the most notable weather. However, Novak’s ultimate snow prediction was considerably too high.
Bill, Thanks for posting the grades. I agree with your earlier post, it's not much fun grading this year with the wimpy storms we have seen, you simply don't get a big enough spread among the outlets that you do during the bigger events.
ReplyDeleteHaving said that I agree with all the grades but the NWS. Being the only institution that can issue advisory or warnings I think they did their best with the data that was presented to them on the overnight runs. Given the fact that it was a heavy travel weekend and lack of storms I believe that the warning was warranted. I see on their facebook page that blizzard criteria was considered. Their biggest error I believe was waiting to long to downgrade it once the short term models (HRRR,RUC and the VSREF)showed the proper solution. I think a C might be in order. Thoughts anyone?
Say what you will about the decision to hoist a winter storm warning. I think it was a good call. Remember, the job of the National Weather Service is to protect life and property. With so many folks planning on travelling during the snow event, combined with the expected deteriorating travel conditions, it was a good decision to call attention to the situation. Good weather forecasting isn't only about getting the conditions correct, but also providing the masses with information to keep themselves safe. I've come across anecdotal reports that the roads were quite poor in some locations last evening. All in all, good forecast and good decision to issue the winter storm warning.
ReplyDelete-Kevin.
But by definition, if the NWS ultimately downgraded to an advisory, doesn't that suggest that if they had it to do over again, they'd have just gone with an advisory? Also, an advisory IS supposed to provide the masses with info to keep themselves self. There has to be a warning differential between the type of storm we had last night and what most people would consider to be a "major storm."
ReplyDeleteThis could be an interesting debate.
ReplyDeleteSnow total wise they we're similar to WCCO I believe who got a C.
So the sole reason for the D is the warning which as mentioned no one else deals with.
Personally I had no problem with the warning. Roads became icy after it started with rain and with the potential for blowing snow on a very busy travel night if it stopped people from driving then it did its job.
Rhetorical question for Bill - If the NWS gets docked a full grade for missing on the warning, to they gain a full grade over everyone else when they nail it?
@CWY2190 You make a real good point. In retrospect, perhaps they should get the same grade as WCCO (so official are these grades, after all). I saw a tweet from one television met to another asking whether they thought the warning was proper... so I think there's a feeling out there that it was questionable. Almost reminds me of when Paul Douglas tossed out a foot of snow a week before a storm (that didn't amount to much) and got the whole region on high alert. Sometimes, a forecaster, because of status or visibility, sets the tone for expectations.
ReplyDeleteI agree with @CWY2190.
ReplyDeleteI'll say it again, there is life "outside the loop." If I'd have chosen to drive home last night, I would have put my safety at significant risk. I experienced no problems on the freeways or major side streets inside the loop this morning; however, there were slippery spots on 212 between Eden Prairie and Carver and the two-lane roads I took were covered in ice chunks.
Unless the NWS changes its advisory model so that the inner ring of the Twin Cities is given separate advisories from the rest of the communities in those counties, they have to warn the masses. I certainly think it's worth it to have a million people in the core doubting a warning/advisory in hindsight than having a million people just outside that core area insufficiently prepared for deteriorating conditions.
"But by definition, if the NWS ultimately downgraded to an advisory, doesn't that suggest that if they had it to do over again, they'd have just gone with an advisory?"
ReplyDeleteNo, I don't interpret it that way.
I also don't agree that an advisory has the same impact as a winter storm warning.
When generating these products the forecaster must have leeway for situations not specifically covered in the "product definition" or the warning would be issued by a computer based on the forecast the person creates.
I feel that allowing the forecaster to nudge one way or another depending on the situation is necessary and appropriate.
-Kevin.
Kevin,
ReplyDeleteI think my writing was rather sloppy. I agree with you that an advisory does not have the same impact as a winter storm warning. What I was trying to say is that I think an advisory still has some impact... i.e., a tiered kind of thing where there's a minor event, an advisory-level event and a warming-level of event. Anyway, I'm sure I'm beating this into the ground by now.
This does not effect us, but it is worth noting-
ReplyDeleteOne of the biggest lake effect snow events in a long time is currently occuring over the great lakes- encomapassing all five of them
Bill - at best I would give the NWS a "D" in this case. You can't just through out a winter storm warnning - no matter what night it is - have 1.7" of snow fall, and then say, it was an accurate call. Why beat around the bush? There is probably someone at the NWS that we could email that could tell us how they would have graded their own forecast (I am assuming they have their own internal record keeping/analysis of each event). If they don't "grade themselves" then how is there ever going to be improvement?
ReplyDeleteTo follow up to my last comment...of course there are going to be mistakes with winter weather forecasts. Every situation is unique, which I think is an aspect we all love about the weather. If I was confused about the use of a WS Warning, as well as media outlets, how do you think the general public perceived it?
ReplyDeleteI've seen earlier posts about how "people rely on winter weather as a business" (snow plow operators, ski resorts, etc). . Just the same, there are businesses that are effected by someone over-predicting poor travel weather. How would you grade the NWS if you owned a bar or a restaurant and you lost 25% of your revenue compared to a 1-2" forecast?
You guys don't have to wonder why the NWS issued a warning over an advisory. You can go back and read the forecast discussion on weather.gov...
ReplyDeleteA VERY BUSY AND SOMEWHAT COMPLEX FORECAST SCENARIO WILL UNFOLD
OVER THE UPCOMING 24 TO 36 HOURS... PARTICULARLY IN CONTRAST TO
THE EXTREMELY BENIGN WEATHER RECENTLY. THE PRIMARY CONCERNS ARE
WINTER WEATHER AND WINDS. WINTER WEATHER CONCERNS REVOLVE AROUND A
BAND OF NORTH-SOUTH ORIENTED PCPN WHICH SHOULD QUICKLY TRANSITION
FROM RAIN TO SNOW AS IT MOVES EAST ACROSS THE FORECAST AREA LATE
THIS AFTERNOON THROUGH TONIGHT. THE MODELS STILL DISAGREE ON SOME
OF THE DETAILS... BUT THERE IS GENERAL AGREEMENT ON THE POTENTIAL
FOR SEVERAL INCHES OF SNOW ACROSS AT LEAST THE CENTRAL AND A GOOD
CHUNK OF THE EASTERN FORECAST AREA DURING THAT TIME FRAME. WIND
ISSUES WILL ARRIVE SHORTLY AFTER PCPN DOES... AND WILL INCREASE
QUICKLY AND DRAMATICALLY DURING THE EVENING AND INTO THE OVERNIGHT
HOURS. WIND GUSTS ACROSS THE AREA OVERNIGHT WILL TOP 40 MPH... AND
SOME LOCATIONS OVER THE SOUTHERN CWFA COULD SEE GUSTS TOP 50 MPH.
GIVEN THE CONCERNS ABOUT WIND AND ACCUMULATING SNOW... EVEN WITH
SNOWFALL TOTALS NOT EXPECTED TO EXCEED 3 OR 4 INCHES IN MOST
AREAS... WINTER WEATHER ADVISORIES AND WARNINGS WILL BE ISSUED DUE
TO EXPECTED IMPACTS FROM THE COMBINATION OF SNOW... WIND... AND
FALLING TEMPERATURES. IMPACTS MAY BE ENHANCED DUE TO INCREASED
TRAVEL DUE TO IT BEING NEW YEARS EVE... AND INDEED THE OVERLAP OF
FALLING SNOW AND WIND LOOKS TO OCCUR AROUND THE 06Z TIME FRAME
OVER THE CENTRAL PART OF THE FORECAST AREA... WHICH COULD SET THE
STAGE FOR SIGNIFICANT PROBLEMS ON AREA ROADWAYS.
I'd argue there were more people on the roads during the daytime hours during a Saturday snow event earlier this season in November where more snow fell than did during New Years Eve and there wasn't an advisory hoisted (until late). Did this past WS WarnIng make up for that, or was this an attempt to make up for New Years Eve 2010? S
ReplyDeleteI love reading those discussions, I just wish they were updated more than once every 12 hours, especially during an event.
I know this is all Monday morning "quarterbacking" (forecasting), but just trying to think of ways to improve the process. Maybe this is as good as it gets?
The NWS keeps dropping their forecast for tonight. The first sub-zero readings of this winter are likley.
ReplyDeleteAccuweather has snow ten days out.
ReplyDeleteNow, normally, would not post this, but it appears that the weather channel also has snow ten days out.
Also, forcea (the british centered internet weahter) has rain predicted late that night, but also predicts a low of 5, adusted to be the farinheight tempature.
Does anyone know how they came up with this? Novak? Hammer? Somebody else?
How about a bit more education for the naysayers and amateur graders?
ReplyDeleteI found another directive speaking to issuance of winter weather products for NWS. It seems that the forecasters may, at their discretion, issue products based on perceived impact of the event even if specific criteria may not be met. Makes it hard to "grade" for the folks on this site and others, but makes a more useful product.
http://www.nws.noaa.gov/directives/sym/pd01005013curr.pdf
Here's a quote from section 6... "However, if it is early in the season or during a critical time of day such as rush hour when the impact will likely be high, then a Winter Storm Warning might be warranted. The forecaster has the discretion and should not be held back from issuing what best describes the impending winter hazard even if traditional criteria may not be met in the strictest sense."
Given this new information, I think the NWS grade should be adjusted upward by at least one, if not two letter grades and since the winter storm warning "criteria" not being met is a moot point. What do you think, Bill?
- Kevin
Even accepting that the NWS can issue a warning for what appears to be an increasing range of circumstances, I still maintain that they downgraded the warning to an advisory before the snow even began. This suggests to me -- heck, it's the only way to interpret it -- that the storm was NOT as major as they initially expected it would be. The issue has less to do with warning/advisory than the fact that they overestimated the storm. I could see raising the grade to a C (because, as noted earlier in the thread, they shouldn't be penalized for being the outlet that has to label the storm with "warning" or "advisory"). Essentially their performance on forecast alone (not issuance of advisory/warning) was similar to WCCO. In the future, maybe we should just analyze the NWS based on its actual forecasts and leave debate of whether warnings and advisories should be issued as a separate discussion.
ReplyDelete@Bill-
ReplyDeleteGood point. The way I see it, the NWS is the only one who has to issue statements, and, due to that, the have to deal with more pressure to get things done, along with more criticism.
@ bemaki a cold front will be sliding in and may drop a few flurries as it moves through.
ReplyDeletehttp://mag.ncep.noaa.gov/GemPakTier/MagGemPakImages/gfs/20120102/18/gfs_namer_216_1000_500_thick.gif
@ everyone else, if you want to look a accu's interpretation of the latest ECMWF weekly's ...
http://www.accuweather.com/en/weather-blogs/anderson/new-look-at-the-extended-long-range-model-forecast/59714
I gotta agree with bemaki and previous anon poster. If you're going to grade all the different weather outlets, it should be based on the same criteria that is the common denominator (during the winter, I'm guessing you'd be looking at snow amounts and temperatures). NWS downgrading the warning to an advisory could have also been due to less blowing snow, timing of the snow (not as big of an impact on travel), the "worst" weather has ended and only advisory level "bad" weather is expected, or yes less snow.
ReplyDeleteFor this event, looking at the warning, it was downgraded at 854 PM. It stopped snowing at 1130PM at KMSP (using what data I have quick access to). The actual text from the warning-to-advisory product is below. Clearly the reasoning was slightly less snowfall and less blowing snow due to "stickiness" of it. Call that a bad forecast if you must, I guess, but the initial WS.W was for 2-4" and it looks like 1-5" fell. Also note that the WS.W was initially surrounded by counties under an advisory, where a little less snow was anticipated (1-2").
I'd also suggest looking more at average snowfall amounts since shifts in snow bands of just 20 miles can mean 12" of snow over KMSP or that snow falling primarily south of the metro (then who cares amiright?).
.THE WINTER STORM WARNING THAT COVERED MUCH OF SOUTH CENTRAL
MINNESOTA...THE TWIN CITIES METROPOLITAN AREA...PARTS OF CENTRAL
MINNESOTA AND PARTS OF WEST CENTRAL WISCONSIN HAS BEEN DOWNGRADED
TO A WINTER WEATHER ADVISORY FOR THE REST OF THE NIGHT. THE
WINTER WEATHER ADVISORY THAT COVERED WESTERN MINNESOTA FROM LONG
PRAIRIE TO REDWOOD FALLS HAS BEEN CANCELLED.
A BAND OF SNOW THIS EVENING FROM NORTH OF THE TWIN CITIES THROUGH SOUTH CENTRAL MINNESOTA WILL PUSH EAST RATHER QUICKLY DURING THE LATE EVENING AND EARLY MORNING HOURS...EXITING WEST CENTRAL
WISCONSIN BY 4 AM SUNDAY. SNOW ACCUMULATIONS OF ONE TO THREE INCHES ARE LIKELY WITH THE HIGHEST AMOUNTS OVER SOUTH CENTRAL MINNESOTA NEAR THE INTERSTATE 90 CORRIDOR. THERE MAY ALSO BE SOME
LIGHTNING AND THUNDER WITH THE SNOW OVER FAR SOUTHERN MINNESOTA. IT WILL BE A RATHER WET SNOW WITH TEMPERATURES RIGHT AROUND THE FREEZING MARK. ALTHOUGH NORTHWEST WINDS ARE INCREASING ACROSS THE REGION...THE AMOUNT OF BLOWING SNOW IS NOT EXPECTED TO BE AS MUCH OF A PROBLEM AS IT WAS BEFORE DUE TO THE NATURE OF THE SNOW.
-P
How about some comments on a possible shift in the pattern by the end of next week? I think this will happen as it seems like this ridging is shows signs of breaking down. It has to level off and even out. There are some big signs, even indicated in the NWS discussion this afternoon, about a possible storm mid to late next week and some significant cold air, even below zero. Below zero? What is that, again??
ReplyDeleteChris Shaffer just mentioned cold snap between 17th and 20th possible. I agree, a change is coming, I'm just kind of waiting for this evening's run of the GFS to post some things.
ReplyDelete