For fans of weather accuracy assessment, today was a good day. In the process of launching its new "Best Forecast" feature, Weather Underground is also providing an unprecedented level of transparency by providing an accuracy comparison with the National Weather Service's National Digital Forecast Database.
Given Minnesota's reputation as a notoriously challenging location for weather forecasting, it will be interesting to see how the "Best Forecast" feature fares for our area. Either way, we applaud WU's new innovation in weather forecasting.
For more information about the new system, click here.
Do you think it's fair to lump TV stations together as a whole, as opposed to grading individual forecasters? I don't. No one can choose who their co-workers are, especially in TV. Giving 1 grade to an entire weather team is like giving 1 grade to entire class of 5th graders. Certain forecasters are FAR more skillful than their co-workers, and vice-versa.
ReplyDelete@Anonymous Point well taken and that's something we've discussed on TMF from time to time. Part of the deal, however, is the way the stations promote their *team* of forecasters... so it sort of begs an analysis from that perspective. Also, individual forecasters don't always work all days so there's no continuity if assessed individually.
ReplyDeleteAs you readily admit you have no meteorogical training, I'll fill you in: the standard for forecast rankings is individually. It's how it's done in the National Weather Service. It's how it's done in colleges. It's how it's done in the National Forecasting Game, in which college forecasters compete from across the country. Respectfully... your method is something that would be expected of someone who is way out of their depth. You get a D.
DeleteAs it has been duly emphasized by TMF on a previous post, the real problem is when different forecasters from the same TV station give apparently inconsistent and sometimes almost opposite forecasts within a short period of time.
ReplyDeleteKSTP is typical example, with the Barrow/Dahl duo alternating almost evert half hour with "no big storms / big storm chance next week" kind of statements that are somewhat embarassing.
It would be interesting to see if there is a way to track the consistency of a station's forecast across its meteorologists.
From what I understand, most stations have unwritten policies that significant forecast changes (i.e., from one forecast to the next) need to have a bit of discussion/deliberation before changing. However, I may not be getting that quite right.
ReplyDelete@Anonymous So how do you propose to do that when forecasters, particularly the ones on TV, don't provide forecasts each day? My sense is that television forecasting is a bit of a different ball of wax than the way things are done at the NWS. Finally, nobody should take things on here overly seriously. I've NEVER said I'm a professional at this.
ReplyDelete@anon 859 am:
ReplyDeleteAlthough I don't know with 100% certainty I would think most outlets have some means of grading their forecaster's. Many of us that make up the viewership of these outlets all have our favorite individual forecasters, but it would be next to impossible to grade all of them for each event unless they all are forecasting every day. What I do think is important, is to grade each outlet over time (ie: last year was fun, this year? well we all know the answer to that) and try to show those who are interested which of our outlets consistently performed at a higher level over a given period, particularly during the winter season. So I for one applaud Bill for what he does.
"Giving 1 grade to an entire weather team is like giving 1 grade to entire class of 5th graders."
ReplyDelete...and yet that's how our government has decided to rank school districts, schools, etc...for the past decade. Don't worry, I won't get on my Soapbox.
I think the best thing to do is look at them together and individually. It would be crazy to expect that people don't hold an entire newscast/station responsible for its weather team, news team, sports team, etc. At the same time, Belinda leaves at 10:30 pm and Sven cmoes in at 4:00 am. Inbetween, there's a lot that can change with the weather and obviously one has to judge for him or herself at some point.
Good points @Randy and @AB.
ReplyDeleteIts up to each TV station's cheif meteorologist to make sure the staff works together as a team and provides consistently accurate forecasts. Many stations brand their team as one cohesive unit and they should be graded as such; just as NWS, weather.com and Wunderground produce forecasts as a team. If there's one bad apple on the team its up to the station mgmt to address that. It may be that the news station doesn't value forecast accurracy as much as the guy across the street, so I see no problem in the team based approach.
ReplyDelete-Pete
This comment has been removed by the author.
ReplyDeleteToday's storm chasers are equipped with some of the latest technology to observe and record mother nature's fury. Continuing technological development makes these instruments affordable to the average hobbyist.Owen
ReplyDelete