Wednesday, July 13, 2011
National Forecasters Outperforming Locals for This Week
Last Friday night, temperature trends varied by forecast outlet in a rather interesting way. Virtually all national forecasters (weather.com, accuweather and intellicast) saw the current week as being cooler than the local prognosticators. The national weather providers envisioned several days in the 70s, whereas the hometown studs maintained temps in the 80s throughout. Given the rather odd dichotomy in temps, we captured the forecast for later review. Our thinking -- our guess, really -- was that the locals would be better at forecasting local weather than the nationals. However, as this week has played out, and given the forecast for Thursday, that's not been the case. If you had your money on Weather.com, you were a winner.
Subscribe to:
Post Comments (Atom)
What about all that talk that we were supposed to have a cool summer? This has to be one of the hottest summers ever.
ReplyDelete@ anonymous not even close you should check out the summer of 1988, this week will be close to that, but back then it lasted for almost 60 days, it was nasty
ReplyDeleteYes, 1988 was killer. Extended heat waves with multiple days in the 102-105 range and little to no rain. It was a brutal, hot, drought. This meteorological summer (June-August) has been one of extremes, with very few truly nice days like today. When it hasn't been extremely hot, it's been cool and rainy. Those extremes even out to near average.
ReplyDeleteanon, June was barely 1º F warmer than normal. We are 5.8º warmer this July. Don't judge the summer based on a 5-7 days of hot weather. Of coarse having two days of 100º or more is rare.
ReplyDeleteThe forecast for "summer" is based on the average temperature for June-July-August. It is NOT a forecast for individual days during the season, the number of heat waves, mugginess, etc. While at this point the mathematical probability of a below normal summer (defined as being in the lower third tercile of summers based on the past 30 years of data) is decreasing, a cool August could tip the scales.
ReplyDeleteAs for the content of the main post...I don't see what you're getting at here. You're taking a sample size of FIVE and saying the local guys suck? How are you even computing this stat? I plugged your numbers above into Excel and found the mean absolute error (MAE) for the different outlets to be, in decreasing order of accuracy, to be Weather.com (1.2 F), Accuweather (2.4), Intellicast (2.6), KARE (2.8), KSTP (3.0), WCCO (3.6), NWS (4.0). Besides NWS stinking it up and Weather.com doing well, the rest are bunched in the middle. Again, the sample size is so small it is hard to draw any conclusions outside of "these few days".
I will say that I *really* wish that ALL stations/agencies/companies that provide forecasts were more transparent with their verification. Internally it is something that is definitely looked at and is really cared about, but those numbers usually don't make it out to the public (possibly because people don't care about the gritty details of verification, which can be exhaustive).
@P There's no major scientific import here to be sure. Just seemed somewhat interesting that all of the locals saw it warming up more than the nationals.
ReplyDeleteAs for verification, transparency, etc., I don't get the impression that anyone besides the big nationals do that. ForecastWatch.com provides great comparative data on how certain national weather providers are performing. See something like this: http://forecastadvisor.com/Minnesota/SaintPaul/55116/. There's also a company called weatherate.com that assesses (in a systematic way) the performance of local television stations; however, unless the winning TV station purchases the rights to use the certification, there's no way for anyone to know how the individual stations fared. In the meantime, with no resources to speak of, all I can do is take snapshots of certain periods where there is variance in the forecast (such as the one in this blog posts). Again, nothing scientific, but perhaps moderately entertaining. Thanks, as always, for your great input to the blog.
I've seen the Forecast Advisor site and WeatherRate has issues (essentially you're just paying to be labeled as the most accurate by an unbiased third party, how convenient!).
ReplyDeleteTransparent verification should be one of your "demands" of forecasters in general.
How would you define transparent verification? What does it actually look like? Should they make available a "scoresheet" that shows what they predicted against actual?
ReplyDeleteThe national weather service has a whole department on verification. Temp, rain/prob of precip, snow, tornado warnings, thunderstorm warnings, ect.
ReplyDeleteHell, I bet if you sent an email to the Twin Cities NWS office they'll give you their verification data. After all they work for you, Mr. Taxpayer.
I doubt they would. While there are a few people employed full time with NWS to be in charge of verification, it isn't publicly accessible, isn't readily available to a NWS meteorologist, and therefore isn't much use except for maybe briefing congressional types who care about that crap.
ReplyDeleteI'd love to see offices provide quarterly verification reports on their websites. Or even on the point-click forecasts, provide an "Accuracy" link that'll show verification statistics for that point.
Either of those ideas would be "transparent verification" in my mind, but honestly ANYTHING at this point would be an improvement. It is something you don't see any forecasting agency/business provide to the average user.