How Cold for New Years Eve?

With little interesting weather to track, we're focusing on the overnight low temps forecast for this New Years Eve (and the overnight time period) for the MSP airport. Interestingly, all local television meteorologists were forecasting temps to be colder than virtually all national outlets. As of 6 p.m. this New Years Eve, here's the info we gathered.

WCCO: -8

KSTP: -6

KMSP: -8

KARE: -7

NWS: -3

Accuweather: -3

TWC: 0

Intellicast: 0

The Search for Snow: Christmas and Beyond (and farewell to TMF dog)

Eddie, official TMF dog (Oct. 4, 1995 - Dec. 21, 2012)

Twin Cities snow lovers (and haters) continued to scan the horizon for the next notable snowfall. As of this writing (Sunday afternoon), there appeared to be a chance of a little snow on Christmas Eve day and a chance for a more appreciable snow for late in the week.

Forecasters Challenged by Expected Wintery Mess This Weekend

For forecasters challenged by last weekend's snowstorm, things don't get any easier. Mother Nature appears to be throwing another vexing riddle at the region's finest meteorological prognosticators.

As of Wednesday evening, forecasters were not ready to commit to a clearcut forecast. Here are a few forecast thoughts we saw across social media Wednesday afternoon and evening.

KARE11, during the evening news, mentioned that an accumulating snowfall of perhaps 2-4" is expected if the storm path continues in its current manner.

From MPR weather blog

National Weather Service

Analysis of Forecaster Performance for Dec. 9 Storm

The final total for this weekend’s snowstorm was 10.5 at the official MSP airport recording station. The core metro area snowfall ranged from 9 to 14 inches with an even foot of snow a common observation, including here at TMF headquarters in St. Paul.

No forecaster anticipated the possibly of a truly major snowfall until late Saturday afternoon. Even then, most forecasts did not fully capture the extent of the burgeoning snowstorm. (Note, for weather geeks looking to get a meteorological explanation for the late-changing forecast, MPR provides a good analysis here.)

Rather than provide forecaster grades as we have in the past, we’ll focus on a few general observations. Given that we don’t capture forecast activity in a truly systematic, day-by-day manner, we feel that the assignment of grades suggests a level of authority and exactness we simply don’t have. Still, we believe we monitor the forecasts enough to offer reasonable opinions. As always, take them for what they’re worth. We invite readers to provide grades and thoughts in the comment section to this post.

Our observations:

The best performers for this storm were the National Weather Service (NWS) and Novak Weather. Most other forecast outlets did not distinguish themselves.

Virtually all forecasters eyed this storm for the better part of a week. However, several minimized the storm as recently as Thursday. On Thursday, KMSP tweeted, “Winter returns the next few days, flurries late Fri afternoon, flurries Sat and wicked cold by Mon.” On Thursday evening, the Star Tribune noted the possibility of 1-3 inches of “oatmeal-like slush,” jokingly suggesting it would be cause to contact MnDOT and the National Guard. Likewise, the Weather Channel noted the possibility of mere “snow showers” over the weekend. At the same time, however, other forecasters were not ready to conclude the storm would be a minimal, back-page item.

KMSP was Jekyll & Hyde on this one. After the overly minimizing tweet on Thursday, they had the best standing prediction on Friday night with a forecast of 6-8 inches. But by Saturday, they’d revised it downward to 4.1 inches (the “snow meter.”)

The NWS consistently predicted higher amounts than the majority of forecasters 36 hours prior to the storm’s arrival. Some readers/viewers may have wondered whether a winter storm warning for MSP would prove to be warranted; clearly, it was.

Novak Weather also lead the way on this forecast, staying the course of a forecast of 6+ inches for MSP from early on. On Saturday afternoon, Novak Weather was also the first to communicate the possibility that accumulations could “get out of hand quickly for MSP” on Sunday as they ultimately did.

The separation of men from boys on this forecast took place Saturday afternoon and evening. Some forecasters picked up on the late-developing possibility of a truly significant snowfall while others did not. As detailed in the previous post, forecasts by KSTP (Jonathan Yuhas) and KMSP (Steve Frazier) missed the boat in not noting the late change in the storm’s evolution (though it should be noted that other members of the KSTP forecasting team tweeted 5-8 inch forecasts at the same time Yuhas was on air forecasting 3-6 inches).

It's Post Time... ehrrr Snow Time: Final Forecasts!

This weekend's snowstorm appears to have a mind of its own. A more southerly track has caused a rapid change in most, though not all, forecasts from Minneapolis/St. Paul weather prognosticators.

With snow on the doorstep, here was the latest thinking from Twin Cities forecasters late this Saturday evening. KSTP, KMSP and Accuweather stand out on the low side (about 4.5 inches) while WCCO, MPR and the Star Tribune are more along the lines of 8 inches.

WCCO: 5-10
KSTP: 3-6 (most snow along Highway 23 storm)
KMSP: 4.6 per snow tracker on news
KARE: 6
Star Tribune: 6-10+ per tweet at 9:50
MPR: 5-10 as of 5:40 per blog
National Weather Service: 6-10
The Weather Channel: 4-8
Weather Underground BestForecast: 8
Accuweather: 4.2
Novak Weather: 6-10 per tweet at 10:30
RandyinChamplain: 10
DDwx (Minnesota met from Atlanta): 8

Dec. 8-9 Snow Forecasts Locked and Loaded

A dusting of snow fell on ornamental grass in St. Paul on Friday.

Updated forecasts in blue font reflect forecasts as of late afternoon/early evening on Saturday. In general, there appeared to be a late afternoon trend toward somewhat higher snow totals and greater confidence in the storm.

We're about 24 hours away from the expected onset of the largest snowstorm to hit the Twin Cities in what seems like a long time. The highest snow forecast we found for the central MSP metro was for Fox/KMSP of 7 inches. The lowest was The Weather Channel with 3 inches. The most common forecast seemed to be 4-5 inches. Here are the latest forecasts that we captured late this Friday evening.

WCCO: 3-6 inches (10 p.m. news). Call it 4.5
KSTP: 3-5 inches (Jonathan Yuhas/KSTP Weather tweet). Call it 4. 3-6 as of 5pm news
KMSP: 6-8 inches (10 p.m. news). Call it 7. 4.1 with snow meter.
KARE: 4.6 inches (10 p.m. news). (4.6 for Bloomington) 3-6 (showed 4 for MSP)
Star Tribune Video (startribune.com/weather/5.39 p.m.): 6-7 inches. Call it 6.5
Star Tribune Paul Douglas (as of 11:45 p.m. blog): 2-5 inches. Call it 3.5 3-6 as of 4:40 tweet
MPR: 3-5 @MSP airport (as of 10:27 p.m. blog post). Call it 4. 3-7 as 4:10 update/5:40 tweet suggests 5-10" for metro.
NWS: "Around 6 inches" per warning statement. Call it 6. 5-9 inches as of 5:02 update
TWC: 2-4 inches per iPad forecast. Call it 3. 4-8 inches per iPad forecast at 5:40
Accuweather: 3-6 inches. Call it 4.5. 3-6 as of 5:40 p.m.
Weather Underground Best Forecast: 6 Up to 5 inches as of 5:45
Novak Weather: 6 inches (based on graphic). Call it 6. Solid 6 per afternoon graphic.
DDwx-MN met from Atlanta: 4
ShakopeeWeather: 4-6 inches. Call it 5

Note that we've made the call that the amount for the airport will be in the middle of the snow range forecast, a reasonable deduction given that the airport essentially represents the central core of the metro.

Roll the dice and let the games begin. Follow us on Twitter and like us on Facebook to get more timely updates.

Winter Weather on the Horizon (Redux)?

Forecasters in Minneapolis/St. Paul began to talk in cautious tones about a snow-making system for next Sunday/Monday. This no doubt is enough to pump the adrenaline of snow lovers or those who simply want some winter weather excitement. And so, we shall watch the progress.

Winter Weather Finally on the Horizon?

For snow and winter lovers, the wait may be almost over. The National Weather Service included the following in their afternoon statement this Monday, Nov. 19:

Long range models are beginning to hone in on the transition to a 
more active regime for next week. The 12z European model (ecmwf) was the most 
bullish in bringing a significant winter storm to the area between 
Monday night and Wednesday. The forecast was consistent with the 
Panhandle Hooker composite for 6+ inch snowfalls at kmsp. The 12z 
GFS trended toward this stormier scenario as well. There is still 
plenty of room for fluctuation in temporal/spatial details with 
this system...but the first notable snowfall of the season is not 
out of the question for early next week.

Tracking Sandy From Afar

As Minnesotans suffer through the first real cold jolt of the season, our friends on the Mid-Atlantic and Northeast coasts are under the gun from an already much-publicized tropical-ish storm. It's taken the weather world by storm -- no pun intended. Use this space to discuss your thoughts on this potentially historical weather event.

If you're looking for a great weather site that provides detailed, non-hyped commentary on Sandy, we recommend following the Capital Weather Gang.

"Frankenstorm" Nickname for Hurricane Sandy Stirs Controversy

The need to name significant, and sometimes insignificant, weather events has reached a new peak. Partially brought on by The Weather Channel's decision to unilaterally name weather events it considers important, the idea of naming storms has stirred great interest -- and now controversy. Many have nicknamed Hurricane Sandy "Frankenstorm." CNN has gone on record as denouncing the use of a name that it feels trivializes a potentially destructive storm.

The Capital Weather Gang reports that the "Frankenstorm" nickname stemmed from use in a National Weather Service-issued statement. And so the plot thickens.

Here are some twitter responses on the subject. Do you agree? Disagree?

The Curse of the Weather Radar: The Night That Weather Paranoia Ran Baseball

Baseball’s Murderer’s Row used to be the first six hitters of the powerful 1927 New York Yankees. Today, the game’s most feared opponent may well be the nasty little green and yellow pixels your neighborhood weatherman points to on the evening news.

What transpired tonight in Detroit was absurd. With a line of showers closer to Indiana than Detroit at the 8:00 p.m. ET starting game time, Major League Baseball inexplicably delayed the start of the fourth game of the ALCS. Finally, at 9:25, with light rain still 30 minutes away, MLB postponed the game.

This is an outrage on so many levels and rivals the idiocy of the 2002 all-star game that ended in a tie. Had the rain been minutes away from Comerica Park – truly close to meteorological inevitability – the delay would have been a bit more understandable. But it was far enough away that: 1) a good chunk of the game could have been completed, and 2) the precipitation could have dried up or circled around Detroit.

Unlike the situation in St. Louis, the precipitation heading to Detroit was merely steady rain, hardly the stormy weather that produced a Severe Thunderstorm Watch for the St. Louis area. It’s quite possible the game’s final innings could have been played before conditions worsened to the point where a legitimate delay may have been warranted. There was also a remote possibility that the entire game could have completed before the first drops of rain came down.

Just as it’s been since Abner Doubleday invented the game, October baseball is fraught with weather issues. It’s likely to be chilly, windy and rainy. It happens. And it’s been happening for a hundred years.

To its credit, MLB recently changed the rules regarding rain delays in the playoffs to ensure that no game would ever end without going its full length. The fact that this rule exists makes the decision to delay and then cancel a game nearly two hours before the onset of rain even more unconscionable.

To be sure, there are two interests that baseball is protecting. One it will admit and one it won’t. It’s possible, maybe even likely, that starting pitchers may not have been able to resume the game after a long delay. (We’ll let the babying of baseball pitchers be another topic for another day.) But, so what. That’s part of the game. Perhaps a weather-savvy manager might even be bold enough to hold out a starting pitcher in expectation of inevitable rain.

The second reason is – this just in – MLB wants to keep New York on the baseball radar for as long as possible. Had the team’s ace, CC Sabathia, been forced to come out of the game after a delay, it would have hurt the Yankees already slim chances of winning the series. The postponement effectively keeps general interest in the Yankees team for one more day.

We can only conclude that Bud Selig was doing his best Jim Cantore impression, sitting in front of a large monitor trying to make sense of those annoying little pixels. Clearly, he confused the green radar echoes with greenbacks.

P.S. A whopping .04" of rain fell during what would have been the third full hour of the game.

2012 Winter Forecasts

Please keep your eyes and ears open for the 2012 winter forecasts. We'll be collecting them and presenting them in a future blog entry.

To date, here's what we've heard of for winter forecasts:
Dave Dahl's forecast (61 inches of snow)
Live Weather Blogs (44.3 inches of snow, near average temperatures)
Paul Douglas forecast on Oct. 23:
I've mentioned it a few times on the weather blog. I suspect our dry bias will hang on into at least the first half of winter, thru December. A weak El Nino may steer the most significant storms well south and east of Minnesota, but my gut (a "wish-cast"?) is that we'll see more plentiful snows after the first of the year. I refuse to believe that we could see two 20" winters, back to back. The odds of this happening are slim. If I had to throw out an inch figure (dangerous) it would be in the 40-45" range, so more than last winter, but still almost half the snow that fell during 2010-2011, when a strongly negative NAO (North American Oscillation) blocking pattern kept a fresh supply of sloppy, southern storms pushing north across the Plains. My gut is telling me this winter will be closer to last winter than the 86" winter of 2011-2011, but I still suspect we'll see more snow (and more cold) than last winter. Stay tuned...

Here are last year's less-than-spot-on winter forecasts.

Fall Foliage (and Precip, Where Art Thou?)

This just in: the Twin Cities is in the midst of quite a dry period. What's your guess as to when the next substantial rainfall will occur? And/or when do you think we'll accumulate an inch of rain over successive precipitation events? And when, pray tell, do you figure the Twin Cities will see the first flakes of snow cascade down from the sky?

Here are a few shots of the emerging fall color that were taken from the Ford Bridge over the Mississippi River, not far from TMF world headquarters. They were also mentioned in the must-read MPR Weather Blog.

Fair Amount of Forecaster Variance for Weekend Temps

Actual weather: The official high temp at MSP for Saturday and Sunday was 85 and 80, respectively. Who was the best forecaster? Tough to say in this case. One conclusion is that most forecasters did not fully note the drop in temp between Saturday and Sunday. 

Friday update: As of Friday evening, KARE (locally) and Accuweather (nationally) continue to be on an island in predicting the coolest weather this weekend.

Will the weekend be quite warm or just pleasantly warm? There was a fair amount of spread among the forecasters as of Thursday evening for the Saturday/Sunday temperatures. Most shocking is that there’s a 7-degree difference in the forecast for Saturday, though it’s just 36 hours from this writing. In general, KARE is on the cool island for this weekend’s forecast while Weather Underground’s BestForecast is a clear outlier for Sunday’s forecast.

WCCO: 84/82 (Th), 84/88 (Fri)

KSTP: 82/82 (Th), 82/84 (Fri)

KMSP: 85/84 (Th), 83/85 (Fri)

KARE: 78/77 (Th), 80/80 (Fri)

Strib: 81/80 (Th), 82/83 (Fri)

NWS: 85/84 (Th), 82/84 (Fri)

Weather.com: 83/85 (Th), 84/83 (Fri)

Weather Underground BestForecast: 84/70 (Th), 84/88 (Fri)

Accuweather: 79/78 (Th), 80/78 (Fri)

Intellicast: 84/86 (Th), 85/84 (Fri)

Game, Set and Match: Weather Underground

If you enjoy watching and comparing the various forecasts issued by local and national weather sources, today made for great entertainment. We love it when forecasters separate from the crowd. It gives us something to watch when someone takes a chance.

We began monitoring the forecasts for today last Friday. It was hard not to miss the fact that Weather Underground BestForecast was out on a limb, five degrees higher than any other weather outlet. Comments to a recent post derided Weather Underground's renegade forecast, pronouncing it to be a "joke." Well, they're certain laughing back at WU's San Francisco headquarters in China Basin after today's high reached exactly what WU forecast last Friday. Well done, gentleman.

Here's how far off (under predicted) the various weather sources were based on today's actual temperature and Friday's forecast high.

Wunderground BestForecast: 95 (exact!)

Weather.com: 89 (-6)

Accuweather.com: 85 (-10)

Weather.gov: 89 (-6)

Wcco: 86 (-9)

Kstp: 85 (-10)

Kmsp: 87 (-8)

Kare: 85 (-10)

Strib: 80 (-15)

It's interesting to note that the best local television forecast was 8 degrees off, yet the national and governmental sources, excluding Accuweather, were within 6 degrees. Not a good showing for the locals.

How Hot for Next Tuesday?

There's considerable forecast variability for next Tuesday's high temperature as of this Friday evening. Here's a recap based on information available as 6 p.m. Friday.

Wunderground BestForecast: 95

Weather.com: 89

Accuweather.com: 85

Weather.gov: 89

Wcco: 86

Kstp: 85

Kmsp: 87

Kare: 85

Strib: 80

Excessive Heat Watch—Dead on Arrival

It was hard not to notice that the NWS office of the Twin Cities stood tall with the issuance of an Excessive Heat Watch some 60 hours before the time it went into effect. On Monday evening, the NWS advised us that excessive heat levels might be reached during the day on Thursday. It may not have been an unprecedented amount of warning, but it’s not something we don’t see often. One would guess that the issuance of a watch so far in advance would suggest the likelihood of serious heat, but that does not appear to be the case based on the “regular” hot forecast in effect as of this writing (6 p.m. Wednesday)

Something seemed amiss about this watch from the start. When the watch went into effect the only counties within many miles (five states perhaps) of the Twin Cities were Hennepin and Ramsey. This suggested it was a marginal at best. And even while the forecast temps at that point were close to 100, the dew point forecast did not go above 60—hardly oppressive.

We understand that a watch does not always mean a warning will follow. But for the NWS to trumpet the heat watch so many hours in advance suggested a certainty and confidence that didn’t bear out. In the end, the NWS/Twin Cites looks a little silly.

Dubious Heat Advisories

In a season when heat advisories and excessive heat warnings are making regular appearances in Twin Cities forecasting lexicon, we’re starting to wonder whether they serve much purpose or at least question the proper use of them. A comparison of today’s weather with yesterday’s provides a good example.

On Monday, the Twin Cities was under a heat advisory. The temperature reached 98 degrees but the humidity was only moderate, and the heat index never reached advisory levels. And while it doesn’t figure in to the formal mathematical calculation of heat index, a persistent, gusty southwest wind made the heat more tolerable than it might otherwise have been.

In contrast, today’s high temperature reached 94 but the humidity was more oppressive. Toss in light and variable winds and it’s fair to say that most people would think today was more uncomfortable. The NWS did not issue a heat advisory and the heat index technically touched advisory levels (100). On June 27, a formal heat advisory was issued, yet the heat index never reached 100.

With the benefit of hindsight, an advisory (technically) should have been issued for Tuesday rather than Monday. And when you throw in the ill-advised heat advisory on June 27, that’s three days where the issuance/nonissuance of an advisory was wrong. To us, this merely points up the dubious nature of the formality of heat advisories in the first place. Throw in the oddly worded “excessive heat warning” and you’ve got even more confusion.

It’s simply very hot and uncomfortable weather! Nothing more, nothing less. In our minds, the advisories and warnings that occur in winter have far more relevance and significance to our daily lives. If the windchill is a gazillion degrees below zero and you take a long walk without being properly dressed, you’re going to suffer frostbite at a minimum and you could potentially freeze to death. If heavy snow arrives, particularly during rush hour, it’s going to mess with your commute and you might need to plan accordingly. But if the heat index one day is 100 and another day is 98, is that really much of a meaningful difference? And more importantly, is one day more worthy of a heat advisory than the other? We think not.

If you're curious to see official definitions for advisories, warnings, watches and the like for the Twin Cities, click here.

Follow the Minnesota Forecaster on Twitter and Facebook. 

One Hot Week

A hazy, smoke-enhanced sunset heralded the end of a 99-degree day in St. Paul.

Whether it's labeled with "excessive heat warning," "heat advisory," or merely "stinking hot," it's hot, headline-stealing hot. The only suspense, it would appear, is whether the mercury will hit the century mark this week.

Hot, But Not Advisory Hot

Yes, it was hot today. And the oppressiveness came late and continues late into this summer evening. Make no mistake, if you don't have air conditioning, you're not a happy puppy.

The NWS issued a heat advisory today. A portion of the official wording from a statement issued at 10:09 a.m. read as follows:
 

... Heat advisory remains in effect from 1 PM this afternoon to 8 PM CDT this evening... temperature... in the mid to upper 90s this afternoon. The combination of hot temperatures and humidity will make it feel like 100 to 110 degrees.

Let's put the advisory to the test. The official high temperature at MSP was 93, not the middle to upper 90s. The highest heat index reported through 9 p.m. was 98. Not once did the heat index register in the predicted 100-to-110 degree range. One could even argue that the worst of the heat actually occurred from 4 p.m. to 9 p.m., but that would be splitting hairs.

A busted advisory? We think so. It's tempting to say, "Well, it was hot enough, so what's the difference?" But if that's the case, what's the point of issuing an advisory in the first place?

Do you think the concept of issuing a heat advisory has merit? Let us know your thoughts.

A Busted Summer Weekend Forecast (Like You Didn't Know)

Not that you needed us to tell you, but yes, it was a busted forecast this weekend. In fact, it could be argued that the inverse of what was forecast occurred for much of the weekend.

As of Friday night, the consensus of forecasters was that Saturday would likely bring rain by evening, possibly heavy, and that Sunday would be a delightful day for dads desiring good weather. This is a tweet that's representative of many others with a similar message.

In general, the complete opposite happened. Saturday morning provided an unexpected washout (approximately .5 inch of rain fell in St. Paul) and the afternoon (for the central Twin Cities) provided only fleeting light showers that were hardly worth noting. By 1 p.m. Sunday, rain had begun to fall in St. Paul, and the radar suggested that rain would threaten for the rest of the day.

Think the anger over a missed summer forecast isn’t real? Think again. Here’s what one tweeter had to say:

We applaud Jerrid Sebesta for invoking a little humor into the situation:

Tornado or Straight-Line Winds: When Does the NWS Investigate?

After seeing firsthand the significant storm damage from powerful winds that occurred in and around TMF headquarters in St. Paul last Sunday night, we had cause to wonder: When does the National Weather Service decide to further investigate whether storm damage was caused by a tornado or straight-line winds? Obviously, to a property owner who’s looking at a tree over their car, the exact cause is not likely to be a pressing concern.

We took our question to Todd Krause, the warning coordination meteorologist at the Twin Cities National Weather Service in Chanhassen. Here is what he told us:

There are a couple reasons why we do surveys. One is if there is a great deal of interest (usually by the media and public) in what caused the damage (that is a subjective call on our part). We also will do a survey if we are trying to figure out scientifically what happened, so that we can review the radar data and other environmental clues and learn something for the next time. Even if we have had warnings in place, it is still important to learn something about what we’ve seen on radar etc.

Having said that, there will be times when we do not perform a survey every time there is damage somewhere. Sometimes staffing is such that we cannot spare anyone to go take a look. Or maybe more severe weather is anticipated and we need to remain at the office.  Or maybe somebody has provided pictures/video of a tornado or downburst and it is patently obviously what caused the damage. Or the scenario is obviously a certain type of damage (e.g. the Goodhue County storm damage on June 14 – see http://www.crh.noaa.gov/images/mpx/NewsStory_June14.pdf).

Follow the Minnesota Forecaster on Twitter and Facebook.

Storm Damage from June 10, 2012

Thanks to WCCO for including our input on the 6 p.m. newscast. Our work was also featured in the Star Tribune as well as the MPR News Updraft weather blog.

Considerable damage was sustained from a severe storm in Highland Park, St. Paul. There were no sirens prior to the storm. Here are a few glimpses of damage that occurred within a few short blocks of each other. We counted at least five trees that landed on cars and several streets and alleys were impassable. Trees generally seemed to fall in a southwest-to-northeast direction.

With wind gusts reaching 59 mph at MSP, approximately three miles away, it seems like that winds approached or exceed 70 mph in this area.

(Photos copyright Bill Stein.)

Follow the Minnesota Forecaster on Twitter and Facebook.

Considerable Forecaster Variability for Mem Day Weekend

Analysis: The Memorial Day Weekend forecast proved to be a challenge. The official high temperatures at MSP for Saturday, Sunday and Monday were 64, 92 and 80. None of the forecasters shined, though we'd say the Strib probably did the best, coming very close to nailing both Sunday and Monday temps. WCCO was out to lunch with the Saturday forecast, missing it by 15 degrees and standing out from the crowd in the process. This just in: long-range forecasts, particularly for this part of the country, is not quite an exact science!

A changeable forecast for Memorial Day Weekend means great variability in forecasts from your beloved local and national weather outlets. For Saturday, Sunday and Monday, there are spreads of 7 degrees, 14 degrees and 9 degrees respectively between forecasters. In general, the Star Tribune is the most bullish on weekend warmth (followed closely by KSTP) while WCCO is generally the coolest. Here are temperature forecasts as obtained at noon on Tuesday.

#4: 76/77/72

#5: 74/86/82

#9: 74/81/73

#11: 75/na/na

NWS: 74/83/73

Accuweather: 70/81/70

TWC: 69/85/72

Strib: 72/91/81

Putting the Rain-Predicting Apps to the Test

RainAware Reigns Supreme in Precip-Predicting App Market

The Background

Two things are abundantly clear at this point in civilization. One, weather technology is becoming ever more complex and sophisticated, capable of detecting things once thought impossible. And two, man is always seeking to gain some measure of “life management” over the whims of Mother Nature. And so it seems only natural that new precipitation-predicting weather apps have emerged on the scene, ready to guide us through the day without getting wet.

The Test

We decided to give RainAware, Dark Sky and Ourcast, three of the newer rain-predicting apps, a test on a stormy night in Minnesota. (See also our exclusive interviews with the founders of RainAware, Dark Sky and Ourcast.) We began checking each app beginning at 6:45 p.m. and subsequently recorded their predictions every 15 minutes thereafter until the rain began. Likewise, once it became clear the rain would eventually end, we recorded the apps’ predictions for rain-ending times starting at 12:45 a.m., and then rechecked the apps every 15 minutes until the rain ended.

The Results

As our results in the accompanying graphic reflect, RainAware was the most accurate in determining both the beginning and ending times of the rain. RainAware locked on to the precipitation early and rather accurately. It came quite close to predicting the actual time of rain onset a full hour and a half before it arrived. And while it initially waffled a bit on the actual start time and experienced a server problem that made updates inconsistent for a short period, it provided a rain starting time nearly three hours in advance. In contrast, Ourcast seemed to think it was raining a full two hours before a single drop fell from the sky. Dark Sky, which doesn’t predict rain until it sees its arrival within a one-hour window from the current time, was slow to pick up on the ultimate arrival of the rain. At 8:45, Dark Sky predicted rain would begin at 9:35, when in fact it began at 9:15.

An analysis of predicted starting and stopping times revealed that RainAware was the most accurate.

RainAware was equally impressive in predicting an accurate time for the end of the rain. At 12:45 a.m, RainAware predicted the rain would end at 1:23. Dark Sky predicted the rain to last through 1:45 and Ourcast predicted the precipitation to last through at least 2:10 a.m. The rain ended at our location at 1:20.

Ground Clutter a Challenge for Dark Sky and Ourcast

Both Dark Sky and Ourcast also had challenges grasping the ultimate end of the rain. Both apps – to varying degrees – continued to think it was raining after the rain had actually stopped. The inability to decipher ground clutter from precipitation appears to be a continuing problem for both Dark Sky and Ourcast, as we’re seeing a reoccurrence of the problem as of this writing (May 2, 9:50 p.m.). While not perfect, it’s clear to us that RainAware is the superior app when it comes to detecting real rain from radar noise.

RainAware

In addition to RainAware’s actual performance in predicting rainfall, we also think the app’s features are generally the best of the apps tested. RainAware provides the longest lead time in rain prediction with a three-hour window. The three-hour window “messages” also come with informative statements about possible rain events even when there are no specific rain times. For example, it will suggest “showers could develop at any time,” or “dry now but a growing chance of rain” that we think provide a valuable “heads up” to users.

We also like the very simple but effective 7-day weather forecast that RainAware includes. While the main purpose of the app is to provide start and stop times for precipitation, the big-picture forecast means there’s no need to consult other apps for more general weather information.

Users desiring a pretty or interactive radar may be disappointed by RainAware. However, we think the radar is far secondary to the main function of the app, which is to provide start and stop times for precip. Besides, there are a number of other apps on the market dedicated exclusively to radar.

Dark Sky

Dark Sky brings undeniable beauty to radar depictions, which historically have been clunky and jittery. We also appreciate that all the information is boiled down to one screen, which includes confidence and forecast of precipitation strength. The app also provides the ability to backtrack two hours on the radar so that one can see what amount of precipitation passed through the area. Clearly, there’s some good innovation at work in this app.

However, we think the one-hour forecast window is insufficient, particularly when there’s no other information related to the overall forecast. If it’s noon and you’re wondering about the odds of getting in an evening softball game, Dark Sky is not going to help you.

Ourcast

The feature we liked best about Ourcast, the only free app among the three we tested, was the ability to move quickly and smoothly from one point on the map to another. This functionality is not present in Dark Sky or RainAware. Also, if you’re a fan of being social with your weather, Ourcast provides the opportunity to commiserate with your neighbors. Otherwise, we weren’t particularly impressed by Ourcast.

Conclusion

For our money, based on both the results of our test and its overall features, RainAware is the best precip-predicting app on the market.

The Minnesota Forecaster provides analysis of both the weather and those who forecast it. For periodic updates, follow us on Twitter and Facebook.

Feature App Profile: Ourcast

Ourcast is one of several new apps on the market that seek to provide specific times for the arrival and departure of precipitation (we previously featured RainAware and Dark Sky). We sought to learn more about the app and had the following Q&A with

Mark Hohmann, Ourcast founder.

We’ve read that the Ourcast app predicts rain and snow with "unprecedented accuracy." Can you tell me more about what’s been done to test and verify this accuracy?

The biggest challenge is in making an accurate correlation between what radar is showing and what is actually happening on the ground in terms of rain observations. We found that when you can correlate the past behavior of the radar with past rain observations, you get a much better indication of precipitation. Our accuracy refers to the ability to correlate the radar to a rain observation. It comes as a surprise to a lot of people that radar is a technical signal and while a good indicator, it’s not always directly correlated with what’s happening on the ground.

It sounds like statistical modeling is a major part of the app. How much meteorological knowledge was used in developing the app?

We found that if you’re going to predict the next two hours of precipitation, the dynamics associated with a statistical model are better than a meteorological approach which would include a numerical weather model or a mesoscale model.

Do you envision your app being used more by weather enthusiasts or common folk?

Both. We’re definitely focused on the typical consumer that’s not a weather enthusiast. We think that’s where the biggest gap is in peoples’ ability to use the weather forecast in the short term. We want to give those people that ability. But weather enthusiasts should find it useful as well as it can help them determine when radar is really describing precipitation on the ground.

Ourcast makes use of “check ins” as a component in the app. What do check-ins provide?

Check-ins are user reports about what is going on on the ground. The more information that users provide, the more we can build on our accuracy. Check-ins also provide a social aspect to the app by providing an ability to discuss weather with people near you.

How dependent on check ins are you for accuracy?

We launched with a product we think is accurate. It will get more accurate as our community grows. Check-ins and user reports help us to tease out what’s ground precipitation from what’s clutter, ultimately adding more accuracy.

The way that we correlate radar to precip is through a large set of weather stations located around the U.S. This includes backyard weather stations, government sources, proprietary sources and Weather Underground stations. This provides real-time information and gives more accuracy, almost to neighborhood level reports. These weather stations also provide more data beyond precipitation including temperature and wind speed.

Are users more apt to be overwarned or underwarned about precip?

We try to be as accurate as possible. However, we have built in a slight bias for overprediction. We think people would rather be warned more than warned less – they’d rather be warned of rain that doesn’t happen than rain that comes without being predicted. In other words, there’s a bias to false positives more than false negatives.

How far out does the forecast go?

In general, our forecast window is two hours. However, depending on when you check it, you may catch the system in a cycle and it can be a little less. But it should always provide at least a one hour and 40 minute forecast. Beyond two hours, the model right now degrades, and at that point mesoscale models and traditional numerical models become better.

Are there locations where the performance of Ourcast is less reliable or precipitation patterns that are more problematic?

There are regional issues such as where the radar signal is weakest or concentration of ground stations (for observations) is weakest. Mountainous regions have a lot of radar blockage and also tend to have fewer ground observations because of less population. There can also be issues with storm types. Storms that are strongly delineated, have a defined leading edge and are part of regional fronts usually provide the most reliable accuracy. Light snow in winter can be more of a challenge. However, as we grow our community it will mean a system that’s less reliant on radar and we’ll be able to build a forecast that’s more robust.

What do you consider the strengths or uniqueness of your app?

We’re unique because of a combination of a strong forecast system and a strong social app. We’ve designed it to have both. When you can make it more interactive, fun, social, etc., it takes on a new dimension. The social aspect can mean that even when plans are cancelled because of the weather, it can be a more fun, social experience.

Where do you see the most opportunities for improvement in future upgrades?

We want to keep making the app more interactive and fun. We want to add the concept of rankings and badges, so that users who report will get rewarded more. We are also working to provide an alert for users that will warn them ahead of time when precip is coming or when it’s about to end. We think that will really help to add value to the product.

In testing the app on an iPod Touch, we’ve had a few instances when “low memory” issues have caused the app to crash. Is that unusual?

No, it’s not and that’s because the app is not made to perform on the iPod Touch but rather on iPhones. However, it can still work on an iPod Touch but it may have some bugs until we support that platform more effectively.

The app categorizes rain into drizzle, light rain or heavy rain, but there’s no designation for moderate rain. Is that by design?

Yes, that’s by design. We want to keep things simple and just have a few separate categories. The app includes dBZ levels that reflect the intensity of precipitation, so people can see the see the relative changes in strength of precip over time.

Severe Tuesday?

Forecasters are eyeing Tuesday as a day of potential severe weather in the Twin Cities and the region. Here's a depiction from the Storm Prediction Center as of the morning of Monday, April 30. Use this thread to discuss your thoughts regarding what may be Minnesota's first significant severe weather of the spring.

Feature App Profile: Dark Sky

Dark Sky is one of several new apps on the market that seek to provide specific times for the arrival and departure of precipitation (we previously featured RainAware). We sought to learn more about the app and had the following Q&A with Adam Grossman, co-founder of Dark Sky.

What are the differences between the iPad and iPhone versions?

They both have the same data, but the interfaces are different to reflect the difference in form-factor. On the iPad, for example, we show the radar map and future prediction on the same screen, whereas we split them up on the iPhone. Personally, I like exploring the radar on the iPad's big screen and use the iPhone to check the weather on the go.

Were meteorologists included in the development of the app?

Actually, no. My schooling was in physics, not meteorology, and the other two co-founders are both computer guys. The whole thing started as a side project a couple years back, just to see if something like this were possible. I got sick of getting stuck out in the rain, and decided to experiment with applying statistics and machine learning to the problem (since I lacked traditional meteorology experience). Against all odds, it seemed to work really well for prediction precipitation in the near term.

Is weather knowledge built in to the app or is it more an interpretation of the movement of cells on the radar?

We take a statistical approach, rather than using physical/meteorological models. So the predictions we make are based on how the particular storm -- and storms like it -- have moved and developed in the past.

What do you see as the advantage of your radar depiction (vs. traditional radar)?

For a lot of people -- especially those who aren't weather nerds and aren't used to looking at and interpreting weather radar -- conventional weather animations out there can be very confusing. Because the doppler radar stations only take new images every five to ten minutes, they tend to result in clunky, jerky animations that are hard to follow. So what we've done is take our prediction algorithms and apply them to the time periods in between the radar frames, allowing us to create smooth and fluid animations. It really makes it a lot easier to see how the storms are moving and changing, and where they're headed.

Have you tested the accuracy?

An important part of our statistical approach is that the system constantly monitors its own accuracy: Every time a new radar images comes in, we use it to compute the error of past predictions. Because of this, we can tell in real-time which storms we have accurate predictions for, and which ones we don't. We reflect this in the interface as a "wobble" in our graphs. The more wobble, the less confident in our predictions we are.

Are users more apt to be overwarned or underwarned about precip?

It depends on the area and the type of precipitation. Light, spotty, slow moving precipitation is the hardest for us to predict, so in those conditions you might get some light sprinkling that we didn't anticipate. On the other hand, radar images also have a lot of "noise" in them (i.e. regions that look like precipitation that actually aren't) so if we don't do a perfect job cleaning them up, it can lead to false-positives. Fortunately, as we gather more data, we're constantly improving both our cleaning and prediction algorithms.

Are we correct in assuming the window of projection is one hour?

Right now we're restricting it to a forecast for the next hour. We plan to expand beyond the hour in future app releases, as we improve our prediction capabilities.

Does the radar show the past hour of precip as well?

Yes. You can scrub back in time over the past 2 to 3 hours. On the iPad, there's a history button to load this past data (which will be clearer in an update of the app we have coming out in the next day or two).

Are there locations where the performance of Dark Sky is less reliable or precipitation patterns that are more problematic?

We're less reliable in places with off-and-on sporadic light rain that just sort of sits over an area all day: think Seattle. We're most accurate for stronger storms, such as thunderstorms and those nice cohesive squall lines you'll often see rolling down the plains. 

What do you consider the strengths of your app?

Our goal was to make a weather app that was easy and fun to use for everyone, not just weather junkies. So we've put an emphasis on design, usability, and making the best radar visualization out there. I think that really sets us apart from the other apps out there.

  

Where do you see the most opportunities for improvement in future upgrades?

We have a big list of improvements we want to make, some minor and some major. One of our biggest priorities is a notification system: The app is only helpful if you remember to consult it, and I've personally been caught off guard by the rain because I just didn't think to check the app. So we want to build in notifications that will actively warn people when rain is headed their way.

Improvements to the underlying prediction algorithm are also a huge priority, of course. We're constantly improving things behind the scenes, and users should get the benefit of those improvements even without having to update the app.