Tuesday, June 18, 2013

The case for better public weather data

This morning, I was pointed by Chris St. Clair (@cstclair1) to an article on the BBC News site.

The article pointed out (paraphrasing) that "the Jet Stream is one thing they will consider, though the Jet Stream's position is a symptom and not a cause".  This caught my attention because the Jet Stream is of the metrics that I've been looking for in historical data, but it's never recorded anywhere that I can find it at least.  The second thing is it's not just the Jet Stream that is missing from our weather data…

As they correctly pointed out in the article, it's a symptom and not a cause… and so are many other things.  Anyone following my escapade into the Canadian weather knows that the day of the week is a factor, and that is also a symptom (likely pollution) and not a cause, yet they don't record the day of the week either (in my case, that's a pain as I have to recalculate 27 million reading dates to work out the day of the week and put that next to each record).

Other things that are not recorded:
  • What is our distance from the sun?
  • What is our angle of tilt to the sun?
  • Is El Nino or La Nina in effect?
  • What phase of the sun cycle are we in?
  • Are we being hit by a CME during that hour?
  • What is the local tidal phase?
  • What is the local atmospheric tidal phase?
  • What are the AQI readings at that time?

All these are just as important as the time, the date, the temperature or the atmospheric pressure at that moment, mainly because they add to the bigger picture of what's going on.  

When I look at the weather data that the public can generally obtain for free from places like Environment Canada, it feels like I'm only looking at one piece of the puzzle at any one time.  What we really need is to build a bigger picture so we can see what lines up…  Correlation doesn't necessary imply causality, but it helps us to determine where the links are.

In short, we need better data to be made available to the public.