Monday, January 7, 2013

The Age of Discovery Is Not Over

This morning I was reading an old post about the Age of Discovery being over - that we've invented nearly everything we need and know everything about the world that is worth knowing about…  

Let me tell you a little story….

When I was a kid, my Computer Studies teacher made me write a function that could be used instead of the sqrt() programming function, so that provided any starting number, it could return the square root of that number.

Back then, we didn't have the Internet.  I was also relatively bad at maths.  My final code basically did this:

Imagine I was given 100.53 as the number to find the square root of.
1. Start with the number 2.  
2. Square it (2^2 = 4).
3. Is "4" higher or lower than 100.53?
4. It's lower, so try a higher number and do the calculation again.
5. Keep repeating until we're higher than 100.53, then start iterating down in slower increments.
6.  Keep repeating until we're lower than 100.53, then start iterating back up in even smaller increments.
7.  Eventually, with this process, you end up with something that's good enough to about 6 digits that the code returned as the answer.

After handing in my code, the teacher said "Ah, you've basically replicated the Newton Method".  Having never heard of this, he explained that it was very similar in that you revise a guessed number up and down until you arrive back at the answer you're looking for.

I thought about this for a moment…  If I'd been born about 400 years earlier, even with my bad math skills at the time, this could have been called the Coulls method, not the Newton method.

Fast forward to about 2008, and I was now a 30-something year old guy faced with a problem where some computer hardware that hidden in a bus kept failing in winter.  It didn't take a genius to realise that it was a temperature related issue, but the funny thing is it wasn't the cold that was causing the problem, but the vehicle's heating system that was cooking the hardware.

To pin down the parameters that caused the failures, I was eventually driven to trawling through weather data (oddly, I do this task quite often) and I noticed that the failures were happening more on the weekdays and less on weekends.  Eventually, I discovered that weekends were warmer by a degree or two than weekdays, and hence the drivers cranked the heat up less and there were less hardware failures.

I thought I'd discovered something big.  I double-checked my data.  It was accurate.  I then went looking (we had the Internet now) to see if this was a known phenomenon.  I found I'd been beaten to this discovery by just 9 years, no less by a university that's on my doorstep.  (

Remembering my Newton gap, it was clear I'd narrowed the gap from the 400 years to 9 years.  At some point soon, I should be catching up with the best of them, right?

It didn't take long for me to latch on to my next "discovery".  This was also weather related, being related to nocturnal atmospheric pressure patterns that I can pin down to a 7 day cycle, but this time I found that I wasn't beaten to the punch.

What I found this time was that I was chronologically seeing the problem at the correct time, but now there was no accessible data to corroborate with, and even Environment Canada had said that my "discovery" didn't make much sense as weather doesn't follow man-made cycles.  A few years went by and I spoke to a professor in the USA about my observation and what Environment Canada had said.  He basically said he wouldn't be surprised if I was right and to go back out and explore this more.  

It then occurred to me that when the people before had been trying to discover something new, it wasn't like a race where you can see who is ahead of you or catching up behind you.  

No.  Instead, I'm on my own.

Over the past 18 months I've been gradually planning how I would attack the problem and this past few months I gathered my data.  Now I'm just waiting for a bit of spare time where I can process it - but at least I have a plan on how I'll do it.  

More importantly, I have already seen what I'm looking for, so I will recognise it when I see it again - which is weird because nobody else is publicly reporting that they have seen the same thing.

I've often joked that I see a "different world" to the people around me.  Solving a problem like this has meant a lot of "thought-experiment time" over the past few years, and it squarely puts me in a place on a regular basis where there's nobody else.

As a result I disagree with all those that say the age of discovery is over, or that it's inaccessible to the average person.