Bad Statistical Reasoning about Weather and Climate

Yet another reader sent me a link to a really annoying article at a site called “Daily Tech”. The article has been more than adequately debunked by Darksyde at Daily Kos, but it’s a very typical example of a general kind of argument made both for and against global warming, which I find extremely annoying.

The basic argument takes one of two forms:

  1. Wow, look how hot it is today! How can anyone possible deny global
    warming?
  2. Wow, look how cold it is today! How can those idiots believe in global
    warming?

These are both examples of confusing weather with climate. That confusion is a typical example of a common statistical error:
using aggregate data to draw conclusions about specific individuals, or using a single individual to draw conclusions about an aggregate. Individual data points and aggregates are very different things, and you can’t just arbitrarily go from one to another.

Weather is what it’s like outside today. In terms of temperature,
weather is a single data point, like today’s temperature here in New York.
Climate is the total set of data describing the temperature everywhere around
the world, every day, for at least a full year.

If we had an unusual number of 98F degree days here in NY last summer, that
wouldn’t be any kind of evidence that there is global warming. If
Edmonton Alberta had an unusual number of -30C degree days this winter, that
wouldn’t be any kind of evidence that there isn’t global warming.

Global warming is a statistical trend, that global average temperatures are changing by a couple of degrees celsius. It doesn’t mean that we’re not going to see extremely cold weather. It doesn’t mean that we’re going to necessarily see unusually warm weather. It doesn’t mean that it’s going to get warmer everywhere. In fact, it’s almost certain that some places will become
colder as a result of global warming!

I’ve explained before that when you’re dealing with statistics dealing with an aggregate, those statistics are only descriptions of the aggregate; you
can’t just arbitrarily assume that you can apply the aggregate data in a precise
way to a particular individual. For a common example, we know that the average male american will live to be around 73 years old. Pick a random person, and we can’t make any real prediction about how long they’ll live. Some people will die at
30; some will die at 50; some will live to be 100. In my family, my favorite uncle, who my son is named after, was a healthy guy, and he died in his 50s. His brother, my grandfather, was very unhealthy – he had diabetes, high blood pressure, high cholestorol, congestive heart failure and emphazema, and he lived to be 86. Between the two of them, the average of their lifespans were very normal – but one died young, and one lived an unusually long time.

Climate is an aggregate statistic. It’s the cumulative average of the daily weather every day, every place in the world. Global warming is a general directional trend – the global average temperature is changing.

As I said above – there are some places likely to become colder as a result of global warming. For example, Great Britain currently has weather quite a bit warmer than is typical for its latitude. One of the major reasons for that is
an ocean current called the mid-atlantic conveyor, which carries warm water from the equator up towards Europe. Global warming, by causing an influx of cold water from melting ice could change the ocean currents,
essentially halting the conveyor, resulting in Great Britain and much of western Europe to become significantly colder.

You just can’t reason from individual data taken in isolation to
conclusions about an aggregate. You can’t pick an arbitrary sample from the
aggregate, and use it to draw conclusions about the aggregate – you need to carefully select a representative sample. What people commonly do, when they’re looking at weather, is looking at a subjectively selected, non-representative sample, and trying to use it to draw conclusions about
trends in the global aggregate. That’s just not valid statistical reasoning. If you want to make predictions about an aggregate, you have to understand that aggregate, how it’s computed, and what it measures. You have to understand the data
and how it’s collected in order to know what it means to take a representative sample of that aggregate. You have to use a very careful, rigorous process to
select a representative sample and perform any kind of meaningful reasoning
from it.

0 thoughts on “Bad Statistical Reasoning about Weather and Climate

  1. Rachel

    And have you seen the Dec. 5, 2007 article by Ross McKitrick in Canada’s National Post online about the data used to support the global warming hypothesis being contaminated? It’s very interesting. He served as an external reviewer for the IPCC report which was published in May 2007.

    Reply
  2. stewart

    Rachel, you might want to remember:
    a) McKitrick is a conservative economist, not a climatologist.
    b) anyone can be an external reviewer on the IPCC report, all it takes is signing up. The only people who seem to make a big deal of this are those with no qualifications otherwise. Read some of the comments and responses, and you’ll see some wacky stuff.
    Want to be a reviewer? …’If you are interested in reviewing the report, send a message — with your name and affiliation in the subject line — to ipcc-usgrev@climatescience.gov to obtain the username and password required to access the report.’…
    Climate is the integral of weather, over time and space. Any single incident is an anecdote, we need to have the pattern. However, the pattern is profoundly strong, and inconsistent with any of the other hypotheses that have been proposed.

    Reply
  3. bill r

    Good distinction, but you also need to remember that the aggregate (the parameter) is a simple linear function of the parts, and predictions about changes in the aggregate are also predictions about average changes in the individual parts, unless the prediction also includes a cutout. So if you forecast a 1 degree increase over, say, 5 years, with a s.d. of whatever, you are making a forecast about every point in the population at the end of that period.
    The data used to form the model is not in any sense a random sample. Its a convenience sample with some effort to add a systematic grid around the historical points.

    Reply
  4. Anonymous

    “However, the pattern is profoundly strong, and inconsistent with any of the other hypotheses that have been proposed”
    Oh ya. So how profoundly strong is it? Can you quantify?
    Also you say that its inconsistent with other hypotheses. First of all climate models are extremely inaccurate. They don’t make any definite predictions and the usually give ranges. So I would say that the hypothesis is not even a single hypothesis rather it is a large number of different hypotheses which are not consistent with each other.

    Reply
  5. Joshua Steffan

    There isn’t global warming. It’s just scientists feeling the proximity of The Lake of Fire. 😉
    Now go on patting yourself on the back how cleverly you reply to this…

    Reply
  6. Jeff

    I will not say much about this topic other than to redirect your attention to the fantastic work of the many scientist,researchers,engineers,mechanics, and all the support folks at Nasa programs scattered about the U.S.A. and abroad. Many Universities are participating in concrete research programs that have yielded much valuable data.The process of subatomic particle manipulation and transformation by the spheres is paramount in the explanation of climate. As you sit there and read this , a great many satalites orbit overhead dedicated to the collection of this data.Never in the history of modern man have this many people been focused on earth sciences.I believe we are moving in the correct direction.If there is a bright spot for the future,then Mike Griffen and his Nasa is by far… The Brightest!

    Reply
  7. derek

    Old statistical joke: Bill Gates walked into the homeless shelter, and everyone in the room became a millionaire, on average.

    Reply
  8. SteveF

    As I said above – there are some places likely to become colder as a result of global warming. For example, Great Britain currently has weather quite a bit warmer than is typical for its latitude. One of the major reasons for that is an ocean current called the mid-atlantic conveyor, which carries warm water from the equator up towards Europe. Global warming, by causing an influx of cold water from melting ice could change the ocean currents, essentially halting the conveyor, resulting in Great Britain and much of western Europe to become significantly colder.
    Sadly, since I live in the UK and like to ski, this is actually unlikely. Not out of the realms of possibility, but unlikely. The thermohaline circulation seems to be much more stable in interlacial times and it is becoming increasingly apparent that the amount of freshwater required to perturb the system is extraordinarily large (e.g. we don’t have the north American and Scandinavian ice sheets sitting around melting any more).
    Tis a shame indeed.

    Reply
  9. Dunc

    anyone can be an external reviewer on the IPCC report, all it takes is signing up. The only people who seem to make a big deal of this are those with no qualifications otherwise.

    Yup. I’ve now got to the stage than whenever I see the words “IPCC External Reviewer”, I automatically read them as “Unqualified Crank”.
    I am so signing up as a reviewer for the next one.

    Reply
  10. LRM

    Quoth Joshua:

    There isn’t global warming. It’s just scientists feeling the proximity of The Lake of Fire. 😉
    Now go on patting yourself on the back how cleverly you reply to this…

    No need, really. Your words are so profoundly, agonizingly, mind-bogglingly stupid that just about any reply would come off as clever in comparison. I almost find myself hoping that you sincerely believe what you wrote: the idea that anyone in this world would find such sub-moronic tripe humorous is simply too frightening to contemplate.

    Reply
  11. Michael Swart

    And I think you can go further. While weather is not climate. Climate is then not global climate. Listening to Quirks and Quarks (a Canadian Science Radio Show) they mentioned that antarctica is getting a bit cooler (climate-wise) even while the average global temperature is increasing.
    Didn’t I hear a few months ago a story where somebody recognized that we don’t have enough weather data for large portions of the globe. I’m sure the weather/climate is documented very carefully in populated areas like Chicago or New York. But there are probably mega-acres of pacific ocean or Sahara desert that aren’t.

    Reply
  12. Nomen Nescio

    good points all. the analogy i like to use goes, think of the last person you left a tip for — maybe a waitress, bartender, or whatever. whoever it was, they likely make much of their income as tips, and hence their daily take-home pay is likely fairly variable and unpredictable. like the weather.
    got that? okay, then, global warming is the national economy. it’s got nothing to do with whether J. Random Pizza Boy got an extra twenty dollar bills today or got gypped.

    Reply
  13. bill r

    Nice reflexive post, Dunc.
    Back to the main topic, a representative sample is a folk statistics term, not something that is well defined in the statistical sampling literature. In order to make rigorous inference from the observed stations to some generic, not well defined, SAT, one needs to demonstrate something along the lines of the cancellation axioms of conjoint measurement, or Taylor series types arguments. There is a small problem of (practically) restricted temperature ranges though. That always makes additive arguments fun.
    Reminds me of the farmer, the physicist, and the spherical cow.

    Reply
  14. Joshua Steffan

    @LRM: Well done! Goes to show what kind of hateful, bitter people promote global warming and bash genuine Christian scientists.
    I’m using humor, but unlike the quibbling, controversial “science” with its smoke ‘n’ mirrors, I use THE ONLY COMPLETE SOURCE THAT CAN BE COMPLETELY TRUSTED.
    So you may think I’m stupid, but I’m going to spend the eternity with my Father, and you’ll be frying in hell… who’s the stupid one now?

    Reply
  15. Morgan

    Sadly, since I live in the UK and like to ski, this is actually unlikely…
    Tis a shame indeed.

    Yeah, well, I live in Ireland, and I can tell you that we don’t need to get any freakin’ colder. Sheesh.

    Reply
  16. Richard Simons

    Joshua: What I get from your posts is that you do not think that global climate change can be occuring but are too cowardly to come right out and say so, because if you did that you would have to justify your views and you can’t.

    Reply
  17. DouglasG

    All we need to know about global warming is 2 things.
    1) Adding CO2 to a system will warm it. We have experimental proof that CO2 is a “greenhouse gas.” We have had this information for over 100 years.
    2) We have 400,000 years (or so) of CO2 data from Arctic Ice cores. From this data, we know that the CO2 levels in the air were relatively stable for the last 10,000 years. However, starting at about 1800, the CO2 levels began to rise. They have continued to rise since then, and they rise faster the closer you get to now. Thus, beginning at around the Industrial Revolution, the burning of fossil fuels has led to an increase in CO2 in our atmosphere.
    If you hear anything to the contrary, this individual has an agenda.

    Reply
  18. Spot

    The Corpus Callosum made a very similar argument, but it was pertaining to the public not understanding statistics for medication. I see a correlation between this post and theirs, that the public is just bad with statistics. Are they the same mistake? Or are they two different misunderstandings about statistics?

    Reply
  19. Seth Manapio

    I’m using humor, but unlike the quibbling, controversial “science” with its smoke ‘n’ mirrors, I use THE ONLY COMPLETE SOURCE THAT CAN BE COMPLETELY TRUSTED.

    Your source is as completely silent on the topic of anthropogenic climate change as it is on the composition of superconducting ceramics. It is not complete or even relevant in the context of this discussion.

    Reply
  20. FhnuZoag

    I’m pretty sure Joshua is trolling, and in any case he isn’t worth replying to. (In anything other than the third person, I guess.)

    Reply
  21. FhnuZoag

    I have to point out though that Mark is wrong. Anomalous numbers of hot and cold days are evidence for or against GW – in that they have some effects than can be used in some bayesian updating scheme. Obviously, they aren’t going to be any sort of strong evidence, but it isn’t totally accurate to say that they aren’t any evidence at all.
    Even when picking a non-representative sample, if you have some knowledge of how the sample is picked, you can work out likelihoods etc, and hence cope in a way. There’s known ways of getting a distribution of the maximum of a random variable, for example. E.g., if you roll a hundred sided dice three times and find that the max of the rolls is 99, then that is decent evidence that the dice isn’t fair. (Probability less than 99 in all three tries is about 0.94)

    Reply
  22. Flex

    Mark Chu-Carroll wrote, “You just can’t reason from individual data taken in isolation to conclusions about an aggregate. You can’t pick an arbitrary sample from the aggregate, and use it to draw conclusions about the aggregate – ….”
    I’d add one more point. You can’t use the aggregate data to make a non-probabalistic prediction about a single data point.
    To use your example, you can’t use aggregate climate change data to predict the weather on, say, May 23, 2009, in New York City.
    In fact, without additional knowledge beyond the aggregate climate data, you can’t tell me what the weather was in New York City on May 23, 2007.
    Determining where a randomly selected individual data point lies on a aggregate data curve requires additional knowledge (additional tests) than the curve itself provides. At best you could say it has a probability of lying within a certain range, e.g. 94% chance within three standard deviations from the mean on a normal distribution.
    Statistical analysis is an amazingly powerful tool, but it has limitations.

    Reply
  23. cm

    Climate is the total set of data describing the temperature everywhere around the world, every day, for at least a full year.
    As someone pointed out, isn’t that the global climate? Climate can also refer to just one region (“Miami has a warmer climate than New York”).
    Also, some of the commenters seem to think that a single data point can also be called an “anecdote”. That’s not right. Anecdotal evidence just means casual observations that were not done as part of a careful scientific observation/experiment. But one day’s worth of careful weather sampling in an area provides not an anecdote, but a real data point. Mark’s gripe, of course, is that one data point is not enough to go on to make claims about climate or gradual climate change, and that’s importantly correct.

    Reply
  24. -

    E.g., if you roll a hundred sided dice three times and find that the max of the rolls is 99, then that is decent evidence that the dice isn’t fair. (Probability less than 99 in all three tries is about 0.94)
    How do you get that? My probability is rusty. Is it just that 99 and 100 are 2 out of a possible 100 die faces, and so the p for one roll is 0.02, and then you multiply that p by the number of rolls, 3, to get 0.02 x 3 = 0.06 = p(getting above 99), so 1 – 0.06 = 0.94 = p(getting below 99)?
    I don’t recall if the rule is here you can multiply the p of a single roll by the number of rolls (meaning the p’s are addititive). Is that right?

    Reply
  25. Narc

    I use THE ONLY COMPLETE SOURCE THAT CAN BE COMPLETELY TRUSTED.

    You are, of course, talking about the Book of Mormon here, right?

    Reply
  26. Stephen Wells

    My current line on global warming:
    I’m insulating my house, and I’m not turning down the heating. But I don’t expect the house to get any warmer.
    If that makes sense to you, you probably don’t expect global warming either.

    Reply
  27. Jim

    Unfortunately, to many people view the evidence for global warming with a jaundiced political view point. Also the changes occur slowly enough that you can’t “feel” the change. thus the average person gets confused about the whole thing. (they aren’t feeling is get significantly warmer)
    Once the change is large enough to be indisputable it will be to late.
    Right after 9/11 planes were grounded for a few days in the US. I find it fascinating that the pan evaporation rate changed for those few days. (increased) So just using that one parameter change – commercial jets not flying for a few days over the US – we had a measurable perturbation on local climate. To me this evidence is sufficient to convince me that we have a large effect on our climate. One that we need to work on reducing.

    Reply
  28. gwangung

    First of all climate models are extremely inaccurate. They don’t make any definite predictions and the usually give ranges.

    Which is the same for ALL models and ALL research that has predictive capabilities. So, what other distinguishing features do yu think exists?

    Reply
  29. Jeff

    Jim, you are not alone. I too noticed a percievable change in the atmosphere.If anyone in the U,S went outside they too should have experienced the same.

    Reply
  30. bill r

    First off, you can falsify a model with a single observation if it falls outside of the range the model predicts. That’s the whole idea behind null hypothesis testing, or a Bayesian update. We saw a recent example in the subprime meltdown. The hypotheses of “no correlation” and “normal returns” have been pretty soundly rejected.
    Second, for #29, I have been updating my insulation without turning down the heat. I live in Northern Wisconsin. It doesn’t get warmer, but does cut down the drafts.

    Reply
  31. Eric Lund

    E.g., if you roll a hundred sided dice three times and find that the max of the rolls is 99, then that is decent evidence that the dice isn’t fair. (Probability less than 99 in all three tries is about 0.94)

    This comment is false for the same reason the statement “It’s unusually cold today, therefore global warming is probably a hoax” is false. It’s the fallacy of small number statistics. Yes, the expected number of rolls of 99 or higher is 0, but 1 is not statistically significantly different from 0 in this context. Nor is it improbable to have a particularly cold day in an unusually warm season. Now if you rolled the hundred-sided die three times and got a minimum of 99 (the probability of this is 8*10-6), then I would wonder whether the die was loaded, because 3 out of 3 is statistically significantly different from 0 of 3.
    To the respondent who was wondering how he got that probability: The calculation itself is correct (actually, the probability is a little higher than 0.94). Since the results of each roll are in principle independent, a result that has a probability p of occurring once has a probability pn of occurring n consecutive times. For numbers this close to 1 your method comes as close as it does because for |x| n as 1 + nx (in this example x = -0.02).

    Reply
  32. LRM

    Quoth Joshua:

    @LRM: Well done! Goes to show what kind of hateful, bitter people promote global warming and bash genuine Christian scientists.

    You misunderstand, Joshua – I’m not hateful or bitter. I am simply terrified at the thought that anyone could find your “joke” funny or witty. If you want to frighten me into conversion, you would do better to threaten me with having to listen to your jokes for all eternity: the Lake of Fire seems like a tropical paradise in comparison.

    I’m using humor, but unlike the quibbling, controversial “science” with its smoke ‘n’ mirrors, I use THE ONLY COMPLETE SOURCE THAT CAN BE COMPLETELY TRUSTED.

    Rush Limbaugh? Bill O’Reilly? Ann Coulter? Certainly couldn’t be the Bible, since it has nothing to say on the subject of Global Warming.

    Reply
  33. Eric Lund

    Despite my previewing, the last sentence in my comment #34 came out garbled. It should be: “…because for |x| much less than 1 you can approximate (1 + x)n as 1 + nx….”

    Reply
  34. Jeff

    Refering to post #30
    The color shift of the atmosphere shifted to such an extent that it was percievable with the human eye.
    The smell of the troposphere was such that I have not experienced for a very long time.
    This phenomenon has been on my mind since that day.
    Further study is warrented.
    Perhaps one day a year to set a baseline wouldn’t be to much to ask.

    Reply
  35. Torbjörn Larsson, OM

    And have you seen the Dec. 5, 2007 article by Ross McKitrick in Canada’s National Post online about the data used to support the global warming hypothesis being contaminated? It’s very interesting.

    You got to be kidding us! McKitrick is the global warming denialist evangelical scibling Deltoid is skewering. Over and over. Since 2004 in fact, so McKitrick earned his own denialist category. ‘Cause McKitrick is so brain deficit that he can’t distinguish between degrees of angle and radians. Which by any standard must be considered Bad Math:

    If you’re new here: In previous postings on Ross McKitrick I have shown how he messed up an analysis of the number of weather stations, showed he knew almost nothing about climate, flunked basic thermodynamics, couldn’t handle missing values correctly and invented his own temperature scale.

    And that was already 2004.
    Okay, if you by interesting mean that anyone still reads his crap, you may have a point…

    Reply
  36. Jeff

    I’ve studied particle physics for almost 30 years now and agree with the statment (every thing in the universe is recycled)All systems recycle on all scales.Does that fit into any ideology or ism?

    Reply
  37. Jeff

    So if one has an ephiphany(a moment of sudden and great revelation)without any known cause such as empirical,I suppose you could attribute this to divine intervention, reincarnation or perhaps primordal memory.
    In the Vedic belief does one retain memory from the previous life? By the way I thought spinoza felt familiar.
    Re:#17GMBM 2-4-1.Thanks.

    Reply
  38. Stephen Wells

    Just got back from a seminar on climate-change modeling. My new favourite line re. model uncertainty and climate change: think of a roulette wheel. My ability to predict its detailed future behaviour is zero, I have no idea what number will come up next. I can still predict with certainty that the house wins in the long run.
    Interesting historical titbit: Arrhenius, 1880s, predicted that a doubling of atmospheric CO2 would raise global temperature about 5 degrees. Local detail is tricky, but back-of-the-envelope still works globally.

    Reply
  39. Wry Mouth

    Always happy to see someone emphasize that statistics are about populations and not individuals. Persons on both sides of many issues (the economy, global climate change, the Democratic presidential race) are always flogging a day’s or an hour’s datum as indicator of trend, and my brain pops a few capillaries every time.
    Who needs alcohol? I get high on general ignorance. ;o/
    However — every time I get one more senior through a stats course, I feel that that is one more person inoculated against such hysteria.
    (vocations: math, biostatistics and education)

    Reply
  40. Kevin

    “The process of subatomic particle manipulation and transformation by the spheres is paramount in the explanation of climate”
    what?

    Reply
  41. Jeff

    #45-Kevin
    High speed solor wind contains mass of many ion species…and Leptons—electron,muon,tau,carry charge.Their purveyance thru the spheres,thermo,magno,iono,atmo,&litho along with the transitional pauses in between can be altered by any mass that comes between the source(sun) and final deatination.The earth has an energy budget,it is somewhat powered by the sun.ANY disturbance or alteration of these spheres produces a change to the final package.IE:Chemical change in the atmos(pollution+aerisols),or disturbance in the litho(mining,forest depletion,ect).Its much more complcated than this,and no one moderate change will have any great effect.It is the cumulative effect over time that seems to be valid.How does this effect Climate…Well,We’ll have to wait and see…or will we? :~)

    Reply

Leave a Reply to Eric Lund Cancel reply