Iterative Hockey Stick Analysis? Gimme a break!

This past weekend, my friend Orac sent me a link to an interesting piece
of bad math. One of Orac’s big interest is vaccination and
anti-vaccinationists. The piece is a newsletter by a group calling itself the “Sound Choice
Pharmaceutical Institute” (SCPI), which purports to show a link
between vaccinations and autism. But instead of the usual anti-vac rubbish about
thimerosol, they claim that “residual human DNA contamintants from aborted human fetal cells”
causes autism.

Among others, Orac already covered the nonsense
of that from a biological/medical
perspective. What he didn’t do, and why he forwarded this newsletter to me, is because
the basis of their argument is that they discovered key change points in the
autism rate that correlate perfectly with the introduction of various vaccines.

In fact, they claim to have discovered three different inflection points:

  1. 1979, the year that the MMR 2 vaccine was approved in the US;
  2. 1988, the year that a 2nd dose of the MMR 2 was added to the recommended vaccination
    schedule; and
  3. 1995, the year that the chickenpox vaccine was approved in the US.

They claim to have discovered these inflection points using “iterative hockey stick analysis”.

Continue reading Iterative Hockey Stick Analysis? Gimme a break!

Finger Trees Done Right (I hope)

A while ago, I wrote a couple of posts that claimed to talk about finger trees. Unfortunately, I really botched it. I’d read a bunch of data structure papers, and managed to get myself thoroughly scrambled. What I wrote about was distantly related to finger trees, and it was useful to help understand how fingertrees work – but it was not, in any way, shape, or form, actually a description of fingertrees. Since then, I’ve been meaning to write a proper post explaining finger trees – but with the work on my book, and with chaos at work, I just haven’t had the time. This time, in order to do my damnedest to make sure that I don’t screw it up again, I’m basically go to describe finger trees over a couple of posts by walking through the best finger-tree paper that I could find. The paper is “Finger Trees: a simple general-purpose data structure”, by Ralf Hinze and Ross Patterson. This might by the paper that introduced the structure, but I’m not sure.

The point of finger trees is pretty simple. It’s very similar to the point of zippers. Programming in functional languages is terrific. As I’ve described before, there are a lot of advantages to writing functional code. But there are also a lot of places where a naive implementation of an algorithm using a functional data structure is dreadfully inefficient. Functional code may be prettier, more maintainable, and more reusable – but imperative code is frequently much more efficient. When you’re doing an operation that, conceptually, modifies a piece of a complex data structure, then functional code can really suck. Finger trees give you a way around that – for many common updatabale data structures, you can build finger-tree versions that are very close to or fully as good as imperative, updating structures.

Continue reading Finger Trees Done Right (I hope)

Friday Random Ten, 4/23/2010

  1. Stellardrive, Inlandsix: Reasonably good instrumental prog. They’re
    not particularly exceptional, but they’re decent.
  2. Gong, “The Octave Doctors and the Crystal Machine”: Gong is a
    perfect example of one of the differences between the great prog bands,
    and a lot of the neo-progressive stuff. I can’t quite describe exactly what it
    is – but you listen to a band like Gong, and you never get bored. You can listen
    to it over, and over – and it’s always interesting. Even though the individual
    features of the music are similar to what a lot of less brilliant bands do,
    they manage to put them together in a different way. I can listen to a neo-prog
    band like Jadis or Frost once or twice a month; if I listen to them more than
    that, they start to bug me. But I can listen to Gong twice a day, and never
    lose interest.
  3. Parallel or 90 Degrees, “Backup”: One of the really great neo-progressives.
    Po90 is Andy Tillison’s other band, and they are brilliant. Not as brilliant as
    groups like Gong, but pretty damned amazing.
  4. Jadis, “All You’ve Ever Known”: Here’s exactly what I’m talking about.
    The beginning of this Jadis track is actually sort-of like the Gong track above.
    But somehow, it’s dull when Jadis does it. Listening to them right after
    Gong and Po90, they frankly sound terrible. I really like Jadis, but they can’t
    hold a candle to the prog greats.
  5. And So I Watch You From Afar, “If it Ain’t Broke, Break It”: Really good
    post-rock. ASIWYFA is on the louder end of post-rock, and they’re really good
    at it. They’re one of my most recent post-rock discoveries, after being recommended
    to me by a reader of the blog, and I’m really enjoying them.
  6. Genesis, “Your Own Special Way”: And now, my favorite band of all time.
    I love Genesis. Even after Peter Gabriel left, they still wrote some of the
    best prog rock of all time. There’s a reason why so many neo-prog bands were
    inspired by them. Even when they’re doing a song like this, which is basically a silly sappy ballad,
    they make it into something really special.
  7. Jacob Hoffman with Kandel’s Orchestra, “Doina and Hora”: an incredibly old
    recording of traditional klezmer, led by probably the greatest Klezmer xylophone player
    ever. If you have any appreciation for Klezmer, this will absolutely knock your
    socks off.
  8. The Flower Kings, “Soul Vortex”: Ah, the Flower Kings. The only
    neo-progressive band that I’ve found that’s really as good as the original
    prog guys. Whatever that elusive “it” that the great bands had that made them
    endlessly listenable was, Roine Stolt and the Flower Kings have it.
  9. Transatlantic, “The Return of the Giant Hogweed”: On their latest album,
    Translatlantic added a disk of covers of their influences. Naturally, no
    group made up of members of the best neo-progressive bands could possibly
    not include a classic Genesis track. It’s a very faithful cover, and
    it works really well.
  10. Marillion, “Forgotten Sons”: An old favorite of mine: one of the
    lesser known tracks from Marillion’s very first full album. From the
    very start, Marillion was really something special.

Book Update

Very quick post here: the third beta of my AppEngine book “Code in the Cloud” was released this morning. If you’ve bought a copy of the beta, you can go to your pragmatic account, and download a fresh copy with all of the fixes and new material.

If you haven’t bought a copy… Well, if you’re interested in cloud programming, I’d like to think that this book is a pretty good overview of the subject. It’s about Google AppEngine, but I’ve done my best to write it so that it discusses the nature of cloud programming in general, using AppEngine as a specific example of a cloud platform. Buying a copy supports your friendly math blogger and makes me happy; and when I’m happy, I’m more likely to write more posts for the blog :-).

The main reason that I’m mentioning it is that a few people have asked me to provide a forum on the blog for questions. I’m happy to answer questions, and I’d love to hear feedback from anyone who’s read it – both positive and negative. (And, to be honest, the negative feedback is generally more useful, so I’m very serious when I say that constructive negative feedback is welcome. Anything that you can tell me now, before it’s printed, is something that I can fix!)

So if you’ve got any questions or comments about the book, please go ahead and put them in the comments here. On the other hand, if you find any errors, you’re welcome to put them here, but it would help me more if you could file an erratum at the book’s pragmatic press page, linked above.

Shocking Fraud from Financial Scum

Against my better judgement, I’ve ended up writing a lot about the
financial mess that we’re currently going through. If you’ve read that, you
know that my opinion is that the mess amounts to a giant pile of fraud.

But even having spent so much time reading and studying what was
going on, the latest news from the financial mess shocks me.
Even knowing how utterly sleazy and dishonest many people in the financial world
have been, even knowing about the stuff they’ve been doing, the kinds of
out and out fraud that they’ve perpetrated, the latest news makes them
look even more evil than I could have imagined.

Continue reading Shocking Fraud from Financial Scum

I am a racist

(Unfortunately, this post has been linked to by a white supremacist site. Instead of providing a forum for their foulness, I’m shutting down comments on this post.)

Unfortunately, I lost the link that inspired this. But I recently saw a post by a conservative about “reclaiming” the word racist. It went on to list a collection of reasons why he was a racist. The gist of it was that all of us dirty liberals were the real racists – because there’s no possible reason for us to support things like affirmative action, welfare, etc., unless we really, deep down, believe that minorities – particularly blacks – are stupid animals incapable of taking care of themselves.

It’s typical bullshit. So I’m responding in my own way. Because, you see, I am a racist. I’m not proud of that fact – but growing up in a deeply racist and sexist culture, you can’t avoid absorbing racist and sexist messages and attitudes into your worldview. And the blogger who inspired this is, like me, a member of the privileged elite. The difference between us is that I at least try to notice the effects of my privilege. I don’t support social justice programs like affirmative action, welfare, and job training because I think that poor black people need help because they’re less smart than me: I think that people like me have unfair advantages that we rarely appreciate, and that everyone deserves the same advantages that I’ve been lucky enough to receive. But however idealistic I am, however commited I am to social justice, the fact remains: I am, to my shame, a racist.

  1. I am a racist – because I never noticed all of the unearned privileges that are given to me until someone pointed them out.
  2. I am a racist – because even after learning about the unearned privileges
    that I recieve, I still don’t notice them.
  3. I am a racist, because I have grown up in a culture that, at every turn, teaches
    me that to be white is to be better, and smarter, and I have absorbed that lesson.
  4. I am a racist, because I instinctively react to members of minorities with fear.
  5. I am a racist, because I live in a sunset town.
  6. I am a racist, because I believe that I deserve the success I
    have, even though I know people who are more smart, capable, and
    talented than I am never had the chances that I did to
    be successful, because of the color of their skin.
  7. I am a racist – because I am a white man who has directly benefited from
    the unfair preferences that have been directed towards me all of my life.
  8. I am a racist – because every day, I benefit from the denial of
    basic privileges to other people.
  9. I am a racist, because I do not notice the things that are denied to people
    who are different from me.
  10. I am a racist, because I do not notice the advantages that I have over
    others.
  11. I am a racist, because even when I do manage to notice what is denied
    to people of different races and backgrounds, I don’t speak up.

The point of this isn’t just to do a sort of “walk of shame”. The
point is that I am an incredibly lucky person, who has benefited from
all sorts of things – from where I was born, to the color of my skin,
to the background of my parents, to my gender. I have recieved, and
continue to receive benefits because of those, and many other factors
that have nothing to do with my own merit. And except for
very rare occasions, that goes unremarked, unnoticed.

People like me think of ourselves as the default – as “normal”
people. We consider the incredible advantages that we receive to
be normal, unremarkable. We don’t notice just how much we benefit
from that assumption of our own normality – the benefits we
receive fade into invisibility. We don’t even notice that they exist. And
then when someone who doesn’t get those benefits
has trouble, we naturally blame them for not being as successful as we
are.

The underlying theme of people like the jerk who inspired this
post is: “I made it by myself, without any help. So
they should be able to make it by themselves, without any
help either.”

But that’s bullshit, because none of us “made it by ourselves”. We’re
the beneficiaries of the system we live in.

I grew up in a wealthy town in NJ. We didn’t consider ourselves
wealthy – but by comparison to lots of other people, we really were.
I went to a very good school system. We complained about it a lot:
the textbooks were too old; the equipment in the science labs were too
beaten up; the classes were too easy, and so on.

When I was in college, I got to teach a summer program for top
students from schools in Newark, Camden, and Jersey City. And I
discovered that my students went to schools where they didn’t have to
worry about their books being too old – because they didn’t
have any books. I mean that literally: in their english
classes, they didn’t have books, because their schools had
never been able to buy new books since it opened – and the
books had long since fallen apart. They didn’t complain about the
lousy lab equipment – because their schools had never had
science labs at all. How could people coming from schools like that
possibly hope to compete with students from a school like
mine? I didn’t admitted to college over people from their schools because
I was smarter. I got admitted into college over people from their
schools because I was richer and whiter.

And when my students went to the campus bookstore to buy
basic supplies like paper and pencils, the people who worked there
followed them around the store – because what would a
bunch of poor black kids be doing in a bookstore if they weren’t
there to rob it?

I write this math blog for fun. How did I get the background to do
it? I come from a highly educated family. They taught me to read
before I even started preschool. I’d learned about statistics from my
father when I was in third grade. I learned about algebra in sixth
grade, even though my school didn’t teach it until 8th or 9th. I
learned calculus in my freshman year in high school – even though my
school didn’t teach it until a senior year AP class. I was learning this stuff
long before the school taught it to me; and my parents made sure that
they bought a house in a very expensive school district where there would
be things like AP classes. My parents paid for me to go to college – which gave
me the time to take courses not just because I needed them to graduate,
but because they covered things that I wanted to learn, just for fun.

How could a person from a family that just managed to scrape by,
who lived in a school system that couldn’t afford textbooks for the
basic classes, much less the AP classes, how could they compete with
me? It’s damned close to impossible. Not because they’re any less
smart, or any less talented. But because I’ve had an absolutely
uncountable number of advantages. Every day of my life, I’ve been
given benefits which helped make it possible for me to become who and
what I am. I’m here partially because I’ve worked damned hard
to get here. But that work, by itself, wouldn’t have gotten me to where I am,
without luck and privilege.

People like me need to remember that. We didn’t earn what we have
all by ourselves. We may have earned part of it – but only
part. An awful lot of what we have is built on privilege: on the advantages
that we’ve been given because of race, gender, wealth, and family.