{"id":627,"date":"2008-04-14T13:19:43","date_gmt":"2008-04-14T13:19:43","guid":{"rendered":"http:\/\/scientopia.org\/blogs\/goodmath\/2008\/04\/14\/probability-distributions\/"},"modified":"2008-04-14T13:19:43","modified_gmt":"2008-04-14T13:19:43","slug":"probability-distributions","status":"publish","type":"post","link":"http:\/\/www.goodmath.org\/blog\/2008\/04\/14\/probability-distributions\/","title":{"rendered":"Probability Distributions"},"content":{"rendered":"

I’d like to start with a quick apology. Sorry that both the abstract algebra and the new game theory posts have been moving so slowly. I’ve been a bit overwhelmed lately with things that need doing right away<\/em>, and
\nby the time I’m done with all that, I’m too tired to write anything that
\nrequires a lot of care. I’ve known the probability theory stuff for so long,
\nand the parts I’m writing about so far are so simple that it really doesn’t take nearly as much effort.<\/p>\n

With that out of the way, today, I’m going to write about probability distributions.<\/p>\n

<\/p>\n

Take a probabilistic event. That’s an event where we don’t know how
\nit’s going to turn out. But we know enough to describe the set of possible outcomes. We can formulate a random variable for the basic idea of the event, or we can consider it in isolation, and use our knowledge of it to describe
\nthe possible outcomes. A probability distribution describes the set of outcomes of some probabilistic event – and how likely each one is. A
\nprobability distribution always has the property that if S in the set
\nof possible outcomes, and ∀s∈S:P(s) is the probability of outcome s, then Σs∈S<\/sub>P(s)=1 – which is really just another way
\nof saying that the probability distribution covers all possible outcomes.<\/p>\n

In the last post, about random variables, I described a formal abstract
\nversion of what we mean by “how likely” a given outcome is. From here on,
\nwe’ll mainly use the informal version, where an outcome O with a probability
\nP(O)=1\/N means that if watched the event N times, we’d expect to see O occur
\nonce.<\/p>\n

Probability distributions are incredibly important to understand statistics. If we know the type of distribution, we can make much stronger
\nstatements about what a given statistic means<\/em>. We can also do all
\nsorts of interesting things if we understand what the distributions look like.<\/p>\n

For example, when people do work in computer networks, some of the key
\nconcerns are called bandwidth and utilization. One of the really
\ninteresting things about networks is that they’re not generally as fast
\nas we think they are. The bandwidth of a channel is the maximum amount of
\ninformation that can be transmitted through that channel in a given time. The utilization is the percentage of bandwidth actually being used at a particular moment. The utilization virtually never reaches 100%. In fact, on most networks, it’s much<\/em> lower. The higher utilization gets, the harder it is for anyone else to use what’s left.<\/p>\n

For different network protocols, the peak utilization – the amount of the bandwidth that can be used before performance starts to drop off – varies. There’s often a complex tradeoff. To give two extreme cases, you can build
\na protocol in which there is a “token” associated with the right to transmit on the wire, and the token is passed around the machines in the network. This
\nguarantees that everyone will get a turn to transmit, and prevents more than one machine from trying to send at the same time. On the other hand, there’s a family of protocols (including ethernet) called “Aloha” protocols, where you
\njust transmit whenever you want to – but if someone else happens to be
\ntransmitting at the same time, both of your messages will be corrupted, and will need to be resent.<\/p>\n

In the token-ring case, you’re increasing the amount of time it takes
\nto get a message onto the wire during low utilization, and you’re eating up a
\nslice of the bandwidth with all of those token-maintenance messages. But as capacity increases, in theory, you can scale smoothly up to the point where all of the bandwidth is being used by either the token maintenance or by real
\ntraffic. In the case of Aloha protocols, there’s no delay, and no token maintenance overhead – but when the utilization goes up, so does the chance
\nof two machines transmitting simultaneously. So in an Aloha network, you
\nreally can’t<\/em> effectively use all of the bandwidth, because as
\nutilization goes up, the amount of bandwidth dedicated to actual messages
\ndecreases, because so much is being used by messages that had to be discarded due to collisions.<\/p>\n

From that brief overview, you can see why it’s important to be able to
\naccurately assess what the effective<\/em> bandwidth of a network is. To
\ndo that, you get into something called queueing theory<\/em>, which
\nuses probability distributions to determine how much of your bandwidth will
\nlikely be taken up by collisions\/token maintenance under various utilization scenarios. Without the probability distribution, you can’t do it; and if the probability distribution doesn’t match the real pattern of usage, it will
\ngive you results that won’t match reality.<\/p>\n

There are a collection of basic distributions that we like to work with,
\nand it’s worth taking a few moments to run through and describe
\nthem.<\/p>\n