There are a couple of things that we’d like to know from this data. First,
\ncan we predict the failure rate for a particular exposure? And if so, how reliable will that prediction be?<\/p>\n
To answer that, we first need to have some idea of what kind of relationship we’re looking for. In this case, we believe that the relationship should be linear.
\n(If it’s not, then we wouldn’t use linear regression, and this would be a different post.) So what we’re looking for is a linear equation that captures the relationship in that data. Since it’s experimental data, it’s not perfect – there are experimental errors, there are extraneous factor. So no line is going to be
\nan exact fit to the data – so we want to find the best possible fit.<\/p>\n
If you remember your algebra, a linear equation takes Mean-square in the above basically means that we take the difference between So how do we minimize the mean-square y-distance? It’s actually pretty easy. First m = (Σ(xi<\/sub>–x<\/span>) So, for our example data, x<\/span>=16, If we sum up “(y-y<\/span>)(x-x<\/span>)”, we get 29.8; if we sum up Once we have the slope, we need to compute the point where the regression line crosses the y axis, which is simple: b=y<\/span>-mx<\/span>. So the intercept is 11-0.42×15 = 4.7.<\/p>\n The resulting line – the “best fit” line according to our linear regression – is shown in the diagram. You can see it’s a reasonably good fit. But how good? There’s a standard What’s a “good” correlation coefficient? There is no single answer. If you’re working on The definition of the correlation coefficient is a bit messy. In fact, there are several For our particular data set, the correlation coefficient comes out to 0.88 – a very The interesting point is, we constantly hear people talk about correlation. You’ll frequently hear things about “correlation is not causation”, “X correlates with Y”, etc. Most people seem to think that correlation implies nothing more that “X is related to Y”, or “If X increases, then Y increases”. In fact, correlation means something very specific: X correlates with Y if and only if<\/em> there is a linear relationship between X and Y. Nothing more, and nothing less. Correlation means a direct linear relationship.<\/p>\n When you look at data, it’s easy to make the assumption that if, every time you change X Y changes in a predictable linear fashion, then changing X must cause<\/em> Y to change. That’s probably the single most common mistake made in the misuse of statistics. Real examples of this show up all the time. For a trivial example, my dog is more Suppose you’ve got a bunch of data. You believe that there’s a linear relationship between two of the values in that data, and you want to find out whether that relationship really exists, and if so, what the properties of that relationship are.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[61],"tags":[],"class_list":["post-619","post","type-post","status-publish","format-standard","hentry","category-statistics"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4lzZS-9Z","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts\/619","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/comments?post=619"}],"version-history":[{"count":0,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts\/619\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/media?parent=619"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/categories?post=619"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/tags?post=619"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nthe form y=mx+b, where “m” and “b” are constants. In this case, “x” is the
\nradiation level, and “y” is the failure rate. So what we want to do is to take
\nthe set of data, find values for “m” and “b” that provide a best fit<\/em> line for the data. The way that we define best fit is a line where, for a given x-value,
\nthe mean-square difference between the measured y-value, and the y-value predicted by our best-fit line is minimal.<\/p>\n
\nthe observed and the predicted, square it, and take the mean. That mean-square
\nvalue is what we want to minimize. But why square it? The intuitive answer comes
\nfrom an example. What’s a better fit: one where for a given pair of points, the
\ndistances are -10 and 15, or one where the distances are 7 and -2? If we use mean
\ndistance, then the two are the same – the mean distance in each case is 2.5. If we
\nuse mean-square, then we’ve got 162.5 in the first case, and 53 in the second. The “7” and “-2” is a better fit.<\/p>\n
\nwe want to compute the slope. Normally, the slope is defined as ΔY\/ΔX. We
\ntweak that a bit, making it (ΔYΔX)\/(ΔX2<\/sup>). Then we
\nuse that for all of the data points. So what we end up with is:<\/p>\n
\n(yi<\/sub>–y<\/span>)) \/
\n(Σ(x-x<\/span>)2<\/sup>)<\/p>\n
\nand y<\/span>=11. So we compute the
\nfollowing values for our data:<\/p>\n\n
\n y-y<\/span><\/th>\n<\/th>\n x-x<\/span><\/th>\n (x-x<\/span>)2<\/sup><\/th>\n<\/tr>\n \n -2.8<\/td>\n -5<\/td>\n 25<\/td>\n<\/tr>\n \n -0.9<\/td>\n -3<\/td>\n 9<\/td>\n<\/tr>\n \n 0.8<\/td>\n -1<\/td>\n 1<\/td>\n<\/tr>\n \n -0.7<\/td>\n 1<\/td>\n 1<\/td>\n<\/tr>\n \n 1.7<\/td>\n 3<\/td>\n 9<\/td>\n<\/tr>\n \n 1.9<\/td>\n 5<\/td>\n 25<\/td>\n<\/tr>\n<\/table>\n
\n(x-x<\/span>)2<\/sup>, we get 70. That gives us a slope of 0.42.<\/p>\n![\"regress.png\"](\"https:\/\/i0.wp.com\/scientopia.org\/img-archive\/goodmath\/img_302.png?resize=279%2C171\")
<\/p>\n
\nmeasure of the quality of a line-fit in a linear regression, called the correlation coefficient<\/em>. The correlation coefficient describes how well a simple linear relationship matches a set of data (which connects to an interesting point that I’ll mention in a moment). The correlation coefficient is unitless, and varies between -1 and +1. If the data is a perfect negative correlation – meaning that as X increases, Y decreases in a perfectly linear fashion, then the correlation coefficient will be -1. If the data is a perfect positive correlation – that is, a perfect relationship where when X increases, Y also increases in a linear fashion, then the correlation coefficient will be +1. If there is absolutely no linear relationship between X and Y, then the correlation coefficient will be zero.<\/p>\n
\nan experiment where you’ve done a really good job of isolating out any external factors, and
\nwhere you’ve got very precise equipment, the correlation coefficient required to conclude
\nthat you’ve correctly isolated a relationship could be very high – greater than 0.99. On the other hand, in social science work, anything over 0.5 is often considered good enough to
\ninfer a relationship.<\/p>\n
\ndifferent versions of it used for different purposes. The most common one is the Pearson
\ncorrelation, which is what I’m going to describe. The basic idea of the Pearson correlation
\ncoefficient is to compute something called the covariance<\/em> of the two variables in
\nthe equation, and then divide that by the product of the standard deviation of the two
\nvariables. The covariance is something that’s usually defined via probability theory, and I
\ndon’t want to get into the details in this post, but the basic idea of it is: for each data
\npoint (x,y), take the product of the difference between x and mean x, and the difference
\nbetween y and the mean y. Sum that up for all data points, and that’s the covariance<\/em>
\nof the variables x and y.<\/p>\n
\nhigh correlation for such a small data set. <\/p>\n
\nCorrelation shows that there is a relationship – but it doesn’t show why<\/em>.<\/p>\n
\nlikely to have an accident in the house if it’s very cold out. In fact, there’s
\na strong inverse correlation between the temperature outside, and the likelihood of my
\ndog having an accident. Does cold weather cause my dog to have an accident? No. Cold
\nweather causes me to try to avoid taking him out, and because I don’t take him out
\nenough, he has more accidents. The cold doesn’t affect my dog’s likelihood to
\nhave an accident directly. He’d be equally likely to have an accident if the weather
\nwere a balmy 70 degrees out, if I didn’t take him out for a walk. He doesn’t care whether
\nit’s 10 or 70; he just needs to go.<\/p>\n","protected":false},"excerpt":{"rendered":"