How to interpret science: Explaining "sampling bias"

[Naomi Most]

Here's a well-written article explaining the concept of “Sample Bias” and how to avoid drawing over-generalized conclusions based on data that cannot support those conclusions.http://www.guardian.co.uk/science/2011/dec/02/biased-sample-statisticsExcerpted here:"A common type of bias is sampling bias, which arises from the method researchers have used to choose who or what is included in the sample. Sampling bias causes some types of people to be less or more likely to be included than others in the research, and so they are under- or overrepresented in the analysis.For example, if the national census could only be completed online then the sample would be biased by only including those people who have access to the internet. For instance it would exclude those who can't afford a computer or who don't have the necessary skills to use one."

[sckiely]

Thats such good information, so much of this sort of stuff goes unquestioned? I also find it interesting that so many studies utilise people from demographics likely to show the most benefits of the testing being undertaken! Many fitness regimes are tested on un trained people that have not exercised before or followed are sensible nutrition regime, so following ANY regime properly is going to lead to results!Another example would be the study of the benefits of eating breakfast that Kiefer cited where the test group were malnourished so ANY nutrition at ANY TIME of day is going to show positive responses!I am curious about the motivation behind these studies, apart from Dollar value! These people are scientists and professionals who you would assume were looking for realistic and effective results to REALLY BENEFIT the people they are aiming at, yet so many studies are flawed and/ or irrelevant to the vast majority of the people they promote/ market these benefits to

[Jeremy Wade]

Sampling bias:  Something simple like Pfizer leaving out results for anybody with mental disorders, alcohol consumption, disease, etc. from their safety studies submitted to the FDA when applying for approval to market Chantix as a stop smoking aid. (allegedly at this point)Or most behavioral science studies being biased by the subjects being "WEIRD" undergrads as asserted in this abstract:http://www2.psych.ubc.ca/~henrich/pdfs/WeirdPeople.pdf "Behavioral scientists routinely publish broad claims about human psychology and behavior in the world’s top journals based on samples drawn entirely from Western, Educated, Industrialized, Rich, and Democratic (WEIRD) societies."One more reason interpreting research can be such a daunting task and why it is so easy to manipulate the results of research to support your theories, even when the research doesn't actually support you.

[Hurley]

Good points. This is exactly why I never bother reading, much less understanding, any of the science behind what I believe. 

[Author]

Posts