Is there a name for this statistical fallacy

[Bluegoat]

I'm not positive that this is in fact a fallacy - but it is a sort of misuse.  I've seen it a few times in different contexts and am wondering if there is a particular name for it or way to describe it.

 

I'll give an example.

 

The goal of a study is to determine if riding bikes is dangerous.  So they compare the bike injuries among people who ride bikes with bike injuries among people who do not ride bikes.  They conclude all the bike injuries are among the bike riders, therefore, bike rising ought to be avoided.

[SKL]

Could you be thinking of "non sequitur"?

[MSNative]

Biased and poorly designed study would be my name for it. ;). Wikipedia actually has a good list of problems with stats.

 

https://en.m.wikipedia.org/wiki/Misuse_of_statistics

[MyThreeSons]

It sounds like a typical Facebook post that "proves" someone's agenda.

 

[gardenmom5]

Lies, d*mned lies, and statistics.

 

It could also be labeled disingenuous.

[tm919]

Selection bias probably... off the top of my head, it seems like you'd need some sort of Heckman correction for that example.

[MerryAtHope]

It sounds like a typical Facebook post that "proves" someone's agenda.

 

LOL, the Facebook Fallacy!

[Jean in Newcastle]

False comparison or faulty comparison

 

http://www.logicallyfallacious.com/index.php/logical-fallacies/97-faulty-comparison

[Bluegoat]

Thanks for your thoughts.  It was a Facebook post that got me thinking, but I did go to the study abstract from the posted article, and as far as I could see, that was the structure of the article.  It was about FAS and FAE.  I'll have to look later and see if I can get more text, because it seemed to be kind of a crazy way to compare.

[Guest]

Hmm, It's almost, but not quite a post-hoc fallacy.

 

I'd be interesting in hearing what more specific designation you come up with!

[fdrinca]

I'd be interested in the theory behind designing the research project this way. Following for link :)

[greenvneck]

Are you thinking of a syllogism?

[Bluegoat]

I'd be interested in the theory behind designing the research project this way. Following for link :)

 

I think it was a metanalysis, so it may have been a matter of what they could come up with from the data they had.

 

But - it seems very weird to me.  So I am wondering if either the description was misleading, or maybe it isn't really meant to be used to create recommendations.

 

Ok, here is a link.  As you can see, it shows drinking increases the likelihood of a problem compared to the other group, but not much about any kind of objective likelihood of a problem.  But maybe that is in there but not in the abstract.

[Ali in OR]

I'm currently reading How Not to be Wrong--The Power of Mathematical Thinking by Jordan Ellenberg. He talks about lots of ways the general populace misunderstands things--like assuming that since the obesity rate is increasing, everyone will be obese by 2048. That error is assuming every curve is a straight line. I'm less than half way through and have no idea if it answers your question, but you might find it interesting reading.

[Bluegoat]

That does sound really interesting, Ali - I will see if my library has it. 

[fdrinca]

Is this this bit that's problematic?

 

First-trimester drinking, compared to no drinking, results in 12 times the odds of giving birth to a child with FASDs. First- and second-trimester drinking increased FASDs odds 61 times, and women who drink during all trimesters increased the likelihood of FASD odds by a factor of 65. - See more at: https://www.aap.org/en-us/about-the-aap/aap-press-room/pages/AAP-Says-No-Amount-of-Alcohol-Should-be-Considered-Safe-During-Pregnancy.aspx#sthash.s4Ny6Q3r.dpuf

 

I read this as expressing the increased risk associated with drinking/FASD, not to determine the correlation between drinking/FASD and no drinking/FASD.

 

Using "not drinking" establishes the baseline likelihood of observing FASD, which one would assume to be 0 but in reality isn't due to sampling error, weird metabolic issues, etc. 
 

 

 

[Tsuga]

 

 

The goal of a study is to determine if riding bikes is dangerous.  So they compare the bike injuries among people who ride bikes with bike injuries among people who do not ride bikes.  They conclude all the bike injuries are among the bike riders, therefore, bike riding ought to be avoided.

 

It's equivocation. They equivocate between "bike injury" and "injury".

 

People are not concerned about only bike injuries, but about all injuries. They are not concerned about bike fatalities, but about all fatalities.

 

By comparing bike injuries and fatalities between two groups, the argument pulls a bait and switch.

 

Notice how we went from "bike injury" to "injury". They are equivocating between bicycle injuries and all injuries.

 

The authors have proven that bike riding increases your rate of bike injuries, and that's absolutely true. And the best way to avoid bike injuries is to stay off of a bike and off of bike trails (even on foot).

 

However, that is not what people are concerned about. People are concerned about their overall risk of death.

 

(That said bikes actually are dangerous, more so than cars and walking,  in places in which there are not adequate bike lanes and bikes are ridden in the road... )

[fdrinca]

Also, when I posed your original question to DH (stats professor by day, grump by night), he said the statistical term would be "stupid research design," followed by various grumbles about current graduate students. 

[Guest]

Also, when I posed your original question to DH (stats professor by day, grump by night), he said the statistical term would be "stupid research design," followed by various grumbles about current graduate students.

You made me laugh and laugh with this post. :0)

[Deee]

Is this this bit that's problematic?

 

First-trimester drinking, compared to no drinking, results in 12 times the odds of giving birth to a child with FASDs. First- and second-trimester drinking increased FASDs odds 61 times, and women who drink during all trimesters increased the likelihood of FASD odds by a factor of 65. - See more at: https://www.aap.org/en-us/about-the-aap/aap-press-room/pages/AAP-Says-No-Amount-of-Alcohol-Should-be-Considered-Safe-During-Pregnancy.aspx#sthash.s4Ny6Q3r.dpuf

 

I read this as expressing the increased risk associated with drinking/FASD, not to determine the correlation between drinking/FASD and no drinking/FASD.

 

Using "not drinking" establishes the baseline likelihood of observing FASD, which one would assume to be 0 but in reality isn't due to sampling error, weird metabolic issues, etc.

 

Yes, thats how I read it (I used to write systematic reviews for the Cochrane Collaboration). So the increased risk comes from the comparison between 1, 2 or 3 trimesters of drinking and is a very sensible thing to analyse if the sample size is large enough.

 

The bike thing - thats just crappy design so you get the results you'd like (sadly very common, which is why we stats and methods boffins are so grumpy). Its not a post hoc fallacy, though. Thats when you go back and hunt for data AFTER you've designed and run the study. Its usually based on a hunch or trends in the data (statisticians are not fond of "trends" - they make us very uncomfortable). Also known as a "fishing expeditition" , and depressingly common.......

[Bluegoat]

Is this this bit that's problematic?

 

First-trimester drinking, compared to no drinking, results in 12 times the odds of giving birth to a child with FASDs. First- and second-trimester drinking increased FASDs odds 61 times, and women who drink during all trimesters increased the likelihood of FASD odds by a factor of 65. - See more at: https://www.aap.org/en-us/about-the-aap/aap-press-room/pages/AAP-Says-No-Amount-of-Alcohol-Should-be-Considered-Safe-During-Pregnancy.aspx#sthash.s4Ny6Q3r.dpuf

 

I read this as expressing the increased risk associated with drinking/FASD, not to determine the correlation between drinking/FASD and no drinking/FASD.

 

Using "not drinking" establishes the baseline likelihood of observing FASD, which one would assume to be 0 but in reality isn't due to sampling error, weird metabolic issues, etc. 

 

 

I think what I am finding most odd is actually more the recommendation from the APA based on it. So - the likelihood is greater, which is what I think anyone would expect for all kinds of reasons, almost any activity increases the likelihood of a problem related to that activity.   It seems like a totally relative number, and I can't see how a recommendation comes out of that.

[Deee]

Its not true that any activity increases the likelihood. If, for example, FASD developed as result of alcohol exposure only in the first trimester, then drinking in the second and third but not the first trimesters wouldn't necessarily increase the likelihood of the disorder (obviously not the case here). However, it is very easy to confuse causation with correlation, and researchers make this mistake depressingly often. Eg, it is more likely that a woman who drinks in the second and third trimesters (after she knows she is pregnant and against prevailing social and medical advice) will have also been driinking in the first trimester (when she may have had no idea she was pregnant until 4-6 weeks). So drinking in the third trimester may be correlated with drinking in the first, but that doesn't mean its causative. Its thus a confounding variable. Causation is hard to prove and requires large sample sizes so that you can tease out confounding variables.

[Bluegoat]

Its not true that any activity increases the likelihood. If, for example, FASD developed as result of alcohol exposure only in the first trimester, then drinking in the second and third but not the first trimesters wouldn't necessarily increase the likelihood of the disorder (obviously not the case here). However, it is very easy to confuse causation with correlation, and researchers make this mistake depressingly often. Eg, it is more likely that a woman who drinks in the second and third trimesters (after she knows she is pregnant and against prevailing social and medical advice) will have also been driinking in the first trimester (when she may have had no idea she was pregnant until 4-6 weeks). So drinking in the third trimester may be correlated with drinking in the first, but that doesn't mean its causative. Its thus a confounding variable. Causation is hard to prove and requires large sample sizes so that you can tease out confounding variables.

 

I meant very generally - as in, if you do X, you are more likely to have a problem with things cause by X than if you do not.  So - I am not surprised, and i doubt anyone else is, that FAS would be more common among pwwhen mothers drink alchohol. 

 

You are right of course that knowing details about how that risk is distrubuted is useful, but sometimes tricky to pin down.

 

But I don't see that the information that doing X is more likely than not doing X to cause a problem related to X, tells us much about whether we should ban X or avoid it, unless we can put that into a larger context.