YSK that ‘statistically significant’ and ‘statistically meaningful’ are NOT the same things, and understanding this can help you interpret research far more accurately.

YSK that ‘statistically significant’ and ‘statistically meaningful’ are NOT the same things, and understanding this can help you interpret research far more accurately.

In a nutshell:

When a researcher claims their research is 'statistically significant' all this means is that the researcher has used mathematics to determine with confidence that the independent variable's effect on the dependent variable is not due to chance.

For example, a researcher could say, "I think that cellphones cause cancer." They could then do a variety of expensive research methods to come up with the (hypothetical) conclusion that, yes, they're confident that, all variables accounted for, cell phones can cause cancer at a 0.01% higher rate in comparison to the population that doesn't use them. That doesn't sound like much, does it? Like maybe it's so low a percent that it doesn't really even have practical use? Well guess what, that result is still statistically significant, simply because the researchers have determined that the result was not due to chance.

Now this is different from what statistically meaningful/practically significant means in regards to research. For something to be statistically meaningful or practically significant, the results not only must be not due to chance, but they must be large or meaningful enough that we should actually care. Unfortunately, this line can something be fairly arbitrary.

Take the previous example. How much of an increased cancer risk from cell phone use are we willing to tolerate before it makes any difference at all in our behaviour?

0.01%?

0.1%?

1%?

5%?

As high as 10%? Or higher? The line is not clear cut. When do the results become statistically meaningful? Unfortunately, this is probably going to be unique to each specific research topic.

The real takeaway from this is, though, that just because you see research claiming its results are 'statistically significant' doesn't automatically mean you should care. The real question comes from whether or not the results are dramatic enough that they make a practical difference in how we interact with or interpret the topic of research. And that is what being statistically meaningful is all about.

submitted by almostlucid
[link] [1 comment]
Source: You Should Know

Posted in You should know Tagged with: , , , , , , , , , , ,