Hi everyone,

Whenever I find spare time, I share posts about topics that are on my mind.

Today, I would like to focus on the concept of statistically meaningful rates in social science. I believe that only a small percentage of scientists truly understand this concept and the reasoning behind these calculations. Many researchers simply use statistical programs and look at the results without delving deeper into the calculations behind them. Ridiculously, they often don’t understand what they are doing or why they are doing it.

We typically focus on the 0.05 meaningful rate when deciding statistical significance between two different datasets or tests. But why do we focus on this value?

This approach comes directly from mathematical probability. Before explaining it, let me clarify something. We set up hypotheses before every research study. In these hypotheses, we claim that the interventions in the study will have an effect and reveal differences in certain variables. Let’s consider this explanation more broadly: essentially, we are dealing with two possibilities: the hypothesis is confirmed, or it is not. This represents two distinct probabilities. In mathematics, if you have two different probabilities, A and B, the probability of either A or B occurring seems to be roughly five times out of a hundred in natural contexts (assuming no external influences). This means that if you test two different probabilities, the likelihood of one occurring is about 5%—and if you see one outcome 5 times, the sixth time, you should expect the opposite outcome due to the natural probabilities involved. Mathematically, if you observe event A five times, event B will likely occur on the sixth attempt. This relates to the 5% probability (0.05).

Thus, we naturally accept a 5% probability on one side. If the probability drops below 5%, we can infer that another variable is at play. Now, returning to our hypothesis: we had two probabilities—either the hypothesis works, or it does not. We accept the realization of our hypothesis up to the 0.05 level. Anything below this threshold suggests that the result is not due to chance and must be explained by another variable. In other words, the result goes beyond natural probability and indicates that our hypothesis is likely correct.

This is why we focus on the 0.05 meaningful rate in our studies: it helps us determine if there is a significant difference among the variables and whether our hypothesis holds true. If anyone would like to test this mathematical probability, simply take a coin and try your luck. After a certain number of attempts, you will not consistently land on one side more than 5 times under normal conditions.

24/11/24