Statistically significant or completely irrelevant?
Have you ever heard the phrase that 99 % of statistics are made up? The joke is that you can throw any number you want in there because numbers hold no relevancy in statistics. I’m not saying that generalization is accurate, but I do wonder if that phrase came about because the mainstream population didn’t understand the intricate and tangled web that statistics exhibit. In today’s media, statistically significant could mean completely irrelevant: you be the judge. I thought that I would give you two examples here with made up news articles headlines by yours truly. One story however is based on factual events.
“Changing the temperature of your food could improve your health.”
WHOA! That sounds like a home run! Sure if changing the temperature of my food would improve my health why wouldn’t I do it? The reality of the study when you get down to it suggested that in actuality changing the temperature of your food allowed your digestive tract to work at a rate of one minute faster over the course of one year than if you consume at standard food temperatures which in turn does better your health because your body saves one minute of digesting its food. Does that tell us anything though? After looking at the statistical analysis of the study would you change your eating habits after getting the full story? Do we care that our food can be digested one min faster over the course of one year? Does that make or break our daily life? The reality is that most likely it is completely irrelevant.
“Adding in one exercise per week could win you a state championship.”
Last week in Texas we had state cross country meets and a local girl won first out of the entire state for the third year in a row. The community has been so excited for her. In a non hypothetical analysis her time was one min faster than last year when she won it. What changed? I called her aunt to see if she could identify a single component that put her over the top this year to be able to beat her existing state record by one min and she stated that she is constantly pushing herself all year to train for this event. She never takes a break. This year she decided to add in 1 additional strength training session per week. That one session a week contributed to shaving 1 min off of her time to take home a third state championship. This alone is the epitome of statistical significance described in a contributive manner. We can’t prove that adding in that one additional strength training helped. It could have been maturity of her body or emotional preparedness as she continues to run the same race. We also cant prove that adding it in didn’t allow her body that extra boost for her to win the state championship again.
Frederick Mosteller once said, “While it’s easy to lie with statistics, it’s even easier to lie without them.” (1) This brings us to Simpson’s Paradox. The idea as a whole is perhaps my biggest excitement of reading this particular book this week. Which I might add I could not put down this title. I did not follow the assignment and only read a portion and on occasion I believe thats ok. In my opinion this particular work by the Chivers family required you to read it in its entirety to understand the content fully. I was reminded this week how exiting statistical analysis can be. I’m curious to continue my research on how numbers that are so definitive both have a personality of their own and are completely misleading. 
The following graph to the left (2) is a great pictorial example of what Simpson’s Paradox might look like with the same set of data. On the far left with blue and orange it shows two separate groups with a positive correlation between x and y. However on the right when you combine the groups into one collective number the data then shifts and becomes a negative outcome.
As my final note I really appreciated the optional boxes the authors included to hone in on specific mathematical concepts. While I did get an A in my last statistics class, that may or may not have been eighteen years ago. The ability to quickly re-grasp a concept is both appreciated and necessary to understand the book as a whole.
(1) Chivers, Tom, and David Chivers. “How Numbers Can Mislead.” Essay. In How to Read Numbers: A Guide to Statistics in the News (and Knowing When to Trust Them), 7–7. London: Weidenfeld & Nicolson, 2022.
(2) “Simpson’s Paradox and Interpreting Data.” Accessed November 11, 2022. https://towardsdatascience.com/simpsons-paradox-and-interpreting-data-6a0443516765.
About the Author
Alana Hayes
Alana is a mother to four beautiful children and wife to a farmer in Texas. She is an avid world traveler with a heart for both the world and education. She is the president of the nonprofit Against the Grain Texas where they focus on providing education to children overseas and at risk adults in the states. To date the nonprofit has given almost $100,000 to individuals around the world. In her free time she loves spending meaningful time with people and reading to further her personal education.12 responses to “Statistically significant or completely irrelevant?”
Leave a Reply
You must be logged in to post a comment.
Alana,
I enjoyed reading your blog!
The two stories you mention make it difficult to trust the numbers/statistical data. I love that you were able to call someone and verify what possibly made a difference in the girls running time! Very interesting!
I have a pretty good idea what you are pursuing with your NPO. I am curious if you will use statistical research with your NPO? You did so well in statistics! You may enjoy seeing how statistical data might support some of the initiatives you hope to achieve through Fred’s Garage. I hope you find some relevant findings to support your NPO!!
Hey Kristy!
You know I haven’t thought of it, but I do not think that is a bad idea! I am meeting with someone next week that would be a great resource to inquire more!
Stay tuned.
Hi Alana,
I loved reading the fake headlines. I have increased the temperature of my food by one degree and I have added an additional day of strength training to my week! Seriously, it is interesting see how these kind of headlines are so pervasive in our culture and are convincing if you don’t stop to think about the information that is shared. I agree that this book needs to be required reading due to the nature of how statistics are used everyday. Thank you for sharing.
Thanks Chad!
It is truly mind boggling how much our society can follow along with anecdotal words in print as if its law. What do we do to instill thinkers instead?
Alana,
I am in agreement with Chad that this book ought to be required reading at some point for either high school students or at the very least, college students. It is enlightening. We are completely overwhelmed in our society today with messages based on numbers that likely are not true at all. Yet so many people will take what the media shares as truth, whatever form that media takes. Your question to Chad is a relevant one … how do we get people to think instead of following along with whatever the message of the day is?
Tonnette,
This brings me to where I am in my current research. Who is responsible for teaching this? Does this responsibility lie on the parents alone or the educational system?
Ultimately we are responsible for teaching the youth of our nation. We are looking at a generational change that is needing to take place. Now, what does that look like?
Alana, I love your fake headlines. What a great way to highlight how misleading statistics can be. It amazes me how quick people are to look at a headline and believe something to be true. I remember an acquaintance on Facebook once sharing an article that indicated the fire in Notre Dam was because of a terrorist attack. When you looked at the article and the picture it showed, it was from years before the fire and had no connection to terrorism. This same person has also on more than one occasion posted articles from The Onion or Babylon Bee (known satire sites) as fact. As you commented, “What do we do to install thinkers instead?”
Becca,
I was at Notre Dam just two weeks prior to the fire and it was a lot to take in.
In the quick response/non-critical thinking age that we are due to mostly in part to social medial what can we do to slow down people like your friend from sharing information blindly?
Hey Alana,
The way you wrote “Changing the temperature of your food could improve your health” almost had me convinced. I would have fallen for it if I did not know you wrote it in jest.
In reading your post, I was reminded of the line about a gasoline-smelling colgne in the movie Anchorman: “They’ve done studies you know. 60% of the time, it works every time.”
David,
When I was explaining the importance of the Chiver’s book to Ben and how I wanted him to read it I gave him an overview as well as my initial thoughts. He immediately referenced Anchor Man as well.
HA! Great minds think alike!
Alana – I was thinking about your portfolio project as I read your blog and wondered if you have done research on key performance indicators (KPIs) for your coffee shop? Figuring out your KPIs and tracking them is really helpful for any new business venture! If you like numbers, then that might be fun for you to investigate.