Faulty Numbers
“I told ya’ll!” That was the declaration during our car ride as my friend exuberantly pointed out that he is the winner of the bet, and that he is (almost) always right. The latest bet was on the price of pomegranate at Wal-Mart and who can guess the closest price. The bet ensued to raise the stakes of the argument, throw in a quick website check, and our price is verified and a winner declared. I elected not to point out the “sample size fallacy” with the amount of guesses and bets that are made on a near daily basis [1].
What are the statistical chances that my friend does have an ability to know the price of produce at any given moment? Statically speaking, probably not high, but he is bound to get some correct if he keeps guessing [2]. What If my friend pointed out that he has won the last eight-out-ten bets? That might change our perspective to be more in line with my friends. Imagine that after he points out the amount of bets he does get correct, our group decides that we will no longer bet my friend because he always wins. While this could be true, it is not based on enough statistical information that should be convincing.
This is the argument by Tom Chivers and David Chivers in the their book, How to Read Numbers (A Guide to Stats in the News (and Knowing When to Trust Them) as they reveal how numbers are often used in ways that “…give misleading impressions” [3]. Chivers and Chivers go on to say that the implications for the faulty use of numbers impact our lives to the extent that it even contributes to how people participate in our democracy [4]. It is necessary that we have an understanding of the numbers we use and see in the media because we rely on numbers to understand our world [5]. Despite any grand conspiracy that one might think is at hand, the misses of numbers is not necessarily malicious, it is due to the fact that we are narrative formed people and the numbers aid in our understanding of reality, therefore, it is up to use to be better informed about the numbers we see [6].
As a church leader, I see the use of statistics in the evaluation of ministries. This evaluation is often based on anecdotal evidence that goes something like, “worship attendance is decreasing because we switched to hymns. That is what I have heard from a few church members, anyway.” Not only is this a small sample size, it does not follow that there is a correlation between the music that Sunday and church attendance. It could be that it was a home game for the Dallas Cowboys and a number of families in the church were in attendance or that it was a holiday weekend and people were traveling. As Chivers and Chivers point out, we use this kind of anecdotal evidence to create a narrative to make sense and it generally works well for personal use, but can have detrimental effects if applied in broader contexts [7].
Whether it is a bet with a few friends or making sense of fluctuating attendance, it is eye-opening to see how numbers shape the decisions we make on a daily basis. It is not surprising that people who are not good with numbers come to different conclusions when the numbers lack context [8]. Perhaps I should point out that my friend is relying on a small sample size to gloat in his victory. While it will not change the outcome, perhaps it will be an invitation to scrutinize the numbers we rely on and why we even bet in the first place.
- Tom Chivers and David Chivers, How to Read Numbers: A Guide to Statistics in the News (and Knowing When to Trust Them) (Weidenfeld & Nicolson, 2022), 21.
- See the “Texas Sharpshooter Fallacy,” Chivers and Chivers, 137.
- Ibid., 2.
- Ibid, 3.
- Ibid.
- Ibid, 4.
- Ibid., 19.
- Ibid., 2.
About the Author
Chad McSwain
Chad is a systematic creative serving in pastoral ministry for nearly 20 years, Chad is a professional question-asker and white-board enthusiast, who enjoys helping people discover their own passions and purpose. A life-long learner, he has a B.A, Philosophy - Univ. Central Oklahoma, M.A Theology - Fuller Seminary, M.Div. Perkins School of Theology at SMU and is pursuing a Doctor of Leadership - George Fox University. He is an ordained Elder in the United Methodist Church, currently serving as Lead Pastor of Whitesboro UMC. Chad and his wife, Brandi live in Prosper, Texas along with their three children, two pugs and a chameleon.9 responses to “Faulty Numbers”
Leave a Reply
You must be logged in to post a comment.
Chad,
Betting on the price of pomegranates at WalMart!! You have a fun friend group!!
It sounds like you have a good perspective on attendance fluctuation. Do you record the Dallas Cowboy games so you can view after church? 🙂
Thanks for the fun read!
So, the story is inspired by the Israeli trip. We stopped to get fresh squeezed pomegranate juice and it cost four dollars. That lead to a discussion of the price of pomegranates. I argued that we were charged the America price. We had to settle for the Wal-Mart price since we were on a bus going to our next destination. This didn’t prove anything but it was a fun bet.
Church gets out before the game starts because we have priorities. I did serve a congregation where a high percentage of people would attend games and miss church, particularity if they were playoff bound.
Chad,
If we continue to make an educated guess we continue to increase our odds of winning at some point. The danger comes when we stop guessing.
All of that to say, if the church body continues to make assumptions or generalized statements about worship music you can reframe that into the church body cares about the attendance, and therefore is evaluating an outcome. The danger of the church body would be if they stop guessing. That would mean they didnt care.
How can we lead through empowering others, even when their guess is wrong?
Thanks for the comment. I’m not sure that an educated guess will evidentially lead to winning, although I think I understand your point. Our guesses can continue to be made based on false assumptions of those we are making guesses about. We have to engage and actually listen to a significant number of people to make better guesses, which I believe is the point you are making.
I agree that attendance may not be the best thing to measure in a church setting, yet as Chivers and Chivers point out, the numbers represent people (2). I get that it is easier to count heads than measure transformation in a person. Its buzzy to talk about engagement, which is really just more numbers in disguise of another setting. The key to counting is remembering that the numbers are people and to know their stories as much as possible.
To your point, I would take guessing people to complacent people. It’s the inertia principle: it is easier to change course than to set sail. To your question on leading guessing people, I would say reminding people that we fail forward and we know more now than we did before. That requires lots of conversation and overcoming fear of failure, which is better than giving up.
Exactly. Your church body is so lucky to have you!
Hey Chad, great post. Fun fact, I used to be a Cowboys Cheerleader. Okay, full disclosure, it was the Conejo Cowboys, a PeeWee football league and I was 10, but it is fun to say I was a Cowboys Cheerleader. The Dallas Cowboys used to train in Thousand Oaks. It just goes to show how we can make information misleading by how we present it, similar to the argument about numbers and statistics. We need to be mindful of this in a ministry context. As you noted, a decline in attendance could be the result of a change in the worship, but it could also be due to vacations or other considerations. How do we use the numbers to our benefit without being beholden to the numbers in ministry?
Hey Chad,
I am guilty of making judgments based on small samples sizes. Recently, I watched a video about how young men are in crisis. The points in the video were valid, however, I quickly applied that to my young adults ministry I lead. The young adults group is about 30-40 weekly attendance and probably 60 filter in and out through a regular month. In one week I had two conversations with guys in their twenties without jobs, girlfriends, and living with their parents. This was concerning for me, but with that video in mind I jumped to the conclusion that the young men in my young adults group need to get it together! Now, is that true? Maybe, but maybe not. I can’t base this conclusion off of this small sample size. And this doesn’t paint the whole picture. Maybe these guys do have vision for their lives, are living at home to save money and pay off student loan debt, and maybe they are not waking up late, playing video games, and not helping pay for groceries.
Now, research and statistics do help us as leaders get a quick read on the people and community we serve. So we can’t throw all statistics out the window. So my question for you Chad is this: after reading this book, beside “How big was the sample size?” what are some of the questions you will ask yourself when you come across numbers-based research?
Hi Chad, I agree with you on scrutinizing the numbers before relying on or even betting on them, especially regarding essentials like ministry. You reminded me of Gladwell’s conclusion of 10,000 hours for mastering complex skills and materials. I wonder how many decided to take him at his word to count those hours. Some of us can be firm believers in our numbers and statistics, taking them at face value.
Thanks for your post, Chad. What metrics does your church use for evaluating health and progress besides worship attendance? I find that it’s increasingly difficult to measure things like engagement and spiritual growth, but I do understand why it’s important to try. Just curious if your church has a process for this?