Kansas City Chiefs, NBA referee bias, and pastoral discouragement
“Your football team is always playing on primetime!” That’s what my Bears-loving friend and fellow ministry leader from Chicago told me last fall. Being a long-time Kansas City Chiefs fan, my response was: “Really? Well, it’s about time!” Of course, his statement didn’t set well with me. I mean, did he really conduct the necessary research to make his assertion? Truth be told, KC probably IS on prime time a lot these days. However, are they REALLY on that much? Perhaps my friend was leaning on anecdotal evidence, “observing (that KC was playing on a couple of Thursday nights) and drawing a conclusion.”[1] Or, was I guilty of “cherry-picking”[2] statistics about how long KC had been “bad?” In the grand scheme of things, they have been a decent team. After all, they did win a Super Bowl just a few years ago (only to lose the next year to the evil Tom Brady), and in fact they’ve been to three Super Bowls…and now a FOURTH coming up on February 12, 2023. That’s more than most teams have been to in the Super Bowl era! Context is important.
Alas, Tom Chivers and David Chivers, in their super-fun book How to Read Numbers, affirmed what I’ve suspected all along. In rooting for my favorite teams, or even in conducting research within the ministries I’ve served, I’m no stranger to “sampling bias.” I tend to take samples that aren’t “representative of the population (I’ve) taken (them) from.”[3]
Case in point: NBA officiating and the Atlanta Hawks. (Yes, I’m going to stretch the sports theme out a bit more here. Sorry.) Our family has loved the Hawks for a LOOONG time. If we aren’t working or traveling and there’s a Hawks game on, we’re glued to the TV or even at the game at State Farm Arena.
But here’s the thing. We’ve always felt like ESPN had a bias toward the Lakers, or whatever team had bigger superstars. For years, my sons and I would say things like: “Did you see that Trae Young dropped 40 points last night and leads the league in points and assists, yet ESPN STILL headlined the Lakers game with Lebron James?” Here’s another favorite in our home: “Scott Foster is single-handedly destroying the Hawks.” Poor Scott Foster. Steve Holman, Atlanta’s long-time play-by-play announcer yelled that line on-air during a game a few years ago against Lebron’s Cleveland Cavaliers (at least, that’s what we remember him saying…and we could be wrong). He, along with thousands of other homers, believed Foster was making, well…some bad calls. Foster is an NBA referee that Atlanta fans loathe – at least some of us do – as he and other referees seem to always conspire to make calls that go against the Hawks. Now, if I stepped back, and took the emotion out of it, even though I feel like “the referee is biased against my team,”[4] I’d see that I’ve chosen to highlight a few anecdotes and small samples that certainly do not represent the bigger picture. I’m sure Mr. Foster is an excellent referee. After all, think how hard it must be to become a referee in the NBA, let alone be one for many years! Certainly, more data and more sample sizes would have corrected our faulty conclusions.
On a more serious note, I want to make sure I’m asking the right questions about my NPO. I want to look at the stats with a greater awareness that context matters when it comes to research. I want to make sure that what I’m reading and writing about, even now with this semester’s research surrounding my NPO, “is a fair representation of the literature.”[5]
Upon reading How to Read Numbers, I realize that my NPO statement could be truncated by small sample sizes, anecdotal evidence, biased samples, or research that may or may not be completely accurate due to any number of factors.
My current research question is this: What foundations, characteristics, and rhythms are critical for the flourishing of both individual ministry leaders and their associations? My research has to do with certain challenges that ministry leaders experience – challenges that include burnout, isolation, and discouragement. These are challenges that I’ve either experienced first-hand or observed through small-scale research and conversations I’ve had with local groups of pastors and church planters. They are also issues that have been addressed in national research studies such as Barna Group’s report on the percentages of pastors who have considered quitting[6] or Barna’s “State of the Church” findings.[7]
A few years ago, I surveyed a network of pastors to find out what issues they believed were the most pressing leadership issues they experienced, at that time. That survey, conducted before the pandemic, yielded certain findings from which we developed our training topics for that ministry year. Those findings were localized to a group of less than 65 ministry leaders and may not be indicative of pastors around the country. Chivers and Chivers believe that “you should be wary of any study that has fewer than 100 participants, especially if it’s used to make some quite surprising and/or subtle claim.”[8]
I’ve not conducted a survey like that one since 2019, but I hope to do so within the association of pastors and church planters where I currently serve. Chivers and Chivers have helped me to think about how I will go about that research, as well as how I hope to communicate my findings, making sure to “give the sample size of the study – and be wary of small samples.”[9]
How to Read Numbers will be a book I return to as my NPO project unfolds, or as we worry about the officiating at the next Hawks game on Tuesday night.
[1] Chivers, David, and Chivers, Tom. How to Read Numbers. (London: Weidenfeld and Nicolson, 2021), 16.
[2] Ibid., see chapter 16.
[3] Ibid., 29.
[4] Ibid., 29.
[5] Ibid., 166.
[6] See https://www.barna.com/research/pastors-well-being/, last accessed on 2/5/2023.
[7] See https://www.barna.com/stateofthechurch/, last accessed on 2/5/2023.
[8] Chivers and Chivers, 28.
[9] Ibid., 167.
About the Author
Travis Vaughn
15 responses to “Kansas City Chiefs, NBA referee bias, and pastoral discouragement”
Leave a Reply
You must be logged in to post a comment.
Hi Travis, you wrote… “I realize that my NPO statement could be truncated by small sample sizes, anecdotal evidence, biased samples, or research that may or may not be completely accurate due to any number of factors.”
This is a fear that I too am encountering in my NPO (Immigration). Lots of numbers and widely separated opinions/agendas. Red Flags abound.
My own survey also falls short..”Chivers and Chivers believe that “you should be wary of any study that has fewer than 100 participants, especially if it’s used to make some quite surprising and/or subtle claim.”
Ouch. My survey does not even come close to that benchmark. Thanks for your comments…Shalom…Russ
In our presbytery, we have more than 100 pastors, so it would be possible to surpass 100 participants, theoretically speaking (they may not all want to participate). However, I think Chivers and Chivers have a more widespread sort of study in mind. Surveying that number of pastors in our group may reveal findings true of this presbytery, but not necessarily true of every presbytery, or even the majority of pastors for that matter. This is partly why I waver between focusing in on my immediate context, or perhaps I should have a broader focus. For your project, what is the scope of your immigration study? Is it localized, or is it a wider study? Is it specific to a particular immigrant group, or more broad?
Travis, thanks for asking. I think this whole discussion has been generated by my stakeholder’s meeting. The questions I asked led to a survey and now I question the validity of the mini survey I did.
But I don’t want to be distracted by the numbers. The question posed by Matthew Sorens the World Relief Church Mobilizer was WHY do many churches avoid the issue of loving the “international newcomer” in their midst?
I reformatted the problem to wonder, what would it take to “jumpstart” churches planning process for immigrant outreach (a vision without a plan is a wish). My answer was a telephone app that would assist the refugee, the caseworker and volunteer to help them move into the integration process. I have started Ukrainian, Dari, Spanish and Burmese versions. Sigh….Miles to go and promises to keep.
Travis,
Thank you for your sports analogies as I find the image helpful in thinking about statistics and numbers.
You write: A few years ago, I surveyed a network of pastors to find out what issues they believed were the most pressing leadership issues they experienced, at that time. That survey, conducted before the pandemic, yielded certain findings from which we developed our training topics for that ministry year. Those findings were localized to a group of less than 65 ministry leaders and may not be indicative of pastors around the country. Chivers and Chivers believe that “you should be wary of any study that has fewer than 100 participants, especially if it’s used to make some quite surprising and/or subtle claim.”[8]
I read that line, too, and it caused me to pause. I am sure Chivers and Chivers are much more accurate than I will ever be but I am not sure I completely discount ethnographic research that uses less than 100 individuals. Do you think perhaps the “less than” might go deeper with questions whereas the “more than” research that cares more about the quantity as opposed to quality just barely scratch the surface? It sounds like what you researched is interesting and deeper. So for your NPO are you planning to survey more than 100 ministry leaders? Or will you rely on other researchers’ work? Thanks for the thoughtful post!
Also along the same lines, I am curious what other ways you might want to “pressure test” your sample size. For example, will you also consider age, education status, gender, race? I am not sure that any of these are good ideas- just curious!
Great question! I do think age, education, gender, race/ethnicity…all of these (and probably more) categories should be considered. By adding these, I do know this could change the “who,” in some ways. I will have to do a good job of walking the reader through my context so that they can understand what might NOT be representative of the population of “ministry leaders” as a whole. Those are categories I did not factor — nor was I necessarily trying to — a few years ago when our network conducted research to determine training topics, along with potential services. After reading Chivers and Chivers, are there some research categories you are adding that you had not previously thought about to your own NPO project?
Pam, these are great questions. What I/we researched a few years ago was specific to the pastors/leaders in the network where I served as executive director at the time. We looked for themes across the participants’ answers and categorized them. I’m sure our categories were informed by my/our/popular narratives we encountered beyond our local network. Of course, some of those categories were drawn from anecdotes or perhaps even biased samples. I do think we were as transparent as we could have been, though. We did not claim that the issues we unearthed in our local study were necessarily true for every network or assocation, everywhere. I agree with Chivers and Chivers, that it’s important to be transparent about the context. Being transparent solves a host of problems. Short answers to your other questions — Yes, I plan to survey more than 100 ministry leaders, with my current NPO. But I wrestle with whether or not I want to tweak the context and the question(s) my NPO rais(es). I do think the “less than” approach has some merit, but I’m not sure it has to be a quality vs. quantity. There has to be a way to do both. I think I need to be careful not to make statements that make things sound like every ministry leader everywhere is experiencing this or that. Context and transparency are important.
Travis,
You had me at NFL!
And then you lost me at MLB….never could understand the attraction to that game. I believe 85% of people who attend MLB games drink too much beer to ensure they enjoy themselves. That’s a fact.
Seriously…like you I have been relating this book to my upcoming research and, as I read, I became increasingly convinced that I am dependent on my denomination for providing me with current and accurate stats as it relates to the leadership crisis within our denomination in Canada. If they don’t have those statistics, I’m not sure what I am going to do!
I will be waiting with anticipation to see what your research reveals…as I suspect a good chunk of your content should likely be included in my NPO that is seeking to layout what effective training for ministry leaders needs to include.
Did you mean “NBA” when you said “MLB? I don’t think I mentioned MLB. But…since you brought it up, how do you know that 85% of people who attend MLB games drink too much beer to enjoy themselves? Are you thinking about a study that is a representative sample across the league, or are you referring ONLY to Seattle Mariners fans? Isn’t that the closest MLB team to you?
But seriously, regarding your denomination, I wonder how much research you will be doing focused exclusively on your denomination compared/contrasted with a broader multi-denominational landscape. In terms of training, how much will you be trying to answer problems that are endemic to your group compared with what you might find to be more ubiquitous? In addition to our own research, I think you and I will have to find what we can from our respective denominations, but I would imagine we’ll be looking at secondary sources to provide a bigger picture.
Travis….I’m not sure how I ended up thinking MLB after reading your post–you clearly articulated the very real problem of biased officiating against your Atlanta Hawks which I’m sure is totally accurate and very frustrating.
Perhaps I had consumed too many beers myself while posting my comment! Before I write further….I’d like to confirm that I am sober.
Your comment, “I wonder how much research you will be doing focused exclusively on your denomination compared/contrasted with a broader multi-denominational landscape.” got me curious: My faculty advisor has directed me towards a smaller focus for my NPO rather than a larger one. As a result, I named my denomination as the scope of my NPO. He even suggested that I consider narrowing further to “Western Canada” which I didn’t feel necessary as I have relational connections in the National office that I can leverage for my research. So my question to you: were you being directed to limit your scope at all as it relates to your NPO? I have another question unrelated to this particular blog that I would text to you if were willing to text me your number (mine is: 604 308-8797).
Travis, you and I are swimming in some similar waters with our proposed NPO’s. That’s interesting and affirming for me. I too found myself concerned about the “sample size.” Right now, much of my NPO Gut tells me we have a problem, a struggle in the pastorate with isolation, depression, joylessness, etc. But I don’t want to be just “going with my gut” on this whole deal, so I hope and pray to God the empirical evidence and support is out there, somewhere. And even just saying that makes me feel funny inside, because I sincerely wish it wasn’t an issue and that pastors are doing better than I think they/we are. Alas, I think my gut (and the antidotal conversations I’ve had) are correct. What do you think?
John, I think the anecdotel conversations + the secondary research that groups like Barna and Christianity Today have produced is helpful and does seem to confirm the gut suspicions. However, I’m not sure that every association, denomination, church, network, or tribe has experienced things the same way. Perhaps some of the issues have been around forever, but because we measure different things now and have non-stop data to peruse, some of my assumptions could be (and likely are) wrong. Perhaps VERY wrong. Or at least misinformed. I would love to know more about your NPO and the scope of your project.
Hi Travis, I didn’t realize our NPO’s were so similar. For my NPO, I am initially working with about 30 pastors and ministry leaders in Southeast Asia that represent a broader number of pastors within our family of churches. I am keeping my sample size small intentionally at this point. I hope to expand once my initial project is complete. Can you unpack a bit more your concerns about a small sample size?
Jenny, is your NPO focused exclusively on those 30 pastors/ministry leaders, or are you addressing things on a larger scale within a particular denomination or group? I’d love to learn more. When I say concerns about a small sample size, in my situation, I am wanting to figure out some things on a scale that is beyond the group (presbytery) of ministry leaders that I serve, but my NPO is focused narrowly on a small group of pastors and church planters and ministry leaders. We do have over 100 pastors in our presbytery (if one counts assistant pastors and other ministry positions), so we could have a good number to work with locally, but what’s going on here may not be the same experience for others outside of this group. I want to focus locally — and I may end up tweaking my NPO — but I have interests in my research that are beyond just this group. I’d love to know more about what you are working on with those 30 pastors.
Appreciate your posts, this book has challenged me to get more numbers for a survey I’m doing which will require some recruitment. Like you, I want to make sure I’m getting some diversity regarding the need I’m addressing and not building a solution for my NPO that does not actually impact people in the way I’m perceiving it is or the amount of people I believe it is.