Data Nerd
I’m a data nerd at heart – always have been.
When I was a kid, I had to do the dishes as my chore. I would have fun timing myself to see if I could beat my previous record of loading or unloading the dishwasher.
Today, I’m still tempted to collect data for household chores, but my attention has been redirected to statistics for the Bible translation movement. In my role at SIL International, half of my job is being the director of ProgressBible.[1] Our aim is to inspire informed action amongst Bible translation organizations, enabling them to be most effective in their work. As a part of ProgressBible, I am heavily involved in the Every Tribe Every Nation (ETEN) [2] collective impact alliance. We follow Stanford’s model [3] for collective impact, in which one of the conditions is shared measurement. ProgressBible serves as the engine that collects and analyzes data across all organizations participating in the collective impact alliance.
One of our current areas of focus (and my NPO) is providing an analysis on Scripture usage, and reasons why there is a lack of Scripture usage by some language communities. I’ve had over a hundred conversations with missiologists, sociolinguists, strategists, Scripture engagement specialists, church leaders, field workers, and community members to try to understand the reasons for lack of usage. Unfortunately, this is an exceptionally complex field, and there were not any easy answers (or solutions) to this problem. But as I have been asking around, I have come to the realization that I have to fight the temptation to give into my confirmation bias [4]. I have a hypothesis for the major reasons why Scripture is unused, and my tendency is to be biased towards that hypothesis. So I’ve had to ‘Search my feelings’ as described in rule one of Tim Harford’s How to Make the World Add Up. [5] How were so many blinded to counterfeit Vermeer paintings? They were blinded because they wanted to believe that the paintings were legitimate.
Another contribution that ProgressBible makes to the Bible translation movement is to provide a forecast of when every language community will have access to Scripture in a language they understand best (named the All Access Goals by ETEN). Rule three of was especially relevant to the forecasting we do.
Rule three tells us to avoid premature enumeration.[6] Harford opens the chapter with an example about infant mortality being higher in the rural areas of England than in London. In reality, this was due to a difference in reporting choices between classifying deaths as a miscarriage or as an infant death.
In Bible translation, we have been tracking the forecast to reach the All Access Goals closely, and publish monthly updates based on the most recent data. At times, there can be major fluctuation in the data, and therefore the forecast. There are many reasons we can have a ‘good’ month in which it does not actually mean that the pace of Bible translation has sped up. It could be that we discovered Scripture that was translated some time ago, that an organization was behind on reporting and just caught up in a single month, or many other reasons. This premature enumeration can lead us to inaccurate conclusions, just as the case with infant mortality in England.
I have long felt that statistics can be manipulated or used strategically and the difference between the two may not always be clear. Some may consider either the visualization on Iraq’s Bloody Toll or the alternative view of the data Iraq: Death on the Decline [7] to be manipulative or pushing an agenda. In working with and presenting data, I have felt a burden to steward this responsibility with integrity, and yet there is a tension between showing compelling visualizations and presenting the data in a way that promotes an agenda.
May the Lord give us all wisdom in using data wisely!
References:
[1] https://progress.bible.
[2] https://eten.bible.
[3] https://ssir.org/articles/entry/collective_impact.
[4] Schulz, Kathryn, Being Wrong: Adventures in the Margin of Error. New York, HarperCollins, 2011.
[5] Tim Harford, “How to Make the World Add Up: Ten Rules for Thinking Differently About Numbers”, The Bridge Street Press, 2020, 21-50.
[6] Ibid, 71-92.
[7] Ibid, 248.
About the Author
Christy
14 responses to “Data Nerd”
Leave a Reply
You must be logged in to post a comment.
Hi Christy,
I was tickled by the title of your post, ‘Data Nerd’.
ProgressBible tag line shows, [1]” Gathering and sharing data to get the word of God into the hands of every person.” From your post, I perceive you to be a data expert (someone who enjoys numbers). I am just the opposite – numbers intimidate me. Do you find Harfort’s book contributing to your NPO, or this book is too elementary for you?
[1] https://progress.bible.
Hi Shela! Harford’s book is certainly helpful – as much as I love data – there is always a risk of misinterpreting the data – so his 10 points were a good reminder!
Christy,
Great post and I appreciate some insight to what you do. I’ve met many people over the years that have been involved in Bible translation. I am curious, what is the benefit of the monthly statistics? Are they a way to show that the worker’s time is well spent? Are they to create competition amongst translators? Monthly to me, an untrained person in this realm, seems overkill. I think I would be happy with yearly numbers.
Hi Jeff – we have some very large financial investors who want to see progress every month. Some of them consume these numbers to the point of internalization (they know them very well and have an extremely good pulse on what’s going on). There are also those that desire to support in projects that are lagging (or failing), and these monthly numbers help to identify potential problem areas.
It may seem overkill – but the trend is to go more frequent than monthly!
Christy,
The data nerd part I would have never guessed, so thanks for leading with that.
You mentioned falling into the temptation of confirmation bias when it comes to your research.
How do you imagine you will reconcile the data with your feelings so that the statistics won’t be manipulated or used to sway others to go in a particular direction?
And yes, may you steward the data with great wisdom and not be afraid to allow your life experience to be a part of your decision making process.
Blessings to you!
Hi Nancy,
I’ve just recently expanded my research to include voices that I have disagreed with. They will be key in keeping me grounded.
There is a risk of me misinterpreting the data due to confirmation bias, and then there is also the risk of others using the data I produce in inappropriate ways. It’s tricky for all of us!
Christy,
I am fascinated by the work you are doing for your NPO. It does sound complex. You mentioned both rules 1 and 3 from Harford. Was there anything you read that will definitively change your approach?
Hi Chad – I’m adding in some other voices into my research that will help keep me grounded and avoid confirmation bias and force me to ‘search my feelings’. Thanks for asking!
Christy, yes, data can easily be used or manipulated, based on how it is presented. It’s great that you are recognizing the the pitfalls of premature enumeration. Are there any other data pitfalls that are becoming more clear to you as you work on your project?
Oh there are so many! One of the challenges is that data isn’t easily accessible in the Bible translation space. It takes on average 16 years to complete a project, which results in tremendously slow lagging indicators (ie – sometimes, decisions we make today in Bible translation will be realized in 16 years. In other words – the data we see today could be based on decisions made 16 years ago! This makes understanding the little data that exists incredibly complex!
Christy, Thanks for sharing about your work with Bible translation readership. What I take away from what I read is that it is one thing to translate the Bible but what good does it do if people aren’t reading those translations. I am wondering if your research is giving any insights yet and if so do they lead to ways to lessen the gap?
Hi Diane – this is a very complex issue, and I wish I had the answers. There are a variety of reasons why a Bible translation could be unused (the language community has shifted to using a majority language, the community didn’t have ownership in the project, persecution, and many more reasons). While some of these things are known and understood, it’s not clear to me why we struggle to implement the learnings. I suspect there are some leadership issues at play.
Hi, Christy, since you are really into data, you must be really liking this book by Tim Harford. You talk about fluctuations in the data collected, you think applying the ‘ten statistical commandment’ by Harford would help explain the fluctuations in your data?
Christy. Great post. I have so many outside view/inside view questions about your dishwasher make and model, numbers of household members’ dishes you were loading for, type of detergent used, whether a pre-rinse was sometimes or always required, time of day you loaded and unloaded.
And so, related to your NPO, thank you for letting us feel your personal tension between showing the statistics that support your argument, while not manipulating or enabling confirmation bias.
What is one insight this book has provided that you think will help you refine your research?