While statistics may appear to be all about numbers, without the proper words to start with, the numbers may not lead to the proper conclusions.
Judith Tanur has been working on getting the words and the numbers right for decades.
After a long distinguished career as a professor of statistics, sociology and survey methodology at Stony Brook University in Stony Brook, New York, retirement has not slowed Tanur in her quest to educate, inspire, and use statistics to help solve problems and improve lives.
She’s an active participant in the AAAS On-call Scientists initiative.
That effort connects scientists, engineers, and health professionals who volunteer their expertise with researchers and human rights organizations in need of technical expertise.
Sometimes her assistance involves just one long phone call, where Tanur gives advice on basic issues like, how do you find a sample? Or, how do you construct a questionnaire?
Other times it is much more complex, such as a case exploring how legal aid for the indigent is handled in various countries. And the initial discussions with the researcher she assisted focused on choosing the right words.
“I would question her about whether the question was necessary; whether the wording was appropriate; whether her potential respondents would actually be able to answer it; or if it was too complicated, or whether it was within their province to know,” she said. “But unless you have had experience asking people questions on questionnaires, you are not aware of the pitfalls, and how impatient people get; and how hard it is to make sure that the meaning the researcher means is shared by the respondent.”
Eventually the questionnaire was finalized and sent out to respondents. Then slowly the responses started trickling in.
“We were in contact only sporadically for some time. While we waited, both of us were holding our breath, hoping we got a reasonable response rate. It becomes ‘we’ at this point; not ‘she and I’ but ‘we.’”
With the responses in hand, the next task was how to make sense of it.
There was a lot of consultation by phone, but the plan for the analysis jelled when the researchers worked side by side. A trip to Tanur’s home on Long Island helped make it happen.
“We spent many hours together in a lovely atmosphere, and she got to enjoy Montauk. She got the hang of how to do the analysis and she carried out the rest of it and presented it to the corporate sponsors. She really did an absolutely heroic job, and I was very impressed. I was pleased to be part of it,” said Tanur. Beyond just achieving solid research “I think personal contact is important in getting life right,” she said.
Tanur says her field of expertise is seldom acclaimed for its charisma.
“When I say I’m a statistician, most people’s eyes glaze over, and the most frequent response is, ‘That was my worst class in college.’ That is funny but true. I think there is no way to turn that around,” she said.
So by sharing examples of how statistical analysis improves lives, Tanur can turn anxiety about statistics to a better appreciation of its real power.
For more than a decade, she’s volunteered on a team evaluating health care interventions in Vietnam, sponsored by Atlantic Philanthropies.
“We did surveys and home visits and interviews at community health centers. It was fascinating stuff,“ she said.
This project had an additional layer of complexity: Language.
“A good deal of my research has to do with the effects of question wording on answers. There are a lot of important but quite subtle differences. In this questionnaire, we wrote it in English, it was translated into Vietnamese and I therefore had absolutely no idea of how the questions were really worded. I worked very closely with a Vietnamese colleague to write the questionnaire. We did, I think, a very good job, but I had to count on him to do the final check to see if we were asking what we meant to ask,” she said.
“By seeing how well the needs of women, the elderly, the poor, and ethnic minorities were being served by the innovations we were able to see which interventions were most useful and help specify the conditions under which they should be implemented," she said.
Whether examining legal systems, healthcare, or politics, an important tenet of statistics is, if you don’t have a good sample, your conclusions are suspect. So if policy is being made on the basis of a poor survey; then virtually everything about that policy can be wrong. And that translates to real-world consequences, whether access to funds for education, health care, jobs, infrastructure improvements, or voting rights.
In the United States, there is currently a dramatic example of how the questions on a survey can have a significant impact on public policy. A proposal to add the question, “Are you a U.S. citizen?” to the 2020 U.S. census is a subject of great controversy, already facing some legal challenges.
“The feeling is that adding that question will increase non-response among immigrants. Immigrant families live mostly in cities; and cities are mostly Democrat. One can see a nefarious design there if one is inclined to think that way; that it may be a deliberate attempt to suppress responses to the census in certain areas. The census is used for redistricting, and if there are fewer people counted in cities, there are fewer House seats in cities, fewer House seats likely to be Democratic. So it is consequential,” she said.
Whether the outcomes involve better access to healthcare, proper representation in Congress, or other forms of justice, statistics matter.
While she no longer gets a paycheck, Tanur still uses her statistics skills in her volunteer work. She said, “There is the satisfaction of being helpful to people who need my help. I’m not really good at killing time without feeling productive.”