Saturday, February 1, 2014

Some Ethical Limits of Statistics

My piece titled "Some Benefits and Limits of Statistics" has been in the top five of my all-time most viewed posts for a while now. I have no clue why this is so, but in any case, I am glad people like to learn! (Or there are some very zealous cyber bots searching for posts about statistics in the United States and Russia.)

Either way, an observation that my latest statistics professor, Nicholas Jewell, made on the first day of class this semester (yes, I am taking more statistics) really stuck with my heart, and I thought it a good time to share. In so many ways, I am preaching to myself in this post, as I have had many a moment of poor character in my thoughts and words over the years, many times as a result of my numbers bent, and it is something I am constantly praying for God to redeem in me. So here goes.

In statistics, we typically do not have information about every single person or thing we want to understand. Instead, we take samples from a study population and try to generalize what we find in the sample to the whole population. Think of this like asking one hundred random strangers in your town whether they like your hair do. If 80 of the 100 people you stop all over your town tell you it's a bad hair do, you might reasonably guess that about 80% of people in the whole town would think the same, regardless of whether you have actually asked every single one of them. If you care much about what others think, you go get a new hair cut, or if not, you pride yourself on being a hair rebel.

Intuitively, we generally prefer larger sample sizes, because, all other things being equal, they make our results more credible. For example, if you stopped 5 random strangers some place in your town, and 4 of them did not like your hair do, you might chalk it up to bad luck in your stranger selection. You might figure, hey, if I had asked five other people across the street or across town, they might have said something different. But if you ask 1,000 random strangers, and 800 say you look like Medusa, you would be less likely to assume it was an "unlucky 800." As it turns out, this intuitive understanding is also mathematically true in statistics, although I won't get into too many details here.

"The top portion of this graphic depicts probability densities that show the relative likelihood that the "true" percentage is in a particular area given a reported percentage of 50%. The bottom portion shows the 95% confidence intervals (horizontal line segments), the corresponding margins of error (on the left), and sample sizes (on the right). In other words, for each sample size, one is 95% confident that the 'true' percentage is in the region indicated by the corresponding segment. The larger the sample is, the smaller the margin of error is."
(Source: http://en.wikipedia.org/wiki/Margin_of_error)

As Professor Jewell reminded us during our first class, in public health in particular, we are often using epidemiological methods to compare people who experience a certain health outcome with people who do not. For example, we might study the characteristics of people who get pancreatic cancer in their lifetime versus people who do not get pancreatic cancer in their lifetime, in the hopes of figuring out what those who get cancer share in common and how they are different from those who do not get cancer. If we conduct a study of 1,000 people, where we compare 999 individuals who did not get the cancer with one person who did, we are not likely to feel confident that the one person who got cancer in our sample is statistically representative of the multitude of people who get pancreatic cancer around the world. (This is just like the hair-do example above, where we believe the results more when we ask more people.) Let's assume that the single cancer patient in our study is also the only one of the 1,000 people we study who has ever visited the North Pole - so our one cancer patient has visited the North Pole, but our 999 non-cancer patients have never visited the North Pole. Do we then have evidence that the North Pole is associated with pancreatic cancer? Most of us would say no, because we intuitively sense that we cannot draw a conclusion based solely on a single person's experience. On the other hand, if we compare 500 people who did not get cancer with 500 people who did and find that a history of North Pole vacations is much more common among the cancer patients, we are more likely to believe and perhaps further investigate an association between cancer and the North Pole.

In the above example, more cases of disease in our study population gives us more confidence in our conclusions. We are more likely to learn something and believe it could be true more generally by studying 500 cancer patients than by studying only one cancer patient. As statisticians, then, we may find ourselves in the position of wishing for more cases of disease in our sample size in order to have greater confidence in our results. And this is where I really appreciated the insight of Professor Jewell. In moments like those, when we are wrapped up in data and numbers, we should take a moment to step back and realize exactly what we have just thought. Should anyone really wish for more cases of a disease, just for the sake of analytic rigor? The obvious answer should be no. We may wish that we had crafted a better study design to interview more individuals who already have the disease, but it is very easy to forget that each of those data points is a real person.

Every single cancer patient enrolled in a study represents a person and family and community who had to go through an agonizing experience. Participating in a study in and of itself is often no walk in the park, either. Furthermore, the results of a study into what causes a given outcome may benefit future patients, but are often too late to benefit the ones under study at the time. This pain and trial are not to be so callously wished on anyone.

And it is in this sense that as students, teachers, researchers, etc., those who use statistics have an ethical responsibility to consider the human costs of what we quantify. Professor Jewell said he reminds himself of this by watching one particularly compelling and emotionally complex movie about human subjects research every year, which helps re-center him on the human aspects of his work. You may find your own unique ways to heighten your awareness. I believe this need applies not only to public health, but also to many fields that utilize mathematical inquiry beyond public health - education, public policy, social work, and business, just to name a few. It is not that our methods will necessarily be changed in any concrete way by exercising compassion in our thoughts (although they might, in some cases); rather, we can seek an attitude of empathy and a posture of humility as acts of internal integrity. It is not that we should neglect the study of disease or other negative life outcomes, as our numerical inquiry is typically not the cause of such outcomes, and is often ultimately aimed at improving health. But even as we work for these good causes, we should take time to reflect, empathize, and get outside our little boxes to just be human - to appreciate the lived experiences of those studied or affected, and to stop ourselves from so recklessly throwing around words and cold thoughts about those who it is all too easy to see as data point number 732.

No comments:

Post a Comment