How to Evaluate Consumer Health Reporting

NASA image

In April I had the privilege of attending Science Writing in the Age of Denial, a conference held at the University of Wisconsin-Madison, that featured science writers, journalists and scientists from around the world discussing some of the perils, pitfalls, challenges and amazing opportunities of covering science, medical and health stories in today’s media landscape. I reveled in the two days of intense discussion.

My notebook from the conference is filled with notes, sketches, web addresses to visit and names of books that I simply must read. One of the talks that hit home hard was given by Gary Schwitzer, a consumer healthcare journalist who is publisher of the web site

Schwitzer’s talk, “Cheerleading, Shibboleths and Uncertainty” addressed the status of consumer healthcare reporting which, in his opinion, often tends to be little more than “cheerleading” for the latest greatest drug, technology or screening test. His talk addressed some of the cult-like following for screening tests (the shibboleths) and the tendency to convey with certainty the “upside” of screening without discussing adequately the risks or downsides (uncertainty).

His focus on the screening issue was particularly poignant to me, because I have experienced my own transformation of thought regarding health screening. As a young graduate student studying reproductive genetics, I was amazed and awed at all of the potential information available from screening, and I had no doubt that the more a person knew the better off a person would be. Knowledge is power, right? I never stopped to consider other sides of the equation: information that did not yield actionable data, ambiguous results, what the term “increased risk” actually means (in truth, it often has little meaning), false positive, and anxiety.

Then I worked as a lay chaplain in a research hospital and began to see that knowledge does not always come with power. Sometimes it comes with confusion, anxiety and disappointment. Our medical and establishment is not very skilled at dealing with confusion, anxiety and disappointment. Actually, our society seems not to be particularly good at dealing with confusion, anxiety and disappointment.

Then I got pregnant at age 40 and allowed myself to be talked into the latest greatest screening test, which delivered only ambiguous results—not anything actionable, only something that led a genetics counselor to suggest more screening. Fortunately the “more screening” suggested was something with which I was very familiar, and I declined. I opted instead to do my own research, take responsibility for my own healthcare, and I decide what tests would be conducted for the rest of my pregnancy, based on the kind of information offered and what my husband and I would or would not be able to do as a result of that information (e.g., know whether we would need to find extra assistance for a special needs child). My actions were not typical, and I got the impression that most health care providers would prefer to march patients lock-step through a “if this, then this” scenario.

We were fortunate. In the end the ambiguous results were just that, ambiguous results that couldn’t be explained, and we had a beautifully developed, healthy baby girl. But still, I spent the better part of my pregnancy with a nagging worry in the back of my head. I wonder how much better an experience I would have had throughout my pregnancy if I had skipped that first trimester screening. I’ll never know.

I’m not a fan of screening for screening’s sake, but it took an experience on the receiving end of nonactionable, ambiguous results that generated a lot of anxiety to drive home the downside of some screening tests. When I see fliers in the newspapers advertising traveling screenings (usually ultrasounds) held at senior centers or churches to search for, as Schwitzer called it, “weapons of mass destruction inside us”, I cringe. It’s disease mongering, often to vulnerable populations, and it annoys me.

So, I was excited to hear about and the eight other websites around the world that are devoted to holding consumer health reporting to a higher standard. I need help reading through the hype and understanding the often convoluted statistics. Most people do. reviews consumer-directed health stories based on ten criteria:

  1. Does the story adequately cover the total cost of the treatment/screening being discussed?
  2. Does the story adequately quantify the benefits (how often do the benefits occur)?
  3. Does the story adequately quantify the harms (how often do harms/risks occur)?
  4. Does the story address the quality/strength of the evidence? Does it use causal language to describe results of an observational study (correlation does not imply causation)?
  5. Is the condition exaggerated (i.e., is there disease mongering going on)?
  6. Is the treatment/screening/finding really new?
  7. Is the treatment/screening readily available?
  8. Does the story discuss alternative choices available to the consumer?
  9. Who is promoting this treatment/screening/finding?
  10. Do the promoters have a financial or other conflict of interest?

These are excellent criteria, questions that all people should be asking each time they read a consumer health news item or even listen to a direct-to-consumer pharmaceutical ad (or for that matter read a press release from any corporation, institution or scientific journal). In his presentation, Schwitzer indicated that of 1,700 stories reviewed, 70% of them failed on items 1–4. With those kinds of data, consumers of consumer-directed health reporting cannot afford to be less-than-critical thinkers.

No study ever stands on its own. Good science writing considers every "new" result within the context of the entire body of work in the field.

To Schwitzer’s comments I would add this: No single study ever stands alone. Scientific research is incremental, and no study should ever be reported outside of the context of the larger body of work of the scientific field. I know we live in a society that wants the dramatic headline and doesn’t like to take time for context, but science is rarely dramatic. It inches along, it takes two steps forward and one step back; our understanding comes bit by bit. Hypotheses are proposed, experiments conducted, hypotheses revised, new experiments conducted. That is what science is about.

We can’t make all medical advances look harmless, good and free, because they simply aren’t. We can’t go around disease mongering, having people stressed out continually about their health. We need responsible, thoughtful reporting, and responsible, thoughtful reading. I know I will be a far more thoughtful consumer and writer now.

The following two tabs change content below.

Michele Arduengo

Social Media Manager at Promega Corporation
Michele earned her B.A. in biology at Wesleyan College in Macon, GA, and her PhD through the BCDB Program at Emory University in Atlanta, GA. Michele manages the Promega Connections blog. She enjoys leisure reading, writing creative nonfiction and knitting, and the occasional cross-country skiing jaunt.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.