Episode 107: Research – The Truth Behind the Headlines with Dr. Sarah Ballantyne

This post may contain affiliate links. Click here to see what that means.

Leave a Comment / By /

podcast graphic with photos of host and guest

Understanding Scientific Research

Here’s the deal – headlines are designed to grab our attention, and it’s not uncommon for reporters to simplify, exaggerate, or completely misrepresent a research study’s results. So, when we care about science and how our choices impact our health, how can we be sure we’re getting accurate information? Today, I’ve invited a research expert onto the podcast who many of you know and love: Dr. Sarah Ballantyne, also known as The Paleo Mom. She’s going to teach us how to critique a scientific study for quality, bias, and limits of proof.

Listen to the Show

Podcast: Play in new window | Download

Show Notes

  • Intro (0:00)
  • Thank You to Our Podcast Sponsor: ShopAIP (2:34)
    • When you visit ShopAIP’s website, you can quickly find ingredients by category. Today, I’m highlighting crunchy snacks. Part of making a healing diet sustainable is having healthy options when cravings hit. See ShopAIP’s entire chip selection here.
    • An online store, where everything sold is compliant with the elimination-phase of the Paleo Autoimmune Protocol. ShopAIP sells a wide variety of products, including protein bars, AIP-friendly spice blends, cooking and baking ingredients, snacks, non-toxic skincare, and more.
    • If you’re a first-time customer, use the code PHOENIX for 10% off your order. Purchase here.
  • Meet Sarah (5:08)
  • What Are the Different Types of Research Studies? (13:41)
    • Epidemiological (Correlative) Study – looking to see if two things are linked. For example, consuming broccoli is linked to a lower cancer risk. However correlation doesn’t equal causation. It doesn’t mean we should dismiss these studies, but they need to be confirmed by other types of research.
    • Mechanistic (Experimental) Study – looking at exactly how two things are linked. Is it a direct link or a chain effect? What systems are being impacted? It usually takes many different mechanistic studies to explain hypotheses presented in epidemiological studies. Some are done on cells in a petri dish (in vitro). Some are done on animals.
    • Interventional Study (Clinical Trial) – once an epidemiological study is supported with mechanistic research, the next step is to test it as a treatment. For example, if consuming broccoli lowers cancer risk, and we’ve learned on a biochemical level how this works, can we take a population with a high risk of cancer, have them eat more broccoli, and see if their cancer risk is lowered? A wide variety of things can be tested as an intervention (i.e. diet, lifestyle, supplement, medication, etc.) The gold standard for this type of study is an RDBPC clinical trial (randomized, double blind, placebo control, clinical trial). With diet & lifestyle interventions, sometimes the mechanistic step will be skipped, since the intervention is low-risk. For medication interventions, mechanistic studies are an important part of the process, and drugs are also required to go through 4 phases of clinical trials. Phase 1 establishes safety, phase 2 looks at efficacy and side effects, phase 3 involves a larger pool of patients, and phase 4 follows its long-term use in the marketplace. Diet and lifestyle studies don’t usually go through this level of rigor.
    • Systematic Review & Meta-Analysis – a systematic review provides a thorough look at all the research on a specific topic, analyzing the quality of the research, noting where studies agree and disagree in their results, and summarizing what has been proven and what still remains uncertain. A meta-analysis is a statistical analysis of all of those studies put together (usually a combination of clinical trials). Note: studies often disagree with each other because there are so many potential variables – everything from gender and age of test subjects, to the testing environment and recording methods used. When a study has an aberrant result, Sarah doesn’t discount it. They might have stumbled upon a co-factor that can be important in future research.
  • Thank You to Our Podcast Sponsor – Paleo on the Go (31:33)
    • A frozen meal delivery service, 100% of their menu is compliant with the elimination phase of the paleo autoimmune protocol (AIP). They have over 5o items, including entrees, side dishes, broth, AIP-friendly bacon, and desserts.
    • Use the code PHOENIX for 10% off your first order.
  • Evaluating the Strength of a Study (33:14)
    • Appropriate Controls:
      • You need a point of comparison for your results, i.e. comparing intervention to no intervention, comparing intervention to standard treatment, comparing the effect of different dosages, etc.
      • You need the intervention to be standard – making sure that everyone in the treatment group is receiving an identical treatment, controlling for variables accordingly.
      • Randomization: when subjects are randomly assigned to the treatment vs. control group (rather than choosing themselves). This isn’t vital to a quality study, but it does strengthen it.
      • Blinding: when the subject doesn’t know what group they are in, this controls for the placebo effect; when the researcher doesn’t know which group the subject is in, this controls for research bias. A double-blind study controls for both. This is common in drug trials, but uncommon in mechanistic studies. Obviously in diet/lifestyle studies, you cannot blind the subject groups, but you can use blind controls for the researchers who are compiling the results.
    • Looking at Sample Size:
      • Multiple studies with large, diverse populations is ideal. However, most studies begin with small sample sizes. This is both a safety and a funding issue. Studies shouldn’t be dismissed because they are small. If the effect is very large, a small sample size is sufficient to show that (look at the P value). When the effect is smaller, a large sample size is needed to determine whether or not it’s a true effect.
    • Understanding the Statistics:
      • P Value: This represents the likelihood that the results of the study are a product of chance as opposed to the research intervention. A strong study result has a P value of less than .05. This means there’s a 95% chance that the results were caused by the intervention (not chance). Below .01 is even better. The bigger the effect + low variability in results = a lower P value (even with small studies). For example, Terry Wahls’ 2014 pilot research study had a P value of <.0005 for reduced fatigue after a 12-month implementation of The Wahls Protocol. That means there’s a 99.95% chance that the reduced fatigue was a result of the intervention (not chance). Those are strong results, even with a small sample size of just 20 people.
      • Absolute vs. Relative Risk: A study should state if the statistic it’s reporting is relative risk; otherwise the assumption is it’s absolute risk. Absolute risk relates to the general population. Relative risk relates to the affected population. For example: If within a general population of 10,000 people, 100 of them develop a certain kind of cancer, and you do a study with broccoli consumption and only 50 develop cancer, that’s a 50% reduction in relative risk, but only a .5% reduction in absolute risk. That’s a very big difference. They both have value, but it’s important to know what type of statistic is being shared. In research, you tend to see relative risk statistics shared in drug trials and absolute risk statistics shared in larger epidemiological studies. Be aware that headlines often choose to share relative risk, because it implies a larger effect.
    • Understanding Bias:
      • Demographics: Most medical research has been done on male subjects, and we’re discovering that results are often different in females. This is starting to change, but work still needs to be done, especially when it comes to people of color and diverse ethnic backgrounds.
      • Funding: It’s important to know who funded a study, because it often impacts the result. A glaring example is research by the soda industry into the “safety” of artificial sweeteners. It doesn’t mean that privately funded studies are always biased, but it’s something to keep in mind. Independently funded studies are more likely to be objective. Most peer-reviewed medical journals require conflicts of interest to be disclosed. You can usually find this information at the bottom of a research article.
      • Researcher Bias: Researchers often expect certain results in their study, so any steps they take to control for this bias strengthens the study results, i.e. blinding, objective measuring tools, etc.
      • Confirmation Bias: As human beings, we are more prone to look for research that supports our beliefs and dismiss research that contradicts our beliefs. If a well-designed study contradicts our beliefs, it’s important to be open to the fact that we might have been wrong. This doesn’t just apply to researchers. It applies to everyone reading the research, like us.
  • Breaking Down a News Headline: Coconut Oil Is Pure Poison (51:01)
    • This wasn’t even a medical study. It was a lecture given by a Harvard professor at a university in Germany that was recorded on YouTube and went viral. Note: the speech was in German. The only reason it went viral was due to the explosion of news headlines quoting the video in English and linking to the video. Since then, the video has been removed from YouTube and can no longer be viewed, yet the fear from the news headlines remains.
    • In this video, the professor was conflating general saturated fat research with specific coconut oil recommendations.
    • It is true that a diet high in saturated fat can be unhealthy, but not in the ways originally thought. Most of the research connecting it to high cholesterol has been disproven. However, there is a concern that saturated fat can act as a carrier molecule to bring an inflammatory endotoxin (LPS) into our bloodstream. This endotoxin is naturally produced by certain bacteria in our gut, but doesn’t usually get absorbed into the bloodstream without assistance (through leaky gut, alcohol intake, or high saturated fat intake). High vegetable intake produces a microbiome with less of this bacteria (one of many reasons that vegetables are the foundation of the AIP food pyramid). High saturated fat intake increases this bacteria (particularly in combination with a Standard American Diet). So why is saturated fat allowed on the paleo diet? It contains beneficial nutrients as well that aren’t present in plant-based fats. We don’t want to avoid it altogether; it’s simply detrimental in high quantities. Sarah recommends limiting saturated fat intake to 15% of the daily diet (which matches the average intake of hunter-gatherer populations). Fats that are safer to include in higher quantities are omega-3 fatty acids (found in seafood and grass-fed meats), and mono-unsaturated fats (found in avocados and extra-virgin olive oil).
    • What about coconut oil? While it does contain some saturated fat, 63% of the fat in coconut oil is medium chain triglycerides (MCTs), which have unique health benefits. They don’t act as carrier molecules for endotoxin. In fact, they strengthen the gut barrier (which helps prevent the transfer of endotoxins into the bloodstream). MCTs also increase the secretion of IgA antibodies in the gut (and secretory IgA deficiency increases autoimmune disease risk). MCTs also boosts HDL cholesterol (the good form of cholesterol that promotes health). Coconut oil is simply not the same as meat-based saturated fats.
    • Sarah’s personal diet: She focuses on diversity in her food choices. She enjoys fatty fish and fatty cuts of grass-fed meat. She eats a wide variety of vegetables, both raw and cooked. When cooking vegetables, she rotates through a wide variety of fats since they each have a unique nutrient profile: coconut oil, extra-virgin olive oil, avocado oil, lard, tallow, ghee. For things like kale salad, she likes macadamia and walnut oils for a change of flavor and nutrition. She doesn’t go out of her way to add a lot of extra fat to her diet, but in reasonable amounts, it’s an important part of a healthy diet.
    • Resources:
      • Sarah’s detailed blog post in response to this headline.
      • Props to the New York Times for being one of the only news outlets that tried to give a balanced view and didn’t use the word “poison” in its headline.
  • Outro (1:05:49)

You May Also Be Interested In

Spreading the Word

If you like the podcast, please leave a positive review in iTunes. It would mean the world to me, and also helps others find the podcast. Here are some quick instructions using your iPhone:

  • If you are already subscribed to my podcast: (1) Click the purple podcast icon. (2) At the bottom of the screen, click Library. (3) At the top of the screen, click Shows. (4) Click the Phoenix Helix podcast image. (5) Scroll down the page, and you’ll see Ratings and Reviews. Scroll down a little bit more and click on Write a Review. This will bring up the review screen. Tap 5 stars (if you love the podcast), and then click in the title box, and it will bring up the keyboard. Enter a title and short review. (6) Click Send in the upper right corner. (7) Thank you! Positive reviews give the podcast a higher search ranking in iTunes, helping people find it and letting them know it’s a quality podcast and worth their time to listen.
  • If you haven’t subscribed to my podcast: (1) Click the purple podcast icon. (2) In the lower right corner, click the magnifying class. (3) Type Phoenix Helix in the search box. (4) Click the podcast cover in the Show list. (5) If you’d like to subscribe, click the + sign at the top of the screen. (6) To write a review, scroll down the page, and you’ll see Ratings and Reviews. Scroll down a little bit more and click on Write a Review. This will bring up the review screen. Tap 5 stars (if you love the podcast), and then click in the title box, and it will bring up the keyboard. Enter a title and short review. (7) Click Send in the upper right corner. (8) Thank you! Positive reviews give the podcast a higher search ranking in iTunes, helping people find it and letting them know it’s a quality podcast and worth their time to listen.
Do You Have My Books?

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top