There are companies, professionals, and influencers that do cite research. Citing research is a good thing for the most part, but there are still some issues.  

Frequently they make claims based on research when the research does not entirely say what they claim it says. They leave out pertinent facts.

Oftentimes they cite studies with small sample sizes or control issues. They will cite pilot studies.

Pilot studies are a good starting point for research; however, nothing can be claimed definitely. This is because of the small sample sizes or control issues that are inherent in pilot studies. Pilot studies are good for determining if more research should be conducted in a particular area. The researchers know this. Researchers even say that in their paper. Researchers will acknowledge whether the sample size was too small; they will acknowledge control issues. They might even discuss the confidence interval.

This is irresponsible on the part of the companies, professionals, or influencers. They rarely ever say that it is a pilot study. They rarely ever mention the sample size or any control issues that the researchers discuss in their paper. They use much stronger language than the researchers.

They may believe that they are helping by sharing the information. They may have good intentions, but when they post incomplete information this leaves the reader misinformed. The reader might give the research more confidence than it deserves. It would almost be better in some cases if the reader never read it in the first place.

There is another possibility though. They may only be reading part of the conclusion instead of the whole study. That or they are just interested in creating clickbait or generating sales.

Some will defend their use of small studies by bringing up the fact that AAFCOs minimum for feeding trials is 8 dogs. While this is true, that by no means makes it right. That is an issue with AAFCO approved feeding trials that also needs to be addressed.

We should not be contributing to the idea that 8 dogs are enough. It is a good starting point. It shows that more research should be conducted. It shows the potential value of more extensive research to validate the results.

Would you place a significant amount of confidence in a study on eight humans? If they only tested the Covid-19 vaccine on 8 people, would you take it?

Many of these same people also discuss how AAFCO Feeding Trials are inadequate and how only 6 of the 8 dogs need to complete the trial. However, for a study with a small sample size that validates their beliefs, it is okay that it only has 8 dogs. Some will claim that it is common in veterinary studies to only use 8 dogs. This is not truly the case. While it does occur there are countless studies that use well more than 8 dogs.  

Example 1:

New Study: Adding Fresh White Cheese to dog food (Kibble) improved the digestive and immune system of dogs in 8 weeks.

The influencer fails to mention the fact that the sample size was 15 dogs. This was further broken down into 3 categories. 5 No Cheese, 5 White Cheese, and 5 White Cheese with Bifidobacterium Longum added. More Critically, they fail to mention that the white cheese that had the effect was the one with Bifidobacterium Longum added to it. The researchers specifically said, “Queso Blanco cheese without B. longum does not change the SCFA concentrations.” Furthermore, the researches wrote

“Supplementation of Queso Blanco cheese containing B. longum KACC 91563 increased proliferation of PBMCs more than the control group. Proliferation of PBMCs, which comprise T cells, B cells, monocytes, and NK cells, is generally used as an immune index in experimental and clinical studies. … These finding indicate that Queso Blanco cheese containing B. longum KACC 91563 improved proliferation of PBMCs response to T lymphocyte in healthy dogs. … These results suggest that B. longum KACC 91563 supplementation may enhance the function of gut immune barrier by increasing the level of pro-inflammatory cytokines. Overall, Queso Blanco cheese containing Bifidobacterium longum KACC 91563 administration in healthy beagle was effective in improving fecal microbiota and immunity.”

Example 2:

New Study Shows Raw Food Fed Dogs had an enhancement of innate immunity & Decrease in oxidative Stress. Which may prevent Hypersensitivites & Disturbed Immunity at a young age.

What is missing is the sample size was a grand total of 8 dogs. The study was broken down into 4 atopic dogs and 4 healthy dogs. This was further broken down to 2 atopic dogs fed kibble and 2 atopic dogs fed Raw, 2 healthy dogs fed raw, and 2 healthy dogs fed kibble. There were also several control issues.

The researchers acknowledge the control issues. They acknowledge that it is a small sample size.. Here is what the researchers actually wrote.

“Since this was a pilot study, the sample size was limited. To counteract this, two different algorithms were used to analyze the data. However, our results might still in part reflect the individual genetic differences between the dogs. Because these dogs were client-owned, it cannot be ruled out that different environmental factors may have affected the results. The KD and RMBDs had both very different macronutrient profiles and ingredients and their comparison was performed intentionally as a test between two common types of canine diet. Although this complicates the interpretation of the results it nevertheless shows the differential effect that diets had on gene expression. A larger sample size with more controlled diets should be used to validate the results of this study.

The present study showed that lipid metabolism and differentiation of keratinocytes were possibly altered in the skin of atopic dogs. Additionally, compared to the KD fed group, the gene transcription profile of dogs induced by the RMBD in this study is consistent with an enhancement of innate immunity and decreased oxidative stress and may have an important role in preventing hypersensitivities and a disturbed immunity. As there were two major factors differentiating the diets, processed vs. non-processed and high carbohydrate vs. high fat, further studies must be conducted to determine which, or to what extent these factors influenced the results seen in the present study.”

I’m not saying don’t post pilot studies. I am not saying don’t post studies with small sample sizes, or control issues.

Many of these studies can provide valuable insights, but they need to be viewed for what they are, not what we want them to be.

I am saying when you do post it include the sample size. I am saying include any control issues the researchers identify. I am saying include all of the important details. I am saying don’t pull one sentence from the study because you want it to generate traffic.

Will including this information dampen the ability of the post to spread, probably. Will it dampen enthusiasm, probably.  These however are a small price to pay in exchange for properly sharing information.

It’s easy to make broad or strong claims, and these claims do drive traffic. They generate more likes, follows, shares, and sales.

The people and companies know most people won’t take the time to read the research. They know people have a short attention span. They know people don’t like to read long words that they would have to look up the definition. They know people would rather read a 20-30 word image than read even a 100 word summary of a study. Whenever you devolve something like that, you are making it, so the reader is missing information. Can you read a 30 word summary of Crime and Punishment, and still have a good grasp of the book? Obviously, the answer is no.

When your only concern is trying to drive traffic or generate sales, 30 words works phenomenally better.

This is why it’s important to actually read the study instead of just trusting someone’s interpretation. It’s important to look at the sample size. It’s important to look at the controls. It’s important to look at potential conflicts. It’s important to actually read what the researchers say.

When I commented on an influencer’s post about the second study. In their private pay $10.00 a month to be a member Facebook group, they said I or the one other person who said something were food warriors, or a food troll they said I or the other person was making small digs. They said It’s easy to criticize from the cheap seats or from the peanut gallery.

Does this sound like a proper response to criticism? This leads me to believe that they know there is a problem. They know they should have referenced the fact that it was a pilot study with size and control issues, but instead of acknowledging that, they resorted to name-calling.

Researchers rarely ever make firm claims. They often say things like the results suggest ________, or ____ may help with _____, or ____ contributes to _____, or ____ may improve ______, or _____  appears to _____.

This is why it is so important to compare what the research actually says vs. what these companies, professionals, and influencers claim it says. There can be a very big difference. This is also why some people who write books or have a book deal need a science writer/editor.

Some of them could benefit from a science writer or editor for their social media posts.

Trust but Verify

We all want what is best for our pets. None of this means everything these companies professionals or influencers post is bad information. It just means the information shared should be taken with a grain of salt. I fully support raw feeding, but relying on studies with small sample sizes or control issues will only come back to bite us at some point. If we are supposed to be better than the big kibble companies, we need to be better.