Credibility of Scientific Papers. - Page 2
Page 2 of 4 FirstFirst 1234 LastLast
Results 21 to 40 of 65
  1. #21
    Join Date
    Feb 2018
    Location
    Lambton Shores, Ontario, Canada
    Posts
    280

    Default Re: Credibility of Scientific Papers.

    I'm a card-carrying scientist (I'm browsing beesource as a way to avoiding working on editing a study), and I have to say, threads like this are hard for me to read. The degree to which the general public misunderstands how science works and how scientists operates is aggravating to say the least. A few points, in response to some of what was written in this thread.

    Quote Originally Posted by jim lyon View Post
    Interesting. One of the problems I have is that these “studies” are usually legit to the degree that the public needs to understand that they are quite often funded by a group looking for a particular result and run by a scientist quite interested in the accompanying publicity that comes with his “discovery”. The phrase “publish or parish” may well accurately describe the pressure a researcher may be feeling. Amazingly they always seem to find the result they are looking for but one always needs to look carefully at the numbers to determine how significant the conclusion really is and are all variables truly factored in? I guess I’ve gotten so jaded by this whole process that when I read the headline trumpeting the results of “a new study” I either ignore it or read it with a great deal of suspicion.
    This is a classical example of several common misconceptions of how scientists work, how science itself works, and science funded. There is a lot to unpack here.

    Firstly, the vast majority (internationally, ~80%) of published research is performed by independent scientists (e.g. in university, research centre, or government labs) and is funded via governmental grants or funds from charities/NPO's. And when we are funded by a company or organisation that may create the appearance of a conflict, it is openly declared. The very reason you know that some science is funded by industry or other sources of conflict is because of the formal ethical framework we work under that requires us to disclose. Failing to disclose a conflict can be a career-ending event, as is faking results to make them match a desired outcome. To put that into context, those are two of the few things that can get a tenured prof like myself fired, and it (quite publicly) ends a number of careers each year.

    Secondly, we rarely find the result we are "looking for", but when you author a study there is a degree of "storytelling" involved that can make it appear that way. As a general rule, less than one in ten tested hypothesis are found to be "correct" (meaning the data aligns with the hypothesis - it can still subsequently be found to be false). But failed hypotheses are rarely the focus of studies, and in many cases do not even get discussed. So when you read the literature you tend to see the 1-in-10 things that worked, while the 9-in-10 that didn't are either given minimal coverage or are simply unmentioned.

    Thirdly, science is an iterative process. What that means is that any one study, on its own and in isolation, has nearly zero scientific value. Confidence in scientific conclusions comes not from single studies, but rather from consistent results between studies, results that build upon previous results, and replication.

    Lastly, looking to the media for scientific information is guaranteed to lead you astray. Media isn't interested in accurately reflecting the science or what a particular study/advancement really means. What they are interested in is sexy headlines which drives revenue. As one example, back in 2003 I led a study that discovered a treatment that provided a modest improvement in one secondary clinical aspect experienced by HIV patients (we found that a very old and commonly used cancer drug could help reduce some of the susceptibility to infection HIV patients have, even when their infection is effectively controlled by antiretroviral drugs). And the drug had pretty horrible side-effects, so it would never actually be used in patients. According to a story carried by Reuters, we had discovered a new drug that cured HIV. A lot of Reuters affiliates bought the story - so Reuters got their $$$ - but they didn't even manage to spell my name right.

    Also, 50-60% of scientists are women, so our discoveries are not "his discovery"

    Quote Originally Posted by Robert Holcombe View Post
    Control is making a problem "go away, return and go away", black and white. Biology, with so many independent variables repeatability and control issues makes it really difficult. Is "statistically significant" really "significant" or can it be used to influence results and sell an idea?
    Controlling things in biology is not as hard as this - in a lab setting we can usually keep everything constant but for the one variable we manipulate. In the field (clinic, etc) we cannot control things to that extent, but there are very well established and successful methods for quantifying and accounting for confounding factors that cannot be controlled. The real issue tends to be one of money - addressing confounds usually requires a larger study population and more measurements; both of which are costly. Which is why smaller studies are often treated as unreliable until repeated or replaced with a larger study population.

    As for the question "is statistically significant also biologically significant"...that is actually an easy question to answer. Most fields use some sort of effect size measurement as part of their routine statistical analyses (e.g. odds ratios in clinical studies). These remove the question of how significant a significant result is, as it explicitly measures the size of the impact your experimental manipulation actually had.

    Quote Originally Posted by GregV View Post
    In the cases of bee products consistency and repeatability is impossible.
    With this in mind, consistency and repeatability of the results is impossible.
    With this in mind, consistency and repeatability of the conclusions is impossible.
    Sorry, but this is utter nonsense. The entirety of statistics (and thus science) is predicated on the assumption that everything we measure will be subject to variation - e.g. different honey compositions, differences in biology between individuals, etc. The whole entire point of the science of statistics is to provide tools that allow us to accurately quantify and understand that variation, and to be able to compare between groups in the presence of that variation. Which is why, despite literally everything in this universe being subject to variation, science has been able to make the advances that it has made, and continues to make.

    Quote Originally Posted by Cloverdale View Post
    I’ve been thinking on this also; being beekeepers some of us want to sell everything from the hive, including “bee pollen” and making propolis tinctures, and selling royal jelly. The science available for the aforementioned list is for the benefit of honey bees. But is there any science on how beneficial it would be to human beings? All we have is our own experience, or those experiences of others, to go on. Is there any papers or science behind these claims that I can pass on to customers who ask about these products? I’m not trying to judge anyone who does sell these products.
    While somewhat OT of the thread, the evidence for any sort of clinical benefit of any hive product is pretty slim (with, in some cases, data explicitly showing some claims to be wrong) . Most of the claims are either unevidenced nonsense (i.e. people just made it up...probably because they had something to sell and needed a pitch), or radical and unsupported extension of a minor observation into claims of medical efficacy.

    One quick example - honey is often touted as a a treatment for a variety of infectious diseases (flu, infected cuts, etc). This idea is based on the well supported fact that, in its undiluted form, honey has a modest antimicrobial activity. The mechanism of that effect is known and well established (a combination of microbriostatic sugar concentrations + enzymes that create peroxides). But as a clinical intervention...it doesn't really work. Pure honey if placed on a wound is mildly antimicrobrial (about as effective as an OTC agent like neosporin), and is far inferior to conventional clinical treatments (e.g. sutures + medical-grade antiseptics). So it is not wrong to say honey is antimicrobial...but how we often see honey advertised to be used as an antimicrobial (e.g. consumed, in teas, in tinctures, etc) involves doing things that would eliminate that effect. For example, anything that dilutes the honey - even only slightly - dramatically curtails its antimicrobial effect. And that lost effect is completely predictable. The suppressible effect on microbial growth created by the sugar in honey is right on the verge of where suppression ends. As little as a 2% dilution (e.g. going from 18% to 20% moisture) eliminates that effect for many microorgnaisms, including many pathogens. Likewise, the peroxides in honey are created by an enzyme called glucose oxidase, which is instantly inactivated in even modestly hot water (65C/150F), in our digestive tract, and its activity slows quickly as honey is diluted.

    Essentially, people who market in this way are taking a small truth and stretching it beyond both reason and into territory where studies have directly shown those effects to be eliminated.

    Bryan

  2. Remove Advertisements
    BeeSource.com
    Advertisements
     

  3. #22
    Join Date
    Mar 2012
    Location
    Catskills, Delaware Cty, New York, USA
    Posts
    1,757

    Default Re: Credibility of Scientific Papers.

    Bryan, thank you for this post! Deb
    Proverbs 16:24

  4. #23
    Join Date
    Aug 2014
    Location
    England, UK
    Posts
    1,684

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by SuiGeneris View Post
    Sorry, but this is utter nonsense.
    Nonsense ? It's most certainly not - and it's got sod-all to do with statistics.

    For example, there is no such substance as 'pure honey' - it cannot be defined in anything other than the vaguest terms. Within the last few years, the EU mandarins (who are obsessed with defining as many aspects of everyday life as they possibly can) tried to define honey precisely - and failed. One of the problems they couldn't surmount was that of the presence of pollen. They'd already provisionally 'defined' honey as being a natural substance to which nothing must either have been added or removed. But pollen - being a frequent constituent of honey - then gave them a headache. It clearly isn't a sweet sugary liquid as is the bulk of honey, and so perhaps ought to be removed to generate a more 'pure' product - but that would entail contravening their own prior working definition.
    They even considered imposing a condition that honey must never be extracted from any comb which had previously held brood - until it was pointed out to them that this contravened the basic concept of a 'natural' product, and would thus impose some degree of artificiality. So they had to come up with a mealy-mouthed compromise which, afaik, remains to this day along with the nonsensical legal requirement to provide a 'Best Before' date on jars of honey.

    Keeping this critique rather more science-oriented - how do guys running NMR scans determine whether their test sample is adulterated or not ? Well they can't - precisely because there's no such substance as a 'standard' honey against which to compare it with. So - the best they have come up with is to take samples from a region over a period of time which are then considered to be representative, against which the suspect sample is compared. But a good lawyer can have a field day with such an approach, precisely because (in Greg's words) "In the case of bee products consistency and repeatability is impossible."

    Some of us do not have a very high regard for statistics - let's kick off by examining the choice of alpha (p-value) which is invariably set at 0.05 - which corresponds to a 5% chance that results have occurred at random. Why choose 5% ? Statisticians are loathe to admit that a p-value of 0.05 is completely arbitrary. R.A. Fischer, the father of modern statistics, plucked 0.05 out of fresh air, and it has simply stuck.

    The value of alpha needs to be chosen before an experiment is conducted, because if chosen afterwards, a number could be selected to indicate that the data is significant no matter what it consisted of - and so by manipulating the value of alpha (and thus altering the significance threshold) it becomes possible to even 'prove' the opposite conclusion of what the data actually shows.

    Hence the well-known expression that, "There are lies, damned lies, and statistics", a phrase which describes perfectly the persuasive power of numbers, particularly the use of statistics to bolster weak arguments.

    'Significance' itself has two completely different meanings - one being that the data is unlikely to have occurred by chance - the other being that the data differs from the control to such a degree that the results themselves have some meaningful interpretation. So that it would be entirely possible to have data which is highly unlikely to have occurred by chance, and yet the experiment itself be so badly formulated as to provided worthless conclusions: significant, yet not significant. (I have one or two of these in mind as I write this).

    Personally I wouldn't describe statistics as being a science - I'd say it's one method which can be employed to avoid having to admit that you've discovered nothing of value, either that or have got something hopelessly wrong.
    LJ
    A Heretics Guide to Beekeeping http://heretics-guide.atwebpages.com/

  5. #24

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by little_john View Post
    Keeping this critique rather more science-oriented - how do guys running NMR scans determine whether their test sample is adulterated or not ? Well they can't - precisely because there's no such substance as a 'standard' honey against which to compare it with.

    https://www.news-medical.net/news/20...using-NMR.aspx


    Also fits nicely here: https://entomologytoday.org/2019/02/...et-discovered/

  6. #25
    Join Date
    Apr 2016
    Location
    Rutland County, Vermont,USA
    Posts
    2,291

    Default Re: Credibility of Scientific Papers.

    Thank you Bryan for taking the time to make a thoughtful response. I MIGHT take issue on the practical value of honey as a anti-microbial. Our local hospital has been using Medihoney which is labeled 100% Manuka honey for a wound dressing. This is an external use and I am unsure if it has been used if the wound is seriously infected, but it commonly used here for open wounds and bedsores with good results. I have no idea what studies have been performed on it, but they no longer use neosporin. Thanks again. J

  7. #26
    Join Date
    May 2011
    Location
    Algoma District Northern Ontario, Canada
    Posts
    5,045

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by Fivej View Post
    Thank you Bryan for taking the time to make a thoughtful response. I MIGHT take issue on the practical value of honey as a anti-microbial. Our local hospital has been using Medihoney which is labeled 100% Manuka honey for a wound dressing. This is an external use and I am unsure if it has been used if the wound is seriously infected, but it commonly used here for open wounds and bedsores with good results. I have no idea what studies have been performed on it, but they no longer use neosporin. Thanks again. J
    Isn't the honey used for wound dressing sterilized by radiation or UV light or some such process. Natural honey has some anti microbial properties but it is not sterile.
    Plain old white sugar poultice makes a good emergency wound dressing.
    Frank

  8. #27
    Join Date
    Feb 2006
    Location
    Herrick, SD USA
    Posts
    6,639

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by SuiGeneris View Post

    Also, 50-60% of scientists are women, so our discoveries are not "his discovery"


    Bryan
    You got me there Bryan but given the fact that the English language lacks a singular unisex pronoun I suppose I should have referred to said scientist as “his/her” or maybe “her/his”.
    "People will generally accept facts as truth only if the facts agree with what they already believe."- Andy Rooney

  9. #28
    Join Date
    Feb 2013
    Location
    Barbados
    Posts
    9

    Default

    Normally I keep quiet, but two things need to comment.

    First with research: one must remember got have a problem to get grant money.
    No problem with bees no grants.
    Take all research with a skeptical eye.

    Second: about small cell and mites.
    I live in a closed environment. A island, we only have small cell bees.
    We have no real problem with mites.
    So if you need proof, try it.. you just might like the results.😁

    Quote Originally Posted by JWChesnut View Post
    I would say in beekeeping the far more serious problem is the hucksters that sell completly undocumented claims, such as, "I changed to small cell and my mites ceased to be a concern".

    No actual data to support these claims have ever been presented. The hucksters evade and procrastinate when confronted with a demand to back up their nostrums.

  10. #29
    Join Date
    Apr 2017
    Location
    Aylett, Virginia
    Posts
    4,257

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by Airwreck View Post
    I live in a closed environment.
    We have no real problem with mites.
    The perfect location for TF beekeeping, small cell or otherwise.
    Thankfully, the bees are smarter than I am. They are doing well, in spite of my efforts to help them.

  11. #30
    Join Date
    Sep 2018
    Location
    Northern Lower Michigan, USA
    Posts
    809

    Default Re: Credibility of Scientific Papers.

    LJ
    +1 agree, a study is those bees, that year, in that location, with the weather from that year. Very little is repeatable.
    GG

  12. #31
    Join Date
    Sep 2018
    Location
    Northern Lower Michigan, USA
    Posts
    809

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by Cloverdale View Post
    I’ve been thinking on this also; being beekeepers some of us want to sell everything from the hive, including “bee pollen” and making propolis tinctures, and selling royal jelly. The science available for the aforementioned list is for the benefit of honey bees. But is there any science on how beneficial it would be to human beings? All we have is our own experience, or those experiences of others, to go on. Is there any papers or science behind these claims that I can pass on to customers who ask about these products? I’m not trying to judge anyone who does sell these products.
    The issue with "papers" on pollen , propolis and Jelly is the "funding" In general funds are made available to a study of some "compound" that the funding party owns the patent on. so make a unique product spend 10 K on studies to then make millions. Basically the business plan for most drug companies. Also pollen and jelly and propolis are not "patentable" and, every one in this forum would sell and compete in a market if one was created. So the effort is to po po the evidence of health claims made by bee product proponents, as not proven , no studies, etc to allow them to sell their products. to add insult to injury, IF you claim health effects and give/sell the hive product, that is "prescribing with out a Doctors license" and you get to go to jail or be fined. One can offer the product, and I would not make any claims. with the internet information is out there if they seek they will find. There is a reason most of the Russian athletes take bee pollen. they also have reason to not give away the methods, we are in an odd times. You may be better off to look for a natural type Doctor and offer the products to them, to offer to their clients. they CAN prescribe and do understand the need. Or offer your card as a source.
    GG

  13. #32
    Join Date
    Sep 2018
    Location
    Northern Lower Michigan, USA
    Posts
    809

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by JWPalmer View Post
    Deb, with Ruth swearing by it, I am planning to put propolis screens on two of my hives. If there is any arthritic benefit at all, I am game. The hip hurts so bad at the end of the day I can hardly stand upright. If the bees can help, I'll let them. Being caucasians, I expect they will have the first screen ready for me up in no time.
    JWP Pantothenic acid is in Royal Jelly it helps with Arthritic conditions. My Grand Dad would sting his hand every week to get relief from Arthritic finger joints. A couple things to perhaps try. Do you notice a difference in your hip when stung a few times?
    GG

  14. #33
    Join Date
    Sep 2018
    Location
    Northern Lower Michigan, USA
    Posts
    809

    Default Re: Credibility of Scientific Papers.

    Thank you Bryan for the reply from the "research side"

    However I disagree with this statement:
    Firstly, the vast majority (internationally, ~80%) of published research is performed by independent scientists

    Seriously you believe 80 percent of the studies we see in mags, news paper, TV etc is unbiased in any way?

    Just look at Man made global warming, that should help iron this out.

    I cannot accept 80% unbiased as fact, Sorry if I sound like a Skeptic.
    GG

  15. #34
    Join Date
    Apr 2017
    Location
    Aylett, Virginia
    Posts
    4,257

    Lightbulb Re: Credibility of Scientific Papers.

    Quote Originally Posted by Gray Goose View Post
    Do you notice a difference in your hip when stung a few times?
    No, but then again, I have not ever been stung in the hip. I am just a bit skeptical of apitherapy, but I understand the body producing it's own cortisone in response to a bee sting. Propolis is entirely different, a plant material (tree sap for the most part) that the bees use to seal cracks. Who would have thought? But, now we have glucosamine on store shelves and we all know where that comes from.
    Thankfully, the bees are smarter than I am. They are doing well, in spite of my efforts to help them.

  16. #35
    Join Date
    Apr 2016
    Location
    Rutland County, Vermont,USA
    Posts
    2,291

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by crofter View Post
    Isn't the honey used for wound dressing sterilized by radiation or UV light or some such process. Natural honey has some anti microbial properties but it is not sterile.
    Plain old white sugar poultice makes a good emergency wound dressing.
    Yes, it is irradiated with Gama rays

  17. #36

    Default Re: Credibility of Scientific Papers.

    Hi Greg,

    While exact reproducibility of bee products is impossible, it is quite possible that the active chemicals which produce some effect are common to all or at least a substantial majority of a particular bee product. For example, bees collecting propolis may be looking for a particular aromatic compound or compounds present in many plant resins, and only taking resins which have a significant proportion of those chemicals. The rest of it could be anything not toxic. It is noteworthy that propolis always smells the same (at least to me). It is a delightful smell, and in the summer you can tell you are near a bee tree when you smell it in the woods.

    Of course, that isn't good enough for the FDA, so there is no strong motivation to test bee products, as it would be difficult to make commercial pharmaceuticals from them. That is why all of the studies are small and preliminary.

    That said, my own observation is propolis is a more potent topical antibiotic, and broader spectrum, than any commercial products. (It has cured infections that resisted all other antibiotics, and appears effective against bacterial, fungal, and viral infections). I make an ointment, and share it with relatives, who share it with friends... It is hard to find enough propolis in my hives. I only collect about 100g per year.

  18. #37

    Default Re: Credibility of Scientific Papers.

    The problem with honey as a wound dressing is the possible presence of clostridium botulinum spores. These spores are common in nature, and can be in nectar. The botulinum bacterium is an obligate anaerobe - it can't survive in an oxidizing environment. Since honey is slightly acidic, the spores stay dormant. However in a wound, with poor blood supply, the spores can germinate, and cause an infection. (Wound botulism) In order to kill the spores, it is necessary to either heat the honey to something like 270F (118C) or irradiate it with gamma radiation, which kills everything. Since heating it that hot will ruin it, they use gamma radiation I suppose.

    The actual risk is low, I suspect.

    The possible presence of these spores is also the reason not to give honey to infants. Since infants - particularly breastfed infants - have a very low acid digestive tract, the spores in honey could germinate and cause an infection. (Infant Botulism).

    While I certainly wouldn't recommend feeding honey to infants, it is not an uncommon thing. Muslims in particular include feeding a small amount of honey to an infant as a religious ritual of some sort, based I think on something in the hadiths. A large study by Kaiser Permanente in California looked into the causes of infant botulism, and both living in a rural area and being exclusively breast fed had higher odds ratios for infant botulism than feeding honey to infants. So we are back to the credibility of scientific papers...

  19. #38
    Join Date
    Feb 2018
    Location
    Ridgecrest, CA USA
    Posts
    10

    Default

    Perhaps one of the best examples is the reported but non-existent link between autism and vaccinations. Look where that has put us.
    Unfortunately that claim has been debunked as if you are African-American and male and an infant under 3 there is a link. I suspect acetaminophen is the actual link as African-Americans have processing issues with that more often and there are other studies indicating it probably is a culprit. More study is needed.

  20. #39

    Default Re: Credibility of Scientific Papers.

    Quote Originally Posted by Gray Goose View Post
    Thank you Bryan for the reply from the "research side"

    However I disagree with this statement:
    Firstly, the vast majority (internationally, ~80%) of published research is performed by independent scientists

    Seriously you believe 80 percent of the studies we see in mags, news paper, TV etc is unbiased in any way?

    Just look at Man made global warming, that should help iron this out.

    I cannot accept 80% unbiased as fact, Sorry if I sound like a Skeptic.
    GG
    Bias is a tricky thing. There is intentional bias, and unintentional bias. I suspect unintentional bias is present in every study, but is most times not strong enough to substantially affect the results, although it might somewhat affect how they are presented. We all have our biases, and the most difficult ones to deal with are the ones we don't know about. The most common bias is the bias toward a successful outcome. Intentional bias - putting your thumb on the scale, not recording all of your data (outliers), is basically fraud. It is more rare, but probably happens about half of the time (personal estimate- just my biased opinion.

    I did a bit of research when I was younger. I found the temptation to censor my data to show the expected outcome more clearly was very strong. If I recall correctly, I didn't entirely resist it. This is part of the reason I decided on applied science for my working career. Research science frightened me, because it exposed my tendency to cheat a little. This is just basic human wickedness. We all have it. It is why we need a savior.

    One thing I appreciate about Randy Oliver is that he shows all of his raw data, and discusses how he wasn't able to execute all of his plans as initially conceived. It is a good practice.

    So my opinion is that bias is quite common. This doesn't always affect the outcome. It looks like Gregor Mendel, who didn't understand statistics, modified his data a bit to show the proportions of peas of different sorts would match his theory of inheritance more accurately than you might expect. That doesn't change the fact that he was right, and that he was the first to propose a theory of genetic inheritance. His bias (or maybe he was just very lucky) didn't affect the outcome.

    Perhaps a bigger problem is the "p" value of 0.05. in a world where 90% of the time a hypothesis is false, a "p" value of 0.05 represents closer to a 50/50 chance the result is repeatable. I can try and explain that, but it might not make sense. If you see a "p" value of 0.005, there is a good chance the result is repeatable.

    Statistical inference is the weakest form of evidence in the scientific world. However, it is often the only kind we can find, because we do not understand the underlying cause and effect well enough test variables directly.

  21. #40
    Join Date
    Mar 2015
    Location
    Tehachapi, California, USA
    Posts
    20

    Default Re: Credibility of Scientific Papers.

    Peer reviewed studies are the only documents to take seriously.

    My 2 cents. I don't see any deception of this shop towel method work in process. Randy Oliver appears to be transparent and honest.
    http://scientificbeekeeping.com/oxal...towel-updates/

Page 2 of 4 FirstFirst 1234 LastLast

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •