inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #76 of 90: Virtual Sea Monkey (karish) Mon 10 Jul 17 16:26
    
There are some stupid, aloppy statements in Gelman's blog post. I
don't think it's going to be productive to engage him.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #77 of 90: Gary Greenberg (gberg) Tue 11 Jul 17 19:06
    
I think he was getting at the mismatch between the way we study
disease and its treatment--mostly through the double-blind
controlled trial-- and the kind of disease that Julie was writing
about. He is correct about this, and as time goes on, more diseases
will surface that are not suitable to the clinical trial--because
their symptoms are heterogeneous (as in CFS/ME), or because the
pathways that lead to the symptoms are heterogeneous (as in
depression) or because the major symptoms are subjective, and so on.
Thedouble-blind controlled study is aa historical, rather than a
scientific development, after all, and there's no reason to think
it's a good way to assay every disease. With these mismatches on the
rise, history may be  in the process of correcting itself.

One thing in that article that bothered me. He desribes the clinical
trial by saying "you gather a bunch of patients and randomly assign
some to a control group, which receives no treatment, and some to an
experimental group, which does." But this is not accurate. Generally
the control group receives treatment, but it is placebo treatment.
This is not the same as no treatment, not even close. And especially
with disorders like CFS/ME, placebo responses can be very strong.
I'm pretty surprised the New Yorker let that through, but then again
I can say from personal experience that the fact checking on the
website is nowhere near as robust as the fact checking for theprint
version.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #78 of 90: Mark McDonough (mcdee) Tue 11 Jul 17 19:16
    
Yeah, that's a startling and substantive error.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #79 of 90: Gary Greenberg (gberg) Thu 13 Jul 17 13:53
    
I did email him. Here's the exchange.

>
> Prof. Gelman--I'm catching up with your critique of clinical
trials, which I think is well taken. But with me you're preaching to
the converted, and there's an error in your New Yorker piece that is
not going to help you with the unconverted. Clinical trials in
general do not compare a treatment group to a no-treatment group.
They compare a treatment group to a placebo group. The treatment
effect is not the change in the treatment group minus the change in
the no-treatment group, but rather treatment minus placebo. That
means that placebo effects are part of every treatment effect.
>
> Two implications for your well-taken critique: 1. The clinical
trial assumes that the placebo effect is noise, and is designed to
filter it out. Obviously a problem if indeed there is a difference
between placebo and no treatment (which there is), because it means
that an entire category of potential treatment is simply being used
as a stalking horse rather than being investigated. 2. Clinical
trials in general don't have a no-treatment condition, which means
that the actual signal, of medication, placebo or both, can't really
be discerned. That is a crucial weakness in an experimental design
that is the foundation of medical research. 


His response:

Hi, Gary, thanks for the note.  Good point:  "control" can include a
placebo or indeed any alternative treatment.  I don't think I can
easily correct on New Yorker site but I can correct on blog.
Andrew


I think he misses the point. "Control," in the context of clinical
trials nearly always involves a placebo treatment. Sometimes it
involves a known drug.  It never, that I know of, includes an
alternative treatment. I did not bother responding. 

But the larger point (his and mine) remains: conditions like Julie's
do not lend themselves well to the current regime.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #80 of 90: (fom) Thu 13 Jul 17 15:08
    
I read it as "alternative" in the sense of "some other drug," not 
"alternative" as in some new agey treatment or whatever. That is the 
basic meaning of alternative, I think.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #81 of 90: Julie Rehmeyer (jrehmeyer) Thu 13 Jul 17 21:09
    
Thanks for writing to him, Gary.

His review was an odd experience for me. Of course, it's delightful
to be positively reviewed in the New Yorker! But it's very odd to
have the PACE trial held up as an example of the limitations of
randomized control trials, when the real problem here isn't that,
but that it was so dreadfully done. 

I just posted this comment on Gelman's site:

Hi Andrew,

Thanks so much for this review! Three points:

1. There was an important error in the New Yorker version. It says:
One of their chief objections is that the pace research team, while
collecting its data, changed the main metric of “recovery” from
objective measures, such as fitness and employment status, to
subjective ones, such as the patients’ self-evaluations over time. 

The primary outcomes were always the subjective ones, so this isn't
accurate. However, they weakened their thresholds on the subjective
measures for "recovery" (as well as for every other benchmark in the
trial) enormously -- so much so that (as you point out in your
longer version) patients could get worse on fatigue and physical
function and still be considered "recovered." When the objective
measures didn't improve, the researchers dismissed them, even though
they themselves had chosen them.

2. You make the point that RCTs tell you less when you're looking at
heterogeneous groups. But in the PACE trial, the researchers chose a
broad definition of ME/CFS that exacerbated this problem, including
a far more diverse population than was necessary or appropriate.

While it's true that ME/CFS (or SEID) may well be a cluster of
related illnesses, it's also the case that definition really
matters. The Mayo Clinic definition that you offer is pretty much
the worst one out there -- it negates itself, giving a medical
explanation by demanding that there is no medical explanation. The
one the PACE trial used isn't quite that bad, but it's close,
requiring only six months of disabling fatigue as the primary
symptom. That can include a significant percentage of depressed
people, which is most likely part of why the treatments appeared
effective -- increasing activity is a good treatment for depression.
And that definition doesn't require the hallmark symptom of the
disease, exercise intolerance (as the term SEID emphasizes). 

Though you se the term SEID, you didn't use the SEID definition,
which requires exercise intolerance (the hallmark of the disease)
along with fatigue, unrefreshing sleep, and either cognitive
problems or difficulties regulating heart rate or blood pressure
when standing. The SEID definition is designed to be simple for
clinicians to use and able to catch all patients with the disease,
at the cost of including some others. That's useful for clinical
practice, but not so useful for research. The best current
definition for research (IMO) is the Canadian Consensus Criteria,
which is more restrictive and also more complex:
http://me-pedia.org/wiki/Canadian_Consensus_Criteria. It adds
requirements for pain and neuroimmune symptoms.

Even among patients who meet the Canadian Consensus Criteria, there
is likely heterogeneity, but far, far less so than simply looking at
really tired folks.

The PACE investigators and their intellectual heirs persist in using
very broad definitions to this day.

3. Finally, I just wanted to point out that actually you do have
access to a chunk of the data. Patients managed to get the data
relevant to evaluating recovery by the original defintion, which
they made publicly available here:
https://sites.google.com/site/pacefoir/pace-ipd_foia-qmul-2014-f73.xlsx?attred
irects=0.

The data showed that accordingly to the original definition of
recovery, the researchers had null results. That is, neither CBT nor
GET led a statistically significant number of patients to "recover."
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #82 of 90: Julie Rehmeyer (jrehmeyer) Thu 13 Jul 17 21:10
    
Gary, I'm trying to make sure I understand your point. Are you
essentially saying that placebo effects can be valuable?
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #83 of 90: Gary Greenberg (gberg) Fri 14 Jul 17 05:03
    
Yes, and the clinical trial design implicitly endorses that view by
assuming that placebo responses are part of every response to a
treatment. But, perversely, the design also creates the impression
that placebo treatments are not valuable by treating them as noise.
Which often means researchers are on to the next drug rather than
trying to figure out why the placebo group improved.


The cynical view is that the reason for our relative ignorance about
placebo effects is that there is no money in sugar pills. But that's
only part of the explanation. It's also the case that modern medical
research, and especially the clinical trial, is extremely ill-suited
to placebo treatments. The farther away you get from diseases with
clear molecular causes that are remedied by treatments aimed at
them, the less valuable that approach is. 
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #84 of 90: Julie Rehmeyer (jrehmeyer) Fri 14 Jul 17 05:19
    
Fwiw, One study showed that the placebo effect is weaker with
quantity syndrome treatments in typical:
https://www.ncbi.nlm.nih.gov/m/pubmed/15784798/
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #85 of 90: Gary Greenberg (gberg) Fri 14 Jul 17 12:48
    
That is an interesting study. And it's not unlike other
meta-analyses of placebo effects: they tend to indicate that placebo
responses are not as strong as individual studies make them out to
be. And in general, finding a placebo signal has proven to be
difficult. I'm not sure what this means, but it isn't entirely clear
thata it means that placebo effects are weak. 

REading between the lines here, Julie, I wonder whether you have a
sense that if there were a strong placebo response in CFS/ME, that
this would somehow deligitimize the diagnosis or the people  who
have it.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #86 of 90: Julie Rehmeyer (jrehmeyer) Fri 14 Jul 17 12:55
    
Not necessarily. But I do think it's interesting that there's a
general assumption that placebo effects in ME/CFS would be
particularly strong, even though the evidence doesn't support that.
Why did you say that?
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #87 of 90: Julie Sherman (julieswn) Mon 24 Jul 17 07:26
    
Julie will be interviewed on Your Call on KALW today at 10am.
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #88 of 90: David Gans (tnf) Mon 24 Jul 17 07:49
    
(streaming live at <http://www.kalw.org> - 10-11am Pacific
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #89 of 90: Jon Lebkowsky (jonl) Mon 24 Jul 17 16:11
    
Apologies for being late to close this discussion and thank Julie
Rehmeyer and everyone who participated! 
  
inkwell.vue.497 : Julie Rehmeyer, Through the Shadowlands
permalink #90 of 90: Julie Rehmeyer (jrehmeyer) Mon 24 Jul 17 19:02
    
Thanks so much, everyone! It was a great conversation.

And by the way, the interview is available here:
<http://kalw.org/post/your-call-julie-rehmeyers-long-road-recovery-chronic-fati
gue-syndrome#stream/0>
  



Members: Enter the conference to participate

Subscribe to an RSS 2.0 feed of new responses in this topic RSS feed of new responses

 
   Join Us
 
Home | Learn About | Conferences | Member Pages | Mail | Store | Services & Help | Password | Join Us

Twitter G+ Facebook