-

Also posted over at Rogue Medic (now at EMS Blogs) and at Research Blogging.

-

There is a new research podcast specifically for EMS – EMS Research Podcast. On episode 2 we discuss several topics, including the research on the RAD-57 non-invasive monitor.

-

In How Not to Respond to Negative Research, I forgot to address one of the more important problems with the response of Dr. Michael O’Reilly.

-

Because the study used the same type of device and sensor and evaluated patients similar to those in previous studies, the differences in results were likely due to different methods by the investigators. The possible reasons for the discrepancy between the results reported in this study include multiple items addressed in the directions for use of the device and sensor, including:

•inappropriate finger positioning in the sensor

•inappropriately sized sensor for the subject’s finger

•timing of SpCO and COHb measurements not being exactly simultaneous

•increased methemoglobin level (which can be ruled out by measuring noninvasive methemoglobin levels [SpMet] with the Rad-57)

•patient motion

•external light interference

•device or sensor malfunction (there were many zero readings by Rad57 in which HbCO was considerably higher, which could be due to device malfunction and may indicate a need for service)^[1]

Dr. O’Reilly is telling us that the RAD-57 is too difficult to use in the ED (Emergency Department), but miraculously acquires accuracy and reliability at a fire scene.

timing of SpCO and COHb measurements not being exactly simultaneous

Exactly simultaneous?

Exactly?

Does carboxyhemoglobin vary that much that a few seconds later, the HbCO is wildly different?

If that is the case, why buy a machine that will only give us a snap shot of a rapidly fluctuating and unreliable number?

Is there any reason to believe that carboxyhemoglobin changes that rapidly and unpredictably?

No.

-

Three RAD devices and training in their use were supplied by Masimo Corporation for the duration of the study. Clinicians underwent training in use of the RAD device before study initiation. Measurement of RAD carboxyhemoglobin was performed simultaneously with sampling of arterial or venous blood for laboratory determination of carboxyhemoglobin level.^[2]

Simultaneously, but not exactly simultaneously?

Dr. O’Reilly is creating the impression that this is a difference. Should we believe that, in the ED, the levels of carboxyhemoglobin are rising dramatically between the application of the RAD-57 and the simultaneous drawing of blood, just because the word exact was not used?

The big problem with the RAD-57 was that it missed over half of the significantly elevated carboxyhemoglobin levels – some while indicating a carboxyhemoglobin level of zero. Did patients have a zero carboxyhemoglobin level one minute and a significantly elevated carboxyhemoglobin level the next minute?

Dr. O’Reilly seems to be indicating that carboxyhemoglobin operates on the same principle of uncertainty as Scrödinger’s Cat, but with a free random number generator.

-

There are several problems with researching EMS equipment in the much more stable environment of the ED.

The ED environment has much less variability than the EMS environment.

If vasoconstriction is a problem, the warmer ED is much less likely to produce vasoconstriction than being out in the cold, which is often where the RAD-57 will be used.

The people using the equipment are generally paying much more attention to what they are doing, if only because they have to document compliance with study protocols.

As stated in the study, the participants are often trained by the people most familiar with the equipment – not somebody who read a package insert, or watched a video, or was once trained by someone from the manufacturer.

Compared with use in the EMS environment, the ED environment can be seen as much closer to the ideal testing environment.

Is Dr. Reilly complaining that Masimo provides bad training on the use of the RAD-57?

If our device does not work in your hands, it’s your fault.

It is always a pleasure to deal with someone who stands behind a product and is looking out for the patients assessed/treated with that product.

Dr. Michael O’Reilly, who is Executive Vice President of Masimo Corporation, an officer of the corporation, and holds stock options in Masimo^[1], is not that person.

-

The RAD-57 may have some role in identifying elevated carboxyhemoglobin levels, but so far nobody can tell what that role is. As I pointed out earlier –

Less than half of the patients with elevated COHb were correctly identified.

If we screen a fire fighter for COHb, then we need to keep that fire fighter out of the fire.

Should anyone ever use a low RAD-57 reading to justify returning a fire fighter to a fire?

No.

-

-

Footnotes:

-

^[1] Performance of the Rad-57 pulse co-oximeter compared with standard laboratory carboxyhemoglobin measurement.
O’Reilly M.
Ann Emerg Med. 2010 Oct;56(4):442-4; author reply 444-5. No abstract available.
PMID: 20868919 [PubMed - indexed for MEDLINE]

Free Full Text of letter and author reply from Ann Emerg Med with links to Free Full Text PDF download

-

^[2] Performance of the RAD-57 pulse CO-oximeter compared with standard laboratory carboxyhemoglobin measurement.
Touger M, Birnbaum A, Wang J, Chou K, Pearson D, Bijur P.
Ann Emerg Med. 2010 Oct;56(4):382-8. Epub 2010 Jun 3.
PMID: 20605259 [PubMed - indexed for MEDLINE]

Free Full Text Article from Ann Emerg Med with links to Free Full Text PDF download

.

-

Also posted over at Rogue Medic (now at EMS Blogs) and at Research Blogging. There is a new research podcast specifically for EMS – EMS Research Podcast. On episode 2 we discuss several topics, including the research on the RAD-57 non-invasive monitor.

-

Continuing from The RAD-57 Pulse Co-Oximeter – Does It Work – Part I and from The RAD-57 Pulse Co-Oximeter – Does It Work – Part II. More on the use of the RAD-57.

-

To the Editor:

Masimo Corporation is the manufacturer of the Rad-57, a multiwavelength pulse carbon monoxide oximeter that measures noninvasive carboxyhemoglobin (SpCO) in the blood. The Rad-57 is the subject of a study by Touger et al¹ and an accompanying editorial² in this edition of Annals. Masimo appreciates the journal’s willingness to publish these comments about the study and editorial.^[1]

So far, so good.

Since the introduction of the Rad-57 in 2005, Masimo has received countless reports from clinicians that the device has enabled them to save lives and limit the damaging effects of carbon monoxide poisoning.^[1]

Only the second paragraph and Dr. O’Reilly is already wandering into the street corner sales pitch. This is what the alternative medicine charlatans always seem to say.

Countless? Why is it that none of the people making these anecdotal claims are able to count, but they expect us to trust them with our lives?

Given this accuracy specification, approximately 95% of SpCO measurements are expected to be within 2 SDs of the COHb value. If this specification were applied to limits of agreement calculation, the accuracy could be stated as 5.9% to +5.9%. The study by Touger et al¹ stated: “We determined a priori that a difference of ±5% carboxyhemoglobin would be considered clinically significant.” No rationale for the ±5% threshold was provided, but we do not believe it is appropriate to initiate a study with expectations greater than the stated performance of a device or drug. In short, there was a strong likelihood the study would result in a negative conclusion by the investigators before it even started.^[1]

Accuse the ones performing the study of approaching things with a bias.

So, what did the study state about the reasons for their range?

Finally, the selection of ±5% carboxyhemoglobin as a boundary for acceptable limits of agreement was based on presumed clinical significance but may be considered somewhat arbitrary. The actual limits of agreement (–11.6% to 14.4% carboxyhemoglobin) demonstrated in our study substantially exceeded this value, suggesting that the inference would have been the same even if a larger value, eg, ±10% carboxyhemoglobin, had been chosen.^[2]

Dr. O’Reilly is complaining that ±5% is narrower than the ±5.9% that he states should be used as the limit of 2 standard deviations. Dr. Touger points out that ±5.9% would not make the RAD-57 look good. Even ±10% wouldn’t make the Rad-57 look good.

The study results are significantly different from those of other available studies, as well as from Masimo’s internal test data from subjects with 1% to 40% COHb levels. However, the study results by Touger et al¹ are discussed as being representative of device performance, and strong conclusions are made by both the authors and the editors according to the study results.^[1]

This study was of real emergency department patients (at a burn center with a hyperbaric chamber) being evaluated for possible CO (Carbon monOxide) poisoning. As far as being different from other available studies, that is not true.

Here is another study, which does not encourage faith in the RAD-57 readings.

A total of 36.4% of the patients transported during the study had SpCO documented. Of the 1,017 adults included in this group, 11 (1.1%) had an SpCO >15%.^[3]

1% of the patients included in the study had RAD-57 levels above 15%.

Of the 11 patients with a SpCO >15%, 10 were transported to a hospital for which the investigators had institutional review board (IRB) approval to review the patient’ s medical record. Of those 10, none had confirmatory venous carboxyhemoglobin levels. The two patients with an SpCO level of 21% did have a repeat SpCO documented at triage upon arrival to the emergency department. Their repeat levels were 8% and 2%. None of the 10 patients with levels >15% ultimately were diagnosed with and treated for carbon monoxide exposure or toxicity.^[3]

The medical records were available for 91% of that 1%.

None of the 10 patients with levels >15% ultimately were diagnosed with and treated for carbon monoxide exposure or toxicity.

But, none of them had blood drawn to check carboxyhemoglobin levels.

Were they misdiagnosed?

More important. Did the researchers let the hospitals know that they were studying the ability of the RAD-57 to identify carboxyhemoglobin?

From the study, the answer appears to be, No.

Do we know if the RAD-57 missed any patients with elevated carboxyhemoglobin?

No.

This study does not appear to have been designed to identify patients missed by the RAD-57.

Data obtained from other studies indicate that there are false positive results using the new technology, but the benefit of identifying true positive CO toxic patients outweighs the burden of false positive results.³ In this cohort, there were at least two patients who had discordant results when a repeat SpCO was obtained, further emphasizing that all positive results obtained with the non-invasive method should be confirmed with a blood carboxyhemoglobin level.^[3]

The RAD-57 readings were inconsistent. The patients with high readings were not diagnosed with, or treated for carbon monoxide toxicity. This does not support Dr. O’Reilly’s claims that the RAD-57 is accurate and just being unfairly evaluated.

The RAD-57 may have some role in identifying elevated carboxyhemoglobin levels, but so far nobody can tell what that role is. As I pointed out earlier –

Less than half of the patients with elevated COHb were correctly identified.

If we screen a fire fighter for COHb, then we need to keep that fire fighter out of the fire.

Should anyone ever use a low RAD-57 reading to justify returning a fire fighter to a fire?

No.

Masimo stands by its products’ performance and knows that when SpCO-enabled devices are used according to their directions for use, they provide accurate SpCO measurements that provide significant clinical utility, helping clinicians detect carbon monoxide poisoning in patients otherwise not suspected of having it and rule out carbon monoxide poisoning in patients with suspected carbon monoxide poisoning.^[1]

That advice from Dr. O’Reilly may encourage us to return fire fighters to an environment that has already made them toxic, but with the mistaken belief that they have carboxyhemoglobin levels of zero, when their carboxyhemoglobin is really very high.

Dr. O’Reilly’s advice is bad for Masimo investors.

Dr. O’Reilly’s advice is bad for patients.

Dr. O’Reilly’s advice misrepresents the research.

Find a way to make the RAD-57 reliable, then sell a lot of them. Right now, it isn’t reliable.

Right now, the RAD-57 is an accident waiting to happen.

-

Continued in How Not to Respond to Negative Research – Addendum.

-

-

Footnotes:

-

^[1] Performance of the Rad-57 pulse co-oximeter compared with standard laboratory carboxyhemoglobin measurement.
O’Reilly M.
Ann Emerg Med. 2010 Oct;56(4):442-4; author reply 444-5. No abstract available.
PMID: 20868919 [PubMed - indexed for MEDLINE]

Free Full Text of letter and author reply from Ann Emerg Med with links to Free Full Text PDF download

-

^[2] Performance of the RAD-57 pulse CO-oximeter compared with standard laboratory carboxyhemoglobin measurement.
Touger M, Birnbaum A, Wang J, Chou K, Pearson D, Bijur P.
Ann Emerg Med. 2010 Oct;56(4):382-8. Epub 2010 Jun 3.
PMID: 20605259 [PubMed - indexed for MEDLINE]

Free Full Text Article from Ann Emerg Med with links to Free Full Text PDF download

-

^[3] Non-invasive carboxyhemoglobin monitoring: screening emergency medical services patients for carbon monoxide exposure.
Nilson D, Partridge R, Suner S, Jay G.
Prehosp Disaster Med. 2010 May-Jun;25(3):253-6.
PMID: 20586019 [PubMed - indexed for MEDLINE]

.