We all want accurate glucose meters that we can trust to tell us correctly what our blood sugars are at any given moment.
But even after meters are cleared by the FDA and put on the market for sale, we’re seeing those devices aren’t as accurate as they should be.
A new error grid to monitor accuracy for already-approved meters could change that.
Remember our report in May about new efforts to create a post-market surveillance program to assess meters, including a new error grid being developed by the Diabetes Technology Society? Well, it’s finally out there for the public to see!
Unveiled at the American Diabetes Association’s 74th Scientific Sessions earlier this month, their new Surveillance Error Grid (SEG) has the potential to give doctors and people with diabetes the ability to keep track of how these meters perform in the years after initial review, as they continue to be sold on the market.
This new assessment tool replaces two previous error grids created in the 80s and 90s, and will be the first developed for the 21st century Diabetes Community based on modern diabetes science and knowledge about blood sugars.
“This is not for clearance (of glucose meters) by the FDA, but to monitor clinical performance of glucose meters once they’re on the market,” confirmed Diabetes Technology Society founder Dr. David Klonoff of UCSF, who presented to a standing-room only crowd of 200+ people during his recent ADA conference presentation of the grid. “You are going after that sweet spot, like a tennis racket where you want to have the best contact with the ball.”
There was a paper published about this on June 13, going into the nitty-gritty of how it was created and what each part of the grid means. But for our purposes, here’s the run-down:
You can see on the color-coded chart what Klonoff means by that special spot extending along the “green zone” line that represents zero risk — or basically, where the numbers meet up showing the least amount of inaccuracy. The further the numbers are from each other, the higher the risk of high or low blood sugar inaccuracies (see, that’s the brown color on the grid). Of course, the BG numbers translate to a wide range of actual risk levels that can all be plotted on the grid, for those interested.
In creating this grid, Klonoff says his group surveyed 206 clinicians about how they would react to blood meter inaccuracies when it comes to patient care; they were asked a series of questions about how they’d respond, relating to hypoglycemia and hyperglycemia, and how those BG levels would correspond to one of five types of actions: emergency treatment for low blood sugar, taking oral glucose, no action needed, take insulin, or emergency treatment for high BG. From there, the participating doctors were asked to assign a ‘risk level’ to the patient for each case. Of course, very high and very low blood sugar inaccuracies represent the highest risk.
Based on the risk score chart they created, the data on meter models can be used in multiple ways — it could be given a pass-fail grade used to determine if it’s worth a doctor recommending or patient using, or manufacturers or regulators could use it to chart out and compare the risk of potential meter inaccuracies when reviewing what’s on the market.
Interestingly, the DTS found in surveying doctors that there wasn’t any clinical reason to create different error grids for PWDs with type 1 or type 2, those who use insulin or don’t use insulin, whether someone uses Multiple Daily Injections (MDI) or an insulin pump and CGM. The only possible reason for separate error grids might be when assessing meters being sold in the U.S. versus internationally, based on the differing standards used overseas.
For the most part, the answer is simple: Accuracy is important and the degree to which the clinical risk is rated is the same for most PWDs.
This SEG is the first to actually be created since the Diabetes Control and Complications Trial (DCCT) came out in 93, which means the past error grids didn’t take into consideration contemporary knowledge about the risk of hyperglycemia and how tighter glucose levels impact A1C and the future risk of diabetes complications. The widely-used Clarke Error Grid was created in 1987 and emphasized treatment based on science at the time, while the Parkes Error Grid was first developed in 1994 and focused on outcomes. Still, that was two and three decades ago– and a lot has changed since then, from how we treat our diabetes, to the faster analog insulins and delivery methods like insulin pumps that have become more mainstream.
Why do error grids matter? Just think about those materials we get from manufacturers whenever a new meter hits the market… they cite data on one of the two grids, when reporting on how accurate these devices are for patients. That’s the role of these error grids.
Yet historically, regulators haven’t relied much on these kind of error grids to assess how accurate these devices are for sale. As we’re told by Dr. Courtney Lias, director of the Division of Chemistry and Toxicology Devices at the FDA:
“The error grids were not developed for that purpose, and do not enable a robust assessment of the data sets. We use alternate analytical methods to review the data we get for glucose meters and CGMs.”
Lias went on to tell us: “Surveillance is done to monitor the performance of glucose monitors in use by patients and healthcare providers. Adverse event data are collected and investigations are performed when devices malfunction and/or when patients or device users are injured. Post-market meter problems are varied and can include, for example, meter systems that provide erroneous results (e.g., due to test strip manufacturing errors, design deficiencies, etc.). Sometimes, the root cause of device problems is not definitively identified.
“When these post-market issues occur, the FDA and manufacturers must work together to assess the potential health impact of the problem. For meter systems, the health impact generally relates to the likelihood that the meter will generate erroneous results, and that patients or healthcare providers will act on those results. Therefore, the frequency and magnitude of the error is evaluated to try to assess the overall risks to the public. A new error grid that can help to indicate the clinical risk of device errors can facilitate decision-making for manufacturers and the FDA to allow for efficient and appropriate post-market actions.”
That last part of what Lias tells us coincides with what Klonoff said during his presentation: That this new SEG has the potential to help regulators in their decision-making, and if that happens this error grid approach could be mirrored for other types of medical devices beyond those focused on diabetes self-management.
Creating this new error grid was no small feat, and it wasn’t limited to a handful of experts; rather, it had broad international participation. About three dozen players from the regulatory, academic, clinician, non-profit and industry came together to assist in creating this. And it’s a just part of the larger post-market monitoring program that the DTS first announced at the start of May. Just before the ADA began and a few days before Klonoff made his presentation on the new error grid, the DTS issued a news release stating that the official Steering Committee for that program has been assembled, and the group will meet in July for the first time in Washington D.C.
Unfortunately, we haven’t been able to obtain details from Klonoff or the DTS about who exactly is part of the new Steering Committee… and honestly we’re unsure why the list of names would not be made public? We as patient advocates have a lot of question, such as: is one meter manufacturer over-represented while others may be missing from the table? Are patient advocacy orgs represented, including the grassroots StripSafely initiative that’s turned heads and sparked action on this whole topic of glucose meter and test strip accuracy?
All we were able to get from the DTS at this time was: “A Steering Committee has been formed consisting of world-class experts in blood glucose monitoring, diabetes, and laboratory methods from academia, medical practice, government, industry, and medical organizations. Many of them have experience in establishing surveillance programs and testing verification programs for other medical tests. Patient advocacy group(s) will be represented in a future Advisory Board that will be established soon. A protocol to assess BGM System performance will be developed and then product testing will begin.”
We look forward to hearing more about how this error grid will be used, and how it will fit into the bigger picture of assuring accurate devices for people with diabetes.