Tuesday, 23 June 2015

Sensor (1-3) Performance

It was time to change sensors again, starting my fourth sensor today. I had some accuracy issues with my 3rd sensor, so I decided to have a closer look at the data. From the exported .csv file, I wrote a small piece of software that find pairs of blood glucose and Libre scanned values. Normally I scan with the Libre just before or after I take my BG, so extracting the data was not a big issue. Ideally, the BG and Libre values should match almost perfectly. The performance of these kind of sensors are usually measured by the Mean-Absolute-Relative-Difference (MARD). Abbott claims that the Freestyle Libre have a MARD of 11.8% after the 3rd day. The lower the value, the better.

The BG values can be seen in the x-axis, while the y-axis shows the Libre values. For a perfect sensor and BG combination, all the points should be on the y=x line. My first sensor results shown in red, second sensor in green and third in blue.

The following plot shows the values for all 14 days for each sensor. (click on it for a larger view).
Clearly sensor 3 seems to always measure a bit low. Values below the line indicates that the Libre measured a value lower than the BG value, the inverse applies to values above the line. Sensor 2 seems to be measuring little too low in the [2,8] range, but too high in the [8,16] range. Sensor 1 measured a little too high on most cases. Abbott should really add the option to calibrate. The device already have a BG meter build-in, so why not add an optional calibration / offset process ???

The following plot shows the values for 13 days, skipping day 1 for each sensor.
You can see that most of the values from day 1 are outliers when comparing the two plots.

The MARD values can be seen in the following table:
Day 1-14 Day 2-14 Day 3-14 Day 4-14 Day 1-7 Day 8-14
Sensor 1 14.7% 15.0% 15.3% 16.2% 14.5% 15.0%
Sensor 2 12.1% 12.0% 12.6% 12.6% 13.2% 10.9%
Sensor 3 18.4% 14.7% 13.3% 12.1% 20.26% 11.8%
This shows that the accuracy do improve over time. Although sensor 3 was looking very bad in the early days, the accuracy quickly improved and overall performed better than sensor 1.

The mean error can be seen in the table below:
Day 1-7 Day 8-14 Day 1-14 Day 2-7
Sensor 1 -0.20 -0.68 -0.36 -0.30
Sensor 2 -0.07 0.15 0.03 0.30
Sensor 3 1.39 0.58 1.21 1.05
The mean error table clearly shows that sensor 1 was giving values slightly high and sensor 3 gave very low values, a simple calibration process could have improved the accuracy a lot.

Let's assume a simple calibration procedure: After 7 days, take the mean error (of day 2-7) and use that as a simple calibration offset. If we use the mean error as the offset (100%) or half of the mean error as the offset (50%), we get the following MARD from day 8-14: (0% no calibration)
100% 50% 0%
Sensor 1 12.4% 13.4% 15.0%
Sensor 2 9.17% 10.0% 10.9%
Sensor 3 12.7% 9.68% 11.8%

The system can definitely get some benefit from a calibration process.

No comments:

Post a Comment