Saturday, February 10, 2018

Accuracy of Activity Trackers

Millions of people wear some kind of wristband activity tracker and use the device to monitor their own exercise and health, often sharing the data with their physician. But is the data accurate?

A Stanford inquiry evaluated the accuracy of seven devices: Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2. Six out of the seven devices measured heart rate within 5 percent. In contrast, none of the seven devices measured energy expenditure accurately. Even the most accurate device was off by an average of 27 percent. And the least accurate was off by 93 percent.

Sixty volunteers, 31 women and 29 men, wore the seven devices while walking or running on treadmills or using stationary bicycles. Each volunteer’s heart was measured with a medical-grade electrocardiograph. Metabolic rate was estimated with an instrument for measuring the oxygen and carbon dioxide in breath — a good proxy for metabolism and energy expenditure. Results from the wearable devices were then compared to the measurements from the two “gold standard” instruments.

“The heart rate measurements performed far better than we expected,” said Ashley, “but the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me.”

Neither Ashley nor Shcherbina could be sure why energy-expenditure measures were so far off. Each device uses its own proprietary algorithm for calculating energy expenditure and it’s likely the algorithms are making assumptions that don’t fit individuals very well. “All we can do is see how the devices perform against the gold-standard clinical measures,” she said. “My take on this is that it’s very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height and weight, etc.” Heart rate is measured directly, whereas energy expenditure must be measured indirectly through proxy calculations.

Have you noticed similar inconsistencies in your own activity tracker? If wearing an activity tracker motivates you to move and exercise more often, then keep using it. However, try not to base your calorie consumption on the caloric expenditure measurement.