Wearable tech has become an increasingly popular way to measure daily steps, heart rate, energy expenditure, and sleep. However popular they may be, it’s important to know if you’re actually spending your money on something that’s effective. In a recent study in the Journal of Strength and Conditioning Research, investigators took a look at the accuracy of eight trackers on energy expenditure and sleep time.
To determine the accuracy of each of the devices in energy expenditure, the researchers compared each of them to a known quantity: the SenseWear Armband. This armband is research grade wearable tech that measures acceleration in three directions, takes skin temperature, galvanic skin response, and more. Its validity has been determined in several studies in which it was compared against other scientific gold-standards like indirect calorimetry. This device was considered suitable for scientific research in free-living conditions, which was the goal of the study. The participants wore the SenseWear simultaneous with the monitor being tested.
For sleep, the participants used a self-report log to measure the length of sleep, although the SenseWear included a function to measure both the time asleep and the time laying down, (which were included in the study).
There were 95 participants in the study, and they each wore the SenseWear in addition to two or three of the other monitors as available. They wore each monitor for a couple days, collecting data for one full day and night each.
On average, the energy expenditure was almost universally underreported. Only the Polar Loop measured higher than the SenseWear Armband on average, and only by 50 calories. Every other device measured lower than the armband for calories burned by 400 to 500 calories (or more). The Polar Loop also had the lowest standard deviation, meaning it measured calories in a pretty tight band. This could suggest a more consistently accurate reading of energy expenditure as well, however, it is possible that this is by chance as well.
When it came to how long they slept, the results were closer on average, but with a greater degree of variation. It seems that wearables aren’t so great at determining sleep on a day-to-day basis, but over time will probably come pretty close to matching how well people report that that they sleep.
In terms of which devices did the best, the Polar Loop was the best at determining energy expenditure, with a standard error of 13%, followed by the Misfit Shine (15.2%) and the Fitbit Flex (15.5%). For sleep, the Garmin Vivofit (4%), Fitbit Flex (8.8%), and Jawbone UP 24 (10.2%) were the top three respectively. However, it is worth it to note that the three that did best on reporting sleep required the user to initiate sleep mode, which wasn’t automatic. It would appear that an automatic sleep detection lowers the accuracy of the device, the tradeoff being convenience.
Before you run out to buy the Garmin if you’re most interested in monitoring your own sleep, there’s a further consideration. Sleep was measured relative to a sleep log, and the devices without automatic detection performed best. This should come as no surprise, because it more or less means that the devices work as an electronic sleep log.
What’s most interesting about the sleep log is that it was closer to the SenseWear’s laying-down-detection feature than it was the SenseWear’s sleep detection feature, which indicated much less sleep than participants reported. If we were to assume for a moment that people are likely to report their sleep time as a function of their laying down time, and not, say, high quality sleep time, then we would have a pretty good explanation for the SenseWear’s low reported sleep time. And if that sleep time was more accurate than the self-reported data, it might be true that the average device overreports sleep. If all of that is true, and I suspect it is, then it’s the Polar Loop that wins yet again, followed by the Jawbone Up 24, and Fitbit Charge HR. Please note, that’s speculative, and wasn’t reported by the researchers, but seems apparent in their data.
When compared to the SenseWear, the Polar Loop is the clear winner for overall accuracy and ease of use and would be a good buy, with a strong showing from Fitbit as well. If you’re interested in a more user-controlled sleep log type feature, the Garmin Vivofit should be your choice.
