Let's dive into the world of wearables, smartwatches, fitness bands, movement trackers, or whatever you want to call them to find out how much or little you can actually rely on them.
The biggest takeaway was that none of the seven devices measured energy expenditure accurately at all. Even the most accurate device was off by an average of 27 percent and the least accurate was off by 93 percent. That is basically me throwing darts at some graph paper that I stapled to the wall—and I am not a good dart player!
The biggest takeaway was that none of the seven devices measured energy expenditure accurately at all.
The researchers said that manufacturers may or may not test the accuracy of these devices thoroughly and it’s hard for consumers to know how accurate such information is or the process that the manufacturers used in testing the devices.
The heart rate measurements performed far better than expected, based on the previous versions of the devices but again, what was truly surprising was just how crappy the energy expenditure measures were.
The take-home message appears to be that a user can pretty much rely on a fitness tracker’s heart rate measurements but basing the number of doughnuts you can eat on how many calories your watch says that you burned is really not a great idea… any way you dip it.
10 Trackers at Once
Back in May 2016, a reporter for the Big Crunch did a fun experiment (fun for us nerds that is). He wore 10 different trackers at once and took them through their paces. He did this in response to people actually suing Fitbit, saying that its devices didn’t track the data accurately enough.
The reporter went to an electronics store and purchased a bunch of devices like Fitbit, Garmin, Polar, Misfit, Jawbone, and Withings. He also borrowed another Fitbit and two Apple Watches from his very trusting friends.
He then ran the devices through three specific tests: step counting, heart rate measurement and total distance travelled.
Now he was the first to admit that this wouldn’t qualify as a scientific study but he still tried to be more thorough than most of us would - I have to give him that. If it had been me, I would have likely given up the minute the clerk at Best Buy showed me the bill.
The results were, as he put it: depending on your perspective, either quite variable — or pretty close... whatever that means.
First Test - Step Counting
He did two different tests of step counting. In the first, he wore all the devices for a couple hours, doing a variety of tasks. The step totals varied widely by more than 20 percent. That's pretty significant when you consider that people often have a goal of getting to 10,000 steps in per day. With the chance of a 20 percent inaccuracy rate, one device might show 10,000 steps while another would show around 8,000. Not cool, man. Not cool.
In the second test he counted 500 steps out loud (I can’t help but picture someone following a treasure map). In this test, he still saw inaccuracies ranging between 446 and 513.
The most accurate device was the Fitbit chargeHR, which showed 505 steps. Interestingly another Fitbit (a cheaper model) showed only 486 steps. So, you get what you pay for, even within the same company.