Today I dove into my activity tracker data; specifically, the sleep score that Oura calculates daily. It ended up being the perfect scenario to explore a much bigger question I am begging everyone to ask more often:
Does this problem need AI or a simple math formula?
Hear me out here!
Activity tracking data
Now, I'm a really big fan of tracking my data and using gamification to improve my health. I still use my fitbit and garmin depending on activity (i.e. gym or hiking), however, for day-to-day use, I love the Oura ring. It's more subtle and elegant and it doesn't annoy me like sleeping with a watch on does.
That being said, sleep tracking is an important element of why I use my Oura, and therefore worth looking into a bit more.
If you're not familiar with this metric, you can read about the sleep score from the Oura blog here.
The magical Sleep Score
One of the big negatives about Oura is that a lot of the insights and breakdowns are hidden behind a subscription paywall - and the 'free' version only provides you with the sleep score itself without any further explanation. This is quite a departure from the fitbit and garmin models that provide you with a comprehensive dashboard once you've bought the device.
So what is this magical sleep score insight we go get then? And is it worth a monthly fee to peek behind the veil at the unique insights?
The Hypothesis
As a skeptic and data scientist, I had to take it upon myself to look into this question. Rather than approach it as a fancy AI insight I can throw chatGPT at (which seems to be the go-to first step these days) I will be going the Occam's razor way.
My intuition/assumption about it is pretty simple -
More deep sleep and lower heart rate correlate with better scores.
Can it really be that basic?
Let's take a look in a quick data science experiment.
Oura Developer API
First up I found the Oura developer API and got a quick dump of my activity/sleep data.
As Oura tends to have the most restrictive native way to look at your data (vs fitbit or garmin) this is already a great find/idea if you want some more insights!
def get_data(type):
url = 'https://api.ouraring.com/v2/usercollection/' + type
params={
'start_date': '2021-11-01',
'end_date': '2025-01-01'
}
headers = {
'Authorization': 'Bearer ' + auth_token
}
response = requests.request('GET', url, headers=headers, params=params)
return response.json()["data"]
data = get_data("sleep")
with open('oura_data_sleep.json', 'w', encoding='utf-8') as f:
json.dump(data, f, ensure_ascii=False, indent=4)
Elasticsearch Index
Step 2: we need a few extra lines to add the new data into an Elasticsearch index so we can look through it easily.
You can check out some more starter examples for Elastic & Python here.
This was also super easy - as it tends to be with json files - so we didn't need any additional mapping or data processing steps.
client = Elasticsearch(
cloud_id=ELASTIC_CLOUD_ID, # cloud id can be found under deployment management
api_key=ELASTIC_API_KEY, # your username and password for connecting to elastic, found under Deplouments - Security
)
index_name = 'oura-history-sleep'
# Create the Elasticsearch index with the specified name (delete if already existing)
if client.indices.exists(index=index_name):
client.indices.delete(index=index_name)
client.indices.create(index=index_name)
with open("oura_data_sleep.json", "r") as f:
json_data = json.load(f)
documents = []
for doc in json_data:
documents.append(doc)
load = helpers.bulk(client, documents, index=index_name)
Now comes our tiny data science experiment for our hypothesis.
And I do mean tiny. Remember - we're going for the simplest possible explanation first.
I gathered the days with the highest sleep scores with a simple sort on the score field.
index_name = 'oura-history-sleep'
response = client.search(index = index_name, sort="readiness.score:desc")
for hit in response["hits"]["hits"]:
print("Day: {day} and sleeping score: {score}".format(day=hit['_source']['day'], score= hit['_source']['readiness']['score']))
Looking a bit deeper at these days, and at the fields I already suspect had a pretty high influence on the scores, we notice some consistent values:
Finally, the experiment.
Can it be as simple as just making a basic formula around sleeping times and heart rate?
I build a query with Elasticsearch filters looking at deep sleep over 1.5 hours, heart rate under 60, and sorting the hits by highest REM time:
query = {
"range" : {
"deep_sleep_duration" : {
"gte" : 1.5*3600
}
},
"range" : {
"average_heart_rate":{
"lte" : 60
}
}
}
response = client.search(index = index_name, query=query, sort="rem_sleep_duration:desc")
Now we can look at what days this gets back and check what the sleep scores for those days were. The validation step if you will.
Surprise, surprise. It's pretty much the same dates we got in the initial query. It's not perfect - just like our formula or filter selection is far from 100% accurate - but that's exactly the point.
This confirms that the intuition/hypothesis was pretty much spot on and I can "predict" my sleep score with very simple math.
Here are a few Kibana visualizations of a larger subset of my data to further illustrate this connection:
Why does this matter?
In a world of AI buzzwords and the race to the most over-the-top model we can get - you can look at a simple "sleep score" and believe it's a fancy, groundbreaking AI insight that's worth a monthly fee to unlock.
However, more often than not - it's either a reasonable & intuitive formula or maybe even a basic regression.
Simpler. More accurate. Cheaper to compute.
And that's something we need to keep reminding ourselves of. This is why I believe data scientist/analyst jobs will continue to be safe, and why learning the basics of intuitive modeling and machine learning is much more impactful than having unrestricted access to an LLM.
Because as undoubtedly amazing as the advanced tech we have access to today is, it's even more important to understand when you don't need to use it at all.
Top comments (0)