Your phone knows how you feel

[MUSIC PLAYING] NOAH LEAVITT: Coming up on Harvard Chan– This Week in Health, your phone knows how you feel.

JP ONNELA: We actually spend a lot of our waking hours interacting with this device and interacting through this device. Every one of these interactions in isolation is just a small digital breadcrumb. But if you aggregate them together, you really get to see a pretty detailed picture of a person.

NOAH LEAVITT: How researchers are mining data from our cell phones to improve everything from our mental health to recovery from surgery.

[MUSIC PLAYING]

Hello, and welcome to Harvard Chan– This Week in Health. I’m Noah Leavitt. It is Thursday, October 18, 2018. And this week, we’re digging back into the archives to share a story that we first broadcast two years ago.

My co-host Amie Montemurro and I spoke to JP Onnela, a researcher in the Department of Biostatistics here at the school, about his work to mine smartphone data to improve health. Take a listen.

Amie, I’m going to ask you and our listeners to envision a scenario.

AMIE MONTEMURRO: OK.

NOAH LEAVITT: You’ve just had surgery to remove a brain tumor. You’ve recovered in the hospital, and then you head home. And then you get a text from a friend asking how you’re feeling.

[PHONE CHIMES]

SPEAKER 1: Hey, how are you feeling?

SPEAKER 2: OK. Some pain.

NOAH LEAVITT: So maybe that’s a few days after you return home. And then a couple of weeks later, another text.

[PHONE CHIMES]

SPEAKER 1: Just checking in again. How are things going today?

SPEAKER 2: Much better. I’ve been able to start cooking again.

NOAH LEAVITT: And then a few weeks beyond that, you get one more text.

SPEAKER 3: I heard about your surgery. How are you feeling?

SPEAKER 2: I’m doing well. There’s very little pain. Today, I was able to get outside and walk around the neighborhood.

NOAH LEAVITT: These conversations and the progression can actually tell your doctor a lot about your condition.

JP ONNELA: So the simple act of just typing a text message actually requires a ton of things to come together.

NOAH LEAVITT: That’s JP Onnela, an assistant professor of biostatistics at the Harvard Chan School.

AMIE MONTEMURRO: He runs a lab that collects smartphone data and extracts insights about human health and behavior.

NOAH LEAVITT: So back to that brain tumor example and why text messaging can be so insightful–

JP ONNELA: You need to have something to say. You need to be interested in saying something. You need to be able to see. You need to be able to type.

So actually looking at things like how long does it take for a person to respond to a text message, how long are these messages, in terms of number of characters, how frequent are they, and so on, I think all of these will provide very interesting and valuable information about a person’s social and behavioral state.

So initially, you might assume that the person, if they’re just recovering from major surgery, they might only send out very short messages– “yes,” “no,” “doing OK,” and so on. But as time goes forward, if they start to recover better, their text messages might become more frequent. They might become longer, and so on.

NOAH LEAVITT: All of this is an example of something called “digital phenotyping,” and that’s what we’ll be taking an in-depth look at today.

AMIE MONTEMURRO: So what is that? Well, the traditional definition of a phenotype is that it’s a collection of an organism’s many traits, such as its anatomy or hormone levels or its behavior.

NOAH LEAVITT: So digital phenotype involves mining data from electronic devices, such as smartphones, and using statistical tools to extract insights about a person’s behavior. And this is significant, says Onnela, because behavior has typically been very difficult to study because it’s dynamic, it’s always changing, and it’s context-dependent, which means our behavior changes depending on the situation. Again, with that brain tumor example, your behavior will be very different after the surgery compared to before it.

AMIE MONTEMURRO: And this is why smartphones are such a powerful tool. Everything we do– using our GPS for directions, making a phone call, texting a friend– they all leave behind these digital breadcrumbs. On their own, they may not tell us much. But taken together on a larger scale–

JP ONNELA: If you aggregate them together for a day or a week or a month, you really get to see a pretty detailed picture of a person.

AMIE MONTEMURRO: And those are just the active things we’re doing on our phone. But it’s the passive data that can really tell a story.

NOAH LEAVITT: And this is why smartphones are more powerful than something like a Fitbit or another piece of wearable technology. Onnela says that after a few months of usage, people use those devices less frequently, which means they’re collecting less data. But that’s not a problem with phones, which people are always using.

AMIE MONTEMURRO: And our phones are always collecting information, even if we don’t realize it. For example, doctors can have someone with depression complete a survey about their behavior. But looking at phone data can give remarkable insight into a person’s sleep habits.

JP ONNELA: So just to give you an example, in a typical hour of a day, your smartphone screen might be on 10 times or 50 times. But during night when you’re sleeping, it’s almost never on. The only time it would be on is if somebody sends you a message in the middle of the night.

[PHONE VIBRATES]

So just by looking at the frequency of screen on/off events alone, we can get a pretty good estimate of how long you’re sleeping and whether you woke up in the middle of the night. Now, if we combine this with other layers of data, we can get even more precise estimates.

[MUSIC PLAYING]

NOAH LEAVITT: But this smartphone data is really only one part of the picture.

AMIE MONTEMURRO: Where the real breakthroughs happen is when researchers like Onnela can take that smartphone data and marry it to clinical information, things like blood work or a lab test or a surgical outcome.

NOAH LEAVITT: So if we return to our brain tumor example, we can see this in action. Onnela and his team are working with Tim Smith, a neurosurgeon at Brigham and Women’s Hospital. After a patient has surgery, they’ll complete surveys with Smith’s team explaining how they’re feeling, and they’ll also follow-up in-person visits.

AMIE MONTEMURRO: But what about all that time at home? That’s where Onnela can use smartphone data to fill in the gaps. We gave the text message example, which measures cognitive functioning. But there are other things to look at, as well.

JP ONNELA: So in this particular study, for example, one of the key sensors we are using is GPS. We would expect that the person, right after surgery, they will be at the hospital. Once they’re released from the hospital, they will likely stay in their home for several days. They will not be leaving the house.

But we can learn things like, how long do they spend in their house? When they leave their house, how long are they gone for? How many different locations do they visit? How far do they venture out from their house?

NOAH LEAVITT: So there’s all this data out there. So how do scientists like Onnela actually collect it?

AMIE MONTEMURRO: Well, they’ve gone beyond creating a simple app, and they’ve developed an entire research platform. It’s called “Beiwe,” named after the Nordic goddess of sunlight and mental health.

NOAH LEAVITT: And it’s particularly powerful because Onnela and his team have designed it so it can be customizable for each study. So let’s say a surgeon like Tim Smith from Brigham and Women’s wanted to measure GPS data, text message length, and phone calls. Each patient would then download a customized version of the app that would collect only that information. Onnela explains how that works on the patient side.

JP ONNELA: And so what happens is that when we enroll patients in our studies, patients get a unique Beiwe ID number, which is just, let’s say, “ABC123,” or something like that. And they get a temporary password. And with these two pieces of information, they can go to iTunes or iStore. They can download the app, and they can enter their details into the app.

And based on the Beiwe user ID, the system automatically gives them the right version of the app. So basically, that connects them to the right study. So they get the right questions, they get the right kind of passive data collection, and so on.

And Beiwe uses what’s called “store and forward architecture.” So the idea is that the data are initially stored on the device. And next time when a Wi-Fi network becomes available, then the data are uploaded to the study server and erased from the device.

And the data are always encrypted. And this is critical, especially for a biomedical context. To preserve patient privacy, we first hash all identifiers. And on top of that, we have a second layer, which is very strong encryption.

[MUSIC PLAYING]

NOAH LEAVITT: Onnela said it’s actually been fairly easy to recruit patients for these studies, especially once they know what data will be collected and how it will be used.

AMIE MONTEMURRO: And so you might be wondering, when will I be able to download an app like this and use it?

NOAH LEAVITT: It turns out that’s still pretty far away. But Onnela says that in their current studies, they’re working on ways to integrate reports generated from smartphone data into one-on-one patient care.

JP ONNELA: But the idea is that perhaps once a week or once every two weeks, we can take the data that we have collected from a patient. And we can produce, say, a one-page report that tells something about this patient’s mobility, their appetite, their mood– their social connectivity. And then the doctor can look at that report and discuss the past week or past two weeks with the support of having that extra document there.

NOAH LEAVITT: Onnela’s goal is to make this Beiwe platform open-source by 2017, which means that other researchers can use it for their own studies.

AMIE MONTEMURRO: And widespread adoption opens the doors to some incredible research opportunities, says Onnela. At the heart of it is that passive data we talked about earlier. Because patients don’t technically have to do anything, researchers can do incredibly long follow-up times, from six months to a year or two years. All the patient needs to do is to keep the app installed on their phone.

NOAH LEAVITT: The technology also offers doctors and researchers an opportunity to get a picture of a patient’s life before they became sick or injured.

JP ONNELA: So typically, we only get to have the post-data. So I go to see my doctor after I’ve hit my knee or my back is hurting or my head is hurting. But typically, we have no data from what happened before.

And the nice thing about the approach of digital phenotyping is that we could potentially instrument a large cohort– say, 10,000 people or more– and then also have the pre-data. So we’d have the data from a person before something happens to them, before they get into a car accident or before they hit their knee or whatever the condition is. And I think this is potentially very valuable.

NOAH LEAVITT: Another group that could benefit– scientists studying the environment and exposure to pollution.

JP ONNELA: So for example, you could imagine a study that follows people using their smartphone GPS to track their mobility patterns. So we can learn where these people are located physically at any point in the day. Now, if we have some kind of estimates of exposure to pollution at those locations, we can essentially calculate, reverse-engineer, a person’s specific exposure to, say, whatever the pollution you’re interested in or particle you’re interested in. And that’s essentially your custom-made daily exposure to the environment.

NOAH LEAVITT: As we talked, Onnela emphasized the importance of collecting raw data from devices.

AMIE MONTEMURRO: Not only does that make the research he mentioned possible, but it will also make it easier for scientists in the future to replicate studies.

JP ONNELA: And we know from a fairly recent study that about 2/3 of medical studies are inconsistent when retested, and only about 6% of studies are completely reproducible. The reason for collecting raw data is that we or somebody else can later reanalyze the data using whatever methodology they want to use.

But also, what’s important is that if we relied on a proprietary summary statistic of someone’s mobility, for example, what if that summary statistic changes in the middle of a study? So for example, if Google or iPhone or Apple, they decide to change some of the summary statistics that are proprietary, we would never find out. One day, it would just seem that 200,000 people in the United States are now walking 20% more. And this is obviously a problem.

And actually, one of the features of our platform that we’re building in now is one-click replication. So we were inspired by Amazon’s one-click Purchase button. The idea here is that the platform stores all the key settings of a study.

So if you wanted to replicate my study next year, you would have to only find my study in the list of studies, click that, and you would essentially immediately be able to run your own study which uses the exact same settings for data collection as my study. So the idea here is to make the system as transparent as possible and to make it as easy as possible to replicate existing studies.

[MUSIC PLAYING]

NOAH LEAVITT: Onnela wants to make it clear that there’s no financial gain here. No one is buying or selling this data. And the goal, he says, is to simply better understand behavior and hopefully develop better treatments or interventions based on that information.

AMIE MONTEMURRO: Before we wrapped up our conversation with Onnela, we had to ask him, does mining all the smartphone data make him think any differently about the way we all use our phones?

JP ONNELA: I think I have become a little bit more observant about the ways people use their phones. And sometimes, I see people walking around, even in Longwood, and the people seem to be glued to their smartphones. And I’m thinking, wow, this is going to be great data for us.

But at the same time, I’m worried about these people walking into buildings or being hit by someone. So it’s almost gotten to a point where we’ve become so dependent on these devices that it does make you think about the future. But it certainly is an opportunity for us to get very rich behavioral data.

AMIE MONTEMURRO: So keep using those devices, but maybe avoid texting and walking.

NOAH LEAVITT: Always very sound advice. And if you want to learn more about Onnela’s work, you can visit our website, hsph.me/thisweekinhealth, and a reminder that you can always find our podcast on iTunes, SoundCloud, Stitcher, and Spotify.

[MUSIC PLAYING]