Can AI be used to help patients with heart failure?
Each year cardiovascular disease (CVD) causes 3.9 million deaths in Europe and over 1.8 million deaths in the European Union (EU). CVD accounts for 45% of all deaths in Europe and 37% of all deaths in the EU. In 2015, almost 49 million people were living with CVD in the EU.(13)
Adherence is considered to be key to success of the treatment plan. (10, 16, 17)
In this essay, we will survey alternatives for improving adherence in heart failure and CVD in general.
It is notable that at some point, I think around 2014, the literature started using the term adherence instead of compliance for patients who consistently stick with a treatment plan. I believe that this is related in some way to PC (politically-correctness) that is prevalent in the US and Europe – where adherence is more about what the patient does when well-informed and educated as opposed to compliance to strict guidelines from a physician. We will use the term adherence in this article.
Patient adherence approaches
There seem to be 3 main approaches today; each has a different business model.
1 – Institutional approach that patient health is a physician-patient partnership.(12)
2 – A more tech-oriented approach that patient health is about optimizing pill consumption. This initially seemed strange to me since treating congestive heart failure is not just about medication but also related to diet and physical exercise not to mention support of family and care-givers. One of the pill-adherence companies is an Israeli company called Medisafe who have been around for almost 10 years and combine a mobile app, pillbox sensor and patient-engagement. The company collaborates with pharma where there is a direct ROI story – the more pills patients take, the more money the pharma generates. On the strength of collaboration with pharma, Medisafe raised $14.5M in 2017 – so they seem to be getting traction.
Medisafe will be focusing on three aspects of its platform: improving medication adherence, increasing patient engagement, and generating data-driven insights about patient behavior. The new personalization tool highlights all three.
3 – A more human-oriented approach to adherence is remote coaching. There are a number of companies doing this and Livongo is getting a lot more investor traction than Medisafe. The coaching business model is clearer and more direct than the pill model and is based on a simple notion that maintaining healthy employees is good for business, so companies are willing to pay for Livongo subscriptions for their employees to keep them healthy and at work and perhaps reduce insurance premiums.
Livongo gets $105 million, signs deal with insurer Cambia Health Solutions
April 11, 2018
Digital chronic disease management company Livongo has raised $105 million in new Series E funding, mostly from existing investors. The company also announced a partnership with nonprofit health insurance company Cambia Health Solutions. General Catalyst and Swedish investment company Kinnevik led the round, with additional participation from existing investors DFJ, Kleiner Perkins Caufield…
Why the institutional approach is losing ground to coaching and AI-based adherence
Existing strategies of health-care providers for improving patient adherence to treatment rely on data collection and tracking in order to improve physician-patient interactions so that the physician can monitor the patient for side-effects, improvement or degradation in condition.
In spite of important advances in heart failure therapy, derived from better physiopathological understanding, hospital readmissions rate continued to increase(1) which suggests that the strategy of improving patient adherence via education and physician follow-up does not work.
The data is telling us that education does not work well for protocol compliance (since patients quickly forget or do not believe in the efficacy of treatment and/or relevance of the treatment to their condition) and physician follow-up does not work because the vast majority of physicians do not have the time to follow-up on their patients – only on the most critical and high-risk and even then, physician follow-up is iffy.
Big noisy data is not a good way to train compliance models
Another barrier to the institutional approach is that big health provider electronic health records are often cited but rarely used effectively to train AI compliance models. The data in EHR systems is poorly structured, noisy and generally unreliable considering that physicians have 7′ to talk to patients, type on their keyboard and fill out scrips.(15)
Use of AI in a mobile agent – a simple example of how RL might improve adherence.
A mobile agent may communicate directly with patients and suggest actions using reinforcement learning that maximizes a reward of better health in order to achieve a goal of adherence. (9)
The mobile agent reinforces its knowledge in the course of user interaction.(2,3,4,5,6,7,8)
In reinforcement learning, an agent who is both a learner and decision maker interacts with its environment, the patients. In the case of patient adherence, the interaction is a continuing task – where the agent proposes actions to the patient and evaluates a continuous reward function using discounted returns.(14)
If its that simple, why isn’t everyone doing it?
There are I think, 4 challenges to using AI-based mobile agents to improve patient adherence
1. What is the reward function? Are we maximizing pills or patient health?
2. What about usability? Is the mobile agent really usable by older, sick people?
3. What about accessibility? Do older sick people have the equipment to use the app?
4. What is the right computational tool? Perhaps not AI.
A very tricky and non-trivial question in AI-based adherence is “What is the reward function?”. Is it pill-taking (the approach that Medisafe and others have taken) or is it a more analytical and measurable clinical metric such as EF – Ejection fraction.
The left ventricle is the heart’s main pumping chamber that pumps oxygenated blood through the ascending (upward) aorta to the rest of the body, so ejection fraction is usually measured only in the left ventricle (LV). An LV ejection fraction of 55 percent or higher is considered normal.
The excellent review article Deep Reinforcement Learning Doesn’t work yet – talks to the depth of problems with RL including proper choice of reward.
Another tricky and non-trivial problem that is usually neglected is accessibility.
The mobile agent is usually implemented in a smartphone mobile app on IOS and Android. Perhaps there is logic in a much simpler app-less/server-side solution that communicates with patients using text messaging and a dumb phone that costs $10 in order to make the service accessible to people with limited means and connectivity.
A third constraint is usability.
Much work needs to be done on the UX (user experience) in order to enable older people to interact effectively with mobile adherence agents since the majority of CVD patients are over 60. Having said that – this problem will go away by itself over time as the young people of today become the older people of tomorrow – confident and fluent in digital media.
And finally – perhaps deep RL is not the right computational tool for adherence.
Perhaps we should consider Model predictive control (MPC) as a simpler and easier to implement alternative to AI. Model predictive control is an advanced method of process control that is used to control a process while satisfying a set of constraints. It has been in use in the process industries in chemical plants and oil refineries since the 1980s. Certainly no reason not to consider it as an alternative to AI.
The social value of AI
But really the biggest concern is the question of what problem are we trying to solve.
Is patient adherence a process control-optimization problem from a patient perspective in order to reduce adverse events and improve well-being?
Or is patient adherence a revenue-optimization problem from a pharma perspective in order to grow sales?
Are we trying to maximize pill-taking so that the pharma can keep sales high or do we want to minimize medication and medical interventions where appropriate and rely more on alternative medicine and physical fitness strategies to keep people healthy?
This is why pill-taking is such a cool benchmark (besides being great for pharma business). Not only is it easy to get lots of samples, the goal in every interaction with a patient is to maximize pill-taking, so you never have to worry about defining your reward, and you know everyone else has the same reward function.
I believe that we need to balance dosing, physical and social activity based on risk-assessment of the patient’s clinical state and enable an AI agent to take into account the risk level of a patient and modulate reinforcement for dosing, physical or social activity accordingly. In this respect, training models from coaching data and using evidence-based medicine for heart-failure derived from from clinical trial data may be a valuable approach.
An AI-based adherence agent should be able to free up physicians from low-risk patients and time-consuming office visits to focus on treatment of high-risk patients. AI-based adherence agents should be able to help people live well and be healthier while also reducing the cost and risk of interaction with healthcare providers and risk of adverse events due to drug interactions between concomitant medications.
But whatever mix of strategy we use, it seems to me that the priority of AI-based adherence should be to add value to society by improving the lives of people and reducing the impact of heart disease by using clinical, evidence-based metrics such as EF.
The cost to society justifies usage of AI.
CVD (cardiovascular disease) is estimated to cost the EU economy €210 billion a year. Of the total cost of CVD in the EU, around 53% (€111 billion) is due to health care costs, 26% (€54 billion) to productivity losses and 21% (€45 billion) to the informal care of people with CVD.(13)
1-2% of health care expenditure are attributed to heart failure in Europe and 74% of heart failure patients suffer from at least 1 comorbidity: more likely to worsen the patient’s overall health status.
Combining approaches of coaching and smart mobile apps with AI moves the communications focus from a physician-patient partnership based on rules (what you must take, what you cannot eat) and reframes rules as boundaries(11), as “if . . . then” actions that are more positive, more effective and less power-oriented than rules. We believe that the reward (10) of positive outcomes empowers patients better than the threat of negative consequences by their physician. Recommendations for action may also include informative explanations, which may further motivate the patient.
Behavior-modification – the next stage of evolution in AI-based adherence?
Mobile medical apps that measure signs like heart rate and sugar generally use measurements and statistics for informational and educational purposes rather than trying to change the patient’s state. However – AI can help modify behavior and not just coach.
The notion of reward and learning from interaction with patients data to achieve a goal of better health or stability is immensely more valuable to a patient than maximizing pill-taking or simply taking measurements and providing education.(9)
Flaskdata.io | The team behind the world’s fastest patient compliance system Helping over 300 sites around the world and thousands of patients/month
References
1. Krumholz et al. Randomized trial of an education and support intervention to prevent readmission of patients with heart failure. J Am Coll Cardiol 2002;39:83-9.
2. Harvest: an open platform for developing web-based biomedical data discovery https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3932456/ https://github.com/ohdsi
3. Reuse of Clinical Data C. Safran https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4287069/
4.Recycling side-effects into clinical markers for drug repositioning https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3334551/
5. Barriers to sharing https://www.nap.edu/read/18267/chapter/4#22
6. Clinical study data sharing https://www.clinicalstudydatarequest.com/
7. CRF standardization https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3078665/
8. Clinical trial data sharing http://www.phrma.org/codes-and-guidelines/phrma-principles-for-responsible-clinical-trial-data-sharing-certification
9. A Reinforcement Learning System to Encourage Physical Activity in Diabetes Patients. Hochberg et al… https://arxiv.org/abs/1605.04070
Geriatr Nurs. 2010 Jul-Aug;31(4):290-8.
10. Medication adherence is a partnership, medication compliance is not.Gould E1, Mitty E. https://www.ncbi.nlm.nih.gov/pubmed/20682408
11. Rules and boundaries in patient-centered clinical trials
12. Bluestein, D. Lieberman – Rules and boundaries in clinical trials
13. What to teach to patients with heart failure and why: the role of nurses in heart failure clinics. Eneida Rejane Rabelo et al. http://www.scielo.br/scielo.php?script=sci_arttext&pid=S0104-11692007000100024
14. European Cardiovascular Disease Statistics 2017 http://www.ehnheart.org/cvd-statistics.html
15. Reinforcement Learning, An Introduction, Sutton and Barto, MIT Press, 1999
16. Reliability of SNOMED-CT Coding by Three Physicians using Two Terminology Browsers, Chiang et al https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1839418/
17. Depression Is a Risk Factor for Noncompliance With Medical Treatment: Meta-analysis of the Effects of Anxiety and Depression on Patient Adherence. DiMatteo et al http://jamanetwork.com/journals/jamainternalmedicine/fullarticle/485411
18. Importance of medication adherence in cardiovascular disease and the value of once-daily treatment regimens. Frishman. https://www.ncbi.nlm.nih.gov/pubmed/17700384
19. Comparison of Approaches for Heart Failure Case Identification From Electronic Health Record Data. Blecker, Katz, Horwitz et al. http://jamanetwork.com/journals/jamacardiology/article-abstract/2557840
20. Automating the integration of clinical studies into medical ontologies Mark Roantree, Jim O’Donoghue, Noel O’Kelly, Martin van Boxtel, Sebastian Kohler
21. A configurable deep network for high-dimensional clinical trial dataJim O’Donoghue, Mark Roantree, Martin Van Boxtel
22. AI shows promise as a clinical development tool
23. Srinivas Karri , Oracle
24. Artificial intelligence in healthcare: past, present and futureFei Jiang, Yong Jiang, Hui Zhi, Yi Dong, Hao Li, Sufeng Ma, Yilong Wang, Qiang Dong, Haipeng Shen, Yongjun Wang DOI: 10.1136/svn-2017-000101 Published 21 June 2017
25. Mining Patterns of Adverse Events Using Aggregated Clinical Trial ResultsZhihui Luo, PhD, Guo-Qiang Zhang, PhD, and Rong Xu, PhD
26. Data mining for better clinical designMichelle Marlborough, Medidata in the book “Re-Engineering Clinical Trials : Best Practices for Streamlining the Development Process”
27. The Digital Revolution comes to US Healthcare Goldman Sachs, Equity Research Report, 29/6/2015 Roman et al
28. Using Artificial Intelligence to Reduce the Risk of Nonadherence in Patients on Anticoagulation Therapy Daniel L. Labovitz, Laura Shafner, Morayma Reyes Gil, Deepti Virmani, Adam Hanina (all physicians from Montefiore Medical Center, Bronx)
29. Method and apparatus for fractal multilayered medication identification, authentication and adherence monitoring Hanina, Yaron Ganor et al
30. Mobile Health Strategies for Veterans With Coronary Heart DiseaseThe purpose of this study is to determine whether text messages/messaging (TM) or a mobile application (app), compared with an educational website-control provided to all Veterans, can improve adherence to antiplatelet therapy among patients following acute coronary syndrome or percutaneous coronary intervention (ACS/PCI). Will start recruitment beginning of 2018.
31. How to Truly Solve the Patient Engagement Problem with AIWaqaas Al-Siddiq, CEO and Founder of Biotricity Inc.
32. What’s wrong with mobile apps for medication adherence?
33. Assessment of medication adherence app features, functionality, and health literacy level and the creation of a searchable Web-based adherence app resource for health care professionals and patients Heldenbrand, et al.
34. BCT – Behavior Change Technique taxonomyMichie et al 2013
35. The Potential Impact of Intelligent Systems for Mobile Health Self-Management Support: Monte Carlo Simulations of Text Message Support for Medication Adherence Piette et al. 2014 Annals of Behavioral Medicine.
36. Learning from Demonstrations for Real World Reinforcement Learning, Hester et al, “Deep Q-learning from Demonstrations (DQfD), that leverages this data to massively accelerate the learning process even from relatively small amounts of demonstration data and is able to automatically assess the necessary ratio of demonstration data while learning thanks to a prioritized replay mechanism.
37. Contextual Bandits with Linear Payoff Functions Chu, et al.
38. Contextual Linear Bandit Problem and applications Feng et al.
