Terry Miller:
Data science, generally, is about seventy percent the best practices of software engineering and thirty percent statistics and voodoo. Of that five trillion, five percent of the sickest patients contributed to half that amount. A large language model is still a pattern-matching algorithm, just like every other algorithm. I almost think of them as commodities now, very expensive commodities. Why would anyone come to you, your company, over ChatGPT or Claude? I feel like, for us, and for companies in our position, the data is the moat, but I think more importantly.
Saket Saurabh:
Hi everyone. Thanks for listening to another episode of Data Innovators and Builders. Today I’m speaking with Terry Miller, Vice President of AI and Machine Learning at Omada Health. Terry, thank you for chatting with us today.
Terry Miller:
Thank you, Saket. I’m glad to be here. Thank you.
Saket Saurabh:
Excellent. We were just chatting about AI in healthcare, Terry. I feel like most AI leaders worry about accuracy when it comes to healthcare. Relying on AI is almost, I feel, essential, a huge opportunity, but it also comes with various challenges. Give us a lay of the land on AI in healthcare and what you see there.
Terry Miller:
There’s a broad spectrum of applications, probably similar in most verticals in healthcare, ranging from administrative tasks through computer vision, which is fairly mature, especially in radiology, and then there’s more emerging agentic workflows and predictive prevention.
So there’s a broad spectrum of applications, but all of them are more challenging, I think, than any of the sectors I’ve worked in previously in data science, because there are constraints around how you train models with respect to PHI, compliance, and governance, and then, once in production, the controls and guardrails to keep patient data safe and secure are challenging. There’s also the veracity and provenance of your model’s output, especially if you’re surfacing it directly to a member, there are some challenges with those workflows as well. So, a lot of opportunity, it’s moving fast, a lot of folks are really pushing the boundaries, even with frontier models. But there are some challenges in building that are unique to healthcare.
Saket Saurabh:
And Omada Health itself is a pretty unique provider, being a virtual care provider. Tell us a little bit about what that means, and where your role comes in at the company.
Terry Miller:
There are several folks in the digital health space, and I’ll be interested to see the evolution of this space generally over the next few years, I have some personal thoughts there, but I think a digital health provider serves a really important, a crucial, function. I think the number is something like, in 2024 and 2025, five trillion dollars was the spend in healthcare, and of that five trillion, five percent of the sickest patients contributed to half that amount. What defines the sickest patients? They typically have multiple comorbid, chronic conditions. So we serve those patients that fall into those categories, hypertension and diabetes, or broad-spectrum cardiometabolic challenges, we serve all of those.
I think the fundamentals of digital health are crucial, because, given the scale challenges and shortages we all know about in primary care, digital health fills that void, and there’s so much opportunity for digital health to become even more impactful, leveraging tools like AI and other emerging technologies.
Saket Saurabh:
And in this particular case, when you talk about that five percent of cases, there’s a lot of data signals also available, from continuous glucose monitors to DEXA scans and so on, and what I’ve interestingly seen is that all these providers are providing MCP servers so you can chat with the data. Tell us a little bit about how you see that aspect.
Terry Miller:
I come from the industrial space, and to me, the synergies of individual, and by individual I mean single-person, healthcare, what that looks like two to three years from now, with manufacturing, are strong. There are a lot of devices, as you mentioned, from the home, worn on the body, generated from labs, all of that decentralized data, not coming from the physician’s office, that will inform treatment paradigms for individuals going forward. I’m excited to see where that goes.
I think the convergence of these devices is now, I’d say, show-ready. They’ve been getting better over time. We have clinical-grade devices now, the Apple Watch, for example, is an FDA-registered ECG machine, and I think as more of these devices get smaller, more powerful, and more accurate, that’s only going to improve healthcare for folks in the home. So anyone in the digital health space, anyone in healthcare generally, who doesn’t have a plan for ingesting large amounts of in-home or decentralized data, should probably rethink their strategy pretty quickly.
Saket Saurabh:
So when you say show-ready, you’re basically saying this isn’t a demo, this is production-worthy stuff, right?
Terry Miller:
Demo to production is the big story in AI.
Saket Saurabh:
So, what lessons have you learned in navigating from demo-ware to a production-worthy solution?
Terry Miller:
It hasn’t been a big learning curve for me, and hopefully this doesn’t sound strange, but it hasn’t been a big learning curve because I’ve anchored on the fundamentals of data science since I started my career.
There’s one really significant challenge, especially with LLMs today, and it’s not necessarily the LLM itself, it’s the application, the engineered system that an LLM is part of, or the workflow around it. Folks are relying on output from frontier models as their whole workflow, or they’ve got maybe a crude RAG pipeline, some pretty rudimentary system, and then it doesn’t generalize to new data, and they’re confused why. It’s crucial, especially in healthcare but really in any system, that you still treat whatever the application is, an agent, an LLM, a chatbot, as what it is: a large language model is still a pattern-matching algorithm, just like every other algorithm, and you have to test it. You have to do evaluations, you have to do offline monitoring. It’s even more crucial in healthcare that these tools are part of well-engineered systems with robust evaluation and monitoring pipelines.
So, for me, that hasn’t been a stretch. Whether I’m doing linear regression, tree models, or my teams are doing these things, it’s still the best practices of data science. So we’re pretty secure in our spot with that respect.
Saket Saurabh:
I think you’re very right. Those of us who’ve been grounded in data and data science, in rigorous building, testing, and evaluation, that’s not new. I do think that, at various board and executive leadership levels, it’s sometimes hard to demarcate between what your team showed you as a demo versus, “can we really roll this out,” and I think that’s even more true for a healthcare use case versus, say, creating marketing content. What advice or guidance would you share with people trying to build production-grade systems in a high-stakes situation like that?
Terry Miller:
I really like the distinction you made there, specifically around marketing, because I’ve made the same point internally a lot, about tools like Cursor, Claude Code, Codex, and so on, for personal productivity versus workflows. If a firm, especially in healthcare but really any firm, is turning a workflow over to an agent or a large language model, that’s a very different thing, especially if there isn’t a human in the loop, than me using Cursor for personal productivity.
I think your point about marketing is a good one. The distinction is that there are fundamentals in data science, irrespective of the application, fundamentals of what a well-engineered system is, and if we, as a healthcare company, are turning over workflows that are member-facing, patient-facing, or health-coach-facing, it’s imperative that we’ve built robust evaluation frameworks and monitoring pipelines, and that the veracity of the data is checked. There are some core fundamentals that go into a well-engineered system, and I think the application that a given company or team is trying to execute dictates how much those tenets apply. The further outside of personal productivity an application is, the more the tenets of a well-engineered system apply. I think that’s the gap.
Saket Saurabh:
One of the terms coming up a lot is “harness.” The way I feel about this is that these AI models are very capable, but also a little chaotic, hard to manage and control, great in some cases but not always reliable. The job of the harness becomes how you really take this and get the right output, whether that’s running things in a loop, making it repeat, orchestrating across multiple systems, and so on. How did you approach that, in terms of what harness ultimately worked well for you, when you were talking about production?
Terry Miller:
I won’t talk about specific tools we’ve used, but we’ve built an agentic harness for the enterprise, for Omada proper, not personal productivity, but for deploying agents at scale, member-facing and otherwise. In the vein of a well-engineered system, it’s built on reusable templates, we call them tools internally, in a low-code, no-code environment we’ve stood up, leveraging some commercial tools, but we’ve really erred on the side of reusable templates. That’s the technological part.
The more challenging part is finding workflows with really tightly bounded outcome expectations. A lot of times, the scope of what an agent or an LLM can or can’t do, or is good at, is so broad that it doesn’t generalize to new data in production. So really tightly bounded, clean applications with reusable templates, that’s what we’ve found success with internally.
I’ll give you an example. We built an agent that reviews member messages to our health coaches, and messages that were more appropriate for member support, like “my device isn’t connecting” or “my scale doesn’t work,” the LLM determines that intent and automatically routes it, we use Zendesk, so it opens a ticket with member ops. If the message is more intended for a clinical specialist, something about medication changes or a change in health status, we automatically route that to a clinical specialist with context. That’s built as a reusable template.
We have evaluation frameworks on the LLM, and on the agentic workflow as a whole: how well does it call the correct tool? That’s pretty easy in this case since there’s not a broad spectrum of tools to choose from. And then, how well does it deliver the payload? So the LLM, the tool call itself, and the execution of the tool call, all of that’s evaluated and modeled. Tightly bounded application, well-engineered, reusable templates, that’s the secret to success, I think.
Saket Saurabh:
Very well said. Now, as we were starting the conversation, you mentioned something very important in terms of outcomes, the five trillion dollar cost of healthcare, and how you’re targeting that five percent of conditions that cause half of that amount. From a real-world outcome perspective, maybe from your own example or things you’ve seen in the industry, what sort of outcomes are you seeing, people becoming healthier, or things that weren’t possible before, that you find exciting?
Terry Miller:
Yes, the answer is yes, specifically for us, and I think many other digital health companies are doing pretty well because they’re delivering outcomes. Employers, payers, or pharmacy benefit managers believe in this model, but they wouldn’t invest in it if it weren’t delivering outcomes. We’ve seen the success of our program, and what we lean on specifically is that it’s very evidence-based, clinically evidence-based, care. We’ve done a number of randomized controlled trials and other things that demonstrate the causal impact of our behavior change programs.
You mentioned that five percent metric earlier, and I think there are two ways to look at that. There’s getting the sickest five percent healthier, but there’s also preventing the next wave, or the wave before that, from becoming part of that five percent. A lot of focus has been on the former, and we’re in the middle of building a pretty innovative approach to the latter, preventing the next group from rising into that risk category or becoming the kind of contributors we talked about.
Saket Saurabh:
That’s a very good point, identifying people on that path toward becoming part of that five percent, catching the leading indicators, and being able to work on that. Would you say that some of these things we, in the industry, were thinking of solving with machine learning in the more traditional way, which would have taken months or years of work, has certainly become faster? Are you seeing that happen?
Terry Miller:
This is a fantastic question. Definitively, yes, and it’s something I’m a big fan of, for good reason. Large language models, frontier models, and the workflows they enable are all the rage today, and they should be. But there’s still such a place for traditional machine learning and deep learning, augmented with, and combined with, those large language models. There’s a world of opportunity there.
What have been the challenges with machine learning and deep learning? Clean data sets, signals that are reusable and repeatable. We’re in a really good position because we have fourteen years of data, and in places where unstructured or semi-structured data haven’t always been available from a presentation standpoint for traditional machine learning, we now have tools and methodologies for leveraging large language models or computer vision models to structure that data, or to train models that can leverage both structured and unstructured inputs. So I love that question, we have a big focus on combining those two technologies for some of the things we’ve talked about here.
Saket Saurabh:
That brings me to the question of moats, building moats in this world today, since models are more ubiquitously available. Tell me how you see that landscape, in terms of what drives moats for companies like yours and related areas.
Terry Miller:
You’re exactly right, the model itself, I almost think of them as commodities now, very expensive commodities, but commodities. I don’t think that’s an especially innovative position, a lot of people have arrived there, but the moat, for us specifically, is two things. It’s the data set, fourteen years of longitudinal data for over a million members, that’s something a startup today, how would they even acquire that? So I feel like, for us and companies in our position, the data is the moat, but more importantly, the workflows created with the data are moats.
There are two primary areas we think about. One is knowledge, “insights” is such a cliché word I don’t think it’s substantive enough, but how do you help members understand their journey in non-complex ways. I think knowledge is one, but the other is, I’m a big believer in agents and the ability to take action on behalf of members, do a thing, or multiple things, and integrate into their life. So data is the moat, but those two paths built on the data are what’s just hard for a startup today, without that foundation, to replicate.
Saket Saurabh:
Great point. So I think the other cliché word going around these days is “context.”
Terry Miller:
Yeah.
Saket Saurabh:
Right. It is a very top-of-mind thing, but I think it’s becoming a bit cliché, it seems to me like it’s translating that knowledge of how things are done, as you said, the workflow, getting encapsulated. What are your thoughts on context, and are you seeing every company sort of becoming a context company?
Terry Miller:
Most of the people deeply involved in this space have arrived at the point where something more than the algorithm is needed. Whether you’re fine-tuning a model, I’m not sure how pervasive that still is, or not, all the things that are valuable require doing something with the model beyond a chatbot. I use a few of the frontier models just for Q&A, or sort of quirky, niche things everyone uses them for, as a running coach for me personally, but you don’t build a business on that, because that’s already established by OpenAI, Claude, and those companies.
How do you take the capabilities of conversational AI and make it unique to your workflow and your business model? That’s why I think context is important, from an enterprise perspective, and for our patients, or the consumers of any enterprise’s content, context matters in terms of what it allows them to do differently than if they just opened up ChatGPT. I believe deep personalization, for an enterprise, is table stakes, an expected outcome for anyone consuming content in any vertical, not just healthcare, because otherwise, why would anyone come to you, your company, over ChatGPT or Claude? You have to give them context that’s unique to them, in the right moment. That’s what a lot of folks are chasing, because it’s not an entirely easy problem to solve.
Saket Saurabh:
True context that’s specific to your customer, context about the problem, the domain. The model, of course, as we said, is a commodity, but it’s about bringing all those pieces together. I feel like the knowledge of workflows itself, especially in a complicated situation like healthcare with many moving parts, the payer, the PBM, the insurer, and so on, workflows can become very complex the moment you introduce multiple parties together, and that becomes part of the context too, in how you operate your agents.
Terry Miller:
This is something I’ve thought a lot about, and alluded to earlier. I think the individual’s home is a little manufacturing plant, two to three years from now, the convergence of robotics, AI, and all the wearables that are production-ready now, labs, social determinants of health, healthcare specifically, what environment does the person live in, how do they eat, how do they sleep, all of those things. It’s about orchestrating.
I think there’s a big, not-well-defined space right now to orchestrate in-home care. It doesn’t replace primary care, it doesn’t replace brick-and-mortar care, but to orchestrate and solve problems, scheduling critical specialist appointments based on readings that are out of range, or in a number of other ways, I think there’s a big undefined space for coordinating individual healthcare in the home, and a lot of folks are working toward solutions in that space.
Saket Saurabh:
That’s very interesting. So the home itself becomes a sort of first-level healthcare facility, with the right AI, data, robotics especially, and telehealth and other things brought together.
Terry Miller:
I coined the term, and I think you’ll put two and two together here, “edge healthcare,” from my time in manufacturing. I think that’s what it is, the decentralization, if I’m thinking about the end state of all the technologies emerging today for healthcare, if I had to sum it up in one word, it’s decentralization. It’s the move away from clinics where everyone has to go, toward some hybrid form, urgent and acute cases still have to go somewhere, maybe, but the bulk of healthcare takes place in the home. We have all the tools for that today, there’s a coordination layer and an integration layer needed with existing systems, and there are a bunch of policy changes that would have to come along with that, how we pay, how we reimburse.
I think that’s one of the big things for AI in healthcare specifically, autonomous AI even more so, how do insurance companies pay for it? There are some policy levers that have been piloted and extended, but I’m curious to see where this goes over the next two to three years, because for me, the logical end is decentralization.
Saket Saurabh:
When I look at the challenges of going to a healthcare facility, the scheduling, the waiting, most people are a lot more comfortable at home, feel more positive, easier to care for, there’s a lot of upside in how much of this can move to the edge, I guess.
Terry Miller:
Those are definitely challenges, but the biggest problem is scaling primary care physicians, there’s such a shortage of frontline, brick-and-mortar healthcare workers that it’s not a viable model long term, unless, again, medications like GLP-1 agonists change that. It’ll be interesting to see what impact that has on the sickest of our patients. But the model we have today isn’t sustainable from an execution standpoint. So I think that’s another thing that factors in.
Saket Saurabh:
You mentioned that you’re a big believer in agents, and I can see that, ultimately, agents are the ones that affect action, right, it’s not just about having a chat and getting knowledge. You also mentioned human in the loop. When I look at agents I’m running on my local computer, the human in the loop always ends up feeling like, “okay, allow, allow,” and then, “just don’t ask me questions, I’ll move on.” How do you balance that in a customer environment?
Terry Miller:
I keep coming back, hopefully I’m not belaboring the point, to what I said earlier: even with agents, the more bounded the application, the more successful it is. I think it’s a bit of a fallacy, or a knowledge trap, to think you can build an agent on an unbounded application and expect impact. A tightly bounded application, a well-engineered system, they work like magic, all the efficiencies promised by today’s tools actually show up in that environment: tightly bounded application, well-engineered system.
I think the challenge is when the scope of what one tries to do with a single agent expands, that’s where the coordination of multiple agents, agents of agents, comes in. That’s also something we’re working on now, an architecture that can solve for some of that. But I’m still a believer that the more tightly bounded the application, the more successful you’ll be with agents.
Saket Saurabh:
So, Terry, you were talking about how it’s important that the use of AI is well defined, almost like good product and engineering design, build a limited set of features that work well and are one hundred percent reliable. It’s easy, with AI, to just add more features, it’s cheaper to do, but that doesn’t necessarily mean they all work reliably. That principle holds, I believe, from what you said. Correct?
Terry Miller:
Yeah. I think, especially, data science generally is about seventy percent the best practices of software engineering and thirty percent statistics and voodoo. So I think it’s really important, in non-deterministic systems, to bound the application as much as possible, and to try to be sure that, in the build phase, all the scenarios the system is expected to complete or execute on, it’s been trained on, and then, how well does it do that, and are you monitoring that in production? So, again, not to belabor the point, but that bounded application, reusable templates, a well-engineered system, is the best path to success with AI and agents generally, today.
Saket Saurabh:
I know we’re coming close to our time, so I want to move through a couple of quick questions. One is, healthcare is very tightly coupled with public policy in this country. From your perspective, or where you see the industry with the impact of AI, what would you hope for in terms of policies that allow for responsible AI along with good public health outcomes?
Terry Miller:
There are a number of things downstream of AI that I’m interested in, like broadband access being considered part of healthcare infrastructure, I’m deeply interested in that, for reasons we’ve already talked about. But specifically on AI, there’s a very fuzzy line between what’s traditionally been classified as an FDA medical device, whether software or an actual device, and AI. I think those lines are fuzzy, and no one really knows. There will probably be a number of lawsuits brought, and eventually I think the guidance is going to change.
I don’t have the foggiest idea what that will look like, I’m not sure anyone does, but what we know for certain is there will be quite a bit of policy around AI, the ability to run inference, and whether that constitutes a software medical device. I think that’s the nexus of where policy is going to be most impactful in the near term. I don’t have a position on it either way, I think we’re ready to build in whatever environment the guidance sets up. But I’ll be interested to see how that’s resolved over the next few years.
Saket Saurabh:
And for AI leaders listening to this, what measures or metrics should they track to assess their own success or the success of their programs?
Terry Miller:
For us, it’s all about, well, we have engineering metrics we use to track the performance of our systems, but in a vacuum, does that matter if it’s not attached to people becoming healthier? So, is our members’ A1C going down? Is their blood pressure going down? Are they losing weight? I think, at the end of the day, even for engineering, we may not design the features, but we’re most interested in what health impact what we build actually has on our population. So there are metrics we pay attention to, obviously, but we’re most interested in the impact on outcomes for our members.
Saket Saurabh:
And any advice to leaders in terms of what they should be communicating to executives at the company, regarding AI, outcomes, and what can happen in the next few years?
Terry Miller:
AI leaders should be very pragmatic and practical about the discipline required to build successful solutions. For good reason, a lot of the hype around AI is the theatrical, best-case scenario. I think the pragmatic, realistic position today is that it’s still a lot of work and a lot of discipline to build things that work in production. So I’d encourage AI leaders, and more importantly, executives at companies that want to invest in AI transformation, to understand the distinction between tools for personal productivity, or marketing applications, and things you build to actually run your business on. There’s a discipline and rigor associated with the latter that I think everyone wants to hand-wave away, or, out of a lack of experience, just doesn’t appreciate. So I’d still ground everything in the best practices of a well-engineered data science workflow, and execute with discipline and rigor on that.
Saket Saurabh:
Thank you, Terry. I’m personally very excited about what AI can do for healthcare, whether it’s drug discovery or, as you talked about, healthcare on the edge. Thank you so much for chatting with us today, a lot of great insights for, I think, everybody, whether in healthcare or not.
Terry Miller:
Yeah, thank you so much. I enjoyed the conversation.
Saket Saurabh:
Thank you.
