Building Differentiated Patient Experiences | Kerem Ozkay, COO and Ayo Omojola CPO, Carbon Health

[00:00:05] Nikhil: So we just did our two-on-two podcast, uh, which I thought was pretty fun. We talked with Ayo and Kerem at Carbon. For people in the health tech universe. I feel like Carbon is one of the companies that has probably, you know, fucked around and found out the most when it comes to like AI tech and how it's useful in frontline care and all this kind of stuff.

They had some really cool insights into experiments they run all the patient acquisitions or what happens in the clinic. We talked about their AI charting tool. They actually showed us a demo, which is very cool. Learned a lot, and they seem to have like a very cool working, you know, environment and atmosphere.

[00:00:40] Danielle: Yeah, absolutely. If you are listening on audio, the part where they go over the demo, that might seem a little bit off. So I definitely encourage you to go on YouTube for this episode, but Nikhil, you covered that all. I think this was such a, a meaty episode that I don't even wanna talk all that much.

Folks are gonna get a lot of value outta this one. 

[00:00:56] Nikhil: Yeah, I think just listen to it. A lot of good experiments you can run in your own [00:01:00] orgs and, let us know what you think.

All right, we are back with Ops. I did it again. Uh, this time we have Ayo and Kerem from Carbon Health. Ayo scrabbles the world of FinTech healthcare, and a lot of like really, really he's like attracted to Unsexy industries and I guess, uh, unsexy podcast, which is always here. Uh, and we're very, very excited to talk about all the nitty-gritty operational details that both of you have sort of experienced at Carbon.

So thanks for coming to hang out with us. 

[00:01:29] Kerem: Thank you for having us. 

[00:01:30] Ayo: Thank you for having us. 

[00:01:32] Danielle: This is our first, our first four sound on the pod. So let's see. I know how this goes. 

[00:01:38] Nikhil: Very excited. 

[00:01:39] Ayo: We'll, we'll aim not to disappoint. 

[00:01:41] Danielle: All right. I feel like it's a good segue. Nikhil, you already eluded to Ayo's writing, so I did some, some heavy duty recon, which was like an hour's worth of reading actually on, on one of your blog posts.

[00:01:52] Nikhil: So, so one post.

[00:01:53] Kerem: One, one post! 

[00:01:57] Danielle: I tried to summarize it. Uh, but actually [00:02:00] one of them that really stood out to me was talking about, um, basically like four levers to build differentiated tech-enabled healthcare services. So I'm gonna summarize them back for listeners. The first one is like, you need to get a differentiated way to get paid versus incumbents.

Two, you need step wa step function, improvement in clinical outcomes. Three, a differentiated operational model. Basically how you deliver the service for a lower cost. And four, a differentiated patient acquisition model, how you acquire patients for cheaper. Um, there was like a ton in there and I wanted to like riff off those and had 50 questions, but I picked one.

[00:02:37] Ayo: You know, you know what's the best thing? The best thing about that post is that the, the last two patient acquisition and operational model are all Kerem. Like he's the guide actually . 

[00:02:50] Danielle: What do you mean by that? 

[00:02:53] Nikhil: Kerem actually goes into the streets and just be like, please come to our office.[00:03:00] 

[00:03:00] Danielle: He's the floaty guy. 

[00:03:00] Ayo: No, I mean, I mean, like more seriously the, like, when I think of a lot of the stuff that we've done to drive patient acquisition and retention, and reduce barriers to patients coming in, a lot of that, like innovation has come out of our marketing stack. Things that Kerem does super well.

High quality, literal, high quality like Google Ads, SEO funnel optimizations, like all the digital stuff. And then, you know, obviously like in, in the way Carbon works, we're also a retail business, which is part of Kerem's background. And so a lot of the, like what happens when a patient walks in the door, what happens when a patient calls the clinic?

A lot of that stuff has come out of the marketing orders, which, which Kerem used to run. And then on the operational side, like a lot of the improvements, and I dunno if we'll get into this today that we've made in our business have really happened over the last year since Kerem was COO. Like I do a lot of stuff and like, I, I like to say words, but like... 

[00:03:56] Nikhil: Maybe we could riff a little on this... every direct-to-consumer healthcare [00:04:00] company has talked about, rising customer acquisition costs and like how it's become very saturated to do things like direct-to-consumer advertising online, all this kind of stuff. Like what is Carbon doing that's like, so notably different from everyone else.

[00:04:13] Kerem: Yeah. So there, there are a couple things there. One, we also see the rising cost of direct-to-consumer patient acquisition, especially digitally. Everybody's playing in that space, 

[00:04:22] Kerem: but, uh, we tend to go run towards all the white space that exists and, uh, it's like an easier thing to say than actually do, but identifying what we do within our business.

Where we understand demand exists and then opening up the operations in a way that allows us to go acquire patients into that. And so if, if I was just sitting in the, in the marketing seat, which I have in, in the past, I would be banging on the doors of the operator to say, here are the types of patients that I wanna bring in.

And they would be telling me, no, no, no, no. Go bring me more of these. And it just doesn't work that way. 'cause they go bring in more of the ones they're looking for are generally more expensive. [00:05:00] 'cause that's what all the operators are looking for. Um, and so a thing that, that we do really well here, and I, I know I hope pass the compliment my way, but the reality is we set up the tech stack in a way that we can, uh, understand where we have gaps in the operational side or the scheduling side or where we have opportunities for revenue.

And then line up what we can do on the patient acquisition front with that and go and try and fill that. I'll give an external example than an internal example, but we have, um, groups that we've worked with or are working with. Or have looked at through various CorpDev opportunities where they're booked out for weeks on, on hand.

And they, they, the marketing teams there say that's a huge success and they have low acquisition costs, but they're, they're booked out for very low value visits or they're, or, or low value customer types. And, um, what, what we've done at Carbon is we've kind of segmented our, our flow of traffic and, um, schedule in a way that we can take in some of that 

[00:05:57] Ayo: Can I interrupt for second, actually?

[00:05:58] Kerem: Optimize the marketing machine. Yeah, go for it.

[00:06:00] Ayo: The thing Kerem's not saying, which is that like the consequence of that is actually you are reducing provider supply effectively. Mm-Hmm. Because you have providers basically not practicing at the top of their license for like really extended periods of time.

And so for us, like there's like this revenue component, but there's like a patient who has actually a complex problem that needs to be seen and can't.

[00:06:23] Kerem: And, and it just clogs the system in a lot of different ways. And we've optimized both the, the, the tech, but then also how it plugs into the marketing side to go drive higher utilization of more complex patients, therefore more revenue on, on the side.

So it's not necessarily just like a, a widget on the, on the, on the website or a thing that we can go do through the ops model. It does require great integration with, with the product side. 

[00:06:48] Danielle: Okay. I, I'm, I think I'm following, but maybe let me explain that like I'm five back. So like there's some correlation that, or some strategy around making sure your scheduling is [00:07:00] reflective of the types of patients you want coming through the door and being strategic about availability based on condition type or medical necessity.

And, and the, what you're doing there... 

[00:07:13] Ayo: And like, and like time of year, like everything. 

[00:07:16] Kerem: And then optimizing the top of funnel according to that. So we don't go and invest our. Our advertising dollars into those lower value visits. Or even though they have a higher, they, they have lower CACs, they absolutely do lower cost per acquisition.

If I was just sitting in the marketing seat, I would say, look at my conversion. Look at my cost per acquisition. Go fund, go fund more for me. But as you pull it through on the, uh, through the entirety of the business, just it doesn't make sense. And then also where we've chosen to invest more of our SEO and um, kind of more of these programmatic efforts that we have through product is on these higher value visits as well.

Where, again, when, when we talk about filling white space, yeah, we're pulling in 1, 2, 3, 4, 5 patients per day. But in [00:08:00] aggregate it adds up to a bunch. 

[00:08:02] Nikhil: I do feel like it's sort of interesting 'cause um, you guys kind of straddle this line between, you have these urgent care visits that are probably like one-off and people come every so often and maybe it's like a sporadic usage.

And then I'm guessing you also have these like more chronic slash complex patients that are coming through the door. Right. And I'm guessing those are very, very two very different ops models, very different CAC, very different like customer acquisition strategies and all that kind of stuff. Is there, like, when one pa do you or do you try to like serve both or are you trying to pick one or the other when you think about these things?

[00:08:34] Kerem: We try and serve both to a certain extent and, and I know it's a little bit of a cop-out answer, but, um, it just depends on the, the care model within a specific clinic and the relationships. We have the pairs in the areas, so it's not as binary as. We're gonna go drive this type of patient acquisition everywhere. And then beyond that, what Ayo touched on is there are just seasonal aspects to this where, uh, respiratory, you know, is high demand right now. It's winter, it's [00:09:00] raining, people are sneezing on each other. Great, we'll, we'll go bring that in, and that's important. But we, we do want to bring in, uh, more of these high value chronic, patients and get them into our primary care services where it makes sense for them and us.

Where like, it wouldn't make a ton of sense for us to just go pull, pull in a bunch of patients that are, um, diabetic. If we don't have a robust diabetes program. We, we've, we've since kind of transitioned that off to our core primary care and it's a portion of what we do. Um, but yes, those are high value patients.

Yes, we can make a ton off of them. Um, long term with, with relationships we have, it just would not believe them or us in this case. 

[00:09:34] Danielle: You had talked a little bit about some of the SEO strategy. Can we unpack what, what you're doing on the SEO side? This long tail and AI cross hybrid thing? 

[00:09:44] Nikhil: Yeah. Are you guys doing AI generated shitposts on LinkedIn 

[00:09:48] Kerem: hahaha, no yet!

[00:09:50] Danielle: That's Ayo's growth strategy right now on LinkedIn. 

[00:09:52] Ayo: For that, for us to be doing that, you'd have to zoom that I'm actually intelligent.

There's like a bunch of ways a patient can find you. A patient [00:10:00] might find you because they're just looking for something near them. So like urgent care near me is that, a patient might find you because they're looking for a specific condition.

So, I have a UTI and then a patient might find you because they have a specific symptom like I'm coughing. And in practice there's just like, actually not really a way to flatten that into like one path. And so I would say like one of our heaviest investments over the last few years has been. Making sure, like we always had a structure internally.

Like one of the early things that Aaron built was this ontology of like every condition and all the questions that we would ask to intake a patient so you wouldn't have to come in and fill out a paper form. So we had that always. What we didn't have is a, way to reflect the patient's actual intent into that flow.

So, so roughly speaking, the way to think about it is if I go search urgent care near me, we have a very good landing spot for you that is like really focused on a geographic [00:11:00] search. So it's like this location, this is where it is, here's how to get there, here are the providers. You'll see there, this is what the clinic looks like, here's the parking detail.

Like you can, it's just like it's, and then here's the, the services we provide in that clinic. Here are the conditions we, we consistently see patients for. In contrast, if you look at like actually most healthcare providers that actually do. Sort of physical, like retail. What ends up happening is they have like one entry point and then you like kind of pick from a list of providers from a list of locations, and then you like kind of go down.

And our insight was, there's actually the, the patient's actually making a lot of choices in one choice, and we need a entry point that models all those choices. If a patient is coming in and they have like a mindset of, Hey, I have this clinical problem I want solved, we have a totally different frame.

Like we have this tool that we call care discovery, which you can think of roughly as like for the types of conditions that we see patients for or for the types of problems Clinically that we solve for patients. [00:12:00] We model every, like, we're like, Hey, how would the patient think about this? How would they ask about this?

If you were talking to a friend, what would you say? What are the common questions that a patient might ask? And then what are all the locations like? Obviously we have, I think right now we have something like 120, 130 clinics. And the service mix, like the things we do in each clinic is not actually identical.

And so we basically map, you come in for a clinical reason. We have, we help you understand like, hey, this is kind of what you're in for. And then we also help you understand these are the things that, these are the places that you might go to solve this problem, that carbon services. we had the first taste of that, I guess.

During early Covid, we did this like crazy thing where, in like March, 2020, we hired like a mechanical type group to call every hospital in the country and ask them if they had covid tests in stock. And we would do it like every other week. And then we built this map that surfaced. 

[00:12:56] Kerem: Mind you, this is when nobody could find testing, at all. It just was [00:13:00] non-existent. 

[00:13:00] Ayo: Like in March, 2020, you would sneeze and you'd be like, "do I have covid? Like, what's happening right now?" And it, we, it kind of, I, I think, I think at one point actually, like something like 10% of covid tests in the United States were routed from this look, from this site.

Like we helped Apple and Google, like fill out where to find covid testing in Apple Maps and Google Maps. And it kind of gave us this insight that like, our previous orientation was you would find carbon and then you would search for the, the clinical reason you were here, or you would find a location and you would search for a clinical reason you were here.

And we just broke out the clinical reason to the top level of intent is like the way to think about it. And then very specifically on ai, it's basically something we use a little bit as like content generation testing, helping us understand like what questions patients might ask. Filling that stuff out and then like doing some editing, passing it through a clinician filter before putting it all in.

[00:13:53] Kerem: Yeah. And by the way, I mean that Workstream started in 2021. There were various AI tools we're using. Then we had a [00:14:00] pretty robust marketing team and a, and a copywriting team. And they're wonderful. But for them to write 5,000, 6,000, 7,000 posts in a way that they were happy with, just wasn't realistic.

And we got through about a hundred of them in three months with them. And then, we moved them onto a separate project and we ran this through that process that we're using on the AI side with the clinical o oversight on it. And we turned out 6,000 of them in about a week at that point. And then we could kind of let the floodgates open and, you know, it helps optimize what we're looking to do.

It helps with the iteration, but also we needed a lot of signal to see if this was gonna work. And a hundred pages wasn't gonna get us the, get us the signal we needed. But the 5,000 plus, got it for us very, very quickly. 

[00:14:40] Nikhil: And are these posts that are just like, "Hey, this is what, like if you feel like you have this, here's some things that like," stuff like that? 

[00:14:48] Kerem: Yes. Very simple. Like, and if you think about what the patient, they know if they have a stuffy nose or not. Right? It's, it's, it's not so much like, "Hey, if you have a stuffy nose, you have these symptoms, so on and so forth." It's just, that signal between what they're [00:15:00] looking for and what we can offer in that location.

And that's the high fidelity piece that, that makes the, the situation easier for them. They don't have to think about does this place offer the thing I'm looking for at that point. 

[00:15:11] Ayo: Yeah, and I, I would say the one other nuance I would add is patients use Google all the time to basically navigate to care in some way. The big difference between like what we do and say like what, what a WebMD does is doctors work here. And so...

[00:15:24] Kerem: yeah.

[00:15:24] Ayo: ...so the, the, the entire purpose isn't like, "Hey, let's tell you clinically what's happening to you online." It's that like, if, if you're in a place where you have a problem that we can solve for you, come find us.

[00:15:35] Advertisement: The following message is brought to you by Out of Pocket. People who have been tasked with building a call center have many questions. How did I get this job? What are my options? How will this affect my loved ones? Will I ever be happy again? You need answers. That's why we made a course led by call center veterans who've been there and recovered.[00:16:00] 

Join nearly 100 others who have not learned the hard way. To enroll, call toll free today to reach no one, or go to outofpocket. health. Hear from Stephen, a student who said, quote, This course was hugely validating and packed in many invaluable insights that we would have otherwise had to learn the hard way.

Five out of five stars. You can be Stephen. Save money. Start learning. Enroll today. For 100 off your seat. Use code ANSWERS. That's code ANSWERS for 100 off.

[00:16:38] Nikhil: This episode of Ops, I did it again is brought to you by me because no one believes in me more than me I'm teaching a healthcare 101 crash course that starts soon Uh, you can find it on the out of pocket website at outofpocket. help We go over all the major stakeholders in healthcare payers providers pharmacy all those In between acronyms that you've been too scared to ask people [00:17:00] what they mean, we'll talk about how the money flows, major laws in the industry.

It's the fastest way to get up to speed on U. S. healthcare. You can use the code IBELIEVEINME to get 100 off.

[00:17:11] Danielle: How important, so I went through your landing page flow where I, I can get a clear mental model of like, how you say primary care, urgent care, and so on. Right next to it, you indicate when I can be seen next. Yep. And that in three minutes apparently I can be seen. So, sounds like pretty fast. Have you always had like, the time to next visit as part of that experience or was that also an insight, uh, as you were transitioning to buNPIng up the, the types of carefirst? 

[00:17:39] Nikhil: And did you steal this from the emergency department billboards that always are like, "here's the next, here's the next open slide?" 

[00:17:47] Kerem: You know, we debated moving to the DMV model where they just give you a random number where you don't really know if it's live.

The speed to care is the most important thing.

we've always shown it, we've always realized that it is [00:18:00] important we see a fall off in conversion when it's out there. And, covid is a time where we saw a ton of patient demand, we weren't able to keep up with it, and we would see conversion rates fall flat... and we kind of flipped the operating model then as well, where we had, in-person visits you could do virtual, but if you really needed testing, was virtual really gonna do anything for you?

The answer is no. But we allowed patients to be seen within the clinics for, clinical providers that aren't doctors or APCs to do the swabs with virtual oversight. So screens on providers there, and it really would allow one provider to see, oversee two, three clinics at a time.

There's a physical provider in the clinic as well, but just as a way to, to increase throughput. And the reason why we knew we could afford to do that was, again, knowing what the drop-off in conversion was and kind of how to drive the forward-looking patient acquisition as things were spiking.

I would say to your question though, I mean, 95% of our bookings today are same day. That is by [00:19:00] design. We really optimize towards having availability. You know, a thing that we haven't really touched on, and I imagine we will, is we have really restructured our scheduling platform and our operating model to optimize towards when patients are looking for care and that that is not like a flat thing by we, we don't just have a flat schedule by clinic. Yes, there are clinics that have fixed schedules, but we optimize the staffing and how we drive patient demand depending on what we know about what's happening seasonally.

And then in that market and, specifically in that clinic and when we expect demand to spike in those clinics as well. We have a clinic in SoCal won't name the exact, spot, but, generally, if we can see 3 or 4 patients in the first hour, we're great 'cause then that leads to a pretty good day for us.

We have one clinic where we realized between the eight and nine A.M. hour there's, ton of demand and, if we just opened up the top of funnel and then we can drive about 16 patients in, in [00:20:00] that first hour that's, like, half the target that they would need for the entirety of the day to be breakeven profitable, so on and so forth.

And, we flexed the scheduling system, the advertising, and then the way that they staff bringing in some more support staff, having the manager on site be ready for that, where we just basically opened up the doors and then, we're proactive about communicating with the patients on when to expect to be seen.

'cause some of them, if we had just put up an 11 A.M. slot for them and said. Yeah, these slots prior to you coming in or booked, would never have booked with us. They would've never come in but us allowing them to say, yeah, you can get in, you can come into this clinic, but your wait's gonna be 30, 40, 45 minutes.

That was more optimal for them. So increased throughput in that clinic. I think they're, don't quote me on this, well, you're going to 'cause this video, but, um, their bookings were up about 35% between when we turned this thing up, Part of me. Their actual finished appointments and visits were up 35% between when we turned this thing on and prior to when we had it.

[00:20:54] Ayo: Yeah. 

[00:20:55] Danielle: Is the takeaway here operationally, like, to really open up your [00:21:00] schedule from, if you are a, whether...

[00:21:02] Kerem: No! 

[00:21:02] Danielle: That's not the takeaway? So what is the takeaway that somebody would listen?

[00:21:05] Kerem: Yeah, so the NPS was super high too at the time. It was that we, we not only opened up the schedule, but we staffed up accordingly.

We didn't, and we couldn't afford to staff up for the entire day. And we worked with the clinical teams on the ground and say, we know these first few hours are gonna be tough. It will taper off. Here's the support we're offering for you, plus we aligned incentive structure, so on and so forth for them, for this.

Where if we had just opened it up, they would've all walked out. Maybe the first day they would've seen it, maybe the second day, but the third day and they would've been added there. 

[00:21:35] Nikhil: Hmm. It's kind of a learned behavior in healthcare. I think a little bit that like wait times are sort of like, like you come in and waiting is sort of like a natural part of the process.

And weirdly, that is like I, for better or worse, that is like a normalized part of us healthcare is like people will come, there's a waiting room. Like it's fucking called a waiting room, right? The other thing too is, like, someone logically, if you really think about it, yeah, people wanna like see the doctor before work,

right? They'd rather not take time out of the workday if they can [00:22:00] before or after. But most clinics want to operate during work hours, right? So it's like a very weird, it's a very weird mismatch here, I think. But you know, I think you're...

[00:22:09] Kerem: We have this crazy problem actually. We have, we have this fun problem, where

we in California, we have to have people have to take the lunch after whatever, four or five hours it like aligns with everybody else's lunch hour. But when people want to come into the clinic is the lunch hour. And so, we've again, signal from product actually on this of, by the way, this time when you have the, schedule turned off is when the most people want to come in.

So we've shifted hours where we are operating an hour earlier. We're taking, this is not everywhere, but certain clinics were shifted an hour earlier. We have the lunch hour happen 11 to 12 rather than 12 to 1 or whatever the combination is there. So we can maybe... 

[00:22:50] Nikhil: Make these people take a brunch hour.

That's a brunch hour! 

[00:22:54] Ayo: Ooh, that's good branding actually. 

[00:22:57] Kerem: Brunch hour. 

[00:22:59] Ayo: The [00:23:00] one thing I was gonna add actually to Karim's point earlier, and this is Nikhil to answer your question about, was showing real time schedules like a signal from Covid. I think actually real-time schedules predated both Kerem and I here.

Yeah. The nuance for us is, most EHRs don't actually have a concept of a schedule. Like they don't have a concept of like, these are the hours available to book they have. And so the way that most urgent cares, at least, and I think that this is true for a lot of retail clinics solve this, is they say, "Hey, you can just walk in."

So you can think of it roughly like 95% of our visits are same day, but something like 50 plus percent of our visits are booked. And for most urgent cares, ninety-five percent of their visits are same day, but seventy-five percent of their visits are walk-ins. And you, you can only do that, like, that distinction is only really possible if your health record actually has a concept of times that are available that people can book in.

Like the concept of a walk-in that can be like. You know, [00:24:00] entered into the system upstream of a patient who might be booked for that same time, things like that. you know, I would say Kareem, keep me honest here, but I would say that, overall basically having the concept of time in the schedule.

Actually this thing we've iterated, I mean, I think we're on, scheduling V-five now actually. Yeah. We've like rebuilt this thing. We rebuilt this thing like once a year for like the last five years. 

[00:24:20] Kerem: The staff hates it. 

[00:24:22] Ayo: Yeah.

Exactly. 

[00:24:23] Danielle: They hate, they hate the, the software itself or the rebuild process? 

[00:24:27] Kerem: No, no. They love the software. They, they're fine with the rebuild process. This, the concept of scheduling is just very for, I don't wanna say it's foreign to them, but any iteration we put out there is gonna feel awkward to 'em. 'cause everybody thinks about time in terms of. How much time they have with the patient.

So support staff sees a patient for 2, 3, 4 minutes, right? On the, the dot, the MAs spend a different amount of time with the patients, the the provider spend a different amount of time with the patients. And anything that you put in that has like some block of time, they look at it and they say, oh yeah, that doesn't work for me.

'cause they're just [00:25:00] thinking about themselves within it. And even for the patient, like when we give 'em a 20 minute slot, they expect to be out in 20 minutes. 

[00:25:06] Ayo: 20 minutes. And what we're saying is, regardless of what their problem is. 

[00:25:09] Kerem: Yeah, exactly. And what we're saying in aggregate, you get 20 minutes of provider time depending on when you start whatnot.

So we remove that concept entirely. And we remove some of the concepts around a lot of the concepts around how much time each person has with the patient for that reason. But I wouldn't say we're at a perfect model at this point, but we are at a place where, we've removed a lot of the friction from the process and, make it optimal for both walk-ins and appointments for, for the staff.

But, they're always gonna hate it, I feel like, and that's, that's just a thing we're gonna have to work through. 

[00:25:42] Nikhil: I mean, it's basically like, imagine if someone could just put Zoom meetings on your calendar without your...

[00:25:47] Kerem: Totally. 

[00:25:47] Nikhil: You have any control.

[00:25:48] Ayo: What, what do you mean? Imagine? 

[00:25:50] Kerem: And, and, and then someone walking in the door and saying, "I need a meeting with you right now."

[00:25:54] Nikhil: Yeah. Well this is why I quit my job, so, um, I'm not gonna Right. Not the right person to ask [00:26:00] this. No, but I think, I mean, I think it's funny if you talk to docs at hospitals, right? Scheduling, I feel like is one of the core tensions in the entirety of the hospital, right? Because at the end of the day, it is representative of the agency that you have in your job in some capacity, right?

Is like, do I have control over my schedule or not? So of course there's always gonna be some tension around it. I do, I feel like, from what I'm hearing from you guys, that having a little bit more, at least visibility as well as like. To like how where you're gonna like when there's gonna be spikes and not what support you're gonna give them in that process.

Gives a little bit more agency back to them. But you know, I, I don't think it'll ever be a solved thing of like, hey, just like, here's my Calendly link. Like book it. 

[00:26:41] Kerem: Totally. Well, I mean, the current version that we use gives them visibility of where the patient is in, in the clinic in the process.

That has actually helped a lot before everybody was just kind of together, in a single scheduling board. So having that visibility has been good. But I know, I don't know if this is you, Ayo, or Aaron who I've debated with in the past where, there's an idea out there [00:27:00] of do we just show them any of it actually?

The reason why is, you know, if you see that there are 10 more patients coming in down the road. It might change some of the decisions you make now for better or for worse, or if you're just dealing with what's dealt your way to your point, it, it maybe changes your point of view on how much time you go spend with said patient and, the care that they need.

So that is a thing that we will try at some point and see what the results are. 

[00:27:23] Ayo: But like, you don't necessarily want your provider thinking about the 19 patients they have after you. That's true. While they're with you. And we, we used, we actually, a previous version of it was like, you just see like a giant list of just a scrolling list.

Yeah. It's, it's 11, it's 11:00 AM and you're just like, I gotta see like 50 people today. 

[00:27:43] Nikhil: That's true. 

[00:27:43] Kerem: And by the way, two-thirds of those people are gonna cancel and like why stress them out over that? 

[00:27:48] Nikhil: That's true. Yeah. Yeah. I can imagine if you, like we woke up in the morning and there's like just a task list of everything gets get, has to get done in the day.

It'd be pretty intimidating. 

[00:27:57] Ayo: I mean, Nicky will believe it or not, but some people have that. [00:28:00] 

[00:28:00] Nikhil: Not me, bro. I'm just waking up vibing. That's all. 

[00:28:04] Danielle: Are the doctors themselves seeing any sort of performance on a daily basis against that schedule? Like are there, you know, performance metrics that they're tapping into?

Yeah. And like, what are they? Uh,

[00:28:15] Kerem: We use a concept called RVUs here, which is pretty common in the medical field. They base their performance on that and, uh, we incentivize them based on that. Pretty standard on that front. And then the clinical staff within, they get to work, I would say highly analytical as an organization, but we really like to distill the analytics down to the fields and, limit it to call it, probably even too many at this point, but 5-8 metrics max.

And, a lot of those, they're all very specific. None of 'em are really interdependent with one another. You can control each individually and they would have different outcomes. And most of them are to, back to Ayo's team, either there's an AI aspect to them in terms of how it gets sent to them and what, they're [00:29:00] intended to do with it,

so there's some culling in that or, there are actual actions that we can take in within the EHR on that given day to go drive some of them as well. So none of them are just like report out metrics and say, "Here's what you have, deal with it." They all, they're all very actionable within a given day.

[00:29:18] Ayo: Yeah, and typically I think our approach to metrics is typically it's part of a process of accountability up and down the management chain, which allows both the more senior folks to like kind of understand what's happening on the ground and people in the field to push feedback back up the chain.

I, I think that was like one of the questions you guys were asking around, like, how do you design these feedback loops? And I think, you know, just get all credit to Kerem and his team for this, but these like tight feedback loops between people actually doing the work and whether it's like on Slack or in sort of a daily or weekly huddle, that sort of thing is critical.

[00:29:54] Nikhil: So maybe like, just segueing off this a little bit, 'cause one of the things, you talk about in the post is like being two [00:30:00] x better, like a ton of different things, versus 10x better at one thing. Can you like, walk through a couple of, like, what are some like 2x things that you've done that have just like compounded on each other or have just like you've seen massive improvements from? 

[00:30:13] Ayo: Yeah, well, let, let me, let me do a list. So like, just think of the effect of, think of the effect of care discovery and what it means for SEO and that driving up your organic traffic and reducing how much you have to spend to acquire. So like your blended cost of patient acquisition goes down. And then you, and then think of what it means, like what Kerem was saying around like being strategic about like which patient intent do you invest in and how you fluctuate that by season and what that means for like a, again, your overall blended cost of customer acquisition.

That just means like, you know, there's, you're spending less as an organization. That's one content is another example. Where, we took, like, this came from Kerem and I actually almost exactly a year ago, like would sit down and like listen to call transcripts [00:31:00] of like a patient calls a clinic and what does the person at the front desk say? Because we have like a hundred clinics, they're not in the same state. We just didn't really have like a good standardized way of understanding what was happening inside the visit, inside the call, and so what we did was we extracted the call transcripts, threw them into a large language model, transcribe, and then asked the transcript a bunch of questions about "why did the patient call?"

A lot of it was driven by the suspicion, like when you listen through to a bunch of them, you realize that there a bunch of these are like, there's a patient who's trying to come in and they, they're not getting a clear answer on, like, on, on a question that is gating them, making the decision to come see you.

And sometimes the clear answer is like, "yes, we, we take your insurance. So please come in." Sometimes it is, no actually like that specific procedure you need, like, you want, you need your medication renewed, actually you need to be a primary care patient and the provider needs to like do a full evaluation on you.

 When we started, our thought [00:32:00] process was, hey, let's like do some analysis and then do some automation to close the loop and like send the patient a text and like send them a booking link. And eventually what, what ended up happening? I, I think where, where we landed and Kerem, keep me honest here, but basically it, it, it now exists in a report that's part of a feedback loop.

For clinic managers to understand who is calling them every day and why, and then they can use that to call a patient back. They can use that to coach a front desk person, et cetera. So it's like part of like, you know, and this is just like one of the things, you know, when, when you talk about the TenX thing, at the end of the day our business is seeing patients like we, there isn't there, like the atomic unit of value we provide is like a provider is talking to a patient about their care today.

You can't software that. I'm sure at some point in the future somebody will come something, but today you can't software that away. And as a consequence, a lot of our feedback loops are optimized around how do we just make that interaction [00:33:00] better? And you know, if, if, if you think of the concept of TenX, like if I come in 'cause I have a cold or flu or like I, I have a laceration, like the best thing that can happen is.

I get the person I meet understands me, understands my problem, and makes the right decision, the right clinical decision that can help me get over the problem, at or better than standard of care. I dunno how you 10x that. Like you imagine like you walk in with a thigh laceration, like what are they gonna do?

Give you a new leg. Like, do you know what I mean? Just like there's not that much you can do. And so, um, anyway, that just a long way of saying like a lot of the optimizations are, like what is the actual best we can do? And like, let's do it. And, and for example, the contents thing, I think the care discovery thing's.

Another example, scheduling. 

[00:33:49] Kerem: Well those two are tied together though. And, and the way that we Yeah. If we understand, if we under one, we knew that we were gonna invest more into the care discovery side this year than [00:34:00] we did the prior year. And we will next year more than we do this year. Um, but a place of waste for us, beyond just the quality of the, the, the, the calls and the service you were getting that's a, either a page that we created that was driving some organic traffic or an ad that we went and bid on that's going to increase our c we, they get the wrong answer on that end. And so we, we wanted to be sure that we understood, what we were converting on those phone calls, how we were communicating with those patients, and what barriers we had to go solve ahead of time.

It's not just a, I'm gonna go turn the advertising on or off, depending on what the result is on this side, it's, oh yeah, a lot of these patients are calling about this insurance type that we don't actually take. Let's go figure out how we can go take that insurance type. And then go and, go do that,

right? And so once we did that, and once we understand what's going on, then cool, let's go Now, push on the patient acquisition lever and then go see what call intent looks like and if it's improved. And if the answer's yes, great, continue to invest on that front. And so, as I mentioned there, there were multiple variations that we looked at within the call content piece.

And there's a quality element, there's a conversion element, and then there's, there's [00:35:00] definitely within both of those a training element. And so there's definitely human touch that has to happen. And those kind of advantages are all over that. Or they're supposed to be all over it, and they are, for the most part.

And, um, where they're reviewing the results, they're going back and listening to, it's really important to hear tone and inflection within it, but they at least know where to go search and where to go look instead of having to listen to every call like Ayo and I did a year ago. 

[00:35:21] Ayo: Yeah, the, the tone of inflection thing is like crazy.

It's crazy actually how important it is because actually many times, much more than the words that are said, it's what the patient walks away with. And you, just, like you, it's very, very difficult to like replicate that without actually listening. And so in the reporting we have like some things around sentiment that, that give us a sense of like the soft, the soft part of the interaction. 

[00:35:49] Danielle: Was this, um, entire build totally done in-House?

You've mentioned like the LLMs and, and so on. You fed the LLMs yourself, you did the sentiment [00:36:00] analysis. You're surfacing up all these insights. There are so many companies that do that today too. 

[00:36:04] Ayo: Yeah, we, we use, we use a lot of off-the-shelf tools. So we, like, we use OpenAI, obviously AWS, Medical Transcribe.

So we're like not picky about, we just try and use like the best technology off the shelf, but like the implementation like this, this, this is something Kerem and I talk about that. I'm like, I can imagine that if you are, if you are a healthcare provider who's like very sensitive, you have to acquire patients every day and you're very sensitive to the patient experience.

Like this is very, very useful for you. But like. Many providers are not that, like many providers are like, Hey, we have a patient. We just see them for a very long period of time. We don't get that many calls. Like, so. Yeah, exactly. Yeah. 

[00:36:42] Kerem: We didn't go down this path saying, we're gonna go build this thing. By the way, this was never the intention with this one in particular.

Yeah. It, it was just iterations of, Hey, did you hear this conversation? Did you see this? Hey, what if we were to go do this? It was like six months of us just throwing ideas at each other. Not over the wall, but literally [00:37:00] like, Hey, I, like, I don't even know how to think about this. Can you listen to this as well?

Yeah. And, um, it, it, so it, there was no, we, we didn't go about it being like, here's a thing, we gotta go build. It just kind of one day happened where Ayo called me. He's like, I think we built a thing. He's like, I don't even know how you'd use it. And then he rolled it to me and I listened to, I like, oh, I know how I use it, actually. Let's go use it this way. It kinda just happened orgaorganically, as well. 

[00:37:24] Nikhil: Do you find that, because you guys operate clinics in a lot of different places, right? Do you find Mm-Hmm. I mean, I feel like the, the meme about AI generally is like deploying it into the real world, especially on a clinic by clinic basis, is just like a fucking, you know, shit show, right?

Uh, yeah. Do you find that when you, when you roll it out to different, that you have to like make a lot of tweaks when you roll it out to specific clinics? Or do you find that like a Yes. One-time generalized thing works relatively well. 

[00:37:49] Kerem: So one thing I've learned, and I always heard me say this a few times now, a, a year into this role, I can't convince doctors or providers to do anything they don't wanna do.

Like quite literally, no, no one's gonna do [00:38:00] it just by me saying, "please go do this." But I would say that one thing that we. Have in our back pocket is, um, as we build and iterate, Ayo and its product team work really, really closely with the provider groups here, or specific providers on the side and do create evangelist groups and then rolling it out's a lot easier 'cause they're able to either key in on what's beneficial to them or at least get some tweaks in ahead of time where it's not just, Hey, let's go throw this at the docs and make sure they, you know, use it and go make them use it.

 The feedback cycle they get is from either the operators on or the providers in real time and you're naturally building evangelists through that process as well. And it doesn't hurt that they're their managers as well generally. 

[00:38:43] Nikhil: You don't have to optimize for like Oakland slang versus like New York?

[00:38:46] Kerem: No, no, no, no. 

[00:38:48] Ayo: I think, to Kerem's point, like there's a bunch of adoption that's just driven by a bunch of the providers were in the room when the software was being developed. 

[00:38:54] Kerem: Yeah. 

[00:38:54] Ayo: So they're like, like there is something magical about seeing something Yeah.

That you kind of birthed come to [00:39:00] life. At scale you kind of have to do something that doesn't scale, which I think is like, there're like a provider using it has to be like, go to another one and be like, "Hey, you should try this 'cause it will, like this problem that you complained to me about all the time, it's gonna make it better."

And so, like a lot of our adoption is just like that sort of fractal, somebody collides with somebody else and then like that person collides. Well, al also a thing that I, I can't do as an operator, I can't do as a product person is, tell them how to use it in a way that makes sense for their workflows.

So the, the, the, the AI charting tool, uh, I don't know if we are, you all are familiar with it, but essentially. Device in the room, listens to the visit, creates a chart care plan. I'll let Ayo talk more about it in detail, but it, it, it, it requires the provider to actually do their visit in a very different way.

They have to articulate more of the visit which is beneficial for the patient. 'cause they're hearing feedback in real time. It's not just someone listening to their lungs, but I'm gonna butcher all this, but a lung sound clear, so on and so forth. A lot of medical lingo and slang, [00:40:00] whatever it is.

 It is a different visit type that, that is not unfamiliar to the providers. If they've used a scribe in the past or if they do a lot of physical type visits. However, it is not how they would naturally use it if you just said, here's the EMR, go run with it. So to Ayo's point, yes, having someone in, in the building who helped build it, build it and see it come to life is one thing.

But then also having someone sit down with them, oh, actually I see you did this this way. This is why you didn't get the intended outcome. But here's how I use it and here's why I get this intended outcome is a very different thing. And, and us as. Both a, a care provider, a medical group, and technology company allows us to go do that in a way that you wouldn't be able to be really be able to do as a technology provider.

'cause you just have some doctor that's a consultant somewhere saying, trust me, I used this thing. Yeah. Instead of actually being able to sit in the visit with them and actually shadow them and say, oh yeah, here's what I would go do different. 

[00:40:52] Danielle: How does that scale that like, uh, provider evangelist, when you have so many different clinics? 

[00:40:58] Kerem: There's a simple answer and a [00:41:00] difficult, the simple answer is, it kind of spreads like wildfire

if you have a good tool and product and workflow, and you can just drop the one evangelist in, in certain areas, and then they spread within the, the, the hardest thing about our business is the providers never really work with one another. For the most part, our clinics are single providers in, in, in a given day.

So, making sure that they have the touch points with one another to go do that is, is, is a different problem to solve. 

[00:41:25] Danielle: There's just like a massive Slack group that somebody's posting in 

[00:41:28] Ayo: Literally, there's massive Slack group 

[00:41:31] Kerem: we killed Slack actually.

[00:41:34] Nikhil: At the company?

[00:41:36] Kerem: um, in the field, yeah. On the, on the medical group side. In the field too. Yeah. 

[00:41:40] Danielle: He went to teams and staff. 

[00:41:43] Kerem: Part of it, actually, a very real reason is, you'd roll out something like, uh, AI charting and one person would have one complaint and then 40 people would see that and say, Oh I'm not gonna use that thing. So, um, there was a lot of, I don't wanna say misinformation, that's like the wrong thing, but, [00:42:00] well, There's a lot of inference that happens in, Slack that everybody can see that we wanted to really limit and mitigate.

And so the way that we roll things out now is. Very hands-on, but through the clinic managers, through their leadership structure. And really it's up to the product team to really make sure work with those folks and make sure they understand what they're rolling out and what they see so that the rollouts are cleaner in that way.

[00:42:22] Ayo: Yeah, we're very okay with negative feedback. It just needs to come from someone who's actually used it. 

[00:42:27] Nikhil: Yeah, that's fair. Yeah. I feel like at a company of your, like, I've only worked with small companies. I feel like at a company at your size, a Slack is like a little mini Reddit, basically, where people like, like... 

[00:42:39] Kerem: information and information integrity is crazy important.

And a thing that, that we've invested in is, Ayo's team built this out-of-pocket calculator that does a good job of telling the patient what their, what their, um, uh, their re their patient responsibility is, for the type of insurance they have and the type of visit that they've come in for.

But on the flip [00:43:00] side, the way that we used to do it here is we had a Slack thread for it. And you could post some Slack, do we take this insurance for this visit type? And the answer is maybe, and the reason the answer is maybe is if I'm in Southern California and I have Blue Shields, but you're in Northern California and you have Blue Shield, the answer's gonna be different for the, for the two of us actually.

So, um, we, we did want to go and build that in a way that was a, very clear answer to the patient, not what someone said in a different region about that thing. 'cause that person could be telling the truth. It just was false for the patient coming in in a different area. 

[00:43:32] Danielle: That's a good segue to RCM, which I'm curious about.

So insurance front. You like that segue right there? 

[00:43:39] Ayo: Let's go. 

[00:43:40] Danielle: Do I take your insurance and tell me about all your RCM bills? 

[00:43:43] Nikhil: Ayo, I feel like it's an ex, as an ex-FinTech guy, were they like, did they force you to do RCM? Did you like Go Cry? 

[00:43:51] Kerem: Ayo avoided it for as long as possible and now he is so deep in it.

[00:43:55] Nikhil: Of course I can, I can just tell by his facial re reaction when we brought this [00:44:00] up that he, he has PTSD from dealing with something in the RCM side. I don't know what it is. 

[00:44:05] Ayo: I didn't quite avoid it. I think like my healthcare experience is like, I came in thinking I would work on healthcare and mostly I worked on Covid up until like a year and a half ago.

[00:44:14] Kerem: Which is healthcare.

[00:44:15] Ayo: I love RCM. 

[00:44:16] Nikhil: You love RCM, I knew it! There's so much. There's so much. 

[00:44:20] Danielle: Let's go! So do I! Let's go. 

[00:44:22] Ayo: Like, it's so rich. It's the best. My hot, the hottest take that I have.... 

[00:44:27] Nikhil: I love it...

[00:44:28] Ayo: and I think this is like every, everything you see in healthcare, CM will make sense if you believe that what I'm about to say is true, which is Payers do not wanna pay you if you start there.

Everything else makes sense. The paper, the friction, the clearing houses, how no standards exist. How like, like, you know, we like as, as Kerem mentioned, we built this out of pot calculator and there's this like really funny thing where when you get a, a claim fully adjudicated, when you get it back, they tell you how much of the patient responsibility is like copay, co-insurance deductible, [00:45:00] etc.

And you look at it and then you look at what came back in the eligibility response and you're like, how are these not the same thing actually? And they're frequently not. And it is because Payers do not wanna pay you. They have no incentive to make sure they, the data's accurate. I don't, I don't think it's like a malicious thing.

I think it's just the incentive is that like not paying you is profitable and corporate entities do profitable things. 

[00:45:27] Nikhil: It's also, for what it's worth, that's not different than any insurance, right? That's not a health insurance thing right? That is like a true, that's fair.

[00:45:32] Ayo: That's fair, that's fair.

[00:45:33] Nikhil: That is every insurance.

[00:45:34] Ayo: But, but I think the nature, the nature of life is that you hope you don't use most insurances. 

The other types of insurance you use very infrequently, right? And I think this is like, the core tension is like life insurance auto, it's like a, just in case it happens. And it's probably one episode, right?

Yeah. Versus insurance for life 

insurance is definitely one episode. 

[00:45:52] Nikhil: Oh, I dunno, man, I'm Hindu. We have a whole different thing on this. 

Um, so, but, uh, [00:46:00] health insurance, it's like every visit you're, you're pulling teeth on like nickels and dimes that they're paying you. So I assume that's a big part of it.

Have you guys done stuff at, at Carbon to like, you know, work with We, your hot take is probably correct. So now we're working within that matrix that you've now uncovered for us. Uh, like what, uh, what have you guys done basically to like navigate through that?

[00:46:23] Ayo: So I wouldn't say we were like as good as we could be.

Like we, even,, even in the meeting Kerem and I just came from, at the end of the meeting I was like, Hey, there's these two things we need to do for RCM. And there's like a lot. So I, I would actually say we're good, not great on RCM. I would say there's probably like two or three key things. One is, there's a woman who'd been at Carbon for a long time named Ann Lee, who is incredible and took over RCM and basically, I dunno, I think like this woman is like one of the most pathologically effective people that I know.

And basically took the approach [00:47:00] of, like she didn't have an RCM background she organized these like Scrum teams that would go in and just look at a specific problem. And her team obviously is like a whole RCM organization of people who've been doing it for a long time.

And she'd be like, "just bring insight to me." And some insights are just, you gotta, you know, like workers' comp claims are all like paper manual. You just gotta like, put people on it. And some insights are like, oh, we don't have a real email for this patient that we've been sending emails for six months and that's why they haven't paid us.

So it's like, everything from, it's just everything from like, I'll give you actually like a really, really stupid example. I'm like, so shame to admit this actually. Um, 

we built this entire like revenue-cycle pipeline where you would walk in as a patient, we take your payment, we collect your credit card, and then we would send you emails, push notifications, texts, um, [00:48:00] to, get paid.

And we just ultimately, like, it just didn't work that well. And then we started sending people paper statments, and it was like, we were like, "how have we not been doing this for six years?" all the way to like really complex stuff. Like our RCM system is, is in-house and it's built on the same infrastructure as our EHR, which is built on the same infrastructure as our patient application there.

So like information doesn't ever have to like, hop from system system. It's all just like reads off the same database. On top of that, we've actually just been able to build, I would say, and I would, I, I would guess Kerem, keep me, keep me honest here, but I would guess we're maybe 40% of the way there and all the automation that we know to build, and it's everything from.

[00:48:41] Ayo: As of a month ago, we're now like calculating full patient responsibility upfront and collecting it from them. And that's all like automated and built-in. It's not, like, a separate application you have to log into. It just reads the, eligibility and, and and does it. We have a rules engine that like acts on denials and just will say, "Hey, this claim got [00:49:00] denied because we put the unique providers NPI, but this provider accepts group NPI as part billing.

So just like flip it to the group NPI and resubmit the claim." all the way to, like right now one of the areas that we're exploring is some automation on the, like, after a claim is adjudicated, there's this like crazy thing that can happen where a payer will come to you and say, Hey, we know we paid you $100 for that visit 18 months ago and we know we paid it to you a year ago, but actually we overpaid you by $99 So we need it back. And then the payer's just like, please, please write us check and send it within 14 days. And they send that as a physical piece of paper. Yeah. Um, and so we just have like, we're like, you, you, you can just imagine like, just think of it as every insight that we believe actually we can automate.

We are going to, and it's just we are really realistically, you know, a year max, 18 months into, into thinking about the world this way. [00:50:00] Karim, anything you would add? Like, what did I miss? 

[00:50:02] Kerem: 40 percent's probably too high even on how far in we are but yes. 

[00:50:10] Ayo: So brutal. 

[00:50:11] Danielle: I remember, uh, when we were, we were starting at Better Health.

We didn't know about the concept of a virtual mailbox and like first 50 claims that we submitted every day. My job was to go to the mailbox and open like a hundred pages of mail. And then when we went to the virtual mailbox, it turned into like a whole other mess and a half of like how to manage the virtual mailbox.

So now anytime somebody starts a company with claims, I'm like, please just get a virtual mailbox. Like it will at least make 1% of your job better. Yeah, yeah. Yeah. It's hardcore. All right. One last AI thing we, we promised we would talk about on this episode, which is the charting, the AI charting. This was a pretty big release of y'all, of y'all's recently.

Can you share a little bit about like what you did in that space and [00:51:00] what some of the wins are that, that you're most proud of there? 

[00:51:03] Nikhil: Also just like what is the, like metrics, what is the metrics of success for AI charting for you guys? I feel like it's like a, you know that too there, there's like a, you know, the lower the documentation time for the doc, but is it also like throughput on the clinic?

Like just curious, like how you're measuring, like if this is working or not. 

[00:51:20] Kerem: On the metrics success, it's definitely throughput as we think about it from a business. Um, however, there is. We do track number of edits made within a chart, quality of the chart. It's really important for provider satisfaction.

If, if we're increasing throughput and they're, but they're still having to go through and update that chart and do all the things, it's like less useful actually. It like to the point where providers who had to edit the chart on their first try actually had, I wanna say it was a 0% adoption throughout, we had to go back and come back to him and say, no, no, here's again how to go use it.

Maybe it's not quite zero. It was closer to zero than it was to 5% [00:52:00] that, but it was very, very low. But the, the thing that, that, that I like to look at, is definitely around throughput, on the staffing side for sure. But I'm, I'm also on the, the business end of it. My clinical counterpart, our chief Medical Officer, Dr. Sujal Mandavia, he, he's really keyed in on, like I said, the quality chart, the amount of time that it takes the provider to actually go through it.

And then the steps that are kind of like contracted into a single, instance within that. So it's not just the, charting that has to happen, but also the coding that happens after. And the care plan that's created within it. And any notes. So work notes and things with, work comp, et cetera.

And so, a lot of that just reduces complexity of. The operations variation from provider to provider. And we know, like we don't really have to worry about, the way with which we train providers on how to use the EHR for administrative use cases. We're really focused on, training up the providers on the quality of care aspect more than anything else.

So it, it has changed the way, like being complete. [00:53:00] Totally. it has changed the way we spend training time. There's a little bit of EMR training that happens. I would say it's like. It's a magnitude of closer to half a day than anything else. But we don't really have to worry about. Great. Now we have to educate them on, on what to do in coding and, and here's how you handle this work comp thing and here's how you go and do work notes here.

'cause it's different than how you do work notes somewhere else. And it's like this thing that's so simple, but also annoying for providers at the end of the day. But I'll let Ayo describe what the thing is, how it works and, and what it produces. Because I, I think he does a much better job at it than I do.

[00:53:31] Ayo: Actually, one question...

[00:53:32] Danielle: is he pulling it up right now? 

[00:53:34] Ayo: Would that be helpful to refresh? Remember? Would that be helpful actually to see it? 

[00:53:38] Nikhil: Sure. 

[00:53:39] Danielle: Yeah. Let's do it. First time! You can share your screen at the bottom. 

[00:53:43] Ayo: Yeah. I'm gonna let, lemme pull up

[00:53:44] Kerem: Test patient, ayo test patient! 

[00:53:46] Ayo: Yeah. No, no, no. 

[00:53:47] Danielle: We're gonna have to like Ai scribe it in parallel though, so that listeners who are on Apple. 

[00:53:54] Nikhil: This is gonna get too meta when we have like an Ai transcript scribe of ai.

You know? [00:54:00] 

[00:54:00] Ayo: Yeah. Okay, cool. This will do. Um, so...

this is, uh, one of my test patients. So basically, roughly, actually this is our provider app. Easy way to think about it is schedule boards, all the stuff Kerem was talking about on scheduling happens here. Messages, patient search. This is where we spend most of the time tasks.

So like a lab result comes in, who does it go to and like, why? Or does it automatically go to the patient, et cetera. All that stuff is here. And then like settings. and so in my test patient, I think we had a demo yesterday where we actually showed this. So, you know when, when you think of, like, you think of like what Nuance does.

So like what Nuance does is they, there's literally a recording device that just like sits on the provider's computer and they like tap it and then they dictate. To the device and they say, you know, you like, it runs through the provider's, like cognitive filter is the way I like to think about it. And [00:55:00] in the model that we've adopted is you go and have the conversation with your doctor that you want to have.

So you tell them a bunch of your problems, they ask you questions to find out related things they vocalize or articulate their, like they're thinking out loud in the process and they're like, okay, I examined this and this was, this was present or this wasn't present, et cetera. And we capture all of that context, like that entire audio, capture it, transcribe it, and then utilize an LLM to generate a bunch of stuff.

Wow. So in this particular case, it generates like this rich soap note that essentially is the like corpus of what's going to get placed in the patient chart. And then the other thing it does, I'm trying to see if I can show this. Yes. We, here we go, is we have this concept of a care plan and you can think of the care plan as our, encapsulation of all the actions that need to be done, either by a patient or on a patient's behalf in a visit. So a care plan will include any educational materials that need to be sent to the patient, any medications that need to be prescribed, orders [00:56:00] and tests, labs, imaging, follow up visits if you need to come back.

We have some automation where we can like send you a note after, after your prescription course is over and ask you, "Hey, are you okay?" And what I turn actually does for us is it generates, it basically takes the conversation and generates all that. So the provider says, "Hey, hey, do I need to write you a work note?"

And the, and the patient says, "Yes," it generates the work note. The provider says, Hey, I'm gonna prescribe you a sedaminophen. It'll generate that for the patient. And then all of that stuff is pushed into a queue that the provider reviews at the end of the visit and says, yes, no, yes, no, yes, no. And then the other thing that it does is, we built a long time ago this concept that we call like a charge navigator.

And you can think of it as like there's a bunch of clinical work a provider has to do for a visit, so they have to like, take care of you. And then there is some administrative work around coding. Like how, like is this level one or level two visit where, where they newer, established patient, etc. The way our scribe works is it takes the full context of the patient charts, like [00:57:00] everything we know about you historically, and then it takes the visit, like everything that happens is in the visit and it generates, like in this case, like it generates, Hey, this was, I was here for a problem-focused visit rather than a Medicare annual wellness visit as one example. And then it tells, it will suggest the E&M codes that should be used for the patient. It didn't in this case because I haven't filled out a bunch of stuff. But it generates the E&M codes that should be used for the patient and then the provider goes through selecting which ones they agree with and then that gets dropped into the claim.

 So that's, so, so like at, at a high level there's, I I would say like for at the moment we have like a couple of insights. So like, I'd say the big one is there is actually a lot of grunt work in being a provider. And a lot of our effort is in taking that away. And, and if we do our jobs right, what will happen is the provider's gonna spend their cognitive energy just doing clinical work and take care of patients.

And then like anything that's just like administrative or clerical work, we can, we can actually use [00:58:00] AI to, to, for lack of a better way to describe it, autofill. Um, I think the second, and I think this is like a long term insight that like, you know, just is gonna take some years to play out, is that, you probably can think of, the audAyoof the visit as kind of a new primitive actually.

So like if you think of like, if you think of any sort of longitudinal view of patient record today, it's all provider notes. And labs and tests and imaging, et cetera. And like what's happening now I think is that we, like, as of as of, you know, a year ago we started creating these records that are like exactly what the patient said in the visit and how they said it.

And you can imagine fast forward like a decade, imagine like if you have, if you have a chronic condition that like shows up in year nine, imagine being able to like run, you know, GPT-19 in 2029 on that entire history and having it both like due prediction or just basically [00:59:00] like, having it say, "Hey, was there anything that happened in Ayo's visits over the last 10 years that would've indicated that this condition would, would emerge?"

[00:59:10] Nikhil: I think people like, forget that a lot of ca like in a lot of cases the data that exists in healthcare today, I. Is filtered through the compression of what doctors think is important, right? Which is I come to the visit, I say this, all this stuff that I think is, you know, a problem. Yeah. The doctor picks the parts that are relevant to the kind of like encounter that they're putting in and then that is like becomes the truth record of truth, right?

Yeah. That can totally change. 

[00:59:37] Ayo: Actually, it's even worse. It's what doctors think is important. That was compressed through whatever XML or CCDA format, right? Yeah. Yeah. It's actually so lossy, it's insane. 

[00:59:47] Nikhil: Yeah, exactly. And if you start with the raw data as the premise of the entire encounter, totally new things can come up.

Yeah. I remember one of the interesting, um, one of the interesting things that happened during Covid is when the Johnson Johnson vaccine came [01:00:00] out, uh, one of the things people talked about was the blood clotting and one of the, um, symptoms. At, uh, people were talking about, our women were talking about missing their periods, but although these doctors just did not think it was relevant during the encounter because it wasn't showing that it showed up in the side effect profile during the vaccine trials.

And so they were just like, eh, it's like, it's probably just something else. Right. But it was happening sort of systemically and it was, but also, you know, to, to the docs. Not to the docs fault, it's just there was no like, uh, benchmark that you could use of being like, yeah, like a thousand patients have brought this up.

Right. Um, they have no way of knowing that. Right. They know it in, in an isolated thing. So the other secondary part of this is like, you can create this like global brain, a little bit of like all of the encounters at once, right? Where it's like, hey. You don't know this, but actually in these raw encounters, a ton of people have been bringing this up and we can now proactively like tell you that, right?

Which is sort of like interest, very different paradigm change for how, how you think about like, [01:01:00] uh, evidence, right? Because now it's not doc seeing evidence anecdotally, it's, it's, well we have actually raw data. It's systemic, but yeah. Um, well I know we're a little bit over time. Uh, we like to close our podcasts with basically three experiments that pe that you think people could run at their org today.

Um, if they wanted to just like, try something out that you think worked really well. I mean, you have like 1,000,002 x things. So maybe, uh, there's, you know, for people listening to this, like what are just some things, whether it's like org design stuff, whether it's like external facing...

[01:01:31] Danielle: In clinic... 

[01:01:31] Ayo: one thing I think we did almost accidentally that had a surprising amount of upside, is there's this like small pocket of clinicians who also are.

Technical thinkers or business thinkers. Yeah. So they have like both the clinical brain and then whatever the other thing is and having the right one that's culture fit in your organization and is aggressive are actually like, it's an incredible, you just see these things like they, they carry [01:02:00] things forward that are, that are so not obvious to you, but like are very obvious to them.

And the ability to simultaneously, you know, like execute on an initiative and then like be the provider in the clinic, delivering the care as part of that initiative actually is like insanely powerful. 

[01:02:17] Kerem: We actually, here at Carbon, a thing that we did is we kind of abandoned general patient acquisition just very specifically.

We, we doubled down on local. In a lot of different ways. Ways. And, it took both Aaron and I who spent a lot of our career in digital, direct to consumer patient acquisition, customer acquisition. It took a leap of faith on our part to say, "Hey, we're gonna actually not do these things that we know will work and we're now gonna go build up.

What, what would this look like if we were a single operator in a single clinic or a single provider? What would our marketing stack look like? And then how do we go layer those two things together?" So a thing that I would, say go out and try, is if you [01:03:00] have a digital means of patient acquisition, try turning it off and try doing some things that you would've done, say 15, 20 years ago.

Because those are the things that people aren't doing anymore. 

[01:03:08] Danielle: Billboards?

[01:03:09] Kerem: No, yeah. Maybe billboards, direct mail! Back to your patient, back to your patient...

[01:03:13] Danielle: Back to mail.

[01:03:14] Kerem: Mail. Just the human relationships we've built obviously have a lot of value in them. And it's kind of, you, you, you kind of forget in this digital first world that, that there is still value in, in word of mouth.

[01:03:24] Ayo: Yeah. 

[01:03:25] Nikhil: Totally, I love that. 

[01:03:26] Danielle: Any experiments, like inter-team experiments, folks can try for like, basically, um, I dunno, ideas for how to share feedback between product and ops or adoption and either team? 

[01:03:36] Kerem: Ayo calls me after I put my kids down when I have, uh, have given up on the day to throw a bunch of ideas out at me, knowing that I'm more likely to say yes at that point.

'cause I just wanna get off the phone. Anything else? So he catches me when I have my guard down and then the next day he'll have moved on it and it's usually working and I can't say no at that point. 

[01:03:58] Danielle: I love that tip. 

[01:03:59] Ayo: Okay, i, [01:04:00] I'll say I have one that I think is like, it's, it's, it's weird how valuable this is, but whoever your boss, partner, people, you execute with, colleagues, et cetera. Like it's so important to be able to disagree productively. Like it's, yeah, it almost, it matters more than almost anything else. Like if you actually ever, if you can't tell somebody something they don't like and have them like introspect on it as opposed to attack you or just like get defensive, it's like your cycle time's gonna be super slow.

Mostly because then people just won't tell you stuff. If it's just painful to like tell you a thing that you disagree with or that like something's going wrong, actually your cycle time just like slows dramatically. So if I was turn that into an experiment, I would say like, next time somebody tells you something you don't like, instead of being like juNPIng on your throat, just shut up.

Just like Danielle's... 

[01:04:52] Nikhil: I feel like Danielle and I should start fighting more on this podcast to like test the theory out. 

[01:04:58] Kerem: I, I, Ayo [01:05:00] probably tells me maybe Aaron actually does this more, but Ayo probably tells me more than anybody else I ever met. Like, "I kind of hate that idea." It's like just a starting point, but not a, to his point, it's not aggressive.

It doesn't come off that way. It's just more, yeah, you're gonna have to convince me on this one. So why this might work is not on board with that. Which we have, we have a lot of good discourse with one another. 

[01:05:22] Danielle: It seems like it.

[01:05:23] Nikhil: Yeah. I like the soft, I like the softness of that. Yeah. "I kind of hate that." Like, it's like a total head fake. Like, damn, you really, I thought really, really thinking about it, but like, no, you like really fucking hate it. That's funny. Well guys, thank you so much for coming. I know we covered a lot of ground on this. Um, uh, we appreciate you coming. For anyone who's listening, like I think there's a ton of different experiments you can take from that.

A lot of like really interesting things. Carbon, they've done like the work of like 10 different digital health companies under one roof, which is like very cool to see. Um, yeah, so definitely like take some stuff from this and try it, uh, in your own place. [01:06:00] But guys, thank you so much for coming, seriously. 

[01:06:02] Kerem: Thank you for having us. 

[01:06:05] Danielle: I think, uh, after this call, we should make our first swag for the podcast that says "pathologically effective." Because yeah, you said that about so good, about, about somebody on your team, and I was like, that has to be the highest compliment you can ever receive. 

[01:06:21] Nikhil: That's so good. So good. 

[01:06:23] Danielle: Every ops person... 

[01:06:24] Ayo: Once, once you, once you see it, you can't unsee it.

[01:06:27] Nikhil: That's amazing. I love that. 

[01:06:29] Danielle: I love that. Thank y'all. 

[01:06:31] Kerem: Thank you. 

‍