“Siri, where is the nearest Starbucks?”

“Alexa, order me an Uber.”

“Suki, let’s get Mr. Jones a two-week run of clarithromycin and schedule him back here for a follow-up in two weeks.”

Doesn’t sound that crazy, does it? For years, voice assistants have been changing the way people shop, get around, and manage their home entertainment systems. Now they’re starting to show up someplace even a little more personal: the doctor’s office. The goal isn’t to replace physicians with sentient speakers. Quite the opposite. Drowning in a sea of e-paperwork, docs are quitting, retiring, and scaling back hours in droves. By helping them spend more time listening to patients and less time typing into electronic health records, voice assistants aim to keep physicians from getting burned out.

It’s a problem that started when doctors switched from handwritten records to electronic ones. Health care organizations have tried more manual fixes—human scribes either in the exam room or outsourced to Asia and dictation tools that can only convert text verbatim. But these new assistants—you’ll meet Suki in a sec—go one step further. Equipped with advanced artificial intelligence and natural language processing algorithms, all a doc has to do is ask them to listen. From there they’ll parse the conversation, structure it into medical and billing lingo, and insert it cleanly into an EHR.

“We must reduce the burden on clinicians,” says John Halamka, chief information officer at Boston-based Beth Israel Deaconess Medical Center.¹ He’s been conducting extensive early research around how Alexa might be used in a hospital, to help patients locate their care team or request additional services, for example. “Ambient listening—the notion that technologies like Alexa and Siri turn clinician speech and clinician-patient conversations into medical records—is a key strategy.”

Alexa and Siri might be the best known voice assistants, but they’re not the first ones doctors are trusting with their patients. While Amazon and Apple are rumored to be working on voice applications for health care, so far they’re still piloting potential use cases with hospitals and long-term care facilities. They don’t yet have any HIPAA-compliant products on the market.

Not so for Sopris Health, a Denver-based health intelligence company that launched today after starting to roll out its app at the beginning of the year. You don’t summon a name to turn it on, just tap it when you want it to start listening. It automatically converts the audio to free text, then turns that speech into a doctor’s note, thanks to hours of training data from actual doctors’ visits. So “I think I’d like to see you again if things aren’t feeling better within a few days,” becomes “Schedule three-day follow-up.” Or, “We’re going to need to get an MRI of that left knee to figure out what’s going on in there” becomes “Order left knee MRI.”

Much in the same way that Google’s neural networks learned that cats and dogs are different animals that people like to keep as pets, Sopris’ algorithms learned to use context clues to pull out the medically actionable parts of a conversation. A cardinal number becomes an interesting feature—maybe it’s a calendar date or the dose of a medication. The words around it help the app decide to schedule a follow-up or order a prescription. And because it integrates directly with the EHR vendor, no separate orders or emails or phone calls are necessary: You just hit a button.

By doing so, physicians assume responsibility (and liability) that everything in it is correct. Which might sound like a leap of faith, but Sopris CEO and co-founder Patrick Leonard says is actually a positive feature. “What’s really cool is it’s changing physician behavior in a good way,” he says. “The app forces them to practice active listening, double-checking with patients that they got everything right. Which they actually have time for, now that they’re not sitting at a computer for six hours a day.” And if the assistant gets anything off, doctors can manually overwrite it.

Sopris plans to eventually move beyond orthopedics into other specialties; it’s currently in talks with a large children’s hospital about creating a pediatrics module. Another clinical voice company also launching today has even bigger plans. With $20 million in funding and stacked with engineers from Google and Apple, Redwood City-based Suki unveiled its AI-powered digital voice assistant this morning. Former Googler Punit Soni founded the company a year ago (it was originally called Robin), and has since launched a dozen pilots in internal medicine, ophthalmology, orthopedics, and plastic surgery practices in California and Georgia. Preliminary results from the company show Suki cuts physician paperwork by 60 percent.

For now, the app still needs some hand-holding. You have to say “Suki, this patient is 67 years old,” and “Suki, we need to order a blood test.” That’s because Soni’s team gave it just enough seed data to survive. But eventually, with enough data flowing through its neural nets, doctors will be able to say simply, “Suki, pay attention.” And then it’s on to tackling bigger problems.

“We’re starting with documentation, but then we can apply the same methods to billing and coding, and other higher order architectures,” says Soni. Things like prescription management, and maybe even decision support—an algorithm whispering hints in your doctor’s ear about a care plan. “I think it’s unreasonable to imagine that 10 years from now doctors will still be using clunky 1990s-style UI to take care of patients,” says Soni.

The health care system has long been impervious to this kind of disruption. But as deep learning gets even better, these kinds of assistants begin to look more plausible. The space is filling up rapidly; last year a third startup, SayKara, helmed by former Amazon engineers, announced it was developing its own Alexa for health care. Others are sure to follow. And that’s when lawyers focused on privacy and cybersecurity start to get concerned. “When you’re talking about AI in the health care space, the appetite to capture more and more data becomes insatiable,” says Aaron Tantleff, a partner at Foley and Lardner law firm in Chicago. He points out that one of HIPAA’S key privacy protections is a rule that says businesses should only collect the minimal amount of information that is necessary. It’s a provision that is fundamentally at odds with data-hungry neural networks.

Voice assistants also raise questions about unauthorized disclosures in the exam room. “We already know these listening devices can get hacked and allow third parties to record conversations,” says Tantleff. “In a medical setting, there’s a very different level of risk. What are companies doing to prevent that from happening?”

Both Suki and Sopris recognize the significant privacy and security considerations involved with their products. The companies encrypt audio on the device, and in between transit to HIPAA-compliant clouds where their algorithms run. And both apps require a prompt from someone in the room to enable listening. Plus patients have to opt-in; docs can’t just record people who don’t consent. The potential benefit to physicians seems clear. The tradeoff for patients less so. Then again, if you want to keep your doctor around for the long haul, maybe it’s worth asking, “Suki, can you keep my data safe?”

¹ Disclosure: Halamka was also formerly a member of Suki's advisory board

The Algorithm Will See You Now

• When time is brain, AI can help stroke patients get better care, faster.

• Google has developed software that can detect early signs of diabetes-related eye problems, and is testing it in eye hospitals in India.

• To keep pace with all the new ideas for using computers and machine learning in health care, the Food and Drug Administration had to create a new team of digital health experts.

Almost exactly a year ago, 23andMe earned the right to tell people what diseases might be lurking in their DNA. Since then, the consumer genetic testing company has turned tubes of spit into health reports for thousands of its customers. You can learn how your genes might predispose you to eight diseases with a well-known genetic component—things like Parkinson’s, Alzheimer’s, and most recently, breast and ovarian cancers.

But these limited genetic red flags are rare enough that for most people, there’s not much for 23andMe to report back.

Lots of people, though, get migraines. And allergies. And depression. 23andMe says it wants to help them, too—not by extracting insights from their DNA, but by harvesting the wisdom of the crowd. For the last few weeks, the company has been quietly rolling out a new health hub, where customers can share information about how they manage 18 common health conditions. They get to see which treatments work best, according to other users’ personal reports. And 23andMe gets a bunch of data it didn’t have before.

It’s not hard to see who’s getting the better side of the deal.

Each condition page provides some information unique to 23andMe, says product manager Jessie Inchauspe. She highlights how customers can look at the prevalence of a given condition among their spit kit sisters and brothers. Based on the millions of 23andMe customers that consented to participate in research, 27% have self-reported having depression. Most of them were diagnosed by their 30th birthday. And any kids they have will be 20 percent more likely to develop depression themselves.

Unlike the company’s health reports though, the conditions pages won’t tell you how likely your genes are to give you depression, just how much of depression generally is attributable to DNA, according to the company’s data and its reading of the scientific literature. A disclaimer toward the top makes this plain: “This content is NOT based on your genetics. It may not be representative of the general population or of you as an individual.”

The same is true of the treatment ratings: Customers can sort them by reported efficacy and popularity, but not by their own genotype. 23andMe says it does have plans for adding an ethnicity filter at some point—certain drugs can be more or less effective depending on your heritage—but right now there’s nothing personalized about it.

It’s also misleading. “Normally I think 23andMe does a really nice job visually representing genetic risks, but this model brings up some real interpretation concerns,” says Kayte Spector-Bagdady, a bioethicist at the University of Michigan and a former associate director of the Presidential Commission for the Study of Bioethical Issues. The problem, she says, is that people are being asked what treatments they’ve tried and how effective each one is. But that’s not the same as a comparative effectiveness trial. “If I say I have depression and all I ever tried was Zoloft and I had moderate improvement, it doesn’t mean Zoloft was better for me than exercise or Wellbutrin,” says Spector-Bagdady. But on the new pages, colored bars that display reported efficacy of treatments side by side suggest otherwise. “It’s hard for any individual consumer to understand what this information means for them.”

If 23andMe customers want to compare treatments, they don’t have to log on to the company’s health hub to do so. Iodine, a startup co-founded by former WIRED executive editor Thomas Goetz that merged with drug pricing transparency company GoodRx, crowdsources patient reviews and presents them alongside clinical trial data and input from pharmacists. HealthTap’s RateRx app lets doctors from all over the world rate the effectiveness of certain medications for certain ailments. Even Google has been working with the Mayo Clinic to create a database of commonly searched medical conditions and their most frequently used treatments.

So why should 23andMe’s customers turn to them rather than the wilds of the internet for health advice? “We have a nice, closed platform where people feel safe,” says Inchauspe, pointing out that about 80 percent of the company’s 5 million customers consent to participate in research. That means research that 23andMe’s 60 staff scientists do internally, as well as outside studies with data the company shares with academic institutions and sells to pharmaceutical firms. “That gives us an opportunity to crowdsource unique data that just doesn’t exist anywhere else.”

23andMe does have data that other treatment comparison companies don’t: DNA. Theoretically, pairing its massive genetic databases with reports of treatment efficacy could help the company take steps toward offering precision medicine solutions: treatments tailored to your DNA. But for now, that’s not information it can easily share with its customers, at least in the US, on account of federal regulations that treat pharmacogenetic testing—how genes influence someone’s sensitivity to different drugs—as a medical device.

When asked, the company said it has no immediate plans to turn the health hub data into genetic reports. “We view this as a separate product,” says Inschauspe. But 23andMe has already shown its interest in pharmacogenetic testing. In 2014, the company introduced 12 such tests to its customers in the UK—though it stopped offering them in 2017 to make its product uniform on both sides of the pond. But if 23andMe ever plans to bring them back, a little (or a lot) more data certainly won’t hurt.

More Personal Genomics

• Direct-to-consumer genetic tests are more popular than ever. Last year, Ancestry DNA sold 1.5 million spit kits over a four-day period.

• Over on Helix's DNA marketplace, you'll soon be able to request your own clinical tests for 59 disease-causing genetic mutations.

• Upstart Genos takes a different tack, offering financial incentives to its customers for donating their exome sequence data to science.

04/20/18 3:50pm ET This story has been updated to reflect the most up-to-date numbers for how many of 23andMe's consented research participants experience depression; it is 27 percent, not 45 percent, as a previous version of this story stated.

Related Video

Science

Crispr Gene Editing Explained

Maybe you've heard of Crispr, the gene editing tool that could forever change life. So what is it and how does it work? Let us explain.

I know it's an old movie (and it was an even older book before that), but I want to look at the physics of the special submarine drive in The Hunt for Red October. In the story, the Russians build a so-called "caterpillar drive" using hydro-magneto power instead of the traditional propeller. This new drive is way quieter than the traditional type—so quiet that it could sneak up on the United States and blow it up. Spoiler alert: It doesn't.

Here is the cool part: This magnetohydrodynamic drive, which turns water into a sort of rotor, is a real thing. (Although technically in the book version this drive is something other than magnetohydrodynamic. Quibbles.) In fact, it's pretty simple to build. All you really need is a battery, a magnet, and some wires. Oh, also this will have to operate in salt water, so you might need some salt. Here is the basic setup.

How does this work? Well, when you put a positive and negative plate in salt water, it produces an electric field. With salt in water, you get both positive and negative ions—both of these are influenced by the electric field. In the setup shown above, the negative ions would move to the right and the positive ones move to the left. But the ion motion by itself does not produce any propulsion. For that, you also need a magnetic field.

In the diagram, I have a magnet with the north side pointing down. This produces a magnetic field that also mostly points down (as indicated by the red arrow). Now for the awesome physics part. If you have an electric charge moving in a magnetic field, there is a force on that charge—the magnitude of this force depends on the strength of the magnetic field, the value of the electric charge, and the velocity of the charge. This magnetic force can be expressed as the following equation:

If don't have a degree in physics, there are three things that are crazy about this equation. First, there is this weird arrow symbol over some of the variables. Nothing to be alarmed about—this just means these are vector quantities so that the direction also matters. Next there is this vector B. This represents the value of the magnetic field. Honestly, I'm not sure why we (physicists) always use B for the magnetic field—but we do. Lastly, there is that big "X". That is not the sign for multiplication, that is the sign for the cross product. I guess I should also point out that "q" is the symbol for the electric charge.

Multiplication is for scalar quantities—things that don't have direction. So if you want to operate two vector quantities (the velocity and the magnetic field) then you need a different operator (by operator, I mean actions like addition or square root or stuff like that). The cross product operator takes two vectors and produces another vector. The resultant vector depends on both the magnitude and direction of the starting vectors. But for this explanation, the important idea is that the result is a vector that is perpendicular to both of the initial vectors. This means that you have to see this thing in three dimensions in order to grok it.

Maybe this python script will help. Below are three arrows representing the three vectors dealing with the magnetic force. I have labeled the three vectors, so it should be clear which arrow represents which variable. But wait! There are two things you can do. First, you can rotate these three vectors around and view them from different angles—just click-drag with the right mouse button or use ctrl-click. Second, you can change the magnitude and direction of the qv vector by just normal click (or click drag). Go ahead and try it.

You should notice that no matter what you do, the vector F is always perpendicular to both qv and B. OK, that's not quite true. If you change qv such that it is parallel to B, the force would be zero—with no defined direction. If you want to determine the direction of the resultant from a cross product, you need to use the "right hand rule." Here is an older post that goes over the details—just in case you need that.

OK, going back to the hydro-magneto drive. You might notice one problem—some of the ions in the water are moving to the left (positive charges) and some are moving the right (negative charges). However, both the positive and negative ions will be pushed in the same direction (the direction out of the screen). The negative ions are moving in the positive x-direction, but they have a negative electrical charge. This means that they will still have a qv value in the same direction and the magnetic force on these two different ions are still in the same direction.

Now for an actual demonstration. I didn't build this from scratch, but found it as a kit. In this version, there is a circular track for the water so that you don't have to actually go anywhere. I put a drop of blue dye in the water so you can see when it moves. Here is the basic setup.

Once you connect the battery to the two plates—boom, the water starts to move.

I don't know about you, but I think this is pretty cool. Also, if you flip over the magnet the water changes directions. You could also change the water direction by reversing the electrical current. But if this is such a great physics demonstration, why don't they use this for propulsion systems? In short, it doesn't work very well. Sure, the water gets pushed—but you could do much better with just a propeller.

Related Video

Science

Electrical Energy Flow: How it Works

Electrical Energy Flow: How it Works

San Francisco, land of unrestrained tech wealth and the attendant hoodies and $29 loaves of bread, just said whoa whoa whoa to delivery robots.

The SF Board of Supervisors voted on Tuesday, December 5 to severely restrict the machines, which roll on sidewalks and autonomously dodge obstacles like dogs and buskers. Now startups will have to get permits to run their robots under strict guidelines in particular zones, typically industrial areas with low foot traffic. And even then, they may only do so for research purposes, not making actual deliveries. It’s perhaps the harshest crackdown on delivery robots in the United States—again, this in the city that gave the world an app that sends someone to your car to park it for you.

Actually, delivery robots are a bit like that, though far more advanced and less insufferable. Like self-driving cars, they see their world with a range of sensors, including lasers. Order food from a participating restaurant and a worker will load up your order into the robot and send it on its way. At the moment, a human handler will follow with a joystick, should something go awry. But these machines are actually pretty good at finding their way around. Once one gets to your place, you unlock it with a PIN, grab your food, and send the robot on its way.

Because an operator is following the robot at all times, you might consider the robot to be a fancied-up, slightly more autonomous version of a person pushing a shopping cart. “But that's not the business model that they're going after,” says San Francisco Supervisor Norman Yee, who spearheaded the legislation. “The business model is basically get as many robots out there to do deliveries and somebody in some office will monitor all these robots. So at that point you're inviting potential collisions with people.”

Unlike self-driving cars, or at least self-driving cars working properly, these bots roll on sidewalks, not streets. That gives them the advantage of not dealing with the high-speed chaos of roads, other than crossing intersections, but also means they have to deal with the cluttered chaos of sidewalks. Just think about how difficult it can be for you as a human to walk the city. Now imagine a very early technology trying to do it. (Requests for comment sent to three delivery robot companies—Dispatch, Marble, and Starship—were not immediately returned.)

More on Delivery Bots

• Matt Simon

  San Francisco Tries to Ban Delivery Robots Before They Flatten Someone's Toes

• Matt Simon

  The Crazy-Hard Technical Challenges of Robots Delivering Falafel

• Alex Davies

  This Delivery Robot Isn’t Just Charming. It’s Stuffed With Pizza

What happened in that Board of Supervisors meeting was the manifestation of a new kind of anxiety toward the robots roaming among us. Just this last year has seen an explosion in robotics, as the machines escape the lab (thanks in part to cheaper, more powerful sensors) and begin rolling and walking in the real world. They've arrived quickly and with little warning.

And that’s made folks both curious and uneasy. Go to a mall and you may well find a security robot scooting around keeping an eye on things. Robot nurses roam the halls of hospitals. Autonomous drones fill the air. The question is: How are we supposed to interact with these machines? It’s a weird and fundamentally different kind of relationship than you’d form with a human, and not even experts in the field of human-robot interaction are sure how this is going to play out.

The big thing is safety. Machines are stronger than us and generally unfeeling (though that’s changing with robots that have a sense of touch), and can be very dangerous if not handled correctly. Which is what spooked Yee. San Francisco’s sidewalks are bustling with pedestrians and runners and homeless people and dogs and the occasional rat stacked on a cat stacked on a dog. How can the city make sure that roaming delivery robots and citizens get along?

For San Francisco, that means a crackdown. The legislation will require delivery robots to emit a warning noise for pedestrians and observe rights of way. They’ll also need headlights, and each permittee will need to furnish proof of insurance in the forms of general liability, automotive liability, and workers’ comp.

It’s sounds so very un-Silicon Valley. You know, move fast and break things, potentially literally in the case of the delivery robots. But states including Idaho and Virginia have actually welcomed delivery robots, working with one startup to legalize and regulate them early. Though really, San Francisco can better afford to put its foot down here—it’s not like it’s hurting for startups to come in and do business.

Might that seem like San Francisco isn’t as tech-friendly as it may seem? No, says Yee. “If you want to approach delivery, figure out how to do it and be as compatible with our values here,” he says. “Could robots do other things, for instance? Could it be that somebody's accompanying a robot that's picking up used needles in the street?”

If only Silicon Valley wasn't so busy developing parking apps.

Related Video

Business

The Robot That's Roaming San Francisco's Streets to Deliver Food

Hungry? But you don't want to deal with a human? If you live in San Francisco's Mission district, you can get your food delivered by a robot named Marble.