Generating clinical-quality ultrasounds on a smartphone

A startup co-founded by MIT alumnus Nevada Sanchez, Butterfly Network has the vision to make ultrasound imaging as simple as possible!

The aim is actually to make  ultrasound imaging as easy and ubiquitous as blood-pressure or temperature checks – in hospitals and, eventually, in consumers’ homes, to help health care professionals more quickly generate life-saving diagnoses.

The startup has developed a low-cost, handheld scanner, based in part on work done by co-founder Sanchez, that generates clinical-quality ultrasounds on a smartphone. Ultrasounds are uploaded to the cloud, where any expert with permission can give second opinions or help analyze images.

Traditional ultrasound machines rely on vibrating crystals and other components to produce ultrasound images. These are generally large, stationary machines that cost anywhere from $15,000 to $100,000. But the startup’s device, called iQ, which resembles an electric razor that plugs into an iPhone lightning jack, essentially puts an entire ultrasound system on a chip, meaning it’s portable and sells for about $2,000.

In November, the US Food and Drug Administration cleared the device for numerous clinical applications, including urological, abdominal, cardiovascular, fetal, gynecological, and musculo-skeletal. Tens of thousands of orders have been placed and will be shipped over the next few months.

“First users will be doctors and clinicians who are more comfortable with ultrasounds,” said Sanchez, now the startup’s chip design lead. “But, eventually, everyone from paramedics to nurses to doctors who have never used ultrasound will carry with them.”

A future aim is to sell directly to consumers, adds co-founder Jonathan Rothberg, a serial biotech entrepreneur. In the late 2000s, Rothberg moved DNA sequencing onto a semiconductor chip through his startup Ion Torrent, after pioneering the field of high-speed DNA sequencing at 454 Life Sciences – for which President Barack Obama awarded him the 2015 National Medal of Technology and Innovation.

Consider the consumerization of blood-pressure monitors and defibrillators, he said: “Those started in doctors’ offices and are now in people’s homes. In the next few years, we’ll work with the FDA, so anyone who wants a window into the human body can have it.”

In fact, the device has already proven valuable for consumer use: After joining the startup last year as chief medical officer, John Martin, a surgeon, was at a hospital testing the iQ. Having felt a lump in his throat for some time, he scanned his neck, which revealed a tumor that was then diagnosed as cancerous. He recently finished his first round of treatment.

“I actually became the first consumer patient,” Martin said. “That underscores how powerful having a device like this in hands of people themselves will be for the future of medical care. I’m the living evidence that … lives will be saved.”

Ultrasound on a chip

Ultrasound machines rely on transducers, small devices with embedded quartz crystals. Applying an electric current to the crystals makes them vibrate and produce sound waves travelling into a body. When returning echoes hit the crystals, they emit electrical currents that can be translated by electronics into an ultrasound image. The iQ’s low cost and accuracy is in part due to Sanchez’s work designing chips that function like the crystals but are manufactured at a drastically lower price point.

In 2010, Rothberg, inspired by his daughter’s struggle with kidney issues, was seeking ways to advance body imaging. At a conference, he heard a presentation from Max Tegmark, an MIT professor of physics who was working on connecting images from thousands of radio telescopes to measure energy from distant stars. The trick was developing a system – called a butterfly network – that split the data-processing between all antennae efficiently to capture quality images.

Rothberg approached Tegmark with a proposition. “I told Max that I’d love to use his ideas on imaging, and combine them with my recent success of DNA sequencing on semiconductors, to put an ultrasound machine on a chip,” Rothberg said. “I said I would start a company to do that … if he gave me smartest students from MIT.”

Enter Sanchez. After earning two bachelor’s degrees from MIT in electrical engineering and computer science and in mathematics, Sanchez was working in Tegmark’s lab. Rothberg and Sanchez met and founded Butterfly Network in 2011, and Sanchez then began the arduous task of integrating transducers directly onto chips. For help, the startup hired as chief technology officer Keith Fife, who had helped design the DNA-sequencing chips for Ion Torrent.

Rothberg also brought on board MIT alumnus, Kailiang Chen, who worked to enable the transmitting/receiving circuit to talk to the transducer. Jason Gavris joined in Butterfly Network in 2016 to help develop the iQ’s infrastructure – specifically to make it easy to control the device with a single thumb.

Today, the computation power of each chip is extraordinary, Sanchez said: “Essentially, it has almost half a trillion operations per second behind it to swallow data to get ultrasound in and image out in real time.”

After several iterations, Sanchez, Fife, and Chen found a way to integrate stacks of capacitive micromachined ultrasound transducers – basically, metal plates suspended between two electrodes – directly on a chip. (These devices function like crystal transducers.) From this, they created chips with roughly 9,000 transducer channels that, combined with electronics, could send out and receive sound waves and turn those waves into 3D ultrasound images.

User-friendly and universal

To make imaging more user-friendly, the device leverages artificial intelligence (AI) and augmented reality (AR). Typically, Sanchez says, it takes a lot of training to acquire a quality body image without much difficulty — but takes only minutes with iQ.

If a user has difficulty positioning the iQ, AI algorithms detect the probe’s location and recognize what the user is most likely trying to scan. AR symbols direct the user where to position the probe. “The user points the phone’s camera at the probe and they get a 3-D arrow telling them to move up or tilt it,” Sanchez says. “We’ve pulled people off the street and had them find a valid view of the heart in about a minute and a half.”

Another MIT alumnus, Alex Rothberg, who pioneered the deep learning at Butterfly Network, is now developing the artificial intelligence at another of Rothberg’s startups.

The iQ is also the first universal ultrasound device that can image an entire body. Crystals in ultrasound devices resonate at one narrow frequency tailored to individual areas. A user will need one probe to capture, say, a patient’s veins and another to image the heart or kidney. “These pieces have narrow bandwidths tailored to each application,” Sanchez said. “But our device has a very broad bandwidth that can essentially become any other probe at any time.”

Currently, the iQ is selling only in the U.S. But Butterfly Network is in talks with nonprofits, such as the Bill and Melinda Gates Foundation, to bring it to the developing world. In many remote areas, clinicians don’t have access to ultrasound technologies that can be used to, say, detect fetal health issues that cause women to die in childbirth.

“This is a perfect solution,” Sanchez said. “Because it’s low cost, we can put in the hands of doctors in remote locations and instantly save thousands of lives. But, I know there will also be other areas in critical care and in emergency rooms around developed and developing worlds where people’s lives will be saved because of what we built. For me, that’s motivating.”