Facebook
Facebook and NYU School of Medicine’s Department of Radiology are excited to announce fastMRI, a new collaborative research project that will investigate the use of artificial intelligence (AI) to make magnetic resonance imaging (MRI) scans up to 10 times faster. If this effort is successful, it will make MRI technology available to more people, expanding access to this key diagnostic tool.
MRI scanners provide doctors and patients with images that typically show a greater level of detail related to soft tissues — such as organs and blood vessels — than is captured by other forms of medical imaging. But they are relatively slow, taking anywhere from 15 minutes to over an hour, compared with less than a second or up to a minute, respectively, for X-ray and CT scans. These long scan times can make MRI machines challenging for young children, as well as for people who are claustrophobic or for whom lying down is painful. Additionally, there are MRI shortages in many rural areas and in other countries with limited access, resulting in long scheduling backlogs. By boosting the speed of MRI scanners, we can make these devices accessible to a greater number of patients.
Sufficiently accelerated MRI devices could also reduce the amount of time patients must hold their breath during imaging of the heart, liver, or other organs in the abdomen and torso. Increased speed could let MRI machines fill the role of X-ray and CT machines for some applications, allowing patients to avoid the ionizing radiation associated with those scans.
Challenges in advanced medical imaging
This project will initially focus on changing how MRI machines operate. Currently, scanners work by gathering raw numerical data in a series of sequential views and turning the data into cross-sectional images of internal body structures that doctors then use to evaluate a patient’s health. The larger the data set to be gathered, the longer the scan will take.
Using AI, it may be possible to capture less data and therefore scan faster, while preserving or even enhancing the rich information content of magnetic resonance images. The key is to train artificial neural networks to recognize the underlying structure of the images in order to fill in views omitted from the accelerated scan. This approach is similar to how humans process sensory information. When we experience the world, our brains often receive an incomplete picture — as in the case of obscured or dimly lit objects — that we need to turn into actionable information. Early work performed at NYU School of Medicine shows that artificial neural networks can accomplish a similar task, generating high-quality images from far less data than was previously thought to be necessary.
In practice, reconstructing images from partial information poses an exceedingly hard problem. Neural networks must be able to effectively bridge the gaps in scanning data without sacrificing accuracy. A few missing or incorrectly modeled pixels could mean the difference between an all-clear scan and one in which radiologists find a torn ligament or a possible tumor. Conversely, capturing previously inaccessible information in an image can quite literally save lives.
The Facebook and NYU collaboration
NYU School of Medicine, a department of NYU Langone Health, has a long-standing history of pushing the boundaries of medical research and education to benefit the lives of patients. The Radiology Department’s Center for Advanced Imaging Innovation and Research (CAI²R) includes a multidisciplinary team of engineers, physicists, mathematicians, radiologists, and other clinicians and scientists with key expertise in rapid image acquisition, parallel imaging, and advanced image reconstruction. Their focus is on developing novel imaging technologies and rapidly translating those technologies into clinical practice.
Since 2016, CAI²R investigators have been pursuing the idea of utilizing AI to achieve faster MRI scans. Early studies have suggested that scan times could be reduced by an order of magnitude or more. Realizing those potential gains, however, would require additional AI knowledge and large-scale computing resources.
Around the same time, the Facebook Artificial Intelligence Research (FAIR) group, which focuses on open and foundational research that advances the state of AI, was looking for projects in which AI could have significant real-world impact. CAI²R’s image reconstruction work fit those criteria and provided FAIR with an opportunity to combine its deep learning expertise — particularly in the field of computer vision — and its ability to train models at large scale with the medical school’s leading imaging science expertise.
Project data details
The imaging data set used in the project, collected exclusively by NYU School of Medicine, consists of 10,000 clinical cases and comprises approximately 3 million magnetic resonance images of the knee, brain, and liver.
All data, including both images and raw scanner data, are fully stripped of patient names and all other protected health information. The work is fully HIPAA-compliant and approved under NYU Langone’s Institutional Review Board, which oversees all human subject research at the medical center. The project is governed by strict human subject data protection protocols and supported by the world-class information technology team at NYU Langone.
The magnetic resonance images (which generally represent small targeted regions of anatomy) used for this project have been scrubbed of any potential distinguishing features. Comparisons of the performance between AI-based reconstructions and traditional reconstructions will, likewise, be devoid of any identifying information. No Facebook data of any kind will be used in the project.
