Researchers at Purdue University (West Lafayette, IN) have developed a learning-based spatio-spectral imaging process to detect and diagnose medical conditions faster and more accurately than conventional hyperspectral systems (see video).
“Our learning-based method takes advantage of hyperspectral learning,” says Young Kim, a professor and associate head for research at Purdue’s Weldon School of Biomedical Engineering. “This enables the recovery of spectral information from red, green, and blue (RGB) color values acquired using a smartphone camera.”
A smartphone camera can only capture RGB wavelengths in each pixel in a photo, limiting its medical applications. But hyperspectral imaging allows users to capture all wavelengths of visible light in each pixel. With its ability to produce clear, in-depth images across all spectrums, hyperspectral imaging is ideal for detecting ailments such as skin conditions and potentially some cancers.
The challenge? When used alone, hyperspectral systems are slow, bulky, and expensive to operate. There is also a fundamental tradeoff between spatial and spectral resolutions, as well as slow rates of data acquisition. “The intrinsically slow data acquisitions have been the bottleneck,” Kim says. “It’s been impossible to acquire a large amount of hyperspectral image data in an instantaneous manner. As a result, previously snapshot hyperspectral imaging systems and methods heavily rely on complex and bulky instruments. Such approaches are fundamentally limited.”
Combining hyperspectral imaging techniques with a smartphone camera’s numerous sensors, inexpensive operation, and ability to produce multiple images in just seconds speeds up the approach and makes it more economical. And the spectral resolution that’s ultimately achieved is comparable to scientific spectrometers in a range of 1 to 2 nm.
Further boosting the Purdue team’s approach is a machine learning-based model and algorithm that transforms the RGB values from a smartphone-generated photo into hyperspectral image data—also known as hypercube—instantaneously.
“Our inspiration came from the idea that a photograph is more than merely a picture. It contains abundant hyperspectral information,” Kim says. “Our learning-based approach serves as an example that machine learning (statistical or deep learning) can minimize hardware complexity—opening the possibility of hyperspectral imaging using an off-the-shelf smartphone.”
Testing the approach
Several studies using the Purdue team’s patent-pending mobile health (a.k.a. mHealth) approach and technologies are progressing, including work to improve malaria diagnoses and management among school-aged children. The goal is to empower community health workers and healthcare facilities where patients are suffering from or are at risk of diseases like malaria, which is one of the leading causes of death among children, particularly in Sub-Saharan Africa—an estimated 200 million children are at risk of contracting the disease.
“This project will not only provide mHealth-assisted malaria diagnostics and treatments among school-age children at scale, but will also allow digital reporting and collection of patient-level data,” Kim says.
The team’s research also concentrates on assessing blood hemoglobin—a protein in the body’s red blood cells that delivers oxygen to organs and tissues. It transports carbon dioxide from organs and tissues back to the lungs, as well.
“Successful completion of this study will lead to the development of mHealth applications to accurately predict the blood hemoglobin levels,” Kim says. The predictions and measurements are being gathered from a digital photo of the inner eyelid captured using a smartphone camera in a fully automated manner (see figure).
The researchers take advantage of “informed learning” approaches with no additional attachments. “This mobile health application could be fully integrated with an existing electronic health record system in low-resource settings,” says Kim.
The Purdue team is currently looking into applying their hyperspectral learning approach to cervix colposcopy—a procedure for imaging cancerous and other abnormal cells or other health conditions—as well as retinal fundus (the interior surface of the eye opposite the lens) imaging.
Other possibilities
Hyperspectral learning enables ultrafast dynamic imaging, leveraging the ultraslow video recording feature that’s now available on smartphones. Newer smartphone models can capture slow-motion videos as high as 960 to 1920 frames per second, which is even faster than most scientific cameras.
A smartphone video comprises a time series of multiple RGB images, Kim explains, noting each individual frame is an RGB image. These could potentially be transformed into a hemodynamic video, showing blood flow within the body’s organs and tissues, after a learning algorithm is trained.
“We’ve already demonstrated fast imaging with an unprecedented temporal resolution of 0.001 seconds,” Kim says. “This is the world’s fastest hemodynamic imaging, to the best of our knowledge.”
Maximizing advantages
Along with a growing number of potential applications, the team says its learning-based spatio-spectral imaging approach has advantages in its hardware simplicity, it offers high temporal resolution, and there is no known tradeoff between spatial and spectral resolutions.
“We all have seen numerous mHealth applications that require additional accessories and bulky components as a mandatory attachment to the smartphone,” Kim says. “Our method can utilize the built-in camera in a smartphone. This technology can potentially maximize the quality of currently available standards of care and minimize complicated hardware for mobile health solutions, offering mobility, simplicity, and affordability for rapid and scalable adaptation.”
The technology may even pave the way for mobile health implementation in resource-limited or at-home settings.
