Google Research has introduced PhotoScan, a deep learning model that estimates body composition and metabolic health risk directly from smartphone photographs, potentially offering clinic-quality health assessments without expensive medical equipment. The system was trained on over 35,000 participants from the UK Biobank using MRI and DXA scan data, then fine-tuned and validated on separate clinical cohorts totaling nearly 900 participants. PhotoScan measures three key metrics—body fat percentage, android-to-gynoid fat ratio, and visceral-to-subcutaneous fat area ratio—that correlate strongly with insulin resistance, a major precursor to type 2 diabetes and cardiovascular disease.
In clinical validation tests, PhotoScan demonstrated body fat prediction accuracy comparable to DXA scans, the current gold standard for measuring body composition, while outperforming bioelectrical impedance analysis sensors commonly found in smartwatches. More significantly, the AI model calculates fat distribution ratios that BIA sensors cannot measure, providing a more complete metabolic risk assessment. Google researchers reported achieving a mean absolute error of just 2.13 for body fat percentage on independent validation data, nearly matching its 2.15 error on training cohorts.
The research addresses a critical gap in accessible health screening. DXA scans remain expensive, require specialized equipment and trained technicians, and expose patients to ionizing radiation. By enabling accurate metabolic risk assessment through smartphone photography, PhotoScan could democratize early detection of insulin resistance and metabolic dysfunction, conditions that often go undiagnosed until progressing to type 2 diabetes or cardiovascular disease.
Key Points
Google developed PhotoScan, a deep learning model that estimates body composition from smartphone photos with accuracy matching clinical DXA scans
The system can predict insulin resistance by analyzing body fat distribution patterns that standard fitness trackers cannot measure
Trained on 35,000+ UK Biobank records and validated on independent clinical cohorts, demonstrating consistency across diverse populations
PhotoScan could enable accessible metabolic health screening without expensive, radiation-exposing clinical equipment
Measures visceral and subcutaneous fat ratios that predict cardiovascular and metabolic disease risk better than BMI alone