TongueML: A multi-task learning model for multi-attribute tongue image classification based on graph convolution and transformer

Abstract

Tongue diagnosis is an important diagnostic method in Traditional Chinese Medicine (TCM), which assesses health conditions through tongue features such as color, shape, and coating. However, it is often subjective and affected by external factors, limiting its standardization and objectivity. To address this issue, this study proposes TongueML, a multi-task learning model for multi-attribute tongue image classification. Each tongue image is annotated with five diagnostic attributes: tongue body color (shezhi), tongue body shape (shexing), tongue coating (shetai), tooth marks (chihen), and cracks (liewen). TongueML uses a pre-trained EfficientNetB0 for feature extraction, a cross-scale Transformer for local–global feature fusion, and a graph convolutional network to model spatial feature relationships. Experimental results show that TongueML outperforms baseline models, achieving a Macro F1-score of 74.56%. These results suggest that TongueML provides a promising approach for objective and standardized multi-attribute tongue image analysis and may support intelligent tongue diagnosis.

The authors' abstract, as published at the source. Biomedical Signal Processing and Control, 2026 · DOI ↗