Open-data machine learning framework reaches near-perfect breast cancer

Researchers in China have created an open-data framework that other researchers can repeat and check. It combines machine learning, whose decisions can be explained, with measurements of cells to help tell cancerous breast lumps from non-cancerous ones. The researchers also designed the process as a teaching resource for medical artificial intelligence. Min Li of the First Affiliated Hospital of Anhui Medical University and colleagues used a public dataset called Wisconsin Diagnostic Breast Cancer (WDBC). It contains 569 digitized samples collected using fine-needle aspiration, a method that uses a thin needle to collect cells. The team tested seven supervised machine learning methods using a careful validation process with several layers. The results were almost perfect on this dataset. A support vector machine using a radial basis function reached an area under the receiver operating characteristic curve (ROC-AUC) of 0.995. An easier-to-interpret linear support vector machine scored almost as highly, at 0.994. Published in Frontiers in Oncology, the study focuses on transparent testing, checking how well predicted probabilities match real outcomes, and examining how different decision thresholds affect results, rather than relying on one headline accuracy score.

The dataset used in the study is a well-known test collection containing 30 continuous measurements of cell nuclei from digitized fine-needle aspiration samples. It includes ten features: radius, texture, perimeter, area, smoothness, compactness, concavity, concave points, symmetry, and fractal dimension. Each feature has three measurements: the mean, the standard error, and the worst value, defined as the average of the three largest measurements. The dataset contains 212 cancerous samples and 357 non-cancerous samples, so 37.3 percent of the samples are cancerous. The standard version has no missing values. These measurements describe changes in cell shape and texture linked to cancer, such as larger and less regular nuclei, differences in chromatin, greater variation in nuclear shape, and disorganized cell structure. The authors emphasize that this public dataset offers a controlled setting for testing models. It cannot replace clinical data collected from many different medical centers.

The researchers placed great importance on careful testing. They compared seven methods: regularized logistic regression, a calibrated linear support vector machine, a radial basis function support vector machine, random forest, gradient boosting, k-nearest neighbors, and Gaussian Naive Bayes. They tested them using nested stratified cross-validation, with five outer groups and four inner groups. This means they repeatedly split the data into training and testing groups while keeping the balance of cancerous and non-cancerous samples similar. They calculated data-standardization settings, model parameters, probability calibration, and decision thresholds using only the training portion of each outer split. They then fixed these settings before testing on the held-out data. This approach was designed to prevent data leakage, which happens when information from test data influences model development. They used grid search to select model settings, choosing those with the best ROC-AUC score. They used a fixed random seed of 42 for data splitting, bootstrap resampling, and permutation-based feature importance. The study followed TRIPOD+AI guidance for reporting prediction models, while the PROBAST framework helped assess possible sources of bias in the data, predictors, outcomes, and analysis.

All seven methods performed within a narrow range. Their ROC-AUC scores ranged from 0.987 to 0.995, while their areas under the precision-recall curve ranged from 0.980 to 0.994. The radial basis function support vector machine had the highest scores for most measures of how well the models separated and classified the samples. However, formal statistical tests found no significant differences between the leading models. DeLong tests comparing the linear support vector machine with the radial basis function version and with logistic regression produced p-values of 0.430 and 0.829. Exact McNemar tests using the default 0.50 decision threshold produced p-values of 1.000 and 0.688. The researchers chose the linear model as their main model for interpretation because it had high specificity and coefficients that could be examined directly. Its Brier score was 0.021, its calibration intercept was 0.280, and its calibration slope was 1.284. The confidence intervals calculated using bootstrap methods were wide under the conditions tested. These results suggest that the model's predicted probabilities were well calibrated within this dataset, although uncertainty remained about how accurately those probabilities matched the true chances of cancer.

The choice of decision threshold had a major effect on how the model classified samples. Using Youden thresholds calculated separately from each training group, the linear support vector machine achieved an accuracy of 0.981, a sensitivity of 0.962, a specificity of 0.992, an F1-score of 0.974, and a Matthews correlation coefficient of 0.959 on the held-out samples. It correctly identified 204 of the 212 cancerous samples, missed 8 cancerous samples, and incorrectly classified 3 non-cancerous samples as cancerous. The researchers also tested a rule designed to prioritize sensitivity, choosing a threshold that achieved at least 0.99 sensitivity in the training data. With this rule, sensitivity on the held-out samples rose to 0.991, but specificity fell to 0.866. The number of missed cancerous samples dropped from 8 to 2, while false positives increased from 3 to 48. The authors use this trade-off as a teaching example: detecting six more cancerous samples came at the cost of 45 additional false-positive results, which could lead to unnecessary further tests or medical reviews.

The researchers also used decision-curve analysis to examine how useful the model might be when making decisions. They considered whether a patient should be referred for an expert review of cells or a confirmatory tissue examination. The analysis measured net benefit across decision threshold probabilities from 0.05 to 0.95. Before carrying out the analysis, they identified 0.05 to 0.30 as the main range for interpretation. Across this range, the relative weight given to missing a cancer compared with carrying out an unnecessary further examination changes from about 19 times as much to about 2.33 times as much. For most of this range, the model had a higher estimated net benefit than two comparison strategies: treating every case as needing further action or treating none as needing further action. The authors warn that the threshold probability used in decision-curve analysis is not the same as the threshold used to classify a sample. They included values above 0.30 for completeness, not as recommended decision points for actual use.

The features that helped the model make predictions matched established knowledge about cancer cells. Important predictors included worst texture, radius error, worst symmetry, compactness error, area error, and measurements related to concavity. Standardized mean differences showed clear separation between cancerous and non-cancerous samples. For example, the difference was 2.60 for worst concave points and 2.32 for worst radius. The median standardized coefficients of the linear model were positive across all five training folds for nine of the ten reported features. Compactness error consistently had a negative coefficient after accounting for the other features, even though its average value was higher in cancerous samples. This happened because the model adjusted for the effects of other, strongly related measurements of cell shape. The authors highlight this as a useful example of the difference between a feature's overall relationship with cancer and its contribution to a model when other features are considered at the same time. They interpret feature importance in groups of related cell characteristics, including nuclear size, irregular cell boundaries, and variation in texture or symmetry, rather than treating each feature as a separate biological effect.

Reducing the number of features showed that a smaller set of measurements could preserve much of the diagnostic information. The researchers ranked features by mutual information, a measure of how much information one variable provides about another, using only training data within each outer fold. A logistic regression model with 20 features achieved a ROC-AUC of 0.996 and a precision-recall AUC of 0.994. Its performance was not significantly different from that of the full model with 30 features. The difference in ROC-AUC between the two models was 0.0012, with a 95 percent bootstrap confidence interval from -0.0004 to 0.0034. Larger reductions in the number of features caused small but measurable drops in performance. The 10-feature model had a ROC-AUC difference of -0.0054 compared with the 30-feature model, with a confidence interval from -0.0117 to -0.0005. Ten measurements, mainly related to nuclear size and irregular boundaries, appeared among the ten highest-ranked features in all five folds. These included mean radius, mean perimeter, mean area, and worst concave points. Their repeated appearance suggests that these patterns were stable across the different data splits.

The authors clearly describe the limits of their findings. The WDBC dataset is an older collection from a single source containing measurements of cell nuclei that had already been extracted. It does not reflect differences in how samples are prepared, stained, scanned, or processed to identify cells, or differences in which cells pathologists select. It also lacks clinical information such as patient age, imaging results, tumor size, cancer type, receptor status, treatment, and follow-up. The bootstrap confidence intervals measure sampling uncertainty based on the fixed predictions made for held-out samples. They do not capture all the uncertainty that would arise from repeating the entire process of developing the model. The study looks at whether samples are cancerous or non-cancerous at the time of assessment, not at predicting a patient's future outcome. The authors say that prospective studies involving multiple medical centers are needed to test performance on new data, the accuracy of predicted probabilities, reliability across patient groups, and the stability of decision thresholds before the method is used in clinical care.

Beyond the benchmark results, the study's main contribution may be the way it teaches the subject. The framework connects patterns in nuclear measurements and feature importance with the reasoning used in cytopathology, the study of cells to identify disease. Comparisons between classifiers, calibration curves, and confusion matrices show different ways to judge a model's performance. Testing different decision thresholds demonstrates how predicted probabilities can affect diagnostic decisions. Nested validation and feature reduction provide practical examples of how to split data, identify overlapping features, understand model decisions, and prevent data leakage. By bringing these steps together using one open dataset and a workflow supported by scripts with fixed software versions, the researchers offer teachers in pathology, oncology, and medical data science a connected, practical way to teach how cellular features can be used in diagnostic artificial intelligence and how such systems should be critically evaluated.

Subject of Research: Machine learning that can be interpreted and explained, using measurements of cell nuclei from an open benchmark dataset to help diagnose breast cancer

Article Title: A reproducible open-data framework for interpretable breast cancer diagnosis and cytopathology education

Article References: Li, M., Zhang, X., Xie, X., Shi, Y., Hu, X., Huang, N., & Wang, Z. (2026). A reproducible open-data framework for interpretable breast cancer diagnosis and cytopathology education. Frontiers in Oncology, 16, Article 1921082. https://doi.org/10.3389/fonc.2026.1921082

Image Credits: AI Generated

DOI: 10.3389/fonc.2026.1921082

Keywords: breast cancer, cytopathology, machine learning, WDBC dataset, nested cross-validation, model calibration, decision-curve analysis, feature reduction, medical AI education, reproducible research, support vector machine, TRIPOD+AI