Skin Cancer Validation is a competition focused on training a binary skin-lesion malignancy classifier using the BCN20000, MSK-1, and MSK-2 datasets, then evaluating its cross-dataset generalization on the complete HAM10000 dataset of 10,015 dermoscopic images.
The competition addresses the challenge that skin-lesion classification models often perform well on images drawn from the same dataset used for training but may see decreased performance when evaluated on images collected by different institutions, devices, or patient populations. The goal is to evaluate how well a model generalizes to an independent HAM10000 test set.
Classification Task
Participants must predict the probability that each skin lesion is malignant. The diagnoses are categorized as follows:
* Malignant: Melanoma (mel), Basal cell carcinoma (bcc), and Actinic keratosis / intraepithelial carcinoma (akiec).
* Non-malignant: Melanocytic nevus (nv), Benign keratosis (bkl), Dermatofibroma (df), and Vascular lesion (vasc).
For every test image, participants submit one malignancy probability between 0 and 1.
Evaluation Submissions are evaluated using two binary-classification metrics: * ROC AUC: The primary leaderboard ranking metric, which evaluates how well submitted probabilities rank malignant lesions above non-malignant lesions independently of a classification threshold. * Accuracy: Calculated using a fixed probability threshold of 0.5 (a probability ≥ 0.5 is classified as malignant).
Higher values are better for both metrics.
Submission Format
The submission must be a CSV file containing exactly one row for every test image with the following columns:
* ID: Test image identifier.
* probability: Predicted probability of malignancy (0 through 1).
The file must include the header row, contain every test ID exactly once, contain no additional index column, and use only finite probability values.