How to save the label for the positive class (AutoML - setting binary classification)

Question

I'm trying to make a binary classification AutoML-model. And I am spesifying the positive label winthin the advanced settings. For some reason, this doesn't have any impact on the runs. I am always getting analytics metrics: micro, macro and weighted but never binary. When I look at the configuration tab I notice that the positive label that I have typed has not been saved. I have tried also different data types (binary: true/false, integer: 0/1 and string: not default / default)

Answer 1

Hello Soini Juha,

Welcome to the Microsoft Q&A and thank you for posting your questions here.

I understand that despite all your tries, you would like to know how you can save the label for the positive class (AutoML - setting binary classification).

To addresses the problem you described, and provides actionable solutions note the followings:

• This issue could arise from a mismatch between the dataset's labels and the positive label specified in the AutoML settings.
• It could also be due to a configuration issue in the AutoML tool where the setting is not being applied or saved correctly.

Metrics like micro, macro, and weighted averages are typically used for multi-class classification. If binary-specific metrics (e.g., Precision, Recall, F1 for the positive class) are not appearing, it suggests the positive label is either not set or not recognized during the run.

Regarding your question:

How to save the label for the positive class (AutoML - setting binary classification)

Ensure the positive label specified in the settings exactly matches the dataset. AutoML tools often rely on case-sensitive and type-consistent labels. For instance:

• Binary: Use 0 and 1.
• String: Match "positive" and "negative" exactly as they appear in the dataset.
• Some AutoML platforms require explicitly setting the positive label in the configuration file or UI. If using code, confirm that the configuration parameter for the positive class is correctly set (e.g., positive_class_label = 'True').
• After specifying the label, ensure the changes are saved before starting the training run. Check for a "Save" or "Apply" button in the tool’s interface, or verify that the code correctly reflects the setting.

To resolve the current issue:

1. Ensuring the Positive Label Matches the Dataset by double-check that the positive label specified in the settings matches the dataset exactly, including data type (e.g., integer vs. string) and case sensitivity and standardize labels in the dataset before running AutoML (e.g., using consistent labels like 0/1 or "True"/"False").
2. Ensure the positive label is properly configured in the advanced settings and saved. If using a code-based API, confirm the parameter (e.g., positive_class_label) is correctly set and validate that changes to the settings persist after saving. This help to verifying configuration.
3. You can use your Tool-Specific Debugging to check the AutoML tool's documentation for handling binary classification and any peculiarities in specifying the positive label. Then, update the AutoML tool to the latest version to address potential bugs or missing features.
4. Also, if binary-specific metrics are not appearing, explicitly configure the tool to output metrics like Precision, Recall, F1-score, and ROC-AUC. Some AutoML platforms may require these to be manually enabled.
5. Run a small dataset experiment to confirm the positive label is being correctly recognized and that binary-specific metrics are generated. If the issue persists, I recommend:

• Sharing the specific AutoML tool and version you're using for more tailored guidance.
• Reviewing logs from the AutoML tool to identify any warnings or errors related to the label settings.

I hope this is helpful! Do not hesitate to let me know if you have any other questions.

---

Please don't forget to close up the thread here by upvoting and accept it as an answer if it is helpful.