This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 43%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Hi.
I am working on an ML model in Designer.
I have a dataset of c. 55,000 rows.
When I add an "ID" column (unique per row - so 55,000 IDs) to my dataset for training / scoring, I receive the error message:
ModuleExceptionMessage:ColumnUniqueValuesExceeded: Number of unique values in column: "ID" is greater than allowed.
Question: is this error based on a physical cap on number of rows - or capacity based on e.g. Compute power associated with the instance?
I can run 20k rows through the model without the ID column - so it seems the unique rows is the challenge.
But then - how do I keep an identifying column in the scored dataset, if there is a cap on unique values?
Because I need the ID column to join with other data that is not able to be used in modelling as features etc.
Any guidance welcome!
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 1%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EYM%3C/text%3E%3C/svg%3E)
Thanks for reaching out to us.
This error occurs when a column contains too many unique values. For example, you might see this error if you specify that a column be handled as categorical data, but there are too many unique values in the column to allow processing to complete. You might also see this error if there is a mismatch between the number of unique values in two inputs.
Resolution:
Open the module that generated the error, and identify the columns used as inputs. For some modules, you can right-click the dataset input and select Visualize to get statistics on individual columns, including the number of unique values and their distribution.
For columns that you intend to use for grouping or categorization, take steps to reduce the number of unique values in columns. You can reduce in different ways, depending on the data type of the column.
I know you need the ID for training your model, so I am checking internally to see the plan. Will get back to you soon.
Yutong
Hello - were you able to find out more about having an ID column included in the scored dataset?
Hello,
Thanks for your patience, could you please share us the pipeline run URL, pipeline graph screenshot, module error so that we can look into this?
Regards,
Yutong-
Thank you - here is the URL:
^ this is a version of the pipeline that functions as intended, but that can only be run with a sample of 0.01 applied. 
and here is the error from the pipeline when I run e.g. 0.05 partition:
Thank you, I have shared those information to the product manager of AML designer and we are looking into this scenario.
Regards,
Yutong
Hello - please may I ask if there are any update?
Note I can subsititute the ID value to be an integer only - if this would resolve the problem?
User can use Edit Metadata module to mark the ID column as "ClearFeature", and thus this will not be used in Train Model. This should prevent the error. Please have a try and let me know if there is any questions. https://learn.microsoft.com/en-us/azure/machine-learning/algorithm-module-reference/edit-metadata
Regards,
Yutong
Thank you - I had used "Clear Feature" but I think the problem was fixed by swapping to an integer ID, instead of a string ID column.