question

IvanCasanaGallen-3736 avatar image
0 Votes"
IvanCasanaGallen-3736 asked romungi-MSFT answered

Filter Based Feature Selection or Permutation Feature Importance

Hi

I am aware that Feature Based Selection kind of measure the prediction power of each variable before any model has been built.

I know that Permutation Feature importance measures, once the model has been built, how relevant each variable is. Thanks to this we can prune our model and strike a good balance between accuracy and simplicity. Fair enough.

However, most of the time I encounter contradictory messages. There is a variable with 0 importance on the Feature Based Selection but it ends up becoming one of the most relevant variables of my model according to its ranking on Permutation Feature importance.

So I guess I cannot rely on what Feature Based Selection says in order to do a preliminary assessment. I guess I should see it as some theoretical exercise.

Thank you

azure-machine-learningazure-machine-learning-studio-classic
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

romungi-MSFT avatar image
0 Votes"
romungi-MSFT answered

@IvanCasanaGallen-3736 Thanks for adding details about your observations. As documented under Permutation feature importance module the rankings the component provides are often different from the ones we get from Filter Based Feature Selection as the Filter Based Feature Selection calculates scores before a model is created. Thanks!!


5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.