Why does Random Forest variable importance not sum to 100%?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
1
down vote

favorite
1












The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results not sum to 100%?



Here's a simple reproducible example:



library(randomForest)
data(iris)
iris.rf <- randomForest(Species~., importance = TRUE, data = iris, ntrees=1000)
imp <- importance(iris.rf, type = 1)
sum_imp <- sum(imp)
sum_imp # != 100


Thanks






r random-forest importance







share|cite|improve this question

















  • 1




    Why do you assume it should sum to 1? I see no reason for that belief.
    – Firebug
    Sep 8 at 18:31











  • See Measures of variable importance in random forests
    – Firebug
    Sep 8 at 21:22
















up vote
1
down vote

favorite
1












The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results not sum to 100%?



Here's a simple reproducible example:



library(randomForest)
data(iris)
iris.rf <- randomForest(Species~., importance = TRUE, data = iris, ntrees=1000)
imp <- importance(iris.rf, type = 1)
sum_imp <- sum(imp)
sum_imp # != 100


Thanks






r random-forest importance







share|cite|improve this question

















  • 1




    Why do you assume it should sum to 1? I see no reason for that belief.
    – Firebug
    Sep 8 at 18:31











  • See Measures of variable importance in random forests
    – Firebug
    Sep 8 at 21:22












up vote
1
down vote

favorite
1









up vote
1
down vote

favorite
1






1





The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results not sum to 100%?



Here's a simple reproducible example:



library(randomForest)
data(iris)
iris.rf <- randomForest(Species~., importance = TRUE, data = iris, ntrees=1000)
imp <- importance(iris.rf, type = 1)
sum_imp <- sum(imp)
sum_imp # != 100


Thanks






r random-forest importance







share|cite|improve this question













The randomForest package in R has the importance() function to get both node impurity and mean premutation importance for variables. Why, when calculating mean permutation importance, do the results not sum to 100%?



Here's a simple reproducible example:



library(randomForest)
data(iris)
iris.rf <- randomForest(Species~., importance = TRUE, data = iris, ntrees=1000)
imp <- importance(iris.rf, type = 1)
sum_imp <- sum(imp)
sum_imp # != 100


Thanks






r random-forest importance




r random-forest importance






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Sep 8 at 14:02









Micha

1083




1083







  • 1




    Why do you assume it should sum to 1? I see no reason for that belief.
    – Firebug
    Sep 8 at 18:31











  • See Measures of variable importance in random forests
    – Firebug
    Sep 8 at 21:22












  • 1




    Why do you assume it should sum to 1? I see no reason for that belief.
    – Firebug
    Sep 8 at 18:31











  • See Measures of variable importance in random forests
    – Firebug
    Sep 8 at 21:22







1




1




Why do you assume it should sum to 1? I see no reason for that belief.
– Firebug
Sep 8 at 18:31





Why do you assume it should sum to 1? I see no reason for that belief.
– Firebug
Sep 8 at 18:31













See Measures of variable importance in random forests
– Firebug
Sep 8 at 21:22




See Measures of variable importance in random forests
– Firebug
Sep 8 at 21:22










1 Answer
1






active

oldest

votes

















up vote
4
down vote



accepted










As far as I can tell, variable importance is measuring either: a) the percentage that the prediction error increases when the variable is removed, or b) the change in the purity of each node when the variable is removed. (Averaged over all trees in the forest.) Neither of these is a probability, so there's no reason they should add up to 100%.



You can, of course, divide by the sum of all importances to get a percentage, but I think that would create confusion: you now have a percentage of what exactly?



(Welcome to the site!)






share|cite|improve this answer




















  • Thanks for the welcome. I expect I'll be back :-)
    – Micha
    Sep 9 at 6:50










Your Answer




StackExchange.ifUsing("editor", function ()
return StackExchange.using("mathjaxEditing", function ()
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
);
);
, "mathjax-editing");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "65"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: false,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f365955%2fwhy-does-random-forest-variable-importance-not-sum-to-100%23new-answer', 'question_page');

);

Post as a guest

















1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
4
down vote



accepted










As far as I can tell, variable importance is measuring either: a) the percentage that the prediction error increases when the variable is removed, or b) the change in the purity of each node when the variable is removed. (Averaged over all trees in the forest.) Neither of these is a probability, so there's no reason they should add up to 100%.



You can, of course, divide by the sum of all importances to get a percentage, but I think that would create confusion: you now have a percentage of what exactly?



(Welcome to the site!)






share|cite|improve this answer




















  • Thanks for the welcome. I expect I'll be back :-)
    – Micha
    Sep 9 at 6:50














up vote
4
down vote



accepted










As far as I can tell, variable importance is measuring either: a) the percentage that the prediction error increases when the variable is removed, or b) the change in the purity of each node when the variable is removed. (Averaged over all trees in the forest.) Neither of these is a probability, so there's no reason they should add up to 100%.



You can, of course, divide by the sum of all importances to get a percentage, but I think that would create confusion: you now have a percentage of what exactly?



(Welcome to the site!)






share|cite|improve this answer




















  • Thanks for the welcome. I expect I'll be back :-)
    – Micha
    Sep 9 at 6:50












up vote
4
down vote



accepted







up vote
4
down vote



accepted






As far as I can tell, variable importance is measuring either: a) the percentage that the prediction error increases when the variable is removed, or b) the change in the purity of each node when the variable is removed. (Averaged over all trees in the forest.) Neither of these is a probability, so there's no reason they should add up to 100%.



You can, of course, divide by the sum of all importances to get a percentage, but I think that would create confusion: you now have a percentage of what exactly?



(Welcome to the site!)






share|cite|improve this answer












As far as I can tell, variable importance is measuring either: a) the percentage that the prediction error increases when the variable is removed, or b) the change in the purity of each node when the variable is removed. (Averaged over all trees in the forest.) Neither of these is a probability, so there's no reason they should add up to 100%.



You can, of course, divide by the sum of all importances to get a percentage, but I think that would create confusion: you now have a percentage of what exactly?



(Welcome to the site!)







share|cite|improve this answer












share|cite|improve this answer



share|cite|improve this answer










answered Sep 8 at 14:43









Wayne

15.5k13572




15.5k13572











  • Thanks for the welcome. I expect I'll be back :-)
    – Micha
    Sep 9 at 6:50
















  • Thanks for the welcome. I expect I'll be back :-)
    – Micha
    Sep 9 at 6:50















Thanks for the welcome. I expect I'll be back :-)
– Micha
Sep 9 at 6:50




Thanks for the welcome. I expect I'll be back :-)
– Micha
Sep 9 at 6:50

















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f365955%2fwhy-does-random-forest-variable-importance-not-sum-to-100%23new-answer', 'question_page');

);

Post as a guest
































































-

這個網誌中的熱門文章

How to combine Bézier curves to a surface?

圖片
Clash Royale CLAN TAG #URR8PPP up vote 2 down vote favorite My aim is to smooth the terrain in a video game. Therefore I contrived an algorithm that makes use of Bézier curves of different orders. But this algorithm is defined in a two dimensional space for now. To shift it into the third dimension I need to somehow combine the Bézier curves. Given two curves in each two dimensions, how can I combine them to build a surface? I thought about something like a curve of a curve. Or maybe I could multiply them somehow. How can I combine them to cause the wanted behavior? Is there a common approach? In the image you can see the input, on the left, and the output, on the right, of the algorithm that works in a two dimensional space. For the real application there is a three dimensional input. The algorithm relays only on the adjacent tiles. Given these it will define the control points of the mentioned Bézier curve. Additionally it marks one side of the curve as inside t
閱讀完整內容

Carbon dioxide

圖片
"CO2" redirects here. For other uses, see CO2 (disambiguation). Carbon dioxide Names Other names Carbonic acid gas Carbonic anhydride Carbonic oxide Carbon oxide Carbon(IV) oxide Dry ice (solid phase) Identifiers CAS Number 124-38-9   Y 3D model (JSmol) Interactive image Interactive image 3DMet B01131 Beilstein Reference 1900390 ChEBI CHEBI:16526   Y ChEMBL ChEMBL1231871   N ChemSpider 274   Y ECHA InfoCard 100.004.271 EC Number 204-696-9 E number E290 (preservatives) Gmelin Reference 989 KEGG D00004   Y MeSH Carbon+dioxide PubChem CID 280 RTECS number FF6400000 UNII 142M471B3J   Y UN number 1013 (gas), 1845 (solid) InChI InChI=1S/CO2/c2-1-3   Y Key: CURLTUGMZLYLDI-UHFFFAOYSA-N   Y InChI=1/CO2/c2-1-3 Key: CURLTUGMZLYLDI-UHFFFAOYAO SMILES O=C=O C(=O)=O Properties Chemical formula C O 2 Molar mass 44.01 g·mol −1 Appearance Colorless gas Odor Low concentrations: none High concentrations: sharp; acidic [1] Density 1562 kg/m 3 (solid at 1 atm and −78.5 °C) 1101 kg/m 3 (liquid at
閱讀完整內容

Why am i infinitely getting the same tweet with the Twitter Search API?

圖片
up vote 0 down vote favorite My code workes perfectly for other queries, but for one query I am infinitely getting the same tweet. I can't understand what the problem is. API call: query='Allergic asthma OR Nonallergic asthma OR Occupational asthma OR EIB OR Exercise-induced bronchoconstriction OR Nocturnal asthma OR Cough-variant asthma' new_tweets = api.search(q=searchQuery, count=100, since_id=since_id, lang='en',tweet_mode='extended') if not new_tweets: print("No more tweets found") break for tweet in new_tweets: if True: print(jsonpickle.encode(tweet._json, unpicklable=False)) my_file = open('searchTweets.txt','a') my_file.write(jsonpickle.encode(tweet._json, unpicklable=False)+'n') my_file.close() else: continue Output in file: https://pastebin.com/JmRCsTeE These are the JSONs of the same Tweet. Other queries that work: Breast cancer OR Prostate cancer OR Colon cancer OR Lung cancer Type 2 dia
閱讀完整內容