Super learner
This user is a Super Learner. To become a Super Learner, you need to reach Level 8.
Resolved: entropy function
high entropy means high uncertanity, but that doesn't mean that the features values for these classes are similar. the two classes could be very different from each other and not that difficult to split despite the high entropy. so why the alogrithm struggle to split it. can you explain more how it works in this case. thanks
1 answers ( 1 marked as helpful)
Hello Doaa,
Thanks for reaching out!
Great question!
High entropy indeed indicates high uncertainty in class distribution, meaning the classes are more evenly distributed in the dataset at that point. However, the issue isn't necessarily that the feature values for the classes are similar—it’s about how the algorithm measures the "purity" of splits.
Entropy is used in decision trees as a criterion for evaluating how well a split separates the data into pure groups (where each group ideally contains a single class). When entropy is high, the algorithm struggles to find splits that create significantly purer subsets, as each potential split still leaves mixed classes behind. Essentially, it becomes harder to reduce uncertainty through splitting.
Hope this helps!
Best,
The 365 Team
Thanks for reaching out!
Great question!
High entropy indeed indicates high uncertainty in class distribution, meaning the classes are more evenly distributed in the dataset at that point. However, the issue isn't necessarily that the feature values for the classes are similar—it’s about how the algorithm measures the "purity" of splits.
Entropy is used in decision trees as a criterion for evaluating how well a split separates the data into pure groups (where each group ideally contains a single class). When entropy is high, the algorithm struggles to find splits that create significantly purer subsets, as each potential split still leaves mixed classes behind. Essentially, it becomes harder to reduce uncertainty through splitting.
Hope this helps!
Best,
The 365 Team