Decision Tree

Outlook	Temperature	Humidity	Windy	Play
sunny	hot	high	Weak	no
sunny	hot	high	Strong	no
overcast	hot	high	Weak	yes
rainy	mild	high	Weak	yes
rainy	cool	normal	Weak	yes
rainy	cool	normal	Strong	no
overcast	cool	normal	Strong	yes
sunny	mild	high	Weak	no
sunny	cool	normal	Weak	yes
rainy	mild	normal	Weak	yes
sunny	mild	normal	Strong	yes
overcast	mild	high	Strong	yes
overcast	hot	normal	Weak	yes
rainy	mild	high	Strong	no

Outlook	Temperature	Humidity	Windy	Play
2	1	0	1	no
2	1	0	0	no
0	1	0	1	yes
1	2	0	1	yes
1	0	1	1	yes
1	0	1	0	no
0	0	1	0	yes
2	2	0	1	no
2	0	1	1	yes
1	2	1	1	yes
2	2	1	0	yes
0	2	0	0	yes
0	1	1	1	yes
1	2	0	0	no

Play	yes	no	sum	Entropy
	9	5	14	$$-\frac{9}{14}log(\frac{9}{14})--\frac{5}{14}log(\frac{5}{14})=0.940285959 $$

$$Gain(Outlook(0.5))=Entropy(Play)-\frac{4}{14}*Entropy(Outlook_{<=0.5})-\frac{10}{14}entropy(Outlook_{>0.5})$$

Outlook	yes	no	sum	Entropy	Gain
≤ 0.5	4	0	4	0	0.226000244
> 0.5	5	5	10	1	0.226000244
≤ 1.5	7	2	9	0.764204507	0.102243564
> 1.5	2	3	5	0.970950594	0.102243564

Temperature	yes	no	sum	entropy	gain
≤ 0.5	3	1	4	0.811278124	0.01495607
> 0.5	6	4	10	0.970950594	0.01495607
≤ 1.5	5	3	8	0.954434003	0.001339742
> 1.5	4	2	6	0.918295834	0.001339742

Humidity	yes	no	sum	entropy	gain
≤ 0.5	3	4	7	0.985228136	0.151835501
> 0.5	6	1	7	0.591672779	0.151835501

Windy	yes	no	sum	entropy	gain
≤ 0.5	3	3	6	1	0.04812703
> 0.5	6	2	8	0.811278124	0.04812703

Entropy

$$entropy = -\sum{p_ilog_2(p_i)}$$

from scipy.stats import entropy
entropy([9, 5], base=2)

Preprocessing

from sklearn.preprocessing import OrdinalEncoder
from pandas import read_csv
df = read_csv('play.csv')
encoder = OrdinalEncoder(dtype=int)
X = encoder.fit_transform(df[df.columns[:-1]])
y = df['Play']

Build Model

from sklearn.tree import DecisionTreeClassifier
classifier = DecisionTreeClassifier(criterion='entropy')
classifier.fit(X, y)

Plot Tree

from sklearn.tree import plot_tree
from matplotlib import pyplot
plot_tree(classifier, feature_names=df.columns[:-1], class_names=["no", "yes"])
pyplot.show()

Accuracy Score

classifier.score(X, y)

Gini

$$G=1-\sum_k{p_k^2}$$

Reference

Decision Tree	https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html

Outlook	Temperature	Humidity	Windy	Play
2	1	0	1	no
2	1	0	0	no
0	1	0	1	yes
1	2	0	1	yes
1	0	1	1	yes
1	0	1	0	no
0	0	1	0	yes
2	2	0	1	no
2	0	1	1	yes
1	2	1	1	yes
2	2	1	0	yes
0	2	0	0	yes
0	1	1	1	yes
1	2	0	0	no

Outlook	Temperature	Humidity	Windy	Play
2	1	0	1	no
2	1	0	0	no
0	1	0	1	yes
1	2	0	1	yes
1	0	1	1	yes
1	0	1	0	no
0	0	1	0	yes
2	2	0	1	no
2	0	1	1	yes
1	2	1	1	yes
2	2	1	0	yes
0	2	0	0	yes
0	1	1	1	yes
1	2	0	0	no

Outlook	Temperature	Humidity	Windy	Play
2	1	0	1	no
2	1	0	0	no
0	1	0	1	yes
1	2	0	1	yes
1	0	1	1	yes
1	0	1	0	no
0	0	1	0	yes
2	2	0	1	no
2	0	1	1	yes
1	2	1	1	yes
2	2	1	0	yes
0	2	0	0	yes
0	1	1	1	yes
1	2	0	0	no