ITMO_FS.filters.univariate.information_gain

ITMO_FS.filters.univariate.information_gain(X, y)

Calculates mutual information for each feature by formula, I(X,Y) = H(X) - H(X|Y)

Parameters:

• X (numpy array, shape (n_samples, n_features)) – The input samples.
• y (numpy array, shape (n_samples, )) – The classes for the samples.

Returns:

Return type:

Score for each feature as a numpy array, shape (n_features, )

Examples

>>> import sklearn.datasets as datasets
>>> from ITMO_FS.filters.univariate import information_gain
>>> X = np.array([[1, 2, 3, 3, 1],[2, 2, 3, 3, 2], [1, 3, 3, 1, 3],[3, 1, 3, 1, 4],[4, 4, 3, 1, 5]], dtype = np.integer)
>>> y = np.array([1, 2, 3, 4, 5], dtype=np.integer)
>>> scores = information_gain(X, y)
>>> print(scores)