k-Nearest Neighbors
-
non-parametric
-
used for classification and regression
- classification: An object is classified by a majority vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors
- regression: the output is the property value for the object. This value is the average of the values of its k nearest neighbors
-
instance-based learning, or lazy learning: the function is only approximated locally and all computation is deferred until classification.
-
can be useful to assign weight, e.g. 1/d as weight, where d is the distance to the neighbor
-
training: store feature vectors and class labels
-
classification: majority vote among k-nearest neighbors
-
drawback of majority voting: if the class distribution is skewed, the most frequent class tend to dominate the prediction.