k-Nearest Neighbors

Last Updated: 2021-11-19
  • non-parametric

  • used for classification and regression

    • classification: An object is classified by a majority vote of its neighbors, with the object being assigned to the class most common among its k nearest neighbors
    • regression: the output is the property value for the object. This value is the average of the values of its k nearest neighbors
  • instance-based learning, or lazy learning: the function is only approximated locally and all computation is deferred until classification.

  • can be useful to assign weight, e.g. 1/d as weight, where d is the distance to the neighbor

  • training: store feature vectors and class labels

  • classification: majority vote among k-nearest neighbors

  • drawback of majority voting: if the class distribution is skewed, the most frequent class tend to dominate the prediction.