Refined Lower Bounds for Nearest Neighbor Condensation

One of the most commonly used classification techniques is the nearest neighbor rule: given a training set T of labeled points in a metric space (X, rho), a new unlabeled point x is an element of chi is assigned the label of its nearest neighbor in T. To improve both the space & time complexity of this classification, it is desirable to reduce the size of the training set without compromising too much on the accuracy of the classification. Hart (1968) formalized this as the NEAREST NEIGHBOR CONDENSATION (NNC) problem: find a subset C subset of T of minimum size which is consistent with T, i.e., each point t is an element of T has the same label as that of its nearest neighbor in C. This problem is known to be NP-hard (Wilfong, 1991), and the heuristics used in practice often have weak or no theoretical guarantees. We analyze this problem via the refined lens of parameterized complexity, and obtain strong lower bounds for the k-NNC-(Z(d), l(p)) problem which asks if there is a consistent subset of size <= k for a given training set of size n in the metric space (Z(d), l(p)) for any 1 <= p <= infinity: The k-NNC-(Z(d), l(p)) problem is W[1]-hard parameterized by k + d, i.e., unless FPT = W[1], there is no f(k, d) center dot n(O(1)) time algorithm for any computable function f. Under the Exponential Time Hypothesis (ETH), there is no d >= 2 and computable function f such that the k-NNC-(Z(d), l(p)) problem can be solved in f(k, d) center dot n(o(k1-1/d)) time. The second lower bound shows that there is a so-called (Marx and Sidiropoulos, 2014) "limited blessing of low-dimensionality": for small d some improvement might be possible over the brute-force n(O(k)) time algorithm, but as d becomes large the brute-force algorithm becomes asymptotically optimal. It also shows that the is the n(O(root k)) time algorithm of Biniaz et al. (2019) for k-NNC-(R-2, l(2)) is asymptotically tight. Our lower bounds on the fine-grained complexity of NEAREST NEIGHBOR CONDENSATION in a sense justify the use of heuristics in practice, even though they have weak or no theoretical guarantees.