The inverse colormap operation is the process which allows an image to be displayed with a limited set of colors. In order to obtain minimal visual distortion between the input image and the displayed image, inverse colormap algorithms associate each color with its nearest representative. The method presented in this paper is based on a Karhunen-Loeve transformation, which allows us to efficiently approximate the 3D Voronoi diagram implicitly used by inverse color map algorithms by its 2D projection into the plane defined by the first two eigenvectors of the covariance matrix and then performing an additional correction step. The complexity of our algorithm is independent of the size of the colormap. Moreover, its results are equal or quite close to the optimal solution.