A neural network is developed, that implements a learning-capable K-Nearest Neighbor algorithm (KNN). The goal of the paper is to present a structure and algorithms of functioning of this neural network. The main peculiarity of the proposed network is its capability for learning by means of modification of the network’s connection weights. This type of learning is a full-fledged learning, while classic KNN algorithm and methods proposed for improving its efficiency and speed mainly consider learning process as adjustment of KNN parameters based on the data. After learning stage, the proposed neural network provides a significant improvement of recognition performance compared to the initial KNN. Also a new type of features specifically intended for description and recognition of objects’ shapes has been developed in the work. These features are biologically inspired orientation features; but they are not simple edge detectors - they are orientation histograms. The proposed neural network is implemented as a computer program and is trained on a reduced training set of the MNIST database (RMNIST) consisting of only 10 examples for each recognized class. The network performance is evaluated using standard test set of the MNIST database of 10 000 examples. The experiments conducted with the computer program demonstrate a efficiency of the neural network and its capability to learn by means of modification of the network’s connection weights. Compared to the recognition performances of a number of classifiers also trained on RMNIST, the proposed network shows better recognition results.