<p>Hyperspectral image classification is a challenging task due to the lack of ground-truth labels and high dimensionality of spectral bands. For hyperspectral image (HSI) data of cultural heritage artifacts, which are generally man-made objects, we typically have much higher spatial resolution posing extra challenges. Aiming to differentiate subtle spectral differences between varied pigments with little manual effort, we propose a Spatial-Spectral Graph Convolutional Network (SSGCN) as a powerful learning framework for realizing automatic pigment mapping. First, we introduce a novel spatial pattern descriptor “Edge Response Map” to extract structural information of a training ROI, upon which a Spatial-Spectral graph is built by incorporating spatial node connections into the spectral-based graph. Then, we infuse the Spatial-Spectral graph into the graph convolutional network and build a SSGCN learning pipeline. We train the network on the training ROI via semi-supervised classification. By aggregating node features in terms of spectral affinity and spatial context, SSGCN is capable of learning the intrinsic community structures in a non-linear way, and thus can be directly applied to another region for pigment mapping without ground-truth labels required. Furthermore, the graph-based nature of the approach empowers the SSGCN with scalability to learn new pigment classes by flexibly extending the current model. The experiments on multiple ROIs in the Selden Map of China demonstrate the strength and advantages of our method. Compared to an adaptive density-weighted KNN graph, the Spatial-Spectral graph is able to better identify spatial patterns. With only 15% labeled training pixels, the SSGCN achieves mean per-class accuracy above 90%, obtaining automatic pigment mapping on test ROIs with sub-visual spectral differences well discriminated. In terms of classification accuracy and time complexity, the SSGCN greatly outperforms a 3D CNN architecture, HybridSN, and demonstrates much better inductive learning ability on unseen pixels. The proposed graph-based learning pipeline brings a new prospect for processing large-scale HSI classification/clustering in an intelligent manner, and would aid historians in pigment analysis and codicological studies of artifacts with little manual cost.</p>