The construction of a discriminative affinity graph is crucial in graph-based image segmentation, and feature selection directly impacts its effectiveness. We propose a new method using covariance matrices to compute edge weights in the affinity graph. The process involves oversegmenting the image into superpixels, extracting color and covariance features, and fusing them to construct the affinity graph. Since covariance matrices are non-Euclidean, a suitable metric is adopted. We apply the dominant sets algorithm to partition the graph and evaluate our method on the Berkeley database, demonstrating competitive performance compared to standard methods across multiple metrics.