For a prime number \(p\ge 3\) , let \(A=diag[p,p]\) and \(0\in \mathcal {B}\subset {\mathbb Z}^2\) be a p-element digit set satisfying \(\mathcal {Z}(\widehat{\delta }_{\mathcal {B}})=\bigcup _{j=1}^{p-1} \left( \frac{j}{p}(\omega ,\rho )^t+{\mathbb Z}^2\right) \) for some \(\{\rho ,\omega \}\subset \{1,\cdots ,p-1\}\) , where \(\gcd (\rho ,\omega )=1\) and \(\mathcal {Z}(\widehat{\delta }_{\mathcal {B}})\) is the zero set of the Fourier transform of \(\delta _{\mathcal {B}}\) . It is known [38] that the associated self-similar measure \(\mu _{A,\mathcal {B}}\) is a spectral measure, i.e., there exists a countable set \(\Lambda \subset \mathbb {R}^2\) such that \(\{e^{2\pi i\langle \lambda ,x\rangle }:\lambda \in \Lambda \}\) forms an orthonormal basis for \(L^2(\mu _{A,\mathcal {B}})\) . In this paper, we characterize the structure of the maximal orthogonal sets and spectra of the spectral self-similar measure \(\mu _{A,\mathcal {B}}\) . For the special case \(\omega =\rho =1\) , we give a concrete spectrum \(\Lambda \) and find all matrices \(\Re \in M_2({\mathbb {R}})\) such that \(\Re \Lambda \) is also a spectrum of \(\mu _{A,\mathcal {B}}\) .