It is known that no-boundary Cellular Automata (CA) defined by bipermutive local rules give rise to Latin squares. In this paper, we study under which conditions the Latin square generated by a bipermutive CA is self-orthogonal, i.e. orthogonal to its transpose. We first enumerate all bipermutive CA over the binary alphabet up to diameter \(d=6\) , remarking that only some linear rules give rise to self-orthogonal Latin squares. We then give a full theoretical characterization of self-orthogonal linear CA, by considering the square matrix obtained by stacking the transition matrices of the CA and of its transpose, and determining when it is invertible. Interestingly, the stacked matrix turns out to have a circulant structure, for which there exists an extensive body of results to characterize its invertibility. Further, for the case of the binary alphabet we prove that irreducibility is a sufficient condition for self-orthogonality, and we derive a simpler characterization which boils down to computing the parity of the central coefficients of the local rule.