Looking past questions of gene content, we focus on structural variants of the genomes within a pangenome and seek to find a phylogeny where all the ancestral nodes, including the root, are also pangenomes. Representations of pangenomes generally search for compact structures that emphasize common regions or common duplications among the constituent genomes, but necessarily sacrifice some other aspects of gene order. Since the gene order of a monoploid genome is basically just the set of all the gene adjacencies it is composed of, we will consider a pangenome as being made up all the adjacencies of genes appearing in at least one of its constituent genomes. Our key combinatorial tool, phylogenetic validation, does not involve optimization, but is simply a filter that removes any adjacencies present in input (extant) pangenomes which are unlikely to have been present in any ancestor, inspired by Dollo’s law of irreversible changes. In simulations, this tool turns out to be extraordinarily efficient in retrieving only adjacencies in the original ancestor.