Starting from the second half of the eighteenth century, in the reconstruction of the bourgeois city, particular emphasis was placed on creating new types of public buildings designed to meet the requirements of hygiene, prevention, and the physical and moral rehabilitation of individuals, with the aim of utilizing them in a productive manner for society. Under Bourbon crown, this led to an infrastructural transformation of the city, conceived as a machine with a dense network of centers, many of which were the new prison facilities designed according to contemporary Enlightenment theories. Some of the main Bourbon detention structures were relegated to island contexts, ideal locations for confining an ‘inconvenient’ function and optimizing the desire for social separation and deprivation of freedom. However, the complex management of this particular architectural heritage has led to a gradual process of decommissioning, leaving many of these structures in a state of critical decay. The challenge in restoring and reusing this ‘dark heritage’ involves multiple disciplines, from architecture to psycho-social studies, as these places align with what Michel Foucault defines as “heterotopic” spaces par excellence. The aim of this article is to study the historical, typological and material aspects of a ‘hostile’ and little-studied architectural category. The methodology adopted starts from a general analysis of the Bourbon policy of detention and its sublimation in different architectural typologies, allegories of a social and political vision of that historical period. Within a typological study of prisons in southern Italy, specific attention is paid to the island cases, often considered as the ideal location for prison structures. In fact, island prisons respond to different architectural criteria, often being the result of transformations and adaptations of pre-existing structures. This process of adaptation will be particularly evident in the final section, with the case study of the Palazzo d’Avalos and the former prison of Procida (Naples). Built in the sixteenth century as the residence of Cardinal Innico d’Avalos d’Aragona, the complex later became the royal residence of the Bourbons and, in the nineteenth century, a prison. The prison function has deeply marked the identity of the complex, both in architectural and iconic terms. The analysis of the Procida complex, decommissioned in 1988 and acquired by the Municipality of Procida in 2003, will make it possible to reflect on the various problems involved in the conservation and valorisation of a difficult heritage, further compromised by the fragile landscape in which it is located. The ultimate aim of the contribution is therefore to reflect on the possibilities that come into play in the processes of re-signifying a hostile and vulnerable heritage—both in terms of its functional and material aspects—which today is in danger of losing its cultural and material significance.