This article is based on fractal geometry and complex network theory, dedicated to exploring the optimization application of fractal design algorithms in visual spatial design. Through in-depth analysis of fractal theory and sorting out specific design cases, this article proposes two improved design methods based on mathematical models and programming languages, and generates images with unique visual effects in experiments. This article further explores the expressive power of fractal structures in visual presentation and their multi-level aesthetic connotations, with a particular focus on their unique characteristics in the combination of ordered beauty and psychedelic beauty. This study not only expands the application boundaries of fractal algorithms in visual spatial design, but also provides innovative theoretical support for design methods.