This chapter provides an overview of fundamental derivative-free optimization (DFO) algorithms, focusing on evolutionary algorithms (EAs), estimation of distribution algorithms (EDAs), and Bayesian optimization (BO). EAs are inspired by natural evolution. EDAs model the probability distribution of promising solutions to guide the search, while BO uses surrogate models to efficiently optimize expensive black-box functions. The chapter also discusses running time analysis, a key theoretical tool for understanding algorithm performance, and introduces the No Free Lunch Theorem, which highlights the importance of problem-specific knowledge in optimization. These concepts lay the groundwork for analyzing and designing advanced DFO methods, particularly in machine learning and other complex domains.