The prevention of distributed denial of service (DDoS) assaults and ensuring the accessibility of network resources nowadays relies heavily on their identification and remediation. DDoS attack is an evil form of malware that can send a lot of traffic that can potentially harm either a few or all of the desires resources while prohibiting legitimate users from using the network’s amenities. Feature selection is necessary to optimize DDoS attack diagnosis in order to increase a machine learning algorithm’s effectiveness and minimize computing overhead. In the present investigation, an INBOPA (Inculcated Binary Pigeon Optimization algorithm) approach is suggested to speed up the feature selection (FS) process and accurately distinguishing malicious and normal network traffic. Firstly, The BPOA, which is renowned for its capacities of global investigation, explores its feature space in search of pertinent features which are essential for identifying attacks. Meanwhile, Brayden—Fletcher—Goldfarb—Shanno technique is employed in to BPO algorithm as a local improvement strategy that increases the degree of accuracy of FS by adjusting the chosen feature subgroup. With this crossover, the likelihood of hitting local optima is minimized and the method for choosing features executes better throughout. The developed In-BPOA method is experimented on DDoS dataset with Decision tree as a classifier. The presented INBPOA paradigm excels previous binary optimization methods involving the Artificial Bee Colony, Particle Swarm, and Gray—Wolf Optimization techniques.