Fluoride is an essential mineral for the body as it helps strengthen the bone and teeth, but high amounts of fluoride exposure can have dangerous health effects. Fluoride contamination has been addressed as a major problem worldwide, along with the scarcity of water. The World Health Organization has set the maximum consumption limit of fluoride at 1.5 mg/L. The source of fluoridation may include natural occurring minerals, coal mines or anthropogenic action. Several materials (Biomass, rice husk ash, bone char, shale, and low-grade coal) and methods (Adsorption, membrane separation, and column studies) have been in use for defluoridation of water. Several studies have employed different adsorbents to eliminate fluoride from polluted water. However, none have capitalized on the extensive volumes of waste produced by mining industries. This study focuses on utilizing shale, a prevalent mining waste, as an adsorbent for removing fluoride from aqueous solutions. Along with the defluoridation property of carbonaceous shale, we compared the defluoridation property of ferruginous shale from different mines. The characterization tests, such as XRD, SEM-EDS, and FTIR were also done to analyse their various properties. The maximum efficiency of type I carbonaceous shale was obtained as 47.05% compared to other shales for 3 ppm initial fluoride solution for 60 min of contact time in pH range 5–7 using batch adsorption process. The efficiency of carbonaceous shale collected from Samleshwari coal mines was also compared with other carbonaceous shale collected from Basundhara coal mines and ferruginous shales collected from Guali and Barsuan iron mines. Both carbonaceous shale and ferruginous shale have potential for use as an adsorbent for F− removal but a suitable adsorbent should not only have high F− adsorption capacity and cost effectiveness but also be amenable to easy desorption of the adsorbed F− and capable of efficient regeneration for reuse. The desorption and regeneration efficiency of both shales were also studied, the result showed carbonaceous shale was better than ferruginous shale in adsorption of fluoride without affecting the pH and TDS of treated water.