Hashing-based algorithms have recently shown promising performance in image retrieval. However, the majority of supervised hashing techniques are limited by the following: (1) Solving binary hash codes can be interfered with by inaccurate and incomplete label information; (2) Solving complex discrete optimization problems produces significant quantization errors; (3) Solving hash codes is inefficient, and optimization algorithms take a substantial amount of time. We present a novel supervised algorithm named robust fast supervised discrete hashing (RFSDH) to address these limitations. Specifically, RFSDH adopts the flexible \({\text{l}}_{{2,{\text{p}}}}\) loss function and \({\text{l}}_{{2,{\text{q}}}}\) loss function to induce sample sparsity and suppress potential label noise effects. In addition, a discrete optimization scheme is introduced to solve the hash function and codes directly. Extensive experimental results on three image datasets have indicated that RFSDH is superior to some existing traditional hashing algorithms for image retrieval.