Surface defects detection systems (SDDSs) based on machine vision for hot-rolled steel strips (HRSSs) have been installed in many steel mills, but category, size and location of the surface defects of HRSSs still have been judged to look at the images from the SDDSs by operators, because the false or missed detection cases often occur when the SDDSs are applied to detect the HRSSs’ surface images affected by lights, dusts, and water mists. An improved CenterNet (ICN) with convolution attention mechanism is presented in this paper, which can detect the objects on images with or without ambient lights interferences. The ICN is composed by inserting a convolution attention module and a Sigmoid activation function between Resnet 50 and Decoder of CenterNet (CN) model and replacing the loss functions of the Wh-head and Offset-head modules of CN with Smooth L1 loss function. A HRSSs surface images dataset including 8000 images is constructed, which contains the images with one or multi-category defects and the images without defects. Some of these images are those affected by lights, such as the exposed images and low-light images, and the other part are those with good lighting. The ICN model and CN model are trained, tested, and verified by the image dataset. The experiment results show that the ICN model can detect precisely the surface defects of HRSSs and has higher detection accuracy than CN model.