Large Language Models (LLMs) are currently very popular and have been widely used. However, despite the success, LLMs are not entirely safe. This has been confirmed by jailbreak attacks. The main difficulty of fixing the leak is that the training datasets for LLMs are huge and not entirely harmless. There is a risk of generating harmful content based on user prompts when generating outputs, although various developers of large models have made restrictions. In this paper, we analyze the existing security vulnerabilities of LLMs and show the feasibility of automatically mitigating a jailbreak attack in both black-box and white-box settings. Based on this foundation, we propose a simple yet effective framework. We explore leveraging GPT4 to generate a substantial amount of pre-security instructions as an alternative to manual engineering, and evaluate its impact across various commercial and open-source LLMs (including ChatGPT, LLaMa-2 and ChatGLM). Our findings indicate that the generated pre-security instructions effectively prevent the generation of harmful content, thereby successfully blocking the jailbreak attack. We anticipate that this paper will contribute to the advancement of research on defending against jailbreak attacks, and enhance the security of large language models. Warning: This article contains unfiltered content generated by Large Language Models.