Generative AI(GenAI) based applications have been widely adopted by individuals and organizations to automate daily tasks, enhance productivity, and drive innovation. However, integrating GenAI-based applications into internal government departments or agencies presents several challenges, necessitating comprehensive governance to mitigate potential AI risks while still promoting accessibility for government employees. This paper proposes an AI-based guardrail framework within a governmental organizational context. Specifically, we leverage prompt engineering techniques to guide a Large Language Model (LLM) in assessing another LLM-based GenAI application (e.g., ChatGPT) for its alignment with specific public value principles, providing structured numerical outputs with explanations. We conduct both quantitative experiments and human evaluation on two datasets. Results demonstrate that the LLM-based guardrail can understand complex evaluation instructions and generate reasonable explanations, acting as an additional safety layer to flag content that violates given value principles. However, significant differences in language understanding abilities were observed among different LLMs, including the OpenAI GPT models and other open-source LLMs. The implementation of the proposed framework is available in the GitHub repository ( https://github.com/xrkong/autorail ).