Testing Deep Reinforcement Learning (DRL) agents is costly and heavily dependent on environment configurations. Tests based on randomly generated configurations without specific objectives tend to be inefficient. Two key criteria for effective test generation are the difficulty and diversity of configurations. A configuration is considered difficult if it is likely to cause the agent to fail. Diversity aims to broadly cover the space of possible configurations. Automatically identifying configurations that are both difficult and diverse improves the evaluation of DRL agents, but remains a challenging task. Our approach is inspired by existing literature: we build a binary classifier to distinguish configurations, identify explanatory attributes using the Layerwise Relevance Propagation (LRP) method, and then generate new configurations through mutation. Experimental results on the parking and humanoid environments show that our method produces more difficult and diverse configurations, leading to a higher failure rate compared to existing approaches on the same environments.