Organic Computing (OC) systems tend to incorporate a self-adaptation mechanism allowing them to learn from perceptions of their environment. The hereby used self-learning component is typically a variant of the XCS Classifier System (XCS). XCS is seen as the most researched Learning Classifier System (LCS) which are rule-based Machine Learning systems. LCSs evolve and train populations of classifiers, also called rules, represented as condition-action-evaluation mappings, forming their knowledge base. For their rules, XCS variants employ conditions covering the state space of an environment based on static assumptions about the geometry or structure of the state niches. We take a closer look at using neural networks as so-called neural rules and neural conditions that should be able to evolve to cover state niches of arbitrary structure. While neural rules are far from novel, we modernise the approach and seek to improve the application of neural conditions to Reinforcement Learning problems. XCSF using neural rules and neural conditions was applied to different multi-step Reinforcement Learning environments. Our reported experiments show that they outperform the previous variant of neural rules in all cases, including a traditional XCSF using hyperrectangles in almost all environments.