Combinatorial sequential decision-making problems are typically modeled as mixed-integer linear programs (MILPs) and solved via branch-and-bound (B&B) algorithms. The inherent difficulty of modeling MILPs that accurately represent stochastic real-world problems leads to suboptimal performance in the real world. Recently, machine-learning methods have been applied to learn MILP models for decision quality rather than how accurately they model the real-world problem. However, these approaches typically rely on supervised learning, assume access to optimal decisions, and use surrogates for the MILP gradients. In this work, we introduce a proof-of-concept corl framework that end-to-end fine-tunes an MILP scheme using reinforcement learning (RL) on real-world data to maximize its operational performance. We enable this by casting an MILP solved by B&B as a differentiable stochastic policy compatible with RL. We validate the corl method in two illustrative combinatorial sequential decision-making examples.