A major concern of cooperative multi-agent reinforcement learning (MARL) in real-world applications is the ability to coordinate transparently with new teammates for a common goal. Two problem formulations, Zero-Shot Coordination (ZSC) and Ad-Hoc Teamplay (AHT) have garnered particular interest. We focus on AHT because it includes dynamic agents with no prior knowledge, while ZSC assumes a static policy based on shared knowledge of the environment dynamics. Communication is a common factor in both settings. It is standard to follow a pre-defined communication protocol, or to learn arbitrary communication protocols that implicitly suggest actions or share information. We argue that explicit suggestion-based communication allows for a higher theoretical team performance ceiling than information sharing or best response strategies. We also show that teacher-listener relationships can be learned in an ad-hoc settings for any pre-trained agent that can estimating the current value of an environment state. We show how to learn explicit commands in ad-hoc timescales through our algorithm, Multi-Armed Two-way Command Heuristic (MATCH). Finally, we provide an open source minimally complex environment, Lever Tic Tac Toe, which provides a computationally inexpensive equilibrium selection problem. We leverage Battle of the Sexes/Bach or Stravinsky, LeverNvNTTT, and OvercookedAI environments to provide a principled approach for evaluating future coordination algorithms in terms of their ability to address cooperation challenges.