The field of Natural Language Processing (NLP) has undergone transformative changes with the advent of generative pre-trained language models (PLMs), which, despite their proficiency in generating coherent text, exhibit limitations such as misalignment with human values, interpretability issues, a propensity for hallucination, among others. Addressing these limitations, interactive natural language processing (iNLP) has emerged as a promising paradigm, redefining the interaction scope beyond human-in-the-loop to include knowledge bases, models, tools, and environments. This paper introduces iNLP, where language models serve as agents in an iterative loop of observation, action, and feedback with these diverse external entities. We aim to solidify the conceptual foundation of iNLP, offer a systematic classification, and discuss the broader implications, including evaluation methods, applications, ethical considerations, and future research directions. Our survey critically examines the potential of iNLP to overcome current model shortcomings and align with the ultimate goals of AI, guiding future research towards more sophisticated, socially embodied AI systems.