Missing Word Prediction Using Stochastic Steady State and Bi-Graph Transitions
摘要
This paper presents a system for predicting missing words from text using Stochastic Steady State and Bi-graph Transitions, wherein graph-based probabilistic methods are employed. Our frame work utilizes the context-aware Bi-graph model that accepts the bidirectional relationship of words in the corpus. When the test sequence is equivalent to that in the training data. If not, it seamlessly falls back on a steady-state method inspired by PageRank probabilities when the vocabulary of the input sequence goes beyond the training criteria. The bi-graph approach computes candidate scores by combining transition probabilities from the surrounding context while applying a weighting analogous to inverse term frequency, explained in Results and Methodologies, to reduce final selection bias toward common words. A smoothed transition matrix was constructed for out-of-vocabulary sequences, and an iterative convergence algorithm was used to obtain the steady-state probabilities across the vocabulary. Experiments carried out across wide-ranging linguistic contexts show that the system can deal with familiar and novel text patterns. All these features give the hybrid architecture a flexible answer to the missing word prediction problem with no need for a large amount of training data or complicated neural architectures. Yet, it remains interpretable through explicit probability models.