Lack of encyclopedic text contributors, especially on Wikipedia, makes automated text generation for low resource (LR) languages a critical problem. A step towards enabling this is to generate the structural outline of Wikipedia pages for LR languages. Hence, in this work, we propose and study OutlineGen, the problem of generating the outline of Wikipedia pages for LR languages using minimal information in the form of entity name, language and domain. The OutlineGen task is challenging because even within a (language, domain) pair, the outlines vary a lot across entities. Further, given the diversity of Wikipedia editors and audience, the outlines are not consistent across languages. First, we create a dataset, WikiOutlines, which contains Wikipedia section outlines from \(\sim \) 166K Wikipedia pages across 8 domains and 10 languages. Then, we investigate the effectiveness of non-neural weighted finite state automata as well as Transformer-based methods for this task. We make the code and data publicly available.