We propose encoder-centric stepwise models for extractive summarization using\nstructured transformers -- HiBERT and Extended Transformers. We enable stepwise\nsummarization by injecting the previously generated summary into the structured\ntransformer as an auxiliary sub-structure. Our models are not only efficient in\nmodeling the structure of long inputs, but they also do not rely on\ntask-specific redundancy-aware modeling, making them a general purpose\nextractive content planner for different tasks. When evaluated on CNN/DailyMail\nextractive summarization, stepwise models achieve state-of-the-art performance\nin terms of Rouge without any redundancy aware modeling or sentence filtering.\nThis also holds true for Rotowire table-to-text generation, where our models\nsurpass previously reported metrics for content selection, planning and\nordering, highlighting the strength of stepwise modeling. Amongst the two\nstructured transformers we test, stepwise Extended Transformers provides the\nbest performance across both datasets and sets a new standard for these\nchallenges.\n