Discovering the logical sequence of events is one of the cornerstones in\nNatural Language Understanding. One approach to learn the sequence of events is\nto study the order of sentences in a coherent text. Sentence ordering can be\napplied in various tasks such as retrieval-based Question Answering, document\nsummarization, storytelling, text generation, and dialogue systems.\nFurthermore, we can learn to model text coherence by learning how to order a\nset of shuffled sentences. Previous research has relied on RNN, LSTM, and\nBiLSTM architecture for learning text language models. However, these networks\nhave performed poorly due to the lack of attention mechanisms. We propose an\nalgorithm for sentence ordering in a corpus of short stories. Our proposed\nmethod uses a language model based on Universal Transformers (UT) that captures\nsentences' dependencies by employing an attention mechanism. Our method\nimproves the previous state-of-the-art in terms of Perfect Match Ratio (PMR)\nscore in the ROCStories dataset, a corpus of nearly 100K short human-made\nstories. The proposed model includes three components: Sentence Encoder,\nLanguage Model, and Sentence Arrangement with Brute Force Search. The first\ncomponent generates sentence embeddings using SBERT-WK pre-trained model\nfine-tuned on the ROCStories data. Then a Universal Transformer network\ngenerates a sentence-level language model. For decoding, the network generates\na candidate sentence as the following sentence of the current sentence. We use\ncosine similarity as a scoring function to assign scores to the candidate\nembedding and the embeddings of other sentences in the shuffled set. Then a\nBrute Force Search is employed to maximize the sum of similarities between\npairs of consecutive sentences.\n