Controllable Image Captioning is a recent sub-field in the multi-modal task\nof Image Captioning wherein constraints are placed on which regions in an image\nshould be described in the generated natural language caption. This puts a\nstronger focus on producing more detailed descriptions, and opens the door for\nmore end-user control over results. A vital component of the Controllable Image\nCaptioning architecture is the mechanism that decides the timing of attending\nto each region through the advancement of a region pointer. In this paper, we\npropose a novel method for predicting the timing of region pointer advancement\nby treating the advancement step as a natural part of the language structure\nvia a NEXT-token, motivated by a strong correlation to the sentence structure\nin the training data. We find that our timing agrees with the ground-truth\ntiming in the Flickr30k Entities test data with a precision of 86.55% and a\nrecall of 97.92%. Our model implementing this technique improves the\nstate-of-the-art on standard captioning metrics while additionally\ndemonstrating a considerably larger effective vocabulary size.\n