Encoded (or ciphered) manuscripts are a special type of historical documents\nthat contain encrypted text. The automatic recognition of this kind of\ndocuments is challenging because: 1) the cipher alphabet changes from one\ndocument to another, 2) there is a lack of annotated corpus for training and 3)\ntouching symbols make the symbol segmentation difficult and complex. To\novercome these difficulties, we propose a novel method for handwritten ciphers\nrecognition based on few-shot object detection. Our method first detects all\nsymbols of a given alphabet in a line image, and then a decoding step maps the\nsymbol similarity scores to the final sequence of transcribed symbols. By\ntraining on synthetic data, we show that the proposed architecture is able to\nrecognize handwritten ciphers with unseen alphabets. In addition, if few\nlabeled pages with the same alphabet are used for fine tuning, our method\nsurpasses existing unsupervised and supervised HTR methods for ciphers\nrecognition.\n