Getting an overview over the legal domain has become challenging, especially\nin a broad, international context. Legal question answering systems have the\npotential to alleviate this task by automatically retrieving relevant legal\ntexts for a specific statement and checking whether the meaning of the\nstatement can be inferred from the found documents. We investigate a\ncombination of the BM25 scoring method of Elasticsearch with word embeddings\ntrained on English translations of the German and Japanese civil law. For this,\nwe define criteria which select a dynamic number of relevant documents\naccording to threshold scores. Exploiting two deep learning classifiers and\ntheir respective prediction bias with a threshold-based answer inclusion\ncriterion has shown to be beneficial for the textual entailment task, when\ncompared to the baseline.\n