We address the challenge of extracting structured information from business\ndocuments without detailed annotations. We propose Deep Conditional\nProbabilistic Context Free Grammars (DeepCPCFG) to parse two-dimensional\ncomplex documents and use Recursive Neural Networks to create an end-to-end\nsystem for finding the most probable parse that represents the structured\ninformation to be extracted. This system is trained end-to-end with scanned\ndocuments as input and only relational-records as labels. The\nrelational-records are extracted from existing databases avoiding the cost of\nannotating documents by hand. We apply this approach to extract information\nfrom scanned invoices achieving state-of-the-art results despite using no\nhand-annotations.\n