Predicting the success of startup companies, defined as achieving an exit through acquisition or IPO, is a critical problem in entrepreneurship and innovation research. Datasets such as Crunchbase provide both structured information (e.g., funding rounds, industries, and investor networks) and unstructured text (e.g., company descriptions), but effectively leveraging such heterogeneous data for prediction remains challenging. Traditional machine learning approaches often rely only on structured features and achieve moderate accuracy, while large language models (LLMs) offer strong reasoning capabilities but are not readily adapted to domain-specific business data. We present CrunchLLM, a domain-adapted and backbone-agnostic LLM framework for startup success prediction. CrunchLLM integrates structured company attributes with unstructured textual narratives and applies parameter-efficient fine-tuning together with prompt optimization to specialize foundation models for entrepreneurship data. Importantly, our framework introduces a self-verifiable multitask objective, in which the justification loss serves as a training-time constraint on classification, together with a hierarchically ordered input encoding that reduces the tendency of long unstructured company narratives to overshadow structured business attributes. These methodological innovations yield more reliable and feature-grounded predictions than conventional prompt-based LLM adaptation. Our approach achieves 89% accuracy on the Crunchbase startup success prediction task, significantly outperforming traditional classifiers and baseline LLMs. Beyond predictive performance, CrunchLLM generates interpretable reasoning traces that support its predictions, enhancing transparency and trustworthiness for financial and policy decision-makers. Overall, this work demonstrates how domain-aware LLM adaptation and structured–unstructured data fusion can advance predictive modeling of entrepreneurial outcomes, providing both a methodological framework and a practical tool for data-driven decision-making in venture capital and innovation policy.