This study explores the design and application of Complex-Valued Convolutional Neural Networks (CVCNNs) in the domain of audio signal processing, with a focus on utilizing phase information often neglected in real-valued networks. Empirical evaluations are conducted across three stages. First, CVCNNs are benchmarked on standard image datasets (MNIST, KMNIST, FMNIST), where they demonstrate competitive performance with real-valued CNNs, even under synthetic perturbations. Although our focus is audio signal processing, we first evaluate CVCNNs on image datasets to establish baseline performance and validate training stability before applying them to audio tasks. In the second experiment, we focus on audio classification using Mel-Frequency Cepstral Coefficients (MFCCs). CVCNNs trained on real-valued MFCCs slightly outperform real-valued CNNs; however, incorporating phase information into the input does not lead to measurable performance gains, highlighting the difficulty of exploiting phase information without phase-aware architectural design. Finally, a third experiment introduces Graph Neural Networks (GNNs) to explicitly model phase information via edge weighting, where the inclusion of phase yields measurable gains in binary and multi-class genre classification, affirming its relevance when appropriately structured. These results underscore the expressive capacity of complex-valued architectures and confirm phase as a meaningful and exploitable feature in audio processing applications. Whereas current methods show promise, future advances in phase-aware design will be essential to leverage the potential of complex representations in neural networks.