Chest X-ray (CXR) abnormality classification faces several challenges: (i) limited training data; (ii) training and evaluation sets that are derived from different domains; and (iii) classes that appear during training may have partial overlap with classes of interest during evaluation. We propose an integrated framework called Generalized Cross-Domain Multi-Label Few-Shot Learning (GenCDML-FSL), which supports class overlap during training and evaluation, cross-domain transfer, and few-shot learning for multi-label CXR image classification. Additionally, we introduce Generalized Episodic Training (GenET), a strategy that trains models to handle the challenges observed in GenCDML-FSL scenario. Our approach outperforms transfer learning, hybrid transfer learning, and multi-label meta-learning on multiple datasets.