With the rapid proliferation of video content across social media, surveillance, and education platforms, efficiently summarizing long videos into concise yet semantically faithful surrogates has become increasingly vital. Existing supervised methods achieve strong in-domain accuracy by learning from dense annotations but incur high labeling costs and limited cross-dataset generalization, while unsupervised approaches, though label-free, often miss high-level human semantics and fine-grained narrative cues. Recent zero-shot pipelines leverage large language models (LLMs) for training-free video summarization, yet remain highly sensitive to handcrafted prompts and dataset-specific normalization.To address these limitations, we propose a rubric-guided, pseudo-labeled prompting framework that converts a small subset of human annotations into high-confidence pseudo labels and aggregates them into structured, dataset-adaptive scoring rubrics for interpretable scene evaluation. During inference, first and last segments are scored based solely on their descriptions, whereas intermediate segments incorporate brief contextual summaries of adjacent scenes to assess narrative progression and redundancy. This contextual prompting enables the LLM to balance local salience and global coherence without parameter tuning.Our framework achieves consistent improvements across three benchmarks. On SumMe and TVSum, it reaches F1 scores of 57.58 and 63.05, surpassing the zero-shot baseline PROMPTS TO SUMMARIES (56.73 and 62.21) by +0.85 and +0.84, respectively, while approaching the performance of supervised models. On the query-focused QFVS benchmark, our method attains an average F1 of 53.79, exceeding the baseline (53.42) by +0.37 and demonstrating stable performance across all validation videos. These results confirm that rubric-guided pseudo labeling, combined with contextual prompting, effectively stabilizes LLM-based scoring and establishes a general, interpretable zero-shot paradigm applicable to both generic and query-focused video summarization.Code is available at: https://github.com/wuyuanli60-svg/Context-Aware-Pseudo-Label-Scoring-for-Zero-Shot-Video-Summarization.