Modern 5G/6G radio access networks are increasingly programmable through O-RAN, yet their operational complexity has grown with disaggregation, open interfaces, and fine-grained control parameters. While RAN-side analytics and telemetry mechanisms, such as KPI-based monitoring and mobility event reporting, provide visibility into network behavior, operators still face challenges in correlating heterogeneous events and safely translating observations into actionable configuration changes. This paper presents an LLM-based Net Analyzer rApp for the O-RAN Non-RT RIC that enables explainable and safe, human-in-the-loop automation for RAN operations. The proposed rApp adopts an event-informed, batch-triggered reasoning framework in which mobility events are first interpreted, anomalies are confirmed through targeted log inspection, configurations are inspected via tool-gated access, and minimal configuration changes are proposed only after explicit operator approval. The architecture enforces a strict separation between reasoning and actuation, ensuring auditability and operational safety. The system is implemented and demonstrated on a real O-RAN testbed using a reproducible ping-pong handover scenario, illustrating how large language models can function as reasoning co-pilots that transform raw RAN telemetry into structured explanations and controlled remediation workflows, complementing existing analytics-only approaches in the Non-RT RIC.