Aggregating data is fundamental to data analytics, data exploration, and\nOLAP. Approximate query processing (AQP) techniques are often used to\naccelerate computation of aggregates using samples, for which confidence\nintervals (CIs) are widely used to quantify the associated error. CIs used in\npractice fall into two categories: techniques that are tight but not correct,\ni.e., they yield tight intervals but only offer asymptotic guarantees, making\nthem unreliable, or techniques that are correct but not tight, i.e., they offer\nrigorous guarantees, but are overly conservative, leading to confidence\nintervals that are too loose to be useful. In this paper, we develop a CI\ntechnique that is both correct and tighter than traditional approaches.\nStarting from conservative CIs, we identify two issues they often face:\npessimistic mass allocation (PMA) and phantom outlier sensitivity (PHOS). By\ndeveloping a novel range-trimming technique for eliminating PHOS and pairing it\nwith known CI techniques without PMA, we develop a technique for computing CIs\nwith strong guarantees that requires fewer samples for the same width. We\nimplement our techniques underneath a sampling-optimized in-memory column store\nand show how to accelerate queries involving aggregates on a real dataset with\nspeedups of up to 124x over traditional AQP-with-guarantees and more than 1000x\nover exact methods.\n