Color-centers in solids have emerged as promising candidates for quantum photonic computing, communications, and sensing applications. Defects in hexagonal boron nitride(hBN) possess high-brightness, room-temperature quantum emission, but their large spectral variability and unknown local structure significantly challenge their technological utility. Here, we directly correlate hBN quantum emission with its local, atomic-scale crystalline structure using correlated photoluminescence (PL) and cathodoluminescence (CL) spectroscopy. Across 20 emitters, we observe zero phonon lines (ZPLs) in PL and CL ranging from 540-720 nm. CL mapping reveals that multiple defects and distinct defect species located within an optically-diffraction-limited region can each contribute to the observed PL spectra. Through high resolution transmission electron imaging, we find that emitters are located in regions with multiple fork-like dislocations. Additionally, local strain maps indicate that strain is not responsible for observed ZPL spectral range, though it can enable spectral tuning of particular emitters. While many emitters have identical ZPLs in CL and PL, others exhibit reversible but distinct CL and PL peaks; density functional calculations indicate that defect complexes and charge-state transitions influence such emission spectra. Our results highlight the sensitivity of defect-driven quantum emission to the surrounding crystallography, providing a foundation for atomic-scale optical characterization.