Silica aerogels are porous ultralight materials with exceptional physical properties that are promising materials for thermal insulation applications. This paper theoretically and experimentally investigates the spectral scattering and absorption coefficients of a porous silica aerogel. Silica aerogel samples were prepared with the same compositions and various thicknesses using the sol-gel technique and supercritical drying. The spectral normal-hemispherical transmittances and reflectances of the silica aerogel samples with various thicknesses were measured for wavelengths of 0.38–15 μm. The reflectance and transmittance are higher at short wavelengths than in the infrared region due to the strong scattering and weak absorption at short wavelengths. The thicker samples strongly attenuate the spectral normal-hemispherical transmittance, but have little effect on the spectral normal-hemispherical reflectance. A modified two-flux radiative transfer model was used to analyze the radiation propagation in the silica aerogel with a rough surface morphology and millimeter thicknesses to develop theoretical expressions for the spectral directional-hemispherical reflectance and transmittance. Then, the optical constants, including the absorption coefficient and the scattering coefficient, were determined for wavelengths of 0.38–15 μm based on experimental data by the least-squares algorithm. The results show that when considering the radiation propagation inside the sample, the surface reflection at the air–aerogel interface can be neglected for aerogel thicker than 1.1 mm when the absorbing and scattering effects inside the sample are quite important. The analysis provides valuable data for the optical properties for silica aerogel applications.