The effect of local thermal nonequilibrium (LTNE) on the stability of natural convection in a vertical porous slab saturated by an Oldroyd-B fluid is investigated. The vertical walls of the slab are impermeable and maintained at constant but different temperatures. A two-field model that represents the fluid and solid phase temperature fields separately is used for heat transport equation. The resulting stability eigenvalue problem is solved numerically using Chebyshev collocation method as the energy stability analysis becomes ineffective in deciding the stability of the system. Despite the basic state remains the same for Newtonian and viscoelastic fluids, it is observed that the base flow is unstable for viscoelastic fluids and this result is qualitatively different from Newtonian fluids. The results for Maxwell fluid are delineated as a particular case from the present study. It is found that the viscoelasticity has both stabilizing and destabilizing influence on the flow. Increase in the value of interphase heat transfer coefficient $Ht$, strain retardation parameter $\Lambda 2$ and diffusivity ratio $\alpha $ portray stabilizing influence on the system while increasing stress relaxation parameter $\Lambda 1$ and porosity-modified conductivity ratio $\gamma $ exhibit an opposite trend.