Broadband linear polarization from magnetized stellar atmospheres. II: The influence of damping on net spectral line polarization