The binary Be star $\delta$ Scorpii at high spectral and spatial resolution : II The circumstellar disk evolution after the periastron