Stability of a supercritical fluid diffusing layer with mixed boundary conditions