A large number of physical systems present long range interactions : self-gravitating systems, plasmas, vortex interactions... and thus share some properties. In this thesis, we have another experimental system in view: cold atoms; in that case, absorption/emission of photons create effective long range interactions. During a specific time scale, the out-of-equilibrium dynamics of a long range system is described by the Vlasov, or Vlasov-Fokker-Planck, equation. The goal here is to study the out-of-equilibrium behavior for several particles systems with long range interaction, from theoretical to experimental as well as numerical. Firstly, we study the dynamics of a system of particles close to an inhomogeneous stationary state using the Vlasov equation. We show that, while Landau damping is present for short time scales, the long time behavior is governed by a power law relaxation. We check and valid our predictions using numerical simulations for the HMF model (a prototype of long range system). Next, we consider the center-of-mass and breathing oscillations of interacting particles system. Assuming an invariant shape for the particles distribution, we obtain two equations which approximatively describe evolution of these two modes for a large number of systems (long/short range, with/without thermal bath, ...). Finally we use the previous result to explore an instability of a cloud of cold atoms in a magneto-optical trap and analyze a regime possibly analogous to a 1d self-gravitating system.
