The aim of this thesis is to study the properties of a reactive plasma as well as the structural, optical and electrical properties of tantalum oxynitride thin films (TaOxNy) prepared by radiofrequency sputtering. The elaboration of this ternary material by sputtering a pure tantalum target using plasma containing both of argon, oxygen and nitrogen is complex due to the target-poisoning phenomenon. The analysis of the composition of the plasma by optical emission spectroscopy and monitoring the evolution of some representative line of excited species in this environment, allow us to determine the optimal conditions to deposit TaOxNy films over a wide range of composition. Thanks to a study by X-ray diffraction and X-ray photoelectron spectroscopy, we followed the structural evolution of the films subjected or not to a rapid thermal annealing. We showed by what were constituted the amorphous and crystalline parts of the films and determined the size of the crystalline domains. Finally, the optical properties (refractive index, optical gap, Urbach parameter) and dielectric behavior have been correlated with the structure of materials.