Raw earth is one of the most used building material in the world. Each earthen material is a mix of different sizes grains (stones, gravels, sands, silts and clays) in various proportions. The finest particles, which constitute the binding material, differ from other grains by their small size, which is associated with colloidal properties when mixed with water. Clay, as defined by size (particles smaller than two microns), usually contains a mixture of clays, as defined by mineralogy (hydrated phyllosilicates), with iron, aluminum and silicium oxides. To understand this building material, earthen material is considered initially as a granular material. Physics of dry granular media and physics of wet granular media are explored in order to describe the physical behavior of this material. Innovative building systems, which come from this understanding, are presented. In a second step, clay is considered as a colloidal material. Like many natural systems, clay binder is a mixture of particles with different surface properties, containing both permanent negative charges and variable charges, depending of pH and ionic strength. These mixed systems are called heterocoagulated systems. Different sujects for further researches, that follow this understanding, are presented.