Plumes are known to migrate through the ductile quasi-Newtonian mantle, while dikes fracture and propagate through the solid lithosphere. However, depending on the timescale, the lithosphere presents solid as well as viscous properties. To determine what happens in the complex case, where instabilities propagate through a visco-elastic matrix, we performed a combined study of laboratory experiments and numerical simulations. Here we investigate the development of thermal plumes in aqueous solutions of Carbopol, a polymer gel, forming a continuous network of micrometric sponges. This fluid is shear thinning and has a yield-stress σ₀, whereby flow occurs only if the local stress exceeds this critical value σ₀. Below this value, the fluid acts as an elastic solid. The rheological properties of the solutions can be systematically varied by varying the Carbopol concentration. The setup consists of a localized heat-source operated at constant power, placed at the centre of a square tank. Depending on the ratio of the thermally induced stresses and the yield stress, Y₀, three different regimes may be obtained. For low Y₀<Yc₁ no motion occurs, whereas for Yc₁<Y₀Yc₂. We show that the critical parameters (Yc₁,Yc₂) strongly depend on the geometry of the heating. Combined temperature and velocity field measurements show that the morphology of the plume resembles a finger, contrary to the mushroom-like shape encountered in Newtonian fluids. Numerical simulations using a purely viscous description, where the rheology of the fluid is described by a regularized Herschel-Bulkley model, are sufficient to capture the plume dynamics. A detailed parametric study shows that the plume dynamics are governed by the interplay between yield stress, buoyancy induced stress and viscous stresses. We identify two non-dimensional parameters: the yield parameter Ψ comparing the buoyancy induced stress to the yield stress, and the Bingham number Bi comparing the yield stress to the viscous stresses. We show that a plume can rise only if both parameters are supercritical, i.e. if buoyancy induced stress and viscous stresses each overcome the yield stress. Therefore the plume may come to a halt before it reaches the surface. We propose scaling laws for the plume stem velocity, temperature and the size of the shear zone in the steady state. We show that the scaling laws describe the behaviour in the plume stem reasonably well, if the yield stress is neglected and only the shear thinning behaviour is taken into account. Applying the non-dimensional parameters to Earth places severe constraints on the strength of mantle and lithosphere. The maximum strength that allows for thermal instabilities to penetrate the lithosphere or upper mantle is in between 100 kPa and 100 MPa, and strongly depends on the size and buoyancy of the anomaly.