In collaboration with Guillaume Maillard, we study the estimation of a regression function under weak assumptions on the error distribution. In particular, we do not assume that the errors are i.i.d., nor that they have finite variance or exponential moments; we only require them to be independent and centred (hence integrable). In particular, when the errors are square-integrable, they may, for instance, be heteroscedastic.

Within this statistical framework, we introduce a generic estimation method (a surrogate to the classical least squares) that yields estimators whose performance automatically adapts to the integrability properties of the errors. For these estimators, we establish non-asymptotic risk bounds for the L_1-loss. When the regression function belongs to a linear space and the errors are Gaussian but not necessarily i.i.d., these estimators exhibit remarkable robustness properties: they may converge at parametric rate (up to a logarithmic factor) in situations where the least squares estimator is not even consistent!

We shall illustrate the properties of these new estimators for the purpose of estimating a regression function under a shape constraint, such as monotonicity, unimodality, or convexity. We shall show that they are not only robust with respect to these a priori shape assumptions, but also exhibit adaptation properties which are similar to those established for the least squares under the additional assumptions that the errors are i.i.d. and square integrable.