Sign up to save your podcasts
Or
Share Boosting in structured additive models
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
Boosting in structured additive models
Variable selection and model choice are of major concern in many statistical
applications, especially in regression models for high-dimensional data.
Boosting is a convenient statistical method that combines model fitting with
intrinsic model selection. We investigate the impact of base-learner
specification on the performance of boosting as a model selection procedure. We
show that variable selection may be biased if the base-learners have different
degrees of flexibility, both for categorical covariates and for smooth effects
of continuous covariates. We investigate these problems from a theoretical
perspective and suggest a framework for unbiased model selection based on a
general class of penalized least squares base-learners. Making all base-learners
comparable in terms of their degrees of freedom strongly reduces the selection
bias observed with naive boosting specifications. Furthermore, the definition of
degrees of freedom that is used in the smoothing literature is questionable in
the context of boosting, and an alternative definition is theoretically derived.
The importance of unbiased model selection is demonstrated in simulations and in
an application to forest health models.
A second aspect of this thesis is the expansion of the boosting algorithm to new
estimation problems: by using constraint base-learners, monotonicity constrained
effect estimates can be seamlessly incorporated in the existing boosting
framework. This holds for both, smooth effects and ordinal variables.
Furthermore, cyclic restrictions can be integrated in the model for smooth
effects of continuous covariates. In particular in time-series models, cyclic
constraints play an important role. Monotonic and cyclic constraints of smooth
effects can, in addition, be extended to smooth, bivariate function estimates.
If the true effects are monotonic or cyclic, simulation studies show that
constrained estimates are superior to unconstrained estimates. In three case
studies (the modeling the presence of Red Kite in Bavaria, the modeling of
activity profiles for Roe Deer, and the modeling of deaths caused by air
pollution in Sao Paulo) it is shown that both constraints can be integrated
in the boosting framework and that they are easy to use.
All described results were included in the R add-on package mboost.
View all episodes
By Ludwig-Maximilians-Universität München
5
11 ratings
Variable selection and model choice are of major concern in many statistical
applications, especially in regression models for high-dimensional data.
Boosting is a convenient statistical method that combines model fitting with
intrinsic model selection. We investigate the impact of base-learner
specification on the performance of boosting as a model selection procedure. We
show that variable selection may be biased if the base-learners have different
degrees of flexibility, both for categorical covariates and for smooth effects
of continuous covariates. We investigate these problems from a theoretical
perspective and suggest a framework for unbiased model selection based on a
general class of penalized least squares base-learners. Making all base-learners
comparable in terms of their degrees of freedom strongly reduces the selection
bias observed with naive boosting specifications. Furthermore, the definition of
degrees of freedom that is used in the smoothing literature is questionable in
the context of boosting, and an alternative definition is theoretically derived.
The importance of unbiased model selection is demonstrated in simulations and in
an application to forest health models.
A second aspect of this thesis is the expansion of the boosting algorithm to new
estimation problems: by using constraint base-learners, monotonicity constrained
effect estimates can be seamlessly incorporated in the existing boosting
framework. This holds for both, smooth effects and ordinal variables.
Furthermore, cyclic restrictions can be integrated in the model for smooth
effects of continuous covariates. In particular in time-series models, cyclic
constraints play an important role. Monotonic and cyclic constraints of smooth
effects can, in addition, be extended to smooth, bivariate function estimates.
If the true effects are monotonic or cyclic, simulation studies show that
constrained estimates are superior to unconstrained estimates. In three case
studies (the modeling the presence of Red Kite in Bavaria, the modeling of
activity profiles for Roe Deer, and the modeling of deaths caused by air
pollution in Sao Paulo) it is shown that both constraints can be integrated
in the boosting framework and that they are easy to use.
All described results were included in the R add-on package mboost.
More shows like Fakultät für Mathematik, Informatik und Statistik - Digitale Hochschulschriften der LMU - Teil 01/02
View all
Theoretical Physics Schools (ASC)
2 Listeners
Katholisch-Theologische Fakultät - Digitale Hochschulschriften der LMU
0 Listeners
MCMP – Mathematical Philosophy (Archive 2011/12)
6 Listeners
Hegel lectures by Robert Brandom, LMU Munich
6 Listeners
John Lennox - Hat die Wissenschaft Gott begraben?
3 Listeners
MCMP – Philosophy of Science
2 Listeners
MCMP – Philosophy of Mathematics
2 Listeners
Epistemology and Philosophy of Science: Prof. Dr. Stephan Hartmann – HD
1 Listeners
MCMP – Philosophy of Physics
4 Listeners
Center for Advanced Studies (CAS) Research Focus Evolutionary Biology (LMU) - HD
0 Listeners