MIXTURE OF CLASSIFIERS AND MODEL INDETIFIABILITY

Generalized Additive Multi-Models (GAM-M) can be considered as the starting point of a research project concerning semiparametric models. Starting from GAM-M, we introduced a somewhat similar approach based on the joint use of mixtures of classifiers (models as well as smoothers). Main aim was to overcome some of the drawbacks deriving from the use of semiparametric models, such as overfitting and inaccuracy. In this respect, we used bootstrap in order to define a ranking of the different classifiers and a mixing parameters estimation criterion. This approach has been named Generalized Additive Multi-Mixture Model (GAM-MM), and results very effective when dealing with large nonlinear data structures and outliers.
In GAM-MM, the problem of the correct identifiability of the final model has also been studied. In fact, for the estimation algorithms based on the Iterative Reweighted Least Squares (IRLS) principle, the entering order of the predictors in the model could considerably affect the results of the estimation. The estimation procedure used in GAM-MM is also sensitive to this problem. For this reason, we define an ordering entrance of the predictor in the model based on a scoring measure. This measure has been evaluated empirically in a simulation study and for an application on financial data, for the problem of the estimation of the EUR/USD exchange rate on the basis of a set of macroeconomic indicators.
The model identifiability problem has also been studied for Generalized Additive Models (GAM). In particular, an automatic procedure for the identification of both the most appropriate smoother for each predictor and the definition of an entering order of the predictors in the model has been introduced. We referred to the Bagging procedure of Breiman for reducing the risk of model misspecification and introduced an empirical measure for evaluating the adequacy of the estimation deriving from the set of possible smoothers to be associated to each predictor. This procedure has been implemented in the framework of the backfitting algorithm, where is applied in each sub-iteration.