ent values of $\lambda$.

Figure \[fig:cd\_individual\] shows critical difference plots for both subset selection methods. Class balanced selection shows a clear trend that increasing $\lambda$ improves the RMSE, with the average rank for $\lambda=1$ being exactly 4. For random-pair selection, choosing $\lambda=3$ is shown to be statistically equivalent to $\lambda=1$, while higher values of $\lambda$ give superior results on average.

Ensembles of Nested Dichotomies
-------------------------------

Typically, nested dichotomies are utilised in an ensemble setting, so we investigate the predictive performance of ensembles of ten nested dichotomies with multiple subset evaluation, with bagging and AdaBoost employed as the ensemble methods.

### Class Threshold. {#class-threshold. .unnumbered}

[l]{}\

As previously discussed, the number of binary problems is reduced when multiple subset evaluation is applied. This could have negative a effect on ensemble diversity, and therefore potentially reduce predictive performance. To investigate this effect, we built ensembles of nested dichotomies with multiple subset evaluation by introducing a *class threshold*, the number of classes present at a node required to perform multiple subset evaluation, and varying its value from one to seven. We plot the test RMSE, relative to having a class threshold of one, averaged over the datasets from Table \[tab:datasets\], including standard errors, in Figure \[fig:threshold\]. Surprisingly, the RMSE increases monotonically, showing that the potentially reduced ensemble diversity does not have a negative effect on the RMSE for ensembles of this size. Therefore, we use a class threshold of one in our subsequent experiments. However, note that increasing the class threshold has a positive effect on training time, so it may be useful to apply it in practice.

### Number of Subsets. {#number-of-subsets. .unnumbered}

We now investigate the effect of $\lambda$ when using bagging and boosting. Figure \[fig:cd\_bagging\] shows critical