h2o.stackedEnsemble have already have the parameter metalearner_nfolds, but we can't access the cross-validition prediction result.

[Yijuan-w]

ensemble_2 <- h2o.stackedEnsemble(x = predictors, y = response, training_frame = data_h2o,
                                base_models = list(xgb_model,dl_model, rf_model,glm_model, nb_model),
                                metalearner_algorithm = "glm",
                                metalearner_nfolds = 5,  
                                seed = 1)

it works well, but when I want to plot ROC curve to compare the base model and ensemble model, I can't get the prediction result.
it should also have the parameter keep_cross_validation_predictions
please！ Many many thanks!

[tomasfryda]

@Yijuan-w you can get cross-validation ROC curve from the metalearner like this:

perf <- h2o.performance(ensemble, xval = TRUE)
plot(perf, type="roc")

it should also have the parameter keep_cross_validation_predictions

You can do it by using metalearner_params:

ensemble <- h2o.stackedEnsemble(y = response, training_frame = df,
                                  base_models = models,
                                  metalearner_algorithm = "glm",
                                  metalearner_nfolds = 5,
                                  metalearner_params = list(
                                    keep_cross_validation_predictions = TRUE
                                  ), 
                                  seed = 1)

metalearner <- ensemble@model$metalearner_model
fold_id <- 1
cv_preds <- h2o.getFrame(metalearner@model$cross_validation_predictions[[fold_id]]$name)
cv_preds

[Yijuan-w]

wooo
Thank your so much!