`boost_tree()`

is a way to generate a *specification* of a model
before fitting and allows the model to be created using
different packages in R or via Spark. The main arguments for the
model are:

`mtry`

: The number of predictors that will be randomly sampled at each split when creating the tree models.`trees`

: The number of trees contained in the ensemble.`min_n`

: The minimum number of data points in a node that is required for the node to be split further.`tree_depth`

: The maximum depth of the tree (i.e. number of splits).`learn_rate`

: The rate at which the boosting algorithm adapts from iteration-to-iteration.`loss_reduction`

: The reduction in the loss function required to split further.`sample_size`

: The amount of data exposed to the fitting routine.`stop_iter`

: The number of iterations without improvement before stopping.

These arguments are converted to their specific names at the
time that the model is fit. Other options and arguments can be
set using the `set_engine()`

function. If left to their defaults
here (`NULL`

), the values are taken from the underlying model
functions. If parameters need to be modified, `update()`

can be used
in lieu of recreating the object from scratch.

boost_tree( mode = "unknown", mtry = NULL, trees = NULL, min_n = NULL, tree_depth = NULL, learn_rate = NULL, loss_reduction = NULL, sample_size = NULL, stop_iter = NULL )

mode | A single character string for the type of model. Possible values for this model are "unknown", "regression", or "classification". |
---|---|

mtry | A number for the number (or proportion) of predictors that will
be randomly sampled at each split when creating the tree models ( |

trees | An integer for the number of trees contained in the ensemble. |

min_n | An integer for the minimum number of data points in a node that is required for the node to be split further. |

tree_depth | An integer for the maximum depth of the tree (i.e. number
of splits) ( |

learn_rate | A number for the rate at which the boosting algorithm adapts
from iteration-to-iteration ( |

loss_reduction | A number for the reduction in the loss function required
to split further ( |

sample_size | A number for the number (or proportion) of data that is
exposed to the fitting routine. For |

stop_iter | The number of iterations without improvement before
stopping ( |

The data given to the function are not saved and are only used
to determine the *mode* of the model. For `boost_tree()`

, the
possible modes are "regression" and "classification".

The model can be created using the `fit()`

function using the
following *engines*:

R:

`"xgboost"`

(the default),`"C5.0"`

Spark:

`"spark"`

For this model, other packages may add additional engines. Use
`show_engines()`

to see the current set of engines.

For models created using the spark engine, there are
several differences to consider. First, only the formula
interface to via `fit()`

is available; using `fit_xy()`

will
generate an error. Second, the predictions will always be in a
spark table format. The names will be the same as documented but
without the dots. Third, there is no equivalent to factor
columns in spark tables so class predictions are returned as
character columns. Fourth, to retain the model object for a new
R session (via `save()`

), the `model$fit`

element of the `parsnip`

object should be serialized via `ml_save(object$fit)`

and
separately saved to disk. In a new session, the object can be
reloaded and reattached to the `parsnip`

object.

Engines may have pre-set default arguments when executing the model fit call. For this type of model, the template of the fit calls are below:

boost_tree() %>% set_engine("xgboost") %>% set_mode("regression") %>% translate()

## Boosted Tree Model Specification (regression) ## ## Computational engine: xgboost ## ## Model fit template: ## parsnip::xgb_train(x = missing_arg(), y = missing_arg(), nthread = 1, ## verbose = 0)

boost_tree() %>% set_engine("xgboost") %>% set_mode("classification") %>% translate()

## Boosted Tree Model Specification (classification) ## ## Computational engine: xgboost ## ## Model fit template: ## parsnip::xgb_train(x = missing_arg(), y = missing_arg(), nthread = 1, ## verbose = 0)

Note that, for most engines to `boost_tree()`

, the `sample_size`

argument is in terms of the *number* of training set points. The
`xgboost`

package parameterizes this as the *proportion* of training set
samples instead. When using the `tune`

, this **occurs automatically**.

If you would like to use a custom range when tuning `sample_size`

, the
`dials::sample_prop()`

function can be used in that case. For example,
using a parameter set:

mod <- boost_tree(sample_size = tune()) %>% set_engine("xgboost") %>% set_mode("classification") # update the parameters using the `dials` function mod_param <- mod %>% parameters() %>% update(sample_size = sample_prop(c(0.4, 0.9)))

For this engine, tuning over `trees`

is very efficient since the same
model object can be used to make predictions over multiple values of
`trees`

.

Note that `xgboost`

models require that non-numeric predictors (e.g.,
factors) must be converted to dummy variables or some other numeric
representation. By default, when using `fit()`

with `xgboost`

, a one-hot
encoding is used to convert factor predictors to indicator variables.

Finally, in the classification mode, non-numeric outcomes (i.e.,
factors) are converted to numeric. For binary classification, the
`event_level`

argument of `set_engine()`

can be set to either `"first"`

or `"second"`

to specify which level should be used as the event. This
can be helpful when a watchlist is used to monitor performance from with
the xgboost training process.

boost_tree() %>% set_engine("C5.0") %>% set_mode("classification") %>% translate()

## Boosted Tree Model Specification (classification) ## ## Computational engine: C5.0 ## ## Model fit template: ## parsnip::C5.0_train(x = missing_arg(), y = missing_arg(), weights = missing_arg())

Note that `C50::C5.0()`

does not require factor
predictors to be converted to indicator variables. `fit()`

does not
affect the encoding of the predictor values (i.e. factors stay factors)
for this model.

For this engine, tuning over `trees`

is very efficient since the same
model object can be used to make predictions over multiple values of
`trees`

.

boost_tree() %>% set_engine("spark") %>% set_mode("regression") %>% translate()

## Boosted Tree Model Specification (regression) ## ## Computational engine: spark ## ## Model fit template: ## sparklyr::ml_gradient_boosted_trees(x = missing_arg(), formula = missing_arg(), ## type = "regression", seed = sample.int(10^5, 1))

boost_tree() %>% set_engine("spark") %>% set_mode("classification") %>% translate()

## Boosted Tree Model Specification (classification) ## ## Computational engine: spark ## ## Model fit template: ## sparklyr::ml_gradient_boosted_trees(x = missing_arg(), formula = missing_arg(), ## type = "classification", seed = sample.int(10^5, 1))

`fit()`

does not affect the encoding of the predictor values
(i.e. factors stay factors) for this model.

The standardized parameter names in parsnip can be mapped to their original names in each engine that has main parameters. Each engine typically has a different default value (shown in parentheses) for each parameter.

parsnip | xgboost | C5.0 | spark |

tree_depth | max_depth (6) | NA | max_depth (5) |

trees | nrounds (15) | trials (15) | max_iter (20) |

learn_rate | eta (0.3) | NA | step_size (0.1) |

mtry | colsample_bynode (character(0)) | NA | feature_subset_strategy (see below) |

min_n | min_child_weight (1) | minCases (2) | min_instances_per_node (1) |

loss_reduction | gamma (0) | NA | min_info_gain (0) |

sample_size | subsample (1) | sample (0) | subsampling_rate (1) |

stop_iter | early_stop (NULL) | NA | NA |

For spark, the default `mtry`

is the square root of the number of
predictors for classification, and one-third of the predictors for
regression.

#> # A tibble: 5 x 2 #> engine mode #> <chr> <chr> #> 1 xgboost classification #> 2 xgboost regression #> 3 C5.0 classification #> 4 spark classification #> 5 spark regressionboost_tree(mode = "classification", trees = 20)#> Boosted Tree Model Specification (classification) #> #> Main Arguments: #> trees = 20 #>#> Boosted Tree Model Specification (regression) #> #> Main Arguments: #> mtry = varying() #>