R gbm – How to Use R gbm with Distribution=”adaboost”?

boostingr

Documentation states that R gbm with distribution = "adaboost" can be used for 0-1 classification problem. Consider the following code fragment:

gbm_algorithm <- gbm(y ~ ., data = train_dataset, distribution = "adaboost", n.trees = 5000)
gbm_predicted <- predict(gbm_algorithm, test_dataset, n.trees = 5000)

It can be found in the documentation that predict.gbm

Returns a vector of predictions. By default the predictions are on the scale of f(x).

However the particular scale is not clear for the case of distribution = "adaboost".

Could anyone help with the interpretation of predict.gbm return values and provide an idea of conversion to the 0-1 output?

Best Answer

The adaboost method gives the predictions on logit scale. You can convert it to the 0-1 output:

gbm_predicted<-plogis(2*gbm_predicted)

note the 2* inside the logis

Related Solutions

Solved – How to use CART for AdaBoost

AdaBoost makes no assumption about the internals of a classifier, so you should only need the final output.

The weight is used as a coefficient for the classifier (function) trained in the given iteration. Boosting will give you a weighted sum of trees. The leaf regions are typically different from tree to tree, so to approximate the boosted classifier by a single tree, you'd need a tree of a much larger tree depth than for the individual trees.

R Neural Network Model with Target Vector Containing Survival Predictions

Why not just put binary indicator for event as a target variable and length of time period as a explanatory variable (plus other covariates)? If event happens then target is 1 and time period is calculated as time until event happens - start time. For some observations where target is 0 this time period is 36, if measured in months. For some observations where target is 1 it can be much less.

Can there be panel attrition where some observation is removed from the data set before whole surveillance period is over? That must be accounted somehow.

To get individual survival probabilities for various time intervals you score new data set with just developed model object i times where i number of different time periods and particular time period has value n. Then just concatenate i vectors containing time period specific probabilities.

Idea is that measured time period with other covariates accounts for the survival probability conditional on length of time in the observational study.

EDIT:

I looked for R package neuralnet. You can have individual time period specific survival events in the target matrix in the following way. C1 is covariate1, T1 is vector of survival events in the time period 1 etc. Your data frame / matrix could look like this:

ID T1 T2 T3 T4 T5 T6 C1 C2 C3 CN  
1 1 1 1 1 1 1 X11 X12 X13 X1N  
2 1 0 0 0 0 0 X21 X22 X23 X2N  
..

Use following code:

survexample=neuralnet(T1+T2+T3+T4+T5+T6~C1+C2+...+CN,data=example,hidden=n,err.fct="ce",linear.output=FALSE)

This example code does classifying and forces output vector values to be in the range of [0,1].

Best Answer

Related Solutions

Solved – How to use CART for AdaBoost

R Neural Network Model with Target Vector Containing Survival Predictions

Related Question