Forecaster and MVForecaster Attributes

  • You can look up what metrics and estimators are available for you to use by initiating a Forecaster or MVForecaster instance and checking the object’s attributes. This notebook shows how to do that with Forecaster only, but the same attributes exist in MVForecaster, unless specified otherwise.

[1]:

from scalecast.Forecaster import Forecaster
from scalecast.MVForecaster import MVForecaster

f = Forecaster(
    y = [1,2,3,4], # required
    current_dates = ['2021-01-01','2021-02-01','2021-03-01','2021-04-01'], # required, can be a numbered index if dates not known/needed
    future_dates = None, # optional. this accepts an int type that counts the forecast horizon steps. future dates can be generated after the object is initiated.
    test_length = 0, # default is 0, but this accepts int or float types to determine the number/fraction of obs to hold out for model testing
    cis = False, # default is False, change to True if you want confidence intervals. requires a test set.
    metrics = [
        'rmse', # default
        'mape', # default
        'mae', # default
        'r2', # default
        'smape',
        'mse',
        'abias',
    ],
)

Forecaster.estimators

  • These are the the models that forecast and can be set by using f.set_estimator(...).

  • They come from popular machine learning libraries like scikit-learn, keras, statsmodels, and others.

  • More estimators can be added, assuming they follow a basic sklearn API, by using the Forecaster.add_sklearn_estimator() function.

[2]:

print(*f.estimators,sep='\n')

catboost
elasticnet
gbt
knn
lasso
lightgbm
mlp
mlr
rf
ridge
sgd
svr
xgboost
arima
hwes
prophet
silverkite
rnn
lstm
naive
tbats
theta
combo

Forecaster.can_be_tuned

  • The following estimators can be tuned using a grid search and cross validation.

[3]:

print(*f.can_be_tuned,sep='\n')

catboost
elasticnet
gbt
knn
lasso
lightgbm
mlp
mlr
rf
ridge
sgd
svr
xgboost
arima
hwes
prophet
silverkite
rnn
lstm
naive
tbats
theta

As of version 0.18.0, the only model missing here is combo.

Forecaster.sklearn_estimators

  • These all come from scikit-learn or use a basic scikit-learn API and behave similarly, including accepting a normalizer argument, accepting an Xvars argument, and offering any recursive evaluation length.

[4]:

print(*f.sklearn_estimators,sep='\n')

catboost
elasticnet
gbt
knn
lasso
lightgbm
mlp
mlr
rf
ridge
sgd
svr
xgboost

  • Add any other sklearn estimator to the Forecaster (docs) and MVForecaster (docs) classes.

Forecaster.metrics

  • These are all the metrics available for use when optimizing models.

  • All metrics from the metrics class that accept only two arguments are available and can be passed when initiating the object or later using Forecaster.set_metrics().

  • Custom metrics and metric functions also accepted, as long as they only take two arguments (array of actuals and array of forecasted values).

[5]:

print(*f.metrics.keys(),sep='\n')

rmse
mape
mae
r2
smape
mse
abias

Forecaster.determine_best_by

  • These are generated from the metrics in Forecaster.metrics and include in-sample, test-set, and validation-set metrics.

  • Many functions can monitor one of these metrics when applying auto ML methods.

  • Plots and dataframe exports can be ordered best-to-worst according to any of these.

  • The difference between ‘Level’ and non-level only comes into play if Forecaster.diff() has been called to difference a series. However, SeriesTransformer also differences series, in addition to being able to take more dynamic transformations, making the need to use Forecaster.diff() irrelevant. It will soon go away and there will be no distinction between level and non-level metrics.

[6]:

print(*f.determine_best_by,sep='\n')

TestSetRMSE
TestSetMAPE
TestSetMAE
TestSetR2
TestSetSMAPE
TestSetMSE
TestSetABIAS
InSampleRMSE
InSampleMAPE
InSampleMAE
InSampleR2
InSampleSMAPE
InSampleMSE
InSampleABIAS
ValidationMetricValue

Forecaster.normalizer

  • These are all the options to scale your data when using an sklearn estimator.

  • All models receive a MinMax scale by default (since it is highly encouraged to always use scaled data for some scikit-learn models), but None is also available as an argument to avoid scaling.

[7]:

print(*f.normalizer.keys(),sep='\n')

minmax
normalize
scale
robust
None

MVForecaster.optimizer_funcs

  • These are the functions you can use to optimize models in MVForecaster only.

  • This means that if you use the "mean" option, which is the object’s default, when tuning models, it will choose the best one based on which metric had the best average performance on all series

  • You can add your own functions by calling add_optimizer_func(): see the docs.

[8]:

mvf = MVForecaster(f, f.copy())
print(*mvf.optimizer_funcs.keys(),sep='\n')

mean
min
max

[ ]: