SeriesTransformer

This object can be used to perform more complex transformations on your Forecaster object. It can be used to transform the dependent variable to adjust for trends, seasonality, and more, and every transformation is revertible. Revert functions must be called in opposite order as the applied transformation functions.

import pandas as pd
import pandas_datareader as pdr
import matplotlib.pyplot as plt
from scalecast.Forecaster import Forecaster
from scalecast.SeriesTransformer import SeriesTransformer
from scalecast import GridGenerator

GridGenerator.get_example_grids()

df = pdr.get_data_fred('HOUSTNSA',start='1900-01-01',end='2021-06-01')
f = Forecaster(y=df['HOUSTNSA'],current_dates=df.index) # to initialize, specify y and current_dates (must be arrays of the same length)

transformer = SeriesTransformer(f)

f = transformer.LogTransform()
f = transformer.DiffTransform(1)
f = transformer.DiffTransform(12)
f = transformer.ScaleTransform()

f.generate_future_dates(12)
f.set_test_length(12)
f.add_time_trend()
f.add_ar_terms(24)

f.set_estimator('elasticnet')
f.cross_validate(rolling=True)
f.auto_forecast()

# call in opposite order
f = transformer.ScaleRevert()
f = transformer.DiffRevert(12)
f = transformer.DiffRevert(1)
f = transformer.LogRevert()

f.plot()

class src.scalecast.SeriesTransformer.SeriesTransformer(f, deepcopy=True)

__init__(f, deepcopy=True)

Initiates the object.

Parameters:

f (Forecaster) – The Forecaster object that will receive each transformation/revert.
deepcopy (bool) – Default True. Whether to store a deepcopy of the Forecaster object in the SeriesTransformer object.

Methods:

`DeseasonRevert`([m, exclude_models])	Reverts a seasonal adjustment already taken on the series.
`DeseasonTransform`([m, model, ...])	Deseasons a series using the moving average method offered by statsmodel through the seasonal_decompose() function.
`DetrendRevert`([exclude_models])	Reverts the y attribute in the Forecaster object, along with all model results.
`DetrendTransform`([loess, frac, it, ...])	Detrends the series using an OLS estimator or using LOESS.
`DiffRevert`([m, exclude_models])	Reverts the y attribute in the Forecaster object, along with all model results.
`DiffTransform`([m])	Takes differences or seasonal differences in the Forecaster object's y attribute.
`LogRevert`(**kwargs)	Reverts the y attribute in the Forecaster object, along with all model results.
`LogTransform`()	Transforms the y attribute in the Forecaster object using a natural log transformation.
`MinMaxRevert`(**kwargs)	Reverts the y attribute in the Forecaster object, along with all model results.
`MinMaxTransform`([train_only])	Transforms the y attribute in the Forecaster object using a min-max scale transformation.
`Revert`(revert_func[, exclude_models])	Reverts the y attribute in the Forecaster object, along with all model results.
`RobustScaleRevert`(**kwargs)	Reverts the y attribute in the Forecaster object, along with all model results.
`RobustScaleTransform`([train_only])	Transforms the y attribute in the Forecaster object using a robust scale transformation.
`ScaleRevert`(**kwargs)	Reverts the y attribute in the Forecaster object, along with all model results.
`ScaleTransform`([train_only])	Transforms the y attribute in the Forecaster object using a scale transformation.
`SqrtRevert`(**kwargs)	Reverts the y attribute in the Forecaster object, along with all model results.
`SqrtTransform`()	Transforms the y attribute in the Forecaster object using a square-root transformation.
`Transform`(transform_func, **kwargs)	Transforms the y attribute in the Forecaster object.

DeseasonRevert(m=None, exclude_models=[])

Reverts a seasonal adjustment already taken on the series. Call DeseasonTransform() before calling this.

Parameters:

m (int) – The number of observations that counts one seasonal step. If not specified, will use the inferred seasonality from statsmodels.
exclude_models (list-like) – Models to not revert. This is useful if you are transforming and reverting an object multiple times.

Returns:

A Forecaster object with the reverted attributes.

Return type:

(Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.DeseasonTransform(model='mul')  # multiplicative deseasoning
>>> f = transformer.DeseasonRevert() # back to normal

DeseasonTransform(m=None, model='add', extrapolate_trend='freq', train_only=False, **kwargs)

Deseasons a series using the moving average method offered by statsmodel through the seasonal_decompose() function.

Parameters:

m (int) – The number of observations that counts one seasonal step. If not specified, will use the inferred seasonality from statsmodels.
model (str) – Default ‘add’. One of {“additive”, “add”, “multiplicative”, “mul”}. The type of seasonal component.
extrapolate_trend (str or int) – Default ‘freq’. If set to > 0, the trend resulting from the convolution is linear least-squares extrapolated on both ends (or the single one if two_sided is False) considering this many (+1) closest points. If set to ‘freq’, use freq closest points. Setting this parameter results in no NaN values in trend or resid components.
train_only (bool) – Default False. Whether to fit the seasonal decomposition model on the training set only.
**kwargs – Passed to seasonal_decompose() function from statsmodels. See https://www.statsmodels.org/dev/generated/statsmodels.tsa.seasonal.seasonal_decompose.html.

Returns:

A Forecaster object with the transformed attributes.

Return type:

(Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.DeseasonTransform(model='mul')  # multiplicative deseasoning

DetrendRevert(exclude_models=[])

Reverts the y attribute in the Forecaster object, along with all model results. Assumes a detrend transformation has already been called and uses all model information already recorded from that transformation to revert. If the test length changes in the Forecaster object between the time the transformation is called and when models are called, you will get this error when reverting: ValueError: All arrays must be of the same length.

Parameters:: exclude_models (list-like) – Models to not revert. This is useful if you are transforming and reverting an object multiple times.
Returns:: A Forecaster object with the reverted attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.DetrendTransform(ln_trend=True)
>>> f = transformer.DetrendRevert()

DetrendTransform(loess=False, frac=0.5, it=3, poly_order=1, ln_trend=False, seasonal_lags=0, m='auto', fit_intercept=True, train_only=False)

Detrends the series using an OLS estimator or using LOESS. Only call this once if you want to revert the series later. The passed Forecaster object must have future dates or be initiated with require_future_dates=False. Make sure the test length has already been set as well. If the test length changes between the time the transformation is called and when models are called, you will get this error when reverting: ValueError: All arrays must be of the same length. The ols or lowess model from statsmodels will be stored in the detrend_params attribute with the ‘model’ key.

Parameters:

loess (bool) – Default False. Whether to fit a LOESS curve.
frac (float) – Optional. Default 0.5. The fraction of the data used when estimating each y-value. A smaller frac value will produce a more rigid trend line, while a larger frac value will produce a smoother trend line. Ignored when loess is False.
it (int) – Optional. Default 3. The number of iterations used in the loess algorithm. A larger it value will produce a smoother trend line, but may take longer to compute. Ignored when loess is False.
poly_order (int) – Default 1. The polynomial order to use in the fitted trend line. Ignored when loess is True.
ln_trend (bool) – Default False. Whether to use a natural logarithmic trend. Ignored when loess is True.
seasonal_lags (int) – Default 0. The number of seasonal lags to use in the estimation. Ignored when loess is True.
m (int or str) – Default ‘auto’. The number of observations that counts one seasonal step. Ignored when seasonal_lags = 0. When ‘auto’, uses the M4 competition values: for Hourly: 24, Monthly: 12, Quarterly: 4. Everything else gets inferred if possible.
fit_intercept (bool) – Default True. Whether to fit an intercept in the model.
train_only (bool) – Default False. Whether to fit the LOESS or OLS model on the training set only.

Returns:

A Forecaster object with the transformed attributes.

Return type:

(Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.DetrendTransform(ln_trend=True)

DiffRevert(m=1, exclude_models=[])

Reverts the y attribute in the Forecaster object, along with all model results. Calling this makes so that AR values become unusable and have to be re-added to the object.

Parameters:

m (int) – Default 1. The seasonal difference to revert. 1 will undifference once. 12 will undifference seasonally 12 periods (monthly data). Any int available. Use the same values to revert as you used to transform the object originally.
exclude_models (list-like) – Models to not revert. This is useful if you are transforming and reverting an object over and over.

Returns:

A Forecaster object with the reverted attributes.

Return type:

(Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.DiffTransform(1)  # first difference
>>> f = transformer.DiffTransform(1)  # second difference
>>> f = transformer.DiffTransform(12) # first 12-period difference
>>> # call revert funcs in reverse order
>>> f = transformer.DiffRevert(12)
>>> f = transformer.DiffRevert(1)
>>> f = transformer.DiffRevert(1) # back to normal

DiffTransform(m=1)

Takes differences or seasonal differences in the Forecaster object’s y attribute. If using this transformation, call Forecaster.add_diffed_terms() and Forecaster.add_lagged_terms() if you want to use those before calling this function. Call Forecaster.add_ar_terms() and Forecaster.add_AR_terms() after calling this function. Call twice with the same value of m to take second differences.

Parameters:: m (int) – Default 1. The seasonal difference to take. 1 will difference once. 12 will take a seasonal difference assuming 12 periods makes a season (monthly data). Any int available.
Returns:: A Forecaster object with the transformed attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.DiffTransform(1)  # first difference
>>> f = transformer.DiffTransform(1)  # second difference
>>> f = transformer.DiffTransform(12) # first 12-period difference

LogRevert(**kwargs)

Reverts the y attribute in the Forecaster object, along with all model results. Assumes a natural log transformation has already been called.

Parameters:: **kwargs – Passed to Transformer.Revert() - arg exclude_models accepted here.
Returns:: A Forecaster object with the reverted attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.LogTransform()
>>> f = transformer.LogRevert()

LogTransform()

Transforms the y attribute in the Forecaster object using a natural log transformation.

Returns:: A Forecaster object with the transformed attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.LogTransform()

MinMaxRevert(**kwargs)

Reverts the y attribute in the Forecaster object, along with all model results. Assumes the min-max scale transformation has been called on the object at some point. Revert function: array[i]*(array.max() - array.min()) + array.min().

Parameters:: **kwargs – {assed to Transformer.Revert() - arg exclude_models accepted here.
Returns:: A Forecaster object with the reverted attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> f.set_test_length(.2) # specify a test set to not leak data with this func
>>> transformer = SeriesTransformer(f)
>>> f = transformer.MinMaxTransform(train_only=True)
>>> f = transformer.MinMaxRevert()

MinMaxTransform(train_only=False)

Transforms the y attribute in the Forecaster object using a min-max scale transformation. Min-max scale defined as (array[i] - array.min()) / (array.max() - array.min()).

Parameters:: train_only (bool) – Default False. Whether to fit the minmax transformer on the training set only.
Returns:: A Forecaster object with the transformed attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.MinMaxTransform()

Revert(revert_func, exclude_models=[], **kwargs)

Reverts the y attribute in the Forecaster object, along with all model results.

Parameters:

revert_func (function) – The function that will be used to revert the values. If using a user function, first argument must be the array to transform.
exclude_models (list-like) – Models to not revert. This is useful if you are transforming and reverting an object over and over.
**kwargs – Passed to the function passed to revert_func.

Returns:

A Forecaster object with the reverted attributes.

Return type:

(Forecaster)

>>> import math
>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> def log10(x):
>>>     return [math.log(i,base=10) for i in x]
>>> def log10_revert(x):
>>>     return [10**i for i in x]
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.Transform(log10)
>>> f = transformer.Revert(log10_revert)

RobustScaleRevert(**kwargs)

Reverts the y attribute in the Forecaster object, along with all model results. Assumes the scale transformation has been called on the object at some point.

Parameters:: **kwargs – Passed to Transformer.Revert() - arg exclude_models accepted here.
Returns:: A Forecaster object with the reverted attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> f.set_test_length(.2) # specify a test set to not leak data with this func
>>> transformer = SeriesTransformer(f)
>>> f = transformer.ScaleTransform(train_only=True)
>>> f = transformer.ScaleRevert()

RobustScaleTransform(train_only=False, **kwargs)

Transforms the y attribute in the Forecaster object using a robust scale transformation. See the function from scikit-learn: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.RobustScaler.html#sklearn.preprocessing.RobustScaler.

Parameters:

train_only (bool) – Default False. Whether to fit the transformer on the training set only.
**kwargs – Passed to the scikit-learn function.

Returns:

A Forecaster object with the transformed attributes.

Return type:

(Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.RobustScaleTransform()

ScaleRevert(**kwargs)

Reverts the y attribute in the Forecaster object, along with all model results. Assumes the scale transformation has been called on the object at some point. Revert function: array.std()*array[i]+array.mean().

Parameters:: **kwargs – Passed to Transformer.Revert() - arg exclude_models accepted here.
Returns:: A Forecaster object with the reverted attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> f.set_test_length(.2) # specify a test set to not leak data with this func
>>> transformer = SeriesTransformer(f)
>>> f = transformer.ScaleTransform(train_only=True)
>>> f = transformer.ScaleRevert()

ScaleTransform(train_only=False)

Transforms the y attribute in the Forecaster object using a scale transformation. Scale defined as (array[i] - array.mean()) / array.std().

Parameters:: train_only (bool) – Default False. Whether to fit the scale transformer on the training set only.
Returns:: A Forecaster object with the transformed attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.ScaleTransform()

SqrtRevert(**kwargs)

Reverts the y attribute in the Forecaster object, along with all model results. Assumes a square-root transformation has already been called.

Parameters:: **kwargs – Passed to Transformer.Revert() - arg exclude_models accepted here.
Returns:: A Forecaster object with the reverted attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.SqrtTransform()
>>> f = transformer.SqrtRevert()

SqrtTransform()

Transforms the y attribute in the Forecaster object using a square-root transformation.

Returns:: A Forecaster object with the transformed attributes.
Return type:: (Forecaster)

>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.SqrtTransform()

Transform(transform_func, **kwargs)

Transforms the y attribute in the Forecaster object.

Parameters:

transform_func (function) – The function that will be used to make the transformation. If using a user function, first argument must be the array to transform.
**kwargs – Passed to the function passed to transform_func.

Returns:

A Forecaster object with the transformed attributes.

Return type:

(Forecaster)

>>> import math
>>> from scalecast.Forecaster import Forecaster
>>> from scalecast.SeriesTransformer import SeriesTransformer
>>> def log10(x):
>>>     return [math.log(i,base=10) for i in x]
>>> f = Forecaster(...)
>>> transformer = SeriesTransformer(f)
>>> f = transformer.Transform(log10)