One-step-ahead Forecast (Fitted values) from Test Set

[RodolfoViegas]

Hi, is there possible to get the fitted values from test set? After doing cross-validation process, when I use the method .predict() with a new series (new_df=test_set), what the method does are forecasts out-of-sample test set, however a I would like to receive the fitted values (one-step-ahead forecast) from the test set. Similar to predict from sklearn. Is it possible?

Thanks for you help, this is for my research.

[RodolfoViegas]

Anyone? Sorry if my question was stupid, but I need this information

[RodolfoViegas]

Hi, @jmoralez , thanks for the reply. Well, part of my project is to make one-step-ahead forecast from test-set. I would like to do forecast similar to sklearn, put then test set in predict method, like this:

fct.predict(test_set)

That line of code will retun the one-step-head forecasted values, but the Nixtla predict method doesn't offer this sort of prediction.

I think is almost to get the fitted values from the test set.

[jmoralez]

What are the contents of test_set here? And how does the expected output differ from fcst.predict(h=1)?

[RodolfoViegas]

Using Airlines dataset:

train = df[df['ds'] <= '1959-10-01']
test = df[df['ds'] > '1959-10-01']

fcst = MLForecast(
models=LinearRegression(),
freq='MS',
lags=[1,2,3],
)
fcst.fit(df)

preds = fcst.predict(h=1)
preds

0 AirPassengers 1961-01-01 439.35495

Above I got the result from linear regression with h=1, however if I use sklearn lib the result is different

X_train,y_train = split_sequence(train['y'], 2) #spli sequence is a function to create lagged datasets
X_test,y_test = split_sequence(test['y'].reset_index( drop=True), 2)

regr = LinearRegression()
regr.fit(X_train, y_train)
pred = regr.predict(X_test)
pred

array([411.93593271, 412.92639614, 375.87931621, 420.11593714,
464.23595176, 464.28656013, 540.75437937, 630.48651607,
581.26679885, 461.94811955, 434.70001754, 360.00556182])

The sklearn method returned an array of values, the Keras's predict method also return a array. This is one-step-ahead forecast I've used to use in my project.

[jmoralez]

You can define those lags as exogenous features and provide them during fit and predict, but in that case mlforecast won't do much for you, you may be better off using scikit-learn directly.

[RodolfoViegas]

Thanks, @jmoralez

[RodolfoViegas]

@jmoralez If I could do a recommendation, I would like to recommed creating a .forward() method to MLForecast, because that, when we put in it a test_set, for instance, also returns the fitted values (one-step-ahead forecast). Like this:

sf.fitted_[0][0].forward(h=1,
y=test['y'].values,
fitted=True).

Or maybe a possibility to create a parameter fitted in .predict() method; whe it receives new data in "new_df" parameter, so then it could return the fitted values of this new df.

Once again, thanks for you help, I really like Nixtla

[jmoralez]

I don't really understand how you're computing the features, but if you already have them you can just pass them directly to the underlying (scikit-learn) model, e.g. fcst.models_['LinearRegression'].predict(your_features)