Prediction setting specific Datetimes

[glopezzz]

Hi,

My issue is that my data has records for every 30 minutes. The training data is not necessarily ordered by datetime... there could be dates to predict between the training datetimes. Therefore when I fit the modle I want to give the model.predict() function, a dataframe with all those test_dates (as I did with the fbprophet models).

To hand in the DF for prediction, it requires a 't' and a 'y':

                          ds     t     y
0    2020-11-01 00:00:00.000  None  None
1    2020-11-01 00:30:00.000  None  None
2    2020-11-01 02:30:02.000  None  None
3    2020-11-01 05:30:05.000  None  None
4    2020-11-01 08:30:09.000  None  None

Then, when I use the model.predict() function, it returns the following error:

TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.

Don't understand why. Am I doing anything wrong here?

I will also like to add that I set the examples freq='D' as I didn´t find any Documentation about that parameter... What other options are there?

Thank you!

[ourownstory]

Hi, @glopezzz
The workflow diverges from Prophet slightly here. You seem to be running into trouble because you skip make_future_dataframe before predicting. Try using that function.

You should also set freq=30min. Details on the freq parameters are linked in:
http://neuralprophet.com/#simple-model

If that does not solve your issue, please provide a minimal set of code to replicate the issue.

[glopezzz]

Hi @ourownstory , thanks for answering.

The thing here is that I'm not seeking a future dataframe. I want to correct past empty data. That's why I need to give the .predict() function, a list of the dates I want to fill.
That's how I worked with fbprophet...

I've tried using the make_future_dataframe function, but it doesn't work for me, as I explained...

I also tried specifying freq=30min and I still get the same, problem:

TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, int64, int32, int16, int8, uint8, and bool.

Can you pass the link for the freq details please? I can't find it :_(

Thanks again,
Escanor

[ourownstory]

Hi @glopezzz

Do you mind sharing a piece of code for me to reproduce what you are doing?

Also, did you try this?
model.make_future_dataframe(df, periods=0, n_historic_predictions=True)

Can you pass the link for the freq details please?
valid entries (can be multiples of them):
https://pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#timeseries-offset-aliases

[ourownstory]

@mchessl had a similar issue, and ultimately had success with it in: maybe they can share what they did in #262

[glopezzz]

Hi @ourownstory,

That did solve my issue! I'll leave the functions I use here bellow:

def train_neuralprophet_model(X_train,train_dates):
    
    X_train = X_train[:,1:-1]
    X_sep = []

    for var in range(X_train.shape[1]):

        data = {"ds":train_dates, "y":X_train[:,var]}
            
        X_sep.append(pd.DataFrame(data))
        
    models = {}

    for var in range(X_train.shape[1]):
        models[var] = NeuralProphet()
        models[var].fit(X_sep[var], freq='30min')
    
    return models

That function returns a list of models, as I need a model per variable...

With this next function, I predict the data for each variable. As the dates to predict are the same for each variable I need, the DataFrame created by make_future_dataframe is the same for each variable...

def predict_data(models, test_dates):

    predictions_test = []
    dates_to_predict = pd.DataFrame(data={"ds":test_dates, 'y':None})
    df = models[0].make_future_dataframe(dates_to_predict, periods=0, n_historic_predictions=True)
    for i in tqdm(models):
        predictions_test.append(models[i].predict(df))
        
    return predictions_test

I hope this helps...

Thank you!!

[ourownstory]

Great @glopezzz, thank you for sharing your code!

Btw, you could also add your other variables as "future regressors" to each of the models if you would like to improve the filled in data.

If you like, you could create a short notebook "Filling in missing data" explaining what you did here for others.
I'd be happy to approve a pull request!

[glopezzz]

Alright, I'll happily do that, although I cannot share the data due confidentiality... I would change the variables names.

How do I create a notebook and link it to neuralprophet?? I've never done this...