`model_selection.split_data_train_validation_test`: Test set starts one point too early

[don-alejandrino]

When calling model_selection.split_data_train_validation_test() in backtest mode, the test set starts one point too early. Assume you have a DataFrame of size 100, and test_fraction is 0.1.
Then start_date_test = end_date - np.round(number_indices * test_fraction) * delta (line 176) implies that the test set starts 10 time steps earlier than the last timestamp. So, because we have 100 timestamps in this simple example, the test set starts from timestamp 90. Due to this off-by-one-error, the actual test set size is 11, instead of the expected 10.

A simple fix would be to change line 176 in model_selection/model_selection.py to

start_date_test = end_date - (np.round(number_indices * test_fraction) - 1) * delta

[egordm]

Hi, sorry for a late reply.

You are correct, there seems to be a bug in split_data_train_validation_test possibly causing an off by one error. I am not 100% sure if the proposed fix might solve it though or introduce new issues when used on small datasets.

We are actually currently working on openstef v4 where this issue is actually fixed.
See: https://github.com/OpenSTEF/openstef/blob/release/v4.0.0/packages/openstef-models/src/openstef_mode… (stratified_train_test_split).
You can install it using pip install openstef==4.0.0.a7. Depending on your use case you might actually want to prefer using it.

Let me know if v4 works for you. Or if it's too big of a jump and you are in need for this fix in v3 branch.

[don-alejandrino]

Hi @egordm,
thanks for your response. Currently we're still on v3, but we have a workaround that we can live with for the time being. So a fix in v3 is not urgent from my side, especially if it's already fixed in v4. I just wanted to report this bug.