support progress bar in parallel forecast

[jmoralez]

Changes the forecast method to use a concurrent.futures.ProcessPoolExecutor which processes series one at a time instead of chunks of them. This allows two things:

• We can now have a progress bar because we can update it as each serie is processed.
• The work can be better balanced since we previously assumed that each partition would take about the same time, but that wasn't always true in practice, so some CPUs would become idle as some partitions were done while others were still being processed. This means that the process can now be faster, even though I feared the serialization would become a bottleneck it doesn't seem like it.

I ran the following experiment in a c5.4xlarge instance to compare the current version versus the one in this PR to verify that there isn't a slowdown:

Click to expand

from datasetsforecast.m4 import M4, M4Info

from statsforecast import StatsForecast
from statsforecast.models import AutoETS

group = 'Hourly'
series, *_ = M4.load('data', group=group)
series['ds'] = series['ds'].astype('int64')
h = M4Info[group].horizon
sf = StatsForecast(
    models=[AutoETS(season_length=24)],
    freq=1,
    n_jobs=16,
    verbose=True,
)
sf.forecast(df=series, h=h, level=[80, 90], fitted=True)

and it took 1min 36s (current main) vs 1min 32s (this PR).

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB