time_periodic

[ignasivalles]

Hi, I'm running parcels looping over a period of time so I use the option time_periodic. I understand the philosophy, but I'm not really sure what it does, since by mistake, I added a period of time longer than the velocity field (I was confused by the time period I want for my run). The code didn't give any problem, as "time_periodic has to be equal to or longer than the time in the fieldset", so when it is longer, what's going on? When the program loops, do the particles jump from the last time step to the first anyway?

(Btw, if I'm not wrong, I've seen that the option time_periodic=True does not work anymore, but it still appears in the tutorial documentation: https://docs.oceanparcels.org/en/latest/examples/tutorial_timestamps.html)

thank you!

[erikvansebille]

Hi @ignasivalles, thanks for asking this question. I must say that we don't often (never?) use time_periodic in my group, so I'm not entirely sure whether it still works as originally intended.

As for the expected length, I must admit that this is very confusing, but I think the point is that the last time can be duplicated, see the self.grid._add_last_periodic_data_timestep variable on line 203 in the code below:

parcels/parcels/field.py

Lines 197 to 205 in 7e47aa3

	if self.time_periodic is not False:
	if isinstance(self.time_periodic, datetime.timedelta):
	self.time_periodic = self.time_periodic.total_seconds()
	if not np.isclose(self.grid.time[-1] - self.grid.time[0], self.time_periodic):
	if self.grid.time[-1] - self.grid.time[0] > self.time_periodic:
	raise ValueError("Time series provided is longer than the time_periodic parameter")
	self.grid._add_last_periodic_data_timestep = True
	self.grid.time = np.append(self.grid.time, self.grid.time[0] + self.time_periodic)
	self.grid.time_full = self.grid.time

FYI, the code that then converts the periodic time is below:

parcels/parcels/field.py

Lines 1163 to 1193 in 7e47aa3

	def time_index(self, time):
	"""Find the index in the time array associated with a given time.
	Note that we normalize to either the first or the last index
	if the sampled value is outside the time value range.
	"""
	if not self.time_periodic and not self.allow_time_extrapolation and (time < self.grid.time[0] or time > self.grid.time[-1]):
	raise TimeExtrapolationError(time, field=self)
	time_index = self.grid.time <= time
	if self.time_periodic:
	if time_index.all() or np.logical_not(time_index).all():
	periods = int(math.floor((time-self.grid.time_full[0])/(self.grid.time_full[-1]-self.grid.time_full[0])))
	if isinstance(self.grid.periods, c_int):
	self.grid.periods.value = periods
	else:
	self.grid.periods = periods
	time -= periods*(self.grid.time_full[-1]-self.grid.time_full[0])
	time_index = self.grid.time <= time
	ti = time_index.argmin() - 1 if time_index.any() else 0
	return (ti, periods)
	return (time_index.argmin() - 1 if time_index.any() else 0, 0)
	if time_index.all():
	# If given time > last known field time, use
	# the last field frame without interpolation
	return (len(self.grid.time) - 1, 0)
	elif np.logical_not(time_index).all():
	# If given time < any time in the field, use
	# the first field frame without interpolation
	return (0, 0)
	else:
	return (time_index.argmin() - 1 if time_index.any() else 0, 0)

So in summary, I think that the expectation is that the value of time_periodic is exactly the same as the dataset, and that the only freedom you have is whether the last time is a duplicate of the first time or not.
If you want to periodically loop over a shorter time_window, then you should instead use fewer hydrodynamic snapshots when you create your FieldSet.

Oh and good catch about the old statement in the tutorial, I'll fix that immediately

[ignasivalles]

Hi again. Regarding that question, I would like to point out that when you define time_periodic with the timedelta function (e.g., time_periodic = datetime.timedelta(days=6393)), surprisingly, the program crashes. However, it works fine when you simply add an integer number in seconds.

[ignasivalles]

Thanks, @erikvansebille, for your reply. Yes, it's strange, it appears to be a memory issue. However, when I run it with just the number in seconds, it works fine. The terminal was killed with no errors displayed. Here is a snippet of my code. It looks like it gets stuck in FieldSet.from_nemo function. Please note that I'm working with a substantial amount of data (>2TB).

Ufiles  =  sorted(glob(data_file_path+"U2/GLORYS12V1-gridU-*.nc"))
Vfiles  =  sorted(glob(data_file_path+"V/GLORYS12V1-gridV-*.nc")) 
Wfiles  =  sorted(glob(data_file_path+"W2/GLORYS12V1-gridW-*.nc")) 
Tfiles = sorted(glob(data_file_path+"T2/GLORYS12V1-gridT-*.nc"))
Sfiles = sorted(glob(data_file_path+"S2/GLORYS12V1-gridS-*.nc"))

mesh_mask = data_file_path+"GLORYS12V1-mesh_mask.nc"

filenames = {'U': {'lon':mesh_mask, 'lat':mesh_mask, 'depth':Wfiles[0],'time':Ufiles, 'data':Ufiles}, 
             'V': {'lon':mesh_mask, 'lat':mesh_mask, 'depth':Wfiles[0],'time':Vfiles, 'data':Vfiles}, 
             'W': {'lon':mesh_mask, 'lat':mesh_mask, 'depth':Wfiles[0],'time':Wfiles, 'data':Wfiles},
             'T': {'lon':mesh_mask, 'lat':mesh_mask, 'depth':Wfiles[0],'time':Tfiles, 'data':Tfiles},
             'S': {'lon':mesh_mask, 'lat':mesh_mask, 'depth':Wfiles[0],'time':Sfiles, 'data':Sfiles}}

variables = {'U': 'vozocrtx',
             'V': 'vomecrty',
             'W': 'vovecrtz',
             'T': 'votemper',
             'S': 'vosaline'}

dimensions= {'U': {'lon':'glamf','lat':'gphif','depth':'depthw','time':'time_counter'},
             'V': {'lon':'glamf','lat':'gphif','depth':'depthw','time':'time_counter'}, 
             'W': {'lon':'glamf','lat':'gphif','depth':'depthw','time':'time_counter'},
             'T': {'lon':'glamt','lat':'gphit','depth':'depthw','time':'time_counter'},
             'S': {'lon':'glamt','lat':'gphit','depth':'depthw','time':'time_counter'}}


fieldset = FieldSet.from_nemo(filenames, variables, dimensions,allow_time_extrapolation=False,time_periodic=delta(days=6939))

thanks again!

[erikvansebille]

Hmm, very strange. I have no idea how even to start debugging this...

[ignasivalles]

Well, it's not a big deal the workaround (and maybe it's just a question of a conflict between packages in my conda workflow?). In any case it can take some time to find out what's going on, that's why I posted here, just in case someone experiences the same problem :)

[ignasivalles]

Hi again... after setting the time length of my dataset in time_periodic, I encountered the following issue while running parcels for a while (38% progress):

T sampled outside the time domain at time 1980-12-31T12:00:00.000000000. In fset.computeTimeChunk, try setting allow_time_extrapolation to True.

Here, T represents the temperature field that I'm sampling at each particle position, which is defined as U, V, and W in fieldset.from_nemo, as shown above.

I'm currently trying to figure out what is happening but any thoughts on this would be appreciated.

[ignasivalles]

My first time step in the dataset is 2001/01/01 and the last 2018/12/30 (daily frequency).

[ignasivalles]

Hi, following this discussion, I'm still confused about how the time_periodic and sample field work together (if they do). If I'm not wrong I've seen in previous posts that adding a field to be sampled does not implement the time_periodic functionality but maybe this was for previous versions. I also experience problems depending on how my input files are stored. When there is a file for each time step, there is no error. However, if my input files are a single file for each variable with full time dimension, I get the error previously mentioned T sampled outside of the time domain.

In any case, what I'm doing now is setting allow_time_extrapolation=True for T and S before I add them to the FieldSet through the add_field function. So my question is: when time_periodic is on in the FieldSet for UVW, does using allow_time_extrapolation=True, for T and S without time_periodic, theoretically allow you to sample the fields correctly? even when the time index goes beyond the provided time range?

Maybe this is related with #1563 ?