-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
data structure for particle pair #275
Comments
This project has not looked at two-particle statistics yet! We've been focusing on single particle statistics/analysis for now, especially through our |
Thanks @selipot. I just found an old issue here. At that time, I was trying to do something like this project. But I don't know the best data structure for even single particle statistics. I find this project solved my previous issue by introducing the ragged array, which is effective in dealing with particles of different lengths. The regular array is more effective in dealing with those with the same record lengths. Following your basic design, I just define a pair as a list of two def relative_dispersion(pair):
return rd
def velocity_structure(pair):
return S2
def FSLE(pair):
return fsle Then a unified def apply_to_pairs(func, pairs, **kwargs):
res = [func(pair, **kwargs) for pair in pairs]
return xr.concat(res, dim='pair') which is convenient for parallelism and also give rise to a new The thing is that when pairs are large, like ~1000 particles in which FSLE uses all possible pairs (~million pairs), this simple design of pair as [xr.Dataset, xr.Dataset] induces unacceptable large overhead and requires about half day to finish the calculation of FSLE. But if an explicit loop is used in Just want to know if you have any better solution here. I would like to try some benchmark with my data on hand. And if you feel the design is compatible for the project, I definitely love to make a little contribution here. |
Maybe you could store everything in a large single
Then you could use the
|
Thanks @philippemiron. We may still reqiure a time axis, which is generally ragged for different pairs. Note that a pair is grouped according to some criteria (e.g., separation smaller than I open an issue here, but their suggestion of a list of indexes similar to the above solution does not make it much faster (this also makes the dataset much much larger). |
I don't know if I have the most optimal solution to the problem (sorry didn't have chance to try through all your suggestions above). But here are some things that I have done in the past:
![]() |
Hi @dhruvbalwada, that's interesting. How do you calculate relative-dispersion (RD) statistics when you use this data structure? I am really new to this, but I find that the data structure of class RelativeDispersion:
def __init__(self, xpos, ypos, uvel, vvel, time, coord='cartesian', Rearth=6371.2):
# get pairs
def group_pairs(self, particles) -> pairs:
def find_pairs(self, pairs, r0=[lower, upper], chancePair=True) -> originalPairs, chancePairs:
# time- and separation-based statistics
def PDF(self, r, bins) -> PDFs[bins, time]:
def CDF(self, PDFs, bins) -> CDFs[bins, time]:
# time-based statistics
def stat_r(self, pairs, power=2.0, mean=True) -> rx2[pairs, time], ry2[pairs, time], r2[pairs, time]:
def stat_v(self, pairs, power=2.0, mean=True) -> u2[pairs, time], v2[pairs, time], mag2[pairs, time]:
# separation-based statistics
def FSLE(self, pairs, bins, interpT=1, mean=True) -> FSLE[pairs, bins]:
def CIST(self, CDF, lower, upper) -> CIST[pairs, bins]:
# helper functions
def get_separation_bins(self, r_lower, r_upper, alpha=np.sqrt(2.0)) -> bins[bins]: Just feel that this fits well to the thinking of scientists (may be improved further), at least for a newbie like me:
The good side is that this is easy to parallelization (apply a core function to a list of |
Hi, I've recently doing relative dispersion analysis, in which particles are grouped into pairs like
Then I write some diagnostics calculation such as relative dispersion:
If there is 100 particles, through combination one would get ~5000 (100*99/2) possible pairs. So a simple loop like:
would be quite slow due to unacceptable overhead. Although I can do parallel mapping in this case, I just wondering if there is a better solution for this. Do you guys have any suggestions on dealing with pair particle calcualtions?
The text was updated successfully, but these errors were encountered: