Database filtering

[decaluwe]

Enable filtering of database according to user inputs for experiment type, device type, and relevant metadata fields.

Requires some though as to how this is done, but my general thought is:

• Experiment and device type are via dropdown.
• Anode, electrolyte, cathode materials: I think this is also via drop down
• Numeric metadata fields: User input boxes for min and max values.
• Others?

[mdmurbach]

Thanks for keeping the conversation going @decaluwe! I've actually been thinking a fair bit about how to structure all of this so we can do this kind of filtering relatively easily (I think filtering is a potentially really important aspect of the tool!) and have iterated to the following sketch of an initial data structure. I'd love to get people's thoughts on how to improve/make this structure clearer since I think it'll be key to be on the same page when we start to implement all of the upload/download/viz/etc. stuff.

For space/sanity reasons, this doesn't try to capture all of the awesome metadata that was defined in https://github.com/ECSHackWeek/ECSOpenData/tree/master/ecsopendata/metadata_templates but I think we'd simple populate each box with the predefined metadata there.

Essentially, I think we want to have one master table with the common metadata (author, exp_type, refs, device_type) that we could quickly filter by dropdowns for exp_type and device_type. Once you select one of these options, you'd have a new table showing either all of the metadata for a specific experiment type or device type. This new table would show the experimental or device specific metadata which you would be able to filter by again to select a subset of data.

For example, a user looking to find all of the battery cycling data for LFP cells could do the following:

1. Select "cycling" from a dropdown of exp_types
2. Select "battery" from a dropdown of device_types
3. Select "LFP" from a dropdown of cathode materials <-- side note, to do this we have to force a categorical cathode material selection on upload (Data import scripts #26), but I think that's doable

In my mind each of these steps is sequential with the dropdown options populating from whatever data is in the database.

Lastly, this structure has me thinking about the best way of actually implementing this. I know (think?) the point of NoSQL is to basically store metadata linked to documents (which it seems like we have here). Does anyone @dacb @nealde have any thoughts on whether it makes sense to use something NoSQL like MongoDB vs my old-trusty PostgreSQL?

[dacb]

Quick response, but I’m not seeing a reason to abandon a relational data store for this.

…

-- David A. C. Beck, Ph.D. [email protected] http://faculty.washington.edu/dacb/ Research Associate Professor, Dept. of Chemical Engineering Director of Research & Senior Data Science Fellow - eScience Institute Adjunct Associate Professor of Paul G. Allen School of Computer Science & Engineering Adjunct Research Associate Professor, Env. and Occ. Health Sciences Associate Director - NRT Data Intensive Research Enabling Clean Technologies University of Washington, Seattle

On Oct 30, 2018, at 3:30 PM, Matt Murbach ***@***.***> wrote: Thanks for keeping the conversation going @decaluwe <https://github.com/decaluwe>! I've actually been thinking a fair bit about how to structure all of this so we can do this kind of filtering relatively easily (I think filtering is a potentially really important aspect of the tool!) and have iterated to the following sketch of an initial data structure. I'd love to get people's thoughts on how to improve/make this structure clearer since I think it'll be key to be on the same page when we start to implement all of the upload/download/viz/etc. stuff. <https://user-images.githubusercontent.com/9369020/47754101-0a6d4980-dc57-11e8-8e8c-c6dafa2e1b84.png> For space/sanity reasons, this doesn't try to capture all of the awesome metadata that was defined in https://github.com/ECSHackWeek/ECSOpenData/tree/master/ecsopendata/metadata_templates <https://github.com/ECSHackWeek/ECSOpenData/tree/master/ecsopendata/metadata_templates> but I think we'd simple populate each box with the predefined metadata there. Essentially, I think we want to have one master table with the common metadata (author, exp_type, refs, device_type) that we could quickly filter by dropdowns for exp_type and device_type. Once you select one of these options, you'd have a new table showing either all of the metadata for a specific experiment type or device type. This new table would show the experimental or device specific metadata which you would be able to filter by again to select a subset of data. For example, a user looking to find all of the battery cycling data for LFP cells could do the following: Select "cycling" from a dropdown of exp_types Select "battery" from a dropdown of device_types Select "LFP" from a dropdown of cathode materials <-- side note, to do this we have to force a categorical cathode material selection on upload (#26 <#26>), but I think that's doable In my mind each of these steps is sequential with the dropdown options populating from whatever data is in the database. Lastly, this structure has me thinking about the best way of actually implementing this. I know (think?) the point of NoSQL is to basically store metadata linked to documents (which it seems like we have here). Does anyone @dacb <https://github.com/dacb> @nealde <https://github.com/nealde> have any thoughts on whether it makes sense to use something NoSQL like MongoDB vs my old-trusty PostgreSQL? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGL1-m2oX6OAZWmQbJ9EdQ1VeRvZm_nFks5uqNL5gaJpZM4XwOHt>.