Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to import and export data in container object format #399

Merged
merged 2 commits into from
Jun 22, 2023

Conversation

pkalita-lbl
Copy link
Collaborator

This resolves #390.

Currently, the getDataObjects and loadDataObjects methods provide and accept arrays of objects. For various reasons, when working in the LinkML ecosystem we often want those arrays to be wrapped in a top-level or "container" object. That is:

{
  "items": [
    . . .
  ]
}

As opposed to:

[
  . . .
]

In the above example, items would be referred to as the "index slot" of the container object. In many cases the name of the index slot can be inferred.

These changes allow getDataObjects and loadDataObjects to work with both arrays (as they currently do) and arrays wrapped in a container object.

On the exporting side, I've added some interface elements to the Toolbar's "Save As" menu which allows you to choose whether you want to save as an array or an object and, for objects, provide an index slot name (the inferred one will be populated if possible). These values are then passed to new options on the getDataObjects method.

On the loading side, loadDataObjects now detects if an object is provided and if so attempts pick the actual grid data out of the right key.

cc: @turbomam

@pkalita-lbl pkalita-lbl requested a review from ddooley June 13, 2023 23:59
@ddooley
Copy link
Collaborator

ddooley commented Jun 22, 2023

I've approved. It would be good to have a paragraph or two to add to the docs to describe the behaviour we recommend for this visa vis https://w3id.org/linkml/tree_root . I.e. shall we get in the habit of having people add to their schema tree_root="true" for a given class.

I was wondering as well, in case where dataharmonizer finds multiple classes with dh_interface=true, that each one is a candidate for parsing a json data file in a given template. I.e. multiple "tree_roots" actually used from a given schema. So next iteration of this could simply have a data file indicate by metadata which schema and version and "tree root" class it belongs to.

@ddooley ddooley merged commit 0040c7e into master Jun 22, 2023
@pkalita-lbl
Copy link
Collaborator Author

Good points. I'll make a few issues to track that later today.

@pkalita-lbl
Copy link
Collaborator Author

Well, I fully intended to make some issues, but the more I thought about it the more I was unsure about what issues to make 😁. I think maybe a meeting sometime in the near future would be more productive. I think there are some requirements to be laid out and design decisions to be made, and I want to be sure they're not being made in isolation.

@pkalita-lbl pkalita-lbl deleted the issue-390 branch June 23, 2023 16:02
@ddooley
Copy link
Collaborator

ddooley commented Jun 23, 2023

K, I'll touch base in Slack on this!

kennethbruskiewicz pushed a commit that referenced this pull request Aug 21, 2023
Add ability to import and export data in container object format
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document the fact that DH JSON is a bare list and not compatible with LinkML tools as is
2 participants