Merge in latest nmdc-schema (v10.0.0+) #89

turbomam · 2024-02-13T19:52:39Z

I temporarily removed the migration-doctests target from the site target, which is called by the all target

novelty in itself is not a goal

Note: The method isn't run as part of the agenda. TBD.

I expect this class's implementation (i.e. this Python code) to ultimately reside in the `nmdc-runtime` repository.

TODO: - Update documentation - Update migrator creator (template) - Remove "agenda" references from example migrator

…squads-mongodb-via-sparql-in-fuseki

…odb-via-sparql-in-fuseki' of github.com:microbiomedata/nmdc-schema into 1735-referential-ineggrity-checking-of-napa-squads-mongodb-via-sparql-in-fuseki

…ential-ineggrity-checking-of-napa-squads-mongodb-via-sparql-in-fuseki referential integrity checking of napa squads mongodb via sparql in fuseki

…name

…ential-ineggrity-checking-of-napa-squads-mongodb-via-sparql-in-fuseki better differentiation between pure-export and get-study-id-from-filename

…so the funding_sources is not confusing

…g_sources_character-limit issue 598 add funding sources character limit

eecavanna · 2024-02-13T20:35:55Z

I will do the following things on this branch:

Update migrators to use adapters
Re-create the poetry lockfile
Re-add the migration-doctests target to the dependency list of the site target
Update a migrator to check for the presence of a dictionary key before pop-ping it

turbomam and others added 30 commits November 8, 2023 17:25

Update CODE_OF_CONDUCT.md

0c0b359

novelty in itself is not a goal

Implement base and dictionary adapters for migration

20e35f2

Update base migrator to store a reference to the adapter passed in

a3ac9ad

Update migration_recursion.py to instantiate/use a DictionaryAdapter

8aeb87c

Fix typo in comment

f9616b1

Allow omission of an adapter (for now)

3ec0564

Make example doctest more concise

09abbe3

Implement example of renaming a collection via an adapter

295574c

Note: The method isn't run as part of the agenda. TBD.

Avoid accessing adapter's internal db variable from outside class

1f8eb30

Rename db to _db to signify that it is pseudo-private

d2d0af0

Explicitly indicate the base adapter is an Abstract Base Class

09e912d

Implement create_collection adapter method

109f9f4

Implement delete_collection adapter method

65f16df

Implement delete_document_by_nmdc_id adapter method

7a45ed4

Fix typo in comment

e36c95c

Implement process_each_document adapter method

c4e7d09

Allow searches by fields other than id

7fffb43

Reorder document methods to match "CRUD" acronym

95dcd66

Allow deletions by fields other than id

e3bebe4

Clarify comment

996a746

Implement MongoAdapter class (temporary; for illustration purposes)

0e7fabe

I expect this class's implementation (i.e. this Python code) to ultimately reside in the `nmdc-runtime` repository.

Make method name more specific (singular vs. plural)

9e48cc2

Replace _agenda dict with upgrade method

2865322

TODO: - Update documentation - Update migrator creator (template) - Remove "agenda" references from example migrator

Remove remaining reference to _agenda property (oops)

9a0549c

Remove unnecessary __init__ functions from migrators

7be2291

Add tutorial comments to example upgrade method

da3fb57

Update migrator creation script to include upgrade method

a57b4a8

Standardize docstring for migrator class

4571f68

Clarify comment about nested fields

b72956d

Clarify comment about using a "generator expression"

08563e9

turbomam and others added 17 commits January 30, 2024 09:12

Merge branch 'main' into 1735-referential-ineggrity-checking-of-napa-…

390b327

…squads-mongodb-via-sparql-in-fuseki

20240129

0d68ce0

Merge branch '1734-referential-ineggrity-checking-of-napa-squads-mong…

1a31f50

…odb-via-sparql-in-fuseki' of github.com:microbiomedata/nmdc-schema into 1735-referential-ineggrity-checking-of-napa-squads-mongodb-via-sparql-in-fuseki

no type definitions in mixs.yaml

95b116f

Merge pull request microbiomedata#1738 from microbiomedata/1735-refer…

5146980

…ential-ineggrity-checking-of-napa-squads-mongodb-via-sparql-in-fuseki referential integrity checking of napa squads mongodb via sparql in fuseki

better differentiation between pure-export and get-study-id-from-file…

5a24e62

…name

Merge pull request microbiomedata#1746 from microbiomedata/1735-refer…

df7bd5e

…ential-ineggrity-checking-of-napa-squads-mongodb-via-sparql-in-fuseki better differentiation between pure-export and get-study-id-from-filename

add character limit to funding_sources

97dd777

make funding sources limit 250 since it is multivalued

6bcdf7e

adjust illegal funding sources data file and fix other invalid files …

c790fa3

…so the funding_sources is not confusing

adjust funding_sources example file invalid

78eb5de

fix indentation error in data file

13157c3

try moving invalid funding source to just one long line

0d82ae5

move back to multliines

6d1526b

Merge pull request microbiomedata#1748 from microbiomedata/598-fundin…

7ba754b

…g_sources_character-limit issue 598 add funding sources character limit

merged nmdc-schema into berkeley-schema-fy24

22b3604

skipping doctests over migrators

ecd0e00

turbomam requested review from brynnz22 and eecavanna February 13, 2024 19:55

eecavanna added 6 commits February 13, 2024 13:05

Generate poetry.lock by running $ poetry install

2676146

Update migrators to manipulate database via adapter methods

56f9821

Restore migration-doctests as a dependency of $ make all

1501523

Check for existing of part_of key before deleting it

6f4238f

Check for existence of part_of key before accessing it

a50e8f2

Omit associated_studies field when it would be empty

cdd96ae

eecavanna assigned turbomam Feb 13, 2024

eecavanna approved these changes Feb 13, 2024

View reviewed changes

turbomam merged commit 579fe46 into main Feb 14, 2024
2 checks passed

turbomam deleted the berkeley-schema-main-replica branch February 14, 2024 17:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge in latest nmdc-schema (v10.0.0+) #89

Merge in latest nmdc-schema (v10.0.0+) #89

turbomam commented Feb 13, 2024 •

edited

Loading

eecavanna commented Feb 13, 2024 •

edited

Loading

Merge in latest nmdc-schema (v10.0.0+) #89

Merge in latest nmdc-schema (v10.0.0+) #89

Conversation

turbomam commented Feb 13, 2024 • edited Loading

eecavanna commented Feb 13, 2024 • edited Loading

turbomam commented Feb 13, 2024 •

edited

Loading

eecavanna commented Feb 13, 2024 •

edited

Loading