Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up reader-mapper-parser infrastructure, use multiformat reader #19

Open
lukaspie opened this issue Feb 23, 2024 · 0 comments
Open
Assignees
Labels
enhancement New feature or request sub-reader

Comments

@lukaspie
Copy link
Collaborator

Currently, the reader implements a three-layer structure:

  • there is the XpsReader(BaseReader)
  • then there's a layer called "mappers" (see here for an example)
  • these mappers themselves actually call "parsers" (example).

The mappers are used for one file format (like the sle format from SPECS) and then I have a logic that calls a parser for a specific subsets of such files, e.g. depending on the software version that was used for this file. That allows me to keep functionality across different, yet very similar versions of a format by inheritance and abstract base classes.

All of those sub-classes could be readers themselvers, inheriting from our BaseReader class (or the MultiFormatReader developed in FAIRmat-NFDI/pynxtools#250).

The file extension is often not unique, i.e., many vendors have a .txt export, but all the files are actually different. But this logic could probably be handled by passing a function in the extensions dict of the MultiFormatReader that does this. This is already being handled similary. And finally, there should be be a check that the file comes from the list of supported vendors.

@lukaspie lukaspie added enhancement New feature or request sub-reader labels Feb 23, 2024
@lukaspie lukaspie self-assigned this Feb 23, 2024
@lukaspie lukaspie changed the title Clean up reader-mapper-parser infrastructure Clean up reader-mapper-parser infrastructure, use multiformat reader Feb 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request sub-reader
Projects
None yet
Development

No branches or pull requests

1 participant