-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SRTM Version 3 Support #39
Open
nawagers
wants to merge
9
commits into
tkrajina:master
Choose a base branch
from
nawagers:EarthDataLogin
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
deleting *.hgt and *.zip
Using rename instead of replace (could cause errors on windows in the test script if a test previously failed to cleanup). Also changed super() args needed for older versions
GeoElevationData depends on FileHandler, call in module main depend on module data. This creates a circular dependency. Moved FileHandler to the module data to fix. Added unittest for get_elevation. Adjusted unittests for the new FileHandler location.
Set UN and PW in travis, added 3.6, run coverage, coveralls, pyflakes and pycodestyle. Small format changes is data.py. Docstring changes in test.py and data.py.
Python 2.6 support has been dropped by Requests (dependency) and restricts language features. Fixed typo in test.py
Added dev_tools folder for generating srtm.json. Updated or added docstrings and minor code format changes. Cleaned up main.get_data() to remove unneeded code. Made several new functions private (not intended for public API)
Conflicts in master resolved in https://github.com/tkrajina/srtm.py/tree/nwagers-EarthDataLogin |
just a note that I'm using this branch, and haven't experienced problems with it yet |
Is there any special reason why this is not in master? So I need to get a login to use this data set? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New support for SRTM Version 3, Global 1 arc.
The new SRTM v3 is completely void filled and offers 1 arc second resolution tiles over the entire world (between ~56S and ~60N). This addresses issue #38. Users need to obtain Earthdata credentials from NASA Earthdata New User to make use of the new functionality.
Changes
get_data() and main.py: The FileHandler class was moved to data.py due to a previously unknown bug and circular dependencies. get_data() was updated to add version and fallback parameters as well as deprecating srtm1 and srtm3 usage.
EarthData: A new class was added from NASA's programming guide. It's a subclass of requests.Session that overrides the rebuild_Auth function to keep authentication headers when interacting with their server.
Tile Version: A version system was added for the tiles to keep track of which data version (1, 2.1, and 3) and which sampling (1 arcsecond, 3 arcsecond averaged, and 3 arcsecond sampled). The versioning applies to the in memory tile store and the local disk cache. Requests for elevation now accept a version. It's important to know that this will force the old cache to rebuild.
Memory Handling: The tiles are all saved in the dictionary named tiles with keys of the form 'N00E000v2.1a'. This is a class attribute and so is shared between all instances. This was also true before, but
__init__
would reset the dict, so every new object would flush the memory. The index, called tile_index, is also a class attribute dictionary. If the instance sees that the dict is empty during__init__
, it will load it back from srtm.json. The keys to the index are of form 'N00E000' and holds a list of version information. The values in each list correspond to the tile lookup for each version and sampling.Fallback: A bool, GeoElevationData.fallback, turns on and off the fallback behavior. When True, if the requested version can't be loaded, the next version is attempted. This can happen if a server is down or the user supplied bad credentials for version 3 data. The version resolution is laid out in fallback_version(). Users can substitute their own order by subclassing GeoElevationData or saving a new function to the instance's function. An example is in the docstring. The default version is v2.1a which drops to v2.3a and then fails. This is the same behavior as before.
load and fetch: The internal function calls were reorganized to a cleaner and private interface with the functions _get_tilename, _load_tile, _fetch, and _build_url. The other functions that handled similar actions before were deprecated and in some cases, updated to make use of the new functionality.
Travis CI: Dropped support for 2.6. Builds on 2.6 are fragile and restrict language features. Dependency 'Requests' dropped support for 2.6. Added testing for 3.6 and now runs coverage, coveralls, pyflakes, and pycodestyle. The new EarthData credentials are stored in environment variables and loaded by test.py
Test Files: Added a test file folder and three ver2 3 arcsecond tiles. test.py now has a function that replaces fetch() to simulate downloading and reduce network dependency for tests. The new tests written make use of this (except the test for fetch() itself).
Unit tests: The test for batchmode was updated for some small syntax changes and changed some of the assertions to give more helpful failure messages. A comprehensive unittest was added for each of the new functions in GeoElevationData.
srtm.json: A new dict structure was added as srtm.json to replace list.json. The new structure includes information on versions 1, 2.1, and 3 for 1 arc, 3 arc averaged and 3 arc sampled tiles.
Backwards Compatibility: It is expected that the new package code should be completely backwards compatible except for users on Python 2.6 and scripts that directly modified instance and class attributes (such as srtm1_files).
Dev Tools: A script to build the json (or pickle) and the tile lookup for each version are included in the dev_tools directory.