All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Optimize order of the dockerfiles to leverage layer caching by @JoeyEamigh
- Add support for alternate postgres ports and db names in docker by @JoeyEamigh
- Fixed a bug where the map was not loading due to invalid tile layer name
- 17 new tile layers to choose from. Now you can select the tile layer that suits you the best. You can find the list of available tile layers in the map controls in the top right corner of the map under the layers icon.
GET /api/v1/pointsresponse now will includeX-Total-PagesandX-Current-Pageheaders to make it easier to work with the endpoint- The Pages point now shows total number of points found for provided date range
- Link to Visits page in notification informing about new visit suggestion
- Flatten geodata retrieved from Immich before processing it to prevent errors
- Links to view import points on the map and on the Points page on the Imports page.
- The Imports page now loading faster.
- Default value for
RAILS_MAX_THREADSwas changed to 10. - Visit suggestions background job was moved to its own low priority queue to prevent it from blocking other jobs.
- Fixed a bug preventing the application from starting, when there is no users in the database but a data migration tries to update one.
- Support for miles. To switch to miles, provide
DISTANCE_UNITenvironment variable with valuemiin thedocker-compose.ymlfile. Default value iskm.
It's recommended to update your stats manually after changing the DISTANCE_UNIT environment variable. You can do this by clicking the "Update stats" button on the Stats page.
DISTANCE_UNIT is set to mi.
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
APPLICATION_HOST: "localhost"
APPLICATION_PROTOCOL: "http"
APPLICATION_PORT: "3000"
TIME_ZONE: "UTC"
+ DISTANCE_UNIT: "mi"
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
environment:
APPLICATION_HOST: "localhost"
APPLICATION_PROTOCOL: "http"
APPLICATION_PORT: "3000"
TIME_ZONE: "UTC"
+ DISTANCE_UNIT: "mi"- Default time range on the map is now 1 day instead of 1 month. It will help you with performance issues if you have a lot of points in the database.
- GeoJSON import now correctly imports files with FeatureCollection as a root object
- The Points page now have number of points found for provided date range
GET /api/v1/healthendpoint to check the health of the application with swagger docs
- Ruby version updated to 3.3.4
- Visits suggestion process now will try to merge consecutive visits to the same place into one visit.
The GPX and GeoJSON export release
Default exporting format is now GeoJSON instead of Owntracks-like JSON. This will allow you to use the exported data in other applications that support GeoJSON format. It's also important to highlight, that GeoJSON format does not describe a way to store any time-related data. Dawarich relies on the timestamp field in the GeoJSON format to determine the time of the point. The value of the timestamp field should be a Unix timestamp in seconds. If you import GeoJSON data that does not have a timestamp field, the point will not be imported.
Example of a valid point in GeoJSON format:
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [13.350110811262352, 52.51450815]
},
"properties": {
"timestamp": 1725310036
}
}- GeoJSON format is now available for exporting data.
- GPX format is now available for exporting data.
- Importing GeoJSON is now available.
- Default exporting format is now GeoJSON instead of Owntracks-like JSON. This will allow you to use the exported data in other applications that support GeoJSON format.
- Fixed a bug where the confirmation alert was shown more than once when deleting a point.
- Resource limits to docke-compose.yml file to prevent server overload. Feel free to adjust the limits to your needs.
deploy:
resources:
limits:
cpus: '0.50' # Limit CPU usage to 50% of one core
memory: '2G' # Limit memory usage to 2GB- Importing geodata from Immich will now not throw an error in the end of the process
- A notification about an existing import with the same name will now show the import name
- Export file now also will contain
raw_datfield for each point. This field contains the original data that was imported to the application.
PATCH /api/v1/settingsendpoint to update user settings with swagger docsGET /api/v1/settingsendpoint to get user settings with swagger docs- Missing
pageandper_pagequery parameters to theGET /api/v1/pointsendpoint swagger docs
- Map settings moved to the map itself and are available in the top right corner of the map under the gear icon.
- Fixed a bug that prevented data migration from working correctly
- With this release deployment, data migration will work, starting visits suggestion process for all users.
- After initial visit suggestion process, new suggestions will be calculated every 24 hours, based on points for last 24 hours.
- If you have enabled reverse geocoding and (optionally) provided Photon Api Host, Dawarich will try to reverse geocode your visit and suggest specific places you might have visited, such as cafes, restaurants, parks, etc. If reverse geocoding is not enabled, or Photon Api Host is not provided, Dawarich will not try to suggest places but you'll be able to rename the visit yourself.
- You can confirm or decline the visit suggestion. If you confirm the visit, it will be added to your timeline. If you decline the visit, it will be removed from your timeline. You'll be able to see all your confirmed, declined and suggested visits on the Visits page.
- A "Map" button to each visit on the Visits page to allow user to see the visit on the map
- Visits suggestion functionality. Read more on that in the release description
- Click on the visit name allows user to rename the visit
- Tabs to the Visits page to allow user to switch between confirmed, declined and suggested visits
- Places page to see and delete places suggested by Dawarich's visit suggestion process
- Importing a file will now trigger the visit suggestion process for the user
- Dawarich export was failing when attempted to be imported back to Dawarich.
- Imports page with a lot of imports should now load faster.
/api/v1/pointsendpoint now returns 100 points by default. You can specify the number of points to return by passing theper_pagequery parameter. Example:/api/v1/points?per_page=50will return 50 points. Also,pagequery parameter is now available to paginate the results. Example:/api/v1/points?per_page=50&page=2will return the second page of 50 points.
- A user can now trigger the import of their geodata from Immich to Dawarich by clicking the "Import Immich data" button in the Imports page.
- A user can now provide a url and an api key for their Immich instance and then trigger the import of their geodata from Immich to Dawarich. This can be done in the Settings page.
- Table columns on the Exports page were reordered to make it more user-friendly.
- Exports are now being named with this pattern: "export_from_dd.mm.yyyy_to_dd.mm.yyyy.json" where "dd.mm.yyyy" is the date range of the export.
- Notification about any error now will include the stacktrace.
- The
api/v1/statsendpoint to get stats for the user with swagger docs
- Redis and DB containers are now being automatically restarted if they fail. Update your
docker-compose.ymlif necessary
services:
dawarich_redis:
image: redis:7.0-alpine
command: redis-server
networks:
- dawarich
volumes:
- shared_data:/var/shared/redis
+ restart: always
dawarich_db:
image: postgres:14.2-alpine
container_name: dawarich_db
volumes:
- db_data:/var/lib/postgresql/data
- shared_data:/var/shared
networks:
- dawarich
environment:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: password
+ restart: alwaysSee the PR or Swagger docs (/api-docs) for more information.
- Owntracks points are now being saved to the database with the full attributes
- Existing owntracks points also filled with missing data
- Definition of "reverse geocoded points" is now correctly based on the number of points that have full reverse geocoding data instead of the number of points that have only country and city
- Fixed a bug in gpx importing scipt (thanks, bluemax!)
- A bug where an attempt to import a Google's Records.json file was failing due to wrong object being passed to a background worker
- PHOTON_API_HOST env variable to set the host of the Photon API. It will allow you to use your own Photon API instance instead of the default one.
- Pagination to exports page
- Pagination to imports page
- GET
/api/v1/pointsendpoint to get all points for the user with swagger docs - DELETE
/api/v1/points/:idendpoint to delete a single point for the user with swagger docs - DELETE
/api/v1/areas/:idswagger docs - User can now change route opacity in settings
- Points on the Points page can now be ordered by oldest or newest points
- Visits on the Visits page can now be ordered by oldest or newest visits
- Point deletion is now being done using an api key instead of CSRF token
- OpenStreetMap layer is now being selected by default in map controls
- Call to the background job to calculate visits
- Name of background job to calculate visits
- Map areas functionality
- A possibility to create areas. To create an area, click on the Areas checkbox in map controls (top right corner of the map), then in the top left corner of the map, click on a small circle icon. This will enable draw tool, allowing you to draw an area. When you finish drawing, release the mouse button, and the area will be created. Click on the area, set the name and click "Save" to save the area. You can also delete the area by clicking on the trash icon in the area popup.
- A background job to calculate your visits. This job will calculate your visits based on the areas you've created.
- Visits page. This page will show you all your visits, calculated based on the areas you've created. You can see the date and time of the visit, the area you've visited, and the duration of the visit.
- A possibility to confirm or decline a visit. When you create an area, the visit is not calculated immediately. You need to confirm or decline the visit. You can do this on the Visits page. Click on the visit, then click on the "Confirm" or "Decline" button. If you confirm the visit, it will be added to your timeline. If you decline the visit, it will be removed from your timeline.
- Settings for visit calculation. You can set the minimum time spent in the area to consider it as a visit. This setting can be found in the Settings page.
- POST
/api/v1/areasand GET/api/v1/areasendpoints. You can now create and list your areas via the API.
- A route popup now correctly shows distance made in the route, not the distance between first and last points in the route.
- A popup being shown when user clicks on a point now contains a link to delete the point. This is useful if you want to delete a point that was imported by mistake or you just want to clean up your data.
- Added
public/importsandpublic/exportsfolders to git to prevent errors when exporting data
- Some code from
maps_controller.jswas extracted into separate files
- Admin flag to the database. Now not only the first user in the system can create new users, but also users with the admin flag set to true. This will make easier introduction of more admin functions in the future.
- Route hover distance is now being rendered in kilometers, not in meters, if route distance is more than 1 km.
- Hover over a route does not move map anymore and shows the route tooltip where user hovers over the route, not at the end of the route. Click on route now will move the map to include the whole route.
- Fixed a bug where total reverse geocoded points were calculated based on number of imported points that are reverse geocoded, not on the number of total reverse geocoded points.
- Background jobs page. You can find it in Settings -> Background Jobs.
- Queue clearing buttons. You can clear all jobs in the queue.
- Reverse geocoding restart button. You can restart the reverse geocoding process for all of your points.
- Reverse geocoding continue button. Click on this button will start reverse geocoding process only for points that were not processed yet.
- A lot more data is now being saved in terms of reverse geocoding process. It will be used in the future to create more insights about your data.
- Point reference to a user is no longer optional. It should not cause any problems, but if you see any issues, please let me know.
⚠️ Calculation of total reverse geocoded points was changed.⚠️ Previously, the reverse geocoding process was recording only country and city for each point. Now, it records all the data that was received from the reverse geocoding service. This means that the total number of reverse geocoded points will be different from the previous one. It is recommended to restart the reverse geocoding process to get this data for all your existing points. Below you can find an example of what kind of data is being saved to your Dawarich database:
{
"place_id": 127850637,
"licence": "Data © OpenStreetMap contributors, ODbL 1.0. http://osm.org/copyright",
"osm_type": "way",
"osm_id": 718035022,
"lat": "52.51450815",
"lon": "13.350110811262352",
"class": "historic",
"type": "monument",
"place_rank": 30,
"importance": 0.4155071896625501,
"addresstype": "historic",
"name": "Victory Column",
"display_name": "Victory Column, Großer Stern, Botschaftsviertel, Tiergarten, Mitte, Berlin, 10785, Germany",
"address": {
"historic": "Victory Column",
"road": "Großer Stern",
"neighbourhood": "Botschaftsviertel",
"suburb": "Tiergarten",
"borough": "Mitte",
"city": "Berlin",
"ISO3166-2-lvl4": "DE-BE",
"postcode": "10785",
"country": "Germany",
"country_code": "de"
},
"boundingbox": [
"52.5142449",
"52.5147775",
"13.3496725",
"13.3505485"
]
}- Added a logging config to the
docker-compose.ymlfile to prevent logs from overflowing the disk. Now logs are being rotated and stored in thelogfolder in the root of the application. You can find usage example in the the repository'sdocker-compose.ymlfile. Make sure to add this config to bothdawarich_appanddawarich_sidekiqservices.
logging:
driver: "json-file"
options:
max-size: "100m"
max-file: "5"- Visiting notifications page now marks this notifications as read
- Guide on how to setup a reverse proxy for Dawarich in the
docs/how_to_setup_reverse_proxy.mdfile. This guide explains how to set up a reverse proxy for Dawarich using Nginx and Apache2.
MAP_CENTERenv var from thedocker-compose.ymlfile. This variable was used to set the default center of the map, but it is not needed anymore, as the map center is now hardcoded in the application.⚠️ Feel free to remove this variable from yourdocker-compose.ymlfile.⚠️
- Fixed a bug where Overland batch payload was not being processed due to missing coordinates in the payload. Now, if the coordinates are missing, the single point is skipped and the rest are being processed.
- Set
'localhost'string as a default value forAPPLICATION_HOSTSenvironment variable in thedocker-compose.ymlfile instead of an array. This is necessary to prevent errors when starting the application.
- Support for multiple hosts. Now you can specify the host of the application by setting the
APPLICATION_HOSTS(note plural form) environment variable in thedocker-compose.ymlfile. Example:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
APPLICATION_HOSTS: "yourhost.com,www.yourhost.com,127.0.0.1"Note, there should be no protocol prefixes in the APPLICATION_HOSTS variable, only the hostnames.
APPLICATION_HOST to APPLICATION_HOSTS to avoid any issues in the future, as APPLICATION_HOST will be deprecated in the nearest future.
- Support for HTTPS. Now you can specify the protocol of the application by setting the
APPLICATION_PROTOCOLenvironment variable in thedocker-compose.ymlfile. Default value ishttpExample:
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
environment:
APPLICATION_PROTOCOL: "https"- Support for a
location-history.jsonfile from Google Takeout. It turned out, this file could contain not only an object with location data history, but also an array of objects with location data history. Now Dawarich can handle both cases and import the data correctly.
- Notifications system. Now you will receive a notification when an import or export is finished, when stats update is completed and if any error occurs during any of these processes. Notifications are displayed in the top right corner of the screen and are stored in the database. You can see all your notifications on the Notifications page.
- Swagger API docs for
/api/v1/owntracks/points. You can find the API docs at/api-docs.
- Google Takeout geodata, taken from a mobile devise, is now fully supported and can be imported to the Dawarich. The import process is the same as for other kinds of files, just select the JSON file and choose "Google Phone Takeout" as a source.
- Fixed a bug where an imported point was not being saved to the database if a point with the same timestamp and already existed in the database even if it was other user's point.
- First user in the system can now create new users from the Settings page. This is useful for creating new users without the need to enable registrations. Default password for new users is
password.
- Registrations are now disabled by default. On the initial setup, a default user with email
user@domain.comand passwordpasswordis created. You can change the password in the Settings page. - On the Imports page, now you can see the real number of points imported. Previously, this number might have not reflect the real number of points imported.
- New Settings page to change Dawarich settings.
- New "Fog of War" toggle on the map controls.
- New "Fog of War meters" field in Settings. This field allows you to set the radius in meters around the point to be shown on the map. The map outside of this radius will be covered with a fog of war.
- Order of points on Points page is now descending by timestamp instead of ascending.
In new Settings page you can now change the following settings:
- Maximum distance between two points to consider them as one route
- Maximum time between two points to consider them as one route
- New Settings page to change Dawarich settings.
- Settings link in user menu now redirects to the new Settings page.
- Old settings page is now available undeer Account link in user menu.
This release introduces support for GPX files to be imported. Now you can import GPX files from your devices to Dawarich. The import process is the same as for other kinds of files, just select the GPX file instead and choose "gpx" as a source. Both single-segmented and multi-segmented GPX files are supported.
/api/v1/pointsendpoint is removed. Please use/api/v1/owntracks/pointsendpoint to upload your points from OwnTracks mobile app instead.
- Support for GPX files to be imported.
- Couple of unnecessary params were hidden from route popup and now can be shown using
?debug=truequery parameter. This is useful for debugging purposes.
/exports/downloadendpoint is removed. Now you can download your exports directly from the Exports page./api/v1/pointsendpoint is removed.
- A link to Dawarich's website in the footer. It ain't much, but it's honest work.
- Fixed version badge in the navbar. Now it will show the correct version of the application.
- Default map center location was changed.
Please update your docker-compose.yml file to include the following changes:
dawarich_sidekiq:
image: freikin/dawarich:latest
container_name: dawarich_sidekiq
volumes:
- gem_cache:/usr/local/bundle/gems
+ - public:/var/app/public- Added a line with public volume to sidekiq's docker-compose service to allow sidekiq process to write to the public folder
- Fixed a bug where the export file was not being created in the public folder
This is a debugging release. No changes were made to the application.
- Exports page to list existing exports download them or delete them
- Exporting process now is done in the background, so user can close the browser tab and come back later to download the file. The status of the export can be checked on the Exports page.
ℹ️ Deleting Export file will only delete the file, not the points in the database. ℹ️
Volume, exposed to the host machine for placing files to import was changed. See the changes below.
Path for placing files to import was changed from tmp/imports to public/imports.
...
dawarich_app:
image: freikin/dawarich:latest
container_name: dawarich_app
volumes:
- gem_cache:/usr/local/bundle/gems
- - tmp:/var/app/tmp
+ - public:/var/app/public/imports
... ...
volumes:
db_data:
gem_cache:
shared_data:
- tmp:
+ public:- A data migration to remove points with 0.0, 0.0 coordinates. This is necessary to prevent errors when calculating distance in Stats page.
- Reworked code responsible for importing "Records.json" file from Google Takeout. Now it is more reliable and faster, and should not throw as many errors as before.
- Test version of google takeout importing service for exports from users' phones
- Background jobs concurrency now can be set with
BACKGROUND_PROCESSING_CONCURRENCYenv variable indocker-compose.ymlfile. Default value is 10. - Hand-made favicon
- Change minutes to days and hours on route popup
- Improved speed of "Stats" page loading by removing unnecessary queries
- New buttons to quickly move to today's, yesterday's and 7 days data on the map
- "Download JSON" button to points page
- For debugging purposes, now user can use
?meters_between_routes=500and?minutes_between_routes=60query parameters to set the distance and time between routes to split them on the map. This is useful to understand why routes might not be connected on the map. - Added scale indicator to the map
- Removed "Your data" page as its function was replaced by "Download JSON" button on the points page
- Hovering over a route now also shows time and distance to next route as well as time and distance to previous route. This allows user to understand why routes might not be connected on the map.
- Now user can hover on a route and see when it started, when it ended and how much time it took to travel
- Timestamps in export form are now correctly assigned from the first and last points tracked by the user
- Routes are now being split based both on distance and time. If the time between two consecutive points is more than 60 minutes, the route is split into two separate routes. This improves visibility of the routes on the map.
- Routes are now being split into separate one. If distance between two consecutive points is more than 500 meters, the route is split into two separate routes. This improves visibility of the routes on the map.
- Background jobs concurrency is increased from 5 to 10 to speed up the processing of the points.
- Point data, accepted from OwnTracks and Overland, is now being checked for duplicates. If a point with the same timestamp and coordinates already exists in the database, it will not be saved.
- Heatmap layer on the map to show the density of points
BREAKING CHANGES:
/api/v1/pointsis still working, but will be deprecated in nearest future. Please use/api/v1/owntracks/pointsinstead.- All existing points recorded directly to the database via Owntracks or Overland will be attached to the user with id 1.
- Each user now have an api key, which is required to make requests to the API. You can find your api key in your profile settings.
- You can re-generate your api key in your profile settings.
- In your user profile settings you can now see the instructions on how to use the API with your api key for both OwnTracks and Overland.
- Added docs on how to use the API with your api key. Refer to
/api-docsfor more information. POST /api/v1/owntracks/pointsendpoint.- Points are now being attached to a user directly, so you can only see your own points and no other users of your applications can see your points.
/api/v1/overland/batchesendpoint now requires an api key to be passed in the url. You can find your api key in your profile settings.- All existing points recorded directly to the database will be attached to the user with id 1.
- All stats and maps are now being calculated and rendered based on the user's points only.
- Default
TIME_ZONEenvironment variable is now set to 'UTC' in thedocker-compose.ymlfile.
- Fixed a bug where marker on the map was rendering timestamp without considering the timezone.
- Docker volume for importing Google Takeout data to the application
- Instruction on how to import Google Takeout data to the application
- Instruction on how to import Google Takeout data to the application
- Add Points page to display all the points as a table with pagination to allow users to delete points
- Sidekiq web interface to monitor background jobs is now available at
/sidekiq - Now you can choose a date range of points to be exported
- Stop selecting
raw_datacolumn during requests toimportsandpointstables to improve performance.
- Rename PointsController to MapController along with all the views and routes
- Add Points page to display all the points as a table with pagination to allow users to delete points
- Stop ignoring
raw_datacolumn during requests toimportsandpointstables. This was preventing points from being created.
- In right sidebar you can now see the total amount of geopoints aside of kilometers traveled
- Improved overall performance if the application by ignoring
raw_datacolumn during requests toimportsandpointstables.
- Now you can import
records.jsonfile from your Google Takeout archive, not just Semantic History Location JSON files. The import process is the same as for Semantic History Location JSON files, just select therecords.jsonfile instead and choose "google_records" as a source.
- Swagger docs, can be found at
https:<your-host>/api-docs
- Cities, visited by user and listed in right sidebar now also have an active link to a date they were visited
- Dark/light theme switcher in navbar is now being saved in user settings, so it persists between sessions
Breaking changes:
This release changes how Dawarich handles a city visit threshold. Previously, the MINIMUM_POINTS_IN_CITY environment variable was used to determine the minimum number of points in a city to consider it as visited. Now, the MIN_MINUTES_SPENT_IN_CITY environment variable is used to determine the minimum minutes between two points to consider them as visited the same city.
The logic behind this is the following: if you have a lot of points in a city, it doesn't mean you've spent a lot of time there, especially if your OwnTracks app was in "Move" mode. So, it's better to consider the time spent in a city rather than the number of points.
In your docker-compose.yml file, you need to replace the MINIMUM_POINTS_IN_CITY environment variable with MIN_MINUTES_SPENT_IN_CITY. The default value is 60, in minutes.
- A test for CheckAppVersion service class
- Replaced ActiveStorage with Shrine for file uploads
ActiveStorage::FileNotFoundErrorerror when uploading export files
- Set Redis as default cache store
- Consider timezone when parsing datetime params in points controller
- Add rescue for check version service class
- Application version badge to the navbar with check for updates button
- Npm dependencies install to Github build workflow
- Footer
- Disabled map points rendering by default to improve performance on big datasets
- Map controls to toggle polylines and points visibility
- Added content padding for mobile view
- Fixed stat card layout for mobile view
- Removed strong_params from POST /api/v1/points
ActiveStorage::FileNotFoundError: ActiveStorage::FileNotFoundErrorerror when uploading export files
You can now use Overland mobile app to track your location.
- Overland API endpoint (POST /api/v1/overland/batches)
You can now specify the host of the application by setting the APPLICATION_HOST environment variable in the docker-compose.yml file.
- Added version badge to navbar
- Added APPLICATION_HOST environment variable to docker-compose.yml to allow user to specify the host of the application
- Added CHANGELOG.md to keep track of changes
- Specified gem version in Docker entrypoint