diff --git a/README.md b/README.md index 3a613b1..d44992e 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Subversion Offline Solution (SOS 1.1.7) # +# Subversion Offline Solution (SOS 1.2) # [![Travis badge](https://travis-ci.org/ArneBachmann/sos.svg?branch=master)](https://travis-ci.org/ArneBachmann/sos) [![Build status](https://ci.appveyor.com/api/projects/status/fe915rtx02buqe4r?svg=true)](https://ci.appveyor.com/project/ArneBachmann/sos) @@ -52,14 +52,23 @@ SOS supports three different file handling models that you may use to your likin - Has a small user base as of now, therefore no reliable reports of compatibility and operational capability except for the automatic unit tests run on Travis CI and AppVeyor ### Compatibility ### -- SOS runs on any Python 3 distribution, including some versions of PyPy. Python 2 is not fully supported yet due to library issues, although SOS's programming language *Coconut* is generally able to transpile to valid Python 2 source code. Use `pip install sos-vcs[backport]` for Python 2 installation +- SOS runs on any Python 3.4 distribution or higher, including some versions of PyPy. Python 2 is not supported anymore due to library issues, although SOS's programming language *Coconut* is generally able to transpile to valid Python 2 source code. Use `pip install sos-vcs[backport]` to attemüt running SOS on Python 3.3 or earlier - SOS is compatible with above mentioned traditional VCSs: SVN, Git, gitless, Bazaar, Mercurial and Fossil - Filename encoding and console encoding: Full roundtrip support (on Windows) started only with Python 3.6.4 and has not been tested nor confirmed yet for SOS ## Latest Changes ## -- Version 1.2 released on 2018-02-xx: - - ... +- Version 1.2 released on 2018-02-04: + - [Bug 135, 145](https://github.com/ArneBachmann/sos/issues/135) Fixes a bug showing ignored files as deleted + - [Bug 147](https://github.com/ArneBachmann/sos/issues/147) Fixes `sos ls` problems + - [Enhancement 113](https://github.com/ArneBachmann/sos/issues/113) Usability improvements + - [Enhancement 122](https://github.com/ArneBachmann/sos/issues/122) Complete rework of merge logic and code + - [Enhancement 124](https://github.com/ArneBachmann/sos/issues/124) Use enum + - [Enhancement 137](https://github.com/ArneBachmann/sos/issues/137) Better usage pages + - [Enhancement 142, 143](https://github.com/ArneBachmann/sos/issues/142) Improved `sos config` and added local configurations + - [Enhancement 153](https://github.com/ArneBachmann/sos/issues/153) Removed Python 2 leftovers + - [Enhancement 159](https://github.com/ArneBachmann/sos/issues/159) Metadata updates + - [Feature 134, 161](https://github.com/ArneBachmann/sos/issues/134) Added dump option - Version 1.1 released on 2017-12-30: - [Bug 90](https://github.com/ArneBachmann/sos/issues/90) Removed directories weren't picked up - [Bug 93](https://github.com/ArneBachmann/sos/issues/93) Picky mode lists any file as added @@ -71,7 +80,7 @@ SOS supports three different file handling models that you may use to your likin - [QA 79](https://github.com/ArneBachmann/sos/issues/79) Added AppVeyor automated testing - [QA 94](https://github.com/ArneBachmann/sos/issues/94) More test coverage - Many little fixes and improvements - - Downloads until today: 3575 + - Downloads: 4870 - Version 1.0 released on 2017-12-14: - First release with basic functionality - Lots of test cases, good test coverage @@ -85,72 +94,77 @@ SOS supports three different file handling models that you may use to your likin While completing version 1.0 of SOS, I incidentally discovered an interesting [article by Gregory Szorc](https://gregoryszorc.com/blog/2017/12/11/high-level-problems-with-git-and-how-to-fix-them/) that discusses central weaknesses in the design of popular VCSs, with a focus on Git. Many of his arguments I have intuitively felt to be true as well and were the reason for the development of SOS: mainly the reduction of barriers between the developer's typical workflow and the VCS, which is most often used as a structured tool for "type and save in increments", while advanced features of Git are just very difficult to remember and get done right. - While Git is basically a large key-value store with a thin access interface on top, SOS keeps a very clear (folder) structure of branches, revisions and files -- Compared to SVN, SOS's file store is much simpler and doesn't require an integrated database +- Compared to SVN, SOS's file store is much simpler and doesn't require an integrated database, and recovery is manually possible with little effort -Here is a comparison between SOS and VCS's commands: -- `branch` creates a branch from the current file tree (or last commit), but also switches to it immediately (unless told not to). There is no requirement to name branches, removing all barriers +Here is a comparison between SOS and traditional VCS's commands: +- `branch` creates a branch from the current file tree, but also switches to it immediately. There is no requirement to name branches, removing all barriers - SOS allows to branch from the latest committed revision via `sos branch [] --last`; this automatically applies when in tracking and picky mode. In consequence any changes performed since last commit will automatically be considered as a change for the next commit on the branch unless `--stay` was added as well to not switch to the new branch -- `commit` creates a numbered revision similar to SVN, but revision numbers are only unique per branch, as they aren't stored in a global namespace. The commit message is optional on purpose (since `sos commit` serves largely as a CTRL+S replacement) +- `commit` creates a numbered revision from the current file tree, similar to how SVN does, but revision numbers are only unique per branch, as they aren't stored in a global namespace. The commit message is strictly *optional* on purpose (as `sos commit` serves largely as a CTRL+S replacement) - The first revision (created during execution of `sos offline` or `sos branch`) always has the number `0` - - Each `sos commit` increments the revision number by one; revisions are referenced by this numeric index only - - You can tag a commit. This way, the commit message serves as a tag name and is assured to be unique. Referring to a revision by its tag name can be used instead of numeric revision index, but works not only for tagged revisions and finds the first matching revision with a matching commit message -- `delete` destroys and removes a branch. It's a command, not an option flag as in `git branch -d ` -- `move` renames a file tracking pattern and all matching files accordingly; only useful in tracking or picky mode. It supports reordering of literal substrings, but no reordering of glob markers (`*`, `?` etc.), and no adjacent glob markers. Use `--soft` to avoid files actually being renamed in the file tree. Warning: the `--force` option flag will be considered for several consecutive, potentially dangerous operations. TODO allow and consider `--force` several times on the command line + - Each `sos commit` increments the revision number by one; revisions are referenced by this numeric index, the revision's optional commit message if given, or a tag + - Tagging a commit means that the commit message serves as a tag name and is assured to be unique. Referring to a revision by its tag name can be used instead of numeric revision index, but works not only for tagged revisions and finds the first matching revision with a matching commit message + - You may use negative revision indexes, just like Python does. `-1` refers to the latest revision, `-2` to the second-latest + - You may specify a revision of the current branch by `/`, while specifying the latest revision of another branch by `/` (note the position of the slash) +- `delete` destroys and removes a branch. It's a command, not an option flag as in `git branch -d ` for usability's sake +- `add` and `rm` add and remove tracking patterns, if the repository was created in tracking or picky mode. Patterns are never recursively applied, but always apply for a specific file tree path. They may contain, however, globs in their filename part, which makes it different from any other VCS in existence +- `move` renames a file tracking pattern and all matching files accordingly; only useful in tracking or picky mode. It supports reordering of literal substrings, but no reordering of glob markers (`*`, `?` etc.), and of adjacent glob markers. Use `--soft` to avoid files actually being renamed in the file tree. Warning: the `--force` option flag will be considered for several consecutive, potentially dangerous operations - `switch` works like `checkout` in Git for a revision of another branch (or of the current), or `update` to latest or a specific revision in SVN. Please note that switching to a different revision will in no way fix or remember that revision. The file tree will always be compared to the branch's latest commit for change detection -- `update` works a bit like `pull` in Git or `update` in SVN and replays the given branch's and/or revision's changes into the file tree. There are plenty of options to configure what changes are actually integrated. This command will not switch the current branch like `switch` does. Note, that this is not a real 3-way *merge*, just an enhanced *diff* logic. +- `update` works a bit like `pull` and merge in Git or `update` in SVN and replays the specified other (or "remote"'s) branch's and/or revision's changes into the file tree. There are plenty of options to configure what changes are actually integrated, plus interactive integration. This command will not switch the current branch like `switch` does. Note, that this is not a real 3-way *merge*, or *merge* at all, just a more flexible way to insert and remove text output from *diff*. When differing contents are to be merged, there is always a potential for conflict; not all changes can be merged automatically with confidence. SOS takes a simplistic and pragmatic approach and largely follows a simple diff algorithm to detect and highlight changes. Insertions and deletions are noted, and modifications are partially detected and marked as such. There are different layers of changes that SOS is able to work on: - - File addition or removal in the file tree, e.g. when updating from another branch and/or revision or switching to them - - Line insertion or deletion inside a file, e.g. when merging file modifications during update - - Character insertion or deletion on a text line, e.g. when non-conflicting intra-line differences are detected - - Updating state from another branch in the `--track` or `--picky` mode will always combine all tracked file patterns. To revert this, use the `switch --meta` command to pull back in another branch's and/or revision's tracking patterns to the currently active branch - - There may be, however, blocks of text lines that seem inserted/deleted but may have actually just been moved inside the file. SOS attempts to detect clear cases of moved blocks and silently accepts them no matter what. TODO implement and introduce option flag to avoid this behavior + - File addition or removal in the file tree, e.g. when updating from another branch and/or revision or switching to them, can be controlled by `--add`, `--rm` and `--ask`, which applies only for conflicts. Default is to replay both + - Line insertion or deletion inside a file, e.g. when merging file modifications during update, via `--add-lines`, `--rm-lines`, `--ask-lines`. Default is replay both + - Character insertion or deletion on a single text line being mergedf, e.g. when non-conflicting intra-line differences are detected, via `--add-chars`, `--rm-chars`, `--ask-chars`. Default is to replay both + - Updating state from another branch in the `--track` or `--picky` mode will always combine (build the union of) all tracked file patterns. To revert this, use the `switch --meta` command to pull back in another branch's and/or revision's tracking patterns to the currently active branch (may require to switch first to the other side). There is currently no check, if the pulled in tracking patterns are supersets or subsets of the onces being already there + - There may be, however, blocks of text lines that seem inserted/deleted but may have actually just been moved inside the file. TODO: SOS attempts to detect clear cases of moved blocks and silently accepts them no matter what. TODO: implement and introduce option flag to avoid this behavior ### Working in *Track* and *Picky* Modes ### -Use the commands `sos add ` or `sos rm ` to add or remove file patterns. These patterns always refer to a specific (relative) file path and may contain globbing characters `?*[!]` only in the filename part of the path. +Use the commands `sos add ` or `sos rm ` to add or remove file patterns. These patterns always refer to a specific (relative) file paths and may contain globbing characters `?*[!]` only in the filename part of the path. ## Configuration Options ## -These options can be set or unset by the user and apply globally for all offline operations the user performs from that moment on. -Some of these options can be set on a per-repository basis during creation (e.g. `sos offline --track --strict`), others can only be set in a persistant fashion (e.g. `sos config set texttype "*.xsd"`). +These options can be set or unset by the user and apply either globally for all offline operations the user performs from that moment on, or locally to one repository only (using the `--local` option flag). +Some of these options can be defined on a per-repository basis already during offline repository creation (e.g. `sos offline --track --strict --compress`), others can only be set in a persistant fashion (e.g. `sos config set texttype "*.xsd"`), or after repository creation (e.g. `sos config set texttype "*.xsd;*.xml" --local`). ### Configuration Commands ### - `sos config set` sets a boolean flag, a string, or an initial list (semicolon-separated) -- `sos config unset` removes a setting -- `sos config add` adds a string entry to a list -- `sos config rm` removes a string entry from a list -- `sos config show` lists all defined configuration settings +- `sos config unset` removes a boolean flag, a string, or an entire list +- `sos config add` adds a string entry to a list, and creates it if necessary +- `sos config rm` removes a string entry from a list. Must be typed exactly as the entry to remove +- `sos config show` lists all defined configuration settings, including storage location/type (global, local, default) +- `sos config show ` show only one configuration item +- `sos config show flags|texts|lists` show supported settings per type ### User Configuration and Defaults ### -SOS optionally uses the [`configr`](https://github.com/ArneBachmann/configr) library to manage per-user global defaults, e.g. for the `--strict` and `--track` flags that the `offline` command takes, but also for file and folder exclusion patterns. -By means of the `sos config set ` command, you can set these flags flag with values like `1`, `no`, `on`, `false`, `enable` or `disabled`. +SOS uses the [`configr`](https://github.com/ArneBachmann/configr) library to manage per-user global defaults, e.g. for the `--strict` and `--track` flags that the `offline` command takes, but also for often-used file and folder exclusion patterns. +By means of the `sos config set ` command, you can set these flags with values like `1`, `no`, `on`, `false`, `enable` or `disabled`. ### Available Configuration Settings ### -- `strict`: Flag for always performing full file comparsion, not relying on file size and modification timestamp only. Default: False +- `strict`: Flag for always performing full file comparsion, not relying on modification timestamp only; file size is always checked in both modes. Default: False - `track`: Flag for always going offline in tracking mode (SVN-style). Default: False - `picky`: Flag for always going offline in picky mode (Git-styly). Default: False -- `compress`: Flag for compressing versioned artifacts. Default: True -- `defaultbranch`: Name of the initial branch created when going offline. Default: Dynamic per type of VCS in current working directory (e.g. `master` for Git, `trunk` for SVN) +- `compress`: Flag for compressing versioned artifacts. Default: False +- `defaultbranch`: Name of the initial branch created when going offline. Default: Dynamic per type of VCS in current working directory (e.g. `master` for Git, `trunk` for SVN, no name for Fossil) - `texttype`: List of file patterns that should be recognized as text files that can be merged through textual diff, in addition to what Python's `mimetypes` library will detect as a `text/...` mime. *Default*: Empty list -- `bintype`: List of file patterns that should be recognized as binary files that cannot be merged textually, overriding potential matches in `texttype`. Default: Empty list -- `ignores`: List of filename patterns (without folder path) to ignore during repository operations. Any match from the corresponding white list will negate any hit for `ignores` -- `ignoresWhitelist`: List of filename patterns to be consider even if matched by an entry in the `ignores` list +- `bintype`: List of file patterns that should be recognized as binary files which cannot be merged textually, overriding potential matches in `texttype`. Default: Empty list +- `ignores`: List of filename patterns (without folder path) to ignore during repository operations. Any match from the corresponding white list will negate any hit for `ignores`. Default: See source code, e.g. `["*.bak", "*.py[cdo]]"` +- `ignoresWhitelist`: List of filename patterns to be consider even if matched by an entry in the `ignores` list. Default: Empty list - `ignoreDirs`: As `ignores`, but for folder names - `ignoreDirsWhitelist`: As `ignoresWhitelist`, but for folder names ## Noteworthy Details ## - SOS doesn't store branching point information (or references); each branch stands alone and has no relation whatsoever to other branches or certain revisions thereof, except incidentally its initial file contents -- File tracking patterns are stored per branch, but not versioned with commits. This means that the "what to track" metadata is not part of the changesets. +- File tracking patterns are stored per branch, but **not** versioned with commits (!). This means that the "what to track" metadata is not part of the changesets. This is a simplification stemming from the main idea that revisions form a linear order of safepoints, and users rarely go back to older revisions - `sos update` will **not warn** if local changes are present! This is a noteworthy exception to the failsafe approach taken for most other commands ## Hints and Tipps ## -- To migrate an offline repository, simple move the `.sos` folder into an (empty) target folder, and run `sos switch trunk --force` (or use another branch name). For compressed offline repositories, you may simply `tar` all files, otherwise you may want to create an compressed archive for transferring the `.sos` folder +- To migrate an offline repository, either use the `sos dump .sos.zip` command, or simple move the `.sos` folder into an (empty) target folder, and run `sos switch trunk --force` (or use whatever branch name you're wanting to recreate). For compressed offline repositories, you may simply `tar` all files, otherwise you may want to create an compressed archive for transferring the `.sos` folder - To save space when going offline, use the option `sos offline --compress`: It may increase commit times by a larger factor (e.g. 10x), but will also reduce the amount of storage needed to version files. To enable this option for all offline repositories, use `sos config set compress on` -- When specifying file patterns including glob markers on the command line, make sure you quote them correctly. On Linux (bash, sh, zsh), but also recommended on Windows, put your patterns into quote (`"`), otherwise the shell will replace file patterns by any matching filenames instead of forwarding the pattern literally to SOS -- Many commands can be shortened to three, two or even one initial letters, e.g. `sos st` will run `sos status`. Using SOS as a proxy to other VCS requires you to specify the form required by those, e.g. `sos st` works for SVN, but not for Git (`sos status`, however, would work) -- It might in some cases be a good idea to go offline one folder higher up in the file tree than your base working folder to care for potential deletions or renames +- When specifying file patterns including glob markers on the command line, make sure you quote them correctly. On Linux (bash, sh, zsh), but also recommended on Windows, put your patterns into quotes (`"`), otherwise the shell will replace file patterns by the list of any matching filenames instead of forwarding the pattern literally to SOS +- Many commands can be shortened to three, two or even one initial letters, e.g. `sos st` will run `sos status`, just like SVN does (but sadly not Git). Using SOS as a proxy to other VCS requires you to specify the form required by those, e.g. `sos st` works for SVN, but not for Git (`sos status`, however, would work) +- It might in some cases be a good idea to go offline one folder higher up in the file tree than your base working folder to care for potential deletions, moves, or renames - The dirty flag is only relevant in tracking and picky mode (?) TODO investigate - is this true, and if yes, why - Branching larger amounts of binary files may be expensive as all files are copied and/or compressed during `sos offline`. A workaround is to `sos offline` only in the folders that are relevant for a specific task @@ -163,4 +177,5 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for information. - Increase version number in `setup.py` - Run `python3 setup.py clean build test sdist` to update the PyPI version number, compile and test the code, and package it into an archive. If you need evelated rights to do so, use `sudo -E python...`. - Run `git add`, `git commit` and `git push` and let Travis CI and AppVeyor run the tests against different target platforms. If there were no problems, continue: +- Don't forget to tag releases - Run `twine upload dist/*.tar.gz` to upload the previously created distribution archive to PyPI. diff --git a/setup.py b/setup.py index 40ba51b..199ee5f 100644 --- a/setup.py +++ b/setup.py @@ -5,7 +5,7 @@ import os, shutil, subprocess, sys, time, unittest from setuptools import setup, find_packages -RELEASE = "1.1.7" +RELEASE = "1.2" print("sys.argv is %r" % sys.argv) readmeFile = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'README.md')