-
Notifications
You must be signed in to change notification settings - Fork 232
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #455 from ipfs/remove-kubo
chore: remove Kubo userland specifications
- Loading branch information
Showing
7 changed files
with
6 additions
and
650 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,258 +1,3 @@ | ||
| #  Keystore | ||
| # Keystore | ||
|
|
||
| **Authors(s):** | ||
| - [whyrusleeping](github.com/whyrusleeping) | ||
| - [Hector Sanjuan](github.com/hsanjuan) | ||
|
|
||
| * * * | ||
|
|
||
| **Abstract** | ||
|
|
||
| This spec provides definitions and operations for the keystore feature in IPFS. | ||
|
|
||
| # Table of Contents | ||
|
|
||
| TODO | ||
|
|
||
| ## Goals: | ||
|
|
||
| To have a secure, simple and user-friendly way of storing and managing keys | ||
| for use by ipfs. As well as the ability to share these keys, encrypt, decrypt, | ||
| sign and verify data. | ||
|
|
||
| ## Planned Implementation | ||
|
|
||
| ### Key storage | ||
|
|
||
| Storage layout and format is defined in the [`REPO_FS`](REPO_FS.md) part of the spec. | ||
|
|
||
| ### Interface | ||
|
|
||
| #### ipfs key | ||
|
|
||
| ``` | ||
| ipfs key - Interact with ipfs keypairs | ||
| SUBCOMMANDS: | ||
| ipfs key gen - Generates a new named ipfs keypair | ||
| ipfs key list - Lists out all local keypairs | ||
| ipfs key info <key> - Get information about a given key | ||
| ipfs key rm <key> - Delete a given key from your keystore | ||
| ipfs key rename <key> <name> - Renames a given key | ||
| ipfs key show <key> - Print out a given key | ||
| ipfs key send <key> <peer> - Shares a specified private key with the given peer | ||
| Use 'ipfs key <subcmd> --help' for more information about each command. | ||
| DESCRIPTION: | ||
| 'ipfs key' is a command used to manage ipfs keypairs. | ||
| ``` | ||
|
|
||
| #### ipfs crypt | ||
|
|
||
| ``` | ||
| ipfs crypt - Perform cryptographic operations using ipfs keypairs | ||
| SUBCOMMANDS: | ||
| ipfs crypt sign <data> - Generates a signature for the given data with a specified key | ||
| ipfs crypt verify <data> <sig> - Verify that the given data and signature match | ||
| ipfs crypt encrypt <data> - Encrypt the given data | ||
| ipfs crypt decrypt <data> - Decrypt the given data | ||
| DESCRIPTION: | ||
| `ipfs crypt` is a command used to perform various cryptographic operations | ||
| using ipfs keypairs, including: signing, verifying, encrypting and decrypting. | ||
| ``` | ||
|
|
||
| #### Some subcommands: | ||
|
|
||
| ##### ipfs key Gen | ||
|
|
||
|
|
||
| ``` | ||
| ipfs key gen <name> - Generate a new ipfs keypair | ||
| OPTIONS: | ||
| -t, -type string - Specify the type and size of key to generate (i.e. rsa) | ||
| -s. -size int - Size of the key to generate | ||
| -p, -passphrase string - Passphrase for encrypting the private key on disk | ||
| DESCRIPTION: | ||
| 'ipfs key gen' is a command used to generate new keypairs. | ||
| ``` | ||
|
|
||
| * * * | ||
|
|
||
| ##### Key Send | ||
| ``` | ||
| ipfs key send <key> <peer> - Send a keypair to a given peer | ||
| OPTIONS: | ||
| -y, -yes bool - Yes to the prompt | ||
| DESCRIPTION: | ||
| 'ipfs key send' is a command used to share keypairs with other trusted users. | ||
| It will first look up the peer specified and print out their information and | ||
| prompt the user "are you sure? [y/n]" before sending the keypair. The target | ||
| peer must be online and dialable in order for the key to be sent. | ||
| Note: while it is still managed through the keystore, ipfs will prevent you from | ||
| sharing your nodes private key with anyone else. | ||
| ``` | ||
|
|
||
| ##### Comments: | ||
|
|
||
| Ensure that the user knows the implications of sending a key. | ||
|
|
||
| * * * | ||
|
|
||
| ##### Crypt Encrypt | ||
| ``` | ||
| ipfs crypt encrypt <data> - Encrypt the given data with a specified key | ||
| ARGUMENTS: | ||
| data - The filename of the data to be encrypted ("-" for stdin) | ||
| OPTIONS: | ||
| -k, -key string - The name of the key to use for encryption (default: localkey) | ||
| -o, -output string - The name of the output file (default: stdout) | ||
| -c, -cipher string - The cipher to use for the operation | ||
| -m, -mode string - The block cipher mode to use for the operation | ||
| DESCRIPTION: | ||
| 'ipfs crypt encrypt' is a command used to encypt data so that only holders of a certain | ||
| key can read it. | ||
| ``` | ||
|
|
||
| ##### Comments: | ||
|
|
||
| This should probably just operate on raw data and not on DAGs. | ||
|
|
||
| * * * | ||
|
|
||
| ##### Other Interface Changes | ||
|
|
||
| We will also need to make additions to support keys in other commands, these changes are as follows: | ||
|
|
||
| - `ipfs add` | ||
| - Support for a `-encrypt-key` option, for block encrypting the file being added with the key | ||
| - also adds an 'encrypted' node above the root unixfs node | ||
| - Support for a `-sign-key` option to attach a signature node above the root unixfs node | ||
|
|
||
| - `ipfs block put` | ||
| - Support for a `-encrypt-key` option, for encrypting the block before hashing and storing | ||
|
|
||
| - `ipfs object put` | ||
| - Support for a `-encrypt-key` option, for encrypting the object before hashing and storing | ||
|
|
||
| - `ipfs name publish` | ||
| - Support for a `-key` option to select which keyspace to publish to | ||
|
|
||
| ### Code changes and additions | ||
|
|
||
| This sections outlines code organization around this feature. | ||
|
|
||
| #### Keystore package | ||
|
|
||
| The fsrepo carries a `keystore` that can be used to load/store keys. The keystore is implemented following this interface: | ||
|
|
||
| ```go | ||
| // Keystore provides a key management interface | ||
| type Keystore interface { | ||
| // Has returns whether or not a key exist in the Keystore | ||
| Has(string) (bool, error) | ||
| // Put stores a key in the Keystore, if a key with the same name already exists, returns ErrKeyExists | ||
| Put(string, ci.PrivKey) error | ||
| // Get retrieves a key from the Keystore if it exists, and returns ErrNoSuchKey | ||
| // otherwise. | ||
| Get(string) (ci.PrivKey, error) | ||
| // Delete removes a key from the Keystore | ||
| Delete(string) error | ||
| // List returns a list of key identifier | ||
| List() ([]string, error) | ||
| } | ||
| ``` | ||
|
|
||
| Note: Never store passwords as strings, strings cannot be zeroed out after they are used. | ||
| using a byte array allows you to write zeroes over the memory so that the users password | ||
| does not linger in memory. | ||
|
|
||
| #### Unixfs | ||
|
|
||
| - new node types, 'encrypted' and 'signed', probably shouldn't be in unixfs, just understood by it | ||
| - if new node types are not unixfs nodes, special consideration must be given to the interop | ||
|
|
||
| - DagReader needs to be able to access keystore to seamlessly stream encrypted data we have keys for | ||
| - also needs to be able to verify signatures | ||
|
|
||
| #### Importer | ||
|
|
||
| - DagBuilderHelper needs to be able to encrypt blocks | ||
| - Dag Nodes should be generated like normal, then encrypted, and their parents should | ||
| link to the hash of the encrypted node | ||
| - DagBuilderParams should have extra parameters to accommodate creating a DBH that encrypts the blocks | ||
|
|
||
| #### New 'Encrypt' package | ||
|
|
||
| Should contain code for crypto operations on dags. | ||
|
|
||
| Encryption of dags should work by first generating a symmetric key, and using | ||
| that key to encrypt all the data. That key should then be encrypted with the | ||
| public key chosen and stored in the Encrypted DAG structure. | ||
|
|
||
| Note: One option is to simply add it to the key interface. | ||
|
|
||
| ### Structures | ||
| Some tentative mockups (in json) of the new DAG structures for signing and encrypting | ||
|
|
||
| Signed DAG: | ||
| ``` | ||
| { | ||
| "Links" : [ | ||
| { | ||
| "Name":"@content", | ||
| "Hash":"QmTheContent", | ||
| } | ||
| ], | ||
| "Data": protobuf{ | ||
| "Type":"Signed DAG", | ||
| "Signature": "thesignature", | ||
| "PubKeyID": "QmPubKeyHash", | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Encrypted DAG: | ||
| ``` | ||
| { | ||
| "Links" : [ | ||
| { | ||
| "Name":"@content", | ||
| "Hash":"QmRawEncryptedDag", | ||
| } | ||
| ], | ||
| "Data": protobuf{ | ||
| "Type":"Encrypted DAG", | ||
| "PubKeyID": "QmPubKeyHash", | ||
| "Key": "ephemeral symmetric key, encrypted with public key", | ||
| } | ||
| } | ||
| ``` | ||
| Moved to https://github.com/ipfs/kubo/blob/master/docs/specifications/keystore.md |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,128 +1,3 @@ | ||
| #  IPFS Repo Spec | ||
| # Repository | ||
|
|
||
| **Author(s)**: | ||
| - [Juan Benet](github.com/jbenet) | ||
|
|
||
| * * * | ||
|
|
||
| **Abstract** | ||
|
|
||
| This spec defines an IPFS Repo, its contents, and its interface. It does not specify how the repo data is actually stored, as that is done via swappable implementations. | ||
|
|
||
| # Table of Contents | ||
|
|
||
| TODO | ||
|
|
||
| ## Definition | ||
|
|
||
| A `repo` is the storage repository of an IPFS node. It is the subsystem that | ||
| actually stores the data IPFS nodes use. All IPFS objects are stored | ||
| in a repo (similar to git). | ||
|
|
||
| There are many possible repo implementations, depending on the storage media | ||
| used. Most commonly, IPFS nodes use an [fs-repo](REPO_FS.md). | ||
|
|
||
| Repo Implementations: | ||
| - [fs-repo](REPO_FS.md) - stored in the os filesystem | ||
| - mem-repo - stored in process memory | ||
| - s3-repo - stored in amazon s3 | ||
|
|
||
| <center> | ||
| <img src="img/ipfs-repo.png" width="256" /> | ||
| </center> | ||
|
|
||
| ## Repo Contents | ||
|
|
||
| The Repo stores a collection of [IPLD](https://github.com/ipld/specs#readme) objects that represent: | ||
|
|
||
| - **config** - node configuration and settings | ||
| - **datastore** - content stored locally, and indexing data | ||
| - **keystore** - cryptographic keys, including node's identity | ||
| - **hooks** - scripts to run at predefined times (not yet implemented) | ||
|
|
||
| Note that the IPLD objects a repo stores are divided into: | ||
| - **state** (system, control plane) used for the node's internal state | ||
| - **content** (userland, data plane) which represent the user's cached and pinned data. | ||
|
|
||
| Additionally, the repo state must determine the following. These need not be IPLD objects, though it is of course encouraged: | ||
|
|
||
| - **version** - the repo version, required for safe migrations | ||
| - **locks** - process semaphores for correct concurrent access | ||
| - **datastore_spec** - array of mounting points and their properties | ||
|
|
||
| Finally, the repo also stores the blocks with blobs containing binary data. | ||
|
|
||
|  | ||
|
|
||
| ### version | ||
|
|
||
| Repo implementations may change over time, thus they MUST include a `version` recognizable across versions. Meaning that a tool MUST be able to read the `version` of a given repo type. | ||
|
|
||
| For example, the `fs-repo` simply includes a `version` file with the version number. This way, the repo contents can evolve over time but the version remains readable the same way across versions. | ||
|
|
||
| ### datastore | ||
|
|
||
| IPFS nodes store some IPLD objects locally. These are either (a) **state objects** required for local operation -- such as the `config` and `keys` -- or (b) **content objects** used to represent data locally available. **Content objects** are either _pinned_ (stored until they are unpinned) or _cached_ (stored until the next repo garbage collection). | ||
|
|
||
| The name "datastore" comes from [go-datastore](https://github.com/jbenet/go-datastore), a library for swappable key-value stores. Like its name-sake, some repo implementations feature swappable datastores, for example: | ||
| - an fs-repo with a leveldb datastore | ||
| - an fs-repo with a boltdb datastore | ||
| - an fs-repo with a union fs and leveldb datastore | ||
| - an fs-repo with an s3 datastore | ||
| - an s3-repo with a cached fs and s3 datastore | ||
|
|
||
| This makes it easy to change properties or performance characteristics of a repo without an entirely new implementation. | ||
|
|
||
| ### keystore | ||
|
|
||
| A Repo typically holds the keys a node has access to, for signing and for encryption. | ||
|
|
||
| Details on operation and storage of the keystore can be found in [`REPO_FS.md`](REPO_FS.md) and [`KEYSTORE.md`](KEYSTORE.md). | ||
|
|
||
| ### config (state) | ||
|
|
||
| The node's `config` (configuration) is a tree of variables, used to configure various aspects of operation. For example: | ||
| - the set of bootstrap peers IPFS uses to connect to the network | ||
| - the Swarm, API, and Gateway network listen addresses | ||
| - the Datastore configuration regarding the construction and operation of the on-disk storage system. | ||
|
|
||
| There is a set of properties, which are mandatory for the repo usage. Those are `Addresses`, `Discovery`, `Bootstrap`, `Identity`, `Datastore` and `Keychain`. | ||
|
|
||
| It is recommended that `config` files avoid identifying information, so that they may be re-shared across multiple nodes. | ||
|
|
||
| **CHANGES**: today, implementations like js-ipfs and go-ipfs store the peer-id and private key directly in the config. These will be removed and moved out. | ||
|
|
||
| ### locks | ||
|
|
||
| IPFS implementations may use multiple processes, or may disallow multiple processes from using the same repo simultaneously. Others may disallow using the same repo but may allow sharing _datastores_ simultaneously. This synchronization is accomplished via _locks_. | ||
|
|
||
| All repos contain the following standard locks: | ||
| - `repo.lock` - prevents concurrent access to the repo. Must be held to _read_ or _write_. | ||
|
|
||
| ### datastore_spec | ||
|
|
||
| This file is created according to the Datastore configuration specified in the `config` file. It contains an array with all the mounting points that the repo is using, as well as its properties. This way, the `datastore_spec` file must have the same mounting points as defined in the Datastore configuration. | ||
|
|
||
| It is important pointing out that the `Datastore` in config must have a `Spec` property, which defines the structure of the ipfs datastore. It is a composable structure, where each datastore is represented by a json object. | ||
|
|
||
| ### hooks (TODO) | ||
|
|
||
| Like git, IPFS nodes will allow `hooks`, a set of user configurable scripts to run at predefined moments in IPFS operations. This makes it easy to customize the behavior of IPFS nodes without changing the implementations themselves. | ||
|
|
||
| ## Notes | ||
|
|
||
| #### A Repo uniquely identifies an IPFS Node | ||
|
|
||
| A repository uniquely identifies a node. Running two different IPFS programs with identical repositories -- and thus identical identities -- WILL cause problems. | ||
|
|
||
| Datastores MAY be shared -- with proper synchronization -- though note that sharing datastore access MAY erode privacy. | ||
|
|
||
| #### Repo implementation changes MUST include migrations | ||
|
|
||
| **DO NOT BREAK USERS' DATA.** This is critical. Thus, any changes to a repo's implementation **MUST** be accompanied by a **SAFE** migration tool. | ||
|
|
||
| See https://github.com/jbenet/go-ipfs/issues/537 and https://github.com/jbenet/random-ideas/issues/33 | ||
|
|
||
| #### Repo Versioning | ||
|
|
||
| A repo version is a single incrementing integer. All versions are considered non-compatible. Repos of different versions MUST be run through the appropriate migration tools before use. | ||
| Moved to https://github.com/ipfs/kubo/blob/master/docs/specifications/repository.md |
Oops, something went wrong.