new config #166

daviddias · 2018-02-20T08:42:50Z

For #46. Takes into account:

Would love to get a ton of feedback from libp2p users.

Add support to receive transport instances or Classes
Needs:
- feat: add tag, update deps and fix tests js-libp2p-bootstrap#75
- Add tag js-libp2p-mdns#74

alanshaw · 2018-02-20T09:24:10Z

README.md

    }

-    super(modules, peerInfo, peerBook, options)
+    // overload any defaults of your bundle
+    Object.assign(options, _options)


Double checking you want an assign? You've gone to a lot of trouble to establish defaults which are easily removed. For example, if I pass an options object like { config: { addrs: { listen: ['/ip4/0.0.0.0/tcp/5555'] } } } I blow away all the other defaults that have been set for peerDiscovery, protocolMuxing etc.

Do you need a merge or a smart assign instead?

You are right, it need to be a deepMerge.

look into lodash defaults/deepDefaults instead, they do what you need.

mkg20001 · 2018-02-20T13:20:31Z

README.md

-      // DHT is passed as its own enabling PeerRouting, ContentRouting and DHT itself components
-      dht: DHT
+      config: {                       // The config object is the part of the config that can go into a file, config.json.
+        addrs: {                      // Multiaddrs


The peerInfo already contains multiaddrs. What happens with that?

They get added/merged. This is to enable the config to be a single JSON file that has the addrs without any complex setup.

mkg20001 · 2018-02-20T13:25:18Z

README.md

+      peerInfo: _peerInfo             // The Identity of your Peer
+      peerBook: _peerBook             // Where peers get tracked, if undefined libp2p will create one instance
+      modules: {
+        transport: [


I think some kind of first and second class transport system is needed.
For example: the new ws-star needs the libp2p swarm to dial the server via WS but WS is independent from the swarm.
(Edit: That means that WS would be a first class transport and ws-star a second class)
So any listen/dial stuff for first class transports would be setup before the second class transports are setup (so dialing does not fail)

Both transports will be first class with #130 occurring during the #159. Not the concern of this proposal/PR

we had a proposal for transport priorities at some point, but it was rejected.

@dryajov link it please

@diasdavid https://lodash.com/docs/4.17.5#defaults and https://lodash.com/docs/4.17.5#defaultsDeep

I'm confused, wrong c&p?

my bad, this is the correct one - libp2p/js-libp2p-tcp#78

@dryajov that was a proposal for a pseudo priority based on what the module says. It needs to be a user defined.

mkg20001 · 2018-02-20T13:27:44Z

README.md

+          transport: {
+            secio: {
+              spdy: {
+                '*': '*'


What does this exactly mean? Is the thing on the left the multiprotocol for secio and the thing on the right the protocol that is actually being dialed?

Good questions.

The nested'ness here is to represent order/options. This says "After the transport, always do SECIO, after SECIO you can do SPDY or MPLEX, then do whatever"

re: '*': '*' - My intention is to mean "Now protocol mux whatever protocol you want".

mkg20001 · 2018-02-20T13:29:30Z

README.md

+          ann: []                     // To announce/share with the network (i.e DNS addr)
+          notAnn: []                  // To keep private
+        },
+        peerDiscovery: {


How will the options be passed to the module here?

They will, will present these details after this first pass on the looks of the config.

victorb · 2018-02-20T13:45:46Z

README.md

+            interval: 1000            // ms
+            enabled: true
+          },
+          webrtc-star: {              // webrtc-star options


Not consistent, should be webrtcStar

good catch 👍

victorb · 2018-02-20T13:48:18Z

README.md

+      config: {                       // The config object is the part of the config that can go into a file, config.json.
+        addrs: {                      // Multiaddrs
+          listen: []                  // To listen
+          ann: []                     // To announce/share with the network (i.e DNS addr)


Not a fan of these abbraviations, makes it difficult to understand what it really is unless you have context.

Also, taking inspiration from go-ipfs, it would turn out something like this:

{ addresses: { swarm: [] // to listen announce: [] // for sure to announce noAnnounce: [] // stop these from announcing at all cost api: '' gateway: '' } }

daviddias · 2018-03-20T02:25:16Z

I've continue my work on this PR. Made some progress and overall I like where this is headed (which is eventually enabling simpler configuration and #130).

I tried passing the responsibility completely to libp2p to instantiate all of its modules (which was not the case for Transports and Discovery services) but I hit the 🐓&🥚 of transports such as WebRTC that might require special binding modules such as wrtc.

I do want to make sure that everything that goes on config is serializable to JSON and I do want libp2p to be able to control the instantiation of modules, however, I believe I can not escape the need to be able to pass already instantiated modules too. It is not hard, just annoying because now I have to add a hint to the module to tells me if the reference is to an instance or to a class.

If this just sounded like alien 👽 mumbo jumbo to you, do not worry, I'll carry on with the coding and once it is fully ready, it will provide more clarity. If you are interested in understanding, let me know :)

dryajov · 2018-03-20T15:49:53Z

I have to add a hint to the module to tells me if the reference is to an instance or to a class.

This sounds like you're recreating IoC (inversion of control) containers all over :) There are some out there that do that already.

daviddias · 2018-06-05T11:06:29Z

README.md

+      EXPERIMENTAL: {                 // Experimental features ("behind a flag")
+        pubsub: true,
+        dht: true
+      }


Make EXPERIMENTAL part of the config.

daviddias · 2018-06-05T11:07:25Z

README.md

+              mplex: '*'
+            }
+          }
+        }


This feature will have to be implemented at the Switch Level

Before merging, I'll remove these options from the README and continue working on this feature design at #201

daviddias · 2018-06-05T13:38:04Z

README.md

@@ -101,47 +101,90 @@ libp2p becomes very simple and basically acts as a glue for every module that co
 ```JavaScript
 // Creating a bundle that adds:
 //   transport: websockets + tcp
-//   stream-muxing: SPDY
+//   stream-muxing: spdy & mplex
 //   crypto-channel: secio
 //   discovery: multicast-dns

 const libp2p = require('libp2p')
 const TCP = require('libp2p-tcp')
 const WS = require('libp2p-websockets')
 const spdy = require('libp2p-spdy')


daviddias · 2018-06-05T13:38:17Z

README.md

 //   crypto-channel: secio
 //   discovery: multicast-dns

 const libp2p = require('libp2p')
 const TCP = require('libp2p-tcp')
 const WS = require('libp2p-websockets')
 const spdy = require('libp2p-spdy')
+const mplex = require('libp2p-mplex')


uppercase + needs to be included in the modules

daviddias · 2018-06-05T13:38:23Z

README.md

 //   crypto-channel: secio
 //   discovery: multicast-dns

 const libp2p = require('libp2p')
 const TCP = require('libp2p-tcp')
 const WS = require('libp2p-websockets')
 const spdy = require('libp2p-spdy')
+const mplex = require('libp2p-mplex')
 const secio = require('libp2p-secio')


alanshaw · 2018-06-27T15:23:57Z

src/index.js

-        setImmediate(() => discovery.stop(() => {}))
+    if (this._modules.peerDiscovery) {
+      this._discovery.forEach((d) => {
+        setImmediate(() => d.stop(() => {}))


Something for another PR, but this smothers any error while stopping and means that when callback is called for stop all the discoveries might not have stopped yet.

alanshaw · 2018-06-27T15:25:44Z

src/index.js

@@ -269,25 +296,26 @@ class Node extends EventEmitter {
    this._getPeerInfo(peer, (err, peerInfo) => {
      if (err) { return callback(err) }

-      this.switch.hangUp(peerInfo, callback)
+      this._switch.hangUp(peerInfo, callback)
    })
  }

  ping (peer, callback) {
    assert(this.isStarted(), NOT_STARTED_ERROR_MESSAGE)


Again for another PR - we need to call callback with this error or the process will quit.

alanshaw · 2018-06-27T15:38:33Z

Can we merge and release this as is and finish the examples in separate PRs?

jacobheun · 2018-06-27T16:48:58Z

@alanshaw I am working on this today. I think we need to resolve, #166 (comment) first, which I will work on. With the changes to the config, I feel it's important to make sure we have proper validation there so users aren't confused why their config is not working.

I should have this resolved today or tomorrow.

alanshaw · 2018-06-27T21:04:37Z

🚀 awesome @jacobheun! Apologies I hadn't clocked #166 (comment)

docs: update echo example docs: update libp2p in browser example docs: update pubsub example docs: update peer and content routing examples docs: update discovery mechanisms example docs: update encrypted comms example docs: update protocol and stream muxing example

fixed examples and tests as a result of the config validation test: add config tests and fix bugs fix: linting

jacobheun · 2018-06-27T22:21:09Z

Okay, so I think I have everything checked off. I've rebased from master so everything is up to date (@diasdavid fyi on the rebase in case you pull).

I've coupled the latest changes into two commits, hopefully this will help keep things clear.

The first commit is the updated examples. I ran through all of them a final time after my last changes and everything runs. Woo!
I added Joi to validate the configuration. The second commit is pretty much about that, but also includes some of the misc changes @alanshaw mentioned to add to future PRs. This resulted in some code clean up, as we're able to rely on thrown errors from config validation to avoid some unnecessary property checks. It also exposed some issues with the tests not using proper configs, so those are fixed too.

I added tests for the new config validation. There is some custom logic there for peerDiscovery in modules and config, as well as experimental dht.

Pubsub: pubsub should now only be initialized if it's enabled via EXPERIMENTAL. Calling any methods on pubsub when it's not enabled will result in an error.

Provided there are no changes requested from those 2 latest commits, I think this should be good to go.

daviddias · 2018-06-28T07:58:29Z

ci/Jenkinsfile

@@ -1,2 +1,2 @@
 // Warning: This file is automatically synced from https://github.com/ipfs/ci-sync so if you want to change it, please change it there and ask someone to sync all repositories.
-javascript()
+javascript(['nodejs_versions': ['8.11.3']])


@jacobheun @alanshaw switched to Node.js 8 for now until wrtc has builds for node 10

daviddias · 2018-06-28T07:59:53Z

examples/chat/README.md

+1. Run the listener in window 1, `node listener.js`
+2. Run the dialer in window 2, `node dialer.js`
+3. Type a message in either window and hit _enter_
+4. Tell youself secrets to your hearts content!


Thank you for writing the tutorial. Helps for #186

daviddias

Awesome! Thank you for taking this one to the finish line with flying colors, @jacobheun ❤️

daviddias · 2018-06-28T08:02:42Z

Only test that failed is Jenkins related (classic Mac OS failure) //cc @victorbjelkholm

…2p#166)

## [2.0.2](libp2p/js-libp2p-webrtc@v2.0.1...v2.0.2) (2023-05-15) ### Bug Fixes * use transport manager getListeners to get listen addresses ([libp2p#166](libp2p/js-libp2p-webrtc#166)) ([2e144f9](libp2p/js-libp2p-webrtc@2e144f9))

ghost assigned daviddias Feb 20, 2018

ghost added the status/in-progress In progress label Feb 20, 2018

daviddias requested review from pgte, olizilla, alanshaw, dignifiedquire, dryajov, kumavis and mkg20001 February 20, 2018 08:43

This was referenced Feb 20, 2018

Improve libp2p config #46

Closed

libp2p.next() #159

Closed

alanshaw reviewed Feb 20, 2018

View reviewed changes

mkg20001 reviewed Feb 20, 2018

View reviewed changes

victorb reviewed Feb 20, 2018

View reviewed changes

This was referenced Mar 20, 2018

Add tag libp2p/js-libp2p-mdns#74

Merged

feat: add tag, update deps and fix tests libp2p/js-libp2p-bootstrap#75

Merged

daviddias added the P0 Critical: Tackled by core team ASAP label Jun 4, 2018

daviddias mentioned this pull request Jun 4, 2018

Awesome Endeavour: DHT Part II ipfs/js-ipfs#856

Merged

35 tasks

daviddias changed the title ~~new config proposal~~ new config Jun 4, 2018

daviddias force-pushed the better-opts branch from bcd98ea to 6b01652 Compare June 4, 2018 11:02

daviddias mentioned this pull request Jun 4, 2018

RFC: allow passing constructor for the libp2p modules ipfs/js-ipfs#1383

Closed

daviddias commented Jun 5, 2018

View reviewed changes

daviddias force-pushed the better-opts branch from 920e8a2 to abc65c3 Compare June 5, 2018 11:54

daviddias commented Jun 5, 2018

View reviewed changes

pgte mentioned this pull request Jun 27, 2018

use an official version of js-ipfs peer-base/peer-base#1

Closed

ghost assigned jacobheun Jun 27, 2018

alanshaw approved these changes Jun 27, 2018

View reviewed changes

daviddias and others added 2 commits June 27, 2018 22:42

feat: (BREAKING CHANGE) overhaul libp2p config and constructor

c07ca62

refactor: use different defaultsDeep and clean up some code

8372433

jacobheun added 2 commits June 28, 2018 00:05

feat: add config validation

d74e18b

fixed examples and tests as a result of the config validation test: add config tests and fix bugs fix: linting

jacobheun force-pushed the better-opts branch from e3a4c01 to d74e18b Compare June 27, 2018 22:10

test: update CI configs, use only node 8

8533936

daviddias commented Jun 28, 2018

View reviewed changes

daviddias merged commit 6905f1b into master Jun 28, 2018

ghost removed the status/in-progress In progress label Jun 28, 2018

daviddias deleted the better-opts branch June 28, 2018 08:06

daviddias mentioned this pull request Jun 28, 2018

the example readme of "echo" still empty #146

Closed

vasco-santos mentioned this pull request Jun 28, 2018

docs: 2018 Q3 OKRs #207

Merged

pgte mentioned this pull request Jul 18, 2018

libp2p new config ipfs/js-ipfs#1451

Closed

snyk-bot mentioned this pull request Jan 12, 2022

[Snyk] Upgrade libp2p-gossipsub from 0.9.2 to 0.12.1 adamlaska/js-libp2p#9

Open

snyk-bot mentioned this pull request Jan 21, 2022

[Snyk] Security upgrade libp2p-gossipsub from 0.9.2 to 0.12.0 adamlaska/js-libp2p#12

Open

snyk-bot mentioned this pull request Mar 21, 2022

[Snyk] Security upgrade libp2p-gossipsub from 0.9.2 to 0.12.0 adamlaska/js-libp2p#18

Open

maschad pushed a commit to maschad/js-libp2p that referenced this pull request Jun 21, 2023

sync: update CI config files (libp2p#166)

ef955a2

maschad pushed a commit to maschad/js-libp2p that referenced this pull request Jun 21, 2023

ci: update Unified CI configuration (libp2p#166)

e10e505

maschad pushed a commit to maschad/js-libp2p that referenced this pull request Jun 21, 2023

fix: use transport manager getListeners to get listen addresses (libp…

2e144f9

…2p#166)

new config #166

new config #166

Conversation

daviddias commented Feb 20, 2018 • edited Loading

alanshaw Feb 20, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mkg20001 Feb 20, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daviddias commented Mar 20, 2018

dryajov commented Mar 20, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alanshaw commented Jun 27, 2018

jacobheun commented Jun 27, 2018

alanshaw commented Jun 27, 2018

jacobheun commented Jun 27, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

daviddias left a comment

Choose a reason for hiding this comment

daviddias commented Jun 28, 2018 • edited Loading

daviddias commented Feb 20, 2018 •

edited

Loading

alanshaw Feb 20, 2018 •

edited

Loading

mkg20001 Feb 20, 2018 •

edited

Loading

jacobheun commented Jun 27, 2018 •

edited

Loading

daviddias commented Jun 28, 2018 •

edited

Loading