Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Panic while running daemon #3262

Closed
whyrusleeping opened this issue Sep 26, 2016 · 3 comments · Fixed by #3271
Closed

Panic while running daemon #3262

whyrusleeping opened this issue Sep 26, 2016 · 3 comments · Fixed by #3271
Assignees
@whyrusleeping
Labels
panic

Comments

@whyrusleeping
Copy link
Member

Version information:

master

Type:

Panic

Priority:

P1

Description:

Got a panic while running the daemon.

Initializing daemon...
Swarm listening on /ip4/10.5.8.247/tcp/4001
Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip6/::1/tcp/4001
API server listening on /ip4/127.0.0.1/tcp/5001
Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready
panic: runtime error: index out of range

goroutine 95 [running]:
panic(0xc00500, 0xc420010110)
        /home/whyrusleeping/go/src/runtime/panic.go:500 +0x1a1
github.com/ipfs/go-ipfs/merkledag.(*Node).getPBNode(0xc42502c4b0, 0xc42023ec88)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/merkledag/coding.go:61 +0x33f
github.com/ipfs/go-ipfs/merkledag.(*Node).Marshal(0xc42502c4b0, 0xc422c3e760, 0xc422c3e760, 0x11387e0, 0xc422c3e760, 0x2)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/merkledag/coding.go:45 +0x2f
github.com/ipfs/go-ipfs/merkledag.(*Node).EncodeProtobuf(0xc42502c4b0, 0x0, 0x3, 0x22, 0x22, 0x22, 0x30)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/merkledag/coding.go:80 +0xb3
github.com/ipfs/go-ipfs/merkledag.(*Node).Multihash(0xc42502c4b0, 0xc42023ed98, 0x449915, 0xc42023eda8)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/merkledag/node.go:271 +0x34
github.com/ipfs/go-ipfs/merkledag.(*Node).Key(0xc42502c4b0, 0xc420063020, 0x0)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/merkledag/node.go:249 +0x2b
github.com/ipfs/go-ipfs/exchange/bitswap.(*Bitswap).provideCollector(0xc420234a00, 0x113ae20, 0xc420130d80)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/exchange/bitswap/workers.go:138 +0x2c4
github.com/ipfs/go-ipfs/exchange/bitswap.(*Bitswap).startWorkers.func4(0x113f840, 0xc42019cde0)
        /home/whyrusleeping/gopkg/src/github.com/ipfs/go-ipfs/exchange/bitswap/workers.go:40 +0x3c
gx/ipfs/QmSF8fPo3jgVBAy8fpdjjYqgG87dkJgUprRBHRd2tmfgpP/goprocess.(*process).Go.func1(0xc4208fe980, 0xc42019cde0, 0xc42019cf00)
        /home/whyrusleeping/gopkg/src/gx/ipfs/QmSF8fPo3jgVBAy8fpdjjYqgG87dkJgUprRBHRd2tmfgpP/goprocess/impl-mutex.go:112 +0x3c
created by gx/ipfs/QmSF8fPo3jgVBAy8fpdjjYqgG87dkJgUprRBHRd2tmfgpP/goprocess.(*process).Go
        /home/whyrusleeping/gopkg/src/gx/ipfs/QmSF8fPo3jgVBAy8fpdjjYqgG87dkJgUprRBHRd2tmfgpP/goprocess/impl-mutex.go:115 +0x375
kf6nux pushed a commit to kf6nux/go-ipfs that referenced this issue Sep 28, 2016
resolves issue ipfs#3262
8753bf8 
It appears that the slice size of `n.Links` is being modified after the destination capacity of `pbn.Links` is set, thereby causing a panic when attempting to access an index out of range on the new slice.  By using append, we can avoid using mutexes and retain most of the original performance.
@whyrusleeping
Copy link
Member Author

Got this again in teamcity sharness tests:

Error: api not running
[19:31:38][Step 4/4] > cat daemon_err
[19:31:38][Step 4/4] panic: runtime error: index out of range
[19:31:38][Step 4/4] 
[19:31:38][Step 4/4] goroutine 102 [running]:
[19:31:38][Step 4/4] panic(0xbc95c0, 0xc420018130)
[19:31:38][Step 4/4]    /usr/local/go/src/runtime/panic.go:500 +0x1a1
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/merkledag.(*Node).getPBNode(0xc421eddc70, 0xc42027cc88)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/merkledag/coding.go:61 +0x33f
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/merkledag.(*Node).Marshal(0xc421eddc70, 0xc421ef19c0, 0xc421ef19c0, 0x10db480, 0xc421ef19c0, 0x2)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/merkledag/coding.go:45 +0x2f
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/merkledag.(*Node).EncodeProtobuf(0xc421eddc70, 0x0, 0x3, 0x22, 0x22, 0x22, 0x30)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/merkledag/coding.go:80 +0xb3
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/merkledag.(*Node).Multihash(0xc421eddc70, 0xc42027cd98, 0x449eb5, 0xc42027cda8)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/merkledag/node.go:271 +0x34
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/merkledag.(*Node).Key(0xc421eddc70, 0xc420263ce0, 0x0)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/merkledag/node.go:249 +0x2b
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/exchange/bitswap.(*Bitswap).provideCollector(0xc4201f0500, 0x10dda40, 0xc420011040)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/exchange/bitswap/workers.go:137 +0x2c4
[19:31:38][Step 4/4] github.com/ipfs/go-ipfs/exchange/bitswap.(*Bitswap).startWorkers.func4(0x10e2360, 0xc42023faa0)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/github.com/ipfs/go-ipfs/exchange/bitswap/workers.go:39 +0x3c
[19:31:38][Step 4/4] gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess.(*process).Go.func1(0xc42024ee80, 0xc42023faa0, 0xc42023fbc0)
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess/impl-mutex.go:112 +0x3c
[19:31:38][Step 4/4] created by gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess.(*process).Go
[19:31:38][Step 4/4]    /teamcity/work/3144e2a06f62fd48/src/gx/ipfs/QmQopLATEYMNg7dVqZRNDfeE2S1yKy8zrRh5xnYiuqeZBn/goprocess/impl-mutex.go:115 +0x375

@whyrusleeping whyrusleeping added the status/ready Ready to be worked label Sep 28, 2016
@whyrusleeping
Copy link
Member Author

Ah, found it. The issue here is that we're passing the block (which after the cid changes is a node still) down the newBlocks channel in bitswap, and the Add function returns. So now a bitswap worker has a handle to a reference to the node concurrently to the original caller having a reference to it. Going to fix by making the bitswap newBlocks channel be a channel of keys (soon to be cids)

@whyrusleeping
Copy link
Member Author

Related to #3246 and #3199 i beleive

@whyrusleeping whyrusleeping mentioned this issue Sep 29, 2016
Merged
@whyrusleeping whyrusleeping self-assigned this Sep 29, 2016
@whyrusleeping whyrusleeping added status/in-progress In progress and removed status/ready Ready to be worked labels Sep 29, 2016
@whyrusleeping whyrusleeping removed the status/in-progress In progress label Sep 30, 2016
@whyrusleeping whyrusleeping mentioned this issue Sep 30, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@whyrusleeping whyrusleeping

Labels
panic
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

only pass keys down newBlocks chan in bitswap ipfs/kubo
1 participant
@whyrusleeping