Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sync: Invalid justification provided #1147

Open
sandreim opened this issue Aug 25, 2023 · 3 comments
Open

sync: Invalid justification provided #1147

sandreim opened this issue Aug 25, 2023 · 3 comments
Labels
I2-bug The node fails to follow expected behavior.

Comments

@sandreim
Copy link
Contributor

We are getting a lot of this messages on master while load testing on Versi. This leads to a lot of peers getting disconnected 


  |   | 2023-08-25 12:33:07.399 | 2023-08-25 09:33:07.360  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWMuXCqgDJCoV2uBqdbreKg5XBwq3ij3i4QBsX6TFKnMQs for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.398 | 2023-08-25 09:33:07.358  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWPcVYiq5Ri22YBLaiHCk1Ueaduha24ZXQqLGJVQ8k8tHw for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.398 | 2023-08-25 09:33:07.352  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWGLox7NTcTSee7mKZPDEJwUakNdMPWmut4zna6XffR7PW for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.348 | 2023-08-25 09:33:07.326  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWFam2mpmteqG9QG7JCAyZt7NHq5Cs89d5uZ1UectHtage for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.202 | 2023-08-25 09:33:07.162  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWC3xyQ5xLU1NoQxFmw4T6RzehFyHjoKkPpS5Vva7yQoL6 for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.202 | 2023-08-25 09:33:07.159  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWMRSmcTjRpsLTv3mnJMjhq5bqG3gxqXQ7tfivsNCCdUWU for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.198 | 2023-08-25 09:33:07.153  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWBprMYJDaeLnC8TrEgDF9431GumUm9acnXwhPs4HN5ivW for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.198 | 2023-08-25 09:33:07.150  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWFnt83mfipo5VK78tCLMAmZJ5AeT6cKXNihPw3imakLeV for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.155 | 2023-08-25 09:33:07.108  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWC3xyQ5xLU1NoQxFmw4T6RzehFyHjoKkPpS5Vva7yQoL6 for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.148 | 2023-08-25 09:33:07.146  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWDuKxh1CQoiWQfhd2B96NxPfrrvHUidTfRfrEz4DnTwfo for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.148 | 2023-08-25 09:33:07.142  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWNvqbzQzEyRgznkSwrTxLhfzbjVFptjxymJk6cMaw5CcK for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.148 | 2023-08-25 09:33:07.139  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWMSshZvcWUSnK5EqPf8TBahXZg8Pk7Sf4FqVS7TqSD7Yu for #0x1f43…0b00 |  
  |   | 2023-08-25 12:33:07.148 | 2023-08-25 09:33:07.135  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWJNFCzuys84GThP6sDv91rjMhho1NY1mM9cf9swcqSo9N for #0x

https://grafana.teleport.parity.io/goto/e3ndQURIR?orgId=1
@alexggh
Copy link
Contributor

alexggh commented Aug 25, 2023 •
edited
Loading

Adding a bit more context after a fresh reset we don't see the problem, but once we change the image in the helm-chart and the image starts rolling out the errors start showing up and produce a lot of disconnects of peers.

2023-08-25 09:23:06.240  INFO tokio-runtime-worker sync: 
💔 Invalid justification provided by 12D3KooWKkLBwmdFWeoBEfPsYTG7xH5cHTcvQNC4GrCsgTnpvt1b for #0x5021…2f21    

2023-08-25 09:23:06.317 DEBUG tokio-runtime-worker sync::import-queue: 
Justification import failed for hash = 0x50218bb98076a0f1387a91cf0289c98f4b5b9b7b85702ab3ee1693443e9b2f21 with number = 20915 coming from node = PeerId("12D3KooWDVthfKEDq8cPY6Az7FActkSkvcpacxfEzo1QAZTTXtus")
 with error: Import failed: bad justification for header: invalid signature for precommit in grandpa justification

@the-right-joyce the-right-joyce added I2-bug The node fails to follow expected behavior. and removed I3-bug labels Aug 25, 2023
@alexggh
Copy link
Contributor

alexggh commented Aug 25, 2023

A few more details, it seems the problem happens when a new Era occurs, in versi that is configured to happen every 2 minutes, hence it was heavily affecting the network, we can bypass it in Versi by configuring the staking pallet to not change the era, but I still think there is a small bug in there that happens at era changes.

@alexggh alexggh mentioned this issue Aug 29, 2023
Closed
@alexggh
Copy link
Contributor

alexggh commented Jul 31, 2024 •
edited
Loading

This seems to have happened on kusama as well at session changed,

 2024-07-31 15:28:15.061  INFO tokio-runtime-worker sync: 💔 Invalid justification provided by 12D3KooWACUhxW2AXSotDLH3JvhxELwEqUPzLhWzw3eSHm3Mi8Jd for #0xff3d…c43c

see: https://grafana.teleport.parity.io/goto/d6l40I9IR?orgId=1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
I2-bug The node fails to follow expected behavior.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@alexggh @sandreim @the-right-joyce