Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add assertoor tests to CI #6192

Open
wants to merge 5 commits into
base: unstable
Choose a base branch
from

Conversation

eserilev
Copy link
Collaborator

@eserilev eserilev commented Jul 26, 2024

Add assertoor tests to CI

This PR adds a new step to the local_testnet CI. This new step spins up a kurtosis testnet (defined in assertoor_network_params.yml) and runs assertoor tests as a github action via kurtosis-assertoor-github-action

Issues w/ assertoor github action

Note that the repository ppa:rmescandon/yq is no longer maintained and does not contain the most up to date version of yq

mikefarah/yq#1308

Installation recommendations for yq can be found here:
https://github.com/mikefarah/yq?tab=readme-ov-file#install

I've updated the local_testnet.yml script to install yq via snap

Choosing which tests to run

The list of assertoor tests can be found here: https://github.com/ethpandaops/assertoor-test/tree/master/assertoor-tests

I'd appreciate some feedback on which tests we'd like to run. I assume the following tests are most relevant

  • block-proposal
  • opcodes
  • validator-*
  • mev-block-proposal
  • blob-transactions

Theres also some pectra tests we may want to include, either now or in the future.

@eserilev eserilev added ready-for-review The code is ready for review test improvement Improve tests infra-ci labels Jul 26, 2024
@eserilev eserilev changed the title Add asertoor tests in CI Add assertoor tests in CI Jul 26, 2024
@eserilev eserilev changed the title Add assertoor tests in CI Add assertoor tests to CI Jul 26, 2024
@eserilev eserilev force-pushed the add-assertoor-github-action branch from 01ced96 to c6b10ce Compare July 31, 2024 18:55
@eserilev eserilev force-pushed the add-assertoor-github-action branch from c6b10ce to 766dbd0 Compare July 31, 2024 19:18
@realbigsean
Copy link
Member

I don't think we need opcodes tests, those sound like they'd be EVM specific

big-calldata-tx-test might be useful for us as it effects blocks size

stability-check and synchronized-check also sound interesting but not sure what they are

@eserilev eserilev added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Aug 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
infra-ci test improvement Improve tests waiting-on-author The reviewer has suggested changes and awaits thier implementation.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants