Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Khalil/1895 Add config package unit test #4523

Merged
merged 21 commits into from
Jul 6, 2023

Conversation

kc1116
Copy link
Contributor

@kc1116 kc1116 commented Jun 27, 2023

This PR adds some unit test coverage to the new config package. It also adds a README file with details about the package and instructions on how to add a new configuration value.

@codecov-commenter
Copy link

codecov-commenter commented Jun 27, 2023

Codecov Report

Merging #4523 (78716a4) into master (12f898b) will increase coverage by 3.65%.
The diff coverage is 71.77%.

@@            Coverage Diff             @@
##           master    #4523      +/-   ##
==========================================
+ Coverage   54.09%   57.74%   +3.65%     
==========================================
  Files         563      511      -52     
  Lines       56220    47124    -9096     
==========================================
- Hits        30414    27214    -3200     
+ Misses      23459    18072    -5387     
+ Partials     2347     1838     -509     
Flag Coverage Δ
unittests 57.74% <71.77%> (+3.65%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
engine/access/rest/middleware/metrics.go 0.00% <0.00%> (ø)
engine/execution/state/state.go 35.38% <ø> (ø)
fvm/crypto/hash.go 100.00% <ø> (ø)
module/buffer/backend.go 87.01% <ø> (ø)
module/buffer/pending_blocks.go 0.00% <ø> (ø)
module/buffer/pending_cluster_blocks.go 0.00% <ø> (ø)
module/chunks/chunk_assigner.go 58.62% <ø> (ø)
module/mempool/herocache/backdata/heropool/pool.go 89.79% <ø> (ø)
module/mempool/queue/queue.go 96.03% <ø> (ø)
module/metrics/execution.go 0.00% <ø> (ø)
... and 59 more

... and 398 files with indirect coverage changes

@kc1116 kc1116 requested a review from peterargue June 28, 2023 16:51
Copy link
Contributor

@peterargue peterargue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I added a few comments mostly to help break up the text and highlight key words. Feel free to take or leave them.

config/config.go Outdated
Comment on lines 250 to 252
func init() {
initialize()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can you put this at the top of the file? It's easier to find that way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config/network/config_test.go Outdated Show resolved Hide resolved
config/README.md Outdated Show resolved Hide resolved
config/README.md Outdated Show resolved Hide resolved
config/README.md Outdated Show resolved Hide resolved
config/README.md Outdated Show resolved Hide resolved
config/README.md Outdated Show resolved Hide resolved
config/README.md Outdated Show resolved Hide resolved
Copy link
Contributor

@yhassanzadeh13 yhassanzadeh13 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

Comment on lines 124 to 131
func defaultConfig(t *testing.T) *FlowConfig {
initialize()
c, err := DefaultConfig()
require.NoError(t, err)
require.Equalf(t, "./default-config.yml", c.ConfigFile, "expected default config file to be used")
require.NoErrorf(t, c.Validate(), "unexpected error encountered validating default config")
return c
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: For enhanced reusability and encapsulation, it would be prudent to relocate this to the unittest package as an exportable test utility function. Additionally, please substitute all instances of config.DefaultConfig() within the test with unittest.DefaultConfig(t). Thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config/README.md Outdated Show resolved Hide resolved
config/README.md Outdated
The entire default config can be overridden using the `--config-file` CLI flag. When set the config package will attempt to parse the specified config file and override all the values
defined. A single default value can be overridden by setting the CLI flag for that specific config. For example `--network-connection-pruning=false` will override the default network connection pruning
config to false.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on this documentation, a comprehensive example might resemble the following command. If this aligns with the intended usage, please incorporate it into the readme. Alternatively, if there are any deviations, kindly add a complete, workable example. Users frequently find it more straightforward to emulate a practical example than to extract one from the documentation.

<rest of CLI command> --config-file=config/config.yml --network-connection-pruning=false

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config/README.md Outdated
### Adding a new config value
Adding a new config to the FlowConfig can be done in a few easy steps.

1. Create a new subpackage for the config value. This package will define the configuration structs and CLI flags for overriding.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the recommended package where the user can define this subpackage to. For example, are we encouraging the users to add the sub-packages to /config package?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2. `mapstructure` - mapstructure tag is used for unmarshalling and must match the CLI flag name defined in step or else the field will not be set when the config is unmarshalled.
```go
type MyComponentConfig struct {
AppWorkers int `validate:"gt=0" mapstructure:"app-workers"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the above example, it is important to elucidate how the configuration is processed, for example, focusing on the AppWorkers field. The AppWorkers field is populated with the value corresponding to app-workers from the configuration file. It's essential to note that there is a validation check in place to ensure that this value is greater than zero. This validation is critical because a non-positive value for the number of workers would not make sense and could cause runtime issues. In case the validation fails, meaning if the value is set to zero or a negative number, the code is expected to handle this scenario appropriately. Typically, this would involve throwing an error or an exception to indicate that an invalid configuration value was provided. Please document what would be such expected behavior.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

config/README.md Outdated Show resolved Hide resolved
kc1116 and others added 18 commits July 6, 2023 01:36
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Peter Argue <89119817+peterargue@users.noreply.github.com>
Co-authored-by: Yahya Hassanzadeh, Ph.D. <yhassanzadeh@ieee.org>
Co-authored-by: Yahya Hassanzadeh, Ph.D. <yhassanzadeh@ieee.org>
…w/flow-go into khalil/1895-config-system-unittest
@kc1116 kc1116 merged commit b94310b into master Jul 6, 2023
33 checks passed
@kc1116 kc1116 deleted the khalil/1895-config-system-unittest branch July 6, 2023 18:31
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants