Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: etcd sync data checker should work #11457

Merged
merged 2 commits into from
Aug 2, 2024

Conversation

ronething
Copy link
Contributor

@ronething ronething commented Aug 1, 2024

Signed-off-by: ashing axingfly@gmail.com

Description

Fixes #11216

fix part 2 of this issue

Background

the load_full_data function in the file config_etcd.lua(load all exist data from etcd when apisix start), checker does not depend on item_schema to have a value

if data_valid and self.checker then
data_valid, err = self.checker(item.value)
if not data_valid then
log.error("failed to check item data of [", self.key,
"] err:", err, " ,val: ", json.delay_encode(item.value))
end
end

But in the sync_data function (watch new events from etcd when apisix running), checker relies on item_schema should have value.

if data_valid and res.value and self.item_schema then

As a result, the plugin metadata resource cannot reach the checker logic when etcd watch.

apisix/apisix/plugin.lua

Lines 816 to 821 in 507df12

local plugin_metadatas, err = core.config.new("/plugin_metadata",
{
automatic = true,
checker = check_plugin_metadata
}
)

So we need to fix it.

about the test case:

The admin api is not used for the operation, because the admin api will directly fail to verify the schema and cannot go to the etcd checker logic.

error-log-logger-clickhouse.t: The request header sent to clickhouse should be decrypted to be correct.
There was a problem with the previous modification of this PR
https://github.com/apache/apisix/pull/11076/files#diff-2974594095136be865db5d415f3da92d2a435211e30f93f693f1ed46a4a90037

PS: This PR is a partial fork of #11430.

about failed CI

My PR changes have nothing to do with this CI error. The reason for the error is a problem with the image version used by the github runner.

image

ref: https://github.com/apache/apisix/actions/runs/10193727573/job/28202213850?pr=11457

  • The correct github runner image version is as follows,
image

ref: https://github.com/apache/apisix/actions/runs/10193727573/job/28202215238?pr=11457

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have updated the documentation to reflect this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

Signed-off-by: ashing <axingfly@gmail.com>
Signed-off-by: ashing <axingfly@gmail.com>
@shreemaan-abhishek
Copy link
Contributor

I am confused about one thing, in this line:

if data_valid and self.item_schema then

load_full_data validates the item_schema is not falsy.

Then why do you say this:

checker does not depend on item_schema to have a value

@ronething
Copy link
Contributor Author

checker does not depend on item_schema to have a value

@shreemaan-abhishek What I mean is that the execution of the checker function does not depend on the item_schema having a value.

ref:

if data_valid and self.checker then
data_valid, err = self.checker(item.value)

@moonming
Copy link
Member

moonming commented Aug 1, 2024

@ronething I rerun test cases and passed.

Copy link
Contributor

@shreemaan-abhishek shreemaan-abhishek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ronething
Copy link
Contributor Author

@moonming can you merge it, thanks.

@moonming moonming merged commit 8728863 into apache:master Aug 2, 2024
37 checks passed
@ronething ronething deleted the fix/etcd_checker branch August 2, 2024 03:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: the plugin metadata obtained from etcd has not been validated by schema
4 participants