Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VAULT-11510 Vault Agent can start listeners without caching #18137

Merged
merged 11 commits into from
Dec 5, 2022

Conversation

VioletHynes
Copy link
Contributor

@VioletHynes VioletHynes commented Nov 29, 2022

This enables the usage of listeners and the API proxy even without the caching block. Everything works in both cases, and requests to authenticated endpoints (like token lookup) work in both.

I made the decision to make cache-clear return a 200 in the case that a cache does not exist, as I think that behaviour probably makes the most sense, but I'm welcome to being challenged on it.

I also updated a lot of the docs around this area, including a new page for the API Proxy.

There were also config items that were using "caching" to refer to the requests made by the API proxy. I allowed "proxying" as an alternative to make things clearer, hopefully for both us and customers. Similarly, I factored out the configuration items in the cache stanza that configured the API proxy to a new api_proxy stanza.

@VioletHynes
Copy link
Contributor Author

Oh, woops, there are merge conflicts. Looking to fix those.

@@ -36,7 +90,7 @@ func Handler(ctx context.Context, logger hclog.Logger, proxier Proxier, inmemSin
}

// Parse and reset body.
reqBody, err := ioutil.ReadAll(r.Body)
reqBody, err := io.ReadAll(r.Body)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ioutil.ReadAll is deprecated in favour of io.ReadAll

@@ -45,7 +99,7 @@ func Handler(ctx context.Context, logger hclog.Logger, proxier Proxier, inmemSin
if r.Body != nil {
r.Body.Close()
}
r.Body = ioutil.NopCloser(bytes.NewReader(reqBody))
r.Body = io.NopCloser(bytes.NewReader(reqBody))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ioutil.NopCloser is deprecated in favour of io.NopCloser

Copy link
Contributor

@peteski22 peteski22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, I added some random 'bits' just so you know I read it cover to cover 👍🏼

I think there's only 1 typo that the suggestion probably needs accepting for, I'll approve it anyway and you can deal with that rather than 'request changes' seems a bit drawn out for a tiny thing.

website/content/docs/agent/caching/index.mdx Show resolved Hide resolved
command/agent/config/config.go Outdated Show resolved Hide resolved
command/agent/cache/handler.go Outdated Show resolved Hide resolved
command/agent/cache/cache_test.go Outdated Show resolved Hide resolved
@peteski22
Copy link
Contributor

@finnstech: #18092 can be closed once this (18137) PR is merged.

@VioletHynes
Copy link
Contributor Author

After a discussion over Slack, I'm going to update this PR to rely upon a new HCL stanza, api_proxy, which contains the values from the cache stanza relating to api proxying (everything but persist). That way, these values can be supported without using a cache.

@VioletHynes
Copy link
Contributor Author

VioletHynes commented Nov 30, 2022

I just pushed an update to support the api_proxy stanza - I think most of the code is done, though I'm not sure yet (I'm going to run through a few more manual tests), though basic manual tests seem to work great.

I'm going to write a new docs page about the API Proxy in general, as we don't actually have one.

Lastly, it might be nice to refactor the code so that the handler and api_proxy stuff isn't in the cache package. Doing so naively would result in circular dependencies, so I might look at this as part of a separate PR, once this has been merged.

Apologies that the PR is getting a little chunky. I think the only thing that's left is the new docs page.

tl;dr: the PR isn't done yet, and I'll re-request review when it is.

}

// Create a muxer and add paths relevant for the lease cache layer and API proxy layer
mux := http.NewServeMux()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions have been moved over from the cache_test.go file, though the next ~25 lines of code were modified on the way.

@VioletHynes
Copy link
Contributor Author

Okay! I think we're good to go for review. Find the new doc page here: https://vault-hffnd61a7-hashicorp.vercel.app/vault/docs/agent/apiproxy

We now support the api_proxy stanza and note that the values in the cache stanza are deprecated (though they still work, for backwards compatibility). We error if both are provided. Everything I have tested manually seems to work, and all of our tests (that test the proxy and the cache) all seem to pass.

I think I've updated everywhere that needs to be updated, but let me know if I missed anything.

Copy link
Contributor

@peteski22 peteski22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR Violet, sorry the re-review took me longer than I wanted.

I spent a while going over the docs since I'm newer to Vault Agent and wanted to make sure I understood it.

I've made a few suggestions which I think we need (typo type stuff), some that I think make the docs clearer (to me) which you can take or leave, and the occasional rambling where I was trying to figure things out.

website/content/docs/agent/apiproxy.mdx Outdated Show resolved Hide resolved
website/content/docs/agent/index.mdx Outdated Show resolved Hide resolved
website/content/docs/agent/index.mdx Outdated Show resolved Hide resolved
website/content/docs/agent/index.mdx Outdated Show resolved Hide resolved
website/content/docs/agent/caching/index.mdx Show resolved Hide resolved
website/content/docs/agent/apiproxy.mdx Outdated Show resolved Hide resolved
website/content/docs/agent/apiproxy.mdx Outdated Show resolved Hide resolved
website/content/docs/agent/apiproxy.mdx Outdated Show resolved Hide resolved
command/agent.go Outdated Show resolved Hide resolved
command/agent.go Outdated Show resolved Hide resolved
@peteski22
Copy link
Contributor

Random, and not your code/docs etc. but any chance while you're in there you can fix this one thing that for some reason has been making my eye twitch: docs/agent#configuration:

These are the currently-available general configuration option:

to

These are currently the available general configuration options:

@peteski22 peteski22 self-requested a review December 5, 2022 13:39
@VioletHynes VioletHynes merged commit 672cdc0 into main Dec 5, 2022
@VioletHynes VioletHynes deleted the violethynes/VAULT-11510 branch December 5, 2022 15:51
AnPucel pushed a commit that referenced this pull request Jan 14, 2023
* VAULT-11510 Vault Agent can start listeners without caching

* VAULT-11510 fix order of imports

* VAULT-11510 changelog

* VAULT-11510 typo and better switch

* VAULT-11510 update name

* VAULT-11510 New api_proxy stanza to configure API proxy

* VAULT-11510 First pass at API Proxy docs

* VAULT-11510 nav data

* VAULT-11510 typo

* VAULT-11510 docs update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants