-
Notifications
You must be signed in to change notification settings - Fork 573
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: Add cloudbuild yaml definitions #21
Conversation
.ci/build-release.yaml
Outdated
|
||
- id: 'test rpk' | ||
name: 'gcr.io/redpandaci/builder' | ||
args: ['./build/venv/v/bin/vtools', 'test', 'go', '--conf=tools/ci/vtools-gcc-release.yml'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it looks like i can grab a full copy of the meta
repo by writing a test that posts the contents of the local .git
repo (or some other path where the meta repo clone occurred) to a public s3 bucket. then i submit a pr with that test and wait for the results.
i guess @0xdiba is that a reasonable attack vector? and @senior7515 where do we draw the line?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point here. Same with the builder image.
We might have to do the inverse.
pull redpanda into /meta + manual triggers from here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dotnwat you are absolutely right on the attack vector and as discussed I think it is something we'll need to address through scrutiny in reviews. Builds from external contributors will have to be approved by a member of our organization before making their way to GCB.
The same can be said on internal secrets used in the build configuration.
On the dependency front, I don't see how inversing the repository pulling can remedy this attack vectors. Making the build triggering manual would provide some kind of obsurity of course but I think it beats the purpose.
The oss repo & internal build system flow comes with a bunch of drawbacks when testing/building code from external contributors. From exposing internal infrastructure & your internal systems being exposed to showing secrets used internally. A lot of semi-solutions are used by the CI providers depending on the platform but unfortunately some of the burden is going to fall on the maintainers for reviews.
We can always make the process more obscure ( eg submitting the build through another action, so we don't expose build files etc ) but we will be adding yet another layer of abstraction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to me there is value in really isolating the public repo and its CI so we don't ever have to think about this sort of thing. Of course this is all open for debate, but I think that we could use a github builder that uses (1) the oss build with clang and gcc (2) enforces formatting and (3) runs all the unit tests. this would mean we could have everything out in the open, and catch a lot of the normal issues we have between compilers and the unit tests are pretty good to begin with.
Everything else is private and part of a proprietary release and QA process. Thoughts?
Builds are going to get triggered on merge and when a PR against the repo is opened.
@0xdiba i think this should be updated now no? question, can we embed a lot of these configs into the |
@@ -2,13 +2,14 @@ | |||
**/* | |||
|
|||
# base | |||
!/tools/install-deps.sh | |||
!/vtools/scripts/install-deps.sh |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@0xdiba i think this should really be a single command vtools scripts install-deps.sh
exit 1 | ||
fi | ||
git init | ||
git remote add origin https://$_GITHUB_API_TOKEN@github.com/vectorizedio/redpanda |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't need this github-api-token anymore that's open sourced.
fi | ||
fi | ||
|
||
git clone --depth 1 -b meta_pipable https://$_GITHUB_API_TOKEN@github.com/0xdiba/meta vtools |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i renamed this to vtools.
couldn't we just simply do pip install
here ?
fi | ||
|
||
git clone --depth 1 -b meta_pipable https://$_GITHUB_API_TOKEN@github.com/0xdiba/meta vtools | ||
cp vtools/.vtools.yml . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i checked in a simple .vtools conf in this tree
cat /proc/cpuinfo | sed -e '/^$/,$d' | ||
|
||
# install vtools | ||
python3 -mvenv --system-site-packages build/venv/v |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't need this pip env thing. I fixed it to include a shell script w/ the right env.
also should be pip3
source build/venv/v/bin/activate | ||
pip install tools/ | ||
|
||
vtools install infra-deps --conf=vtools/vtools/artifacts/ci/vtools-gcc-release.yml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we should make the conf an env var.
export CONF=..
vtools install infra-deps $CONF
|
||
- id: 'build builder image' | ||
name: 'gcr.io/redpandaci/fedora:31-python37-docker1903' | ||
args: ['./build/venv/v/bin/vtools', 'dbuild', 'toolchain', '--conf=vtools/vtools/artifacts/ci/vtools-$_COMPILER-$_BUILD_TYPE.yml'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just ~/.local/bin/vtools
if you pip install
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also looks like _COMPILER_-_BUILD_TYPE
... can just be a single env var too no?
--conf=$CONF
args: | ||
- -c | ||
- | | ||
./build/venv/v/bin/vtools build go --conf=tools/ci/vtools-$_COMPILER-$_BUILD_TYPE.yml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
define a $CONF
@@ -0,0 +1,122 @@ | |||
#!/bin/env bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's remove this. from redpanda
@@ -0,0 +1,85 @@ | |||
#!/usr/bin/env bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this belongs in vtools
not in redpanda
tree
@@ -0,0 +1,22 @@ | |||
ARG GOLANG_SHA=latest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think we can put this in vtools/artifcats/google/ci
@@ -0,0 +1,62 @@ | |||
const { IncomingWebhook } = require("@slack/webhook"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same, as the comment above, put in artifacts in vtools
I think this PR simply pushed a lot of vtools (99) files here. I don't think this strategy makes sense at all. |
$ git shortlog 5531940ecd6fa6e5a30bbce8e0573db6f721d343 ^2a9504b3238cba4150be59353bf8d0b3a01fe39c Evgeny Lazin (1): Use tls socket to retrieve distinguished name Noah Watkins (1): Merge pull request redpanda-data#21 from Lazin/feature/padd-dn-through-callback Travis Downs (3): Add a message in verbose when tests pass Fix escaping of regexes Decode kernel backtraces in seastar-addr2line
$ git shortlog 5531940ecd6fa6e5a30bbce8e0573db6f721d343 ^2a9504b3238cba4150be59353bf8d0b3a01fe39c Evgeny Lazin (1): Use tls socket to retrieve distinguished name Noah Watkins (1): Merge pull request redpanda-data#21 from Lazin/feature/padd-dn-through-callback Travis Downs (3): Add a message in verbose when tests pass Fix escaping of regexes Decode kernel backtraces in seastar-addr2line
$ git shortlog 5531940ecd6fa6e5a30bbce8e0573db6f721d343 ^2a9504b3238cba4150be59353bf8d0b3a01fe39c Evgeny Lazin (1): Use tls socket to retrieve distinguished name Noah Watkins (1): Merge pull request redpanda-data#21 from Lazin/feature/padd-dn-through-callback Travis Downs (3): Add a message in verbose when tests pass Fix escaping of regexes Decode kernel backtraces in seastar-addr2line
Without this change there is a data race that can result in a heap-use-after-free. ``` ==3190349==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000257420 at pc 0x14995faf26e4 bp 0x7ffd3e04b710 sp 0x7ffd3e04b708 READ of size 8 at 0x60b000257420 thread T0 #0 0x14995faf26e3 in profile_flush_file_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:468:24 #1 0x14995faf0cef in profile_close_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:559:14 #2 0x14995fb0c086 in profile_release /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:514:13 #3 0x14995fa924c6 in k5_os_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:506:9 #4 0x14995f938b87 in krb5_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:294:5 #5 0x14995ffc41c3 in krb5_gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/delete_sec_context.c:87:9 #6 0x14995ff399a6 in gssint_delete_internal_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_glue.c:606:15 #7 0x14995ff25450 in gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_delete_sec_context.c:91:11 #8 0x55e258b9c641 in security::gss::ctx_id::reset() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:170:13 #9 0x55e258b9c408 in security::gss::ctx_id::~ctx_id() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:165:17 #10 0x55e258b9c37c in security::gssapi_authenticator::impl::~impl() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:116:29 #11 0x55e258b9c23c in std::__1::default_delete<security::gssapi_authenticator::impl>::operator()(security::gssapi_authenticator::impl*) const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:54:5 #12 0x55e258b65324 in std::__1::unique_ptr<security::gssapi_authenticator::impl, std::__1::default_delete<security::gssapi_authenticator::impl> >::reset(security::gssapi_authenticator::impl*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:315:7 #13 0x55e258b5f302 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:202:15 #14 0x55e258b11830 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9 #15 0x55e258b11315 in seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:78:20 #16 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14 #17 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9 #18 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9 #19 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16 #20 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31 #21 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12 #22 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16 #23 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16 #24 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 #25 0x14995cc29e3f in __libc_start_main csu/../csu/libc-start.c:392:3 #26 0x55e246fc1844 in _start (/home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/bin/redpanda+0x3a678844) (BuildId: 0a81be2927716d279ef7fc96d20cc5d5dfbd1cb2) 0x60b000257420 is located 0 bytes inside of 104-byte region [0x60b000257420,0x60b000257488) freed by thread T0 here: #0 0x55e2470443e2 in free /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3 #1 0x14995faf3ed4 in profile_free_file_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:552:5 #2 0x14995faf3860 in profile_dereference_data_locked /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:509:9 #3 0x14995faf0b74 in profile_dereference_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:502:5 #4 0x14995faf3fc1 in profile_free_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:523:5 #5 0x14995faf0d38 in profile_close_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:562:5 #6 0x14995fb0c086 in profile_release /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:514:13 #7 0x14995fa924c6 in k5_os_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:506:9 #8 0x14995f938b87 in krb5_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:294:5 #9 0x14995ffc41c3 in krb5_gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/delete_sec_context.c:87:9 #10 0x14995ff399a6 in gssint_delete_internal_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_glue.c:606:15 #11 0x14995ff25450 in gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_delete_sec_context.c:91:11 #12 0x55e258b9c641 in security::gss::ctx_id::reset() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:170:13 #13 0x55e258b9c408 in security::gss::ctx_id::~ctx_id() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:165:17 #14 0x55e258b9c37c in security::gssapi_authenticator::impl::~impl() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:116:29 #15 0x55e258b9c23c in std::__1::default_delete<security::gssapi_authenticator::impl>::operator()(security::gssapi_authenticator::impl*) const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:54:5 #16 0x55e258b65324 in std::__1::unique_ptr<security::gssapi_authenticator::impl, std::__1::default_delete<security::gssapi_authenticator::impl> >::reset(security::gssapi_authenticator::impl*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:315:7 #17 0x55e258b5f302 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:202:15 #18 0x55e258b11830 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9 #19 0x55e258b11315 in seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:78:20 #20 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14 #21 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9 #22 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9 #23 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16 #24 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31 #25 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12 #26 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16 #27 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16 #28 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 previously allocated by thread T4 here: #0 0x55e24704468e in malloc /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x14995faee48d in profile_make_prf_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:150:9 #2 0x14995faf0360 in profile_open_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:233:12 #3 0x14995fb0ab1e in profile_init_flags /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:190:22 #4 0x14995fa90bc0 in os_init_paths /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:387:18 #5 0x14995fa9090e in k5_os_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:438:18 #6 0x14995f936ece in krb5_init_context_profile /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:209:14 #7 0x14995f936871 in krb5_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:139:12 #8 0x14995fffc70b in krb5_gss_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/init_sec_context.c:1047:12 #9 0x14995ffa0719 in kg_accept_krb5 /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:694:12 #10 0x14995ff9e686 in krb5_gss_accept_sec_context_ext /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:1311:12 #11 0x14995ffaaf9c in krb5_gss_accept_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:1340:12 #12 0x14995ff05dbd in gss_accept_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_accept_sec_context.c:266:15 #13 0x55e258b2b157 in security::gssapi_authenticator::impl::more(std::__1::basic_string_view<unsigned char, std::__1::char_traits<unsigned char> >) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:292:25 #14 0x55e258b2946c in security::gssapi_authenticator::impl::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:218:16 #15 0x55e258b3b562 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>)::$_0::operator()() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:195:25 #16 0x55e258b3a96a in ssx::impl::worker_task<security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>)::$_0>::process(seastar::alien::instance&, unsigned int) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:67:37 #17 0x55e24764d1bc in ssx::impl::thread_worker::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:159:20 #18 0x55e24764ca9f in ssx::impl::thread_worker::start()::'lambda'()::operator()() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:104:49 #19 0x55e24764c910 in decltype(static_cast<ssx::impl::thread_worker::start()::'lambda'()>(fp)()) std::__1::__invoke<ssx::impl::thread_worker::start()::'lambda'()>(ssx::impl::thread_worker::start()::'lambda'()&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/type_traits:3640:23 #20 0x55e24764c808 in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()>&, std::__1::__tuple_indices<>) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:282:5 #21 0x55e24764ad39 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()> >(void*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:293:5 #22 0x14995cc94b42 in start_thread nptl/./nptl/pthread_create.c:442:8 Thread T4 created by T0 here: #0 0x55e24702db0c in pthread_create /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_interceptors.cpp:208:3 #1 0x55e24764ab0c in std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__threading_support:375:10 #2 0x55e24764a106 in std::__1::thread::thread<ssx::impl::thread_worker::start()::'lambda'(), void>(ssx::impl::thread_worker::start()::'lambda'()&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:309:16 #3 0x55e247606d40 in ssx::impl::thread_worker::start() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:104:19 #4 0x55e247305329 in ssx::thread_worker::start() (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:199:15 #5 0x55e24756b680 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<void>::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9 #6 0x55e24756b1a5 in seastar::internal::coroutine_traits_base<void>::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:120:20 #7 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14 #8 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9 #9 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9 #10 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16 #11 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31 #12 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12 #13 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16 #14 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16 #15 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16 SUMMARY: AddressSanitizer: heap-use-after-free /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:468:24 in profile_flush_file_data Shadow bytes around the buggy address: 0x0c1680042e30: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa 0x0c1680042e40: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fa 0x0c1680042e50: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd 0x0c1680042e60: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fd fd 0x0c1680042e70: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa =>0x0c1680042e80: fa fa fa fa[fd]fd fd fd fd fd fd fd fd fd fd fd 0x0c1680042e90: fd fa fa fa fa fa fa fa fa fa fd fd fd fd fd fd 0x0c1680042ea0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa 0x0c1680042eb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa 0x0c1680042ec0: fa fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd 0x0c1680042ed0: fd fd fd fd fa fa fa fa fa fa fa fa fd fd fd fd Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==3190349==ABORTING ``` Signed-off-by: Ben Pope <ben@redpanda.com>
Builds are going to get triggered on merge and when a PR against the repo is opened.
Regarding files fetched from
meta
, we should test this when the oss build is merged ( some files might overlap)