Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement cache capabilities for CI Pipelines #74

Closed
SergK opened this issue Nov 20, 2023 · 0 comments
Closed

Implement cache capabilities for CI Pipelines #74

SergK opened this issue Nov 20, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@SergK
Copy link
Member

SergK commented Nov 20, 2023

We must ensure the build cache is updated to speed up CI pipelines. At the same time, the implementation should be cloud-agnostic and doesn't depend on the specific platform. We can go two ways here:

For this task, we are going to implement the second approach.
We have to avoid a lot of volume mounts since there is a limit to the number of mounted volumes per instance (for example, in EKS). Another point: Tekton doesn't allow mounting two PVC volumes to a task without disabling affinity assistance, see tektoncd/catalog#1187

So, the idea behind this is to use a single volume with a subpath. See example here

/ws/cache    <--- use for cache, either NPM, go, maven
   /source   <--- use to checkout
@SergK SergK added the enhancement New feature or request label Nov 20, 2023
epmd-edp pushed a commit that referenced this issue Nov 22, 2023
* Add two tasks: get-cache, save-cache
* Merge 'build' and 'test' steps into single 'build' step for golang
* Run get-cache with root provileges (should be fixed later)
* Use single PVC with the following structure:
   /codebase/cache
            /source
* Checkout to /source subPath for all git servers
* Update all pipelines to work with code in source subPath

We need to add cache steps to all pipelines and align general
steps like 'helm-docs', 'dockerbuild-verify', etc.

Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
Change-Id: I033f3d15f7855ec0d9c3947ce513cc110509ba06
epmd-edp pushed a commit that referenced this issue Nov 28, 2023
JIRA: EPMDEDP-12953

Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
Change-Id: I43a8e2fae738f4a182cca1e088366a117afb8e60
epmd-edp pushed a commit that referenced this issue Nov 28, 2023
We use /subdirectory param for the 'fetch' step so we don't need
to mount workspace with subPath

JIRA: EPMDEDP-12953

Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
Change-Id: I8f90132d464a6bdf054d8a5a7b16ce9373f03f20
epmd-edp pushed a commit that referenced this issue Nov 28, 2023
Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
Change-Id: Ib56ad1b09b1d7b48aaf721206efcd7d8408f0b6a
epmd-edp pushed a commit that referenced this issue Nov 29, 2023
Signed-off-by: Sergiy Kulanov <sergiy_kulanov@epam.com>
Change-Id: Id82a3d914064a337306ec987e081ad96925563d4
epmd-edp pushed a commit that referenced this issue Dec 1, 2023
* Update url to cache server
* Add space parameter to name definition

Related #74

Change-Id: I365e0d827a586d9f8a875da55355bd82eebc047d
epmd-edp pushed a commit that referenced this issue Dec 1, 2023
before:

   init-values -> compile -> test -> sonar -> build -> push

after:

     get-cache -> compile -> test -> sonar -> build -> save-cache
              /                                       \
init-values -                                          -> push

Change-Id: Ie8b1bb33a17d26f33e7dd16482747d7c79f55e8b
epmd-edp pushed a commit that referenced this issue Dec 5, 2023
* Update and rename python task
* Update source dir in python task
* Add cache tasks to python pipelines

Related #74

Change-Id: If5f620dda2bdb1928a945704353c3d7c0a8ab595
epmd-edp pushed a commit that referenced this issue Dec 5, 2023
Change-Id: I89050f28af7f086ad89e999472626cbabb7ca8d4
epmd-edp pushed a commit that referenced this issue Dec 6, 2023
Change-Id: I5ae92e9d711815d3dbbde113a894a5fd338061f7
epmd-edp pushed a commit that referenced this issue Dec 7, 2023
Change-Id: I6f5def4824f1d7f849b45a66cea05d24144f7e3f
epmd-edp pushed a commit that referenced this issue Dec 7, 2023
Change-Id: I193c3c38ea1aed6bec33604c630fc80e405b9f7a
epmd-edp pushed a commit that referenced this issue Dec 8, 2023
Change-Id: Ib926f15567837f08f8c123bf3b93777fc0b1f2ef
epmd-edp pushed a commit that referenced this issue Dec 8, 2023
* Fix maven sonar path
* Refactor autotests pipelines

Change-Id: Id0a26d1baecde640866ee15d366d1df83d7d627e
epmd-edp pushed a commit that referenced this issue Dec 8, 2023
Change-Id: I0da5a86b59e3808ae08c1b218277b30b6f3a1327
epmd-edp pushed a commit that referenced this issue Dec 8, 2023
Change-Id: I40ae0df1b15927b41455ccb31e5e5c05b4847add
epmd-edp pushed a commit that referenced this issue Dec 8, 2023
Change-Id: Ia606da0fe7c2b077f0469e251e5b48f21a9185bb
epmd-edp pushed a commit that referenced this issue Dec 11, 2023
Jira: EPMDEDP-12931

Related: #74
Change-Id: If3c9463cb54113a438acea78e58ee659050c8d77
epmd-edp pushed a commit that referenced this issue Dec 11, 2023
Change-Id: I955f8e6a266a74835b91714ed65aeea9368424cb
epmd-edp pushed a commit that referenced this issue Dec 15, 2023
* Make possible run npm pipelines on OKD cluster

Change-Id: Ie8766f93384930fa37383ea9e794c7648271819a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

3 participants