Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jib integration #5623

Merged
merged 34 commits into from
Mar 13, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
2482fe3
JIB integration
eddumelendez Aug 1, 2022
b1827a1
Update
eddumelendez Aug 1, 2022
2798200
Move to core and use jackson 2.10.0 required by jib-core
eddumelendez Aug 1, 2022
8c172a6
Add test with DockerDaemonImage
eddumelendez Sep 5, 2022
ef68af5
Remove spi, add singleton instance and JibImage
eddumelendez Sep 6, 2022
dba6a21
Move from api to implementation
eddumelendez Sep 6, 2022
8a5a671
Add docs
eddumelendez Sep 6, 2022
5d43c4c
Restore jackson version to 2.8.8
eddumelendez Sep 6, 2022
468e1e9
Merge branch 'main' into jib
eddumelendez Oct 5, 2022
4d7eb0b
Use new loadImageAsyncCmd and use pipes
eddumelendez Oct 5, 2022
2689f81
Merge branch 'main' into jib
eddumelendez Oct 15, 2022
39b6f0b
Add link to jib-core
eddumelendez Oct 17, 2022
aa7edc6
Merge branch 'main' into jib
eddumelendez Nov 23, 2022
0f8036f
Update docker-java version to 3.2.14
eddumelendez Nov 23, 2022
b218d4f
Update docs/features/jib.md
eddumelendez Dec 1, 2022
7691f8d
Update docs/features/jib.md
eddumelendez Dec 1, 2022
eaad8d9
Update build.gradle
eddumelendez Dec 1, 2022
e1cd9f7
Update docs/features/jib.md
eddumelendez Dec 1, 2022
6f06dc5
Merge branch 'main' into jib
eddumelendez Dec 1, 2022
6f286a7
Add unstable annotation and improve docs
eddumelendez Dec 1, 2022
6e28331
Merge branch 'main' into jib
eddumelendez Mar 3, 2023
75566bf
Merge branch 'main' into jib
eddumelendez Mar 6, 2023
0a88f0f
Enhance Jib integration
eddumelendez Mar 6, 2023
da454ec
Add default labels to drop images automatically
eddumelendez Mar 7, 2023
3ea5dff
Add proper tests for labels
eddumelendez Mar 7, 2023
938f6db
Polish test
eddumelendez Mar 7, 2023
25f6806
Fix checkstyle
eddumelendez Mar 7, 2023
97df73f
Update docs/features/jib.md
eddumelendez Mar 10, 2023
082e70a
Removed class
eddumelendez Mar 10, 2023
0c47470
Merge branch 'main' into jib
eddumelendez Mar 10, 2023
ace1635
Add comments
eddumelendez Mar 10, 2023
b29b327
Polish
eddumelendez Mar 13, 2023
7c9f18c
Merge branch 'main' into jib
eddumelendez Mar 13, 2023
c96b77f
Move to jib package
eddumelendez Mar 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions core/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ configurations.all {
resolutionStrategy {
// use lower Jackson version
force 'com.fasterxml.jackson.core:jackson-databind:2.8.8'
force 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.8.8'
}
}

Expand All @@ -75,6 +76,8 @@ dependencies {
exclude(group: 'org.jetbrains', module: 'annotations')
}

provided 'com.google.cloud.tools:jib-core:0.22.0'

shaded 'org.awaitility:awaitility:4.2.0'

api platform('com.github.docker-java:docker-java-bom:3.3.0')
Expand Down
84 changes: 84 additions & 0 deletions core/src/main/java/org/testcontainers/jib/JibDockerClient.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
package org.testcontainers.jib;

import com.github.dockerjava.api.command.InspectImageResponse;
import com.github.dockerjava.api.command.LoadImageCallback;
import com.google.cloud.tools.jib.api.DockerClient;
import com.google.cloud.tools.jib.api.ImageDetails;
import com.google.cloud.tools.jib.api.ImageReference;
import com.google.cloud.tools.jib.http.NotifyingOutputStream;
import com.google.cloud.tools.jib.image.ImageTarball;
import com.google.common.io.ByteStreams;
import lombok.Cleanup;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.UnstableAPI;
import org.testcontainers.images.RemoteDockerImage;
import org.testcontainers.utility.DockerImageName;

import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Map;
import java.util.function.Consumer;

@UnstableAPI
class JibDockerClient implements DockerClient {

private static JibDockerClient instance;

private final com.github.dockerjava.api.DockerClient dockerClient = DockerClientFactory.lazyClient();

public static JibDockerClient instance() {
if (instance == null) {
instance = new JibDockerClient();
}

return instance;
}

@Override
public boolean supported(Map<String, String> map) {
return false;
}

@Override
public String load(ImageTarball imageTarball, Consumer<Long> writtenByteCountListener) throws IOException {
@Cleanup
PipedInputStream in = new PipedInputStream();
@Cleanup
PipedOutputStream out = new PipedOutputStream(in);
LoadImageCallback loadImage = this.dockerClient.loadImageAsyncCmd(in).exec(new LoadImageCallback());

try (NotifyingOutputStream stdin = new NotifyingOutputStream(out, writtenByteCountListener)) {
imageTarball.writeTo(stdin);
}

return loadImage.awaitMessage();
}

@Override
public void save(ImageReference imageReference, Path outputPath, Consumer<Long> writtenByteCountListener)
throws IOException {
try (
InputStream inputStream = this.dockerClient.saveImageCmd(imageReference.toString()).exec();
InputStream stdout = new BufferedInputStream(inputStream);
OutputStream fileStream = new BufferedOutputStream(Files.newOutputStream(outputPath));
NotifyingOutputStream notifyingFileStream = new NotifyingOutputStream(fileStream, writtenByteCountListener)
) {
ByteStreams.copy(stdout, notifyingFileStream);
}
}

@Override
public ImageDetails inspect(ImageReference imageReference) {
new RemoteDockerImage(DockerImageName.parse(imageReference.toString())).get();

InspectImageResponse response = this.dockerClient.inspectImageCmd(imageReference.toString()).exec();
return new JibImageDetails(response.getSize(), response.getId(), response.getRootFS().getLayers());
}
}
59 changes: 59 additions & 0 deletions core/src/main/java/org/testcontainers/jib/JibImage.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package org.testcontainers.jib;

import com.google.cloud.tools.jib.api.Containerizer;
import com.google.cloud.tools.jib.api.DockerClient;
import com.google.cloud.tools.jib.api.DockerDaemonImage;
import com.google.cloud.tools.jib.api.Jib;
import com.google.cloud.tools.jib.api.JibContainer;
import com.google.cloud.tools.jib.api.JibContainerBuilder;
import lombok.SneakyThrows;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.utility.Base58;
import org.testcontainers.utility.LazyFuture;
import org.testcontainers.utility.ResourceReaper;

import java.util.Map;
import java.util.function.Function;
import java.util.stream.Collectors;
import java.util.stream.Stream;

public class JibImage extends LazyFuture<String> {

private final DockerClient dockerClient = JibDockerClient.instance();

private static final Map<String, String> DEFAULT_LABELS = Stream
.of(
DockerClientFactory.DEFAULT_LABELS.entrySet().stream(),
ResourceReaper.instance().getLabels().entrySet().stream()
)
.flatMap(Function.identity())
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));

private final String baseImage;

private final Function<JibContainerBuilder, JibContainerBuilder> jibContainerBuilderFn;

public JibImage(String baseImage, Function<JibContainerBuilder, JibContainerBuilder> jibContainerBuilderFn) {
this.baseImage = baseImage;
this.jibContainerBuilderFn = jibContainerBuilderFn;
}

@SneakyThrows
@Override
protected String resolve() {
JibContainerBuilder containerBuilder = Jib.from(this.dockerClient, DockerDaemonImage.named(this.baseImage));
Function<JibContainerBuilder, JibContainerBuilder> applyLabelsFn = jibContainerBuilder -> {
for (Map.Entry<String, String> entry : DEFAULT_LABELS.entrySet()) {
jibContainerBuilder.addLabel(entry.getKey(), entry.getValue());
}
return jibContainerBuilder;
};
JibContainer jibContainer =
this.jibContainerBuilderFn.andThen(applyLabelsFn)
.apply(containerBuilder)
.containerize(
Containerizer.to(this.dockerClient, DockerDaemonImage.named(Base58.randomString(8).toLowerCase()))
);
return jibContainer.getTargetImage().toString();
}
}
42 changes: 42 additions & 0 deletions core/src/main/java/org/testcontainers/jib/JibImageDetails.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
package org.testcontainers.jib;

import com.google.cloud.tools.jib.api.DescriptorDigest;
import com.google.cloud.tools.jib.api.ImageDetails;

import java.security.DigestException;
import java.util.ArrayList;
import java.util.List;

class JibImageDetails implements ImageDetails {

private long size;

private String imageId;

private List<String> layers;

public JibImageDetails(long size, String imageId, List<String> layers) {
this.size = size;
this.imageId = imageId;
this.layers = layers;
}

@Override
public long getSize() {
return this.size;
}

@Override
public DescriptorDigest getImageId() throws DigestException {
return DescriptorDigest.fromDigest(this.imageId);
}

@Override
public List<DescriptorDigest> getDiffIds() throws DigestException {
List<DescriptorDigest> processedDiffIds = new ArrayList<>(this.layers.size());
for (String diffId : this.layers) {
processedDiffIds.add(DescriptorDigest.fromDigest(diffId.trim()));
}
return processedDiffIds;
}
}
88 changes: 88 additions & 0 deletions core/src/test/java/org/testcontainers/containers/JibTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
package org.testcontainers.containers;

import com.github.dockerjava.api.command.InspectImageResponse;
import org.junit.Test;
import org.testcontainers.DockerClientFactory;
import org.testcontainers.containers.output.OutputFrame.OutputType;
import org.testcontainers.containers.startupcheck.OneShotStartupCheckStrategy;
import org.testcontainers.jib.JibImage;

import java.time.Duration;
import java.util.Collections;

import static org.assertj.core.api.Assertions.assertThat;

public class JibTest {

@Test
public void buildImage() {
try (
// jibContainerUsage {
GenericContainer<?> busybox = new GenericContainer<>(
new JibImage(
"busybox:1.35",
jibContainerBuilder -> {
return jibContainerBuilder.setEntrypoint("echo", "Hello World");
}
)
)
.withStartupCheckStrategy(new OneShotStartupCheckStrategy().withTimeout(Duration.ofSeconds(3)))
// }
) {
busybox.start();
String logs = busybox.getLogs(OutputType.STDOUT);
assertThat(logs).contains("Hello World");
}
}

@Test
public void standardLabelsAreAddedWhenUsingJibSetLabels() {
try (
GenericContainer<?> busybox = new GenericContainer<>(
new JibImage(
"busybox:1.35",
jibContainerBuilder -> {
return jibContainerBuilder
.setEntrypoint("echo", "Hello World")
.setLabels(Collections.singletonMap("foo", "bar"));
}
)
)
.withStartupCheckStrategy(new OneShotStartupCheckStrategy().withTimeout(Duration.ofSeconds(3)))
) {
busybox.start();
assertImageLabels(busybox);
}
}

@Test
public void standardLabelsAreAddedWhenUsingJibAddLabel() {
try (
GenericContainer<?> busybox = new GenericContainer<>(
new JibImage(
"busybox:1.35",
jibContainerBuilder -> {
return jibContainerBuilder.setEntrypoint("echo", "Hello World").addLabel("foo", "bar");
}
)
)
.withStartupCheckStrategy(new OneShotStartupCheckStrategy().withTimeout(Duration.ofSeconds(3)))
) {
busybox.start();
assertImageLabels(busybox);
}
}

private static void assertImageLabels(GenericContainer<?> busybox) {
String image = busybox.getContainerInfo().getConfig().getImage();
InspectImageResponse imageResponse = DockerClientFactory.lazyClient().inspectImageCmd(image).exec();
assertThat(imageResponse.getConfig().getLabels())
.containsEntry("foo", "bar")
.containsKeys(
"org.testcontainers",
"org.testcontainers.sessionId",
"org.testcontainers.lang",
"org.testcontainers.version"
);
}
}
11 changes: 11 additions & 0 deletions docs/features/jib.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Using Jib

[Jib](https://github.com/GoogleContainerTools/jib/tree/master/jib-core) is a library for building Docker images.
eddumelendez marked this conversation as resolved.
Show resolved Hide resolved
You can use it as an alternative to Testcontainers default `DockerfileBuilder`.

<!--codeinclude-->
[GenericContainer with JibImage](../../core/src/test/java/org/testcontainers/containers/JibTest.java) inside_block:jibContainerUsage
<!--/codeinclude-->

!!! hint
The Testcontainers library JAR will not automatically add a `jib-core` JAR to your project. Minimum version required is `com.google.cloud.tools:jib-core:0.22.0`.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ nav:
- features/startup_and_waits.md
- features/container_logs.md
- features/creating_images.md
- features/jib.md
- features/configuration.md
- features/image_name_substitution.md
- features/advanced_options.md
Expand Down