Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linter checks go versions in Dockerfile and YAML files #8954

Merged
merged 3 commits into from
Aug 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .golangci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ linters-settings:
- G306 # Poor file permissions used when writing to a new file.

staticcheck:
go: "1.21"
go: "1.22.5"
ffranr marked this conversation as resolved.
Show resolved Hide resolved
checks: ["-SA1019"]

lll:
Expand Down
32 changes: 27 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,20 @@ ANDROID_BUILD := $(ANDROID_BUILD_DIR)/Lndmobile.aar

COMMIT := $(shell git describe --tags --dirty)

GO_VERSION := $(shell go version | sed -nre 's/^[^0-9]*(([0-9]+\.)*[0-9]+).*/\1/p')
GO_VERSION_MINOR := $(shell echo $(GO_VERSION) | cut -d. -f2)
# Determine the minor version of the active Go installation.
ACTIVE_GO_VERSION := $(shell go version | sed -nre 's/^[^0-9]*(([0-9]+\.)*[0-9]+).*/\1/p')
ACTIVE_GO_VERSION_MINOR := $(shell echo $(ACTIVE_GO_VERSION) | cut -d. -f2)

LOOPVARFIX :=
ifeq ($(shell expr $(GO_VERSION_MINOR) \>= 21), 1)
ifeq ($(shell expr $(ACTIVE_GO_VERSION_MINOR) \>= 21), 1)
LOOPVARFIX := GOEXPERIMENT=loopvar
endif

# GO_VERSION is the Go version used for the release build, docker files, and
# GitHub Actions. This is the reference version for the project. All other Go
# versions are checked against this version.
GO_VERSION = 1.22.5
ffranr marked this conversation as resolved.
Show resolved Hide resolved

GOBUILD := $(LOOPVARFIX) go build -v
GOINSTALL := $(LOOPVARFIX) go install -v
GOTEST := $(LOOPVARFIX) go test
Expand Down Expand Up @@ -297,11 +303,27 @@ fmt-check: fmt
@$(call print, "Checking fmt results.")
if test -n "$$(git status --porcelain)"; then echo "code not formatted correctly, please run `make fmt` again!"; git status; git diff; exit 1; fi

#? lint: Run static code analysis
lint: docker-tools
#? check-go-version-yaml: Verify that the Go version is correct in all YAML files
check-go-version-yaml:
ffranr marked this conversation as resolved.
Show resolved Hide resolved
@$(call print, "Checking for target Go version (v$(GO_VERSION)) in YAML files (*.yaml, *.yml)")
./scripts/check-go-version-yaml.sh $(GO_VERSION)

#? check-go-version-dockerfile: Verify that the Go version is correct in all Dockerfile files
check-go-version-dockerfile:
@$(call print, "Checking for target Go version (v$(GO_VERSION)) in Dockerfile files (*Dockerfile)")
./scripts/check-go-version-dockerfile.sh $(GO_VERSION)

#? check-go-version: Verify that the Go version is correct in all project files
check-go-version: check-go-version-dockerfile check-go-version-yaml

#? lint-source: Run static code analysis
lint-source: docker-tools
@$(call print, "Linting source.")
$(DOCKER_TOOLS) golangci-lint run -v $(LINT_WORKERS)

#? lint: Run static code analysis
lint: check-go-version lint-source

#? protolint: Lint proto files using protolint
protolint:
@$(call print, "Linting proto files.")
Expand Down
2 changes: 1 addition & 1 deletion docker/btcd/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.21.0-alpine as builder
FROM golang:1.22.5-alpine as builder

LABEL maintainer="Olaoluwa Osuntokun <laolu@lightning.engineering>"

Expand Down
16 changes: 8 additions & 8 deletions docs/INSTALL.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,16 +100,16 @@ the following commands for your OS:
<summary>Linux (x86-64)</summary>

```
wget https://dl.google.com/go/go1.22.4.linux-amd64.tar.gz
sha256sum go1.22.4.linux-amd64.tar.gz | awk -F " " '{ print $1 }'
wget https://dl.google.com/go/go1.22.5.linux-amd64.tar.gz
sha256sum go1.22.5.linux-amd64.tar.gz | awk -F " " '{ print $1 }'
```

The final output of the command above should be
`ba79d4526102575196273416239cca418a651e049c2b099f3159db85e7bade7d`. If it
`904b924d435eaea086515bc63235b192ea441bd8c9b198c507e85009e6e4c7f0`. If it
isn't, then the target REPO HAS BEEN MODIFIED, and you shouldn't install
this version of Go. If it matches, then proceed to install Go:
```
sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.22.4.linux-amd64.tar.gz
sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.22.5.linux-amd64.tar.gz
ffranr marked this conversation as resolved.
Show resolved Hide resolved
export PATH=$PATH:/usr/local/go/bin
```
</details>
Expand All @@ -118,16 +118,16 @@ the following commands for your OS:
<summary>Linux (ARMv6)</summary>

```
wget https://dl.google.com/go/go1.22.4.linux-armv6l.tar.gz
sha256sum go1.22.4.linux-armv6l.tar.gz | awk -F " " '{ print $1 }'
wget https://dl.google.com/go/go1.22.5.linux-armv6l.tar.gz
sha256sum go1.22.5.linux-armv6l.tar.gz | awk -F " " '{ print $1 }'
```

The final output of the command above should be
`e2b143fbacbc9cbd448e9ef41ac3981f0488ce849af1cf37e2341d09670661de`. If it
`8c4587cf3e63c9aefbcafa92818c4d9d51683af93ea687bf6c7508d6fa36f85e`. If it
isn't, then the target REPO HAS BEEN MODIFIED, and you shouldn't install
this version of Go. If it matches, then proceed to install Go:
```
sudo rm -rf /usr/local/go && tar -C /usr/local -xzf go1.22.4.linux-armv6l.tar.gz
sudo rm -rf /usr/local/go && tar -C /usr/local -xzf go1.22.5.linux-armv6l.tar.gz
export PATH=$PATH:/usr/local/go/bin
```

Expand Down
2 changes: 1 addition & 1 deletion lnrpc/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.21.0-bookworm
FROM golang:1.22.5-bookworm

RUN apt-get update && apt-get install -y \
git \
Expand Down
64 changes: 64 additions & 0 deletions scripts/check-go-version-dockerfile.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/bin/bash

# Function to check if the Dockerfile contains only the specified Go version.
check_go_version() {
local dockerfile="$1"
local required_go_version="$2"

# Use grep to find lines with 'FROM golang:'
local go_lines=$(grep -i '^FROM golang:' "$dockerfile")

# Check if all lines have the required Go version.
if [ -z "$go_lines" ]; then
# No Go version found in the file. Skip the check.
return
elif echo "$go_lines" | grep -q -v "$required_go_version"; then
echo "$go_lines"
echo "Error: $dockerfile does not use Go version $required_go_version exclusively."
exit 1
else
echo "$dockerfile is using Go version $required_go_version."
fi
}

# Check if the target Go version argument is provided.
if [ $# -eq 0 ]; then
echo "Usage: $0 <target_go_version>"
exit 1
fi

target_go_version="$1"

# File paths to be excluded from the check.
exception_list=(
# Exclude the tools Dockerfile as otherwise the linter may need to be
# considered every time the Go version is updated.
"./tools/Dockerfile"
)

# is_exception checks if a file is in the exception list.
is_exception() {
local file="$1"
for exception in "${exception_list[@]}"; do
if [ "$file" == "$exception" ]; then
return 0
fi
done
return 1
}

# Search for Dockerfiles in the current directory and its subdirectories.
dockerfiles=$(find . -type f -name "*.Dockerfile" -o -name "Dockerfile")

# Check each Dockerfile
for file in $dockerfiles; do
# Skip the file if it is in the exception list.
if is_exception "$file"; then
echo "Skipping $file"
continue
fi

check_go_version "$file" "$target_go_version"
done

echo "All Dockerfiles pass the Go version check for Go version $target_go_version."
75 changes: 75 additions & 0 deletions scripts/check-go-version-yaml.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!/bin/bash

# Function to check if the YAML file contains the specified Go version after
# field 'go:'.
check_go_version_yaml() {
local yamlfile="$1"
local required_go_version="$2"

# Use grep to find lines with 'go:'. The grep exist status is ignored.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we use the prefix go:? I thought we always use GO_VERSION:

Copy link
Collaborator Author

@ffranr ffranr Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I think the only example is in .golangci.yml:

  staticcheck:
    go: "1.21"
    checks: ["-SA1019"]

And I don't think even care about that case. See: #8954 (comment)

Do we keep this check anyway because it might come into play and be useful later?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, in the latest commits, I've bumped the Go version in .golangci.yml. And I've modified the lint check so that it's included in the check.

local go_lines=$(grep -i '^\s*go:\s*"[0-9]\+\.[0-9]\+\(\.[0-9]\+\)\?"' "$yamlfile" || true)

# Check if any lines specify the Go version.
if [ -n "$go_lines" ]; then
# Extract the Go version from the file's lines. Example matching strings:
# go: "1.21.0"
local extracted_go_version=$(echo "$go_lines" | sed -n 's/.*go: "\([^"]*\)".*/\1/p')

# Check if the extracted Go version matches the required version.
if [ "$extracted_go_version" != "$required_go_version" ]; then
echo "Error finding pattern 'go:': $yamlfile specifies Go version '$extracted_go_version', but required version is '$required_go_version'."
exit 1
else
echo "$yamlfile specifies Go version $required_go_version."
fi
fi
}

# Function to check if the YAML file contains the specified Go version after
# environment variable 'GO_VERSION:'.
check_go_version_env_variable() {
local yamlfile="$1"
local required_go_version="$2"

# Use grep to find lines with 'GO_VERSION:'. The grep exist status is
# ignored.
local go_lines=$(grep -i 'GO_VERSION:' "$yamlfile" || true)

# Check if any lines specify the Go version.
if [ -n "$go_lines" ]; then
# Extract the Go version from the file's lines. Example matching strings:
# GO_VERSION: "1.21.0"
# GO_VERSION: '1.21.0'
# GO_VERSION: 1.21.0
# GO_VERSION:1.21.0
# GO_VERSION:1.21.0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: remove the spaces

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the leading spaces here are intentional. We need to be robust to leading white space.

local extracted_go_version=$(echo "$go_lines" | sed -n 's/.*GO_VERSION[: ]*["'\'']*\([0-9.]*\).*/\1/p')

# Check if the extracted Go version matches the required version.
if [ "$extracted_go_version" != "$required_go_version" ]; then
echo "Error finding pattern 'GO_VERSION:': $yamlfile specifies Go version '$extracted_go_version', but required version is '$required_go_version'."
exit 1
else
echo "$yamlfile specifies Go version $required_go_version."
fi
fi
}

# Check if the target Go version argument is provided.
if [ $# -eq 0 ]; then
echo "Usage: $0 <target_go_version>"
exit 1
fi

target_go_version="$1"

# Search for YAML files in the current directory and its subdirectories.
yaml_files=$(find . -type f \( -name "*.yaml" -o -name "*.yml" \))

# Check each YAML file.
for file in $yaml_files; do
check_go_version_yaml "$file" "$target_go_version"
check_go_version_env_variable "$file" "$target_go_version"
done

echo "All YAML files pass the Go version check for Go version $target_go_version."
Loading