Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cri check fails when running on docker #66

Closed
ppennanen opened this issue Apr 6, 2020 · 3 comments
Closed

cri check fails when running on docker #66

ppennanen opened this issue Apr 6, 2020 · 3 comments
Labels
bug Something isn't working component/controller v0.x

Comments

@ppennanen
Copy link

Describe what happened:

datadog-agent created by datadog-operator running on docker logs errors like this:

2020-04-06 12:05:53 UTC | CORE | ERROR | (pkg/collector/runner/runner.go:292 in work) | Error running check cri: temporary failure in criutil, will retry later: try delay not elapsed yet
2020-04-06 12:05:58 UTC | CORE | WARN | (pkg/collector/python/datadog_agent.go:118 in LogMessage) | kubelet:d884b5186b651429 | (kubelet.py:429) | GET on kubelet s `/stats/summary` failed: 403 Client Error: Forbidden for url: https://<REDACTED>:10250/stats/summary?verbose=True
2020-04-06 12:06:08 UTC | CORE | WARN | (pkg/collector/corechecks/checkbase.go:165 in Warnf) | Error initialising check: temporary failure in criutil, will retry later: try delay not elapsed yet

This looks like the same issue as criutil failure that was resolved by Do not enable the cri check when running on a docker setup.

The agent has the environment variable

      DD_CRI_SOCKET_PATH:                     /host/var/run/docker.sock

Describe what you expected:

That the agent would not try to use the docker socket as a CRI socket.

Steps to reproduce the issue:

  • Launch a GKE cluster with Ubuntu + Docker.
  • Deploy DataDog operator
  • Deploy an agent CR:
apiVersion: datadoghq.com/v1alpha1
kind: DatadogAgent
metadata:
  name: datadog-agent
  namespace: datadog
spec:
  credentials:
    apiKeyExistingSecret: dd-api-key
  agent:
    config:
      tolerations:
      - operator: Exists
    apm:
      enabled: true
    logs:
      enabled: true
    image:
      name: "datadog/agent:7.18.1"
    process:
      enabled: true
    systemProbe:
      enabled: true

Additional environment details (Operating System, Cloud provider, etc):

GKE. Ubuntu with Docker.

@ppennanen ppennanen changed the title cri check fails when running on docker cri check fails when running on docker Apr 6, 2020
@ppennanen
Copy link
Author

ppennanen commented Apr 7, 2020

To silence the error add the environment value DD_CRI_SOCKET_PATH set to null:

apiVersion: datadoghq.com/v1alpha1
kind: DatadogAgent
metadata:
  name: datadog-agent
  namespace: datadog
spec:
  credentials:
    apiKeyExistingSecret: dd-api-key
  agent:
    config:
      env:
      - name: DD_CRI_SOCKET_PATH
        value: null 
      tolerations:
      - operator: Exists
    apm:
      enabled: true
    log:
      enabled: true
    image:
      name: "datadog/agent:7.18.1"
    process:
      enabled: true
    systemProbe:
      enabled: true

@ahmed-mez
Copy link
Contributor

Hi @ppennanen

Thank you for reporting this and for the workaround that you suggested (by setting DD_CRI_SOCKET_PATH to null)
We're aware of this issue and we're working on a fix
We also saw the kubelet check warning in the logs you've shared and we fixed it.
The two fixes will be included in the next datadog-operator release.

Thanks!

@ahmed-mez ahmed-mez added bug Something isn't working component/controller labels Apr 7, 2020
@levan-m levan-m added the v0.x label Aug 18, 2024
@levan-m
Copy link
Contributor

levan-m commented Aug 19, 2024

Bulk resolving DatadogAgent v1alpha1 issues.

@levan-m levan-m closed this as completed Aug 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working component/controller v0.x
Projects
None yet
Development

No branches or pull requests

3 participants