Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose logs #12

Open
wvengen opened this issue Jan 29, 2024 · 0 comments
Open

Expose logs #12

wvengen opened this issue Jan 29, 2024 · 0 comments
Labels
API API & Web docker Docker enhancement New feature or request k8s Kuberenetes

Comments

@wvengen
Copy link
Member

wvengen commented Jan 29, 2024

Scrapyd's listjobs endpoint shows a logs path for finished jobs (as seen here):

{
    "status": "ok",
    // ...
    "finished": [
        {
            "id": "2f16646cfcaf11e1b0090800272a6d06",
            "project": "myproject", "spider": "spider3",
            "start_time": "2012-09-12 10:14:03.594664",
            "end_time": "2012-09-12 10:24:03.594664",
            "log_url": "/logs/myproject/spider3/2f16646cfcaf11e1b0090800272a6d06.log",
            "items_url": "/items/myproject/spider3/2f16646cfcaf11e1b0090800272a6d06.jl"
        }
    ]
}

We don't have this yet.
First add serving logs to the API (just plaintext output, the container log messages, streaming if you manage to do it).
Then add the extra entry to the listjobs endpoint for finished jobs.

@wvengen wvengen added the enhancement New feature or request label Jan 29, 2024
@wvengen wvengen added k8s Kuberenetes docker Docker API API & Web labels Jan 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API API & Web docker Docker enhancement New feature or request k8s Kuberenetes
Projects
No open projects
Status: Todo
Development

No branches or pull requests

1 participant