Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add start_time and end_time in listjobs.json #11

Open
wvengen opened this issue Jan 29, 2024 · 2 comments · May be fixed by #30
Open

Add start_time and end_time in listjobs.json #11

wvengen opened this issue Jan 29, 2024 · 2 comments · May be fixed by #30
Assignees
Labels
API API & Web docker Docker enhancement New feature or request k8s Kuberenetes

Comments

@wvengen
Copy link
Member

wvengen commented Jan 29, 2024

For running jobs returned by the listjobs endpoint, add start_time.
For finished jobs returned by the listjobs endpoint, add start_time and end_time.

See https://scrapyd.readthedocs.io/en/latest/api.html#listjobs-json for an example output:

{
    "status": "ok",
    "pending": [
        {
            "project": "myproject", "spider": "spider1",
            "id": "78391cc0fcaf11e1b0090800272a6d06"
        }
    ],
    "running": [
        {
            "id": "422e608f9f28cef127b3d5ef93fe9399",
            "project": "myproject", "spider": "spider2",
            "start_time": "2012-09-12 10:14:03.594664"
        }
    ],
    "finished": [
        {
            "id": "2f16646cfcaf11e1b0090800272a6d06",
            "project": "myproject", "spider": "spider3",
            "start_time": "2012-09-12 10:14:03.594664",
            "end_time": "2012-09-12 10:24:03.594664",
            "log_url": "/logs/myproject/spider3/2f16646cfcaf11e1b0090800272a6d06.log",
            "items_url": "/items/myproject/spider3/2f16646cfcaf11e1b0090800272a6d06.jl"
        }
    ]
}

(we'll omit items_url, and log_url has #12)

@wvengen wvengen added the enhancement New feature or request label Jan 29, 2024
@wvengen
Copy link
Member Author

wvengen commented Jan 30, 2024

Note that we'd want to get these dates from the container info.
For Docker, try docker inspect <jobname> and you'll see StartedAt and FinishedAt in the State.
For Kubernetes, try kubectl describe job <jobname> and you'll see Start Time and Completed At.

Make sure to stick to the date format as in the example.

@wvengen wvengen added k8s Kuberenetes docker Docker API API & Web labels Jan 31, 2024
@RoelAdriaans
Copy link

Worked on by Roel and Timmy

@leewesleyv leewesleyv self-assigned this Aug 30, 2024
@leewesleyv leewesleyv linked a pull request Aug 30, 2024 that will close this issue
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API API & Web docker Docker enhancement New feature or request k8s Kuberenetes
Projects
No open projects
Status: In Progress
Development

Successfully merging a pull request may close this issue.

3 participants