Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnknownJob timestamp properties not being cast #1377

Open
yagodorea opened this issue Oct 4, 2022 · 1 comment
Open

UnknownJob timestamp properties not being cast #1377

yagodorea opened this issue Oct 4, 2022 · 1 comment
Assignees
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p3 Desirable enhancement or fix. May not be included in next release.

Comments

@yagodorea
Copy link

Environment details

  • OS type and version: MacOS 12.5
  • Python version: Python 3.8.10
  • pip version: pip 22.2.2
  • google-cloud-bigquery version: 3.1.0

Steps to reproduce

  1. Create a service account that doesn't have the bigquery.jobs.listAll permission
  2. Use that service account to fetch a job created by another user (lots of fields will not come, or come redacted, including jobType, so the _AsyncJob will be an UnknownJob, see here)
  3. Try to access the job.started field, it'll break (here) because the timestamp fields aren't being cast to int for UnkownJob instances

Code example

import json
import google_auth_httplib2
import httplib2
from google.cloud import bigquery
from google.oauth2 import service_account
from googleapiclient.discovery import build

f = open("bigquery_service_account.json")
sa = f.read()

scopes = ["https://www.googleapis.com/auth/bigquery.readonly"]
credentials = service_account.Credentials.from_service_account_info(
    json.loads(sa), scopes=scopes
)
# client
client = bigquery.Client(credentials=credentials)

# service
http = httplib2.Http()
authed_http = google_auth_httplib2.AuthorizedHttp(credentials, http=http)
service = build("bigquery", "v2", http=authed_http, cache_discovery=False)


jobs = (
    service.jobs()
    .list(
        allUsers=True,
        projectId="<project_id>",
        maxResults=1,
        projection="full",
    )
    .execute(num_retries=1)
)
jobs = jobs.get("jobs", [])

for job in jobs:
    job = client.job_from_resource(job)
    print(job.started)

Stack trace

Traceback (most recent call last):
  File "bqtest.py", line 38, in <module>
    print(job.started)
  File "[...]lib/python3.8/site-packages/google/cloud/bigquery/job/base.py", line 336, in started
    return _helpers._datetime_from_microseconds(millis * 1000.0)
TypeError: can't multiply sequence by non-int of type 'float'
@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery API. label Oct 4, 2022
@shollyman
Copy link
Contributor

TODO: revisit the partial projection issue where the backend redacts some job metadata when parsing jobs.list entries.

@shollyman shollyman added the priority: p3 Desirable enhancement or fix. May not be included in next release. label Mar 27, 2023
@Linchin Linchin self-assigned this Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery API. priority: p3 Desirable enhancement or fix. May not be included in next release.
Projects
None yet
Development

No branches or pull requests

3 participants