Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prune autoinstrumentation sitecustomize module directory from PYTHONPATH immediately #1066

Merged
merged 1 commit into from
Apr 21, 2022

Conversation

aabmass
Copy link
Member

@aabmass aabmass commented Apr 20, 2022

Description

This prevents auto-instrumentation of subprocesses launched with exec* as soon as possible. The risk of autoinstrumenting these subprocesses is creating a loop of autoinstrumentation if the initialization code creates the subprocess.

Fixes #1050

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

Tested manually

Does This PR Require a Core Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

See contributing.md for styleguide, changelog guidelines, and more.

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@aabmass aabmass requested a review from a team April 20, 2022 21:29
@ghost
Copy link

ghost commented May 18, 2022

This PR broke auto-instrumentation for me. the server-uninstrumented.py script fails to produce spans. Could you explain the purposed of moving the code block out of "finally"? Here's a seemingly related issue.

@aabmass
Copy link
Member Author

aabmass commented May 19, 2022

Could you explain the purposed of moving the code block out of "finally"?

Check the PR description and #1050.

Can you provide a repro in a new issue? It is working fine for me:

$ python3 -m venv venv
$ source venv/bin/activate
$ pip install flask opentelemetry-instrumentation opentelemetry-distro opentelemetry-instrumentation-flask
$ curl -O https://raw.githubusercontent.com/open-telemetry/opentelemetry-python/main/docs/examples/auto-instrumentation/server_uninstrumented.py
$ opentelemetry-instrument --traces_exporter console python server_uninstrumented.py

# separate terminal do
$ curl localhost:8082/server_request

 * Serving Flask app 'server_uninstrumented' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
None
{
    "name": "/server_request",
    "context": {
        "trace_id": "0x036a62b07231c51c2cd9fb8db5d8de0b",
        "span_id": "0x46378b949ea3c0fb",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": null,
    "start_time": "2022-05-19T03:18:34.192093Z",
    "end_time": "2022-05-19T03:18:34.193532Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "GET",
        "http.server_name": "127.0.0.1",
        "http.scheme": "http",
        "net.host.port": 8082,
        "http.host": "localhost:8082",
        "http.target": "/server_request",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "curl/7.82.0",
        "net.peer.port": 33810,
        "http.flavor": "1.1",
        "http.route": "/server_request",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.12.0rc1",
        "telemetry.auto.version": "0.31b0",
        "service.name": "unknown_service"
    }
}

@ghost
Copy link

ghost commented May 19, 2022

Here are my results running with the 1066 change. I am on Windows 11:

virtualenv auto_autoinstrumentation 
source auto_instrumentation/Scripts/activate
python --version
Python 3.10.4
pip list 
opentelemetry-api                      1.12.0rc1 c:\users\jeremyvoss\workplace\opentelemetry-python\opentelemetry-api\src 
opentelemetry-distro                   0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\opentelemetry-distro\src 
opentelemetry-instrumentation          0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\opentelemetry-instrumentation\src 
opentelemetry-instrumentation-flask    0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\instrumentation\opentelemetry-instrumentation-flask\src 
opentelemetry-instrumentation-requests 0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\instrumentation\opentelemetry-instrumentation-requests\src 
opentelemetry-instrumentation-wsgi     0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\instrumentation\opentelemetry-instrumentation-wsgi\src 
opentelemetry-proto                    1.12.0rc1 c:\users\jeremyvoss\workplace\opentelemetry-python\opentelemetry-proto\src 
opentelemetry-sdk                      1.12.0rc1 c:\users\jeremyvoss\workplace\opentelemetry-python\opentelemetry-sdk\src 
opentelemetry-semantic-conventions     0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python\opentelemetry-semantic-conventions\src 
opentelemetry-util-http                0.31b0    c:\users\jeremyvoss\workplace\opentelemetry-python-contrib\util\opentelemetry-util-http\src 
pip                                    22.1 
opentelemetry-instrument --traces_exporter console python opentelemetry-python/docs/examples/auto-instrumentation/server_uninstrumented.py 

$ Error in sitecustomize; set PYTHONVERBOSE for traceback: 
error: incomplete escape \u at position 2 
* Serving Flask app 'server_uninstrumented' (lazy loading) 
* Environment: production 
   WARNING: This is a development server. Do not use it in a production deployment. 
   Use a production WSGI server instead. 
* Debug mode: off 
* Running on http://127.0.0.1:8082/ (Press CTRL+C to quit) 
None 


#In other terminal in virtualenv: 
python opentelemetry-python/docs/examples/auto-instrumentation/client.py testing 
{ 
    "name": "client-server", 
    "context": { 
        "trace_id": "0x73acdf4567483e0687b68750cfb4f949", 
        "span_id": "0xd64fe9ef65f7755b", 
        "trace_state": "[]" 
    }, 
    "kind": "SpanKind.INTERNAL", 
    "parent_id": "0x38d3f426cefc6cc1", 
    "start_time": "2022-05-19T19:50:19.645699Z", 
    "end_time": "2022-05-19T19:50:21.694709Z", 
    "status": { 
        "status_code": "UNSET" 
    }, 
    "attributes": {}, 
    "events": [], 
    "links": [], 
    "resource": { 
        "telemetry.sdk.language": "python", 
        "telemetry.sdk.name": "opentelemetry", 
        "telemetry.sdk.version": "1.12.0rc1", 
        "service.name": "unknown_service" 
    } 
} 
{ 
    "name": "client", 
    "context": { 
        "trace_id": "0x73acdf4567483e0687b68750cfb4f949", 
        "span_id": "0x38d3f426cefc6cc1", 
        "trace_state": "[]" 
    }, 
    "kind": "SpanKind.INTERNAL", 
    "parent_id": null, 
    "start_time": "2022-05-19T19:50:19.645699Z", 
    "end_time": "2022-05-19T19:50:21.694709Z", 
    "status": { 
        "status_code": "UNSET" 
    }, 
    "attributes": {}, 
    "events": [], 
    "links": [], 
    "resource": { 
        "telemetry.sdk.language": "python", 
        "telemetry.sdk.name": "opentelemetry", 
        "telemetry.sdk.version": "1.12.0rc1", 
        "service.name": "unknown_service" 
    } 
} 

#Back in original terminal: 
$ testing 
127.0.0.1 - - [19/May/2022 12:50:21] "GET /server_request?param=testing HTTP/1.1" 200 - 
# No spans

However, when I comment out the code block and put it back in the finally, I get the full span:

$ Error in sitecustomize; set PYTHONVERBOSE for traceback:
error: incomplete escape \u at position 2
 * Serving Flask app 'server_uninstrumented' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
testing
{
    "name": "/server_request",
    "context": {
        "trace_id": "0x2544d0e60b7f5c83d672be2a73a5ad67",
        "span_id": "0x86d419ab5908eb8b",
        "trace_state": "[]"
    },
    "kind": "SpanKind.SERVER",
    "parent_id": "0x3a0f0c23ddcc5a6d",
    "start_time": "2022-05-19T20:01:04.744223Z",
    "end_time": "2022-05-19T20:01:04.744223Z",
    "status": {
        "status_code": "UNSET"
    },
    "attributes": {
        "http.method": "GET",
        "http.server_name": "127.0.0.1",
        "http.scheme": "http",
        "net.host.port": 8082,
        "http.host": "localhost:8082",
        "http.target": "/server_request?param=testing",
        "net.peer.ip": "127.0.0.1",
        "http.user_agent": "python-requests/2.27.1",
        "net.peer.port": 52518,
        "http.flavor": "1.1",
        "http.route": "/server_request",
        "http.status_code": 200
    },
    "events": [],
    "links": [],
    "resource": {
        "telemetry.sdk.language": "python",
        "telemetry.sdk.name": "opentelemetry",
        "telemetry.sdk.version": "1.12.0rc1",
        "telemetry.auto.version": "0.31b0",
        "service.name": "unknown_service"
    }
}

Notice that in both cases I get, this strange Error in sitecustomize; set PYTHONVERBOSE for traceback: error: incomplete escape \u at position 2 message which might not appear on linux. I will be examining the code to see if there is a reason it would act differently across platforms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

opentelemetry-instrument command can cause recursive creation of subprocesses
4 participants