[GraphBolt][CUDA] Pipelined sampling optimization

[mfbalin]

Description

We overlap the UVA graph accesses with the rest of the computations by launching them in the UVA stream. We use a separate thread becase the index_select_csc is blocking, unlike index_select, which is nonblocking. I suspect the GIL in python is making multithreading inefficient a little bit, so more performance can be unlocked in the future after python with no GIL is available, see https://peps.python.org/pep-0703/#gpu-heavy-workloads-require-multi-core-processing

I am getting %10-%20 end-to-end speedup for sample_layer_neighbor. sample_neighbor is currently not getting much speedup because it does not need to access the whole indices tensor, which we are fetching through this optimization. Will need to specialize for sample_neighbor later.

This optimization will be the core of us providing the users a way to define their own custom samplers using dgl.sparse. My next PR will show how to implement LADIES while still taking advantage of the GraphBolt optimizations.

Checklist

Please feel free to remove inapplicable items for your PR.

• The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
• I've leverage the tools to beautify the python and c++ code.
• The PR is complete and small, read the Google eng practice (CL equals to PR) to understand more about small PR. In DGL, we consider PRs with less than 200 lines of core code change are small (example, test and documentation could be exempted).
• All changes have test coverage
• Code is well-documented
• To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
• Related issue is referred in this PR
• If the PR is for a new model/paper, I've updated the example index here.

Changes

[dgl-bot]

To trigger regression tests:

• @dgl-bot run [instance-type] [which tests] [compare-with-branch];
  For example: @dgl-bot run g4dn.4xlarge all dmlc/master or @dgl-bot run c5.9xlarge kernel,api dmlc/master

[dgl-bot]

Commit ID: e8ae274

Build ID: 1

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 571fd8757ea01f02ef6b02f9d0ed7de9be268ae3

Build ID: 2

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: f3dd0b3

Build ID: 4

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 1a5abae

Build ID: 3

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: ba18b7f

Build ID: 5

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 3547c00

Build ID: 6

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 1dce39e

Build ID: 7

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 0ad178a

Build ID: 8

Status: ❌ CI test failed in Stage [Lint Check].

Report path: link

Full logs path: link

[frozenbugs]

Overall LGTM, questions on python/dgl/graphbolt/impl/neighbor_sampler.py.

[dgl-bot]

Commit ID: b52adea4a117bc8c2369dd6a96d4623d85c9e716

Build ID: 35

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 23ade71

Build ID: 36

Status: ⚪️ CI test cancelled due to overrun.

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 6b21742

Build ID: 37

Status: ❌ CI test failed in Stage [GPU Build].

Report path: link

Full logs path: link

[mfbalin]

@dgl-bot

[dgl-bot]

Commit ID: 6b21742

Build ID: 38

Status: ❌ CI test failed in Stage [Torch CPU].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: 0a21186

Build ID: 39

Status: ❌ CI test failed in Stage [Torch CPU].

Report path: link

Full logs path: link

[mfbalin]

@dgl-bot

[mfbalin]

@Rhett-Ying Does the CI have a problem?

[dgl-bot]

Commit ID: 0a21186

Build ID: 40

Status: ❌ CI test failed in Stage [GPU Build].

Report path: link

Full logs path: link

[mfbalin]

@dgl-bot

[dgl-bot]

Commit ID: 0a21186

Build ID: 41

Status: ❌ CI test failed in Stage [GPU Build].

Report path: link

Full logs path: link

[dgl-bot]

Commit ID: e0fa30b

Build ID: 42

Status: ❌ CI test failed in Stage [GPU Build].

Report path: link

Full logs path: link

[Rhett-Ying]

@dgl-bot

[dgl-bot]

Commit ID: e0fa30b

Build ID: 43

Status: ❌ CI test failed in Stage [CPU Build].

Report path: link

Full logs path: link

[Rhett-Ying]

@dgl-bot

[dgl-bot]

Commit ID: e0fa30b

Build ID: 44

Status: ❌ CI test failed in Stage [GPU Build].

Report path: link

Full logs path: link

[Rhett-Ying]

@dgl-bot

[dgl-bot]

Commit ID: e0fa30b

Build ID: 45

Status: ✅ CI test succeeded.

Report path: link

Full logs path: link