[example] Create EEG-GCNN example. #3186
Conversation
|
To trigger regression tests:
|
examples/pytorch/eeg-gcnn/README.md
Outdated
| This DGL example implements the EEG-GCNN model proposed in the paper [EEG-GCNN](https://arxiv.org/abs/2011.12107). The original code is [here](https://github.com/neerajwagh/eeg-gcnn). | ||
|
|
||
| ## All References | ||
| - Paper can also be found on [PMLR](http://proceedings.mlr.press/v136/wagh20a.html). |
There was a problem hiding this comment.
Is this one the same as the ArXiv version? If so, we can probably remove it and just mention it in the citation entry at the end.
There was a problem hiding this comment.
Hi Mufei,
This is a simplified version. The original code has precomputed patient indices. In this example, I removed the dependency on the indices.
There was a problem hiding this comment.
I see, thanks. Then perhaps you can mention explicitly that this is a simplified version.
examples/pytorch/eeg-gcnn/README.md
Outdated
| First, download the precomputed data, labels, indices and put them in the repo. <br> | ||
| Then run | ||
| ```python | ||
| python main.py --num_feats --num_nodes --gpu_idx --num_epochs --exp_name --batch_size |
There was a problem hiding this comment.
There's no need to add --num_feats --num_nodes --gpu_idx --num_epochs --exp_name --batch_size as you have a default value for them.
examples/pytorch/eeg-gcnn/main.py
Outdated
| num_feats = args.num_feats | ||
|
|
||
| # set up input and targets from files | ||
| memmap_x = f'norm_X' |
There was a problem hiding this comment.
I checked the preprocessed dataset here, which does not seem to have the file norm_X.
There was a problem hiding this comment.
We haven't updated Figshare yet. We were dealing with a bug in the original code. So, some files need to be updated. We will do that next month, including norm_X. Also, norm_X is the normalized version of psd_featureds_data_X.
There was a problem hiding this comment.
In that case, we will need to wait for the files to be updated before we merge the PR.
|
Can you add an entry for this example in the indexing page here? |
examples/pytorch/eeg-gcnn/README.md
Outdated
| | DGL | AUC | Precision | Recall | F-1 | Bal. Accuracy | | ||
| |-------------------|-------------|---------------|--------------|--------------|---------------| | ||
| | Shallow EEG-GCNN | 0.875(0.036)| 0.980(0.013) | 0.735(0.055) | 0.839(0.035) | 0.811(0.035) | | ||
| | Deep EEG-GCNN | 0.890(0.004)| 0.988(0.004) | 0.723(0.035) | 0.834(0.022) | 0.829(0.005) | |
There was a problem hiding this comment.
How can a user reproduce the numbers here using python main.py?
There was a problem hiding this comment.
I don't think they can. We are planning to update the original repo with both pytorch_geometric and dgl implementations, and release the trainning code as well. This should be done sometime next month too. For this example, can I just point people to the original repo for reproducing the stats?
There was a problem hiding this comment.
We expect a DGL example to be self-contained and able to reproduce the results reported in README. In that case, either we can wait till you get everything ready for this PR or we can simply close the PR and add a pointer to your own repo once you get everything ready there.
There was a problem hiding this comment.
I see. Let me talk to Neeraj about this.
There was a problem hiding this comment.
Hi @mufeili, maybe the following might clarify the design of the PR:
-
We built the PR with the following assumption: that people can go to https://github.com/neerajwagh/eeg-gcnn for reproducing published results on the original data while the DGL example here would enable others to bring their own datasets (X, y) and train our model using the data they care about. Compared to our original PyT implementation, here we have skipped: a) subject-level data splitting, b) subject-level aggregate evaluation, c) cross-validation. This makes the example a "simplified" version and therefore will not be able to exactly reproduce published results. Nevertheless, we can update main.py and the results table so that main.py can reproduce the new "simplified" table made from a simpler model evaluation scheme (as long as users pull X, y from FigShare). We can close the PR after that change while keeping a pointer to the PyT repo for the full/"complex" implementation. We'll mention this simplification in the README so users are aware of the difference.
-
We can fix the absence of norm_X in FigShare by adding normalization after X is read in the code. (instead of updating the uploaded FigShare files).
Let me know if that works for you.
There was a problem hiding this comment.
Meanwhile, the updates we make to our original PyT repo will remain independent of this example and this example will remain self-contained.
There was a problem hiding this comment.
Hi @neerajwagh, the proposal sounds good to me in general. To be more specific, this example does not need to fully reproduce the experiments reported in your paper and I'm good as long as a user can run the script following the instructions in the README file and get similar numbers reported there.
There was a problem hiding this comment.
@mufeili Sounds good! We'll make changes accordingly then. Thanks.
examples/pytorch/eeg-gcnn/README.md
Outdated
|
|
||
| ### Contact | ||
|
|
||
| - Issues regarding non-reproducibility of results or support with the codebase should be emailed to _wei33@illinois.edu_ |
There was a problem hiding this comment.
How about
emailed to John(wei33@illinois.edu)
You may also contact the authors:
- Neeraj: nwagh2@illinois.edu / Website / Twitter / Google Scholar
- Yoga: varatha2@illinois.edu / Website / Google Scholar
|
Thank you for the great job! This PR is ready to merge once we hear back from Neeraj and resolve the last conversation. |
* Create EEG-GCNN example. * Update README.md * Remove gitignore file. * Update README.md * change 'datas' to 'datasets'. * Change train.py to main.py * Added an entry in the indexing page. * State "simplified version"; change how to run. * Fix bug in contact * Remove paper link in reference. * Create working branch * Add normalization of x. * Update paper link and tags * Update paper link in readme * Update readme; add patient level indices * Update readme. Add comments to models * Update README.md * change to with; specify location for ch and el; move note * fix bug for note * Add args for models; clean code. * delete = in readme * Add reference for spec_coh_values
Description
This is an EEG-GCNN example using DGL library. The original code using pytorch_geometric can be found here.
Checklist
Please feel free to remove inapplicable items for your PR.
or have been fixed to be compatible with this change
Changes