-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ONNX Model Availability #3
Comments
Hi @john09282922, Not yet, I believe that it shouldn't be a complex conversion since most of the layers are common ones. Maybe the visual feature cropping used as GNN input could not be supported but is the same method used by well knonw models like MaskRCNN. Depending on the impact of the repository, we will consider further improvements of it surch as releasing ONNX models. In the meantime, feel free to submit a PR if you do the conversion. Best, |
Hi, thank you for appreciating your reply.
Would you mind telling me what kind of part I have to replace your gnn
input with MaskRcnn? I need your help...
plz let me know
thanks,
jungmin
…On Tue, Mar 14, 2023 at 12:47 PM Andrés Prados Torreblanca < ***@***.***> wrote:
Hi @john09282922 <https://github.com/john09282922>,
Not yet, I believe that it shouldn't be a complex conversion since most of
the layers are common ones. Maybe the visual feature cropping used as GNN
input could not be supported but is the same method used by well knonw
models like MaskRCNN.
Depending on the impact of the repository, we will consider further
improvements of it surch as releasing ONNX models. In the meantime, feel
free to submit a PR if you do the conversion.
Best,
Andrés
—
Reply to this email directly, view it on GitHub
<#3 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3KJF5IUNDEQKNNEKIV34DDW4COJ5ANCNFSM6AAAAAAVVMQEJE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Sry Jungmin, it is just a guess of a possible problem since it is a sort of "dynamic operation" and ONNX usually doesnt like them. I havent try to convert the model so I cannot know if it is even a problem. Let me know which exact error are you facing. Maybe I can help. |
Hi,
Thank you for reanswering my question.
UnsupportedOperatorError: Exporting the operator
'aten::affine_grid_generator' to ONNX opset version 14 is not supported.
This 'aten' makes error to export onnx file from your model.
Is there any solution? is there put inputs all together?
thanks,
jungmin
…On Tue, Mar 14, 2023 at 1:48 PM Andrés Prados Torreblanca < ***@***.***> wrote:
Reopened #3 <#3>.
—
Reply to this email directly, view it on GitHub
<#3 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3KJF5KZKWHZBQ2QAUZQAEDW4CVPHANCNFSM6AAAAAAVVMQEJE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Jackpot! Your are facing a conversion problem related to the affine transformation used to crop the visual features of the GNN input. This operation is used by Spatial Transformer Networks, take a look at this issue, it seems to be already solved. |
Hi, I read your issue part. unfortunately, It is problem like
aten::affine_grid_generator'
to ONNX opset version 16
…On Thu, Mar 16, 2023 at 7:55 AM Andrés Prados Torreblanca < ***@***.***> wrote:
Jackpot! Your are facing a conversion problem related to the affine
transformation used to crop the visual features of the GNN input. This
operation is used by Spatial Transformer Networks, take a look at this
issue <pytorch/pytorch#27212>, it seems to be
already solved.
—
Reply to this email directly, view it on GitHub
<#3 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3KJF5JAUTZVTP33A6WJTTLW4L5UHANCNFSM6AAAAAAVVMQEJE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Hi,
Unfortunately, this is not working following the issue. the issue handled
on grid sample, not grid generator. instead of using grid? is there other
way?
I changed to onnx version 16, even 17...
it is not works ....
thanks,
jungmin
…On Thu, Mar 16, 2023 at 7:55 AM Andrés Prados Torreblanca < ***@***.***> wrote:
Jackpot! Your are facing a conversion problem related to the affine
transformation used to crop the visual features of the GNN input. This
operation is used by Spatial Transformer Networks, take a look at this
issue <pytorch/pytorch#27212>, it seems to be
already solved.
—
Reply to this email directly, view it on GitHub
<#3 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/A3KJF5JAUTZVTP33A6WJTTLW4L5UHANCNFSM6AAAAAAVVMQEJE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
or am i making some problem to make random input?
thanks,
jungmin
sample_input1 = torch.rand((1, 3, 256, 256))
sample_input2 = torch.rand((1, 98, 3))
sample_input3 = torch.rand((1, 3, 3))
sample_input = [sample_input1, sample_input2, sample_input3]
On Thu, Mar 16, 2023 at 3:30 PM JUNGMIN HWANG ***@***.***>
wrote:
… Hi,
Unfortunately, this is not working following the issue. the issue handled
on grid sample, not grid generator. instead of using grid? is there other
way?
I changed to onnx version 16, even 17...
it is not works ....
thanks,
jungmin
On Thu, Mar 16, 2023 at 7:55 AM Andrés Prados Torreblanca <
***@***.***> wrote:
> Jackpot! Your are facing a conversion problem related to the affine
> transformation used to crop the visual features of the GNN input. This
> operation is used by Spatial Transformer Networks, take a look at this
> issue <pytorch/pytorch#27212>, it seems to be
> already solved.
>
> —
> Reply to this email directly, view it on GitHub
> <#3 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/A3KJF5JAUTZVTP33A6WJTTLW4L5UHANCNFSM6AAAAAAVVMQEJE>
> .
> You are receiving this because you were mentioned.Message ID:
> ***@***.***>
>
|
Hello, I am running into the same problem, and I used the affine_grad reimplementation from pytorch/pytorch#30563. I was able to convert the model to ONNX using opset 16, but when I used it with onnxruntime, the session creation failed: Any idea how I can work with the model in onnxruntime? My ultimate goal is to generate a tensorrt engine from the model, but I am doing it step by step first. Thanks. |
Hi, could you share how to change affine grid reimplementation? |
Hi, To change affine grid, I did it in "spiga\models\spiga.py", in "extract_visual_embedded", replaces "grid = torch.nn.functional.affine_grid(theta, (B * L, C, self.kwindow, self.kwindow))" with the call to the reimplemented method. To set up the input, I did this: model_cfg = ModelConfig('merlrav') Unfortunately, I don't think this is a proper way to do it, as I can't use the onnx model in any way. Neither with OpenCV DNN, nor with ONNXRuntime, nor TensorRT. Any idea how it could be done? |
did u check that original code's input and your input size is same? |
Did anybody able to generate the ONNX model of SPIGA, which successfully runs on ONNX runtime or TensorRT?. Please reply and let us know your changes to convert the model to ONNX type. Any help will be appreciated! |
Someone posted to use
what i add to inference/framework.py
|
No description provided.
The text was updated successfully, but these errors were encountered: