Exception when creating a TextureAtlas

[iahmedmaher]

Hello,

When I try to load a multi-textured object using the instructions here, an exception is thrown while loading with these messages as seen in the attached image below:

I tried to look into the mtl_io.py file and found that some values of x1 are out of the image's range by 1. Setting those to the max possible index loads and renders the image correctly for a human, but I don't know if some faces have wrong textures or not especially that I don't fully understand uv texturing in general. I checked how x1 gets its values and it seems weird to me that it is x0 + 1 while x0 have values (H - 1) as seen below:

Environment

I am using the environment setup in one of the tutorial notebooks, so the latest stable version of Pytorch3D as of the time of writing this issue.

I am using Google Colab, so it's on Linux.

The issue is present when using a GPU. When using a CPU instead, the notebook crashes.

The texture images have size of 4096x4096. I am not authorized to share them or the obj files, sorry.

[nikhilaravi]

Hi @iahmedmaher apologies for the late response. Are you setting texture_wrap="repeat" in the call to load_obj? Please provide a code snippet so we can help debug further.

[iahmedmaher]

@nikhilaravi Here is my code snippet. This is the same snippet shared here. I have tried all texture_wrap options and different texture_atlas_size sizes, but none of them worked.

verts, faces, aux = load_obj(
obj_filename,
device=device,
load_textures=True,
create_texture_atlas=True,
texture_atlas_size=8,
texture_wrap=None
)
atlas = aux.texture_atlas

mesh = Meshes(
verts=[verts],
faces=[faces.verts_idx],
textures=TexturesAtlas(atlas=[atlas]),
)

[iahmedmaher]

@nikhilaravi If you can tell me whether this is an issue with Pytorch3D or not, I would be really grateful.

[nikhilaravi]

@iahmedmaher the device side assert is due to the UV values being out of bounds for the image during indexing.

We have to convert the UV coordinates to integers in order to index into the texture map. x0, x0+1, y0, y0+1 are the indices of the neighbors for each of the interpolation points however this doesn't properly handle the out of bound values in the same way as the grid_sample function does with the padding_mode argument.

Can you try replacing this line with _bilinear_interpolation_grid_sample?:

pytorch3d/pytorch3d/io/mtl_io.py

Line 302 in 44d2a9b

textures = _bilinear_interpolation_vectorized(image, uv_pos)

If this works I will update the original implementation to use this grid sample version.

[iahmedmaher]

@nikhilaravi That did the trick! Thanks a lot for your time. I am leaving the issue as opened since you said this will require changing the original implementation.

[nikhilaravi]

@iahmedmaher great!! I will update the original implementation!

[nikhilaravi]

This has now been fixed in fc156b5.