problem with 'obj_texture'

[caramel-jiao]

I read the .pkl meta file of the obman release dataset and looked at the texture image of the 'obj_texture' path, but found that it does not correspond to the texture of rendered image in rgb_obj.

I want to know how can I obtain correct texture images of dataset samples? Thanks.

[strongBrave]

Did you solve it, I encountered the same problems.

[caramel-jiao]

No I haven't 😥. But I have tried to directly adopt my own rendered images with such textures for some pretraining, and found it also helpful for my tasks.

[strongBrave]

Cause I am right now doing the inverse engineering, which is restore the 3D texture model from the obj file and texture picture in the meta-info. However, due to the problem, this doesn't work.

[caramel-jiao]

Perhaps using the data obtained by re-rendering according to the existing annotation can help alleviate this problem for you. I may not have any other better ideas either.😣

[strongBrave]

Can you specify what's the meaning of 're-rendering', I think i don't undetstand.

[caramel-jiao]

Sorry maybe I didn't explain clearly. It can be understood as following the guidelines of the current repo (https://github.com/hassony2/obman_render?tab=readme-ov-file#launch-) to generate obman dataset from scratch according to the original workflow.
In this way, because both the texture and background are obtained from the files you specify, you can ensure that the data you render has the correct annotations.

[strongBrave]

Cause I am right now doing the inverse engineering, which is restore the 3D texture model from the obj file and texture picture in the meta-info. However, due to the problem, this doesn't work.

I think I found the reason.

obman_render/blender_grasps.py

Lines 226 to 245 in d64c9a6

	for mat_idx, obj_mat in enumerate(obj_mesh.materials):
	materials_tmp.append(obj_mat)
	if random_obj_textures:
	obj_texture = random.choice(obj_textures)
	generated_uv = True
	else:
	obj_texture = os.path.join(
	os.path.dirname(obj_path), 'texture.jpg')
	generated_uv = False
	obj_sh_path = texturing.add_obj_texture(
	obj_mat,
	obj_texture,
	sh_path,
	down_scale=texture_zoom,
	tmp_suffix='tmp',
	generated_uv=generated_uv)
	tmp_files.append(obj_sh_path)
	tmp_files.append(obj_sh_path.replace('.osl', '.oso'))
	obj_scs.append(obj_mat.node_tree.nodes['Script'])
	obj_scs[-1].update()

obman_render/blender_grasps.py

Lines 267 to 268 in d64c9a6

	if random_obj_textures:
	hand_info['obj_texture'] = obj_texture

You can see these lines apply each mat with corresponding texture picture, but the meta_info only save the path of the last texture picture. Consequently, the object's appearance is determined by the combined effect of all material textures, which explains why the rendered image's texture differs from the single texture image referenced in the saved path.