Rendering transparent/translucent objects in DepthPerspective

[VC86]

For training data generation purposes, I would need to render transparent or translucent objects such as glasses, windows, etc. so that they would show up in the DepthPerspective and SurfaceNormals image channels of AirSim.

• Unfortunately, since transparent materials do not render in the SceneDepth buffer, they will not show up by default in the DepthPerspective image (a simple check is obtained by switching the visualisation mode to the SceneDepth buffer in the visualisation). The root cause seems to be that transparent materials are not rendered with the standard rendering pass, but handled separately by UE4.

• A possible workaround would be to use the CustomDepth buffer. Indeed, by ticking the appropriate material setting one can see a CustomDepth buffer with, e.g., windows showing up in depth.

• The next step then would be to replace Z in the XYZ coordinates obtained in the DepthPerspectiveMaterialFunction. Since we take the length from the camera view, we would have to join (X,Y) with Z from the CustomDepth buffer as obtained from the SceneTexture widget. I was not able to do that successfully.

• For now I managed to only modify the the standard DepthPerspectiveMaterialFunction with the SceneTexture:CustomDepth (scaled to meters). However, this is not a DepthPerspective (and I really needed that). I thought of dividing that by the camera unit ray's z coordinate to obtain the appropriate scaling. I was not able to do that successfully.

Did anyone look into the issue before and has tips/solutions so that AirSim can return translucent objects as mentioned above?

[stale]

This issue has been automatically marked as stale because it has not had activity from the community in the last year. It will be closed if no further activity occurs within 20 days.

[lhjackgod]

For training data generation purposes, I would need to render transparent or translucent objects such as glasses, windows, etc. so that they would show up in the DepthPerspective and SurfaceNormals image channels of AirSim.出于训练数据生成目的，我需要渲染透明或半透明的对象，例如眼镜、窗户等，以便它们显示在 AirSim 的 DepthPerspective 和 SurfaceNormals 图像通道中。

• Unfortunately, since transparent materials do not render in the SceneDepth buffer, they will not show up by default in the DepthPerspective image (a simple check is obtained by switching the visualisation mode to the SceneDepth buffer in the visualisation). The root cause seems to be that transparent materials are not rendered with the standard rendering pass, but handled separately by UE4.遗憾的是，由于透明材质不在 SceneDepth 缓冲区中渲染，因此默认情况下它们不会显示在 DepthPerspective 图像中（通过在可视化中将可视化模式切换到 SceneDepth 缓冲区，可以进行简单检查）。根本原因似乎是透明材质不是使用标准渲染通道渲染的，而是由 UE4 单独处理的。
• A possible workaround would be to use the CustomDepth buffer. Indeed, by ticking the appropriate material setting one can see a CustomDepth buffer with, e.g., windows showing up in depth.一种可能的解决方法是使用 CustomDepth 缓冲区。事实上，通过勾选适当的材质设置，可以看到 CustomDepth 缓冲区，例如，窗口显示得很深。
• The next step then would be to replace Z in the XYZ coordinates obtained in the DepthPerspectiveMaterialFunction. Since we take the length from the camera view, we would have to join (X,Y) with Z from the CustomDepth buffer as obtained from the SceneTexture widget. I was not able to do that successfully.下一步是替换在 DepthPerspectiveMaterialFunction 中获得的 XYZ 坐标中的 Z。由于我们从摄像机视图获取长度，因此我们必须将 （X，Y） 与 CustomDepth 缓冲区中的 Z 联接，如从 SceneTexture 小部件获取。我没能成功地做到这一点。
• For now I managed to only modify the the standard DepthPerspectiveMaterialFunction with the SceneTexture:CustomDepth (scaled to meters). However, this is not a DepthPerspective (and I really needed that). I thought of dividing that by the camera unit ray's z coordinate to obtain the appropriate scaling. I was not able to do that successfully.现在，我只设法使用 SceneTexture：CustomDepth（缩放到米）修改了标准的 DepthPerspectiveMaterialFunction。但是，这不是 DepthPerspective（我真的需要它）。我想到将其除以相机单元光线的 z 坐标以获得适当的缩放。我没能成功地做到这一点。

Did anyone look into the issue before and has tips/solutions so that AirSim can return translucent objects as mentioned above?之前有没有人研究过这个问题并有提示/解决方案，以便 AirSim 可以返回如上所述的半透明对象？

bro do you have any solutions? I want to get the translucents' depth and seg , but i do not konw how to get it