How to fix Image aliasing issue?

[ShukuiZhang]

Hi,
I've been recording images for the dataset. However, it looks the images I recorded has significant aliasing. E.g. I get the image below:

and converting to grayscale and zooming the image, you can see the aliasing as below:

This prevents us to go any further for image features. Is there any way that I can maximize the image quality? I tried "pixel_as_float" and uncompressed version, but neither seems to work well... Thanks.

Update: I tried other method in UE4 for anti-aliasing, e.g. MSAA and FXAA, but do not see salient improvement.

[Robin-Wong]

Try to modify the settings.json for large resolution and please remember the rpcClient get RGBA image, unless you modify the file RenderRequest.cpp for uncompress. For pixel_as_float, I think AirSim only trans R channel with float32.

[ShukuiZhang]

Thanks for your reply. @Robin-Wong I checked the code and you are right. But did you meet the same problem and fix them at your end? I don't think changing the resolution can make a change (Actually, I did so but no help).

On the other hand, it looks that this may be a problem more about AirSim because if I take direct screenshot of the scene in the main window, I can see much less jaggies...

[sytelus]

This very much looks like lack of resolution (as you are zooming). Also see here: https://answers.unrealengine.com/questions/293017/scenecapturecube-and-its-anti-aliasing.html. Note that images from Unreal takes much more longer to generate as you increase resolution. You might want to do two test shots at different resolutions and compare them to see if this is the cause.

[ShukuiZhang]

Hi @sytelus The link you provided seems promising. Can you give some hint where to find the "Render Target Texture"? I think previously it comes with the FpvRenderTarget, but it was removed. Thanks.

[sytelus]

Render target gets set up here: https://github.com/Microsoft/AirSim/blob/master/Unreal/Plugins/AirSim/Source/PIPCamera.cpp#L122

Same function also sets up SceneCapture2D (which is just like camera but allows to get image out of it).

You can set properties there as you like and experiment. I was searching about SceneCapture2D and anti-aliasing and there seems to be lots of forum posts that anti-aliasing wasn't working. I would suggest to take look at them as well just to see if any of the settings are missing.

[ShukuiZhang]

I have been searching for the anti-aliasing stuff for yesterday, even with the best settings I can have, the captured image still has jaggy. I also tried to modify the settings @sytelus mentioned, with width = 1920 and height = 1080, I can see the improvement, but seems still not good enough. The following is the captured image:
Default PIPCamera.cpp:

After tuning settings per @sytelus :

The raw screenshot from UE4:

Out of the three, it looks the ue4 screenshot has the best quality of all. So I was wondering if AirSim implicitly degrades the quality of the captured image somewhere?

[paulerikf]

@jonyMarino I realize this issue is closed, but hopefully it's ok if I still post this here:

Adding capture->ShowFlags.SetTemporalAA(true); before line 402 got AntiAliasing working for me.

[jonyMarino]

Thanks a lot @paulerikf! I closed the issue due to inactivity. Do you want to place a PR with your fix?

[paulerikf]

@jonyMarino I've been away for a bit, sorry. I hesitated to place a PR because I wasn't sure if having AA should be the default behavior, or if it should be configurable in some way... (if so, I haven't put in the time to figure out how that'd be done).

If having AA hardcoded on is fine I can easily add a PR, if it should be configurable I'd need some time to figure that out.

[jonyMarino]

@paulerikf You are right. This should be configurable. This impacts performance, and I don't know if it affects another type of image like the depth one.
The settings.json would be the right place to create a new parameter for this.