Take image after rendering is finished OR check status of rendering before taking image

[patrickschulte]

Hey,

Is it possible to check the state of rendering from the UE4 before recording via simGetImages()? Often I still get images of the last position, although the engine already shows me the new one. Current workaround is to use a sleep-timer, but to capture a very large number of pictures with the computer vision mode, this is not an efficient solution.

Thanks in advance!

[sytelus]

I haven't seen this happening. Do you have minimal reproducible example?

[patrickschulte]

from math import * 
import numpy as np
import airsim


# Include plotting libraries
import PyQt4
import matplotlib
matplotlib.use('qt4agg')
import matplotlib.pyplot as plt

# Create AirSim Client
Client = airsim.VehicleClient()
Client.confirmConnection()

# Define step-width and inital y-range
y_range = 2
y_step = 1

# Create x-range grid
x_arange = np.arange(0, -5, -1)

# Create empty image array to save all positions
Image_Array = []

# Pyramid-shape for loops
for x in x_arange:
    for y in np.arange(-y_range, y_range+1, y_step):

        # Set pose of vehicle
        Client.simSetVehiclePose(airsim.Pose(airsim.Vector3r(x, y, 0), airsim.to_quaternion(0, 0, 0)), True)
        
        # Get RGBA Image
        Response = Client.simGetImages([airsim.ImageRequest(0, airsim.ImageType.Scene, False, False)])

        # Take only the first element
        Response = Response[0]

        # Create a numpy-object
        Response_np = np.fromstring(Response.image_data_uint8, dtype = np.uint8)

        # Deal it as RBGA-Image
        Response_np = Response_np.reshape(Response.height, Response.width, 4)
        
        # Delete the alpha channel and add it to the image_array
        Image_Array.append(Response_np[:, :, 0:3])

    # Increase the range to get a pyramid shape
    y_range += 1

# For loop over all images
for i in range(len(Image_Array)):
    plt.imshow(Image_Array[i])
    plt.show()

settings.json:

{
  "SeeDocsAt": "https://github.com/Microsoft/AirSim/blob/master/docs/settings_json.md",
  "SettingsVersion": 1.0,
  "ClockSpeed": 1,
  "EngineSound": false,
  "SimMode": "ComputerVision",
  "ViewMode": "",
  "CameraDefaults": {
    "CaptureSettings": [
      {
        "ImageType": 0,
        "Width": 149,
        "Height": 149,
        "FOV_Degrees": 95
      }
    ]
  },
  "PawnPaths": {
    "BareboneCar": {"PawnBP": "Class'/AirSim/VehicleAdv/Vehicle/VehicleAdvPawn.VehicleAdvPawn_C'"},
    "DefaultCar": {"PawnBP": "Class'/AirSim/VehicleAdv/SUV/SuvCarPawn.SuvCarPawn_C'"},
    "DefaultQuadrotor": {"PawnBP": "Class'/AirSim/Blueprints/BP_FlyingPawn.BP_FlyingPawn_C'"},
    "DefaultComputerVision": {"PawnBP": "Class'/AirSim/Blueprints/BP_ComputerVisionPawn.BP_ComputerVisionPawn_C'"}
  },

  "TimeOfDay": {
    "Enabled": false,
    "StartDateTime": "",
    "CelestialClockSpeed": 100,
    "StartDateTimeDst": false,
    "UpdateIntervalSecs": 10
  }
}

PC:
12 Core Processor
32GB RAM
GeForce 1080Ti

Environment: Landscape Mountain

Problem:
Sometimes objects are missing in the image or the old position is captured instead of the new position (even if only a rotation is applied).