The Virtual World

Building a Virtual World

• Goal: mimic human vision in a virtual world (with a computer)
• Cheat for efficiency, using knowledge about light and the eye (e.g. from the last lecture)
• Create a virtual camera: place it somewhere and point it at something
• Put film (containing pixels, with RGB values ranging from 0-255) into the camera
• Taking a picture creates film data as the final image
• Place objects into the world, including a floor/ground, walls, ceiling/sky, etc.
• Two step process: (1) make objects (geometric modeling), (2) place objects (transformations)
• Making objects is itself a two-step process: (1) build geometry (geometric modeling), (2)
paint geometry (texture mapping)
• Put lights into the scene (so that it’s not completely dark)
• Finally, snap the picture:
• “Code” emits light from (virtual) light sources, bounces that light off of (virtual) geometry, and
follows that bounced light into the (virtual) camera and onto the (virtual) film
• We will consider 2 methods (scanline rendering and ray tracing) for the taking this picture
 Pupil
• Light emanates off of every point of an object outwards in every direction
• That’s why we can all see the same spot on the same object
• Light leaving that spot/point (on the object) is entering each of our eyes
• Without a pupil, light from every point on an object would hit the same cone on our eye,
averaging/blurring the light information
• The (small) pupil restricts the entry of light so that each cone only receives light from a small
region on the object, giving interpretable spatial detail
 Aperture
• Cameras are similar to the eye (with mechanical as opposed to biological components)
• Instead of cones, the camera has mechanical pixels
• Instead of a pupil, the camera has a small (adjustable) aperture for light to pass through
• Cameras also typically have a hefty/complex lens system
 Aside: Lens Flare
• Many camera complexities are (often) not properly accounted for in virtual worlds
• Thus, certain effects (such as depth of field, motion blur, chromatic aberration, lens flare, etc.)
have to be approximated/modeled in other ways (as we will discuss later)
• Example: Lens flare is caused by a complex lens system reflecting/scattering light
• This depends on material inhomogeneities in the lenses, the geometry of lens surfaces, absorption/dispersion
of lenses, antireflective coatings, diffraction, etc.
 Pinhole Camera
• The pupil/aperture has to have a finite size in order for light to be able to pass through it
• When too small, not enough light enters and the image is too dark/noisy to interpret
• In addition, light can diffract (instead of traveling in straight lines) distorting the image
• When too large, light from a large area of an object hits the same cone (causing blurring)
• Luckily, the virtual camera can use a single point for the aperture (without worrying about dark
or distorted images)
 Aside: Diffraction
• Light spread out as it goes through small openings
• This happens when the camera aperture is too small (diffraction limited)
• It leads to constructive/destructive interference of light waves (the Airy disk effect)
 Pinhole Camera (a theoretical approximation)
• Light leaving any point travels in straight lines
• We only care about the lines that hit the pinhole (a single point)
• Using a single point gives infinite depth of field (everything is in focus, no blurring)
• An upside-down image is formed by the intersection of these lines with an image plane
• More distant objects subtend smaller visual angles and appear smaller
• Objects occlude objects behind them
 Virtual Camera
• Trick: Move the film out in front of the pinhole, so that the image is not upside down
• Only render (compute an image for) objects further away from the camera than the film plane
• Add a back clipping plane for efficiency
• Volume between the film (front clipping plane) and the back clipping plane is the viewing
frustum (shown in blue)
• Make sure that the near/far clipping planes have enough space between them to contain the scene
• Make sure objects are inside the viewing frustum
• Do not set the near clipping plane to be at the camera aperture!
 Camera Distortion depends on Distance
• Do not put the camera too close to objects of interest!
• Significant/severe deductions for poor camera placement, fisheye, etc. (because the image looks terrible)
• Set up the scene like a real-world scene!
• Get very familiar with the virtual camera!
 Eye Distortion?
• Your eye also has distortion

• Unlike a camera, you don’t actually see the signal received on the cones
• Instead, you perceive an image (highly) processed by your brain
• Your eyes constantly move around obtaining multiple images for your brain to work with
• You have two eyes, and see two images (in stereo), so triangulation can be used to estimate
depth and to undo distortion

• If your skeptical about all this processing, remember that your eye sees this:
 Dealing with Objects
𝑥
• Let’s start with a single 3D point 𝑥⃗ = 𝑦 and move it around in the virtual world
𝑧
• An object is just a collection of points, so methods for handling a single point extend to
handling entire objects

• Typically, objects are created in a reference space, which we refer to as object space
• After creation, we place objects into the scene, which we refer to as world space
• This may require rotation, translation, resizing of the object

• When taking a (virtual) picture, points on the object are projected onto the 2D film plane,
which we refer to as screen space
• Unlike rotation/translation/resizing, the projection onto screen space is highly nonlinear and
the source of undesirable distortion
 Rotation
𝑥
• Given a 3D point, 𝑥⃗ = 𝑦
𝑧
• In 2D, one can rotate a point counter-clockwise about the origin via:

𝑥 !"# cos 𝜃 − sin 𝜃 𝑥 𝑥

!"# = 𝑦 = 𝑅(𝜃) 𝑦
𝑦 sin 𝜃 cos 𝜃

• This is equivalent to rotating a 3D point around the z-axis using (i.e. multiplying by):

cos 𝜃 − sin 𝜃 0
𝑅$ 𝜃 = sin 𝜃 cos 𝜃 0
0 0 1
 Rotation
• To rotate a 3D point around the x-axis, y-axis, z-axis (respectively), multiply by:

1 0 0
𝑅" 𝜃 = 0 cos 𝜃 − sin 𝜃
0 sin 𝜃 cos 𝜃 cos 𝜃 0 sin 𝜃
𝑅! 𝜃 = 0 1 0
− sin 𝜃 0 cos 𝜃
cos 𝜃 − sin 𝜃 0
𝑅# 𝜃 = sin 𝜃 cos 𝜃 0
0 0 1

• Matrix multiplication doesn’t commute, i.e. 𝐴𝐵 ≠ 𝐵𝐴, so the order of rotations matters!
• Rotating about the x-axis and then the y-axis, 𝑅! 𝜃! 𝑅" 𝜃" 𝑥,
⃗ is different than rotating about
the y-axis and then the x-axis, 𝑅" 𝜃" 𝑅! 𝜃! 𝑥⃗
• 𝑅! 𝜃! 𝑅" 𝜃" 𝑥⃗ ≠ 𝑅" 𝜃" 𝑅! 𝜃! 𝑥⃗ because 𝑅! 𝜃! 𝑅" 𝜃" ≠ 𝑅" 𝜃" 𝑅! 𝜃!
 Line Segments are Preserved
• Consider two points 𝑝⃗ and 𝑞⃗ and the line segment between them:
𝑢 𝛼 = 1 − 𝛼 𝑝⃗ + 𝛼𝑞⃗
• 𝑢 0 = 𝑝⃗ and 𝑢 1 = 𝑞,
⃗ and 0 ≤ 𝛼 ≤ 1 specifies all the points on the line segment

• Multiplying points on the line segment by a rotation matrix 𝑅 gives:

𝑅𝑢 𝛼 = 𝑅 1 − 𝛼 𝑝⃗ + 𝛼𝑞⃗ = 1 − 𝛼 𝑅𝑝⃗ + 𝛼𝑅𝑞⃗
• 𝑅𝑢 0 = 𝑅𝑝⃗ and 𝑅𝑢 1 = 𝑅𝑞,
⃗ and 0 ≤ 𝛼 ≤ 1 specifies all the points connecting 𝑅𝑝⃗ and 𝑅𝑞⃗
• i.e., only need to rotate the endpoints in order to construct the new line segment (connecting them)

• 𝑅𝑝⃗$ − 𝑅𝑝⃗% %% = 𝑅 𝑝⃗$ − 𝑝⃗% %% = 𝑝⃗$ − 𝑝⃗% & 𝑅& 𝑅 𝑝⃗$ − 𝑝⃗% = 𝑝⃗$ − 𝑝⃗% %% shows that the
distance between two rotated points is equivalent to the distance between the two original (un-
rotated) points
 Angles are Preserved
• Consider two line segments 𝑢 and 𝑣⃗ with 𝑢 ⋅ 𝑣⃗ = 𝑢 % 𝑣⃗ % cos(𝜃) where 𝜃 is the angle
between them

• 𝑅𝑢 ⋅ 𝑅𝑣⃗ = 𝑅𝑢 % 𝑅𝑣⃗ % cos(𝜃) >

• 𝑅𝑢 ⋅ 𝑅𝑣⃗ = 𝑢& 𝑅& 𝑅𝑣⃗ = 𝑢& 𝑣⃗ = 𝑢 % 𝑣⃗ % cos 𝜃 = 𝑅𝑢 % 𝑅𝑣⃗ % cos(𝜃)
• So, the angle 𝜃 between 𝑢 and 𝑣⃗ is the same as the the angle 𝜃> between 𝑅𝑢 and 𝑅𝑣⃗
 Shape is Preserved
• In continuum mechanics, material deformation is measured by a strain tensor
• The six unique entries in the nonlinear Green strain tensor are computed by comparing an
undeformed tetrahedron to its deformed counterpart
• Given a tetrahedron in 3D, it is fully determined by one point and three line segments (the
dotted lines in the figure)

• The 3 lengths of these three line segments and the 3 angles between any two of them are
used to compare the undeformed tetrahedron to its deformed counterpart
• Since we proved these were all identical under rotations, rotations are shape preserving
 Shape is Preserved
• Thus, we can rotate entire objects without changing them
 Scaling (or Resizing)
𝑠$ 0 0
• A scaling matrix 𝑆 = 0 𝑠% 0 can both scale and shear the object
0 0 𝑠'
• Shearing changes lengths/angles creating significant distortion

𝑠 0 0
• When 𝑠$ = 𝑠% = 𝑠', then 𝑆 = 0 𝑠 0 = 𝑠𝐼 is pure scaling
0 0 𝑠
• The distributive law of matrix multiplication (again) guarantees that line segments map to line
segments
• 𝑆𝑝⃗$ − 𝑆𝑝⃗% %% = 𝑠 𝑝⃗% − 𝑝⃗% %% implies that the distance between scaled points is
increased/decreased by a factor of 𝑠
• 𝑆𝑢 ⋅ 𝑆𝑣⃗ = s %𝑢 ⋅ 𝑣⃗ = 𝑠 % 𝑢 % 𝑣⃗ % cos 𝜃 = 𝑆𝑢 % 𝑆𝑣⃗ % cos(𝜃) shows that angles between
line segments are preserved
• Thus, uniform scaling grows/shrinks objects proportionally (they are mathematically similar)
 Scaling (or Resizing)

non-uniform

uniform

uniform
 Homogenous Coordinates
• In order to use matrix multiplication for transformations, homogeneous coordinates are
required
𝑥 𝑥𝑤
𝑦𝑤
• The homogeneous coordinates of a 3D point 𝑥⃗ = 𝑦 are 𝑥⃗( = 𝑧𝑤 for any 𝑤 ≠ 0
𝑧 𝑤 𝑥
𝑦
• Dividing homogenous coordinates by the fourth component (i.e. 𝑤) gives 𝑧 or 𝑥⃗
1
𝑥 1
𝑦
• 3D points are converted to 𝑥⃗( = 𝑧 , with 𝑤 = 1, to deal with translations
1 𝑢$
𝑢$
𝑢
• Vectors 𝑢 = 𝑢% have homogenous coordinates 𝑢( = 𝑢% or 𝑢
𝑢' ' 0
0
 Homogenous Coordinates
• Let 𝑀'"' be a 3x3 rotation or scaling matrix (as discussed previously)
• The transformation of a point 𝑥⃗ is given by 𝑀'"'𝑥⃗

0 𝑥
𝑀'"' 𝑦 𝑀'"'𝑥⃗
• To obtain the same result for 𝑥⃗ , use a 4x4 matrix 0
𝑧 =
1 0 1
0 0 0 1 1

0 𝑢$
𝑀'"' 0 𝑢% 𝑀'"'𝑢
• Similarly, for a vector 𝑢' =
0 0
0 0 0 1 0
 Translation
𝑡$ 𝑥
𝑡$
𝐼'"' 𝑡% 𝑦 𝑥⃗ + 𝑡⃗
• To translate a point 𝑥⃗ by 𝑡⃗ = 𝑡% , multiply 𝑧 =
𝑡' 1
𝑡'
0 0 0 1 1
1 0 0
• 𝐼#"# = 0 1 0 is the 3x3 identity matrix
0 0 1

𝑡$ 𝑢$
𝐼'"' 𝑡% 𝑢% 𝑢 has no effect (as desired)
• For a vector 𝑢' =
𝑡' 0
0 0 0 1 0

• Translation preserves line segments and the angles between them (and thus preserves shape)
 Shape is Preserved
• We can translate entire objects without changing them
 Composite Transforms
• Rotate 45 degrees about the point (1,1)

T(-1,-1) T(1,1)
R(45)

• These transformations can be multiplied together to get a single matrix M=T(1,1)R(45)T(-1,-1)

that can be used to multiply every relevant point in the (entire) object:
 Order Matters
• Matrix multiplication does not commute: 𝐴𝐵 ≠ 𝐵𝐴
• The rightmost transform is applied to the points first

T(1,1)R(45) ≠ R(45)T(1,1)
 Hierarchical Transforms
• M1 transforms the teapot from its object space to the table’s object space (puts it on the table)
• M2 transforms the table from its object space to world space
• M2M1 transforms the teapot from its object space to world space (and onto the table)

𝑀" 𝑀!

𝑀!

𝑀"
 Using Transformations
• Create objects (or parts of objects) in convenient coordinate systems
• Assemble objects from their parts (using transformations)
• Transform the assembled object into the scene (via hierarchical transformations)
• Can make multiple copies (even of different sizes) of the same object (simply) by adding
another transform stack (efficiently avoiding the creation of a new copy of the object)

• Helpful Hint: Always compute composite transforms for objects or sub-objects, and apply the
single composite transform to all relevant points (it’s a lot faster)

• Helpful Hint: Orientation is best done first:

• Place the object at the center of the target coordinate system, and rotate it into the desired orientation
• Afterwards, translate the object to the desired location
 Screen Space Projection
• Projecting geometry from world space into screen space can create significant distortion
$
• This is because # is highly nonlinear
(x,y,z)
film plane

pinhole x
x’ (x’,y’,h)
z-axis
z
h

and
 Matrix Form
𝑥 * 𝑤′
𝑦 * 𝑤′ ! * =𝑧
• Writing the screen space result as gives the desired after dividing by 𝑤
𝑧 * 𝑤′ "
𝑤′

𝑥 * 𝑤′ ℎ 0 0 0 𝑥
𝑦 * 𝑤′ 0 ℎ 0 0 𝑦
• Consider: = 𝑧
*
𝑧 𝑤′ 0 0 𝑎 𝑏
𝑤′ 0 0 1 0 1
+" +!
• This has 𝑤 * = 𝑧, 𝑥 * 𝑤 * = ℎ𝑥 or 𝑥 * * * *
= # , and 𝑦 𝑤 = ℎ𝑦 or 𝑦 = # (as desired)

.
• Homogenous coordinates allows the nonlinear to be expressed with linear matrix
$
multiplication (so it can be added to the matrix multiplication stack!)
 Perspective Projection
• The third equation is 𝑧 * 𝑤 * = 𝑎𝑧 + 𝑏 or 𝑧 * 𝑧 = 𝑎𝑧 + 𝑏
• New 𝑧 values aren’t required (projected points all lie on the 𝑧 = ℎ image plane)
• However, computing 𝑧′ as a monotonically increasing function of 𝑧 allows it to be used to
determine occlusions (for alpha channel transparency)

• The near (𝑧 = 𝑛) and far (𝑧 = 𝑓) clipping planes are preserved via 𝑧′ = 𝑛 and 𝑧′ = 𝑓
• 2 equations in 2 unknowns (𝑛% = 𝑎𝑛 + 𝑏 and 𝑓 % = 𝑎𝑓 + 𝑏); so, 𝑎 = 𝑛 + 𝑓 and 𝑏 = −𝑓𝑛

• This transforms the viewing frustum into an orthographic volume in screen space