The white cross is the nadir (a dotted line, the ground track); click any of the four to pick a point on the ground

playproj

Sensor Height above the datum Tilt from the vertical Looking towards Field of view

Terrain Relief DEM for the orthorectification

True ortho (mask hidden ground) Displacement arrows Roads from the map Image in the scene

Click the terrain, or any of the four images, to follow one point on the ground from the map into the image and back.

How it works

A map projection takes the ground to a plane by a rule. A picture of the ground does it by a ray: every pixel is a line from the sensor's perspective centre out through the lens, and the pixel shows whatever that line meets first. On flat ground that is a projective transformation (a plane to a plane, which is why four control points can georeference an aerial photo of a flat place). On terrain it is not, because where a ray meets the ground depends on how high the ground is.

That dependence is relief displacement. Follow a ray from the sensor through a point P at height h and carry it on down to a reference surface at height z: it arrives a distance (h - z) tan(theta) beyond P, where theta is the ray's angle from the vertical. So a picture rectified onto a flat datum puts every hill top too far from the nadir (the point straight below the sensor), the higher the further. For a vertical photo from height H that is the textbook D h / (H - h), D the ground distance from the nadir: tall things lean out from the centre of the photo. Straight roads over the hills come out bent.

Orthorectification undoes it with two things: a sensor model (where the sensor was, how it was pointed, how its pixels map to directions) and a DEM. For each cell of the output map, take (x, y), read z from the DEM, and run (x, y, z) forward through the sensor model to an image position, then read the image there. For a frame camera that model is the collinearity equations: with the perspective centre C, the camera's right, down and forward axes r, d, f and the principal distance c (in pixels), u = cx + c (P - C).r / (P - C).f and v = cy + c (P - C).d / (P - C).f. Nothing is traced backwards; the DEM supplies the height the ray could not.

A pushbroom scanner (most Earth observation satellites) has one line of detectors and builds the image a row at a time as it moves along its track. Each row has its own perspective centre, so the model is a line camera per row. With a fixed attitude the scan plane is always square to the track, so relief moves points across the track only: in the raw image the hills lean sideways, never up or down the strip. Real satellite products often ship the model as rational polynomial coefficients (RPCs) fitted to this physical model; GDAL's gdalwarp -rpc -to RPC_DEM=dem.tif is the same loop over output cells as here.

The DEM has to be right. A smoothed DEM leaves the peaks displaced by their height above the smoothing; a level plane at the mean height fixes the average and leaves the rest. And ground the sensor never saw (the back of a ridge, the ground behind a spire) has no pixel of its own: an ordinary orthorectification fills it from whatever was in front, a doubled image of the slope. A true ortho tests each cell against the DEM, marching from the ground towards the sensor, and masks it (magenta) instead. Buildings in city orthophotos are the same story at a smaller scale.

Everything here is in a local frame in metres, x east, y north, z up, on a synthetic terrain over a 10 km square (a DEM on a 50 m grid, read bilinear). The raw image is traced: one ray per pixel, marched to the ground and bisected, coloured by the map (height tint, hill shading, a kilometre grid and four straight roads). The two rectified panels use the same image and the same sensor model, onto the datum (z = 0) and onto the chosen DEM. The code is src/core/ortho.js, checked in test/check-ortho.mjs (the sensor models invert their own rays, displacement is (h - z) tan(theta), and the true DEM reproduces the map).

Hash keys: s (frame, push), alt (km), tilt, az, fov, terr, rel, dem, occ, arr, ov, img, px, py (the picked point, metres), cam.

Collinearity and relief displacement as in any photogrammetry text, for example Wolf, P. R., Dewitt, B. A. and Wilkinson, B. E. (2014) Elements of Photogrammetry with Applications in GIS, 4th edition, McGraw-Hill.
Pushbroom models: Gupta, R. and Hartley, R. I. (1997) Linear pushbroom cameras, IEEE Transactions on Pattern Analysis and Machine Intelligence 19(9), 963-975. RPC orthorectification as in GDAL's gdalwarp.
Drawn with three.js.