Fisheye lens correction

Written by Paul Bourke
Original November 2016, data being updated regularly

See also: fisheyerectify.pdf (March 2018)
and Classification of fisheye lenses


The source code implementing the projections below is only available on request for a small fee. It includes a demo application and an invitation to convert an image of your choice to verify the code does what you seek. For more information please contact the author.


A normal lens has pincushion or barrel distortion which can be corrected for to give a perfect perspective projection, a pin-hole camera. This is called "rectification" and is often applied before any warp/stitch/blending is applied, for example, in panoramic photography.

In the same way there is a perfect circular fisheye projection, that is, one in which the distance r from the center of the fisheye circle is linearly proportional to the latitude of the corresponding 3D vector. In the following, along any radial line on the fisheye (left), two points separated by the same pixel distance, correspond to the same angular distance (right).

Such a fisheye lens is often referred to as a "tru-theta" lens and while such lenses can and have been manufactured, in real life and for lower cost lenses the relationship is non-linear. The non-linearity normally occurs towards the periphery of the fisheye and results in a compression effect.

In many cases the lens manufacturer can supply data for the curve relating "r", the distance on the sensor or fisheye circular image, to latitude (often called "field angle") of the 3D vector corresponding to that radius. In some cases for high grade lenses the manufacturer will supply curves for the particular lens in question, an acknowledgement that lenses can vary typically by 5% in the manufacturing process. In the situation where no such data is available one needs to construct a rig that will allow the angles to be measured.

The approach to correcting for the non-linear relationship is to fit a suitable polynomial to the data points relating "r" to latitude. A general function for the latitude φ might be

φ(r) = a0 + a1r + a2r2 + ... + airi + ... + anrn

Since the fisheye is assumed to be radially symmetric and r=0 is the center of the fisheye corresponding to a latitude of 0, a0 is zero. In practice the highest order polynomial needed for the fitting is usually n=3, in some extreme examples a 4th order (n=4) is required. So the polynomial for a least squares fit is

φ(r) = a1r + a2r2 + a3r3 + a4r4

The case where the fisheye already has a linear relationship would be a1 = φmax / rmax, a2 = 0, a3 = 0, and a4 = 0. Where φmax is the half the circular fisheye angle and rmax the radius on the image or sensor corresponding to φmax. Noting that the r is often normalised to 1, in which case rmax = 1.

In order to correct a source image, the software actually performs the mapping in the reverse direction. That is, one considers each position on the destination image and derives the matching position and therefore best RGB value in the source image. As such what one actually needs to compute is the radius as a function of field angle, the inverse function of the equations presented earlier, in other words:

r(φ) = b1φ + b2φ2 + b3φ3 + b4φ4

As an example, the following data relating field angle to distance on the sensor is illustrated below for a 190 degree fisheye The samples are in blue and the fitted curve is shown in grey, The red line shows the relationship for a perfectly linear fisheye, this is also the relationship after the correction is performed. The horizontal axis is the field of view (field angle) in radians and the vertical axis is normalised fisheye image coordinates, that is, the radius is unity for the field of view at the fisheye circle.

r(φ) = 0.7284 φ - 0.1461 φ2 + 0.2896 φ3 - 0.2109 φ4

Usage

The usage string for the command line application is shown below. The main option is -p where the fitted parameters are provided. The center of the fisheys circle (-c) and the radius (-r) mean that images directly off the camera sensor can be transformed without first cropping it square. Rotations can be performed about any axis and in any order. The coordinate system is right handed, y is forward along the optical axis of the fisheye (rotation in y is a roll), z is upwards (rotation about z is a pan), and the x axis is to the right (rotation about x is a tilt).

Usage: fishcorrect [options] imagefile
Options
   -w n        sets the output fisheye width, default: same as source fisheye
   -a n        sets antialiasing level, default: 2
   -s n        source fisheye field of view (degrees), default: 180
   -t n        output fisheye field of view (degrees), default: same as source
   -c x y      source fisheye center, default: center of image
   -r n        source fisheye radius, default: half the fisheye image width
   -x n        tilt angle (degrees), default: 0
   -y n        roll angle (degrees), default: 0
   -z n        pan angle (degrees), default: 0
   -ox n       fisheye offset amount in x axis, default: 0
   -oy n       fisheye offset amount in y axis, default: 0
   -oz n       fisheye offset amount in z axis, default: 0
   -p n n n n  fourth order correction terms, default: no correction
   -o s        output filename, default: derived from input file
   -d          debug mode

Updates

  • Increased lens linearity correction to 4th order
  • Added fisheye cropping circle mask (December 2018)
  • STmap file for Nuke (March 2021)
  • Added offset fisheye support (February 2022).

It should be noted that the utility described here transforms a fisheye into a linear fisheye. More often than not one is converting the fisheye into some other mapping, perhaps an equirectangular (fish2sphere) or extracting a perspective projection (fish2persp). In these cases the non-linearity of the fisheye should be incorporated directly into the equations for those mappings, rather than being performed as a separate image transformation process..

Example

The following image is captured with a 210 degree fisheye which has quite significant compression towards the rim.

The following is the undistorted version, that is, radius on the fisheye images is now proportional to latitude.



Lens functions

In the remainder of this document the curves and polynomial correction values for various fisheye lenses are presented. These are collected by the author as lenses and/or data are made available. The functions are all 4th order. Graphs are radians horizontally, and normalised radius of the position on the fisheye image (sensor) vertically.

Entaniya

Entaniya M12 250 degree fisheye

Entaniya HAL 250 (3.0) degree fisheye

Entaniya M12 220 degree fisheye

Entaniya HAL 200 5.0 fisheye

Entaniya M12 280 degree fisheye

While this distortion can be corrected for, it should be noted that the more curvature the less effective resolution one achieves for the widest angles. For many applications this is exactly where one does want resolution, for example, for feature point detection between multiple camera rigs. One should also be aware that if you need 220 degrees then for highly curved functions the relatively small number of pixels for those last 10 or 20 degrees may effectivey mean you only really have a useful lower FOV lens. This can work in ones favour, for example if you only need a 180 degree lens then a 220 fisheye that is linear up to 180 may be a better choice than a 180 degree lens with curvature.

Entaniya RP L220 220 degree fisheye

iZugar

iZugar MKX22, 220 degrees

iZugar MKX200, 200 degrees

iZugar MKX13, 185 degrees
Pretty much perfect f-theta, theta = r*180/(pi*0.5*185)

iZugar MKX19, 190 degrees

Meike

Meike 3.5mm, 220 degree

Meike 6.5mm, 190 degree

Sunex

Sunex miniature DSL315, 190 degree

Sunex miniature DSL239, 185 degree

Sunex miniature DSL415, 195 degree
Pretty much perfect f-theta, theta = r*180/(pi*0.5*195)

Sunex Superfisheye DSLR01, 185 degree

Lensagon

Lensagon BF10M14522S118, 190 degrees

Lensagon BF16M220D, 220 degrees

Aico

Aico ACHIR01028B10M, 245 degree

Aico ACHIR01420B9M, 185 degree
Pretty much perfect f-theta, theta = r*180/(pi*0.5*185)

Evetar

Evetar E3307, 195 degrees

Evetar E3267A, 250 degrees

Evetar E3279, 190 degrees

Arducam

ArduCam part number M25156H18, SKU number LN031, 180 degrees

ArduCam part number M32076M20, SKU number LN010, 220 degrees

Computar

Computar E1228KRY, 185 degrees, stereographic

Computar E1628KRY, 190 degrees

Computar E2328KRY, 190 degrees

Commonlands

Commonlands CIL220, 215°@6.8mm M12 Fisheye Lens
Pretty much perfect f-theta, theta = r*180/(pi*105)

Commonlands CIL227, 185°@7.8mm Fisheye Lens

Commonlands CIL207, 220°@4.0mm FIsheye lens

Xiamen Alaud Optical

ABFE01320, 190 degrees

ABFE01514, 185 degrees

ABFE01720, 200 degrees

ABFE08420, 200 degrees

ABFE09620, 210 degrees

ABFE010528, 245 degrees

Others

Bosch Flexidome IP Panoramic 7000 MP, 180 degrees

Arecont Vision MPL1.55 1.55mm, 185 degrees

Raynox DCR-CF187, 180 degrees

Theia TY180, f1.32

Laowa 4mm, 210 degrees

SMTEC SL-190, 180 degrees

Nikkor 8mm f2.8, 180 degrees

Omnitech ORIFL190-3, 190 degrees

DZO, VRCA 220 degree

Ideal stereographic lens 2tan(theta/2) (185 degrees).

Sigma

Sigma f2.5, 4.5mm, 180 degrees

Sigma seems to be one of those companies who refuse to provide field angle data on their lenses. Totally unreasonable, this is a fundamental characteristic of a lens that an owner has every right to insist upon. Fortunately one can determine it with only a modest effort. The equipment required is shown below, the procedure is as follows

  • Determine the zero parallax position for the lens, this is the position the lens will be rotated about in subsequent steps. The standard way of doing this is to line up a close object and a distant one, adjust the position of the camera on the slider rail until the two objects stay aligned despite rotations of the camera/lens. In the case here of the Sigma 4.5mm you can see below that zero parallax point is closer to the front of the lens than is usual.

  • Choose an object in the scene and align it mid height and width on the lens. I do this with a simple Vuo composition with cross hairs and taking a live HDMI feed from the camera. But simpler, most cameras will have a cross hair or alignment grid.

  • Rotate the camera/lens through the field of view of the lens in 5 degree (or 10 degree) steps and capture a photograph at each step. In the case of the Sigma 4.5mm this is 180 degrees.

  • For each photograph (angle) measure the distance in pixels from the center horizontally to the object chosen, plot this distance against angle and fit a function of your choice. Note in this case I have rotated from -90 to 90 degrees, while the graphs found elsewhere in this document are from 0 to half the field of view.

Sigma f3.5, 8mm, 180 degrees

Canon

Canon 8-15mm, 8mm and 12mm position

Canon is another company who behave badly and don't supply this fundamental information of their products. It is more complicated with this lens since the field of view and the curves change with different zoom values. So there is actually a whole family of curves required in order to deal with any particular zoom value, alternatively one could create a 3D surface fit. This could be automated given that the focal length is supplied in the exif data on the photograph. Two curves are provided, based upon the usual use of the lens by the author: fully zoomed out (8mm) and 12mm is the maximum zoom to fill full frame Canon5D Mk III sensor horizontally.

Canon RF 5.2mm f/2.8L Dual Fisheye
Pretty much perfect f-theta, theta = r*180/(pi*95)