In 1997 Debevec and Malik published an interesting work. They presented there a method that allows to estimate the response function of the image formation system based on a series of photos taken at different exposures. This method finds a function that minimizes (in the terms of the least squares) error of solution to the system of linear equations relating pixel brightness to the exposure.
In order to determine the response function of the Nikon D70 digital SLR encoded by the camera as JPGs, three series of shots have been taken. Each series consisted of eleven consecutive shots taken with different exposure settings changed with 1EV step, in the [-5EV, 5EV] interval around the correct exposure value.
The scenes selected for photos contained only static parts and neutral colors. Entire frame was filled with gray colors without elements with high saturation. The scenes contain areas with contrasting brightness to assure a wide distribution of points in [exposure, pixel value] space for single exposure. This was to improve the quality of stitching curves to each other for individual pixels in the least squares method calculations.
Examples of two series of images used in the calculations are shown in the figures below.
Series 1:
Series 2:
The photos were taken using a tripod, sequentially one after the other, in the shortest possible time intervals which were allowed by the software remotely triggering the camera shutter from a smartphone by USB OTG cable. It was important to take the pictures in the shortest possible time, to avoid changes in lighting conditions during shooting. All series of photos were taken on a cloudy day, when the sun was completely behind the clouds. All the shots were taken using a fixed aperture number, and a variable exposure value was obtained by operating the shutter speed. This allowed to avoid the problems associated with the change of depth of field and vignetting. In order to minimize noise component from the sensor, the minimal allowed sensitivity of 200 ISO was set. The size of the images was set to maximum.
In the calculations, the exact values of shutter time displayed by a camera (eg. 1/500, 1/250 or 1/125) were used, even though Debevec and Malik suggested that the time resulting from exponentiation of 2 (eg. 1/512, 1/256 or 1/128 ) is better approximation of the actual exposure time.
In terms of accuracy to performance ratio of the computations, it is essential to select the adequate number of pixels in the image, which will be used in calculations. Nikon D70 has a 6 megapixel sensor, and the biggest images recorded by the camera in Fine JPG quality is 3008×2000 pixels. The system of linear equations which minimizes the error of estimation of the transfer function is of the order: N*P+Zmax-Zmin. The use of all 6,016,000 pixels, in the series of 11 exposures for 8-bit images, would require to allocate about (3008*2000*11+2^8+1)*(2^8+3008*2000)*8/1000/1000/1000=3,185,066 GB of RAM. Maybe in the future it will be possible to perform such calculations on a cell phone, but they are not currently (according to the author’s knowledge) feasible. Actually, to determine this system of equations, it is sufficient to take into account N pixels that obey inequality: N*(P-1)>(Zmax-Zmin), where N – is the number of pixels , P – the number of exposures; and Zmax-Zmin is the maximum difference in brightness of pixels. With 11 exposures and 8-bit images, a sufficient number of pixels would be: N>(2^8-1)/(11-1) that is N>=26.
The required minimum number of pixels needed to calculate the response curve is known, but it is still necessary to determine how to sample the picture. In the paper, Debevec and Malik manually selected the pixels to be taken into account in the calculations. Here, I decided to select for the calculation, the array of 600 (30×20) pixels, spaced at uniform intervals from each other over the image, while maintaining a 5% margin from the edges of the picture. Due to the fact that the number of pixels was an order of magnitude larger than required, the system was characterized by a high redundancy. It occupied (30*20*11^8+2+1)*(2^8+30*20)*8/1000/1000=46.96MB of RAM. Calculations on typical computers were a matter of seconds.
The following figures show response functions obtained for Nikon D70 with JPG Fine encoding for the three (RGB) colors. In order to decrease impact of extreme brightness values for the estimate of the transfer function, a triangular window function defining the importance of the brightness of pixels has been applied. The illustrations also present points in [exposure , pixel value] space, which were used to estimate the response function.
The response curve of a Nikon D70 camera with JPEG format encoding. Series 1.
The response curve of a Nikon D70 camera with JPEG format encoding. Series 2.
The obtained curves are identical. They show the response curve of the camera for JPEG files.
This post is also available in: Polish
Comments