Statistical Analysis and Modeling of Luminance/Chrominance and Range/Disparity
The 3D Natural Scene Statistics research at LIVE is being conducted in collaboration with Center for Perceptual Systems (CPS, http://www.cps.utexas.edu ).
Introduction
Natural scene statistics (NSS) are important factors both towards understanding the evolution of the human vision system and for designing image processing algorithms. Extensive research has been conducted to explore the link between NSS and neural processing of visual stimuli.
With the increasing popularity of 3D image and video content, the statistics between 3D depth and 2D color image data in natural scenes are of high interest. However, very little work has been done due to the limited access to high quality databases of color images and associated ground-truth range maps.
Towards obtaining a better understanding of the statistical relationships between color and range, we studied both the marginal and joint statistics of luminance/chrominance and range/disparity using the high-resolution color images and accurately co-registered dense ground-truth range maps in the LIVE Color+3D Database .
Color and Depth Priors in Natural Images
By utilizing high-resolution, high-quality color images and coregistered range maps in the LIVE Color+3D Database, we examined the statistical relationships between multi-scale, multi-orientation Gabor decompositions of luminance/chrominance and range/depth data in natural scenes. We showed that the marginal statistics of both image and range magnitude responses follows the well-known 1/f^2 power law, and the conditional statistics of range gradients given image magnitude responses provide evidences supporting the co-occurrence of natural image and range variations. We further derived marginal and conditional priors relating natural luminance/chrominance and disparity, and demonstrated their efficacy with application to the Bayesian stereo algorithm. We also demonstrated that including the chrominance-range models augments the performance of the Bayesian stereo algorithm over using only the luminance information. More importantly, the superior performance incorporating color and range priors to previous luminance-only models bolsters the psychophysical evidence that not only image intensity, but also chromatic information is useful in 3D visual processing.
The statistical analysis we have performed and the color-range priors we have derived in this paper yield insight into how 3D structures in the environment might be recovered from color image data. We believe that these fundamental regularities between luminance/chrominance and range/depth information in natural images can be further utilized in a variety of 3D image and video applications. For example, shape-from-X (shading, texture, etc.) algorithms can generate more accurate three-dimensional structures using additional color-range statistics, and 3D (stereo) quality assessment can better judge distortions from irregular chrominance and range correspondences.
Relevant publications:
- C.-C. Su, L. K. Cormack, and A. C. Bovik, "Color and depth priors in natural images," IEEE Transactions on Image Processing , vol. 22, no. 6, pp. 2259-2274, June, 2013. ( PDF )
- C.-C. Su, A. C. Bovik, and L. K. Cormack, "Statistical model of color and disparity with application to Bayesian stereopsis," IEEE Southwest Symposium on Image Analysis and Interpretation , pp. 169-172, Apr. 2012. ( PDF )
- C.-C. Su, A. C. Bovik, and L. K. Cormack, "Natural scene statistics of color and range," IEEE International Conference on Image Processing , pp. 257-260, Sep. 2011. ( PDF )
Statistical Modeling of 3D Natural Scenes
Using a public coregistered database of luminance and range natural images, we examined the priors and conditional distributions of low level, fundamental image features such as luminance, disparity, and distance. Similar to the well-known properties of wavelet subband histograms of luminance images, the distributions of range and disparity subband coefficients tend to have much higher kurtosis than luminance coefficients, owing to a greater degree of regularity in range and disparity maps as compared to luminance images. Generalized Gaussians generally give good fits to marginal distributions of luminance, range, and disparity wavelet coefficients, but the shape parameter associated with marginal luminance distributions ( [0.6, 0.8]) tends to be significantly larger than those for marginal range and disparity distributions ( [0.2, 0.3]).
We found that the conditional magnitudes of luminance and range (disparity) coefficients mutually depend on each others¡¦ magnitudes. Generally, regions with larger luminance variation tend to have larger range (disparity) variation and vice versa. Our analysis also shows that the correlations between bandpass luminance and bandpass range (disparity) are stronger in coarser scales. The shape parameters of the conditionals display a clear dependency on the scene features that are conditioned on. We developed a stereo correspondence algorithm based on our statistical models of 3D natural scenes. Using a Bayesian framework, we showed that adaptively changing the smoothness cost at different luminance variation can improve the quality of the computed disparity maps. We believe that statistics of this type will also prove useful in other 3D vision and image processing applications, such as shape-from-X (shading, texture, etc), 3D recognition, and 3D (stereo) image quality assessment.
Relevant publications:
- Y. Liu, L. K. Cormack, and A. C. Bovik, "Statistical modeling of 3-D natural scenes with application to Bayesian stereopsis," IEEE Transactions on Image Processing , vol. 20, no. 9, pp. 2515-2530, 2011. ( PDF )
- Y. Liu, L. K. Cormack, and A. C. Bovik, "Dichotomy between luminance and disparity features at binocular fixations," Journal of Vision , vol. 10, no. 12, pp. 1-17, 2010. ( PDF )
- Y. Liu, A. C. Bovik, L. K. Cormack, "Disparity statistics in natural scenes," Journal of Vision , vol. 8, no. 11, pp. 1-14, 2008. ( PDF )
Back to 3D Natural Scene Statistics Research page
Document edited with HTML-Cleaner.com