Reconstruction of multispectral light field (5d plenoptic function) based on compressive sensing with colored coded apertures from 2D projections

Kareth Marcela León López, Laura Viviana Galvis Carreño, Henry Arguello Fuentes


In the last decade, spatio – angular (light field) acquisition systems have advanced due to the inclusion of coded apertures in the optical path. These coded apertures, modulate the light, encoding the information before being captured. Traditionally, these coded apertures are binary, i.e. block and unblock the light rays in the spatial dimension, capturing sparse information of the scene. In this work, the binary coded aperture is replaced by a colored coded aperture which modulates the source not only spatially but spectrally. Thereby, it is possible to capture light fields in multiple wavelengths yielding high spectral resolution. The spectral information provides many features of a scene in different wavelengths, these features are not present in the visible range of the electromagnetic spectrum. In this paper, an algorithm that simulates the light field sampling with colored coded apertures is proposed. The proposed algorithm, exploits the redundant information of the scene based on the compressive sensing theory thus, capturing just a sparse signal. The multidimensional image can be recovered from the underlying signal through a reconstruction algorithm. Several simulations show the quality of the multispectral light field reconstructions. The PSNR (Peak Signal to Noise Ratio) values obtained for the reconstructions are around 25 dB.


Light field; compressive sensing; multispectral Image; multispectral light Field; colored coded Aperture.

Full Text:



E. J. Candès, “Compressive sampling,” in Proceedings of the International Congress of Mathematicians, 2006, pp. 1433–1452.

M. Fornasier and H. Rauhut, “Compressive sensing,” Handb. Math. Methods Imaging, pp. 187 – 228, 2011.

D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289–1306, 2006.

G. A. Shaw and H. K. Burke, “Spectral imaging for remote sensing,” Lincoln Lab. J., vol. 14, no. 1, pp. 3–28, 2003.

K. Marwah, G. Wetzstein, Y. Bando, and R. Raskar, “Compressive light field photography using overcomplete dictionaries and optimized projections,” ACM Trans. Graph., vol. 32, p. 1, 2013.

A. Mian and R. Hartley, “Hyperspectral video restoration using optical flow and sparse coding,” Optics Express, vol. 20, no. 10. p. 10658, 2012.

K. Choi and D. J. Brady, “Coded aperture computed tomography,” in SPIE Optical Engineering + Applications, 2009, vol. 7468, p. 74680B–74680B–10.

E. Adelson and J. Bergen, “The plenoptic function and the elements of early vision,” Comput. Model. Vis. Process., pp. 3–20, 1991.

G. Wetzstein, I. Ihrke, D. Lanman, and W. Heidrich, “Computational plenoptic imaging,” Comput. Graph. Forum, vol. 30, no. 8, pp. 2397–2426, 2011.

H. Rueda, H. Arguello, and G. R. Arce, “Compressive spectral imaging based on colored coded apertures,” in ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, 2014, pp. 7799–7803.

S. D. Babacan, R. Ansorge, M. Luessi, R. Molina, and A. K. Katsaggelos, “Compressive Sensing of Light Fields,” in Proc. IEEE International Conference on Image Processing, 2009, pp. 2337–2340.

M. Aharon, M. Elad, and A. Bruckstein, “K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311–4322, 2006.

A. Brückner, “Microoptical Multi Aperture Imaging Systems,” Friedrich-Schiller-Universität Jena, 2011.

S. Kim, E. Lee, M. H. Hayes, and J. Paik, “Multifocusing and depth estimation using a color shift model-based computational camera,” IEEE Trans. Image Process., vol. 21, no. 9, pp. 4152–4166, 2012.

Y. Bando, B.-Y. Chen, and T. Nishita, “Extracting depth and matte using a color-filtered aperture,” ACM Transactions on Graphics, vol. 27, no. 5. p. 1, 2008.

H. Arguello and G. R. Arce, “Colored coded aperture design by concentration of measure in compressive spectral imaging,” IEEE Trans. Image Process., vol. 23, no. 4, pp. 1896–1908, Apr. 2014.

H. Arguello, H. Rueda, Y. Wu, D. W. Prather, and G. R. Arce, “Higher-order computational model for coded aperture spectral imaging.,” Appl. Opt., vol. 52, no. 10, pp. D12–21, 2013.

E. van den Berg and M. P. Friedlander, “Probing the Pareto Frontier for Basis Pursuit Solutions,” SIAM Journal on Scientific Computing, vol. 31, no. 2. pp. 890–912, 2009.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, 2004.

Abstract : 1351 PDF : 505

Article Metrics

Metrics Loading ...

Metrics powered by PLOS ALM

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Esta publicación hace parte del Sistema de Revistas de la Universidad de Antioquia
¿Quieres aprender a usar el Open Journal system? Ingresa al Curso virtual
Este sistema es administrado por el Programa Integración de Tecnologías a la Docencia
Universidad de Antioquia
Powered by Public Knowledge Project