STUDENT POSTER SESSION

P I C Y T D O C TO R A L P R O G R A M
June 28th - 2024

Computational Intelligence to reduce errors in the analysis of dynamic and static images
with structured light projection
Camilo René Duque Becerra Dr. Francisco Javier Cuevas de la Rosa
[email protected] [email protected]
AUTHOR ADVISOR

ABSTRACT OBJECTIVES
3D reconstruction using fringe patterns is a very active field research, to date one of the most important challenge is increasing processing speed and one promising alternative is phase approxima- Research, study and develop metaheuristic algorithms and compensation to reduce motion in-
tion due to traditional methods of phase measurement either require 3 images for the same measurement at a given instant, which slows down the process or implements a filter that loses data duced errors to perform dynamic 3D reconstruction of opaque solid objects.
and does not allow the processing of complex geometries. Having high-speed phase approximation algorithms wolud allow real-time applications in which the object is in motion, making its use in In order to achieve this, the following specific objectives have been proposed:
real situations more viable. Within the phase approximation methods, metaheuristics offers alternatives with excellent results in recent years. This poster summarizes the progress to date of a re- • Synthesize the state of the art, in order to identify the improvements that can be obtained
search that attempts to obtain phase approximation algorithms with metaheuristic methods through parallel processing at the hardware level. The results improve speed compared to other pre- through algorithms based on metaheuristics.
vious algorithms indicating its potential to contribute to the advancement of this field.
• Develop new algorithms based on computational intelligence based on hardware parallel
processing and metaheuristics applied to fringe demodulation problems using structured
BACKGROUND light projection.
3D reconstruction is a field of Optical Metrology for measuring the dimensions of an object and proximation, machine learning [17] and metaheuristics mainly. Of the latter, its main advantage • Evaluate the performance of the developed algorithms.
generating a 3D model, it is used in robotics, industrial automation, video games, medicine, vir- is the very little need for prior knowledge, as well as the existence of a wide range of optimizati-
tual reality, among others[1]. One of the most used techniques and scientific interest is based on on techniques with which it can be implemented [18,19]. METHODOLOGY
structured light projection (SLP)[2], mainly fringe projection profilometry (FPP)[3], where phase As shown in figure 2, in the case of dynamic measurements and that is, when the object is mo-
The hypothesis posits that phase unwrapping through metaheuristic algorithms and GPU para-
shifting (PSP) stands out[4]. It consists of projecting cosine fringes on a measured object, captu- ving or changing shape (uniformly or not), whatever the phase development technique, an un-
llelization reduces processing time without sacrificing depth measurement precision, thus im-
ring an image and determining the phase difference between the original pattern and that of the certainty term is added to equation 1, the motion induced error Ɛn, which originates from the
proving certain dynamic measurements over existing methods. The experiment compares two
image, this difference is proportional to the depth of the object[5]. The mathematical model is: change in relative position of each point of the object with respect to the coordinate of the asso-
treatments focusing on processing time, maintaining comparable phase error levels. The treat-
ciated pixel at each instant of time, its model is [20]:
ments are recursive unwrapping with Simulated Annealing and parallelized unwrapping with en-
Where, In(x,y) is the image intensity at the pixel of coordinates (x,y) at time n, A(x,y) is the back- hanced Simulated Annealing. Three experiment types—simulated images, static real images, and
ground illumination (experimentally controlled variable), B(x,y) is the reflectance of the object dynamic real images—will test the algorithms.
(experimentally controlled too), φ(x,y) is the phase of the fringe that contains the surface depth
information, (the depth is said to be wrapped in the phase term), δn=(2π(n-1))⁄N are the phase
shifts that are projected for the same instant n; and η(x,y) represents the noise. Figure 1 sche-
C U R R E N T M I L E S TO N E S A N D P R O G R E S S
matizes the experimental arrangement of the 3D reconstruction with FPP. To date, on simulated and real images, the following algorithms have been successfully imple-
mented with hardware parallelism on GPU:
• Edge detection with Sobel operator (findings are set to be published soon).
• Fringes Counting.
• Induced motion error compensation by Structural Similarity Index Measure (SSIM).
• Phase approximation with simulated annealing.
The following experiments are currently being developed and conducted in progress:
• A scene with spheres in circular motion.
• Parametric analysis of the simulated annealing algorithm considering the neighborhood
Figure 2. Schematic of motion induced error in phase shifting profilometry (PSP). function and the cooling scheme.
To mitigate motion induced error, two main trends have been established, increasing the proces- • Otsu adaptive thresholding on GPU.
sing speed during phase unwrapping (for which improved variants of FTP and PSP are used) and • Interferometry.
error compensation (updating the phase map adding the error) [20-25]. Until now, phase appro- The final experimental steps pending initiation are outlined below:
ximation has not been used in the dynamic regime, hence the novelty of this work.
• Phase aproximation with improved Simulated Annealing of rotating human-like figure.
Figure 1. Schematic of 3D measurements by fringe projection profilometry (FPP). On the other hand, in this field, due to the very recent massification and cost reduction, until
In te original PSP, N images (at least three) are required for each instant n, which slows down the • Trial featuring a other metaheuristic algorithm.
now there are few investigations in which parallel processing based on graphics acceleration
process, due to, single shot for phase unwrapping algorithms have been developed, such as Fou- hardware (GPU) is implemented [26-32], so it is a little unexplored path. How to take advantage
rier transform profilometry (FTP)[6], whose advantage is time reduction and simplicity, but loss of the computational power to increase the speed of the algorithms, that is another novel com-
of information (filtering) occurs, therefore is ineffective for analyzing objects with textures, sha- ponent with which we hope to contribute.
dows or non-uniformities, as well as images with closed stripes, undersampling, and noise. Inter-
mediate alternatives, between FTP and PSP[7], include artificial intelligence [8-16] to perform

R E S U LT S A N A LY S I S A N D D I S C U S S I O N
Hardware Parallelism in GPU of the Sobel Edge Detection Algorithm: PyCuda Shared Memory Hardware Parallelism in GPU of Fringes Counting. Phase approximation with simulated annealing.
implementation, evaluation and image Complexity and Image Size analysis.

Hardware: CPU Intel Xeon

@2.20 GHz, 12 GB RAM, Tesla
K80 accelerator, and 12 GB a) d)
GDDR5 VRAM.
Software: CUDA Framework
and PyCUDA.

b)
Figure 5. Sample images and data used to evaluate fringes counting GPU implementation.
In all cases the result was correct in the number of stripes, in the complete image and by regions.
Figure 3. Sample images and data used to evaluate Sobel GPU parallel implementation. Analysis was considered by rows, columns, main diagonal and inverse diagonal. Using the shared
memory model, the results show a large speedup with GPU parallel processing.
Induced motion error compensation by Structural Similarity Index Measure (SSIM).
c) e)
Figure 7. a) Simulated images to evaluate the proposed algorithm. b) Experimental results. c)
Stability and sensitivity analysis. d) Moire shadow experiment. e) Speckle interferogram.
The proposed phase unwrapping algorithm, utilizing Simulated Annealing and GPU parallelism
with shared memory, was evaluated using simulated images (Fig. 7.a). The results demonstrate
significant improvements in processing time without precision loss (Fig. 7.b) compared to [16].
a) b)
Figure 4. Speedup Evaluation of Sobel GPU parallel implementation. a) Comparative with refe- a) b) The algorithm's results remained consistent despite parameter variations in iterations and gene-
rence libraries. b) Speedup over OpenCV and image size analysis. rations per iteration (Fig. 7.c). In the Moire shadow experiment (Fig. 7.d), the algorithm success-
fully reconstructed the hemisphere, though dimensional accuracy verification is still required.
The algorithm's capacity was further evaluated in an interferometry experiment with speckle
noise (Fig. 7.e), identifying the necessity for an adaptive noise reduction stage, currently being
c) d) developed using GPU-based parallel processing with Otsu's method [41,42].
Figure 6. a) Induced motion error compensation proposed algorithm. b) Sample images of scene
with two sphere in circular motion. c) Speedup of calculating SSIM of GPU Shared Memory over
CPU. d) Time execution comparative of DICE similarity coefficient and SSIM similarity index.
CONCLUSION
Based on [22] The proposed algorithm (Fig. 6.a) computes a similarity index for the phase map at The main contributions to date are: a novel, faster edge detection algorithm based on the Sobel
Figure 5. Image Complexity analysis of Sobel GPU parallel implementation. instant i. If this index surpasses an experimentally defined threshold, the phase map is updated; operator that achieves greater speedup with larger images and is insensitive to image complexi-
The parallel implementation of the Sobel operator using PyCuda and the shared memory model otherwise, it assumes the object has moved without changing, thus negating the need to update ty; a new motion-induced error compensation algorithm that reduces processing time by upda-
demonstrates a significant speed increase compared to the OpenCV, SciPy, and Scikit libraries [33 the 3D model. In a scenario involving two rotating spheres (Fig. 6.b), execution times for calcula- ting the 3D model only when the phase map changes above a threshold; and a phase unwrap-
-37]. Additionally, experimental data indicate that this speed enhancement becomes more pro- ting the DICE similarity coefficient and the structural similarity index (SSIM) were compared. Re- ping algorithm using simulated annealing that shows notable runtime improvements. Current
nounced with larger image sizes, surpassing the performance of other published implementati- sults indicate that SSIM is faster (Fig. 6.d), leading to its selection for the motion-induced error experimental progress demonstrate that metaheuristic algorithms with GPU hardware paralle-
ons [38]. However, when analyzing entropy and average spatial information metrics [39-40], no compensation algorithm. Subsequently, parallel implementation on GPU hardware demonstrated lism and the shared memory model, make suitable for 3D reconstruction in dynamic regime.
correlation was found between the speed increase and the complexity of the image. significant speedup compared to CPU execution (Fig. 6.c).

REFERENCES
[1] A. G. Marrugo, F. Gao, and S. Zhang, “State-of-the-art active optical techniques for three-dimensional surface metrology: a review [Invited],” JOSA A, Vol. 37, Issue 9, pp. B60- [15] I. Moré, F. J. Cuevas, and J. Jiménez, “Parallel Demodulation Algorithm for processing independent windows of a fringe patterns using Simulating Annealing,” Opt Commun, [29] P. H. Yelmewad, “Parallel Metaheuristic Approaches to Solve Combinatorial Optimization Problems,” 2021.
B77, vol. 37, no. 9, pp. B60–B77, Sep. 2020, doi: 10.1364/JOSAA.398644. vol. 463, p. 125403, 2020, doi: https://doi.org/10.1016/j.optcom.2020.125403. [30] Y. Sun, S.-C. Chu, P. Hu, J. Watada, M. Si, and J.-S. Pan, “Overview of Parallel Computing for Meta-Heuristic Algorithms,” Taiwan Ubiquitous Information, vol. 7, no. 3,
[2] J. Geng, “Structured-light 3D surface imaging: a tutorial,” Adv Opt Photonics, vol. 3, no. 2, 2011, doi: 10.1364/aop.3.000128. [16] I. Moré Quintero and F. J. Cuevas de la Rosa, “INTELLIGENT ALGORITHMS FOR PROCESSING METROLOGICAL IMAGES,” Centro de Investigaciones en óptica, 2021. 2022.
[3] S. Gorthi, P. R.-O. and lasers in engineering, and undefined 2010, “Fringe projection techniques: whither we are?,” infoscience.epfl.ch, vol. 106, no. 123, pp. 133–140, 2010, [17] H. Yu et al., “Dynamic 3-D measurement based on fringe-to-fringe transformation using deep learning,” Optics Express, Vol. 28, Issue 7, pp. 9405-9418, vol. 28, no. 7, pp. [31] J. Gong and M. Karimzadeh Parizi, “GWMA: the parallel implementation of woodpecker mating algorithm on the GPU,” https://
Accessed: Oct. 16, 2021. [Online]. Available: https://infoscience.epfl.ch/record/140745/files/OLEN.pdf 9405–9418, Mar. 2020, doi: 10.1364/OE.387215. doi.org/10.1080/02533839.2022.2078418, vol. 45, no. 6, pp. 556–568, 2022, doi: 10.1080/02533839.2022.2078418.
[4] J. Xu and S. Zhang, “Status, challenges, and future perspectives of fringe projection profilometry,” Opt Lasers Eng, vol. 135, p. 106193, Dec. 2020, doi: 10.1016/ [18] C. Zuo, L. Huang, M. Zhang, Q. Chen, and A. Asundi, “Temporal phase unwrapping algorithms for fringe projection profilometry: A comparative review Basic Energy Sciences [32] N. Coll, M. Fort, and M. Saus, “Coverage area maximization with parallel simulated annealing,” Expert Syst Appl, vol. 202, p. 117185, Sep. 2022, doi: 10.1016/
J.OPTLASENG.2020.106193. (BES) (SC-22),” 2016. J.ESWA.2022.117185.
[5] C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: A review,” Opt Lasers Eng, vol. 109, pp. 23–59, Oct. 2018, [19] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi, “Optimization by Simulated Annealing,” Science (1979), vol. 220, no. 4598, pp. 671–680, May 1983, doi: 10.1126/ [33] H. Ben Fredj, M. Ltaif, A. Ammar, and C. Souani, “Parallel implementation of Sobel filter using CUDA,” 2017 International Conference on Control, Automation and Diag-
doi: 10.1016/J.OPTLASENG.2018.04.019. SCIENCE.220.4598.671. nosis, ICCAD 2017, pp. 209–212, Oct. 2017, doi: 10.1109/CADIAG.2017.8075658.
[6] K. Mutoh and M. Takeda, “Fourier transform profilometry for the automatic measurement of 3-D object shapes,” Applied Optics, Vol. 22, Issue 24, pp. 3977-3982, vol. 22, no. [20] L. Lu, V. Suresh, Y. Zheng, Y. Wang, J. Xi, and B. Li, “Motion induced error reduction methods for phase shifting profilometry: A review,” Opt Lasers Eng, vol. 141, p. 106573, [34] N. D. Lynn, A. I. Sourav, and A. J. Santoso, “Implementation of Real-Time Edge Detection Using Canny and Sobel Algorithms,” IOP Conf Ser Mater Sci Eng, vol. 1096, no. 1,
24, pp. 3977–3982, Dec. 1983, doi: 10.1364/AO.22.003977. 2021, doi: 10.1016/j.optlaseng.2021.106573. p. 012079, Mar. 2021, doi: 10.1088/1757-899X/1096/1/012079.
[7] J. I. Tan et al., “Deep learning-based method for non-uniform motion-induced error reduction in dynamic microscopic 3D shape measurement,” Opt Express, vol. 30, no. 14, p. [21] Q. Zhang, W. Guo, Y. Liu, Y. Li, and Z. Wu, “Real-time 3D shape measurement with dual-frequency composite grating and motion-induced error reduction,” Optics Express, [35] M. Chouchene, F. E. Sayadi, Y. Said, M. Atri, and R. Tourki, “Efficient implementation of Sobel edge detection algorithm on CPU, GPU and FPGA,” International Journal of
24245, 2022, doi: 10.1364/OE.461174. Vol. 28, Issue 18, pp. 26882-26897, vol. 28, no. 18, pp. 26882–26897, Aug. 2020, doi: 10.1364/OE.403474. Advanced Media and Communication, vol. 5, no. 2–3, pp. 105–117, 2014, doi: 10.1504/IJAMC.2014.060506.
[8] F. Cuevas, H. Sossa, and M. Servín, “A parametric method applied to phase recovery from a fringe pattern based on a genetic algorithm,” Opt Commun, vol. 203, pp. 213–223, [22] B. Li, V. Suresh, and Y. Wang, “Motion-induced error reduction for binary defocusing profilometry via additional temporal sampling,” Optics Express, Vol. 27, Issue 17, pp. [36] A. Akasapu, V. Sailaja, and G. Prasad, “Implementation of Sobel filter using CUDA,” IOP Conf Ser Mater Sci Eng, vol. 1045, no. 1, p. 012016, Feb. 2021, doi: 10.1088/1757-
Jul. 2002, doi: 10.1016/S0030-4018(02)01132-X. 23948-23958, vol. 27, no. 17, pp. 23948–23958, Aug. 2019, doi: 10.1364/OE.27.023948. 899X/1045/1/012016.
[9] F. J. Cuevas and M. Servin, “Closed fringe demodulation by multipolynomial fitting,” https://doi.org/10.1117/12.560979, vol. 5531, pp. 405–412, Aug. 2004, doi: [23] H. Gutiérrez and D. la Vara, Análisis y Diseño de Experimentos. McGraw-Hill Interamericana: México City, Mexico, 2012. [37] B. Lipovský, S. Š.-C. Science, and undefined 2023, “Performance comparison of CPU and GPGPU calculations using three simple case studies,” math.md, vol. 31, no. 1, p.
10.1117/12.560979. [24] G. Wu, T. Yang, F. Liu, and K. Qian, “Suppressing motion-induced phase error by using equal-step phase-shifting algorithms in fringe projection profilometry,” Opt Express, 2023, 2023, doi: 10.56415/csjm.v31.07.
[10] J. F. Jimenez, F. J. Cuevas, J. H. Sossa, and L. E. Gomez, “Fringe-pattern demodulation using a parametric method based on differential evolution,” Lecture Notes in Computer vol. 30, no. 11, p. 17980, 2022, doi: 10.1364/OE.459087. [38] Q. Chang, X. Li, Y. Li, and J. Miyazaki, “Multi-directional Sobel operator kernel on GPUs,” J Parallel Distrib Comput, vol. 177, pp. 160–170, Jul. 2023, doi: 10.1016/
Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 6256 LNCS, pp. 61–70, 2010, doi: 10.1007/978-3-642-15992-3_7/ [25] J. Kofman, T. Tao, X. Liu, Y. Wan, Y. Wan, and J. Kofman, “Real-time motion-induced-error compensation in 3D surface-shape measurement,” Optics Express, Vol. 27, Issue J.JPDC.2023.03.004.
COVER. 18, pp. 25265-25279, vol. 27, no. 18, pp. 25265–25279, Sep. 2019, doi: 10.1364/OE.27.025265. [39] L. M. Herrera, M. I. C. Murguía, and J. A. R. Quintana, “Análisis de Métodos de Medición de Complejidad de Imagen - Image Complexity Measurement Methods: A Sur-
[11] F. Cuevas, N. Alcala, L. Toledo, and J. Sossa-Azuela, “Optical discontinuous fringe metrology by using a genetic algorithm,” in Proccedings of the IASTED International Conferen- [26] A. Borisenko and S. Gorlatch, “Comparing GPU-parallelized metaheuristics to branch-and-bound for batch plants optimization by Russian Science Foundation. View project vey,” ReCIBE, Revista electrónica de Computación, Informática, Biomédica y Electrónica, vol. 7, no. 2, pp. 17–46, Oct. 2018, doi: 10.32870/RECIBE.V7I2.99.
ce Computational Intelligence, 2006, pp. 293–296. Accessed: Aug. 27, 2022. [Online]. Available: https://www.researchgate.net/publication/220939968 Comparing GPU-Parallelized Metaheuristics to Branch-and-Bound for Batch Plants Optimization,” Article in The Journal of Supercomputing, 2019, doi: 10.1007/s11227-018- [40] H. Yu and S. Winkler, “Image complexity and spatial information,” 2013 5th International Workshop on Quality of Multimedia Experience, QoMEX 2013 - Proceedings,
[12] J. Jiménez, H. Sossa, and F. Cuevas, “Particle swarm optimization applied to interferogram demodulation,” Advances in Intelligent Systems and Computing, vol. 175 ADVANCES, 2472-9. pp. 12–17, 2013, doi: 10.1109/QOMEX.2013.6603194.
pp. 327–337, 2013, doi: 10.1007/978-3-642-31519-0_21/COVER. [27] N. Coll, M. Fort, and M. Saus, “Coverage area maximization with parallel simulated annealing ✩,” Expert Syst Appl, vol. 202, p. 117185, 2022, doi: 10.1016/ [41] X. Zhu, Y. Xiao, G. Tan, S. Zhou, C. S. Leung, and Y. Zheng, “GPU-accelerated 2D OTSU and 2D entropy-based thresholding,” J Real Time Image Process, vol. 17, no. 4, pp.
[13] I. Moré Quintero and F. J. Cuevas de la Rosa, “AJUSTE DE FUNCIONES MEDIANTE METAHEURÍSTICAS APLICADO A LA DETECCIÓN DE FASE EN ARREGLOS DE METROLOGÍA ÓPTI- j.eswa.2022.117185. 993–1005, Aug. 2020, doi: 10.1007/S11554-018-00848-5/METRICS.
CA,” 2017. [28] T. G. Crainic, “Parallel Meta-Heuristics and Cooperative Search Parallel Meta-Heuristic and Cooperative Search Dépôt légal-Bibliothèque et Archives nationales du Québec [42] PRAHARA, Adhi, et al. Parallel Approach of Adaptive Image Thresholding Algorithm on GPU. Knowl. Eng. Data Sci., 2021, vol. 4, no 2, p. 69-84.
[14] A. López Martínez and F. J. Cuevas De La Rosa, “Metaheuristic Approaches for 3D-reconstruction-related Problems,” Centro de Investigaciones en Óptica, A.C., 2019. Bibliothèque et Archives Canada, 2017,” 2017.