Journal of Manufacturing Processes 69 (2021) 630–647 Contents lists available at ScienceDirect Journal of Manufacturing Processes journal homepage: www.elsevier.com/locate/manpro Image processing approach to automate feature measuring and process parameter optimizing of laser additive manufacturing process Deepika B. Patil *, Akriti Nigam, Subrajeet Mohapatra Department of Computer Science and Engineering, Birla Institute of Technology Mesra, Ranchi 835215, Jharkhand, India A R T I C L E I N F O A B S T R A C T Keywords: Laser additive manufacturing Image processing Geometric features Measurement Optimization Edge detection The present research work focuses on the development of image processing technique that can automatically extract the deposition geometric features and optimize the process parameters required for manufacturing components by laser additive manufacturing process. This paper reports (i) manufacturing of vertical and hor­ izontal wall components and capturing its images, (ii) developing robust image-processing technique for feature extraction and measurement, (iii) formulating a component sorting methodology with a capability to accept and reject component, and (iv) developing the process parameter optimizing model to identify the optimized com­ bination of process parameters used to manufacture components. The developed image processing algorithm has been validated against the manual measurement method and CAD model. It has been observed that the proposed image processing algorithm can measure the geometric features of the vertical and horizontal wall components with an error of less than 3%. The optimization study gave the optimized value of laser power as 820 W, 850 W, 800 W and 860 W, scanning speed as 500 mm/min, 500 mm/min, 730 mm/min and 700 mm/min, and powder feed rate as 6 g/min, 10 g/min, 7 g/min and 7 g/min for effective vertical wall width, effective vertical wall height, effective horizontal wall width and effective horizontal wall height, respectively. The optimized process parameters were validated experimentally on laser based additive manufacturing process. The optimized values of effective vertical wall width, effective vertical wall height, effective horizontal wall width and effective horizontal wall height are 5.003, 14.003, 20.003 and 6.002, respectively, with corresponding experimental values as 5.028, 14.016, 20.018 and 6.028, respectively. Therefore, for the fast-growing additive manufacturing industry the proposed image processing methodology will offer benefits of automatic feature measuring process and process parameter optimizing with high accuracy and less human interference. In future, the image pro­ cessing algorithm will be further developed for the real-time feature extraction of the depositions done by laser based additive manufacturing process. 1. Introduction Direct energy laser deposition process is widely used additive manufacturing process for manufacturing complex, good quality, high accuracy, 3D components. However, ensuring the accuracy of the components is still a challenging task for additive manufacturing in­ dustries. Consequently, researchers are trying to bridge this challenge either by proposing the optimized process parameters of the deposition process or by developing novel measurement techniques that can ensure the accuracy of the deposition process. Most challenging task faced by the researchers working in the development of the measurement tech­ nique is by making use of image processing techniques. Capabilities of image processing can be used for characterizing surface defects [1], online monitoring of the manufacturing process [2], generating toolpath in CNC machines [3], and assess the surface quality of products [4]. In previous literature, simple and regular geometry features were extracted using different feature extraction techniques such as Hough transformation with genetic algorithm method, standard Hough trans­ formation method, ellipse detection method and randomized Hough transformation method [5–9]. For some of the deposition processes, researchers have used image processing software integrated with me­ chanical measurement equipment. Alfano et al. [10] used an air plasma spring process for the deposition of ceria and yttria co-stabilized zirconia materials. They used image analysis technique on the images obtained by Scanning Electron Microscopy (SEM) to examine the microstructure and porosity in the deposition. For gas metal are deposition process, * Corresponding author. E-mail address: deepikapatil941@gmail.com (D.B. Patil). https://doi.org/10.1016/j.jmapro.2021.07.064 Received 30 December 2020; Received in revised form 12 July 2021; Accepted 28 July 2021 Available online 19 August 2021 1526-6125/© 2021 The Society of Manufacturing Engineers. Published by Elsevier Ltd. All rights reserved. D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 1. Flow chain adopted in present study. Thao and Kim [11] used image analysis software integrated with the microscope to obtain the deposition geometry features, while Sathiya et al. [12] measured deposition geometric features using an optical microscope. Consequently, for the deposition geometry obtained by gas metal arc welding and CO2 laser welding process, Rayes et al. [13] used the stereotype microscope with image analysis software to measure the deposition features. For the metal arc gas deposition process, Cao et al. [14] used an image processing technique known as Canny edge detec­ tion to extract the features and shape of deposition. On the other hand, for a similar process, Singh et al. [15] used an image analysis technique to measure features of deposition geometry. Researchers are also working on the development of robust image 631 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 2. Experimental setup of laser additive manufacturing used for building the components. Fig. 3. CAD model of the components build using laser additive manufacturing (a) horizontal wall and (b) vertical wall. manufacturing process, Vaidya and Anand [18] proposed an image processing method that can slice the computer aided design model. This approach has been known as IPSlicer. In this, the snapshot of the product image has been captured in the perpendicular direction of the heat source. Later this image is exported to IPSlicer to slice the desired ge­ ometry in the snapshot. This helps to get the CAD data by converting it to the STL file. For the powder bed fusion additive manufacturing process, Yao et al. [19] used an image processing technique known as multi­ fractal method. They used this method to detect the defects and char­ acterize the microstructure of the components, in which they identified the dimension and shape of the defect and microstructure. Researchers have used image processing techniques with machine learning to over­ come the problems of additive manufacturing. Straub [20] has effec­ tively used multiple cameras as an image processing technique to detect the live defects generated in deposition geometry during deposition process. Using a similar method Wang et al. [21] evaluated and controlled the grain morphology of the components built by the laser melting deposition process. Past literature suggests that the image processing in additive manufacturing has the ability to characterize the microstructure feature of complex geometry and analysis of the deposition process. Conse­ quently, the aim of the present research work is to develop the robust image-processing based technique that can automate the process of geometric features extraction for the components manufactured by laser additive manufacturing process. To achieve the aim, following objec­ tives were carried out in this research work: Table 1 Combinations of process parameters used to manufacture components. Experiment no Power (W) Scanning speed (mm/min) Powder feed rate (g/min) Number of passes for vertical wall Number of passes for horizontal wall 1 2 3 4 5 6 7 8 9 10 11 12 800 900 1000 800 900 1000 800 900 1000 800 900 1000 500 5 500 10 700 5 700 10 96 96 94 109 95 86 103 98 103 100 103 102 175 197 256 215 203 187 205 256 210 226 244 254 processing techniques that can improve the performance of the additive manufacturing process. For laser cladding process, to build the compo­ nent of the aircraft engine, Tabernero et al. [16] used the image pro­ cessing method to get the porosity size, shape and number. For the laminated Object Manufacturing process, Putthawong et al. [17] pro­ posed the image processing technique to generate the tool path. They used adaptive cross hatching and intersection point detection technique to extract the tool path information. For the laser additive 632 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 4. Cross section of the (a) horizontal wall and (b) vertical wall geometric features to be extracted by image processing. 1. Manufacturing of vertical and horizontal wall components by using laser additive manufacturing process and acquiring the crosssectioned images of the manufactured components. 2. Develop a robust image-processing based technique for feature extraction and measurement of the total and effective width, height, and area of the vertical and horizontal wall components. 3. Propose a component sorting methodology with a capability to accept or reject dimensions based on extracted features. 4. Develop the process parameter optimizing model using genetic al­ gorithm optimization technique for the laser additive manufacturing process. Consequently, the accuracy of the proposed technique was validated with a manual measuring method. 2. Methodology Fig. 1 depicts the flow chain adopted in the present study to auto­ mate the geometric feature measuring process. The flow chain is divided 633 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 5. Image processing on (a) vertical wall and (b) horizontal wall obtained for each experiment. 634 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 into four stages. Initially, in stage 1, the components with the vertical and horizontal walls are manufactured using various process parameters of the laser additive manufacturing process. In stage 2, the images for the manufactured components are captured and the images are preprocessed to remove any noise, shadow condition, poor lighting condi­ tion, and perspective. Thereafter, the images are processed in stage 3 for parametrization of the geometric features such as width, height, and area of the deposition geometry. In the next stage i.e., stage 4, the computed geometric features such as effective width, effective height, and effective area are individually compared with the values obtained from the CAD model. Based on which the decision of accepting or rejecting the dimension is made. For the rejected dimensions, the opti­ mum combination of process parameters is identified by the process parameter optimizing model. The optimized process parameters are sent back to manufacture the components in stage 1 with geometry features close to the CAD model. 2.1. Experimentation and image acquisition In the present study, the laser additive manufacturing process has been used to manufacture the components of Inconel 625 on the mild steel substrate material. Fig. 2 shows the experimental setup of the laser deposition process used for experimentation at Magod Fusion Technol­ ogies Pvt. Ltd., Pune, India. Two Computer-Aided Design (CAD) models of deposition geometries named as vertical and horizontal walls as depicted in Fig. 3a and b respectively were manufactured. The process parameters such as laser power, powder feed rate, and travel rate of heat source selected to obtain these deposition geometries are as given in Table 1. Other parameters of setup consist of laser spot diameter as 2 mm, hatch spacing as 1 mm, slicing thickness as 1 mm and scan pattern as zigzag. The total number of passes used to obtain vertical and hori­ zontal walls are included in Table 1. The deposition geometries as shown in Fig. 3 were manufactured using each combination of process parameters given in Table 1. There­ after, the depositions are sectioned along with the height of deposition geometry using a wire-electro discharge machine. The sectioned sam­ ples were placed on a flat surface plate with black in its background to eliminate the shadow effect. The images were acquired using Canon (Model 1500 D) camera under natural light conditions. Camera has been levelled parallel to the surface plate to avoid any asymmetricity in the captured images. Fig. 4a depicts the vertical wall and Fig. 4b depicts horizontal wall that was sectioned to acquire an image of deposition geometries used for image processing. Fig. 6. Schematic of vertical and horizontal lines drawn at the edge of depo­ sition to trace outer rectangle boundaries. 2.2. Image processing approach This section describes the ability of the image processing approach to automate the feature measuring process and process parameter opti­ mization required in manufacturing components by the laser additive manufacturing process. In the present work, authors have proposed a novel image processing algorithm that can identify the irregular shapes appearing in the deposition geometries. It has features to automatically detect and parametrize the dimensions of the components such as total and effective width, height, and area of the deposition geometries. The process is carried out into two stages, pre-processing (stage II) and parameterization (stage III) of deposition geometry as shown in Fig. 1. The following subsections describe the methodology adopted for image processing. 2.2.1. Image pre-processing In present study, the image analysis has been done to handle noise, shadow condition, poor lighting condition, and perspective using steps described below [22]. Firstly, the camera captured images for each experiment are converted from coloured images to greyscale images as shown in Fig. 5a and b. This conversion helps to identify the edges of the deposition geometry. In the next step, gamma correction is applied to Fig. 7. Pseudocode for the outer rectangle operations. 635 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 8. Schematic of (a) concavities in deposition edges and (b) vertical line drawn at the concavity intersection point to obtain boundaries for inner rectangle. eliminate any darkness and shadow conditions existing in the images. This is done to enhance the dynamic range of dark pixel intensities and shrink the dynamic range of brighter pixel intensities. Further, the Dif­ ference of Gaussian (DoG) filtering as expressed in Eq. (1) is used to remove the shadow effect and suppress illumination gradient. While applying DoG with Gaussian filter, width σ1 = 1.0 is implemented through convolution of image for creating the first smoothed image. The image is again smoothed with width σ2 = 2.0. ( ( ) )⎞ ⎛ 1 ⎜ ⎜1 DoG = √̅̅̅̅̅ ⎜ exp 2π ⎝σ1 − (x+y)2 2σ2 1 − 1 σ2 − exp (x+y)2 2σ 2 2 ⎟ ⎟ ⎟ ⎠ 2.2.2. Parameterization The width, height, and area of deposition geometry are measured on the edges of deposition geometry extracted using the Canny edge detection technique [23] technique. Stage III process has been divided into two operations: 1) Outer rectangle trace, and 2) Inner rectangle trace. Outer rectangle trace will give the values of total width, height, and area of a deposition while the inner rectangle trace will give the values of effective width, height, and area of deposition. Following is the description of the traces used for developing novel image processing algorithm that can parametrize the features of the deposition geometries. (1) (a) Outer rectangle: To extract the actual deposition geometry first, compute the outer boundary rectangle corresponding to the deposition object as shown in Fig. 6. The outer bounding rect­ angle gives the extent of the deposition object in the x and y di­ rections. The pixels in the image obtained after Canny edge detection consists of rows from R1 to Ra (where a is the number of rows) and columns from C1 to Cb (where b is the number of col­ umns). To compute the outer bounding rectangle of the deposi­ tion geometry the classic raster scan technique has been used. Fig. 7 shows the pseudocode used to extract the outer bound rectangle. For locating the topmost edge of the rectangle, the scanning starts from the topmost left pixel of the image i.e., at point 1. Further the image is scanned from left to right i.e., from point 1 to point 2, going down one row at a time until the fore­ ground pixel intensity is obtained. The corresponding row Rtop is selected for the point at which the foreground pixel intensity is identified, and the straight-line for the topmost edge of the rectangle is drawn (dotted horizontal line in Fig. 6). Conse­ quently, for locating the bottommost edge, the scanning will start from the bottommost left pixel of the image i.e., from point 3 to point 4. The scanning takes place from left to right in a similar way but by going up, one row at a time (Rbottom). For locating the leftmost edge of the rectangle, the scanning starts from the topmost left pixel of the image i.e., at point 1. Further, the image is scanned from top to bottom i.e., from point 1 to point 3 going Thereafter, histogram equalization and thresholding are applied to all inputs individually to eliminate the pixel intensity unevenness caused due to dark and bright spots. It also improves the brightness of images. Morphological operation and substrate removal has been done on ver­ tical and horizontal wall images as shown in Fig. 5. Morphological opening and closing operation are applied on the images for removing the light spots on dark background and as dark spots on light back­ ground. Opening operation (I ∘ E) as given in Eq. (2), helps to remove noise such as small, bright spots mainly on the deposition geometry and it preserves the level of intensity of the images. Similarly, closing operation (I • E) as given in Eq. (3), helps to remove small holes present in area of the deposition. Below equation represents the opening and closing operations. I∘E = I ⊖ E ⊕ E (2) I•E =I⊕E⊖E (3) Once the image enhancing operation is completed, the next step is to remove the substrate area from the images captured along with substrate material because the deposition geometry is the main requirement for the image processing. Therefore, the substrate removed images are used for parametrization in stage III. 636 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 where Cx and Cy are the coordinates of the centroid of the outer rect­ angle in the x-axis and y-axis, Outer_R, and Outer_L, is the column index of the right and left edge of the outer rectangle, and Outer_B, Outer_T are the row index of the bottom and top edge of the outer rectangle. To trace the left boundary of the inner rectangle, the scanning process starts from the pixel at the centroid. The ith column of pixels ‘Coli’ that represents a vertical line is moved towards the left side with an increment of one column at a time until the left bound of the outer rectangle is reached. For each column scanned, the number of contiguous foreground in­ tensity regions is counted. The idea behind counting contiguous fore­ ground intensity regions is to identify the intersecting point at a concave shape. In each column, there will be a minimum of two contiguous foreground intensity regions indicating upper and lower object bound­ ary. Any extra contiguous region indicates the presence of a cave as shown in Fig. 8a. The scanning process gets terminated at the first col­ umn which has more than two contiguous foreground intensity regions. The column index at which the process gets terminated is the left bound of the actual deposition. The above process is repeated in a similar manner in the right direction from the centroid to locate the right bound of the actual deposition. To locate the top bound, it is necessary to locate point a and b as shown in Fig. 8b. For this process, the column which is one pixel towards right of the inner left bound is scanned. This scanning starts from the row corresponding to the centroid and moves upward. The scanning stops when the first foreground pixel is encountered, and that point is labelled as ‘a’ as shown in Fig. 8b. The same process is repeated in the right direction. The column which is one pixel towards the left of the inner right bound is scanned, moving upwards starting from the row corresponding to the centroid. Point ‘b’ is located where the first foreground pixel is obtained in the scanning process. If ‘a’ and ‘b’ are located at two different rows, then the one which is closer to the centroid is selected and marked as the inner top bound. The bottom bound of the inner rectangle remains the last row of the deposited object from where the substrate has been removed. Using the left, right, top, and bottom bounds, the inner rectangle for the actual deposition ge­ ometry is created. In the horizontal wall, the top bound is located by using the process adopted to locate the left bound of the vertical wall. While the bottom bound is the last row of the deposited object from where the substrate has been removed. The left and right bound of the horizontal wall are located using the process adopted for locating the top bound in the vertical wall. Fig. 10a and b are the outcome of the outer rectangle and inner rectangle trace performed on the actual deposition geometries obtained on vertical and horizontal wall, respectively. Fig. 9. Pseudocode for the inner rectangle trace. right side, one column at a time until the foreground pixel in­ tensity is obtained. The corresponding column Cleft is selected for the point at which the foreground pixel intensity is identified, and the straight-line for the leftmost edge of the rectangle is drawn (dotted vertical line in Fig. 6). Using a similar process and by reversing the scanning strategy, the rightmost edge (Cright) of the rectangle is identified. For the horizontal wall, the outer rectangle is drawn using a similar method. (c) Feature extraction: Using outer rectangle operation, total width, height, and area of deposition as shown in Fig. 11a has been measured. While using inner rectangle operation, effective width, height, and area of deposition as shown in Fig. 11b have been measured. Pseudocodes as depicted in Fig. 12 were used for calculating these dimensions. (b) Inner rectangle: It will give the effective dimensions of the deposition geometry. The deposition geometries have concavities as represented in Fig. 8a. The concavity for the vertical wall deposition geometry is on the left and right side of the deposition Therefore, for the vertical wall, the left and the right-side edge of the inner bound rectangle is extracted initially. For this, a vertical line is drawn at the two intersection points obtained on the concave-shaped object as shown in Fig. 8b. Pseudocode as depicted in Fig. 9 was used for the inner rectangle trace. The ith column of pixels is represented as ‘Coli’. The procedure to trace the inner rectangle begins with calculating the centroid of the outer bounding box as shown in Eqs. (4) and (5). Cx = Outer R − Outer L 2 (4) Cy = Outer B − Outer T 2 (5) 2.2.3. Component sorting process The effective dimensions such as effective width, height and area of the manufactured components are desired and should be according to the CAD model as shown in Fig. 3. The effective dimensions for each individual experiment measured by image processing were compared against the CAD model width as 5 mm, height as 14 mm, and area as 70 mm2 for vertical wall while width as 20 mm, height as 6 mm, and area as 120 mm2 for the horizontal wall. Almost all the combinations listed in Table 1 are capable of manufacturing vertical and horizontal wall, but few combinations of process parameters are not capable of producing the desired features according to CAD model. Tables 2 and 3 shows the value of effective and total dimensions obtained using image processing technique for the vertical and horizontal wall, respectively. From Table 2 for vertical wall, it can be said that the component manufactured by combinations of process parameters in experiment no. 3 and 5 did not 637 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 10. Images after performing parametrization stage-II operation on (a) vertical wall and (b) horizontal wall obtained for each experiment. give the desired effective width (i.e., 5 mm) while the combinations used for experiment no. 6 and 12 did not give the desired effective height (i.e., 14 mm). From Table 3 for the horizontal wall, it can be said that the component manufactured by the combination of process parameters in experiment no. 3 did not give the desired effective height (i.e., 6 mm) while experiment no. 6 and 7 did not give the desired effective width (i. e., 20 mm). While comparing the effective dimensions obtained by image processing with CAD model dimensions, tolerance of ±0.30 mm for width and height were considered. Therefore, looking at the uncer­ tainty in the process parameters combinations it is necessary to identify 638 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 11. Geometric features of the additive manufactured components extracted using image processing (a) total and (b) effective geometry features. (9) are used as the objective functions in GA. These equations were built by using regression model and by using the values of effective width and height given by image processing technique for 12 experimental runs. The components manufactured in this study are rectangular in shape hence instead of developing a separate model for effective area, the product of width and height has been considered. Figs. 13 and 14 depicts the comparison of the equation predicted effective width and height with the image processed effective width and height of the vertical and horizontal wall, respectively. Table 4 show the bound values of process parameters and the constraints used in GA for optimization. The effec­ tive width and height of the vertical and horizontal wall has been con­ strained as shown in Table 4. In present study, the constrain functions used for optimization were built according to the method proposed by Gunaraj and Murugan [24]. Other values of parameters used in GA optimization are included in Table 5. In the past literature for the direct energy deposition process, the value of crossover fraction used was in between 0.7 and 0.9 [25,26]. To identify the crossover fraction value for the present study, the trial-and-error runs was carried out using 0.7, 0.75, 0.8, 0.85 and 0.9 crossover fraction values. It was observed that the crossover fraction value of 0.8 and 0.85 gave the best fitness value very close to the effective dimensions of horizontal and vertical walls. Therefore, these values are used in present work. The objective functions in this study are nonlinear therefore Augmented Lagrangian GA has been used [27]. To identify the most appropriate optimized process parameters values, optimization was carried out for various combina­ tions of GA parameters as shown in Table 6. The fitness values and process parameter combinations for each GA parameter combinations has been included as the supplementary material. While the combina­ tion of GA parameter that gave fitness value close to the width and height of CAD model of vertical and horizontal wall has been included in Tables 7 to 10. It has been observed that, for vertical wall, combination 4 gave the best fitness values of the effective width i.e., 5.003 (Table 7) and combination 7 gave the best fitness values of the effective height i.e., Fig. 12. Pseudocodes for the feature extractions. the optimum combination of process parameters for vertical and hori­ zontal wall. 2.2.4. Optimization of process parameter In the present study, optimization of power, scanning speed and powder feed rate of laser additive manufacturing process is done by using Genetic Algorithm (GA). Optimization has been done to achieve the targeted value of width as 5 mm and height as 14 mm for vertical wall and width as 20 mm and height as 6 mm for the horizontal wall. For vertical wall, Eqs. (6) and (7), while for the horizontal wall, Eqs. (8) and 639 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Table 2 Effective and total dimensions obtained for vertical wall. Exp no. Image processed effective width (mm) Image processed effective height (mm) Image processed effective area (mm2) Image processed total width (mm) Image processed total height (mm) Image processed total area (mm2) 1 2 3 4 5 6 7 8 9 10 11 12 4.95 4.98 3.75 4.98 3.71 4.91 4.99 5.06 5.01 4.96 4.95 5.12 13.98 13.96 13.91 14.05 13.83 13.07 14.02 13.99 13.94 14.01 14.15 12.59 69.2 69.5 52.1 70.0 50.9 64.2 70.0 70.8 69.8 69.5 70.0 64.5 6.35 6.62 6.38 7.22 6.15 6.02 7.14 6.86 6.96 6.40 6.49 7.23 15.17 14.43 14.72 15.05 15.50 14.23 14.39 14.28 14.83 15.55 15.87 14.10 70.63 75.06 63.86 82.62 60.73 70.63 74.93 73.94 76.48 77.68 70.68 76.33 Bold indicates the experiement combination that gave under or over dimensions for vertical wall. Table 3 Effective and total dimensions obtained for horizontal wall. Exp no. Image processed effective width (mm) Image processed effective height (mm) Image processed effective area (mm2) Image processed total width (mm) Image processed total height (mm) Image processed total area (mm2) 1 2 3 4 5 6 7 8 9 10 11 12 19.71 20.01 20.30 20.29 20.14 22.41 21.45 19.95 19.79 20.02 19.91 19.99 6.12 6.06 6.72 5.92 5.95 6.07 5.93 5.98 6.10 6.02 6.04 5.99 120.59 121.33 136.81 120.30 119.90 136.08 127.11 119.33 120.72 120.48 120.26 119.75 24.90 26.92 31.50 30.61 29.40 28.55 29.27 32.21 30.03 27.64 31.53 31.93 7.02 7.30 8.12 7.03 6.91 6.55 6.99 7.95 6.98 8.16 7.73 7.96 159.30 168.84 194.09 168.57 172.47 158.91 173.32 209.14 176.48 222.67 200.89 206.51 Bold indicates the experiement combination that gave under or over dimensions for horizontal wall. Fig. 13. Comparison of the equation predicted with the image processed effective width and height for vertical wall. 14.003 (Table 8). For horizontal wall, combination 6 gave the best fitness values of the effective width i.e., 20.003 (Table 9) and combi­ nation 2 gave the best fitness values of the effective height i.e., 6.002 (Table 10). Fig. 15 depicts the fitness values of effective vertical wall width (Fig. 15a), effective vertical wall height (Fig. 15b), effective horizontal wall width (Fig. 15c) and effective horizontal wall height (Fig. 15d) captured at various combinations of GA parameters. h = 6.27 + 0.86 × 10− 2 P + 0.56 × 10− 2 Ss + 0.977Pf − 5.64773 × 10− 6 P × Ss − 6.63 × 10− 5 Ss × Pf − 1.1 × 10− 3 P × Pf (7) b) Horizontal wall a) Vertical wall w = 18.82 − 1.71 × 10− 2 P − 1.41 × 10− 2 Ss − 0.586Pf + 1.81 × 10− 5 P × Ss +1.54 × 10− 5 Ss × Pf + 6.4 × 10− 4 P × Pf (6) 640 w = 19.71 + 0.13 × 10− 2 P − 0.154 × 10− 2 Ss + 0.051Pf (8) h = 5.68 + 0.11 × 10− 2 P − 0.66 × 10− 3 Ss − 0.031Pf (9) D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 14. Comparison of the equation predicted with the image processed effective width and height for the horizontal wall. Table 4 Bounds and constraints used in GA for optimization. Laser power ‘P’ Scanning speed ‘Ss’ Powder feed rate ‘Pf’ Effective vertical wall width Effective vertical wall height Effective horizontal wall width Effective horizontal wall height Table 7 Fitness values for the effective vertical wall width for different generations. 800 ≤ P ≤ 1000 500 ≤ Ss ≤ 700 5 ≤ Pf ≤ 10 Vw − 5.02 ≤ 0 Vw − 4.98 ≥ 0 Vh − 14.02 ≤ 0 Vh − 13.98 ≥ 0 Hw − 20.02 ≤ 0 Hw − 19.98 ≥ 0 Hh − 6.02 ≤ 0 Hh − 5.98 ≥ 0 Upper limit Lower limit Upper limit Lower limit Upper limit Lower limit Upper limit Lower limit Combination 4 Table 5 Values of GA parameters used in optimization. GA parameters Values Number of variables Population size Population type Creation function Number of generations Operator Crossover function Crossover fraction Mutation function Elite count Nonlinear constraint algorithm 3 50 and 100 Double vector Uniform 100 Roulette wheel Single point 0.80 and 0.85 Adaptive feasible 1 and 2 Augmented Lagrangian Table 6 Combination of GA parameters used in optimization. Combination Population size Crossover fraction Elite count 1 2 3 4 5 6 7 8 50 50 50 50 100 100 100 100 0.80 0.80 0.85 0.85 0.80 0.80 0.85 0.85 1 2 1 2 1 2 1 2 Generation Laser power Scanning speed Powder feed rate Fitness value 0 1 2 3 4 5 6 7 8 9 10 20 21 28 29 30 31 32 33 34 39 40 41 50 55 56 64 65 80 81 82 819.175 819.175 817.138 819.175 819.175 819.138 819.175 819.175 819.675 819.675 819.675 819.957 819.925 819.957 819.957 819.968 819.968 819.988 819.988 819.988 819.999 819.999 819.999 819.999 819.999 819.999 819.999 819.999 820.000 820.000 820.000 517.633 517.633 502.058 502.631 501.631 501.058 501.058 501.058 501.058 501.058 500.558 500.183 500.058 500.027 500.027 500.058 500.058 500.121 500.027 500.027 500.043 500.027 500.020 500.005 500.005 500.001 500.001 500.000 500.000 500.000 500.000 5.849 5.849 5.916 5.849 5.849 5.916 5.916 5.916 5.916 5.916 5.916 5.978 5.994 5.994 5.994 5.997 5.997 5.997 5.997 5.998 5.998 5.998 5.998 5.998 6.000 6.000 6.000 6.000 6.000 6.000 6.000 5.029 5.029 5.021 5.017 5.016 5.012 5.012 5.012 5.010 5.010 5.009 5.005 5.004 5.004 5.004 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 5.003 Italic indicates that the process parameters stabilizes after attaining minimum fitness value. study, for manual measurement of the deposition geometry features Catia V5 software is used. To measure the values of width, height and area, the deposition geometry is divided by known dimension grids. To obtain the dimensions, a total number of grids covering the desired features is multiplied with the single grid dimension. 4. Results and discussion 3. Manual measurement of deposition geometry features 4.1. Comparison of total dimensions against manual measurement In present study, manual measurement method has been used to validate the deposition geometry features extracted by image process­ ing. Currently, the additive manufacturing industries use a manual measurement technique and measure the deposition width, height, and area by using CAD software known as AutoCAD [28]. In the present In this section, the image processed deposition geometry features such as total width, height and area of the vertical and horizontal wall were compared with the manual measuring method. Figs. 16 and 17 depicts the comparison of the total deposition width and height for the 641 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Table 8 Fitness values for the effective vertical wall height for different generations. Table 10 Fitness values for the effective horizontal wall height for different generations. Combination 7 Combination 2 Generation Laser power Scanning speed Powder feed rate Fitness value Generation Laser power Scanning speed Powder feed rate Fitness value 0 1 2 3 4 5 6 7 8 9 10 11 12 18 19 20 21 22 23 24 25 26 27 28 29 30 36 37 38 39 40 60 61 62 849.321 849.321 849.321 849.321 849.321 849.321 849.821 849.821 849.821 849.821 849.821 849.821 849.946 849.961 849.961 849.961 849.961 849.977 849.993 849.993 849.993 849.993 849.993 849.993 849.993 849.996 849.988 849.996 849.996 849.999 850.000 850.000 850.000 850.000 504.584 501.236 503.584 501.761 500.761 500.761 501.261 500.261 500.261 500.236 500.236 500.236 500.511 500.136 500.011 500.011 500.011 500.136 500.011 500.011 500.011 500.011 500.011 500.011 500.011 500.105 500.103 500.011 500.003 500.011 500.011 500.000 500.000 500.000 10.137 10.118 10.085 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.023 10.008 10.008 10.008 10.008 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 14.009 14.008 14.008 14.007 14.007 14.007 14.005 14.004 14.004 14.004 14.004 14.004 14.004 14.004 14.004 14.004 14.004 14.004 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 14.003 0 1 2 3 4 5 6 7 12 13 14 15 16 17 18 19 25 26 34 45 59 60 61 860.330 860.330 860.330 861.463 861.963 860.330 860.463 860.330 860.080 860.080 860.080 860.048 860.048 860.048 860.048 860.017 860.017 860.005 860.005 860.005 860.001 860.001 860.001 686.832 687.832 687.832 698.540 698.540 698.540 698.540 699.540 699.790 699.790 699.790 699.915 699.915 699.915 699.977 699.977 699.997 699.977 699.997 699.997 700.000 700.000 700.000 6.990 6.990 6.990 6.893 6.990 6.990 6.998 6.990 6.998 6.998 6.998 6.998 6.998 6.998 6.998 6.998 6.999 6.999 7.000 7.000 7.000 7.000 7.000 6.011 6.011 6.011 6.008 6.005 6.004 6.004 6.003 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 6.002 Italic indicates that the process parameters stabilizes after attaining minimum fitness value. vertical wall (Fig. 16a) and the horizontal wall (Fig. 17a) respectively. Figs. 16b and 17b depict the total area of vertical and horizontal wall, respectively. The maximum percentage difference between the features obtained by image processing and manual measuring methods is 1.9% for total deposition width (Fig. 16a), 2.1% for deposition height (Fig. 16a), and 1.9% for total deposition area (Fig. 16b) for a vertical wall. While for the horizontal wall, the maximum percentage difference is 2.2% for total deposition width (Fig. 17a), 2.3% for deposition height (Fig. 17a) and 2.3% for total deposition area (Fig. 17b). Examining the uncertainty in the manual measured geometric features it was observed that the inaccuracy of the measuring method and the importance of the image processing technique to improve and automate the measuring process. Italic indicates that the process parameters stabilizes after attaining minimum fitness value. Table 9 Fitness values for the effective horizontal wall width for different generations. Combination 6 Generation Laser power Scanning speed Powder feed rate Fitness value 0 1 2 3 4 5 6 7 8 9 10 11 19 20 21 22 29 32 46 56 57 58 805.763 805.064 801.307 801.307 800.825 800.307 800.307 800.307 800.057 800.057 800.057 800.057 800.025 800.088 800.088 800.025 800.010 800.017 800.002 800.002 800.002 800.002 727.615 727.615 729.774 729.774 729.615 729.774 729.774 729.774 729.774 729.774 729.774 729.899 729.978 729.961 729.990 729.978 729.996 729.992 729.999 730.000 730.000 730.000 7.009 7.009 7.033 7.009 7.009 7.009 7.009 7.009 7.009 7.009 7.009 7.009 7.009 7.001 7.001 7.001 7.001 7.001 7.000 7.000 7.000 7.000 20.014 20.014 20.007 20.005 20.005 20.004 20.004 20.004 20.004 20.004 20.004 20.003 20.003 20.003 20.003 20.003 20.003 20.003 20.003 20.003 20.003 20.003 4.2. Comparison of effective dimensions against manual measurement Effective geometry features are the actual geometry features of the final components that should match the CAD model. This section de­ scribes the comparison of the image processed effective width, height and area of the vertical and horizontal wall with the manual measuring method and dimensions of CAD model. Figs. 18 and 19 show the com­ parison of the effective deposition width, and height (Figs. 18a and 19a), and deposition area (Figs. 18b and 19b) of the vertical and horizontal wall respectively. The maximum percentage difference between image processed and manual measured effective deposition width, height and area of the vertical wall is 2.1%. While for the horizontal wall, the maximum difference between the image processed and manual measured effective deposition width is 2.1%, 1.9% for effective depo­ sition height and 2.1% for effective deposition area. In the ideal deposition, the effective geometry features are equal to the features designed in the CAD model. In the present study, features of deposition geometries such as width, height and area in the CAD model of the vertical wall (Fig. 3a) are 5 mm, 14 mm and 70 mm2, respectively and of the horizontal wall (Fig. 3b) are 20 mm, 6 mm and 120 mm2, respectively. Comparing the effective geometry feature with the CAD model, it has been observed that the vertical wall manufactured by the process parameters in experiments 3, 5, 6 and 12 gave unacceptable dimensions. For the horizontal wall manufactured by the process Italic indicates that the process parameters stabilizes after attaining minimum fitness value. 642 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 15. Fitness values of the (a) effective vertical wall width, (b) effective vertical wall height, (c) effective horizontal wall width and (d) effective horizontal wall height obtained for various combination GA parameters. 643 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 15. (continued). Fig. 16. Deposition geometry features (a) total deposition width and height and (b) total deposition area of the vertical wall. Fig. 17. Deposition geometry features (a) total deposition width and height and (b) total deposition area of the horizontal wall. parameters in experiments 3, 6 and 7 gave unacceptable dimensions. Process parameters in other experimental runs gave the effective di­ mensions in an acceptable range. Therefore, it can be said that some combination of process parameters can give dimensions desired to CAD model while some combinations are not suitable. Therefore, it become 644 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 19. Deposition geometry features (a) effective deposition width and height and (b) effective deposition area of the horizontal wall. Fig. 18. Deposition geometry features (a) effective deposition width and height and (b) effective deposition area of the vertical wall. Table 11 Optimized process parameters obtained from GA. most important to identify the optimum combination of process pa­ rameters that can give the desired dimensions. Vertical wall 4.3. Validation of optimized process parameters Horizontal wall Table 11 shows the optimized combination of process parameters using GA that can be used to manufacture vertical and horizontal wall. It has been observed that, to manufacture the vertical wall with the acceptable effective wall width the optimized value of laser power as 820 W, scanning speed as 500 mm/min and powder feed rate as 6 g/min should be used. While for the acceptable effective wall height the opti­ mized value of laser power as 850 W, scanning speed as 500 mm/min and powder feed rate as 10 g/min should be used. To manufacture the horizontal wall with the acceptable effective wall width the optimized value of laser power as 800 W, scanning speed as 730 mm/min and powder feed rate as 7 g/min should be used. While for the acceptable effective wall height the optimized value of laser power as 860 W, scanning speed as 700 mm/min and powder feed rate as 7 g/min should be used. Fig. 20 depicts the vertical wall (Fig. 20a) and the horizontal wall (Fig. 20b) manufactured by the optimized process parameters. Table 12 shows the values of the effective width, height and area ob­ tained using optimized process parameters for the vertical and hori­ zontal wall. Comparing the effective dimensions with the CAD model Features Power (W) Scanning speed (mm/min) Powder feed rate (g/min) Width Height Width Height 820 850 800 860 500 500 730 700 6 10 7 7 dimensions it was observed that for the vertical wall, the maximum percentage difference is 0.56% for width, 0.11% for height and 0.63% for area. Consequently, for the horizontal wall, the maximum percent­ age difference is 0.09% for width, 0.46% for height and 0.53% for the area. Comparison shows that the optimized process parameters can manufacture the vertical and horizontal wall in good agreement with the CAD model. 5. Conclusion The present research work has been mainly focused on the devel­ opment of the robust image-processing technique to automate the pro­ cess of geometry features extraction, compare the extracted geometry features with CAD model, and identify the optimum combination of process parameters with a capability to manufacture components by laser additive manufacturing process. Based on the objectives of the 645 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 Fig. 20. Deposition geometries manufactured by using modified process parameters on laser additive manufacturing process. • Comparing CAD model dimensions against the dimensions of the deposition geometry obtained by optimized process parameters confirmed that for the vertical wall, the accuracy of measurement is 0.56% for width, 0.11% for height and 0.63% for area. Consequently, for the horizontal wall, the accuracy of measurement is 0.09% for width, 0.46% for height and 0.53% for the area. Table 12 Effective dimensions obtained by optimized process parameters. Vertical wall Horizontal wall Image processed effective width (mm) Image processed effective height (mm) Image processed effective area (mm2) 5.028 5.021 20.018 20.013 14.010 14.016 6.015 6.028 70.442 70.374 120.408 120.638 In the past literature, it was observed that to measure the features of the deposition geometries obtained by direct energy deposition process, manual measurement techniques were used. This leads to compromise the accuracy and the speed of the measuring process. However, in the fast-growing additive manufacturing industry, to overcome this issue, the proposed image processing technique will benefit the measuring process by automatically extracting the complex features of the component with high accuracy and less human interference. In future, the image processing algorithm of this research will be further devel­ oped for the real-time feature extraction of the depositions done by laser based additive manufacturing process. current research following conclusions are drawn. • The proposed robust methodology for image processing is highly reliable to automate the process of deposition geometry feature extraction and has ability to measure the total and effective geometry features with good accuracy. • Comparing the dimensions between the image processing and manual measurement method, the accuracy is 2.1% for the total width and 2.0% for the total height of vertical wall and the accuracy for the total width and height of the horizontal wall is 2.2% and 2.3% respectively. • Comparison of the effective geometric features by image processing against the manual measuring method it was observed that the maximum percentage difference for vertical wall is 2.2% for width, 2.1% height and 2.1% for area. While for horizontal wall, the maximum percentage difference is 2.2% for width, 1.8% for height and 2.2% for area. • The maximum percentage error between the geometric feature extracted by image processing technique is less than 3% this signifies that the ability and accuracy to measure the geometric features of additive manufactured. • The optimized process parameters suitable for effective vertical wall width are, the laser power as 820 W, scanning speed as 500 mm/min and powder feed rate as 6 g/min. For effective vertical wall height are, the laser power as 850 W, scanning speed as 500 mm/min and powder feed rate as 10 g/min. • The optimized process parameters suitable for effective horizontal wall width are, the laser power as 800 W, scanning speed as 730 mm/ min and powder feed rate as 7 g/min. For effective horizontal wall height are, the laser power as 860 W, scanning speed as 700 mm/min and powder feed rate as 7 g/min. Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgements The authors acknowledge the financial support given under SEED money scheme of Birla Institute of Technology Mesra, Ranchi, Jhark­ hand (India). Appendix A. Supplementary data Supplementary data to this article can be found online at https://doi. org/10.1016/j.jmapro.2021.07.064. References [1] Ranjan R, Khan AR, Parikh C, Jain R, Mahto RP, Pal S, et al. Classification and identification of surface defects in friction stir welding: an image processing approach. J Manuf Process 2016;22:237–53. https://doi.org/10.1016/j. jmapro.2016.03.009. 646 D.B. Patil et al. Journal of Manufacturing Processes 69 (2021) 630–647 [2] Bhat NN, Kumari K, Dutta S, Pal SK, Pal S. Friction stir weld classification by applying wavelet analysis and support vector machine on weld surface images. J Manuf Process 2015;20:274–81. https://doi.org/10.1016/j.jmapro.2015.07.002. [3] Yuan Y, Zhang D, Zhu H, Ehmann KF. Machining of micro grayscale images on freeform surfaces by vibration-assisted cutting. J Manuf Process 2020;58:660–7. https://doi.org/10.1016/j.jmapro.2020.08.051. [4] Rane K, Castelli K, Strano M. Rapid surface quality assessment of green 3D printed metal-binder parts. J Manuf Process 2019;38:290–7. https://doi.org/10.1016/j. jmapro.2019.01.032. [5] Duda RO, Hart PE. Use of the Hough transformation to detect lines and curves in pictures. Commun ACM 1972;15:11–5. https://doi.org/10.1145/361237.361242. [6] Kiryati N, Eldar Y, Bruckstein AM. A probabilistic Hough transform. Pattern Recognit 1991;24:303–16. https://doi.org/10.1016/0031-3203(91)90073-E. [7] McLaughlin RA. Randomized Hough transform: improved ellipse detection with comparison. Pattern Recognit Lett 1998;19:299–305. https://doi.org/10.1016/ S0167-8655(98)00010-5. [8] Chen S, Xia R, Zhao J, Chen Y, Hu M. A hybrid method for ellipse detection in industrial images. Pattern Recognit 2017;68:82–98. https://doi.org/10.1016/j. patcog.2017.03.007. [9] Kasemir KU, Betzler K. Detecting ellipses of limited eccentricity in images with high noise levels. Image Vis Comput 2003;21:221–7. https://doi.org/10.1016/ S0262-8856(02)00155-5. [10] Alfano M, Di Girolamo G, Pagnotta L, Sun D. Processing, microstructure and mechanical properties of air plasma-sprayed ceria-yttria co-stabilized zirconia coatings. Strain 2010;46:409–18. https://doi.org/10.1111/j.14751305.2009.00659.x. [11] Thao DT, Kim IS. Interaction model for predicting bead geometry for lab joint in GMA welding process. Arch Comput Mater Sci Surf Eng 2009;1:237–44. [12] Sathiya P, Aravindan S, Ajith P, Arivazhagan B, Haq AN. Microstructural characteristics on bead on plate welding of AISI 904 L super austenitic stainless steel using gas metal arc welding process. Int J Eng Sci Technol 2011;2:189–99. https://doi.org/10.4314/ijest.v2i6.63710. [13] El Rayes M, Walz C, Sepold G. The influence of various hybrid welding parameters on bead geometry. Weld J 2004;83:147. [14] Cao Y, Zhu S, Liang X, Wang W. Overlapping model of beads and curve fitting of bead section for rapid manufacturing by robotic MAG welding process. Robot Comput Integr Manuf 2011;27:641–5. https://doi.org/10.1016/j. rcim.2010.11.002. [15] Singh AK, Dey V, Rai RN, Debnath T. Weld bead geometry dimensions measurement based on pixel intensity by image analysis techniques. J Inst Eng (India) C 2019;100:379–84. https://doi.org/10.1007/s40032-017-0392-8. [16] Tabernero I, Lamikiz A, Martínez S, Ukar E, Figueras J. Evaluation of the mechanical properties of Inconel 718 components built by laser cladding. Int J [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] [27] [28] 647 Mach Tool Manuf 2011;51:465–70. https://doi.org/10.1016/j. ijmachtools.2011.02.003. Putthawong S, Koomsap P, Chansri N. Image-based adaptive crosshatch toolpath generation for laminated object manufacturing. Virtual Phys Prototyp 2014;9: 233–49. https://doi.org/10.1080/17452759.2014.954823. Vaidya R, Anand S. Image processing assisted tools for pre- and post-processing operations in additive manufacturing. Procedia Manuf 2016;5:958–73. https://doi. org/10.1016/j.promfg.2016.08.084. Yao B, Imani F, Sakpal AS, Reutzel EW, Yang H. Multifractal analysis of image profiles for the characterization and detection of defects in additive manufacturing. J Manuf Sci Eng 2018:140. https://doi.org/10.1115/1.4037891. Straub J. Initial work on the characterization of additive manufacturing (3D printing) using software image analysis. Machines 2015;3:55–71. https://doi.org/ 10.3390/machines3020055. Wang T, Zhu YY, Zhang SQ, Tang HB, Wang HM. Grain morphology evolution behavior of titanium alloy components during laser melting deposition additive manufacturing. J Alloys Compd 2015;632:505–13. https://doi.org/10.1016/j. jallcom.2015.01.256. Patil D, Nigam A, Mohapatra S. Automation of geometric feature computation through image processing approach for single-layer laser deposition process. Int J Comput Integr Manuf 2020;33:895–910. https://doi.org/10.1080/ 0951192X.2020.1815843. Canny J. A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 1986;PAMI-8:679–98. https://doi.org/10.1109/ TPAMI.1986.4767851. Gunaraj V, Murugan N. Prediction and optimization of weld bead volume for the submerged arc process –part 2. Weld J Res Suppl 2000;11:331–8. Liu H, Qin X, Huang S, Jin L, Wang Y, Lei K. Geometry characteristics prediction of single track cladding deposited by high power diode laser based on genetic algorithm and neural network. Int J Precis Eng Man 2018;19:1061–70. https://doi. org/10.1007/s12541-018-0126-8. Kumar A, Maji K. Selection of process parameters for near-net shape deposition in wire arc additive manufacturing by genetic algorithm. J Mater Eng Perform 2020; 29:3334–52. https://doi.org/10.1007/s11665-020-04847-1. Conn AR, Gould NIM, Toint PL. A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds. SIAM J Numer Anal 1991;28:545–72. https://doi.org/10.1137/0728030. Sawant MS, Jain NK, Nikam SH. Theoretical modeling and finite element simulation of dilution in micro-plasma transferred arc additive manufacturing of metallic materials. Int J Mech Sci 2019:164. https://doi.org/10.1016/j. ijmecsci.2019.105166.