3D Reconstruction using convolution smooth method

Received Aug 15, 2019 Revised Sep 23, 2020 Accepted Apr 30, 2021 3D imagery is an image with depth data. The use of depth information in 3D images still has many drawbacks, especially in the image results. Raw data on the 3D camera even does not look smooth, and there is too much noise. Noise in the 3D image is in the form of imprecise data, which results in a rough image. This research will use the convolution smooth methods to improve the 3D image. Will smooth noise in the 3D image, so the resulting image will be better. This smoothing system is called the blurring effect. This research has been tested on flat objects and objects with a circle contour. The test results on the flat surface obtained a distance of 1.3177, the test in the object with a flat surface obtained a distance of 0.4937, and the test in circle contour obtained a distance of 0.3986. This research found that the 3D image will be better after applying the convolution smooth method.


INTRODUCTION
Image is an array containing valuable information that represents a number to represent a color. Each array value in an image is called a pixel [1][2][3]. The number of pixels in an image will determine the image's quality, the more pixels in an image, the better the image [4,5]. The image's information is divided to be two-part, that is 2D images and 3D images [2,3]. All images are a series of 2D images, but for 3D images, the information contained in the image contains depth [6][7][8]. In image processing, there is an image repair method. One of them is the convolution smooth method. The convolution smooth method is a technique for smoothing an image or clarifying the image by replacing the pixel value with a pixel number value corresponding to or adjacent to the original pixel. The convolution smooth method will not change the image size and quality [9][10][11].
In taking 3D image data, two types of cameras are commonly used, namely stereo cameras and cameras with IR sensors. This research will use an Intel RealSense Camera produced by Creative with the name Senz3D. with the model code VF0780 [12,13]. The features provided by the Senz3D camera are gesture control, 3D face sensing, voice control, and dual microphone. The result of this camera is a 2D image with depth information [14,15]. However, the results issued from the camera are still not perfect. Therefore it is necessary to do image processing to improve these results. One of the image processing methods is convolution smooth. From the description problem above, the researcher concludes that need do research about repair 3D image with the theme "3D reconstruction using convolution smooth method".

CONVOLUTION SMOOTH
In this step, the literature review related to research, like chose the related method for image processing, a tool for image processing, and so on. On this step, the 3D data is collected [16,17]. The tool used to get 3D data is Senz3D, shown in Figure 1, and the result from getting data from Intel RealSense is shown in Figure 2. In this research, we will use the 3D camera of Senz3D from Creative. The Senz3D will take two types of data. The first is a 2D image with RGB's information color and a 3D image with information depth. When getting data, data to be saved in the 2D image and 3D image [12,13]. In this research, data will show in Figure 2, from the data will be processed for correction until it can use on the next step, like animation or 3D scanning.

Smooth
Convolution is a mathematical operation on two functions (f and g) that produces a third function (f*g). In this research, the first function to process data from the raw image can be called function f. The second function is to process the average image from raw data in one area and be called function g [18,19]. Convolution has two functions f (x) and g (x), which are defined as follows: In this case, the sign (*) states that the convolution operator and variable α are auxiliary variables. For image processing, the operation performed is discrete because pixel coordinate values are discrete values. Furthermore, the filter or mask used in image processing is usually of limited size, in the sense that the weight or influence of the points far enough is insignificant so that it can be ignored (considered zero). The discrete form of one-dimensional convolution operation in image processing is: For functions with two dimensions, convolution operations defined as follows: -For integral: The g (x, y) filter function is also called the convolution filter, convolution mask, convolution kernel, or template. In a discrete form, the kernel convolution is expressed in a matrix (generally a 3x3 matrix). The size of this matrix is usually smaller than the size of the image. Each matrix element is called the convolution coefficient [20][21][22].

3-Dimension
Image is two-dimensional imagery produced from a two-dimensional analog image that is continuing into a discrete image through a sampling process [2,3]. Analog images are divided into N rows and M columns so that they become discrete images. Crosses between certain rows and columns are called pixels. 3D imagery is a three-dimensional representation of an image. The 3D image contains the parallel X-axis and Y-axis information and contains information about the Z-axis. The 3D image in this research is an image with a two-dimensional array containing information depth of an area. The information contained in the senz3D camera contains contours from the object's surface that are shot [23][24][25]. Figure 3 shows the result of 3D image capture from the Senz3D camera. The results taken from the Senz3D camera contain depth information, which is the distance between objects to the camera [12,13]. The spectrum color from Figure 3 is the indicator closer to the camera. The red color is an indicator that is more close to the camera, and the blue color is farther from the camera. From the color of Figure 3 can get information about the contour from the image.

RESEARCH METHOD 3.1. Analysis
In this step, we will test the system design. This study will use the MATLAB editor as a data processor. The step in data processing is shown in Figure 4. It explained that the data retrieval process gets results from data processing. The step of system design for processing data will make using MATLAB. In the system design implementation, using convolution smooth method and the results of this research are improved 3D images. Figure 4 is the step of processing data from image raw data to the final image after smoothing. The processing data will be doing every array image from one area until all the area. The research result is a good 3D image.

Data normalization
The results taken by 3D cameras are in the form of raw data, and the results of 3D imaging data still show differences in uneven contours. Data in this research will be using 3D face. Data were taken using the Senz3D camera and the unity game engine. 3D image is data getting from Senz3D. It can be seen in Figure 5 the data still looks rough and the data will be processed using the convolution smooth method so that the resulting data becomes smoother and as expected. Figure 6 is represents the 1D from Figure 5 and a sample data from the 3D image before smoothing. Therefore the data in Figure 6 still looks rough.  Figure 6. Sample data from Figure 5 Raw data obtained from the 3D camera will be processed to get maximum results, and the result in Figure 7 and in one line is shown in Figure 8. In Figure 7, the results of image improvement have shown a better result. The research will be conducted from this data development so that data can be collected together with other 3D data. As with reliefs on archaeological objects, such as in temple buildings, the results of this research can be developed to collect relief data in ancient reliefs such as in temple buildings. Because the relief in the temple building is made horizontally and almost surrounds the temple building. Thus the relief in the building can be documented properly in 3D. Figure 8 is the result of processing data in Figure 5. It can be seen in the data in Figure 8 that it has better results and has followed the contours of actual data.

RESULTS AND ANALYSIS 4.1. Testing
The testing phase will be done twice, the first one is testing for a flat plane, and the second is testing objects 3D. In this testing phase, it is used to find out how much error occurred in applying the convolution smooth method for 3D image improvement.

Tests in flat fields
Testing on the flat plane is carried out to measure the validation of the convolution smooth method's application. The test is carried out on objects with a flat surface, as this test is carried out on a wall. Figure 9 shows the shape of the wall being tested, and data collection using the SenZ3D camera can be seen in Figure 10, and Figure 11 is an example of one line data from Figure 10.  Figure 11 Sample data from Figure 10 Raw data from the 3D image shown in Figure 10 is the original data before applying the convolution smooth method. From the raw data shown in Figure 10, it can be seen that there is still a lot of noise. This shows that the results of taking pictures using the SenZ3D camera are not flat. As can be clarified in Figure 11, the wall surface does not show a flat surface.
In the next step, the data in Figure 10 will be smoothed data using a convolution smooth method, the results of smoothing data results can be seen in Figure 12, and Figure 13 is an example of one line data from Figure 12. Implementation of the convolution smooth method gives a good effect. The results of applying this method in image 3D can be seen in Figure 12, and in Figure 13 are the results of taking one line of data from Figure 12. Figure 12. Raw data after convolution smooth Figure 13. Sample data from Figure 12 When compared to the data in Figure 11 with Figure 13, the results are very significant. Figure 12 is more smooth than Figure 10. The noise in Figure 10 can be reduced. This comparison can be seen in Figure 14. Figure 14 compares data that has not been and has been carried out by convolution smooth. The blue line is data where the convolution smooth method has not been applied, and the red line is data that has been applied to the convolution smooth method. In Figure 14, you can see the data improvement with excellent noise removal. The value of data improvement on the implementation of the convolution smooth method is 1.3177. In addition to the implementation to the wall as a flat plane, this research also tested objects that have a flat surface, such as a handphone. Data can be seen in Figures 15-17. Testing on an object with a flat surface with a smaller surface ratio than testing on a wall image will test the sensor on the SenZ3D camera. Cameras with IR sensors to get depth information will give a different response. The noise on objects with a flat surface has less noise than the test on the wall, the data can be seen in Figure 17, and the raw data can be seen in Figure 16. In Figure 17, the noise generated looks lower than in the previous test.  Figure 17. Sample data from Figure 16 The results of applying the convolution smooth method can be seen in Figures 18 and 19. Implementing the convolution smooth method to objects with a flat surface such as Figure 18 has the result 0.4937. The comparison of results on image data 12 and image 18 is due to the sensor's performance on the SenZ3D camera. The accuracy will be higher when working with smaller objects. Figure 18. Raw data Figure 16 after convolution smooth Figure 19. Sample data from Figure 18

Tests on 3D objects in 3D
In addition to testing on a flat surface and objects with a flat surface, testing is also carried out on objects with an uneven surface. The test in this research was carried out on a glass with a circular surface. Test data, as shown in Figures 20-22. The results of data on objects with a circular surface can be seen in Figure 21. Figure 22 shows the data in the form of one line. On objects with a circular surface it can be seen that the IR sensor waves are deflected so that the results of data collection show that the object looks sharp. In Figure 23 is the result of testing on a circular object. In this test, the shape of the object changed from its original shape. This was caused by the IR sensor on the SenZ3D camera. On testing objects with a circular surface, the result is 0.3986. The results are almost the same as testing on objects with a flat surface. Figure 24 shows sample data from Figure 23.

CONCLUSION
In this research, the implementation of the convolution smooth method in improving 3D images was carried out. The test is carried out in three steps, the first on a flat surface, the second on an object with a flat surface, and the third on an object with a circular surface. The results of the first test obtained a distance of 1.3177, and in the second test, a distance of 0.4937 was obtained, and the third test obtained a distance of 0.3986. The results of these tests indicate that the convolution smooth method can work well. Besides that, the IR sensor on the SenZ3D camera will work better on smaller objects and on circular surfaces. The SenZ3D does not work well on data collection.