In computer vision and computer graphics, 3D reconstruction is the process of capturing the shape and appearance of real objects. This process can be solved by active or passive methods. If the model is allowed to change its shape in time, it is called non-rigid or spatio-temporal reconstruction.
Video 3D reconstruction
Motivation and apps
3D reconstruction research has always been a difficult destination. Using a 3D reconstruction one can define any 3D object profile, as well as know the 3D coordinates from any point in the profile. Reconstruction of 3D objects is a generally scientific problem and core technology from various fields, such as Computer Aided Geometric Design (CAGD), Computer Graphics, Computer Animation, Computer Vision, medical imaging, computational science, Virtual Reality, digital media, etc. For example, patient's patient lesion information can be presented in 3D on a computer, offering an accurate approach in diagnosis and thus having a vital clinical value.
Maps 3D reconstruction
Active methods
The active method, ie the range data method, given the depth map, reconstructs the 3D profile with the approach of numerical approach and construct the object in the model based scenario. This method actively disrupts reconstructed objects, either mechanically or radiometrically using a distance meter, to obtain a depth map, ie. light structured, laser range finder and other active sensing techniques. A simple example of a mechanical method would be to use a depth meter to measure the distance to a rotating object placed on the turntable. More useful radiometric methods emit light against the object and then measure the reflected part. Examples range from moving the light source, visible color, laser time-of-flight to microwaves or ultrasound. See 3D scanning for more details.
Passive method
The 3D passive reconstruction method does not interfere with reconstructed objects; they only use sensors to measure the jets that are reflected or emitted by the surface of the object to infer the 3D structure through image understanding. Typically, the sensor is a camera-sensitive image sensor in the visible light and the input to the method is a set of digital images (one, two or more) or video. In this case we are talking about image-based reconstruction and the result is a 3D model. Compared to the active method, passive methods can be applied to a wider range of situations.
Methods of edged signs
The monocular marking method refers to the use of images (one, two or more) from a single point of view (camera) to continue 3D construction. It uses 2D characteristics (eg Silhouette, shadows and textures) to measure 3D shapes, and that is why it is also named Shape-From-X, where X can be silhouettes, shadows, textures, etc. 3D reconstruction through simple and fast monocular gestures, and only one suitable digital image is required so that only one camera is adequate. Technically, it avoids stereo correspondence, which is quite complicated.
Shape-from-shading Due to the analysis of image information in the image, using Lambertian reflections, the normal depth of information from the surface of the object is restored to reconstruct.
Photometric Stereo This approach is more sophisticated than the shape-of-shading method. Images taken in various lighting conditions are used to solve depth information. It should be mentioned that more than one image is required by this approach.
Form-of-texture Suppose such an object with a smooth surface is covered by a replicated texture unit, and projections from 3D to 2D cause distortion and perspective. The distortion and perspective measured in 2D images provide clues to invertly resolve the normal depth of information from the surface of the object.
stereo vision binocular
Binocular Stereo Vision obtains 3-dimensional geometric information of an object from several images based on human visual system research. The results are presented in depth maps. Image objects obtained by two cameras simultaneously in different angles, or by a single camera at different times in different angles, are used to restore 3D geometric information and reconstruct its 3D profile and location. This is more direct than a cutting-edge method like form-of-shading.
The binocular stereo vision method requires two identical cameras with a parallel optical axis to observe the same object, obtaining two images from different angles. In terms of trigonometric relationships, in-depth information can be calculated from disparities. The stereo vision vision method is well developed and stably contributes to profitable 3D reconstruction, leading to better performance when compared to other 3D constructions. Unfortunately, it is computationally intensive, otherwise it performs rather poorly when the baseline distance is large.
Problem statement and basics
The approach using Binocular Stereo Vision to obtain 3D geometric object information is based on visual differences. The following figure provides a simple schematic diagram of a horizontally sighted Stereo Binocular Vision, where b is the base line between the projective centers of the two cameras.
The origin of the camera coordinate system is in the optical center of the camera lens as shown in the figure. Actually, the camera image area is behind the optical center of the camera lens. However, to simplify the calculation, the image is taken in front of the optical center of the lens by f. The u-axis and v-axis of the image coordinate system of O 1 uv are in the same direction as the x-axis and y-axis of the camera coordinate system respectively. The origin of the image coordinate system lies at the junction of the imaging plane and the optical axis. Suppose a world point such as P whose corresponding image point is P 1 (u 1 , v 1 ) and P 2 (u 2 , v 2 ) respectively in the left and right image area. Assume two cameras are in the same plane, then the y coordinates of P 1 and P 2 are identical, that is, v 1 = v 2 . According to trigonometric relationships,
where x is the coordinates of P in the left camera coordinate system, f is the focal length of the camera. Visual disparity is defined as the difference in the location of the drawing point from a given world point obtained by two cameras,
based on which coordinate P can be done.
Therefore, once the coordinates of the image points are known, in addition to the two camera parameters, the 3D coordinates of that point can be determined.
3D reconstruction consists of the following sections:
Acquisition of image
The acquisition of 2D digital images is a 3D reconstruction information resource. The commonly used 3D reconstruction is based on two or more images, although it may only use one image in some cases. There are different types of methods for image acquisition that depend on the occasion and purpose of a particular application. Not only are application requirements to be met, but also visual disparity, lighting, camera performance, and scenario features should be considered.
Camera calibration
Camera Calibration in Stereo Vision Binocular refers to the determination of the mapping relationship between image points P 1 (u 1 , v 1 ) and P 2 (u 2 , v 2 ), and the coordinate space P (x p , y p , z p ) in the 3D scenario. Camera calibration is the basic and important part in 3D reconstruction through Stereo Vision Binoculars.
Feature extraction
The purpose of feature extraction is to obtain the characteristic of the image, through the process of stereo correspondence. As a result, the image characteristics are closely related to the choice of matching method. There is no universally applicable theory for feature extraction, leading to a remarkable diversity of stereo correspondence in the Stereo Vision Binocular study.
Stereo correspondence
The stereo correspondence is to form a correspondence between the primitive factors in the image, ie to match P 1 (sub <1 , v 1 ) and P 2 (u 2 , v 2 ) of the two images. Certain interference factors in the scenario must be considered, eg. lighting, noise, surface physical characteristics and others
Recovery
According to proper correspondence, combined with camera location parameters, 3D geometric information can be recovered without difficulty. Due to the fact that 3D reconstruction accuracy depends on correspondence accuracy, camera location parameter errors and so on, previous procedures must be performed carefully to achieve a relatively accurate 3D reconstruction.
3D Image Reconstruction
Routine clinical diagnosis, patient follow-up, computer-assisted surgery, surgical planning etc. are facilitated by accurate 3D models of desired human anatomical parts. The main motivations behind 3D reconstruction include
- Improved accuracy for multiple aggregate views.
- Detailed surface estimates.
- Can be used to plan, simulate, guide, or assist a surgeon in performing medical procedures.
- The exact position and orientation of the patient's anatomy can be determined.
- Assist in a number of clinical areas, such as radiotherapy planning and treatment verification, spine surgery, hip replacement, neurointerventions and aortic stenting.
Apps:
The 3D reconstruction system finds its applications in various fields
- Medicine
- Movie industry
- Robotics
- Town planning
- Games
- Virtual environment
- Earth Observation
- Archeology
- Augmented reality
- Reverse engineering
- Animation
- Human computer interactions
Problem Statement:
Most of the algorithms available for 3D reconstruction are very slow and can not be used in real-time. Although the algorithm presented is still in the early stages but they have the potential for quick calculation.
Existing Approach:
Delaunay and alpha-form
- The Delaunay method involves extracting the surface of the tetrahedron from the cloud's starting point. The idea of ​​'form' for a series of points in space is given by the concept of alpha-form. Given set of endpoints S, and alpha real parameters, the alpha-S form is a polytope (generalization to every dimension of a two-dimensional polygon and a three-dimensional polyhedron) that is not convex or connected. For large values, the alpha form is identical to the convex-curve S. The algorithm proposed by Edelsbrunner and Mucke removes all tetrahedron bounded by the surrounding sphere smaller than. The surface is then obtained with the external triangle of the resulting tetrahedron.
- Another algorithm called Strict Cocone labeled the early tetrahedron as interior and exterior. The triangles found in and out produce the resulting surface.
Both methods have recently been extended to reconstruct cloud points with noise. In this method the quality of the points determines the feasibility of the method. For proper triangulation because we use the entire cloud point, a point on the surface with errors above the threshold will be explicitly represented on the reconstructed geometry.
Setting Zero Method
Surface reconstruction is performed using a distance function that assigns to any point in the space signed by the distance to the surface of S . A contour algorithm is used to extract a zero-set used to obtain a polygonal representation of an object. Thus, the problem of reconstructing the surface of an unorganized point cloud is reduced to the exact function definition f with a zero value for the sample point and is different from the zero for the rest. An algorithm called marching cubes specifies the use of the method. There are different variants for the given algorithm, some using the discrete function f , while others using the polyharmonic radial base function are used to adjust the set of the starting point. Functions such as Moving Least Squares, basic functions with local support, based on Poisson equations have also been used. Loss of precision geometry in areas with extreme curvature, ie, angles, edges is one of the major problems faced. Furthermore, pretreatment of information, by applying some sort of screening technique, also affects the definition of the angle by softening it. There are several studies related to post-processing techniques used in reconstruction for detection and refinement of angles but this method increases the complexity of the solution.
Teknik VR
The transparency of the overall volume of objects is visualized using the VR technique. The image will be done by projecting the ray through the volume data. Throughout each ray, opacity and color need to be calculated on each voxel. Then the information calculated along each ray will be aggregated to a pixel on the image plane. This technique helps us see comprehensively the entire compact structure of the object. Since this technique requires a lot of calculations, requiring a powerful configuration computer is appropriate for low-contrast data. The two main methods for projecting rays can be considered as follows:
- The object-order method: Projects the light through the volume from back to front (from volume to plane image).
- The method of image-order or casting of light: Projecting rays through a volume from front to back (from image area to volume). There are several other methods for composite drawing, the exact method depends on the user's purpose. Some common methods in medical images are MIP (maximum intensity projection), MinIP (minimum intensity projection), AC (alpha compositing) and NPVR (non-photorealistic rendering volume).
Voxel Grid
In the input space this filtering technique is sampled using 3D grid voxels to reduce the number of points. For each voxel, the centroid is selected as representative of all points. There are two approaches, the selection of voxel centroid or select the center point of the points located inside the voxel. To get the average internal points has a higher computing cost, but offers better results. Thus, a subset of the input space is obtained which roughly represents the underlying surface. The Voxel Grid method presents the same problem as other filtering techniques: the impossibility of defining the final number of points representing the surface, the loss of geometric information due to the reduction of dots inside the voxel and the sensitivity to the noisy input space.
External links
- Synthesize 3D Shapes through Multi-View Depth Map Modeling and Silhouettes with Deep Generative Network - Generate and reconstruct 3D shapes through multi-view depth map modeling or silhouette.
See also
- 3D modeling
- 3D data acquisition and object reconstruction
- 3D reconstruction of many images
- 3D Scanner
- 4D Reconstruction
- Depth map
- Kinect
- Photogrammetry
- Stereoscopy
References
External links
- http://www.nature.com/subjects/3d-reconstruction#news-and-comment
- http://6.869.csail.mit.edu/fa13/lectures/lecture11shapefromX.pdf
- http://research.microsoft.com/apps/search/default.aspx?q=3d reconstruction
- https://research.google.com/search.html#q=3D reconstruction
Source of the article : Wikipedia
0 Comments