Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots

Turan, Mehmet
Pilavci, Yusuf Yigit
Ganiyusufoglu, Ipek
Araujo, Helder
Konukoglu, Ender
Sitti, Metin
Despite significant progress achieved in the last decade to convert passive capsule endoscopes to actively controllable robots, robotic capsule endoscopy still has some challenges. In particular, a fully dense three-dimensional (3D) map reconstruction of the explored organ remains an unsolved problem. Such a dense map would help doctors detect the locations and sizes of the diseased areas more reliably, resulting in more accurate diagnoses. In this study, we propose a comprehensive medical 3D reconstruction method for endoscopic capsule robots, which is built in a modular fashion including preprocessing, keyframe selection, sparse-then-dense alignment-based pose estimation, bundle fusion, and shading-based 3D reconstruction. A detailed quantitative analysis is performed using a non-rigid esophagus gastroduodenoscopy simulator, four different endoscopic cameras, a magnetically activated soft capsule robot, a sub-millimeter precise optical motion tracker, and a fine-scale 3D optical scanner, whereas qualitative ex-vivo experiments are performed on a porcine pig stomach. To the best of our knowledge, this study is the first complete endoscopic 3D map reconstruction approach containing all of the necessary functionalities for a therapeutically relevant 3D map reconstruction.
Machine Vision and Applications


Endo-VMFuseNet: A Deep Visual-Magnetic Sensor Fusion Approach for Endoscopic Capsule Robots
Turan, Mehmet; Almalioglu, Yasin; Gilbert, Hunter B.; Sari, Alp Eren; Soylu, Ufuk; Sitti, Metin (2018-05-25)
In the last decade, researchers and medical device companies have made major advances towards transforming passive capsule endoscopes into active medical robots. One of the major challenges is to endow capsule robots with accurate perception of the environment inside the human body, which will provide necessary information and enable improved medical procedures. We extend the success of deep learning approaches from various research fields to the problem of sensor fusion for endoscopic capsule robots in the...
Optimum Design of 6R Passive Haptic Robotic Arm for Implant Surgery
Yılmaz, Serkan; Konukseven, Erhan İlhan (2010-07-10)
The aim of this research was to design an optimum 6R passive haptic robotic arm (PHRA) to work in a limited workspace during dental implant surgery. Misplacement of an implant during dental surgery causes longer recuperation periods and functional disorders. In this study, a passive guidance robot arm was designed as a surgical aid tool for a dentist during the operation to reduce the surgical complications. Optimum design of a 6R robot is a complex issue since minimum energy has to be consumed while maximu...
Trajectory tracking control of robots with flexible links
Ider, SK; Özgören, Mustafa Kemal; Ay, V (2002-11-01)
A new method is developed for the end-effector trajectory tracking control of robots with flexible links. In order to cope with the non-minimum phase property of the system, the closed-loop poles are placed at desired locations using full state feedback. The dynamic equations are linearized about the rigid motion. A composite control law is designed to track the desired trajectory while at the same time the internal dynamics is stabilized. The proposed method is valid for all types of manipulators with any ...
Enhancing biocompatibility of tantalum via anodization for orthopedic applications
Uslu, Ece; Ercan, Batur (null; 2018-12-06)
Tantalum and its alloys have been investigated as the next generation of orthopedic implant materials in the last decade. Being a valve metal, tantalum forms a naturally occurring stable oxide layer approximately 3-5 nm on its surface at ambient conditions and this layer both prevents heavy ion release from the metal and provides a natural barrier for implant corrosion. In fact, due to its chemically inert nature, tantalum has the highest corrosion resistance of all metals used in orthopedic applications. T...
Control and motion planning for mobile robots via a secure wireless communication protocol
Kaya, Muhammed Çağrı; Eroğlu, Alperen; Temizer, Selim (null; 2013-05-21)
Mobile robot control is receiving considerably more attention in current robotics research. Especially unmanned vehicles are used in important missions such as bomb disposal, patrolling borders of a country by unmanned aerial vehicles (UAVs), etc. This usage increases the importance of unmanned vehicles and they become a target of malevolent people. In this study, a controllable robot system is designed with a secure communication protocol. A Pioneer P3-DX robot is used as a ground vehicle. The robot operat...
Citation Formats
M. Turan, Y. Y. Pilavci, I. Ganiyusufoglu, H. Araujo, E. Konukoglu, and M. Sitti, “Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots,” Machine Vision and Applications, pp. 345–359, 2017, Accessed: 00, 2020. [Online]. Available: