Endo-VMFuseNet: A Deep Visual-Magnetic Sensor Fusion Approach for Endoscopic Capsule Robots

Turan, Mehmet
Almalioglu, Yasin
Gilbert, Hunter B.
Sari, Alp Eren
Soylu, Ufuk
Sitti, Metin
In the last decade, researchers and medical device companies have made major advances towards transforming passive capsule endoscopes into active medical robots. One of the major challenges is to endow capsule robots with accurate perception of the environment inside the human body, which will provide necessary information and enable improved medical procedures. We extend the success of deep learning approaches from various research fields to the problem of sensor fusion for endoscopic capsule robots in the case of asynchronous and asymmetric sensor data without any need of calibration between sensors. The results performed on real pig stomach datasets show that our method achieves high precision for both translational and rotational movements and contains various advantages over traditional sensor fusion techniques.
IEEE International Conference on Robotics and Automation (ICRA)


Sparse-then-dense alignment-based 3D map reconstruction method for endoscopic capsule robots
Turan, Mehmet; Pilavci, Yusuf Yigit; Ganiyusufoglu, Ipek; Araujo, Helder; Konukoglu, Ender; Sitti, Metin (Springer Science and Business Media LLC, 2017-12-27)
Despite significant progress achieved in the last decade to convert passive capsule endoscopes to actively controllable robots, robotic capsule endoscopy still has some challenges. In particular, a fully dense three-dimensional (3D) map reconstruction of the explored organ remains an unsolved problem. Such a dense map would help doctors detect the locations and sizes of the diseased areas more reliably, resulting in more accurate diagnoses. In this study, we propose a comprehensive medical 3D reconstruction...
Optimum Design of 6R Passive Haptic Robotic Arm for Implant Surgery
Yılmaz, Serkan; Konukseven, Erhan İlhan (2010-07-10)
The aim of this research was to design an optimum 6R passive haptic robotic arm (PHRA) to work in a limited workspace during dental implant surgery. Misplacement of an implant during dental surgery causes longer recuperation periods and functional disorders. In this study, a passive guidance robot arm was designed as a surgical aid tool for a dentist during the operation to reduce the surgical complications. Optimum design of a 6R robot is a complex issue since minimum energy has to be consumed while maximu...
Magnetohydrodynamic Flow Imaging Using Spin-Echo Pulse Sequence
Eroğlu, Hasan Hüseyin; SADIGHI, MEHDI; Eyüboğlu, Behçet Murat (2019-04-24)
In this study, magnetohydrodynamic (MHD) flow of conductive liquids due to injection of electrical current during magnetic resonance imaging (MRI) is investigated. A spin-echo based MRI pulse sequence is proposed to image the MHD flow. Magnetic resonance (MR) phase effects of the MHD flow is related to the MRI pulse parameters and injected current. Average velocity distributions of the MHD flow are reconstructed using the MR phase images. The method is validated by numerical simulations. The reconstruction ...
Opto-thermoelectric nanotweezers
LIN, Linhan; WANG, Mingsong; PENG, Xiaolei; LISSEK, Emanuel N.; MAO, Zhangming; SCARABELLI, Leonardo; ADKINS, Emily; Coskun, Sahin; Ünalan, Hüsnü Emrah; KORGEL, Brian A.; LIZ-MARZAN, Luis M.; FLORIN, Ernst-Ludwig; ZHENG, Yuebing (Springer Science and Business Media LLC, 2018-04-01)
Optical manipulation of plasmonic nanoparticles provides opportunities for fundamental and technical innovation in nanophotonics. Optical heating arising from the photon-to-phonon conversion is considered as an intrinsic loss in metal nanoparticles, which limits their applications. We show here that this drawback can be turned into an advantage, by developing an extremely low-power optical tweezing technique, termed opto-thermoelectric nanotweezers. By optically heating a thermoplasmonic substrate, a light-...
Biopolymer based micro/nanoparticles as drug carriers for the treatment of skin diseases
Eke, Gözde; Hasırcı, Vasıf Nejat; Hasırcı, Nesrin; Department of Micro and Nanotechnology (2011)
Controlled drug delivery systems are becoming increasingly interesting with the contribution of nanotechnology. In the case of transdermal applications the greatest limitation is the highly impermeable outermost layer of the skin, the stratum corneum. One promising method of controlled transdermal drug delivery of the skin therapeutics is the use of nanoparticles as carriers. Encapsulation of the drug, as opposed to classical topical application of creams or emulsions, allows the drug to diffuse into hair f...
Citation Formats
M. Turan, Y. Almalioglu, H. B. Gilbert, A. E. Sari, U. Soylu, and M. Sitti, “Endo-VMFuseNet: A Deep Visual-Magnetic Sensor Fusion Approach for Endoscopic Capsule Robots,” Brisbane, Australia, 2018, p. 5386, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/68320.