Early-exit convolutional neural networks

Demir, Edanur
This thesis is aimed at developing a method that reduces the computational cost of convolutional neural networks (CNN) during inference. Conventionally, the input data pass through a fixed neural network architecture. However, easy examples can be classified at early stages of processing and conventional networks do not take this into account. In this thesis, we introduce “Early-exit CNNs”, EENets for short, which adapt their computational cost based on the input by stopping the inference process at certain exit locations. In EENets, there are a number of exit blocks each of which consists of a confidence branch and a softmax branch. The confidence branch computes the confidence score of exiting (i.e. stopping the inference process) at that location; while the softmax branch outputs a classification probability vector. Both branches are learnable and they are independent of each other. During training of EENets, in addition to the classical classification loss, the computational cost of inference is taken into account as well. As a result, the network adapts its many confidence branches to the inputs so that less computation is spent for easy examples. Inference works as in conventional feed-forward networks, however, when the output of a confidence branch is larger than a certain threshold, the inference stops for that specific example. Through comprehensive experiments, we show that EENets significantly reduce the computational cost upto 2% of the original without degrading the testing accuracy. The idea of EENets is applicable to available CNN architectures such as ResNets. On MNIST, SVHN and CIFAR10 datasets, early-exit (EE) ResNets achieve similar accuracy with their non-EE versions while reducing the computational cost to 20% of the original.


Fuzzy Semantic Web Architecture for Activity Detection in Wireless Multimedia Sensor Network Applications
Ozdin, Ali Nail; Yazıcı, Adnan; KOYUNCU, Murat (2019-01-01)
This study aims to increase the reliability of activity detection in Wireless Multimedia Sensor Networks (WMSNs) by using Semantic Web technologies extended with fuzzy logic. The proposed approach consists of three layers: the sensor layer, the data layer, and the Semantic Web layer. The sensor layer comprises a WMSN comprising sensor nodes with multimedia and scalar sensors. The data layer retrieves and stores data from the sink of WMSN. At the top of the architecture, there is a semantic web layer that in...
An energy aware fuzzy approach to unequal clustering in wireless sensor networks
Bagci, Hakan; Yazıcı, Adnan (2013-04-01)
In order to gather information more efficiently in terms of energy consumption, wireless sensor networks (WSNs) are partitioned into clusters. In clustered WSNs, each sensor node sends its collected data to the head of the cluster that it belongs to. The cluster-heads are responsible for aggregating the collected data and forwarding it to the base station through other cluster-heads in the network. This leads to a situation known as the hot spots problem where cluster-heads that are closer to the base stati...
Automatic target recognition of quadcopter type drones from moderately-wideband electromagnetic data using convolutional neural networks
Güneri, Rutkay; Sayan, Gönül; Department of Electrical and Electronics Engineering (2022-12-15)
In this thesis, the classifier design approach based on “Learning by a Convolutional Neural Network (CNN)” will be applied to two different target library/data sets; an ultra-wideband simulation data (from 37 MHz to 19.1 GHz) obtained for a target library of four dielectric spheres, and a moderately-wide band measurement data (from 3.1 to 4.8 GHz) obtained for a target library of four quadcopter type unmanned aerial vehicles (UAVs). While the bandwidth of simulation data for spherical targets is about nine ...
An adaptive, energy-aware and distributed fault-tolerant topology-control algorithm for heterogeneous wireless sensor networks
Deniz, Fatih; Bagci, Hakki; KÖRPEOĞLU, İBRAHİM; Yazıcı, Adnan (2016-07-01)
This paper introduces an adaptive, energy-aware and distributed fault-tolerant topology control algorithm, namely the Adaptive Disjoint Path Vector (ADPV) algorithm, for heterogeneous wireless sensor networks. In this heterogeneous model, we have resource-rich supernodes as well as ordinary sensor nodes that are supposed to be connected to the supernodes. Unlike the static alternative Disjoint Path Vector (DPV) algorithm, the focus of ADPV is to secure supernode connectivity in the presence of node failures...
A temporal neural network model for constructing connectionist expert system knowledge bases
Alpaslan, Ferda Nur (Elsevier BV, 1996-04-01)
This paper introduces a temporal feedforward neural network model that can be applied to a number of neural network application areas, including connectionist expert systems. The neural network model has a multi-layer structure, i.e. the number of layers is not limited. Also, the model has the flexibility of defining output nodes in any layer. This is especially important for connectionist expert system applications.
Citation Formats
E. Demir, “Early-exit convolutional neural networks,” M.S. - Master of Science, Middle East Technical University, 2019.