Ciência-IUL
Publications
Publication Detailed Description
Efficiency and Scalability of Multi-Lane Capsule Networks (MLCN)
Journal/Book/Other Title
International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)
Year (definitive publication)
2019
Language
English
Country
Brazil
More Information
Web of Science®
Scopus
This publication is not indexed in Scopus
Google Scholar
Abstract
Some Deep Neural Networks (DNN) have what we call lanes, or they can be reorganized as such. Lanes are paths in the network which are data-independent and typically learn different features or add resilience to the network. Given their data-independence, lanes are amenable for parallel processing.The Multi-lane CapsNet (MLCN) is a proposed reorganization of the Capsule Network which is shown to achieve better accuracy while bringing highly-parallel lanes. However, the efficiency and scalability of MLCN had not been systematically examined.In this work, we study the MLCN network with multiple GPUs finding that it is 2x more efficient than the origina lCapsNet when using model-parallelism. Further, we present the load balancing problem of distributing heterogeneous lanes in homogeneous or heterogeneous accelerators and show that a simple greedy heuristic can be almost 50% faster than a naıve random approach
Acknowledgements
--
Keywords
Fields of Science and Technology Classification
- Computer and Information Sciences - Natural Sciences
- Civil Engineering - Engineering and Technology