Show/Hide Menu
Hide/Show Apps
Logout
Türkçe
Türkçe
Search
Search
Login
Login
OpenMETU
OpenMETU
About
About
Open Science Policy
Open Science Policy
Communities & Collections
Communities & Collections
Help
Help
Frequently Asked Questions
Frequently Asked Questions
Guides
Guides
Thesis submission
Thesis submission
MS without thesis term project submission
MS without thesis term project submission
Publication submission with DOI
Publication submission with DOI
Publication submission
Publication submission
Supporting Information
Supporting Information
General Information
General Information
Copyright, Embargo and License
Copyright, Embargo and License
Contact us
Contact us
The reusability prior: comparing deep learning models without training
Download
Polat_2023_Mach._Learn. _Sci._Technol._4_025011.pdf
Date
2023-06-01
Author
Polat, Aydın Göze
Alpaslan, Ferda Nur
Metadata
Show full item record
This work is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
.
Item Usage Stats
31
views
7
downloads
Cite This
Various choices can affect the performance of deep learning models. We conjecture that differences in the number of contexts for model components during training are critical. We generalize this notion by defining the reusability prior as follows: model components are forced to function in diverse contexts not only due to the training data, augmentation, and regularization choices, but also due to the model design itself. We focus on the design aspect and introduce a graph-based methodology to estimate the number of contexts for each learnable parameter. This allows a comparison of models without requiring any training. We provide supporting evidence with experiments using cross-layer parameter sharing on CIFAR-10, CIFAR-100, and Imagenet-1K benchmarks. We give examples of models that share parameters outperforming baselines that have at least 60% more parameters. The graph-analysis-based quantities we introduced for the reusability prior align well with the results, including at least two important edge cases. We conclude that the reusability prior provides a viable research direction for model analysis based on a very simple idea: counting the number of contexts for model parameters.
Subject Keywords
deep learning
,
entropy
,
parameter efficiency
,
reusability
URI
https://hdl.handle.net/11511/103050
Journal
Machine Learning: Science and Technology
DOI
https://doi.org/10.1088/2632-2153/acc713
Collections
Department of Computer Engineering, Article
Citation Formats
IEEE
ACM
APA
CHICAGO
MLA
BibTeX
A. G. Polat and F. N. Alpaslan, “The reusability prior: comparing deep learning models without training,”
Machine Learning: Science and Technology
, vol. 4, no. 2, pp. 0–0, 2023, Accessed: 00, 2023. [Online]. Available: https://hdl.handle.net/11511/103050.