Abstract
Convolutional Neural Networks (CNNs) showed state-of-the-art accuracy in image classification on large-scale image datasets. However, CNNs show considerable poor performance in classifying tiny data since their large number of parameters over-fit the training data. We investigate the classification characteristics of CNNs on tiny data, which are important for many practical applications. This study analyzes the performance of CNNs for direct and transfer learning based training approaches. Evaluation is performed on two publicly available benchmark datasets. Our study shows the accuracy change when altering the DCNN depth in direct training to indicate the optimal depth for direct training. Further, fine-tuning source and target network with lower learning rate gives higher accuracy for tiny image classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Krizhevsky A, Sutskever I (2012) Hinton GE: ImageNet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
Lin Y, Lv F, Zhu S et al (2011) Large-scale image classification: fast feature extraction and SVM training. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1689–1696. https://doi.org/10.1109/CVPR.2011.5995477
Wang J, Yang J, Yu K, Lv F, Huang T, Gong Y (2010) Locality-constrained linear coding for image classification. In: 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 3360–3367
Karpathy A, Fei-Fei L (2015) Deep visual-semantic alignments for generating image Des. In: Proceedings of the IEEE conference on computer vision and pattern Recognition, pp 3128–3137. https://doi.org/10.1109/CVPR.2015.7298932
Kokul T, Fookes C, Sridharan S, et al (2017) Gate connected convolutional neural network for object tracking. In: IEEE international conference on image processing (ICIP). IEEE, pp 2602–2606 (2017)
Kokul T, Ramanan A, Pinidiyaarachchi UAJ (2016) Online multi-person tracking-by-detection method using ACF and particle filter. In: IEEE 7th international conference on intelligent computing and information systems ICICIS, pp 529–536. https://doi.org/10.1109/IntelCIS.2015.7397272
Stiller C, Wojek C, Lauer M et al (2013) 3D traffic scene understanding from movable platforms. IEEE Trans Pattern Anal Mach Intell 36:1012–1025. https://doi.org/10.1109/tpami.2013.185
Rasheed N, Khan SA (2014) Khalid: a tracking and abnormal behavior detection in video surveillance using optical flow and neural networks. In: Proceeding IEEE 28th international conference on advanced information networking work. IEEE WAINA, pp 61–66. https://doi.org/10.1109/WAINA.2014.18
Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005
Parkhi OM, Vedaldi A, Zisserman A (2015) Deep face recognition. In: Proceedings Br Mach Vis Conference, pp 41.1–41.12. https://doi.org/10.5244/C.29.41
Weinberger K (2005) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 207–244. https://doi.org/10.1142/S021800141100897X
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv Prepr arXiv:14091556, pp 1–14. https://doi.org/10.1016/j.infsof.2008.09.005
Dalal N, Triggs B (2010) Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05). IEEE, pp 886–89
Lindeberg T (2012) Scale invariant feature transform. Scholarpedia 7:10491. https://doi.org/10.4249/scholarpedia.10491
Jia D, Wei D, Socher R, et al (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255. https://doi.org/10.1109/CVPRW.2009.5206848
Shin H-C, Roth HR, Gao M et al (2016) Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans Med Imaging 35:1285–1298. https://doi.org/10.1109/TMI.2016.2528162
Tan C, Sun F, Kong T, et al (2018) A survey on deep transfer learning. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 11141 LNCS: 270–279. https://doi.org/10.1007/978-3-030-01424-7_27
Fukushima K (1988) Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw 1:119–130. https://doi.org/10.1016/0893-6080(88)90014-7
Szegedy C, Liu W, Jia Y, et al (2015) Going deeper with convolutions. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 1–9
Simonyan K, Vedaldi A, Zisserman A (2013) Deep inside convolutional networks: visualising image classification models and saliency maps, pp 1–8
Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. Lect Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes Bioinformatics) 8689 LNCS: pp. 818–833. https://doi.org/10.1007/978-3-319-10590-1_53
Bengio Y (2009) Learning Deep architectures for AI. Found Trends® Mach Learn 2:1–127. https://doi.org/10.1561/2200000006
CS231n: Convolutional neural networks for visual recognition home page, http://cs231n.stanford.edu/. Last accessed 21 May 2019
Cireşan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. https://doi.org/10.1109/CVPR.2012.6248110
Cireşan DC, Meier U, Masci J et al (2011) Flexible, high performance convolutional neural networks for image classification. IJCAI Int Jt Conf Artif Intell 1237–1242. https://doi.org/10.5591/978-1-57735-516-8/IJCAI11-210
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? pp 3320–3328
Caruana R (1997) Multi-task learning. Kluwer Academic Publishers
Chen H, Wang Y, Shi Y et al (2018) Deep Transfer learning for person re-identification. In: 2018 IEEE 4th international conference on big data, BigMM (2018). https://doi.org/10.1109/BigMM.2018.8499067
Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: 2009 IEEE conference on computer vision and pattern recognition, pp 1717–1724. https://doi.org/10.1109/CVPR.2014.222
Sharif A, Hossein R, Josephine A et al (2014) CNN features off-the-shelf-an astounding baseline for recognition. Computer vision and pattern recognition workshops (CVPRW), pp 512–519
Guyon I, Dror G, Lemaire V et al (2011) Unsupervised and transfer learning challenge. Proc Int Jt Conf Neural Networks 793–800. https://doi.org/10.1109/IJCNN.2011.6033302
Ganin Y, Lempitsky V (2014) Unsupervised domain adaptation by Backpropagation arXiv preprint arXiv:1409.7495
Fei-Fei Li, Fergus Rob, Perona Pietro (2007) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. Comput Vis Image Underst 106:59–70. https://doi.org/10.1016/j.cviu.2005.09.012
Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images. In: Learning multiple layers of features from tiny images. University of Toronto
Hinton GE, Srivastava N, Krizhevsky A et al (2012) Improving neural networks by preventing co-adaptation of feature detectors arXiv:1207.0580v1[cs.NE], pp 1–18
Chollet F (2015) “Keras.”
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chandrarathne, G., Thanikasalam, K., Pinidiyaarachchi, A. (2020). A Comprehensive Study on Deep Image Classification with Small Datasets. In: Zakaria, Z., Ahmad, R. (eds) Advances in Electronics Engineering. Lecture Notes in Electrical Engineering, vol 619. Springer, Singapore. https://doi.org/10.1007/978-981-15-1289-6_9
Download citation
DOI: https://doi.org/10.1007/978-981-15-1289-6_9
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1288-9
Online ISBN: 978-981-15-1289-6
eBook Packages: EngineeringEngineering (R0)