Deep Learning - Plain Version

Deep Learning - Plain Version

By Prof. Dr.-Ing. Andreas MaierEducation
Download on the App Store

Deep Learning - Plain Version episodes

  • Deep Learning - Regularization Part 2

    Deep Learning - Regularization Part 2

    This video discusses classical regularization techniques such as early stopping using a validation set, augmentation, and maximum a-posteriori methods that expand the loss function using a regularization term.

    Video References:

    Lex Fridman's Channel

    Further Reading:

    A gentle Introduction to Deep Learning

    14 min
  • Deep Learning - Regularization Part 3

    Deep Learning - Regularization Part 3

    This video discusses normalization such as batch normalization and self normalizing units and explains the concepts of drop-out and drop-connect

    Further Reading:

    A gentle Introduction to Deep Learning

    10 min
  • Deep Learning - Regularization Part 5

    Deep Learning - Regularization Part 5

    This video discusses multi-task learning.

    Further Reading:

    A gentle Introduction to Deep Learning

    Links:

    Link - for details on Maximum A Posteriori estimation and the bias-variance decomposition

    Link - for a comprehensive text about practical recommendations for regularization
    Link - the paper about calibrating the variances

    References:

    [1] Sergey Ioffe and Christian Szegedy. “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”. In: Proceedings of The 32nd International Conference on Machine Learning. 2015, pp. 448–456.
    [2] Jonathan Baxter. “A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling”. In: Machine Learning 28.1 (July 1997), pp. 7–39.
    [3] Christopher M. Bishop. Pattern Recognition and Machine Learning (Information Science and Statistics). Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2006.
    [4] Richard Caruana. “Multitask Learning: A Knowledge-Based Source of Inductive Bias”. In: Proceedings of the Tenth International Conference on Machine Learning. Morgan Kaufmann, 1993, pp. 41–48.
    [5] Andre Esteva, Brett Kuprel, Roberto A Novoa, et al. “Dermatologist-level classification of skin cancer with deep neural networks”. In: Nature 542.7639 (2017), pp. 115–118.
    [6] C. Ding, C. Xu, and D. Tao. “Multi-Task Pose-Invariant Face Recognition”. In: IEEE Transactions on Image Processing 24.3 (Mar. 2015), pp. 980–993.
    [7] Li Wan, Matthew Zeiler, Sixin Zhang, et al. “Regularization of neural networks using drop connect”. In: Proceedings of the 30th International Conference on Machine Learning (ICML-2013), pp. 1058–1066.
    [8] Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, et al. “Dropout: a simple way to prevent neural networks from overfitting.” In: Journal of Machine Learning Research 15.1 (2014), pp. 1929–1958.
    [9] R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. John Wiley and Sons, inc., 2000.
    [10] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. http://www.deeplearningbook.org. MIT Press, 2016.
    [11] Yuxin Wu and Kaiming He. “Group normalization”. In: arXiv preprint arXiv:1803.08494 (2018).
    [12] Kaiming He, Xiangyu Zhang, Shaoqing Ren, et al. “Delving deep into rectifiers: Surpassing human-level performance on imagenet classification”. In: Proceedings of the IEEE international conference on computer vision. 2015, pp. 1026–1034.
    [13] D Ulyanov, A Vedaldi, and VS Lempitsky. Instance normalization: the missing ingredient for fast stylization. CoRR abs/1607.0 [14] Günter Klambauer, Thomas Unterthiner, Andreas Mayr, et al. “Self-Normalizing Neural Networks”. In: Advances in Neural Information Processing Systems (NIPS). Vol. abs/1706.02515. 2017. arXiv: 1706.02515.
    [15] Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. “Layer normalization”. In: arXiv preprint arXiv:1607.06450 (2016).
    [16] Nima Tajbakhsh, Jae Y Shin, Suryakanth R Gurudu, et al. “Convolutional neural networks for medical image analysis: Full training or fine tuning?” In: IEEE transactions on medical imaging 35.5 (2016), pp. 1299–1312.
    [17] Yoshua Bengio. “Practical recommendations for gradient-based training of deep architectures”. In: Neural networks: Tricks of the trade. Springer, 2012, pp. 437–478.
    [18] Chiyuan Zhang, Samy Bengio, Moritz Hardt, et al. “Understanding deep learning requires rethinking generalization”. In: arXiv preprint arXiv:1611.03530 (2016).
    [19] Shibani Santurkar, Dimitris Tsipras, Andrew Ilyas, et al. “How Does Batch Normalization Help Optimization?” In: arXiv e-prints, arXiv:1805.11604 (May 2018), arXiv:1805.11604. arXiv: 1805.11604 [stat.ML].
    [20] Tim Salimans and Diederik P Kingma. “Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks”. In: Advances in Neural Information Processing Systems 29. Curran Associates, Inc., 2016, pp. 901–909.
    [21] Xavier Glorot and Yoshua Bengio. “Understanding the difficulty of training deep feedforward neural networks”. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence 2010, pp. 249–256.
    [22] Zhanpeng Zhang, Ping Luo, Chen Change Loy, et al. “Facial Landmark Detection by Deep Multi-task Learning”. In: Computer Vision – ECCV 2014: 13th European Conference, Zurich, Switzerland, Cham: Springer International Publishing, 2014, pp. 94–108.

    7 min
  • Deep Learning - Common Practices Part 1

    Deep Learning - Common Practices Part 1

    This video discusses the use of validation data and how to choose optimizers, monitor weights, and set learning rates including their annealing.

    Further Reading:

    A gentle Introduction to Deep Learning

    11 min
  • Deep Learning - Common Practices Part 2

    Deep Learning - Common Practices Part 2

    This video discusses the use of validation data and how to choose architectures and hyperparameters and discuss ensembling.

    Further Reading:

    A gentle Introduction to Deep Learning

    9 min

About Deep Learning - Plain Version

From the publisher's feed

 

More shows like Deep Learning - Plain Version

Public Theology - Religion - Education. Interreligious Perspectives by Prof. Dr. Manfred Pirner

Public Theology - Religion - Education. Interreligious Perspectives

0 Listeners

Lineare Algebra I by Prof. Dr. Peter Knabner

Lineare Algebra I

0 Listeners

Foundations of Quantum Mechanics by Prof. Dr. Florian Marquardt

Foundations of Quantum Mechanics

2 Listeners

Die deutsche Königswahl (1125-1411) by Prof. Dr. Stuart Jenks

Die deutsche Königswahl (1125-1411)

0 Listeners

Lectures on Quantum Theory (Elite Graduate Programme) 2015 -Measurements by Dr. Frederic P. Schuller

Lectures on Quantum Theory (Elite Graduate Programme) 2015 -Measurements

1 Listeners

Einführungsvorlesung Mittelalter by Prof. Dr. Stuart Jenks

Einführungsvorlesung Mittelalter

0 Listeners

Lectures on the Geometric Anatomy of Theoretical Physics by Dr. Frederic P. Schuller

Lectures on the Geometric Anatomy of Theoretical Physics

6 Listeners

Quantum Computing by Prof. Dr. Michael J. Hartmann

Quantum Computing

0 Listeners

Modern Optics 3: Quantum Optics by Prof. Dr. Maria Chekhova

Modern Optics 3: Quantum Optics

0 Listeners