This study examines the training dynamics of deep networks and introduces a novel statistic called local complexity (LC) to measure the concentration of linear regions around data points. The study finds that LC undergoes different phases during training, with linear regions migrating towards the decision boundary in the final phase. These phases are closely related to the memorization and generalization performance of the network.

https://arxiv.org/abs//2310.12977

YouTube: https://www.youtube.com/@ArxivPapers

TikTok: https://www.tiktok.com/@arxiv_papers

Apple Podcasts: https://podcasts.apple.com/us/podcast/arxiv-papers/id1692476016

Spotify: https://podcasters.spotify.com/pod/show/arxiv-papers