How Data Organization Shapes Neural Network Learning: Hierarchical Labels and Spectral Structure

Visual classification depends on two fundamental factors: image structure (the relationships encoded in pixel values) and categorical organization (the labels indicating what they depict). This thesis explores how these data properties shape neural network training and responses. First, I examine hierarchical label structures in visual datasets. While classification tasks are defined at a specific granularity, training on finer-grained labels can boost performance. I show that this benefit depends on dataset size, model capacity, and data geometry, revealing when granularity enhances generalization. Through systematic experiments on synthetic and real datasets, I identify the conditions under which hierarchical training provides advantages over flat classification. Second, I investigate how the power-law spectra of natural images affect neural network sensitivity, testing both standard CNNs and biologically-inspired retinal models. Using synthetic images with controlled spectral exponents, I find that networks across these architectures show peak sensitivity near the characteristic exponent of natural images. I show that this pattern arises from the convolutional structure rather than learning by comparing random convolutional networks to fully connected ones, and find that it depends on the input dimension. Finally, I show that these spectral preferences influence learning, with classification performance varying according to a dataset's spectral properties. These studies show how data properties influence training strategies and how architectural constraints modulate network sensitivity to image structure. Together, they provide insights into the interplay between data organization and neural network behavior, with implications for both understanding these systems and designing better training procedures.