Interpretable Representations in Artificial Neural Networks

Abstract

This thesis investigates interpretable representations in Artificial Neural Networks examining both engineered approaches and structure discovered within existing architectures. Whether imposing structure on networks yields benefits over simply scaling model size and data remains debated; our work contributes to understanding this tension between structure and flexibility.

We first engineer interpretable representations through modular architectures. Extending Slot-Attention and combining it with spatial transformers, we build networks that segment images into objects with disentangled representations of pose, shape, and appearance, and show these are easier to interpret than less structured counterparts.

To probe the link between interpretability and generalisation, we pair these object-centric representations with specialised relational reasoning modules. Although this yields sparser reasoning, combining structured approaches often underperforms both the individual methods and standard monolithic networks. We trace this to compounding rigidity: as multiple structural biases interact, they create information bottlenecks between network stages.

Given these difficulties, we turn to emergent structure in state-of-the-art architectures, which perform complex reasoning without strong inductive biases. Focusing on planning as a form of reasoning, we show that foundation models solving maze navigation develop internal world models reflecting spatial structure. These representations can be causally perturbed to alter the model's behaviour, revealing how the networks internally represent and reason about spatial relationships.

Together, these complementary approaches - engineering interpretable architectures and analysing emergent representations - advance our understanding of structure in neural networks. While imposed structure can enhance interpretability, its relationship with performance and generalisation remains complex. Our findings suggest a promising direction: rather than imposing rigid constraints, future architectures may achieve interpretability by leveraging and amplifying the natural emergence of structured representations in flexible, scaled systems.

BibTeX

@phdthesis{spies2025interpretable,
  title={{Interpretable Representations in Artificial Neural Networks}},
  author={Spies, Alexander Fabian},
  school={Imperial College London},
  year={2025},
  doi={10.25560/126689},
  url={https://hdl.handle.net/10044/1/126689}
}