Accepted to Reliable ML @ NeurIPS 2025
Authors: Claire O'Brien, Jessica Seto, Dristi Roy, Aditya Dwivedi
Behavioral alignment in large language models (LLMs) is often achieved through broad fine-tuning, which can result in undesired side effects like distributional shift and low interpretability. We propose a method for alignment that identifies and updates only the neurons most responsible for a given behavior, a targeted approach that allows for fine-tuning with significantly less data. Using sparse autoencoders (SAEs) and linear probes, we isolate the 3% of MLP neurons most predictive of a target behavior, decode them into residual space, and fine-tune only those neurons using gradient masking. We demonstrate our approach on reducing sycophantic behavior, matching or exceeding state-of-the-art performance on four benchmarks.
