Interpretable attention probing reveals model-specific perceptions of cell states in single-cell foundation models
Interpretable attention probing reveals model-specific perceptions of cell states in single-cell foundation models
原文摘要
出版方登记 · OpenAlexSingle-cell foundation models (scFMs) are increasingly used for transcriptomic analysis, yet it remains unclear whether different models encode cell states according to the same biological principles. Here, we present scGeneLens, an interpretable probing framework for dissecting how Transformer-based scFMs organize gene-level information into cellular representations. scGeneLens combines structured sparse-attention probing, cross-layer gene influence propagation, and complementary attribution and representational-alignment analyses to expose the dominant biological signals prioritized by each model. Applying scGeneLens to scFoundation and scGPT, we found they exhibit pluralistic perceptions of cells: scFoundation emphasizes relations among cell-type marker genes, resulting in stronger cell-type separability, whereas scGPT focuses more on genes involved in shared cellular pathways and core biological activities. These results suggest that differences among scFMs reflect not only performance variation but also distinct biological abstractions of cell identity and state. scGeneLens provides a unified lens for probing what scFMs learn about cells and informing future model design.
页面未能读取全文,以上为出版方登记的摘要;方法与图表见原文
来源:Patterns · doi.org