通过超球面上的对齐和一致性理解对比表征学习（CS ML）---用户7305506

对比表征学习在实践中取得了显著的成功。在这项工作中，我们确定了与对比损失相关的两个关键性质：（1）正对特征的对齐（紧密性）和（2）超球面上（规范化）特征的诱导分布的均匀性。我们证明了，渐近地，对比损失优化了这些属性，并分析了它们对下游任务的积极影响。在经验上，我们引入了一个可优化的度量来量化每个属性。在标准视觉和语言数据集上的大量实验证实了这两个指标与下游任务性能之间的强烈一致性。值得注意的是，对这两个指标的直接优化会导致在下游任务上表现出与对比学习相当或更好的性能。

原文标题：Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

原文：Contrastive representation learning has been outstandingly successful in practice. In this work, we identify two key properties related to the contrastive loss: (1) alignment (closeness) of features from positive pairs, and (2) uniformity of the induced distribution of the (normalized) features on the hypersphere. We prove that, asymptotically, the contrastive loss optimizes these properties, and analyze their positive effects on downstream tasks. Empirically, we introduce an optimizable metric to quantify each property. Extensive experiments on standard vision and language datasets confirm the strong agreement between both metrics and downstream task performance. Remarkably, directly optimizing for these two metrics leads to representations with comparable or better performance at downstream tasks than contrastive learning.

原文作者：Tongzhou Wang, Phillip Isola

原文地址：https://arxiv.org/abs/2005.10242