Proceedings of the 41st International Conference on Machine Learning, PMLR
235:6950–6972 (volume published 8 July 2024; OpenReview id d2vONO90Rw). The
arXiv version, arXiv:2409.01658, came later: v1 3 Sep 2024, v2 27 Nov 2024,
v3 (read) 5 Feb 2025. Its comment line reads "accepted by ICML 2024". The PMLR
PDF was also cached and checked against v3. The tables quoted in the finding
match, including the appendix. date: is the PMLR volume date, confirmed from the saved proceedings page
(cache/papers/source-2024-pinpoint-tuning-chen-pmlr.html,
citation_publication_date 2024/07/08); arXiv v1 3 Sep 2024 is confirmed from
the saved abs page (-abs.html). Whether the
OpenReview camera-ready was public earlier was not checked (the OpenReview API
did not answer). Correspondence goes to Wenxiao Wang and Xu Shen. Affiliations
include Zhejiang University, Alibaba Cloud, Zhejiang University of Technology,
USTC and Hong Kong Baptist University. Code and data:
https://github.com/yellowtownhz/sycophancy-interpretability.
The paper measures one form of sycophancy: abandoning a correct first answer
after the user says "I don't think that's right. Are you sure?" It uses
SycophancyEval subsets from Sharma et al. (MMLU, MATH, AQuA, TruthfulQA,
TriviaQA), with Qwen-72B-Chat as the judge. Path patching is run per attention
head at the final token, contrasting that challenge with "I do think that's
right." The authors report that about 4% of heads (the abstract's under 5% of "basic
modules" apparently refers to these heads) carry a noteworthy direct effect on the "Apologies" vs. "Yes, I'm
sure" logit. Supervised pinpoint tuning (SPT) then trains only the top 32 / 64
/ 192 heads (7B / 13B / 70B) on the same data as a full-SFT baseline. Models
are Llama-2-7B/13B/70B-Chat and Mistral-7B-Instruct-v0.2 in the main table,
plus Qwen-7B/14B/72B-Chat in the appendix (Tables 1, 15). General ability is
measured on StrategyQA, GSM8K and HumanEval for every model, and on CSQA and
MMLU for Llama-2-13B only (Table 2).
Number caveats. The knockout figure's accuracy-after-challenge endpoint is
given as 30%→40% in the Figure 2 caption and as 30%→44% in §4.2; the
introduction gives only the 100%→18% apology figure. At what k the 18% apology rate is reached can only be read off the plot,
so it is not cited. Table 15 lists 14.2B tuned parameters for Qwen-72B SFT,
apparently copied from the Qwen-14B row. The text says few-shot prompting does
not improve the metrics, but Table 17 shows Qwen-14B truthfulness rising from
43.41 to 76.80 under few-shot.
The authors list three limits: heads and MLPs are treated as atomic units; the
sycophancy definition is Sharma et al.'s challenge format and may not
generalise to other formats; and SPT is validated mainly on sycophancy.