公开问题

Differentiating through an eigendecomposition with near-degenerate eigenvalues

问题维基

提问的人可以创建维基，并指定维护者，把共同的进展整理在这里。

发布于 2026-09-02最近更新 2026-09-04264 次浏览

The standard backward rule for a symmetric eigendecomposition has $1/(\lambda_i - \lambda_j)$ in it, which is fine right up until two eigenvalues approach each other. In our case they approach each other exactly where the interesting physics is, so the gradient goes to nonsense precisely where we need it.

Things I have tried: adding a damping $\varepsilon$ to the denominator (biases the gradient in a way I cannot bound), and reformulating to avoid the decomposition (works for two of the four places it appears).

Is there a formulation whose derivative stays bounded through a degeneracy, given that the function being differentiated is smooth there even though the decomposition is not?

提问者 · 2026-09-02Dan Okonkwo

1 个回答

回答

The usual move is to stop differentiating through the decomposition and start differentiating through whatever invariant you actually want.

If the downstream quantity is a spectral function $\sum_i g(\lambda_i) v_i v_i^T$ with $g$ smooth, the whole thing is a matrix function $g(A)$ and its differential is the Daleckii–Krein formula — which has divided differences $[g](\lambda_i, \lambda_j)$ where your rule has $1/(\lambda_i - \lambda_j)$ . Those are perfectly well behaved at a degeneracy; they become $g'(\lambda)$ .

So the singularity was never in the problem, only in the intermediate you chose.

2026-09-04 · Mei Kurosawa

登录后可以回答。