LINEAR-DIOPHANTINE returns a solution exactly when one exists full
This proof shows: correctness.
Let a, b and c be integers. We write A=∣a∣ and B=∣b∣, and σa, σb for the signs used in line L11, so that aσa=A and bσb=B. We use gcd(0,0)=0, and gcd(u,v)=gcd(∣u∣,∣v∣).
Lemma 1 (necessity). If ax+by=c for some integers x and y, then gcd(a,b)∣c.
Proof. g=gcd(a,b) divides a and b, so it divides ax+by. If g=0 then a=b=0 and c=0=g⋅0. □
Lemma 2 (the loop). Assume (a,b)=(0,0). Before each test of line L4, the following invariant holds:
(E) r0=As0+Bt0, r1=As1+Bt1, r0≥0, r1≥0, and gcd(r0,r1)=gcd(a,b).
The loop stops, and after it r1=0 and g=r0=gcd(a,b)≥1.
Proof. Initialization. Line L3 sets (r0,s0,t0)=(A,1,0) and (r1,s1,t1)=(B,0,1). Then A=A⋅1+B⋅0, B=A⋅0+B⋅1, both are nonnegative, and gcd(A,B)=gcd(a,b).
Maintenance. Assume (E) and r1=0, so r1>0. Line L5 sets q=⌊r0/r1⌋, and the new values are r0′=r1, r1′=r0−qr1, s0′=s1, s1′=s0−qs1, t0′=t1, t1′=t0−qt1. The first relation for the new values is the old second relation. For the second,
As1′+Bt1′=(As0+Bt0)−q(As1+Bt1)=r0−qr1=r1′. Also r0′=r1>0 and r1′=r0modr1∈[0,r1). By the GCD recursion theorem, gcd(r1,r0−qr1)=gcd(r0,r1)=gcd(a,b). So (E) holds again.
Termination. In each iteration the new r1 is r0modr1<r1. So r1 is a nonnegative integer that strictly decreases, and the loop stops. When it stops, r1=0, and (E) gives gcd(r0,0)=r0=gcd(a,b). Since (a,b)=(0,0), gcd(a,b)≥1. Line L9 sets g=r0. □
Theorem. LINEAR-DIOPHANTINE(a,b,c) returns a pair (x,y) with ax+by=c if a solution exists, and NONE otherwise.
Proof. Case a=b=0. Then ax+by=0 for all x and y, so a solution exists if and only if c=0. Lines L1 and L2 return (0,0), which is a solution, when c=0, and NONE otherwise.
Case (a,b)=(0,0). By Lemma 2, line L9 sets g=gcd(a,b)≥1, and As0+Bt0=g. If g∤c, then by Lemma 1 there is no solution, and line L10 returns NONE. If g∣c, let m=c/g, an integer. Line L11 sets x=σas0m and y=σbt0m, so
ax+by=(aσa)s0m+(bσb)t0m=(As0+Bt0)m=gm=c, and line L12 returns this solution. ■
The step. In the case (a,b)=(0,0), the vector (s1,t1) at the end satisfies As1+Bt1=r1=0, and it is nonzero, because the matrix with rows (s0,t0) and (s1,t1) starts as the identity and each iteration multiplies it on the left by a matrix of determinant −1, so its determinant stays ±1. Hence s0t1−s1t0=±1, which forces gcd(s1,t1)=1. From As1=−Bt1 with gcd(s1,t1)=1 it follows that (s1,t1)=±(B/g,−A/g), the step between consecutive solutions of the equation with ∣a∣ and ∣b∣.