Barrett reduction returns x mod n after at most two corrections full
This proof shows: correctness.
Claim. Let n≥1 have k binary digits, so 2k−1≤n<2k, and let μ=⌊4k/n⌋ (lines L1 and L2). For every integer x with 0≤x<4k, let Q=⌊x/n⌋ and R=x−Qn, so 0≤R<n. Then the estimate q of line L3 satisfies Q−2≤q≤Q, the loop of line L5 subtracts n exactly Q−q≤2 times, and line L6 returns R=xmodn.
Notation. Let a=x/2k−1 and b=4k/n, two non-negative rationals, and let x1=⌊a⌋, so that line L3 computes q=⌊x1μ/2k+1⌋ with μ=⌊b⌋. Then
2k+1ab=2k−1⋅n⋅2k+1x⋅4k=nx. Bounds on the factors. Since x<4k, a<22k/2k−1=2k+1. Since n≥2k−1, μ≤b≤22k/2k−1=2k+1. Also 0≤a−x1<1 and 0≤b−μ<1.
Upper bound. x1≤a and μ≤b, with all four numbers non-negative, so x1μ≤ab and x1μ/2k+1≤x/n. Taking floors, q≤⌊x/n⌋=Q.
Lower bound. Write
ab−x1μ=a(b−μ)+μ(a−x1). The first term is at most a<2k+1, because 0≤b−μ<1 and a≥0. The second term is less than μ≤2k+1 when μ>0, because 0≤a−x1<1, and it is 0 when μ=0. If x=0, then q=0=Q and there is nothing to prove; otherwise a>0 and the first term is less than a. In all cases ab−x1μ<2k+1+2k+1=2k+2. Dividing by 2k+1,
2k+1x1μ>nx−2≥Q−2. The integer Q−2 is strictly less than x1μ/2k+1, so it is at most the floor of this number: Q−2≤q.
The corrections. Line L4 sets r=x−qn=(Q−q)n+R. By the two bounds, Q−q∈{0,1,2}, so 0≤r<3n and r≡x(modn). Each pass of the loop in line L5 keeps r≡x(modn) and lowers r by n; the loop stops at the first value below n. Starting from (Q−q)n+R with 0≤R<n, it makes exactly Q−q≤2 passes and ends with r=R. Line L6 returns R=xmodn.
Products of residues. For 0≤a′,b′<n, the product satisfies a′b′≤(n−1)2<4k, so line L7 is within the range of the claim and returns a′b′modn. ■