$V_1$: Unifying Generation and Self-Verification for Parallel Reasoners

🍞 Top Bread (Hook): You know how during a school quiz, it’s hard to grade your own answer all by itself, but if you look at two classmates’ answers side by side, it’s easier to say which one seems better?

🥬 Filling (The Actual Concept):

• What it is: Test-time scaling is giving a model extra “thinking time” by letting it try many ideas in parallel and then choosing the best.
• How it works: (1) The model writes several different solution attempts; (2) something picks the best; (3) we return that.
• Why it matters: If we can’t pick the best attempt reliably, generating many attempts just wastes compute.

🍞 Bottom Bread (Anchor): Imagine solving 16 math problems in different ways and then selecting the right one; this only helps if your “chooser” is good at spotting the right solution.

🍞 Top Bread (Hook): Imagine a team brainstorming 10 different ways to build a bridge. More ideas are great, but someone has to judge which design actually holds.

🥬 Filling (The Actual Concept):

• What it is: Parallel reasoning is when an AI tries many different solution paths at once, instead of just one long chain of thought.
• How it works: The model samples multiple answers independently, exploring varied reasoning steps.
• Why it matters: Diversity raises the chance that at least one answer is correct (this is like Pass@N), but we still must pick it.

🍞 Bottom Bread (Anchor): It’s like trying 16 different puzzle strategies—at least one might crack it—but only if the judge notices the winning one.

🍞 Top Bread (Hook): You know how in math class the final answer is either right or wrong? Counting votes can work there: if most kids say “42,” that’s promising.

🥬 Filling (The Actual Concept):

• What it is: Majority voting chooses the most common answer across many tries.
• How it works: Tally answers, pick the one with the most votes.
• Why it matters: It works only when answers can be checked objectively (like exact numbers), not for open-ended tasks like code patches with tricky edge cases.

🍞 Bottom Bread (Anchor): Majority voting helps for 2+2, but not for “Which code fix actually removes the bug?”

🍞 Top Bread (Hook): Grading your own essay without any comparison is tough—you might give yourself a 10/10 even if you missed something.

🥬 Filling (The Actual Concept):

• What it is: Self-verification is when the model judges its own solutions.
• How it works: After generating answers, the model scores them.
• Why it matters: Without a good judge, the best answer might be ignored.

🍞 Bottom Bread (Anchor): A coder model writes 16 functions—self-verification decides which function to submit.

🍞 Top Bread (Hook): If you rate each drawing from 1 to 10 on different days, your “10” today might not match your “10” tomorrow.

🥬 Filling (The Actual Concept):

• What it is: Pointwise verification scores each candidate alone, on a 1–10-like scale.
• How it works: The model reads one solution and assigns a score; higher is better.
• Why it matters: Scores aren’t calibrated across problems, so an 8 in one context doesn’t align with an 8 in another—leading to bad picks.

🍞 Bottom Bread (Anchor): The model might give a buggy code solution 10/10 and miss the truly correct one because it can’t compare them directly.

🍞 Top Bread (Hook): Imagine trying to keep every creative idea while combining them into one mega-idea—sometimes you accidentally throw out the best part.

🥬 Filling (The Actual Concept):

• What it is: Self-aggregation merges several solutions into a new, single solution (e.g., RSA).
• How it works: The model iteratively refines by combining pieces from different answers.
• Why it matters: It can cause diversity collapse—rare but correct answers can get lost during merging.

🍞 Bottom Bread (Anchor): Like blending all paints into brown—useful structure disappears, including that one perfect bright color you needed.

🍞 Top Bread (Hook): Choosing the better of two sneakers is usually easier than guessing each sneaker’s “absolute quality score” from 1–10.

🥬 Filling (The Actual Concept):

• What it is: Pairwise self-verification compares two solutions head-to-head and decides which is better (or if tie), often with confidence.
• How it works: The model reads both, gives each a rating, and the bigger rating “wins.” Repeating this smartly can reveal the true best solution.
• Why it matters: Comparisons are better calibrated than isolated scores, so the model makes fewer judging mistakes.

🍞 Bottom Bread (Anchor): When the model pitted two code fixes against each other, it noticed which one truly passed hidden edge cases, even when pointwise scores were misleading.

The world before this paper: models generated multiple answers, then either (a) judged them one-by-one (pointwise) and often picked wrong due to calibration issues, or (b) merged them (aggregation), risking diversity collapse. The problem: how to choose the best candidate reliably without external test oracles. Failed attempts: pointwise judging over-scored wrong answers; recursive aggregation sometimes reduced Pass@N. The missing piece: use pairwise judging plus an efficient tournament to spend judge-compute where it matters, and also train the same model to be both a great solver and a strong pairwise judge. Real stakes: picking the right code patch, math solution, or policy draft matters for correctness, safety, and speed in everyday tools.

🍞 Top Bread (Hook): Think of a science fair where judges compare projects in pairs—this is way easier than scoring every project “out of 10” in isolation.

🥬 Filling (The Actual Concept):

• What it is: The key insight is that models are much better at deciding “Which of these two is better?” than at giving perfect solo scores, so build everything around pairwise comparisons.
• How it works: (1) Generate many candidate answers in parallel; (2) run a smart, pairwise tournament ($V_I$nfer) that compares the most uncertain pairs more often; (3) train the same model to both solve and compare using RL ($V_P$airRL) so the judge learns from the model’s current outputs.
• Why it matters: This turns scattered attempts into reliable wins, boosting Pass@1 without relying on external tests.

🍞 Bottom Bread (Anchor): On coding benchmarks, this pairwise-first approach improved top-1 accuracy by up to 10% versus classic solo scoring.

Three analogies for the same idea:

• Library game: Give two book summaries to a friend; it’s easier for them to say which is clearer than to grade each out of 10 perfectly.
• Sports ladder: Teams that play near-equals give you the most information about who’s stronger (Swiss tournament), so you schedule more of those matchups.
• Taste test: Comparing two cookies side-by-side is simpler and more accurate than guessing absolute sweetness scores.

Before vs. After:

• Before: Models made many answers but struggled to pick the right one; pointwise scores were unstable; aggregation sometimes erased the one correct outlier.
• After: Use pairwise judging with an uncertainty-guided tournament to surface the best answer, and train the model to excel at both solving and judging in a co-evolving loop.

Why it works (intuition, not equations):

• Pairwise logic: Humans and models alike are more consistent at relative choices than absolute ratings. Pairwise preferences can be aggregated into a stable global ranking.
• Information focus: Comparing near-ties teaches you the most about who’s actually better; Swiss-style pairing concentrates judge-compute on those knife-edge cases.
• Weighted wins: Not all wins are equal—big margin wins should count more, so we weight comparisons by confidence (rating gap), stabilizing the final ranking.
• Co-evolution: As the generator learns to produce better answers, the verifier must learn to distinguish finer differences. Training both together keeps the judge “in tune” with the solver’s current strengths and mistakes.

Building blocks (each with the sandwich pattern):

1. 🍞 Hook: Picking the champion is easier if contenders actually face off. 🥬 Concept: Pairwise self-verification means comparing two candidate answers and rating both. How: Present A vs. B, rate each 1–10, higher wins; repeat across pairs. Why: Side-by-side judgments reduce calibration problems. 🍞 Anchor: Two code patches are judged head-to-head; the one that correctly handles edge cases wins.

2. 🍞 Hook: In tournaments, close matches are exciting—and informative. 🥬 Concept: Uncertainty-guided ranking orders solutions by weighted win rates, giving more weight to decisive wins and focusing new matches on near-ties. How: Use rating gaps as confidence weights; rank by weighted wins; schedule more comparisons where scores are close. Why: You spend effort where it clarifies the ranking the most. 🍞 Anchor: Two nearly-tied answers keep getting compared until a clear winner emerges.

3. 🍞 Hook: Swiss tournaments pair teams with similar records for fair, efficient sorting. 🥬 Concept: Tournament-based refinement (Swiss refinement) pairs neighbors in the current ranking window, preferring unseen, close pairs. How: Sort by current score; compare nearby entries; repeat with updated scores. Why: It quickly resolves local uncertainties with few total matches. 🍞 Anchor: Like chess tournaments where 3–1 players face 3–1 players, rapidly revealing the top performers.

4. 🍞 Hook: A great judge knows when to be sure and when to double-check. 🥬 Concept: $V_I$nfer is the inference-time algorithm that runs the uncertainty-guided, Swiss-style pairwise ranking to pick the best answer. How: Two phases—coverage (everyone compares at least a bit) then refinement (focus on near-ties); final ranking picks the top. Why: Prevents orphan solutions, reduces mis-rankings, and saves compute. 🍞 Anchor: With a $2× verification$ budget, $V_I$nfer beats pointwise and often surpasses aggregation with fewer calls.

5. 🍞 Hook: Practice with feedback makes you better at both playing and refereeing. 🥬 Concept: Reinforcement learning (RL) improves a model using rewards for good behavior. How: The model generates, gets correctness rewards, updates its policy. Why: It steadily nudges the model toward better solutions. 🍞 Anchor: Correct code that passes tests gets reward; the model learns to do that more.

6. 🍞 Hook: If a player and the referee learn together, the matches get cleaner and the calls get sharper. 🥬 Concept: $V_P$airRL co-trains one model to both generate solutions and judge pairs using pairwise rewards. How: Generate a group of solutions; compute generation rewards from tests; form pairs and reward the verifier for scoring close to ground-truth correctness; update one shared model on both. Why: The judge learns from the solver’s current outputs, avoiding mismatch. 🍞 Anchor: On code tasks, $V_P$airRL yields 7–9% better scaling than standard RL and boosts base Pass@1 too.

At a high level: Input problem → Parallel generation (N candidates) → Pairwise verification via $V_I$nfer (coverage + Swiss refinement with weighted wins) → Ranked list → Select top-1 (or top-k). For training: Problems + tests → Generate multiple solutions → Compute generation rewards → Build pairs with at least one correct → Compute verification rewards → Update one shared model ($V_P$airRL).

Step-by-step with sandwiches for key steps:

1. 🍞 Hook: More tries raise your chances—like taking many shots at the hoop. 🥬 Concept: Parallel generation produces N different answers. How: Sample independently with temperature/top-p; store all candidates. Why: Increases Pass@N—the chance at least one is correct. 🍞 Anchor: For N=16, even if single-shot accuracy is modest, having many candidates often includes a correct one.

2. 🍞 Hook: Everyone should play at least one match before declaring a champion. 🥬 Concept: Coverage phase in $V_I$nfer gives every solution minimum comparisons (min-degree) and pairs it with similar-scored opponents. How: Start with random disjoint pairs; then target under-sampled solutions and compare them to close-scored neighbors. Why: Prevents “orphan” solutions that never get seen and reduces early noise. 🍞 Anchor: Each of the 16 candidates is compared at least twice before we get fancy.

3. 🍞 Hook: Close games teach you the most about who’s truly better. 🥬 Concept: Swiss refinement focuses remaining comparisons on near-ties and unseen pairs. How: Sort solutions by current score; compare neighbors within a small window; favor pairs not yet compared and with small score gaps. Why: Maximizes information gain per comparison, raising ranking accuracy with few matches. 🍞 Anchor: If #5 and #6 are nearly tied, we compare them again rather than re-checking #1 vs. #16.

4. 🍞 Hook: A blowout win should count more than squeaking by. 🥬 Concept: Uncertainty-weighted aggregation uses rating gaps as confidence weights. How: In each pair A vs. B, the model outputs ratings rA and rB in [1,10]; weight = max(|rA−rB|, τ). Update each solution’s score as weighted win rate over its opponents. Why: Decisive wins strongly influence the ranking; near-ties contribute gently, reducing variance. 🍞 Anchor: If A=9, B=3, A’s big margin counts more than A=6, B=5.

5. 🍞 Hook: Pick a champion. 🥬 Concept: Final selection takes the top-ranked solution after all comparisons within budget. How: Sort by the aggregated scores and choose top-1. Why: That’s the most likely correct answer given the head-to-head evidence. 🍞 Anchor: On LiveCodeBench, this step delivers higher Pass@1 than pointwise scoring at the same budget.

Concrete example:

• Inputs: N=16 candidate codes, budget B=32 pairwise judge calls, τ=0.1, Swiss window h=8, min-degree=2.
• Coverage: Random pairing (8 pairs), then additional pairings for under-sampled solutions (≈ 16–20 calls total).
• Swiss: Use remaining ~12–16 calls to compare close-score neighbors not yet played.
• Weighted wins: Update each solution’s weighted win-rate μ; pick the highest μ.

Why each step exists (what breaks without it):

• No parallel generation: You have nothing to select from—lower Pass@N.
• No coverage: Some candidates never compared; a hidden gem can’t win.
• No Swiss refinement: You waste comparisons on obvious mismatches, learning little.
• No weighting: Narrow wins distort rankings; decisive victories don’t stand out.
• No budget control: Compute explodes to O($N^2$); the method becomes too slow.

Training recipe ($V_P$airRL):

🍞 Hook: Practice makes perfect—especially when you practice both playing and refereeing. 🥬 Concept: Co-evolving RL trains the same model to solve and to judge pairs at the same time. How it works:

1. Generate G solutions per problem.
2. Generation rewards: run code against tests (1 if all pass, else 0). Compute group-relative advantages.
3. Build pairs that include at least one correct solution (CC or CI) to avoid trivial gaming.
4. Verification rewards: model rates each solution in the pair to [0,1]. Reward is high only if the score is confidently close to ground truth (within 0.2), encouraging bold, correct judgments and discouraging fence-sitting.
5. Update one shared policy with combined loss: J = $J_G$en + λ $J_P$airVerif. Why it matters: The judge learns on in-distribution outputs from the current solver, staying calibrated as the solver improves. 🍞 Anchor: On CodeContests, this co-training improved both the scaled performance and the base Pass@1.

Secret sauce:

• Pairwise over pointwise: Relative comparisons are naturally better calibrated.
• Uncertainty focus: Compare near-ties for maximal information.
• Weighted wins: Turn noisy pair outcomes into a stable global ranking.
• Co-evolution: Keep solver and judge in sync to avoid distribution shift.
• Anti-hacking rewards: Sparsity threshold and CC/CI pairing prevent degenerate strategies (e.g., always rating 0.5 or generating junk to make judging easy).

The test: Can pairwise verification ($V_I$nfer) and co-training ($V_P$airRL) help pick the right answer more often, using the same or less compute?

What they measured and why:

• Pass@1: How often the finally chosen answer is correct—like your final grade.
• Test-time scaling gains: How much accuracy improves when we spend more comparisons wisely.
• Compute budget: Total calls (generation + verification), to ensure fair comparisons.

The competition (baselines):

• Pointwise self-verification: Score each answer alone and pick the highest.
• Recursive Self-Aggregation (RSA): Iteratively merge solutions into new ones (risking diversity collapse).

Scoreboard with context:

• $V_I$nfer vs. Pointwise: On code contests, GPT-OSS-20B jumped from about 66.1% to 73.3% (+7.3% absolute), and Qwen3-4B-Instruct rose from 39.4% to 46.1% (+6.7%). On HMMT math, gains up to +10%. Think of this like moving from a B to a solid A on the leaderboard.
• Budget-matched: With the same total calls, pairwise beats pointwise consistently and scales smoothly with more comparisons, showing compute is used more effectively.
• Versus RSA: On LiveCodeBench-v6 with N=16, $V_I$nfer reached 76% Pass@1 with only 48 verification calls—higher accuracy than RSA achieved with more calls. That’s like finishing the race faster and ahead.
• Real-world SWE-bench Lite: Pairwise verification chose better bug-fix patches—33.3% resolved vs. 28.3% (pointwise) and 26.3% (vanilla). It’s the difference between fixing 1 in 3 issues vs. 1 in 4.

Surprising and insightful findings:

• Biggest wins on hard problems: For the toughest tasks, pairwise verification’s boost was largest (e.g., +23.7% gains with higher budget). That’s where sharp judging matters most.
• Smart pairing beats random pairing: Uncertainty-guided Swiss refinement outperformed random pair selection by ~3.8% in a test—proof that where you compare matters.
• Pairwise + RSA is complementary: Using pairwise scores as a “fitness signal” inside RSA sped up convergence and achieved higher final accuracy than RSA alone—like giving the coach a better scoreboard during tryouts.
• Pointwise saturation: In code, pointwise often gave many candidates 10/10, losing discrimination, while pairwise head-to-head exposed true differences.

$V_P$airRL training results:

• Better test-time scaling: With N=16 and $2× verification$ budget, $V_P$airRL beat pointwise co-training by 6–7% across multiple code benchmarks.
• Better than plain RL even when both used $V_I$nfer at inference: Gains of +1.9% to +8.9% show that teaching pairwise judging during training pays off later.
• Better base Pass@1 (no scaling): Co-training improved generation quality itself, up to +8.7% on CodeContests, indicating stronger underlying reasoning.
• Co-evolving > offline multi-task: Training the verifier online on current model outputs beat using offline pre-collected pairs, across all benchmarks.

Bottom line: Pairwise verification, done smartly and trained jointly, reliably raises top-1 accuracy, especially on hard tasks, and uses compute more wisely than classic approaches.

Limitations:

• Compute overhead: Pairwise comparisons add verification calls. $V_I$nfer makes it efficient (O(N)ish) but it’s still extra cost over picking the first sample.
• Judge quality dependence: The same model judges its own work. While pairwise mitigates bias and calibration issues, self-judging can still err, especially on very subtle or domain-specific nuances.
• Very similar candidates: When two answers are nearly identical and both plausible, even many comparisons may struggle to separate them (diminishing returns).
• Training requires verifiable rewards: $V_P$airRL uses ground-truth tests (e.g., code unit tests). Domains without clear verification signals are harder to train for.
• Prompt and scale sensitivity: The 1–10 rating prompts and hyperparameters (τ, h, min-degree) matter; poor settings can reduce gains.

Required resources:

• For inference: Ability to generate N candidates (e.g., 8–16) and run B pairwise LLM calls (e.g., 1–$3×N$), plus a simple controller implementing coverage + Swiss refinement.
• For training: An RL setup (e.g., GRPO-style), a dataset with executable or checkable rewards (for code/math), and enough compute for shared generation-and-verification rollouts.

When NOT to use:

• Obvious, objective tasks where majority voting with exact answers is cheap and sufficient (simple math)—pairwise may be overkill.
• Extreme latency constraints where any extra judge calls are unacceptable.
• Domains lacking any way to verify correctness during training (then $V_P$airRL’s verifier reward is tricky to define).

Open questions:

• Cross-domain generalization: How well does pairwise judging transfer to long-form writing, science Q&A, or multi-modal tasks without ground truth tests?
• Multi-winner selection: Can we extend beyond top-1 to top-k or solution set curation robustly?
• Hybrid strategies: What’s the optimal schedule for mixing aggregation and pairwise verification under strict budgets?
• Learning the pairing policy: Can we learn to propose pairs end-to-end, further improving efficiency?
• Adversarial robustness: How resilient is pairwise judging to cleverly misleading solutions?

Three-sentence summary: This paper shows that language models are much better at judging two answers head-to-head than at scoring each alone, and builds a full system around that insight. $V_I$nfer is a smart, uncertainty-guided Swiss tournament that finds the best answer with fewer judge calls, while $V_P$airRL trains one model to both solve and verify using pairwise rewards so judging and solving improve together. The result is stronger, more scalable reasoning across code and math, with clear gains in real tasks like bug fixing.

Main achievement: Unifying generation and self-verification around pairwise comparison—both at inference ($V_I$nfer) and during training ($V_P$airRL)—which boosts Pass@1, avoids diversity collapse, and uses compute efficiently.

Future directions: Extend pairwise judging to richer domains (long-form, multi-modal), learn pairing policies end-to-end, blend with aggregation or search more tightly, and develop verifier rewards for tasks without executable tests. Investigate reliability under distribution shift and adversarial cases, and refine training signals to reduce any lingering self-bias.

Why remember this: When you can’t trust absolute scores, compare two answers. That simple switch—plus a clever tournament and co-training—turns many tries into reliable wins, making parallel reasoning truly pay off in practice.