Clarification on Baseline Configuration Differences and Training Temperature

[weiwang1342-wq]

Hello authors,

Thank you for the nice work. I have a few questions regarding some parts of the paper and the experimental setup.

1) Baseline comparison in Table 1

From Table 1, it seems that some baseline results, such as d1, ESPO, SPG, and d TreeRPO, may have been directly taken from the original papers rather than reproduced within the same codebase. My understanding is that the missing entries appear because the corresponding experiments are not reported in the original papers.

However, I am concerned that there may be important configuration differences across these results. For example, the original d1 codebase typically uses quantization plus LoRA training, whereas the codebase used in this paper appears to be based on full finetuning and may also use different reward implementations. In addition, different papers may use different training settings, such as the number of optimization steps, batch sizes, or other hyperparameters.

If the values in Table 1 are directly taken from the original papers, these configuration differences may affect the fairness of the comparison. Could the authors clarify how this issue is handled? Were all baselines reproduced under the same codebase and training configuration, or were some numbers taken directly from prior work? Please let me know if I misunderstood this point.

2) Temperature setting during training

In Figure 8 of Appendix B.1, the authors report the reasoning potential under different temperature settings. I noticed that the arbitrary order strategy, which corresponds to confidence based decoding in LLaDA, outperforms AR by a large margin at temperature 1.0. However, the default temperature in GRPO training appears to be set to 1.0, which does not seem to be the most favorable setting for AR.

Could the authors provide the exact temperature configuration used during training? Also, if temperature 1.0 is indeed the default choice, could the authors explain the reason behind this choice? In particular, why not use sharper temperatures, such as 0.6 or 0.3, where AR seems to outperform the confidence based strategy more clearly?

Thank you for reading my questions. I look forward to your clarification.

[LeapLabTHU]

Thank you for your questions. We respond to each below.

On Q1 (baseline comparison in Table 1).

Thanks for the careful reading. You are right that the Table 1 numbers are quoted from each baseline's originally reported configuration. We agree the configurations are not fully aligned. Since the initial release, we have been revising and extending the paper. As part of this, we have reproduced several representative baselines (e.g., d1, ESPO, SPG) under a configuration aligned with ours, and JustGRPO retains clear margins under this matched setting. These reproduced results will be included in an updated version, which we will release within about a week.

On Q2 (Temperature setting during training).

Yes, the rollout temperature used during GRPO training is 1.0, following our base recipe adapted from LLaDOU. This is a hyperparameter we inherited from that recipe and did not separately ablate. A systematic study of the rollout temperature would be a reasonable direction for future work, and we do not have additional results on it at this point.

On your T = 1.0 observation: one thing to note in reading Figure 8 is that the x axis is on a log scale, which is the natural choice for showing scaling trends but makes the location of the crossover easy to misjudge visually. The 64 to 128 span covers as many values of k as the entire 1 to 64 range. So the region where AR leads is not a small tail at the end. Meanwhile, what matters for our claim, however, is the trend across k. Arbitrary order's lead steadily shrinks as k grows, and AR overtakes it by k = 128, with AR's curve scaling more steeply throughout. Reasoning potential is about large-k coverage, the RL-attainable upper bound, i.e., whether a correct solution can be reached under a large sampling budget rather than in a single shot. The steeper AR slope and the takeover at large k are therefore consistent with our main conclusion.

[weiwang1342-wq]

Respectfully, I'd like this issue reopened (not closed). This appears to be a substantial issue. I hope others in the community will read through it, since their perspective would help establish whether this is as serious as it looks.

[LeapLabTHU]

Thanks again for the comments.

As mentioned, we have just uploaded the major revision (https://arxiv.org/abs/2601.15165v4) as part of our ongoing updates.

Regarding the baseline comparison in Table 1, the revised version explicitly states that the baseline numbers are quoted from the original papers, and it flags the salient configuration differences across them. To reduce ambiguity further, we added a matched-configuration check for several representative baselines under a protocol closer to ours (Table 2). This table is meant to address the concern behind Table 1, namely whether the observed advantage could stem mainly from configuration differences rather than from the method itself. Under this check the qualitative conclusion is unchanged: JustGRPO stays ahead of the representative baselines we examined.

On Q2, beyond the rationale in our earlier reply, we have also updated the relevant analysis in the revised version. As noted before, the rollout temperature 1.0 follows our base recipe from LLaDOU. Building on our earlier point about reading the crossover at T = 1.0, the revised version extends the Pass@k curve at this temperature out to k = 256 (Appendix B.1, Figure 10). Past the crossover the AR lead does not plateau or get reabsorbed; it keeps widening, with AR scaling more steeply throughout. So the trend at T = 1.0 is consistent with our main conclusion.

We hope these additions help clarify the two points.

[weiwang1342-wq]

Thanks for the update. However, I still do not think my main concerns are fully addressed.

First, disclosing that Table 1 numbers are quoted from prior papers does not fully resolve the fairness issue. The paper still uses these results to support the main performance narrative, including Figure 1. If JustGRPO uses full finetuning while some baselines use quantization plus LoRA or different reward/training setups, it is unclear how the gain can be attributed mainly to the AR generation strategy. Could the authors clarify how readers should interpret these comparisons?

Second, for Table 2, I think “matched configuration” needs to be specified more precisely. Are the methods matched only by training steps, or also by actual compute cost? Please report the compute budget for each method, such as number of forward passes, rollout budget, batch size, generated tokens, GPU hours, and whether full finetuning, LoRA, or quantization is used. If possible, please also release the exact scripts and configs for Table 2.

Third, my temperature question is not about whether AR eventually wins in Pass@k at T = 1.0. My question is why GRPO training uses T = 1.0 when the paper’s own analysis suggests AR looks more favorable at sharper temperatures such as 0.6. If rollout temperature was not ablated during training, I think this limitation should be stated clearly.

Overall, I still think this is a substantial experimental fairness issue, not just a clarification. I would appreciate further comments from the authors and perspectives from others in the community. For this reason, I respectfully request that this issue remain open. I want many readers in the community to judge this issue.

[weiwang1342-wq]

One additional clarification I would like to ask:

Were the ICML reviewers and area chair clearly informed during the review process that the main Table 1 comparison used baseline numbers quoted from prior papers, despite substantial training setup differences such as full finetuning versus quantization plus LoRA, and potentially different reward implementations and training budgets?

I think this is important because the paper uses these comparisons to support its main performance narrative. If these details were not clearly disclosed during review, I believe this issue should be treated as more than a minor post-hoc clarification.

[LeapLabTHU]

We provide a quick response to your questions.

The configuration differences in Table 1 are explicitly disclosed in the paper, and reporting originally published baseline numbers is also an established practice in this area, e.g., in LLaDOU (NeurIPS'25), GDPO (ICLR'26), and AGRPO (ICML'26). To ensure fairness under stricter constraints, we provided the matched-setting comparison in Table 2, which was fully presented and evaluated during the peer-review process (the full review record will become publicly available on OpenReview afterwards). Our baseline reproductions follow the official implementations of the respective papers.

The choice of $T=1.0$ follows our base recipe (LLaDOU) as stated above and is a common GRPO default. While sharper temperatures present an interesting scaling property, a comprehensive training-time temperature ablation is beyond the scope of this work. We are happy to see future research on this matter.