Skip to content

JIT: fix FitsIn<int32_t> assert in BitOperations.Rotate{Left,Right} const-fold#129136

Open
AndyAyersMS wants to merge 4 commits into
dotnet:mainfrom
AndyAyersMS:fix-129099-rotate-fold
Open

JIT: fix FitsIn<int32_t> assert in BitOperations.Rotate{Left,Right} const-fold#129136
AndyAyersMS wants to merge 4 commits into
dotnet:mainfrom
AndyAyersMS:fix-129099-rotate-fold

Conversation

@AndyAyersMS

Copy link
Copy Markdown
Member

Note

PR description is AI-generated (GitHub Copilot CLI). The investigation, fix, and regression test are checked by me.

Fixes #129099.

Root cause

NI_PRIMITIVE_RotateLeft / NI_PRIMITIVE_RotateRight's constant-folding path (importercalls.cpp) passes the unsigned fold result directly to gtNewIconNode(ssize_t, TYP_INT):

uint32_t cns1 = static_cast<uint32_t>(op1->AsIntConCommon()->IconValue());
result        = gtNewIconNode(BitOperations::RotateLeft(cns1, cns2), baseType);

For TYP_INT/TYP_UINT operands with the high bit set — e.g. BitOperations.RotateRight(0xFFFFFFFFu, k) which folds to 0xFFFFFFFF — the implicit uint32_tssize_t conversion zero-extends to a positive value (4294967295) that does not fit in int32_t. GenTreeIntCon::SetIconValue then trips its FitsIn<int32_t>(value) assert during Morph - Global.

The ulong overload is unaffected because it uses gtNewLconNode (64-bit).

Fix

Cast through int32_t first so the sign bit is preserved when widened to ssize_t:

result = gtNewIconNode(
    static_cast<int32_t>(BitOperations::RotateLeft(cns1, cns2)), baseType);

RotateLeft(0xFFFFFFFFu, 1) is still 0xFFFFFFFFu; reinterpreting that as int32_t gives -1; widening -1 to ssize_t gives -1; FitsIn<int32_t>(-1) is true. Downstream consumers reading the IconValue as a uint32_t (via static_cast<uint32_t>(IconValue)) recover 0xFFFFFFFF correctly.

Regression test

src/tests/JIT/Regression/JitBlue/Runtime_129099/Runtime_129099.cs exercises both RotateLeft and RotateRight on 0xFFFFFFFFu, 0x80000000u (high-bit-only), and int.RotateRight(-1, _) (signed-int overload, sanity). Each call stores into a volatile static so the fold result must be materialized — without this the JIT can dead-code-eliminate the call before the assert fires.

Wired into src/tests/JIT/Regression/Regression_ro_2.csproj.

Verified locally on osx-arm64.Checked:

  • Before fix: Assert failure ... 'FitsIn<int32_t>(value)' ... 'Runtime_129099:FoldRotateRightUInt():uint' during 'Morph - Global'
  • After fix: all four cases pass.

A ~3,000-trial ReifyCs sweep that previously found 7 instances of this assert now finds 0.

…onst-fold

NI_PRIMITIVE_RotateLeft/Right's constant-folding path passes the unsigned
fold result directly to gtNewIconNode(ssize_t, TYP_INT). For TYP_INT/TYP_UINT
operands with the high bit set (e.g. RotateRight(0xFFFFFFFFu, k) which folds
to 0xFFFFFFFF), the implicit uint32_t-to-ssize_t conversion zero-extends to
a positive value (4294967295) that does not fit in int32_t, tripping
GenTreeIntCon::SetIconValue's FitsIn<int32_t>(value) assert during
'Morph - Global'.

Cast through int32_t first so the sign bit is preserved when widened to
ssize_t. Fixes dotnet#129099.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 8, 2026 20:34
@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jun 8, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a JIT constant-folding bug for NI_PRIMITIVE_RotateLeft / NI_PRIMITIVE_RotateRight where 32-bit rotate results with the high bit set could be represented as a zero-extended ssize_t, ultimately tripping int32-range assertions during later IR mutation.

Changes:

  • Adjust rotate constant-folding to cast the folded 32-bit result through int32_t before creating the GT_CNS_INT node.
  • Add a new JitBlue regression test covering RotateLeft/RotateRight folding for high-bit-set inputs and wire it into the regression project.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/coreclr/jit/importercalls.cpp Casts 32-bit rotate fold results through int32_t to preserve sign when widened and avoid later int32-range asserts.
src/tests/JIT/Regression/JitBlue/Runtime_129099/Runtime_129099.cs Adds a regression test that forces materialization of folded rotate results (via volatile sinks) and validates expected outputs.
src/tests/JIT/Regression/Regression_ro_2.csproj Includes the new Runtime_129099 regression test in the JIT regression build.

Comment thread src/coreclr/jit/importercalls.cpp Outdated
Comment on lines +6411 to +6415
// Sign-extend the unsigned fold result to int32_t so gtNewIconNode's
// FitsIn<int32_t>(value) check for TYP_INT/TYP_UINT folds doesn't trip
// on the high-bit-set case (e.g. RotateLeft(0xFFFFFFFFu, k)).
result = gtNewIconNode(
static_cast<int32_t>(BitOperations::RotateLeft(cns1, cns2)), baseType);
Comment thread src/coreclr/jit/importercalls.cpp Outdated
Comment on lines +6464 to +6468
// Sign-extend the unsigned fold result to int32_t so gtNewIconNode's
// FitsIn<int32_t>(value) check for TYP_INT/TYP_UINT folds doesn't trip
// on the high-bit-set case (e.g. RotateRight(0xFFFFFFFFu, k)).
result = gtNewIconNode(
static_cast<int32_t>(BitOperations::RotateRight(cns1, cns2)), baseType);
Comment on lines +4 to +6
// NI_PRIMITIVE_RotateLeft/Right's const-fold path stored the unsigned
// fold result into gtNewIconNode(ssize_t, TYP_INT). For uint operands
// with the high bit set (e.g. RotateRight(0xFFFFFFFFu, k)) the result
Comment on lines +6414 to +6415
result = gtNewIconNode(
static_cast<int32_t>(BitOperations::RotateLeft(cns1, cns2)), baseType);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the best/correct way to do it?

That is, this seems like a general issue with gtNewIconNode(ssize_t) since we default to TYP_INT and so that scenario should really have an assert(FitsIn<int32_t>(ssize_t)) or insert the static_cast<int32_t>(value) itself, since anything else is just "incorrect IR"

Anything that doesn't fit rather should be TYP_LONG and should've gone through gtNewLconNode instead (or possibly TYP_BYREF on 64-bit for the few cases that have it).

I wonder if even the general signature of gtNewIconNode is "incorrect" and if it rather should be int32_t instead, to help enforce correctness here; particularly since any larger value may need to be LconNode to work on 32-bit.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Reply is AI-generated (GitHub Copilot CLI).

Agreed — the call-site cast is a workaround for a missing API contract. I've pushed an update that also adds the invariant at gtNewIconNode itself:

GenTreeIntCon* Compiler::gtNewIconNode(ssize_t value, var_types type)
{
    assert(genActualType(type) == type);
    assert(genTypeSize(type) > genTypeSize(TYP_INT) || FitsIn<int32_t>(value));
    return new (this, GT_CNS_INT) GenTreeIntCon(type, value);
}

This catches the bug class at construction time rather than per-call-site. Smoke-verified locally:

  • All built Regression_ro_* JIT regression tests still pass
  • A 120-trial ReifyCs sweep produces no firings of the new assert
  • The fix-129099 regression test continues to fail without the call-site fix (the assert at gtNewIconNode would also fire) and pass with it

On the deeper signature question (int32_t vs ssize_t): a much larger refactor that touches 455 call sites — many of which legitimately want ssize_t for TYP_I_IMPL/TYP_BYREF/TYP_LONG use. The assert above gives us the safety net without the API churn. I'd suggest filing a separate issue for the signature change if you want to pursue it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I don't know if int32_t is the right direction but will dig into it some.

There's something unnecessarily clunky about the icon nodes in general. Not sure I want to revisit that right now though.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting... AI is getting ahead of itself here and just replying on its own. Let me reign it in a bit.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, a bit unsure as to the direction myself, but I think assert is a good starting point and will help catch any other issues longer term.

- Comments now correctly attribute the FitsIn<int32_t> assert to the
  downstream SetIconValue/BashToConst call (the construction itself is
  unchecked). Updated both the call-site comments and the regression
  test's header.
- Add the same FitsIn<int32_t> assert to gtNewIconNode itself (per
  @tannergooding) so this invariant is enforced at the IR-construction
  boundary, not just at the rotate call site. Anything larger than int32
  should go through gtNewLconNode (TYP_LONG) or use TYP_I_IMPL on 64-bit.
  Verified: all built Regression_ro_*.dll tests still pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@AndyAyersMS

Copy link
Copy Markdown
Member Author

Looks like this has turned up at least 4 other places needing a fix. Let me think about the right way to proceed here.

@AndyAyersMS

Copy link
Copy Markdown
Member Author

Note

This comment was authored by GitHub Copilot CLI on @AndyAyersMS's machine.
The two candidate patches are diff-only; I rebuilt the JIT locally to confirm both compile and pass a smoke test against the same gtNewIconNode assert.

Looking at the 27 CI failures on this PR's last build (1455021), the new assert(genTypeSize(type) > genTypeSize(TYP_INT) || FitsIn<int32_t>(value)) in gtNewIconNode fires on these distinct callers (deduped from the ILC/JIT logs):

# Caller (method:phase) JIT site
1 System.Numerics.Vector{2,3,4}.IsNegativeInfinity during Importation gentree.cpp:25441 gtNewIconNode(0xFF800000) in gtNewSimdIsNegativeInfinityNode
2 System.Convert:ToUInt32(double):uint during Lowering nodeinfo lowerxarch.cpp:815 gtNewIconNode(static_cast<ssize_t>(UINT32_MAX)) in LowerCastFloatToInt
3 System.Linq.Expressions.Interpreter.NumericConvertInstruction+Unchecked:ConvertDouble(double):System.Object during Lowering nodeinfo same as #2
4 TestConversions:Run() during Lowering nodeinfo same as #2

So really two distinct JIT sites, both the same pattern: a uint/0xFF… literal zero-extends through ssize_t and trips the new TYP_INT-size check.

Candidate patch

--- a/src/coreclr/jit/gentree.cpp
+++ b/src/coreclr/jit/gentree.cpp
@@ -25438,7 +25438,7 @@ GenTree* Compiler::gtNewSimdIsNegativeInfinityNode(var_types   type,
         else if (simdBaseType == TYP_FLOAT)
         {
             simdBaseType = TYP_UINT;
-            cnsNode      = gtNewIconNode(0xFF800000);
+            cnsNode      = gtNewIconNode(static_cast<int32_t>(0xFF800000));
         }

--- a/src/coreclr/jit/lowerxarch.cpp
+++ b/src/coreclr/jit/lowerxarch.cpp
@@ -812,7 +812,7 @@ void Lowering::LowerCastFloatToInt(...)
                         convertIntrinsic = TargetArchitecture::Is64Bit
                                                ? NI_X86Base_X64_ConvertToInt64WithTruncation
                                                : NI_X86Base_ConvertToVector128Int32WithTruncation;
-                        maxIntegralValue = m_compiler->gtNewIconNode(static_cast<ssize_t>(UINT32_MAX));
+                        maxIntegralValue = m_compiler->gtNewIconNode(static_cast<int32_t>(UINT32_MAX));
                         minFloatOverflow = 4294967296.0; // 2^32;
                         break;
                     }

Verification

Applied this PR's two-hunk patch plus the candidate fix locally on windows.x64.Checked. A 4-line C# smoke that exercises Vector2.IsNegativeInfinity(NegInf) and Convert.ToUInt32(2147483648.5) returns 100 (was tripping both asserts before the candidate fix).

Fuzzer cross-check

Just as a sanity input — last night's ReifyCs run produced 31 unique crash seeds across 5 fertile profiles (intrinsic / struct_intrinsic / mixed_sign / mixed_types / stage2_eh) all hitting the original FitsIn<int32_t>(value) assert in BashToConst. All 31 are fixed by this PR's two-hunk const-fold patch alone — none of them tickle the additional gtNewIconNode callsites above, so the fuzzer didn't independently surface #1-#4.

Happy to push the candidate patches to this PR (or to a separate one) if the direction is right — let me know.

These are the additional sites that trip the new assert added to
gtNewIconNode in the previous commit. Same root cause and same fix
shape -- sign-extend a uint bit pattern through int32_t so the
ssize_t parameter fits in int32_t for a TYP_INT-sized const.

* gentree.cpp gtNewSimdIsNegativeInfinityNode: 0xFF800000 (the
  single negative-infinity bit pattern) used as a TYP_UINT broadcast.
* lowerxarch.cpp LowerCastFloatToInt: UINT32_MAX used as the
  saturation bound when lowering double-to-uint.

Surfaced via the prior CI build (1455021) which hit these on
System.Numerics.Vector{2,3,4}.IsNegativeInfinity (Importation) and
System.Convert.ToUInt32(double) / NumericConvertInstruction.ConvertDouble
(Lowering nodeinfo).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings June 13, 2026 16:25
@AndyAyersMS

Copy link
Copy Markdown
Member Author

Pushed eb91446e083 on top of 06e6c92d0eb (no force) with the two candidate hunks:

  • gentree.cpp gtNewSimdIsNegativeInfinityNode0xFF800000static_cast<int32_t>(0xFF800000)
  • lowerxarch.cpp LowerCastFloatToIntstatic_cast<ssize_t>(UINT32_MAX)static_cast<int32_t>(UINT32_MAX)

Built locally on windows.x64.Checked. The smoke that crashed both asserts before now returns 100.

Note

I noticed a local-only wip commit (471ef40) on your machine on this branch fixing TryLowerConstIntUDivOrUMod in lower.cpp — looks like a 3rd site for the same bug class (TYP_INT divisor with bit 31 set). I left it untouched and didn't push it. If you'd like it on this PR too, you can cherry-pick it on top of eb91446e083 and push.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

? NI_X86Base_X64_ConvertToInt64WithTruncation
: NI_X86Base_ConvertToVector128Int32WithTruncation;
maxIntegralValue = m_compiler->gtNewIconNode(static_cast<ssize_t>(UINT32_MAX));
maxIntegralValue = m_compiler->gtNewIconNode(static_cast<int32_t>(UINT32_MAX));
Comment on lines +9062 to +9066
// For TYP_INT-sized constants the value must fit in int32_t; otherwise the
// node is in an invalid state and downstream SetIconValue / BashToConst
// will assert when the constant is updated. Wider values should use
// gtNewLconNode (TYP_LONG) or TYP_I_IMPL on 64-bit targets.
assert(genTypeSize(type) > genTypeSize(TYP_INT) || FitsIn<int32_t>(value));
@AndyAyersMS

Copy link
Copy Markdown
Member Author

Resumed fuzzing with both pushed fixes baked into a runtime-a build (commit eb91446e083 + the existing #129288 rotate-ZF fix from PR #129348). Phase 1A (intrinsic + STRESS, 5000 trials, seed range 20.0B) just finished and surfaced 4 crashes, all the same signature:

Assertion failed 'genTypeSize(type) > genTypeSize(TYP_INT) || FitsIn<int32_t>(value)'
during 'Lowering nodeinfo'   (gentree.cpp:9075 - the new assert)

Instrumented gtNewIconNode with a one-shot printf to capture the bad call. For seed 20000001650:

[BADICON] phase=Lowering nodeinfo value=0xfffa03ea type=int

Source line that triggers it: v30 = v17 % unchecked((uint)0xFFFA03EA); — i.e. a UMOD with a TYP_INT divisor whose bit 31 is set.

That's exactly the 3rd site your local wip commit (471ef406b3e) already fixes in Lowering::TryLowerConstIntUDivOrUMod at lower.cpp:8350. The fuzzer just independently confirmed the bug is real and reachable from C# UMOD with a large uint divisor (no SIMD or HW intrinsic required). All 4 seeds — 20000001650, 20000001931, 20000001985, 20000002772 — are the same shape.

If you'd like, I can cherry-pick your wip commit's lower.cpp hunk on top of eb91446e083 (just the UMOD hunk — leaving the loopcloning.cpp hunk for separate consideration) and push as a third commit on this PR. Just let me know.

Fuzz is continuing through P1B–E + Phase 2 — will report any new signatures as they surface.

Third site for the same bug class. Hits any C# expression of the form
`x %u (uint)C` where C has bit 31 set: divisorValue is computed as
a ssize_t after masking to UINT32_MAX and passed to gtNewIconNode
with TYP_INT, which trips the assert added earlier in this PR.

Sign-extend through int32_t so the constant round-trips correctly
(same shape as the importercalls.cpp Rotate{Left,Right} fix in the
first commit and the IsNegativeInfinity / UINT32_MAX fixes in the
prior commit).

Independently surfaced 31 times by a post-fix fuzz run across 5
profiles -- now the dominant signature for this assert class once
the const-fold and prior gtNewIconNode sites are fixed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@AndyAyersMS

Copy link
Copy Markdown
Member Author

Post-fix fuzz complete — 37,000 trials, 31 crashes, all the same UMOD signature. Pushed d6921fbb580 with the 3rd-site fix.

Run setup

Phase 1 — re-run the 5 originally fertile #129099 profiles

Profile Trials Crashes New shapes
intrinsic (blocks 16-24) 5000 4 UMOD
struct_intrinsic (blocks 12-18) 4000 2 UMOD
mixed_sign (blocks 14-20) 4000 8 UMOD
mixed_types I32-U64 4000 11 UMOD
stage2_eh (blocks 8-14) 4000 4 UMOD

The original BitOperations.Rotate{Left,Right} const-fold pattern (the bug this PR was opened for) is fully cleared. All 29 crashes hit Lowering::TryLowerConstIntUDivOrUMod at lower.cpp:8350.

Phase 2 — exercise the JIT areas my new fixes touched

Profile Trials Crashes Mismatches
intrinsic F32+F64 4000 0 1 (ReifyCs#25 interp FP — all 6 JIT configs agree)
mixed_types F32+F64 4000 0 0
v128_compose 5000 0 0
stage2_eh deeper (blocks 12-18) 3000 2 UMOD

No new bug shapes. The 1 mismatch is the known narrow-int interpreter false positive.

Fix applied (d6921fbb580)

Cherry-pick of just the lower.cpp hunk from your local wip commit (471ef40) — I left the loopcloning.cpp hunk alone since it's for #129176 and a separate concern. Verified locally by re-running 1 representative seed from each fertile profile: all 5 now pass.

Net state of this PR

4 commits, all additive (no force-pushes):

  1. 785e502a46d — const-fold Rotate{Left,Right}
  2. 06e6c92d0eb — review feedback (added gtNewIconNode assert)
  3. eb91446e083 — gentree.cpp IsNegativeInfinity + lowerxarch.cpp LowerCastFloatToInt
  4. d6921fbb580TryLowerConstIntUDivOrUMod

The new gtNewIconNode assert should now cover the full bug class once CI confirms. Happy to launch a deeper/longer fuzz to look for any remaining sites if you want, but the latest run was very clean outside this UMOD signature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

JIT: assertion 'FitsIn<int32_t>(value)' on constant-folded BitOperations.RotateRight(0xFFFFFFFFu, k) with multiple uses

3 participants