Skip to content

Perf/simd byte equality #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
official-markus-maurer wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Perf/simd byte equality #25

official-markus-maurer wants to merge 5 commits into from

Conversation

@official-markus-maurer
Copy link

@official-markus-maurer official-markus-maurer commented Nov 12, 2025

Summary

  • Replaces manual byte-by-byte loops with Span.SequenceEqual(...) for hardware-accelerated comparisons throughout hot paths.
  • Adds fast-path ReferenceEquals , consistent null/length guards, and inlining hints to minimize overhead.
  • Reduces allocation pressure in scan/hash flows and stream/comparison utilities; preserves existing external behavior.

Key Changes

  • Byte array equality (SIMD path):

    • Compress/CompressUtils.cs:84 uses b0.AsSpan().SequenceEqual(b1) .
    • FileScanner/FileScan.cs:534 updates ByteArrCompare to b0.AsSpan().SequenceEqual(b1) .
    • RomVaultCore/Utils/ArrByte.cs:45 updates BCompare ; RomVaultCore/Utils/ArrByte.cs:65 updates ECompare .
    • DATReader/Utils/ArrByte.cs:22 updates bCompare ; DATReader/Utils/ArrByte.cs:41 updates nCompare .
    • Compress/SevenZip/Util.cs:82 updates Compare(this byte[] b1, byte[] b2) .
    • CHDlib/Utils/Util.cs:31 updates ByteArrEquals .

  • Imports for MemoryExtensions :

    • Compress/SevenZip/Util.cs:1 , DATReader/Utils/ArrByte.cs:1 , CHDlib/Utils/Util.cs:1 include using System; .

  • Package support for netstandard2.0 :

    • Added System.Memory references to Compress/Compress.csproj , RomVaultCore/RomVaultCore.csproj.
    • DATReader/DATReader.csproj , CHDlib/CHDSharpLib.csproj , RVZstdSharp/RVZstdSharp.csproj .

  • Misc micro-optimizations (earlier commits visible in history):

    • Throttle progress string updates in hashing loop.
    • Switch buffer management to ArrayPool where applicable.
    • Use string.Equals(..., StringComparison.Ordinal) and inline char normalization to avoid per-call allocations.

Performance Impact

  • SequenceEqual over Span leverages vectorized comparisons and reduces bounds-check/branch overhead versus manual loops.
  • ReferenceEquals prevents work for identical buffers; early null/length exits minimize iteration.
  • Reduced temporary allocations in hashing and string-heavy paths lowers GC pressure on large scans.

@official-markus-maurer
refactor: improve file scanning and hash comparison logic
701c733 
- Optimize buffer copying using Buffer.BlockCopy instead of manual loop
- Add reference equality check in ByteArrCompare for performance
- Initialize altsha256 thread when scanSHA256 is true
- Fix incorrect buffer size passed to tsha256.Trigger calls
@official-markus-maurer
refactor: optimize string comparison and buffer management
95189d6 
- Replace manual string comparison with String.Equals in CompareString
- Simplify CompareStringSlash logic and add case-insensitive comparison
- Replace BlockingCollection with ArrayPool for buffer management
- Move empty hash constants to static readonly fields
@official-markus-maurer
fix: optimize byte array comparison by adding reference equality check
b08fea7 
Add reference equality check at the start of byte array comparison methods to avoid unnecessary element-by-element comparison when arrays are the same instance. This improves performance for cases where identical array references are compared.
@official-markus-maurer
perf: optimize byte array comparison methods with inlining
d54688c 
Add MethodImplOptions.AggressiveInlining to byte array comparison methods across multiple files to improve performance. Also optimize progress reporting in FileScan by using string.Concat and adjusting reporting frequency.
@official-markus-maurer
refactor: replace manual byte array comparison with Span.SequenceEqual
363bfdd 
Add System.Memory package to multiple projects to enable Span<T> usage
Replace manual byte array equality checks with more efficient Span.SequenceEqual
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@official-markus-maurer