Compare commits

..
Author SHA1 Message Date
mlemanczyk 497b1fcf89 Kernels replaced with pre-compiled actions for better performance 2025-11-20 03:00:11 +01:00
mlemanczyk df04ba601c Various small optimizations and cleanups. Methods made static, where possible. Duplicated & slow PrimeTester.BinaryGcd implementation removed. Optimized version was used instead. 2025-11-20 01:43:29 +01:00
mlemanczyk 9939c5d562 Divisor will now be saved to CSV file 2025-11-19 22:14:28 +01:00
mlemanczyk 54fdff0a4e GPU calculations frequency adjusted 2025-11-19 21:38:07 +01:00
mlemanczyk 5cac51479e GPU memory allocations protected with lock to prevent shared memory access issues. GPU usage limited for the best performance. 2025-11-19 16:49:01 +01:00
mlemanczyk 02d16bf05c Tiny cleanup 2025-11-18 06:02:43 +01:00
mlemanczyk a5696ca17f App crash fix related to unassigned value 2025-11-18 06:02:22 +01:00
mlemanczyk 56be12ab95 Cleanup & compilation fixes. 2025-11-18 04:45:18 +01:00
mlemanczyk bc51962f9d Cleanup 2025-11-18 04:37:11 +01:00
mlemanczyk 801b1be3e3 Cleanup 2025-11-18 04:22:39 +01:00
mlemanczyk 66ca969929 Naming improvements and small cleanup 2025-11-18 03:43:52 +01:00
mlemanczyk bf7c4bab39 Accelerator buffers integrated 2025-11-18 01:13:12 +01:00
mlemanczyk 342eff2726 GpuKernelPool.GetOrAddKernels removed in --mersenne=bydivisor mode. PartialFactorResult.WithAdditionalPrime will now modify the current instance instead of creating new copy for better performance. SmallPrimeFactorViews renamed to SmallPrimeFactorGpuTables for consistency with other static tables. 2025-11-17 23:33:59 +01:00
mlemanczyk 2fc74cf98d Cleanup 2025-11-17 20:12:00 +01:00
mlemanczyk f630bfb598 HeuristicCombinedPrimeTesterAccelerator integrated with PrimeOrderCalculatorAccelerator. 2025-11-17 17:38:23 +01:00
mlemanczyk 183b147135 Heuristic tables split into 2 groups 2025-11-17 16:04:27 +01:00
mlemanczyk c413ee1162 Revert "Stream handling simplified to avoid dictionary lookups in favor of array lookups with known indexes."
This reverts commit a84445ef2a.
2025-11-16 17:14:48 +01:00
mlemanczyk a84445ef2a Stream handling simplified to avoid dictionary lookups in favor of array lookups with known indexes. 2025-11-16 15:01:51 +01:00
mlemanczyk cd7636ddbe Manual accelerator stream creations replaced with pool 2025-11-15 13:45:24 +01:00
mlemanczyk 0540411cad Accelerator index added for dictionary-less use-cases 2025-11-15 02:21:45 +01:00
mlemanczyk 6cb63b6e73 Internal accelerator array removed 2025-11-14 12:03:31 +01:00
mlemanczyk 42447c4e43 BinaryGcd switched to the best performing variant according to benchmarks. 2025-11-14 11:43:16 +01:00
mlemanczyk be435b7760 Kernels made part of the accelerators for better performance. 2025-11-13 18:52:46 +01:00
mlemanczyk 0d2a88c45f GpuPrimeWorkLimiter.Acquire() / .Release() removed to save time on the hot-path with billions of calls. We'll rely on AcceleratorStreamPool limits and synchronization. 2025-11-13 17:52:13 +01:00
mlemanczyk 1552e10ff0 VS Code settings updated. 2025-11-13 15:42:09 +01:00
mlemanczyk c53aa517f7 GpuScratchBuffer / -Pool naively integrated with PriemOrderCalculatorAccelerator (formerly Pow2MontgomeryAccelerator). FactorEntry replaced with 2 arrays - unused field removed. Small cleanup. 2025-11-13 14:41:27 +01:00
mlemanczyk 43863d03fb Performance improvements 2025-11-13 00:51:32 +01:00
mlemanczyk 5bc65464d0 Initial PollardRho implementation for GPU. Added pool for accelerator streams shared across the threads. GPU processing performance and stabilization improvements. GpuUInt128.BinaryGcd switched to the version which outperformed other in benchmarks. 2025-11-12 21:18:30 +01:00
mlemanczyk 735bfa1557 GPU accelerator recycled for better performance. 2025-11-12 07:39:52 +01:00
mlemanczyk fdc2da7dae Accelerator leak fixed. 2025-11-11 16:40:44 +01:00
mlemanczyk cf5f78c2ef PrimeTesterByLastDigitAccelerator integrated into Pow2MontgomeryGpuCalculator for better performance. 2025-11-11 13:36:21 +01:00
mlemanczyk 0de9871529 Functionality and classes separated. Kernel initialization made dependent on a flag. Cleanup. 2025-11-10 18:35:50 +01:00
mlemanczyk 45dfa3b724 Console logging extended with kernel warmup progress 2025-11-10 18:18:12 +01:00
mlemanczyk fa89958736 Stabilization improvements. Stream will be released when releasing GPU lease to prevent application crashes. 2025-11-10 14:30:30 +01:00
Marek Lemańczyk 2726c369f6 Merge pull request #256 from mlemanczyk/codex/optimize-evenperfectbitscanner-for-static-arrays
Adopt upstream GPU scratch isolation fixes
2025-11-09 21:53:50 +01:00
mlemanczyk d0b1639a5d GPU stabilization improvements to prevent calculation crashes 2025-11-09 21:52:16 +01:00
mlemanczyk a338bb0d0b Merge branch 'codex/optimize-evenperfectbitscanner-for-static-arrays' of https://github.com/mlemanczyk/even-perfect-numbers-scanner into codex/optimize-evenperfectbitscanner-for-static-arrays 2025-11-07 15:39:11 +01:00
mlemanczyk 8e779c5c9e GpuKernelLease removed 2025-11-07 15:36:45 +01:00
mlemanczyk da3093a9ba GpuKernelLease removed 2025-11-07 15:18:17 +01:00
mlemanczyk 0410fd2c91 PooledContext removal - continuation. GpuKernelLease prepared for removal. 2025-11-07 14:13:11 +01:00
mlemanczyk f4e365194d GpuContextPool removed, because we share context and accelerator between threads, using streams to separate. 2025-11-07 12:52:20 +01:00
mlemanczyk 7c035239b4 Cleanup. Classes & structs extracted to their own files. 2025-11-07 10:34:31 +01:00
mlemanczyk 119690086a ConcurrentDictionary removed. Kernel made dependent on the shared GPU context. 2025-11-07 10:16:33 +01:00
mlemanczyk 9537addf38 ArrayPool<T> replaced with ThreadStaticPools 2025-11-07 00:46:16 +01:00
mlemanczyk c004e140db Ignore words extended 2025-11-06 18:54:18 +01:00
mlemanczyk de2a78b27b Streams added to all kernels and GPU operations to separate threads from each other. 2025-11-06 18:53:55 +01:00
mlemanczyk 64d615c402 Various optimizations to separate non-static data between threads and resolve read/write access issue. 2025-11-04 20:45:56 +01:00
Marek Lemańczyk 5a17fa64d5 Restore per-lease kernel containers 2025-11-04 00:46:59 +01:00
Marek Lemańczyk dbde6337ea Tie GPU scratch buffers to context leases 2025-11-04 00:26:38 +01:00
Marek Lemańczyk 4c4d4243fb Merge remote-tracking branch 'github/codex/optimize-evenperfectbitscanner-for-static-arrays-khd9w8' into work
# Conflicts:
#	PerfectNumbers.Core/Gpu/GpuContextPool.cs
#	PerfectNumbers.Core/Gpu/GpuKernelLease.cs
#	PerfectNumbers.Core/Gpu/GpuKernelPool.cs
#	PerfectNumbers.Core/Gpu/GpuStaticTableInitializer.cs
#	PerfectNumbers.Core/Gpu/Kernels/PrimeOrderSpecialMaxKernels.cs
#	PerfectNumbers.Core/PrimeOrderCalculator.Cpu.cs
#	PerfectNumbers.Core/PrimeOrderCalculator.Gpu.cs
#	PerfectNumbers.Core/PrimeTester.cs
2025-11-03 23:59:12 +01:00
124 changed files with 8587 additions and 7747 deletions

No files matched your search

+3 -1
View File
@@ -9,5 +9,7 @@
"Mersenne", "Mersenne",
"primality", "primality",
"suff" "suff"
] ],
"geminicodeassist.project": "evenperfectbitscanner",
"todo-tree.tree.showCountsInTree": true
} }
@@ -1,4 +1,3 @@
using System;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,6 +1,5 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using PerfectNumbers.Core;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
namespace EvenPerfectBitScanner.Benchmarks; namespace EvenPerfectBitScanner.Benchmarks;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using PerfectNumbers.Core; using PerfectNumbers.Core;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,5 +1,3 @@
using System;
namespace EvenPerfectBitScanner.Benchmarks; namespace EvenPerfectBitScanner.Benchmarks;
public enum ModNumericType public enum ModNumericType
@@ -1,4 +1,3 @@
using System;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -1,12 +1,8 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
namespace EvenPerfectBitScanner.Benchmarks; namespace EvenPerfectBitScanner.Benchmarks;
@@ -1,4 +1,3 @@
using System;
using System.Buffers.Binary; using System.Buffers.Binary;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
@@ -101,7 +100,7 @@ public class Pow2ModBenchmarks
for (int i = 0; i < SampleCount; i++) for (int i = 0; i < SampleCount; i++)
{ {
checksum ^= exponents[i].Pow2MontgomeryModWindowedGpu(divisors[i], keepMontgomery: false); checksum ^= exponents[i].Pow2MontgomeryModWindowedConvertGpu(divisors[i]);
} }
return checksum; return checksum;
@@ -77,7 +77,7 @@ public class Pow2MontgomeryModBenchmarks
for (int i = 0; i < SampleCount; i++) for (int i = 0; i < SampleCount; i++)
{ {
checksum ^= exponents[i].Pow2MontgomeryModWindowedGpu(divisors[i], keepMontgomery: false); checksum ^= exponents[i].Pow2MontgomeryModWindowedConvertGpu(divisors[i]);
} }
return checksum; return checksum;
@@ -1,4 +1,3 @@
using System;
using PerfectNumbers.Core; using PerfectNumbers.Core;
namespace EvenPerfectBitScanner.Benchmarks; namespace EvenPerfectBitScanner.Benchmarks;
@@ -1,4 +1,3 @@
using System;
using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
using PerfectNumbers.Core; using PerfectNumbers.Core;
@@ -34,7 +33,7 @@ public class Pow2MontgomeryModWindowedKeepMontgomeryGpuBenchmarks
for (int i = 0; i < cases.Length; i++) for (int i = 0; i < cases.Length; i++)
{ {
ref readonly Pow2MontgomeryModWindowedBenchmarkCase current = ref cases[i]; ref readonly Pow2MontgomeryModWindowedBenchmarkCase current = ref cases[i];
checksum ^= current.Exponent.Pow2MontgomeryModWindowedGpu(current.Divisor, keepMontgomery: false); checksum ^= current.Exponent.Pow2MontgomeryModWindowedConvertGpu(current.Divisor);
} }
return checksum; return checksum;
@@ -52,7 +51,7 @@ public class Pow2MontgomeryModWindowedKeepMontgomeryGpuBenchmarks
for (int i = 0; i < cases.Length; i++) for (int i = 0; i < cases.Length; i++)
{ {
ref readonly Pow2MontgomeryModWindowedBenchmarkCase current = ref cases[i]; ref readonly Pow2MontgomeryModWindowedBenchmarkCase current = ref cases[i];
checksum ^= current.Exponent.Pow2MontgomeryModWindowedGpu(current.Divisor, keepMontgomery: true); checksum ^= current.Exponent.Pow2MontgomeryModWindowedKeepGpu(current.Divisor);
} }
return checksum; return checksum;
@@ -2,6 +2,7 @@ using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Jobs; using BenchmarkDotNet.Jobs;
using Open.Numeric.Primes; using Open.Numeric.Primes;
using PerfectNumbers.Core; using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu.Accelerators;
namespace EvenPerfectBitScanner.Benchmarks; namespace EvenPerfectBitScanner.Benchmarks;
@@ -25,12 +26,14 @@ public class PrimeTesterBenchmarks
return; return;
} }
_ = _heuristicTester.IsPrimeCpu(_candidates[0]); PrimeOrderCalculatorAccelerator.WarmUp();
_ = _heuristicTester.IsPrimeGpu(_candidates[0]); // HeuristicGroupABPrimeTesterAccelerator.WarmUp();
_ = HeuristicPrimeTester.IsPrimeCpu(_candidates[0]);
_ = HeuristicPrimeTester.IsPrimeGpu(_candidates[0]);
_ = Prime.Numbers.IsPrime(_candidates[0]); _ = Prime.Numbers.IsPrime(_candidates[0]);
} }
[Benchmark] // [Benchmark]
public int HeuristicCpu() public int HeuristicCpu()
{ {
HeuristicPrimeTester tester = _heuristicTester; HeuristicPrimeTester tester = _heuristicTester;
@@ -39,7 +42,7 @@ public class PrimeTesterBenchmarks
for (int i = 0; i < values.Length; i++) for (int i = 0; i < values.Length; i++)
{ {
if (tester.IsPrimeCpu(values[i])) if (HeuristicPrimeTester.IsPrimeCpu(values[i]))
{ {
primeCount++; primeCount++;
} }
@@ -57,7 +60,7 @@ public class PrimeTesterBenchmarks
for (int i = 0; i < values.Length; i++) for (int i = 0; i < values.Length; i++)
{ {
if (tester.IsPrimeGpu(values[i])) if (HeuristicPrimeTester.IsPrimeGpu(values[i]))
{ {
primeCount++; primeCount++;
} }
@@ -134,7 +137,7 @@ public class PrimeTesterBenchmarks
return primeCount; return primeCount;
} }
[Benchmark] // [Benchmark]
public int OpenNumericPrimes() public int OpenNumericPrimes()
{ {
ulong[] values = _candidates; ulong[] values = _candidates;
@@ -19,7 +19,7 @@ internal static class Program
"/help" "/help"
}; };
private const string DefaultHeader = "p,searchedMersenne,detailedCheck,passedAllTests"; private const string DefaultHeader = "p,divisor,searchedMersenne,detailedCheck,passedAllTests";
private static int Main(string[] args) private static int Main(string[] args)
{ {
@@ -3,13 +3,9 @@ using Xunit;
using System.Reflection; using System.Reflection;
using PerfectNumbers.Core; using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using EvenPerfectBitScanner;
using EvenPerfectBitScanner.Candidates; using EvenPerfectBitScanner.Candidates;
using EvenPerfectBitScanner.Candidates.Transforms; using EvenPerfectBitScanner.Candidates.Transforms;
using System.Numerics; using System.Numerics;
using System.Threading;
using System.Threading.Tasks;
namespace EvenPerfectBitScanner.Tests; namespace EvenPerfectBitScanner.Tests;
@@ -102,13 +98,11 @@ public class ProgramTests
var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!; var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!;
var candidatesField = typeof(MersenneNumberDivisorGpuTester).GetField("_divisorCandidates", BindingFlags.NonPublic | BindingFlags.Static)!; var candidatesField = typeof(MersenneNumberDivisorGpuTester).GetField("_divisorCandidates", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
testerField.SetValue(null, new MersenneNumberDivisorGpuTester()); testerField.SetValue(null, new MersenneNumberDivisorGpuTester());
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
candidatesField.SetValue(null, new (ulong, uint)[] { (7UL, 3U), (23UL, 11U) }); candidatesField.SetValue(null, new (ulong, uint)[] { (7UL, 3U), (23UL, 11U) });
forceCpuProp!.SetValue(null, true);
try try
{ {
@@ -122,7 +116,6 @@ public class ProgramTests
candidatesField.SetValue(null, null); candidatesField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -135,13 +128,11 @@ public class ProgramTests
var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!; var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!;
var candidatesField = typeof(MersenneNumberDivisorGpuTester).GetField("_divisorCandidates", BindingFlags.NonPublic | BindingFlags.Static)!; var candidatesField = typeof(MersenneNumberDivisorGpuTester).GetField("_divisorCandidates", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
testerField.SetValue(null, new MersenneNumberDivisorGpuTester()); testerField.SetValue(null, new MersenneNumberDivisorGpuTester());
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
candidatesField.SetValue(null, Array.Empty<(ulong, uint)>()); candidatesField.SetValue(null, Array.Empty<(ulong, uint)>());
forceCpuProp!.SetValue(null, true);
try try
{ {
@@ -155,7 +146,6 @@ public class ProgramTests
candidatesField.SetValue(null, null); candidatesField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -168,12 +158,10 @@ public class ProgramTests
var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!; var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
testerField.SetValue(null, new MersenneNumberDivisorGpuTester()); testerField.SetValue(null, new MersenneNumberDivisorGpuTester());
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, true);
try try
{ {
@@ -186,7 +174,6 @@ public class ProgramTests
testerField.SetValue(null, null); testerField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -199,12 +186,10 @@ public class ProgramTests
var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!; var testerField = typeof(Program).GetField("_divisorTester", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
testerField.SetValue(null, new MersenneNumberDivisorGpuTester()); testerField.SetValue(null, new MersenneNumberDivisorGpuTester());
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, false);
try try
{ {
@@ -221,7 +206,6 @@ public class ProgramTests
testerField.SetValue(null, null); testerField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -233,12 +217,10 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(useIncremental: true, useResidue: true, maxK: 1_000UL), trackAllValues: true)); mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(useIncremental: true, useResidue: true, maxK: 1_000UL), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, false);
try try
{ {
@@ -255,7 +237,6 @@ public class ProgramTests
mersenneField.SetValue(null, null); mersenneField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -267,7 +248,6 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester( mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(
useIncremental: true, useIncremental: true,
@@ -275,7 +255,6 @@ public class ProgramTests
maxK: 1_024UL), trackAllValues: true)); maxK: 1_024UL), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, false);
try try
{ {
@@ -292,7 +271,6 @@ public class ProgramTests
mersenneField.SetValue(null, null); mersenneField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -305,7 +283,6 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester( mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(
useIncremental: true, useIncremental: true,
@@ -313,7 +290,6 @@ public class ProgramTests
maxK: 5_000_000UL), trackAllValues: true)); maxK: 5_000_000UL), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, false);
try try
{ {
@@ -330,7 +306,6 @@ public class ProgramTests
mersenneField.SetValue(null, null); mersenneField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -342,7 +317,6 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester( mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(
useIncremental: true, useIncremental: true,
@@ -350,7 +324,6 @@ public class ProgramTests
maxK: 5_000_000UL), trackAllValues: true)); maxK: 5_000_000UL), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, false);
try try
{ {
@@ -367,7 +340,6 @@ public class ProgramTests
mersenneField.SetValue(null, null); mersenneField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -380,7 +352,6 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester( mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(
useIncremental: true, useIncremental: true,
@@ -390,7 +361,6 @@ public class ProgramTests
maxK: 1_024UL), trackAllValues: true)); maxK: 1_024UL), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, exponent, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, exponent, true), trackAllValues: true));
forceCpuProp!.SetValue(null, true);
try try
{ {
@@ -402,7 +372,6 @@ public class ProgramTests
mersenneField.SetValue(null, null); mersenneField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -417,7 +386,6 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester( mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(
useIncremental: false, useIncremental: false,
@@ -428,7 +396,6 @@ public class ProgramTests
maxK: configuration.ResidueMaxK), trackAllValues: true)); maxK: configuration.ResidueMaxK), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, exponent, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, exponent, true), trackAllValues: true));
forceCpuProp!.SetValue(null, false);
try try
{ {
@@ -440,9 +407,7 @@ public class ProgramTests
mersenneField.SetValue(null, null); mersenneField.SetValue(null, null);
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
GpuContextPool.DisposeAll();
} }
} }
@@ -506,12 +471,10 @@ public class ProgramTests
var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var mersenneField = typeof(Program).GetField("MersenneTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!; var primeField = typeof(Program).GetField("PrimeTesters", BindingFlags.NonPublic | BindingFlags.Static)!;
var compositeField = typeof(Program).GetField("_lastCompositeP", BindingFlags.NonPublic | BindingFlags.Static)!; var compositeField = typeof(Program).GetField("_lastCompositeP", BindingFlags.NonPublic | BindingFlags.Static)!;
var forceCpuProp = typeof(GpuContextPool).GetProperty("ForceCpu");
mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(useIncremental: true, useResidue: false, maxK: 1_000UL), trackAllValues: true)); mersenneField.SetValue(null, new ThreadLocal<MersenneNumberTester>(() => new MersenneNumberTester(useIncremental: true, useResidue: false, maxK: 1_000UL), trackAllValues: true));
primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true)); primeField.SetValue(null, new ThreadLocal<PrimeTester>(() => new PrimeTester(), trackAllValues: true));
CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true)); CandidatesCalculator.OverrideResidueTrackers(new ThreadLocal<ModResidueTracker>(() => new ModResidueTracker(ResidueModel.Identity, 2UL, true), trackAllValues: true));
forceCpuProp!.SetValue(null, true);
try try
{ {
@@ -532,7 +495,6 @@ public class ProgramTests
primeField.SetValue(null, null); primeField.SetValue(null, null);
CandidatesCalculator.ResetResidueTrackers(); CandidatesCalculator.ResetResidueTrackers();
compositeField.SetValue(null, false); compositeField.SetValue(null, false);
forceCpuProp!.SetValue(null, false);
ResetCliArguments(); ResetCliArguments();
} }
} }
@@ -1,6 +1,3 @@
using System.Collections.Generic;
using System.IO;
using System.Numerics;
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core; using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
@@ -1,15 +1,13 @@
using System.Buffers.Text; using System.Buffers.Text;
using System.IO;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Text; using System.Text;
using System.Threading;
using PerfectNumbers.Core; using PerfectNumbers.Core;
namespace EvenPerfectBitScanner; namespace EvenPerfectBitScanner;
internal static class CalculationResultHandler internal static class CalculationResultHandler
{ {
private const string ResultsHeader = "p,searchedMersenne,detailedCheck,passedAllTests"; private const string ResultsHeader = "p,divisor,searchedMersenne,detailedCheck,passedAllTests";
private const string PrimeFoundSuffix = " (FOUND VALID CANDIDATES)"; private const string PrimeFoundSuffix = " (FOUND VALID CANDIDATES)";
internal const int DefaultWriteBatchSize = 100; internal const int DefaultWriteBatchSize = 100;
@@ -95,19 +93,19 @@ internal static class CalculationResultHandler
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static void HandleResult(ulong currentP, bool searchedMersenne, bool detailedCheck, bool passedAllTests, bool lastWasComposite) internal static void HandleResult(ulong currentP, ulong divisor, bool searchedMersenne, bool detailedCheck, bool passedAllTests, bool lastWasComposite)
{ {
// Program flow ensures Initialize(...) and InitializeOutputBuffer() ran before this call. Do not add initialization guards here. // Program flow ensures Initialize(...) and InitializeOutputBuffer() ran before this call. Do not add initialization guards here.
Span<byte> stackBuffer = stackalloc byte[128]; Span<byte> stackBuffer = stackalloc byte[128];
byte[]? rentedBuffer = null; byte[]? rentedBuffer = null;
Span<byte> utf8Span = stackBuffer; Span<byte> utf8Span = stackBuffer;
int byteCount = TryFormatResultLine(utf8Span, currentP, searchedMersenne, detailedCheck, passedAllTests); int byteCount = TryFormatResultLine(utf8Span, currentP, divisor, searchedMersenne, detailedCheck, passedAllTests);
if (byteCount < 0) if (byteCount < 0)
{ {
rentedBuffer = ThreadStaticPools.BytePool.Rent(256); rentedBuffer = ThreadStaticPools.BytePool.Rent(256);
utf8Span = rentedBuffer; utf8Span = rentedBuffer;
byteCount = TryFormatResultLine(utf8Span, currentP, searchedMersenne, detailedCheck, passedAllTests); byteCount = TryFormatResultLine(utf8Span, currentP, divisor, searchedMersenne, detailedCheck, passedAllTests);
if (byteCount < 0) if (byteCount < 0)
{ {
ThreadStaticPools.BytePool.Return(rentedBuffer); ThreadStaticPools.BytePool.Return(rentedBuffer);
@@ -231,7 +229,7 @@ internal static class CalculationResultHandler
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
private static int TryFormatResultLine(Span<byte> destination, ulong currentP, bool searchedMersenne, bool detailedCheck, bool passedAllTests) private static int TryFormatResultLine(Span<byte> destination, ulong currentP, ulong divisor, bool searchedMersenne, bool detailedCheck, bool passedAllTests)
{ {
int offset = 0; int offset = 0;
@@ -248,6 +246,19 @@ internal static class CalculationResultHandler
destination[offset++] = (byte)','; destination[offset++] = (byte)',';
if (!Utf8Formatter.TryFormat(divisor, destination[offset..], out written))
{
return -1;
}
offset += written;
if (offset >= destination.Length)
{
return -1;
}
destination[offset++] = (byte)',';
if (!Utf8Formatter.TryFormat(searchedMersenne, destination[offset..], out written)) if (!Utf8Formatter.TryFormat(searchedMersenne, destination[offset..], out written))
{ {
return -1; return -1;
@@ -1,5 +1,4 @@
using System.Globalization; using System.Globalization;
using System.IO;
namespace EvenPerfectBitScanner.Candidates; namespace EvenPerfectBitScanner.Candidates;
@@ -1,5 +1,4 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Threading;
using Open.Numeric.Primes; using Open.Numeric.Primes;
using PerfectNumbers.Core; using PerfectNumbers.Core;
using EvenPerfectBitScanner.Candidates.Transforms; using EvenPerfectBitScanner.Candidates.Transforms;
@@ -1,6 +1,4 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Threading;
using EvenPerfectBitScanner.Candidates;
using Open.Numeric.Primes; using Open.Numeric.Primes;
namespace EvenPerfectBitScanner.Candidates.Transforms; namespace EvenPerfectBitScanner.Candidates.Transforms;
@@ -1,6 +1,4 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Threading;
using EvenPerfectBitScanner.Candidates;
namespace EvenPerfectBitScanner.Candidates.Transforms; namespace EvenPerfectBitScanner.Candidates.Transforms;
@@ -1,5 +1,4 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Threading;
namespace EvenPerfectBitScanner.Candidates.Transforms; namespace EvenPerfectBitScanner.Candidates.Transforms;
-2
View File
@@ -1,6 +1,4 @@
using System;
using System.Globalization; using System.Globalization;
using System.Numerics;
using PerfectNumbers.Core; using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
@@ -1,7 +1,6 @@
using Open.Collections; using Open.Collections;
using PerfectNumbers.Core; using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using EvenPerfectBitScanner;
namespace EvenPerfectBitScanner.IO; namespace EvenPerfectBitScanner.IO;
@@ -101,6 +100,7 @@ internal static class CalculationResultsFile
int first = 0; int first = 0;
int second = 0; int second = 0;
int third = 0; int third = 0;
int fourth = 0;
ulong parsedP = 0UL; ulong parsedP = 0UL;
ReadOnlySpan<char> detailedSpan = default; ReadOnlySpan<char> detailedSpan = default;
ReadOnlySpan<char> passedAllTestsSpan = default; ReadOnlySpan<char> passedAllTestsSpan = default;
@@ -140,8 +140,15 @@ internal static class CalculationResultsFile
continue; continue;
} }
detailedSpan = span[..third]; span = span[(third + 1)..];
passedAllTestsSpan = span[(third + 1)..]; fourth = span.IndexOf(',');
if (fourth < 0)
{
continue;
}
detailedSpan = span[..fourth];
passedAllTestsSpan = span[(fourth + 1)..];
if (Utf8CliParser.TryParseBoolean(detailedSpan, out detailed) && Utf8CliParser.TryParseBoolean(passedAllTestsSpan, out passedAllTests)) if (Utf8CliParser.TryParseBoolean(detailedSpan, out detailed) && Utf8CliParser.TryParseBoolean(passedAllTestsSpan, out passedAllTests))
{ {
+9 -7
View File
@@ -58,12 +58,14 @@ internal static class Program
bool startPrimeProvided = _cliArguments.StartPrimeProvided; bool startPrimeProvided = _cliArguments.StartPrimeProvided;
int threadCount = Math.Max(1, _cliArguments.ThreadCount); int threadCount = Math.Max(1, _cliArguments.ThreadCount);
int gpuPrimeBatch = Math.Max(1, _cliArguments.GpuPrimeBatch); int gpuPrimeBatch = Math.Max(1, _cliArguments.GpuPrimeBatch);
// int gpuPrimeThreads = SharedGpuContext.Device.MaxNumThreads;
int gpuPrimeThreads = Math.Max(1, _cliArguments.GpuPrimeThreads); int gpuPrimeThreads = Math.Max(1, _cliArguments.GpuPrimeThreads);
UnboundedTaskScheduler.ConfigureThreadCount(threadCount); UnboundedTaskScheduler.ConfigureThreadCount(threadCount);
PrimeTester.GpuBatchSize = gpuPrimeBatch; PrimeTester.GpuBatchSize = gpuPrimeBatch;
GpuPrimeWorkLimiter.SetLimit(gpuPrimeThreads); GpuPrimeWorkLimiter.SetLimit(gpuPrimeThreads);
GpuContextPool.WarmUpPool(threadCount); Console.WriteLine("Warming up GPU kernels");
PrimeTester.WarmUpGpuKernels(gpuPrimeThreads); PrimeTester.WarmUpGpuKernels(1024);
// PrimeTester.WarmUpGpuKernels(gpuPrimeThreads >> 4);
Console.WriteLine("Starting up threads..."); Console.WriteLine("Starting up threads...");
_ = UnboundedTaskScheduler.Instance; _ = UnboundedTaskScheduler.Instance;
int blockSize = Math.Max(1, _cliArguments.BlockSize); int blockSize = Math.Max(1, _cliArguments.BlockSize);
@@ -420,7 +422,7 @@ internal static class Program
_byDivisorStartPrime, _byDivisorStartPrime,
static () => _lastCompositeP = true, static () => _lastCompositeP = true,
static () => _lastCompositeP = false, static () => _lastCompositeP = false,
static (candidate, searchedMersenne, detailedCheck, passedAllTests) => CalculationResultHandler.HandleResult(candidate, searchedMersenne, detailedCheck, passedAllTests, _lastCompositeP), static (candidate, searchedMersenne, detailedCheck, passedAllTests, divisor) => CalculationResultHandler.HandleResult(candidate, divisor, searchedMersenne, detailedCheck, passedAllTests, _lastCompositeP),
threadCount, threadCount,
byDivisorPrimeRange); byDivisorPrimeRange);
@@ -481,7 +483,7 @@ internal static class Program
{ {
p = buffer[j]; p = buffer[j];
passedAllTests = IsEvenPerfectCandidate(p, divisorCyclesSearchLimit, out searchedMersenne, out detailedCheck); passedAllTests = IsEvenPerfectCandidate(p, divisorCyclesSearchLimit, out searchedMersenne, out detailedCheck);
CalculationResultHandler.HandleResult(p, searchedMersenne, detailedCheck, passedAllTests, _lastCompositeP); CalculationResultHandler.HandleResult(p, 0UL, searchedMersenne, detailedCheck, passedAllTests, _lastCompositeP);
} }
continue; continue;
@@ -504,7 +506,7 @@ internal static class Program
} }
passedAllTests = IsEvenPerfectCandidate(p, divisorCyclesSearchLimit, out searchedMersenne, out detailedCheck); passedAllTests = IsEvenPerfectCandidate(p, divisorCyclesSearchLimit, out searchedMersenne, out detailedCheck);
CalculationResultHandler.HandleResult(p, searchedMersenne, detailedCheck, passedAllTests, _lastCompositeP); CalculationResultHandler.HandleResult(p, 0UL, searchedMersenne, detailedCheck, passedAllTests, _lastCompositeP);
if (p == maxP) if (p == maxP)
{ {
@@ -626,7 +628,7 @@ internal static class Program
{ {
if (!(_runPrimesOnCpu if (!(_runPrimesOnCpu
? PrimeTester.IsPrime(p) ? PrimeTester.IsPrime(p)
: PrimeTester.IsPrimeGpu(p))) : HeuristicCombinedPrimeTester.IsPrimeGpu(p)))
{ {
_lastCompositeP = true; _lastCompositeP = true;
return false; return false;
@@ -647,7 +649,7 @@ internal static class Program
if (_cliArguments.UseByDivisor) if (_cliArguments.UseByDivisor)
{ {
// Windowed pow2mod kernel handles by-divisor scans. // Windowed pow2mod kernel handles by-divisor scans.
return _byDivisorTester!.IsPrime(p, out detailedCheck); return _byDivisorTester!.IsPrime(p, out detailedCheck, out _);
} }
detailedCheck = MersenneTesters.Value!.IsMersennePrime(p); detailedCheck = MersenneTesters.Value!.IsMersennePrime(p);
-1
View File
@@ -1,4 +1,3 @@
using System;
using System.Buffers; using System.Buffers;
using System.Buffers.Text; using System.Buffers.Text;
using System.Text; using System.Text;
@@ -1,7 +1,3 @@
using FluentAssertions;
using ILGPU;
using ILGPU.Runtime;
using PerfectNumbers.Core.Gpu;
using Xunit; using Xunit;
namespace PerfectNumbers.Core.Tests.Gpu; namespace PerfectNumbers.Core.Tests.Gpu;
@@ -1,8 +1,8 @@
using System.Reflection; using System.Reflection;
using FluentAssertions;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using PerfectNumbers.Core.Gpu.Accelerators;
using Xunit; using Xunit;
namespace PerfectNumbers.Core.Tests.Gpu; namespace PerfectNumbers.Core.Tests.Gpu;
@@ -114,14 +114,10 @@ public class GpuKernelCompilationTests
typeof(PrimeOrderGpuHeuristics).GetMethod("GetPow2ModKernel", BindingFlags.NonPublic | BindingFlags.Static) typeof(PrimeOrderGpuHeuristics).GetMethod("GetPow2ModKernel", BindingFlags.NonPublic | BindingFlags.Static)
?? throw new InvalidOperationException("PrimeOrderGpuHeuristics.GetPow2ModKernel not found."); ?? throw new InvalidOperationException("PrimeOrderGpuHeuristics.GetPow2ModKernel not found.");
private static readonly MethodInfo PrimeOrderGetPow2ModWideKernelMethod =
typeof(PrimeOrderGpuHeuristics).GetMethod("GetPow2ModWideKernel", BindingFlags.NonPublic | BindingFlags.Static)
?? throw new InvalidOperationException("PrimeOrderGpuHeuristics.GetPow2ModWideKernel not found.");
private static readonly Action<Index1D, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> MersenneDivisorCyclesKernel = private static readonly Action<Index1D, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> MersenneDivisorCyclesKernel =
DivisorCycleKernels.GpuDivisorCycleKernel; DivisorCycleKernels.GpuDivisorCycleKernel;
private static readonly Action<Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>> PrimeTesterSmallPrimeKernel = private static readonly Action<Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>> PrimeTesterSmallPrimeKernel =
PrimeTesterKernels.SmallPrimeSieveKernel; PrimeTesterKernels.SmallPrimeSieveKernel;
private static readonly Action<Index1D, ArrayView<ulong>, ArrayView<byte>> PrimeTesterSharesFactorKernel = private static readonly Action<Index1D, ArrayView<ulong>, ArrayView<byte>> PrimeTesterSharesFactorKernel =
@@ -325,56 +321,67 @@ public class GpuKernelCompilationTests
private static void CompilePrimeOrderPow2ModWideKernel(Accelerator accelerator) private static void CompilePrimeOrderPow2ModWideKernel(Accelerator accelerator)
{ {
_ = PrimeOrderGetPow2ModWideKernelMethod.Invoke(null, new object[] { accelerator }); var kernel = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView1D<GpuUInt128, Stride1D.Dense>, GpuUInt128, ArrayView1D<GpuUInt128, Stride1D.Dense>>(PrimeOrderGpuHeuristics.Pow2ModKernelWide);
_ = kernel;
} }
private static void CompileMersenneDivisorCyclesKernel(Accelerator accelerator) private static void CompileMersenneDivisorCyclesKernel(Accelerator accelerator)
{ {
var kernel = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(MersenneDivisorCyclesKernel); var kernel = accelerator.LoadAutoGroupedStreamKernel(MersenneDivisorCyclesKernel);
_ = kernel; _ = kernel;
} }
private static void CompilePrimeTesterSmallPrimeKernel(Accelerator accelerator) private static void CompilePrimeTesterSmallPrimeKernel(Accelerator accelerator)
{ {
var kernel = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>>(PrimeTesterSmallPrimeKernel); var kernel = accelerator.LoadAutoGroupedStreamKernel(PrimeTesterSmallPrimeKernel);
_ = kernel; _ = kernel;
} }
private static void CompilePrimeTesterSharesFactorKernel(Accelerator accelerator) private static void CompilePrimeTesterSharesFactorKernel(Accelerator accelerator)
{ {
var kernel = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<byte>>(PrimeTesterSharesFactorKernel); var kernel = accelerator.LoadAutoGroupedStreamKernel(PrimeTesterSharesFactorKernel);
_ = kernel; _ = kernel;
} }
private static void CompileGpuKernelPoolKernels(Accelerator accelerator) private static KernelType _kernelType = KernelType.Pow2ModKernelScan | KernelType.OrderKernelScan | KernelType.IncrementalKernelScan | KernelType.IncrementalOrderKernelScan | KernelType.Pow2ModOrderKernelScan;
{ private static void CompileGpuKernelPoolKernels(Accelerator accelerator)
_ = accelerator; {
var lease = GpuKernelPool.GetKernel(useGpuOrder: false);
try
{
_ = lease.OrderKernel;
_ = lease.IncrementalKernel;
_ = lease.Pow2ModKernel;
_ = lease.IncrementalOrderKernel;
_ = lease.Pow2ModOrderKernel;
}
finally
{
lease.Dispose();
}
var orderLease = GpuKernelPool.GetKernel(useGpuOrder: true); GpuPrimeWorkLimiter.Acquire();
try int acceleratorIndex = AcceleratorPool.Shared.Rent();
{ AcceleratorStream stream = AcceleratorStreamPool.Rent(acceleratorIndex);
_ = orderLease.OrderKernel; try
_ = orderLease.IncrementalKernel; {
_ = orderLease.Pow2ModKernel; var kernels = GpuKernelPool.GetOrAddKernels(acceleratorIndex, stream, _kernelType);
_ = orderLease.IncrementalOrderKernel; _ = kernels.Order;
_ = orderLease.Pow2ModOrderKernel; _ = kernels.Incremental;
} _ = kernels.Pow2Mod;
finally _ = kernels.IncrementalOrder;
{ _ = kernels.Pow2ModOrder;
orderLease.Dispose(); }
} finally
} {
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
// GpuPrimeWorkLimiter.Release();
}
stream = AcceleratorStreamPool.Rent(acceleratorIndex);
GpuPrimeWorkLimiter.Acquire();
try
{
var kernels = GpuKernelPool.GetOrAddKernels(acceleratorIndex, stream, _kernelType);
_ = kernels.Order;
_ = kernels.Incremental;
_ = kernels.Pow2Mod;
_ = kernels.IncrementalOrder;
_ = kernels.Pow2ModOrder;
}
finally
{
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
// GpuPrimeWorkLimiter.Release();
}
}
} }
+328 -343
View File
@@ -13,367 +13,352 @@ namespace PerfectNumbers.Core.Tests.Gpu;
[Collection("GpuNtt")] [Collection("GpuNtt")]
public class NttGpuMathTests public class NttGpuMathTests
{ {
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void ForwardGpu_reference_matches_cpu_forward() public void ForwardGpu_reference_matches_cpu_forward()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Reference; NttGpuMath.GpuTransformBackend = NttBackend.Reference;
Span<GpuUInt128> cpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[4];
cpu[0] = new GpuUInt128(0UL, 1UL); cpu[0] = new GpuUInt128(0UL, 1UL);
cpu[1] = new GpuUInt128(0UL, 2UL); cpu[1] = new GpuUInt128(0UL, 2UL);
cpu[2] = new GpuUInt128(0UL, 3UL); cpu[2] = new GpuUInt128(0UL, 3UL);
cpu[3] = new GpuUInt128(0UL, 4UL); cpu[3] = new GpuUInt128(0UL, 4UL);
Span<GpuUInt128> gpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[4];
cpu.CopyTo(gpu); cpu.CopyTo(gpu);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
NttGpuMath.Forward(cpu, modulus, primitiveRoot); NttGpuMath.Forward(cpu, modulus, primitiveRoot);
NttGpuMath.ForwardGpu(gpu, modulus, primitiveRoot); NttGpuMath.ForwardGpu(gpu, modulus, primitiveRoot);
for (int i = 0; i < cpu.Length; i++) for (int i = 0; i < cpu.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void ForwardGpu_staged_matches_cpu_forward() public void ForwardGpu_staged_matches_cpu_forward()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Staged; NttGpuMath.GpuTransformBackend = NttBackend.Staged;
Span<GpuUInt128> cpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[4];
cpu[0] = new GpuUInt128(0UL, 1UL); cpu[0] = new GpuUInt128(0UL, 1UL);
cpu[1] = new GpuUInt128(0UL, 2UL); cpu[1] = new GpuUInt128(0UL, 2UL);
cpu[2] = new GpuUInt128(0UL, 3UL); cpu[2] = new GpuUInt128(0UL, 3UL);
cpu[3] = new GpuUInt128(0UL, 4UL); cpu[3] = new GpuUInt128(0UL, 4UL);
Span<GpuUInt128> gpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[4];
cpu.CopyTo(gpu); cpu.CopyTo(gpu);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
NttGpuMath.Forward(cpu, modulus, primitiveRoot); NttGpuMath.Forward(cpu, modulus, primitiveRoot);
NttGpuMath.ForwardGpu(gpu, modulus, primitiveRoot); NttGpuMath.ForwardGpu(gpu, modulus, primitiveRoot);
for (int i = 0; i < cpu.Length; i++) for (int i = 0; i < cpu.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void InverseGpu_reference_matches_cpu_inverse() public void InverseGpu_reference_matches_cpu_inverse()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Reference; NttGpuMath.GpuTransformBackend = NttBackend.Reference;
Span<GpuUInt128> input = stackalloc GpuUInt128[4]; Span<GpuUInt128> input = stackalloc GpuUInt128[4];
input[0] = new GpuUInt128(0UL, 5UL); input[0] = new GpuUInt128(0UL, 5UL);
input[1] = new GpuUInt128(0UL, 7UL); input[1] = new GpuUInt128(0UL, 7UL);
input[2] = new GpuUInt128(0UL, 9UL); input[2] = new GpuUInt128(0UL, 9UL);
input[3] = new GpuUInt128(0UL, 11UL); input[3] = new GpuUInt128(0UL, 11UL);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
Span<GpuUInt128> transformed = stackalloc GpuUInt128[4]; Span<GpuUInt128> transformed = stackalloc GpuUInt128[4];
input.CopyTo(transformed); input.CopyTo(transformed);
NttGpuMath.Forward(transformed, modulus, primitiveRoot); NttGpuMath.Forward(transformed, modulus, primitiveRoot);
Span<GpuUInt128> cpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[4];
transformed.CopyTo(cpu); transformed.CopyTo(cpu);
NttGpuMath.Inverse(cpu, modulus, primitiveRoot); NttGpuMath.Inverse(cpu, modulus, primitiveRoot);
Span<GpuUInt128> gpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[4];
transformed.CopyTo(gpu); transformed.CopyTo(gpu);
NttGpuMath.InverseGpu(gpu, modulus, primitiveRoot); NttGpuMath.InverseGpu(gpu, modulus, primitiveRoot);
for (int i = 0; i < input.Length; i++) for (int i = 0; i < input.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void InverseGpu_staged_matches_cpu_inverse() public void InverseGpu_staged_matches_cpu_inverse()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Staged; NttGpuMath.GpuTransformBackend = NttBackend.Staged;
Span<GpuUInt128> input = stackalloc GpuUInt128[4]; Span<GpuUInt128> input = stackalloc GpuUInt128[4];
input[0] = new GpuUInt128(0UL, 5UL); input[0] = new GpuUInt128(0UL, 5UL);
input[1] = new GpuUInt128(0UL, 7UL); input[1] = new GpuUInt128(0UL, 7UL);
input[2] = new GpuUInt128(0UL, 9UL); input[2] = new GpuUInt128(0UL, 9UL);
input[3] = new GpuUInt128(0UL, 11UL); input[3] = new GpuUInt128(0UL, 11UL);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
Span<GpuUInt128> transformed = stackalloc GpuUInt128[4]; Span<GpuUInt128> transformed = stackalloc GpuUInt128[4];
input.CopyTo(transformed); input.CopyTo(transformed);
NttGpuMath.Forward(transformed, modulus, primitiveRoot); NttGpuMath.Forward(transformed, modulus, primitiveRoot);
Span<GpuUInt128> cpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[4];
transformed.CopyTo(cpu); transformed.CopyTo(cpu);
NttGpuMath.Inverse(cpu, modulus, primitiveRoot); NttGpuMath.Inverse(cpu, modulus, primitiveRoot);
Span<GpuUInt128> gpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[4];
transformed.CopyTo(gpu); transformed.CopyTo(gpu);
NttGpuMath.InverseGpu(gpu, modulus, primitiveRoot); NttGpuMath.InverseGpu(gpu, modulus, primitiveRoot);
for (int i = 0; i < input.Length; i++) for (int i = 0; i < input.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void Montgomery_twiddle_inverses_multiply_to_r2() public void Montgomery_twiddle_inverses_multiply_to_r2()
{ {
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
GpuContextPool.GpuContextLease gpu = GpuContextPool.Rent(); var accelerator = SharedGpuContext.CreateAccelerator();
try var stream = accelerator.CreateStream();
{
var accelerator = gpu.Accelerator;
var method = typeof(NttGpuMath).GetMethod("GetSquareCache", BindingFlags.NonPublic | BindingFlags.Static); var method = typeof(NttGpuMath).GetMethod("GetSquareCache", BindingFlags.NonPublic | BindingFlags.Static);
method.Should().NotBeNull(); method.Should().NotBeNull();
var cache = method!.Invoke(null, new object[] { accelerator, 8, modulus, primitiveRoot })!; var cache = method!.Invoke(null, new object[] { accelerator, 8, modulus, primitiveRoot })!;
var cacheType = cache.GetType(); var cacheType = cache.GetType();
var twiddlesMont = (MemoryBuffer1D<GpuUInt128, Stride1D.Dense>)cacheType.GetProperty("TwiddlesMont")!.GetValue(cache)!; var twiddlesMont = (MemoryBuffer1D<GpuUInt128, Stride1D.Dense>)cacheType.GetProperty("TwiddlesMont")!.GetValue(cache)!;
var twiddlesInvMont = (MemoryBuffer1D<GpuUInt128, Stride1D.Dense>)cacheType.GetProperty("TwiddlesInvMont")!.GetValue(cache)!; var twiddlesInvMont = (MemoryBuffer1D<GpuUInt128, Stride1D.Dense>)cacheType.GetProperty("TwiddlesInvMont")!.GetValue(cache)!;
var r2 = (ulong)cacheType.GetProperty("MontR2Mod64")!.GetValue(cache)!; var r2 = (ulong)cacheType.GetProperty("MontR2Mod64")!.GetValue(cache)!;
var mod = (ulong)cacheType.GetProperty("ModulusLow")!.GetValue(cache)!; var mod = (ulong)cacheType.GetProperty("ModulusLow")!.GetValue(cache)!;
int count = 7; // length 8 -> 7 twiddles int count = 7; // length 8 -> 7 twiddles
var forward = new GpuUInt128[count]; var forward = new GpuUInt128[count];
var inverse = new GpuUInt128[count]; var inverse = new GpuUInt128[count];
twiddlesMont.CopyToCPU(forward); twiddlesMont.CopyToCPU(stream, forward);
twiddlesInvMont.CopyToCPU(inverse); twiddlesInvMont.CopyToCPU(stream, inverse);
stream.Synchronize();
stream.Dispose();
accelerator.Dispose();
for (int i = 0; i < count; i++) for (int i = 0; i < count; i++)
{ {
ulong prod = (ulong)((((BigInteger)forward[i].Low) * inverse[i].Low) % mod); ulong prod = (ulong)((((BigInteger)forward[i].Low) * inverse[i].Low) % mod);
prod.Should().Be(r2); prod.Should().Be(r2);
} }
} }
finally
{
gpu.Dispose();
}
}
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void ConvolveGpu_reference_matches_cpu_convolve() public void ConvolveGpu_reference_matches_cpu_convolve()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Reference; NttGpuMath.GpuTransformBackend = NttBackend.Reference;
Span<GpuUInt128> leftCpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> leftCpu = stackalloc GpuUInt128[4];
Span<GpuUInt128> rightCpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> rightCpu = stackalloc GpuUInt128[4];
leftCpu[0] = new GpuUInt128(0UL, 1UL); leftCpu[0] = new GpuUInt128(0UL, 1UL);
leftCpu[1] = new GpuUInt128(0UL, 2UL); leftCpu[1] = new GpuUInt128(0UL, 2UL);
leftCpu[2] = new GpuUInt128(0UL, 3UL); leftCpu[2] = new GpuUInt128(0UL, 3UL);
leftCpu[3] = new GpuUInt128(0UL, 4UL); leftCpu[3] = new GpuUInt128(0UL, 4UL);
rightCpu[0] = new GpuUInt128(0UL, 5UL); rightCpu[0] = new GpuUInt128(0UL, 5UL);
rightCpu[1] = new GpuUInt128(0UL, 6UL); rightCpu[1] = new GpuUInt128(0UL, 6UL);
rightCpu[2] = new GpuUInt128(0UL, 7UL); rightCpu[2] = new GpuUInt128(0UL, 7UL);
rightCpu[3] = new GpuUInt128(0UL, 8UL); rightCpu[3] = new GpuUInt128(0UL, 8UL);
Span<GpuUInt128> leftGpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> leftGpu = stackalloc GpuUInt128[4];
Span<GpuUInt128> rightGpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> rightGpu = stackalloc GpuUInt128[4];
leftCpu.CopyTo(leftGpu); leftCpu.CopyTo(leftGpu);
rightCpu.CopyTo(rightGpu); rightCpu.CopyTo(rightGpu);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
NttGpuMath.Convolve(leftCpu, rightCpu, modulus, primitiveRoot); NttGpuMath.Convolve(leftCpu, rightCpu, modulus, primitiveRoot);
NttGpuMath.ConvolveGpu(leftGpu, rightGpu, modulus, primitiveRoot); NttGpuMath.ConvolveGpu(leftGpu, rightGpu, modulus, primitiveRoot);
for (int i = 0; i < leftCpu.Length; i++) for (int i = 0; i < leftCpu.Length; i++)
{ {
leftGpu[i].Should().Be(leftCpu[i]); leftGpu[i].Should().Be(leftCpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void ConvolveGpu_staged_matches_cpu_convolve() public void ConvolveGpu_staged_matches_cpu_convolve()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Staged; NttGpuMath.GpuTransformBackend = NttBackend.Staged;
Span<GpuUInt128> leftCpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> leftCpu = stackalloc GpuUInt128[4];
Span<GpuUInt128> rightCpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> rightCpu = stackalloc GpuUInt128[4];
leftCpu[0] = new GpuUInt128(0UL, 1UL); leftCpu[0] = new GpuUInt128(0UL, 1UL);
leftCpu[1] = new GpuUInt128(0UL, 2UL); leftCpu[1] = new GpuUInt128(0UL, 2UL);
leftCpu[2] = new GpuUInt128(0UL, 3UL); leftCpu[2] = new GpuUInt128(0UL, 3UL);
leftCpu[3] = new GpuUInt128(0UL, 4UL); leftCpu[3] = new GpuUInt128(0UL, 4UL);
rightCpu[0] = new GpuUInt128(0UL, 5UL); rightCpu[0] = new GpuUInt128(0UL, 5UL);
rightCpu[1] = new GpuUInt128(0UL, 6UL); rightCpu[1] = new GpuUInt128(0UL, 6UL);
rightCpu[2] = new GpuUInt128(0UL, 7UL); rightCpu[2] = new GpuUInt128(0UL, 7UL);
rightCpu[3] = new GpuUInt128(0UL, 8UL); rightCpu[3] = new GpuUInt128(0UL, 8UL);
Span<GpuUInt128> leftGpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> leftGpu = stackalloc GpuUInt128[4];
Span<GpuUInt128> rightGpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> rightGpu = stackalloc GpuUInt128[4];
leftCpu.CopyTo(leftGpu); leftCpu.CopyTo(leftGpu);
rightCpu.CopyTo(rightGpu); rightCpu.CopyTo(rightGpu);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
NttGpuMath.Convolve(leftCpu, rightCpu, modulus, primitiveRoot); NttGpuMath.Convolve(leftCpu, rightCpu, modulus, primitiveRoot);
NttGpuMath.ConvolveGpu(leftGpu, rightGpu, modulus, primitiveRoot); NttGpuMath.ConvolveGpu(leftGpu, rightGpu, modulus, primitiveRoot);
for (int i = 0; i < leftCpu.Length; i++) for (int i = 0; i < leftCpu.Length; i++)
{ {
leftGpu[i].Should().Be(leftCpu[i]); leftGpu[i].Should().Be(leftCpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void SquareGpu_reference_matches_cpu_square() public void SquareGpu_reference_matches_cpu_square()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Reference; NttGpuMath.GpuTransformBackend = NttBackend.Reference;
Span<GpuUInt128> cpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[4];
cpu[0] = new GpuUInt128(0UL, 1UL); cpu[0] = new GpuUInt128(0UL, 1UL);
cpu[1] = new GpuUInt128(0UL, 2UL); cpu[1] = new GpuUInt128(0UL, 2UL);
cpu[2] = new GpuUInt128(0UL, 3UL); cpu[2] = new GpuUInt128(0UL, 3UL);
cpu[3] = new GpuUInt128(0UL, 4UL); cpu[3] = new GpuUInt128(0UL, 4UL);
Span<GpuUInt128> gpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[4];
cpu.CopyTo(gpu); cpu.CopyTo(gpu);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
NttGpuMath.Square(cpu, modulus, primitiveRoot); NttGpuMath.Square(cpu, modulus, primitiveRoot);
NttGpuMath.SquareGpu(gpu, modulus, primitiveRoot); NttGpuMath.SquareGpu(gpu, modulus, primitiveRoot);
for (int i = 0; i < cpu.Length; i++) for (int i = 0; i < cpu.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void SquareGpu_staged_matches_cpu_square() public void SquareGpu_staged_matches_cpu_square()
{ {
var original = NttGpuMath.GpuTransformBackend; var original = NttGpuMath.GpuTransformBackend;
try try
{ {
NttGpuMath.GpuTransformBackend = NttBackend.Staged; NttGpuMath.GpuTransformBackend = NttBackend.Staged;
Span<GpuUInt128> cpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[4];
cpu[0] = new GpuUInt128(0UL, 1UL); cpu[0] = new GpuUInt128(0UL, 1UL);
cpu[1] = new GpuUInt128(0UL, 2UL); cpu[1] = new GpuUInt128(0UL, 2UL);
cpu[2] = new GpuUInt128(0UL, 3UL); cpu[2] = new GpuUInt128(0UL, 3UL);
cpu[3] = new GpuUInt128(0UL, 4UL); cpu[3] = new GpuUInt128(0UL, 4UL);
Span<GpuUInt128> gpu = stackalloc GpuUInt128[4]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[4];
cpu.CopyTo(gpu); cpu.CopyTo(gpu);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
NttGpuMath.Square(cpu, modulus, primitiveRoot); NttGpuMath.Square(cpu, modulus, primitiveRoot);
NttGpuMath.SquareGpu(gpu, modulus, primitiveRoot); NttGpuMath.SquareGpu(gpu, modulus, primitiveRoot);
for (int i = 0; i < cpu.Length; i++) for (int i = 0; i < cpu.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
} }
finally finally
{ {
NttGpuMath.GpuTransformBackend = original; NttGpuMath.GpuTransformBackend = original;
} }
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void BitReverseGpu_matches_cpu() public void BitReverseGpu_matches_cpu()
{ {
GpuUInt128[] cpu = new GpuUInt128[8]; GpuUInt128[] cpu = new GpuUInt128[8];
for (ulong i = 0; i < (ulong)cpu.Length; i++) for (ulong i = 0; i < (ulong)cpu.Length; i++)
{ {
cpu[i] = new GpuUInt128(0UL, i); cpu[i] = new GpuUInt128(0UL, i);
} }
GpuUInt128[] gpu = (GpuUInt128[])cpu.Clone(); GpuUInt128[] gpu = (GpuUInt128[])cpu.Clone();
NttGpuMath.BitReverse(cpu); NttGpuMath.BitReverse(cpu);
NttGpuMath.BitReverseGpu(gpu); NttGpuMath.BitReverseGpu(gpu);
gpu.Should().Equal(cpu); gpu.Should().Equal(cpu);
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void DisposeAll_clears_ntt_caches() public void DisposeAll_clears_ntt_caches()
{ {
// Ensure the cache starts empty. // Ensure the cache starts empty.
NttGpuMath.ClearCaches(); NttGpuMath.ClearCaches();
Span<GpuUInt128> values = stackalloc GpuUInt128[2]; Span<GpuUInt128> values = stackalloc GpuUInt128[2];
values[0] = new GpuUInt128(0UL, 1UL); values[0] = new GpuUInt128(0UL, 1UL);
values[1] = new GpuUInt128(0UL, 2UL); values[1] = new GpuUInt128(0UL, 2UL);
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
GpuContextPool.GpuContextLease gpu = GpuContextPool.Rent(); Accelerator accelerator = SharedGpuContext.CreateAccelerator();
try var stream = accelerator.CreateStream();
{ var buffer = accelerator.Allocate1D<GpuUInt128>(values.Length);
var buffer = gpu.Accelerator.Allocate1D<GpuUInt128>(values.Length); buffer.View.CopyFromCPU(stream, ref values[0], values.Length);
try NttGpuMath.SquareDevice(accelerator, stream, buffer.View, modulus, primitiveRoot);
{ stream.Synchronize();
buffer.View.CopyFromCPU(ref values[0], values.Length); stream.Dispose();
NttGpuMath.SquareDevice(gpu.Accelerator, buffer.View, modulus, primitiveRoot); buffer.Dispose();
} accelerator.Dispose();
finally
{
buffer.Dispose();
}
}
finally
{
gpu.Dispose();
}
// Verify that the square cache now holds entries for the accelerator. // Verify that the square cache now holds entries for the accelerator.
var field = typeof(NttGpuMath).GetField("SquareCache", BindingFlags.NonPublic | BindingFlags.Static)!; var field = typeof(NttGpuMath).GetField("SquareCache", BindingFlags.NonPublic | BindingFlags.Static)!;
var cacheObj = field.GetValue(null)!; var cacheObj = field.GetValue(null)!;
int count = (int)cacheObj.GetType().GetProperty("Count")!.GetValue(cacheObj)!; int count = (int)cacheObj.GetType().GetProperty("Count")!.GetValue(cacheObj)!;
count.Should().BeGreaterThan(0); count.Should().BeGreaterThan(0);
GpuContextPool.DisposeAll(); count = (int)cacheObj.GetType().GetProperty("Count")!.GetValue(cacheObj)!;
count.Should().Be(0);
count = (int)cacheObj.GetType().GetProperty("Count")!.GetValue(cacheObj)!; }
count.Should().Be(0);
}
} }
@@ -9,86 +9,83 @@ namespace PerfectNumbers.Core.Tests.Gpu;
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public class NttMontgomeryTests public class NttMontgomeryTests
{ {
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
public void SquareDevice_montgomery64_matches_cpu_square_on_small_input() public void SquareDevice_montgomery64_matches_cpu_square_on_small_input()
{ {
// Small 64-bit modulus and tiny input to keep runtime trivial. // Small 64-bit modulus and tiny input to keep runtime trivial.
var modulus = new GpuUInt128(0UL, 17UL); var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL); var primitiveRoot = new GpuUInt128(0UL, 3UL);
Span<GpuUInt128> cpu = stackalloc GpuUInt128[8]; Span<GpuUInt128> cpu = stackalloc GpuUInt128[8];
for (ulong i = 0; i < (ulong)cpu.Length; i++) for (ulong i = 0; i < (ulong)cpu.Length; i++)
{ {
cpu[(int)i] = new GpuUInt128(0UL, i + 1UL); cpu[(int)i] = new GpuUInt128(0UL, i + 1UL);
} }
Span<GpuUInt128> gpu = stackalloc GpuUInt128[16]; Span<GpuUInt128> gpu = stackalloc GpuUInt128[16];
cpu.CopyTo(gpu); cpu.CopyTo(gpu);
for (int i = cpu.Length; i < gpu.Length; i++) for (int i = cpu.Length; i < gpu.Length; i++)
{ {
gpu[i] = new GpuUInt128(0UL, 0UL); gpu[i] = new GpuUInt128(0UL, 0UL);
} }
// CPU reference // CPU reference
NttGpuMath.Square(cpu, modulus, primitiveRoot); NttGpuMath.Square(cpu, modulus, primitiveRoot);
// Device execution // Device execution
GpuContextPool.GpuContextLease context = GpuContextPool.Rent(); var accelerator = SharedGpuContext.CreateAccelerator();
try var stream = accelerator.CreateStream();
{ using var buffer = accelerator.Allocate1D<GpuUInt128>(gpu.Length);
var accelerator = context.Accelerator; buffer.View.CopyFromCPU(stream, ref gpu[0], gpu.Length);
using var buffer = accelerator.Allocate1D<GpuUInt128>(gpu.Length); NttGpuMath.SquareDevice(accelerator, stream, buffer.View, modulus, primitiveRoot);
buffer.View.CopyFromCPU(ref gpu[0], gpu.Length); buffer.View.CopyToCPU(stream, ref gpu[0], gpu.Length);
NttGpuMath.SquareDevice(accelerator, buffer.View, modulus, primitiveRoot); stream.Synchronize();
buffer.View.CopyToCPU(ref gpu[0], gpu.Length); stream.Dispose();
accelerator.Dispose();
for (int i = 0; i < cpu.Length; i++) for (int i = 0; i < cpu.Length; i++)
{ {
gpu[i].Should().Be(cpu[i]); gpu[i].Should().Be(cpu[i]);
} }
}
finally
{
context.Dispose();
}
}
[Fact] }
[Trait("Category", "Fast")]
public void InverseDevice_montgomery64_matches_cpu_inverse_on_small_input()
{
var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL);
Span<GpuUInt128> input = stackalloc GpuUInt128[8]; [Fact]
for (ulong i = 0; i < (ulong)input.Length; i++) [Trait("Category", "Fast")]
{ public void InverseDevice_montgomery64_matches_cpu_inverse_on_small_input()
input[(int)i] = new GpuUInt128(0UL, i + 1UL); {
} var modulus = new GpuUInt128(0UL, 17UL);
var primitiveRoot = new GpuUInt128(0UL, 3UL);
Span<GpuUInt128> transformed = stackalloc GpuUInt128[input.Length]; Span<GpuUInt128> input = stackalloc GpuUInt128[8];
input.CopyTo(transformed); for (ulong i = 0; i < (ulong)input.Length; i++)
NttGpuMath.Forward(transformed, modulus, primitiveRoot); {
input[(int)i] = new GpuUInt128(0UL, i + 1UL);
}
Span<GpuUInt128> cpu = stackalloc GpuUInt128[input.Length]; Span<GpuUInt128> transformed = stackalloc GpuUInt128[input.Length];
transformed.CopyTo(cpu); input.CopyTo(transformed);
NttGpuMath.Inverse(cpu, modulus, primitiveRoot); NttGpuMath.Forward(transformed, modulus, primitiveRoot);
var original = NttGpuMath.GpuTransformBackend; Span<GpuUInt128> cpu = stackalloc GpuUInt128[input.Length];
try transformed.CopyTo(cpu);
{ NttGpuMath.Inverse(cpu, modulus, primitiveRoot);
NttGpuMath.GpuTransformBackend = NttBackend.Staged;
Span<GpuUInt128> gpu = stackalloc GpuUInt128[input.Length]; var original = NttGpuMath.GpuTransformBackend;
transformed.CopyTo(gpu); try
NttGpuMath.InverseGpu(gpu, modulus, primitiveRoot); {
for (int i = 0; i < input.Length; i++) NttGpuMath.GpuTransformBackend = NttBackend.Staged;
{ Span<GpuUInt128> gpu = stackalloc GpuUInt128[input.Length];
gpu[i].Should().Be(cpu[i]); transformed.CopyTo(gpu);
} NttGpuMath.InverseGpu(gpu, modulus, primitiveRoot);
} for (int i = 0; i < input.Length; i++)
finally {
{ gpu[i].Should().Be(cpu[i]);
NttGpuMath.GpuTransformBackend = original; }
} }
} finally
{
NttGpuMath.GpuTransformBackend = original;
}
}
} }
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core.Gpu;
using Xunit; using Xunit;
namespace PerfectNumbers.Core.Tests.Gpu; namespace PerfectNumbers.Core.Tests.Gpu;
@@ -10,37 +9,23 @@ public class PrimeTesterGpuTests
[Fact] [Fact]
public void IsPrimeGpu_accepts_known_primes() public void IsPrimeGpu_accepts_known_primes()
{ {
try var tester = new PrimeTester();
{ ulong[] primes = [31UL, 61UL, 89UL, 107UL, 127UL, 521UL];
var tester = new PrimeTester(); foreach (ulong prime in primes)
ulong[] primes = [31UL, 61UL, 89UL, 107UL, 127UL, 521UL]; {
foreach (ulong prime in primes) PrimeTester.IsPrimeGpu(prime).Should().BeTrue();
{ }
PrimeTester.IsPrimeGpu(prime).Should().BeTrue();
}
}
finally
{
GpuContextPool.DisposeAll();
}
} }
[Fact] [Fact]
public void IsPrimeGpu_rejects_composites() public void IsPrimeGpu_rejects_composites()
{ {
try var tester = new PrimeTester();
{ ulong[] composites = [33UL, 39UL, 51UL, 77UL, 91UL];
var tester = new PrimeTester(); foreach (ulong composite in composites)
ulong[] composites = [33UL, 39UL, 51UL, 77UL, 91UL]; {
foreach (ulong composite in composites) PrimeTester.IsPrimeGpu(composite).Should().BeFalse();
{ }
PrimeTester.IsPrimeGpu(composite).Should().BeFalse();
}
}
finally
{
GpuContextPool.DisposeAll();
}
} }
} }
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using Xunit; using Xunit;
namespace PerfectNumbers.Core.Tests.Gpu; namespace PerfectNumbers.Core.Tests.Gpu;
@@ -1,7 +1,3 @@
using System;
using System.Numerics;
using System.Collections.Generic;
using PerfectNumbers.Core;
using FluentAssertions; using FluentAssertions;
using Xunit; using Xunit;
@@ -11,7 +11,7 @@ public class MersenneNumberDivisorByDivisorTesterTests
{ {
var candidates = new List<ulong> { 5UL, 7UL, 11UL, 13UL }; var candidates = new List<ulong> { 5UL, 7UL, 11UL, 13UL };
var tester = new FakeTester(); var tester = new FakeTester();
var results = new List<(ulong Prime, bool Searched, bool Detailed, bool Passed)>(); var results = new List<(ulong Prime, bool Searched, bool Detailed, bool Passed, ulong Divisor)>();
int compositeMarks = 0; int compositeMarks = 0;
int clearedMarks = 0; int clearedMarks = 0;
@@ -22,7 +22,7 @@ public class MersenneNumberDivisorByDivisorTesterTests
startPrime: 0UL, startPrime: 0UL,
markComposite: () => compositeMarks++, markComposite: () => compositeMarks++,
clearComposite: () => clearedMarks++, clearComposite: () => clearedMarks++,
printResult: (prime, searched, detailed, passed) => results.Add((prime, searched, detailed, passed)), printResult: (prime, searched, detailed, passed, divisor) => results.Add((prime, searched, detailed, passed, divisor)),
threadCount: 1, threadCount: 1,
primesPerTask: 2); primesPerTask: 2);
@@ -32,10 +32,10 @@ public class MersenneNumberDivisorByDivisorTesterTests
results.Should().Equal( results.Should().Equal(
new[] new[]
{ {
(5UL, true, true, true), (5UL, true, true, true, 0UL),
(7UL, true, true, false), (7UL, true, true, false, 7UL),
(11UL, true, true, false), (11UL, true, true, false, 11UL),
(13UL, true, true, true), (13UL, true, true, true, 0UL),
}); });
} }
@@ -57,10 +57,23 @@ public class MersenneNumberDivisorByDivisorTesterTests
return DivisorLimit; return DivisorLimit;
} }
public bool IsPrime(ulong prime, out bool divisorsExhausted) public bool IsPrime(ulong prime, out bool divisorsExhausted, out ulong divisor)
{ {
divisorsExhausted = true; divisorsExhausted = true;
return prime % 7UL != 0UL && prime % 11UL != 0UL; if (prime % 7UL == 0UL)
{
divisor = 7UL;
return false;
}
if (prime % 11UL == 0UL)
{
divisor = 11UL;
return false;
}
divisor = 0UL;
return true;
} }
public IMersenneNumberDivisorByDivisorTester.IDivisorScanSession CreateDivisorSession() public IMersenneNumberDivisorByDivisorTester.IDivisorScanSession CreateDivisorSession()
@@ -83,7 +96,7 @@ public class MersenneNumberDivisorByDivisorTesterTests
throw new NotSupportedException(); throw new NotSupportedException();
} }
public void Dispose() public void Return()
{ {
} }
} }
@@ -13,14 +13,17 @@ public class MersenneNumberDivisorCpuTesterTests
var tester = new MersenneNumberDivisorByDivisorCpuTester(); var tester = new MersenneNumberDivisorByDivisorCpuTester();
tester.ConfigureFromMaxPrime(11UL); tester.ConfigureFromMaxPrime(11UL);
tester.IsPrime(5UL, out bool divisorsExhausted).Should().BeTrue(); tester.IsPrime(5UL, out bool divisorsExhausted, out ulong divisor).Should().BeTrue();
divisorsExhausted.Should().BeTrue(); divisorsExhausted.Should().BeTrue();
divisor.Should().Be(0UL);
tester.IsPrime(7UL, out divisorsExhausted).Should().BeTrue(); tester.IsPrime(7UL, out divisorsExhausted, out divisor).Should().BeTrue();
divisorsExhausted.Should().BeTrue(); divisorsExhausted.Should().BeTrue();
divisor.Should().Be(0UL);
tester.IsPrime(11UL, out divisorsExhausted).Should().BeFalse(); tester.IsPrime(11UL, out divisorsExhausted, out divisor).Should().BeFalse();
divisorsExhausted.Should().BeTrue(); divisorsExhausted.Should().BeTrue();
divisor.Should().BeGreaterThan(0UL);
} }
[Fact] [Fact]
@@ -47,7 +50,7 @@ public class MersenneNumberDivisorCpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -72,7 +75,7 @@ public class MersenneNumberDivisorCpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -97,7 +100,7 @@ public class MersenneNumberDivisorCpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
} }
@@ -73,14 +73,17 @@ public class MersenneNumberDivisorGpuTesterTests
var tester = new MersenneNumberDivisorByDivisorGpuTester(); var tester = new MersenneNumberDivisorByDivisorGpuTester();
tester.ConfigureFromMaxPrime(43UL); tester.ConfigureFromMaxPrime(43UL);
tester.IsPrime(31UL, out bool exhausted).Should().BeTrue(); tester.IsPrime(31UL, out bool exhausted, out ulong divisor).Should().BeTrue();
exhausted.Should().BeTrue(); exhausted.Should().BeTrue();
divisor.Should().Be(0UL);
tester.IsPrime(37UL, out exhausted).Should().BeFalse(); tester.IsPrime(37UL, out exhausted, out divisor).Should().BeFalse();
exhausted.Should().BeTrue(); exhausted.Should().BeTrue();
divisor.Should().BeGreaterThan(0UL);
tester.IsPrime(43UL, out exhausted).Should().BeFalse(); tester.IsPrime(43UL, out exhausted, out divisor).Should().BeFalse();
exhausted.Should().BeTrue(); exhausted.Should().BeTrue();
divisor.Should().BeGreaterThan(0UL);
} }
[Fact] [Fact]
@@ -105,12 +108,15 @@ public class MersenneNumberDivisorGpuTesterTests
var tester = new MersenneNumberDivisorByDivisorGpuTester(); var tester = new MersenneNumberDivisorByDivisorGpuTester();
tester.ConfigureFromMaxPrime(41UL); tester.ConfigureFromMaxPrime(41UL);
tester.IsPrime(31UL, out _).Should().BeTrue(); tester.IsPrime(31UL, out _, out ulong divisor).Should().BeTrue();
tester.IsPrime(37UL, out bool exhausted).Should().BeFalse(); divisor.Should().Be(0UL);
tester.IsPrime(37UL, out bool exhausted, out divisor).Should().BeFalse();
exhausted.Should().BeTrue(); exhausted.Should().BeTrue();
divisor.Should().BeGreaterThan(0UL);
tester.IsPrime(41UL, out bool divisorsExhausted).Should().BeFalse(); tester.IsPrime(41UL, out bool divisorsExhausted, out divisor).Should().BeFalse();
divisorsExhausted.Should().BeTrue(); divisorsExhausted.Should().BeTrue();
divisor.Should().BeGreaterThan(0UL);
} }
@@ -124,8 +130,16 @@ public class MersenneNumberDivisorGpuTesterTests
var tester = new MersenneNumberDivisorByDivisorGpuTester(); var tester = new MersenneNumberDivisorByDivisorGpuTester();
tester.ConfigureFromMaxPrime(exponent); tester.ConfigureFromMaxPrime(exponent);
tester.IsPrime(exponent, out bool exhausted).Should().Be(expectedPrime); tester.IsPrime(exponent, out bool exhausted, out ulong divisor).Should().Be(expectedPrime);
exhausted.Should().BeTrue(); exhausted.Should().BeTrue();
if (expectedPrime)
{
divisor.Should().Be(0UL);
}
else
{
divisor.Should().BeGreaterThan(0UL);
}
} }
[Fact] [Fact]
[Trait("Category", "Fast")] [Trait("Category", "Fast")]
@@ -138,8 +152,9 @@ public class MersenneNumberDivisorGpuTesterTests
tester.ConfigureFromMaxPrime(43UL); tester.ConfigureFromMaxPrime(43UL);
tester.IsPrime(41UL, out bool divisorsExhausted).Should().BeFalse(); tester.IsPrime(41UL, out bool divisorsExhausted, out ulong divisor).Should().BeFalse();
divisorsExhausted.Should().BeTrue(); divisorsExhausted.Should().BeTrue();
divisor.Should().BeGreaterThan(0UL);
} }
[Fact] [Fact]
@@ -161,7 +176,7 @@ public class MersenneNumberDivisorGpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -189,7 +204,7 @@ public class MersenneNumberDivisorGpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -217,7 +232,7 @@ public class MersenneNumberDivisorGpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -246,7 +261,7 @@ public class MersenneNumberDivisorGpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -275,7 +290,7 @@ public class MersenneNumberDivisorGpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
@@ -309,15 +324,15 @@ public class MersenneNumberDivisorGpuTesterTests
.GetField("_divisorLimit", BindingFlags.NonPublic | BindingFlags.Instance)! .GetField("_divisorLimit", BindingFlags.NonPublic | BindingFlags.Instance)!
.SetValue(tester, 200UL); .SetValue(tester, 200UL);
tester.IsPrime(19UL, out bool divisorsExhausted).Should().BeTrue(); tester.IsPrime(19UL, out bool divisorsExhausted, out ulong divisor).Should().BeTrue();
divisorsExhausted.Should().BeTrue(); divisorsExhausted.Should().BeTrue();
divisor.Should().Be(0UL);
} }
finally finally
{ {
tableField.SetValue(cycles, originalTable); tableField.SetValue(cycles, originalTable);
smallCyclesField.SetValue(cycles, originalSmall); smallCyclesField.SetValue(cycles, originalSmall);
DivisorCycleCache.Shared.RefreshSnapshot(); DivisorCycleCache.Shared.RefreshSnapshot();
GpuContextPool.DisposeAll();
} }
} }
@@ -345,7 +360,7 @@ public class MersenneNumberDivisorGpuTesterTests
} }
finally finally
{ {
session.Dispose(); session.Return();
} }
} }
} }
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Cpu; using PerfectNumbers.Core.Cpu;
using Xunit; using Xunit;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using Xunit; using Xunit;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Cpu; using PerfectNumbers.Core.Cpu;
using Xunit; using Xunit;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
@@ -1,6 +1,5 @@
using System.Reflection; using System.Reflection;
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using Xunit; using Xunit;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
@@ -55,7 +54,6 @@ public class MersenneNumberOrderGpuTesterTests
ulong[]? backupSmall = originalSmall is null ? null : (ulong[])originalSmall.Clone(); ulong[]? backupSmall = originalSmall is null ? null : (ulong[])originalSmall.Clone();
cycles.LoadFrom(tempPath); cycles.LoadFrom(tempPath);
GpuContextPool.DisposeAll();
try try
{ {
@@ -75,7 +73,6 @@ public class MersenneNumberOrderGpuTesterTests
{ {
tableField.SetValue(cycles, originalTable); tableField.SetValue(cycles, originalTable);
smallCyclesField.SetValue(cycles, backupSmall); smallCyclesField.SetValue(cycles, backupSmall);
GpuContextPool.DisposeAll();
} }
} }
finally finally
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Cpu; using PerfectNumbers.Core.Cpu;
using Xunit; using Xunit;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
@@ -1,5 +1,4 @@
using FluentAssertions; using FluentAssertions;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using Xunit; using Xunit;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
@@ -1,6 +1,6 @@
namespace PerfectNumbers.Core; namespace PerfectNumbers.Core.Cpu;
internal static class DeterministicRandom internal static class DeterministicRandomCpu
{ {
[ThreadStatic] [ThreadStatic]
private static ulong s_threadState; private static ulong s_threadState;
@@ -62,7 +62,7 @@ internal sealed class MersenneCpuDivisorScanSession : IMersenneNumberDivisorByDi
} }
} }
public void Dispose() public void Return()
{ {
ThreadStaticPools.ReturnMersenneCpuDivisorSession(this); ThreadStaticPools.ReturnMersenneCpuDivisorSession(this);
} }
@@ -1,487 +1,510 @@
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using PerfectNumbers.Core;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using PerfectNumbers.Core.Gpu.Accelerators;
namespace PerfectNumbers.Core.Cpu; namespace PerfectNumbers.Core.Cpu;
public sealed class MersenneNumberDivisorByDivisorCpuTester : IMersenneNumberDivisorByDivisorTester public sealed class MersenneNumberDivisorByDivisorCpuTester : IMersenneNumberDivisorByDivisorTester
{ {
private ulong _divisorLimit; private ulong _divisorLimit;
private int _batchSize = 1_024; private int _batchSize = 1_024;
[ThreadStatic] [ThreadStatic]
private static GpuUInt128WorkSet _divisorScanGpuWorkSet; private static GpuUInt128WorkSet _divisorScanGpuWorkSet;
private struct GpuUInt128WorkSet private struct GpuUInt128WorkSet
{ {
public GpuUInt128 Step; public GpuUInt128 Step;
public GpuUInt128 Divisor; public GpuUInt128 Divisor;
public GpuUInt128 Limit; public GpuUInt128 Limit;
} }
public int BatchSize public int BatchSize
{ {
get => _batchSize; get => _batchSize;
set => _batchSize = Math.Max(1, value); set => _batchSize = Math.Max(1, value);
} }
public void ConfigureFromMaxPrime(ulong maxPrime) public void ConfigureFromMaxPrime(ulong maxPrime)
{ {
_divisorLimit = ComputeDivisorLimitFromMaxPrime(maxPrime); _divisorLimit = ComputeDivisorLimitFromMaxPrime(maxPrime);
} }
public ulong DivisorLimit public ulong DivisorLimit
{ {
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
get get
{ {
return _divisorLimit; return _divisorLimit;
} }
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public ulong GetAllowedMaxDivisor(ulong prime) => ComputeAllowedMaxDivisor(prime, _divisorLimit); public ulong GetAllowedMaxDivisor(ulong prime) => ComputeAllowedMaxDivisor(prime, _divisorLimit);
public bool IsPrime(ulong prime, out bool divisorsExhausted) public bool IsPrime(ulong prime, out bool divisorsExhausted, out ulong divisor)
{ {
ulong allowedMax = ComputeAllowedMaxDivisor(prime, _divisorLimit); ulong allowedMax = ComputeAllowedMaxDivisor(prime, _divisorLimit);
// The CPU by-divisor run always hands us primes with enormous divisor limits, so the fallback below never executes. // The CPU by-divisor run always hands us primes with enormous divisor limits, so the fallback below never executes.
// if (allowedMax < 3UL) // if (allowedMax < 3UL)
// { // {
// // EvenPerfectBitScanner routes primes below the small-divisor cutoff to the GPU path, so the CPU path still sees // // EvenPerfectBitScanner routes primes below the small-divisor cutoff to the GPU path, so the CPU path still sees
// // trivial candidates during targeted tests. Short-circuit here to keep those runs aligned with the production flow. // // trivial candidates during targeted tests. Short-circuit here to keep those runs aligned with the production flow.
// divisorsExhausted = true; // divisorsExhausted = true;
// return true; // return true;
// } // }
bool processedAll; bool processedAll;
bool composite = CheckDivisors( bool composite = CheckDivisors(
prime, prime,
allowedMax, allowedMax,
out processedAll); out processedAll,
out divisor);
if (composite)
{ if (composite)
divisorsExhausted = true; {
return false; divisorsExhausted = true;
} return false;
}
divisorsExhausted = processedAll || composite;
return true; divisorsExhausted = processedAll || composite;
} divisor = 0UL;
return true;
public void PrepareCandidates(in ReadOnlySpan<ulong> primes, Span<ulong> allowedMaxValues) }
{
ulong divisorLimit = _divisorLimit; public void PrepareCandidates(in ReadOnlySpan<ulong> primes, Span<ulong> allowedMaxValues)
int length = primes.Length; {
for (int index = 0; index < length; index++) ulong divisorLimit = _divisorLimit;
{ int length = primes.Length;
allowedMaxValues[index] = ComputeAllowedMaxDivisor(primes[index], divisorLimit); for (int index = 0; index < length; index++)
} {
} allowedMaxValues[index] = ComputeAllowedMaxDivisor(primes[index], divisorLimit);
}
public IMersenneNumberDivisorByDivisorTester.IDivisorScanSession CreateDivisorSession() }
{
MersenneCpuDivisorScanSession? session = ThreadStaticPools.RentMersenneCpuDivisorSession(); public IMersenneNumberDivisorByDivisorTester.IDivisorScanSession CreateDivisorSession()
if (session is not null) {
{ MersenneCpuDivisorScanSession? session = ThreadStaticPools.RentMersenneCpuDivisorSession();
return session; if (session is not null)
} {
return session;
return new MersenneCpuDivisorScanSession(); }
}
return new MersenneCpuDivisorScanSession();
private bool CheckDivisors( }
ulong prime,
ulong allowedMax, private bool CheckDivisors(
out bool processedAll) ulong prime,
{ ulong allowedMax,
processedAll = false; out bool processedAll,
out ulong foundDivisor)
// The EvenPerfectBitScanner feeds primes >= 138,000,000 here, so allowedMax >= 3 in production runs. {
// Keeping the guard commented out documents the reasoning for benchmarks and tests. foundDivisor = 0UL;
// if (allowedMax < 3UL) // The EvenPerfectBitScanner feeds primes >= 138,000,000 here, so allowedMax >= 3 in production runs.
// { // Keeping the guard commented out documents the reasoning for benchmarks and tests.
// return false; // if (allowedMax < 3UL)
// } // {
// return false;
ref GpuUInt128WorkSet workSet = ref _divisorScanGpuWorkSet; // }
ref GpuUInt128 step = ref workSet.Step; ref GpuUInt128WorkSet workSet = ref _divisorScanGpuWorkSet;
step.High = 0UL;
step.Low = prime; ref GpuUInt128 step = ref workSet.Step;
step.ShiftLeft(1); step.High = 0UL;
step.Low = prime;
ref GpuUInt128 limit = ref workSet.Limit; step.ShiftLeft(1);
limit.High = 0UL;
limit.Low = allowedMax; ref GpuUInt128 limit = ref workSet.Limit;
limit.High = 0UL;
ref GpuUInt128 divisor = ref workSet.Divisor; limit.Low = allowedMax;
divisor.High = step.High;
divisor.Low = step.Low; ref GpuUInt128 divisor = ref workSet.Divisor;
divisor.Add(1UL); divisor.High = step.High;
if (divisor.CompareTo(limit) > 0) divisor.Low = step.Low;
{ divisor.Add(1UL);
processedAll = true; if (divisor.CompareTo(limit) > 0)
return false; {
} processedAll = true;
foundDivisor = 0UL;
// Intentionally recomputes factorizations without a per-thread cache. return false;
// The previous factor cache recorded virtually no hits and only slowed down the scan. }
DivisorCycleCache cycleCache = DivisorCycleCache.Shared;
// Intentionally recomputes factorizations without a per-thread cache.
ulong stepHigh = step.High; // The previous factor cache recorded virtually no hits and only slowed down the scan.
ulong stepLow = step.Low; _ = DivisorCycleCache.Shared;
ulong limitHigh = limit.High;
ulong divisorLow = divisor.Low; ulong stepHigh = step.High;
ulong stepLow = step.Low;
// Keep the divisibility filters aligned with the divisor-cycle generator so the ulong limitHigh = limit.High;
// CPU path never requests cycles that were skipped during cache creation. ulong divisorLow = divisor.Low;
LastDigit lastDigit = (prime & 3UL) == 3UL ? LastDigit.Seven : LastDigit.One;
ushort decimalMask = DivisorGenerator.GetDecimalMask(lastDigit); // Keep the divisibility filters aligned with the divisor-cycle generator so the
// CPU path never requests cycles that were skipped during cache creation.
byte step10; LastDigit lastDigit = (prime & 3UL) == 3UL ? LastDigit.Seven : LastDigit.One;
byte step8; ushort decimalMask = DivisorGenerator.GetDecimalMask(lastDigit);
byte step3;
byte step7; byte step10;
byte step11; byte step8;
byte step3;
byte remainder10; byte step7;
byte remainder8; byte step11;
byte remainder3;
byte remainder7; byte remainder10;
byte remainder11; byte remainder8;
byte remainder3;
if (stepHigh == 0UL && limitHigh == 0UL) byte remainder7;
{ byte remainder11;
step10 = (byte)(stepLow % 10UL);
step8 = (byte)(stepLow & 7UL); if (stepHigh == 0UL && limitHigh == 0UL)
step3 = (byte)(stepLow % 3UL); {
step7 = (byte)(stepLow % 7UL); step10 = (byte)(stepLow % 10UL);
step11 = (byte)(stepLow % 11UL); step8 = (byte)(stepLow & 7UL);
step3 = (byte)(stepLow % 3UL);
remainder10 = (byte)(divisorLow % 10UL); step7 = (byte)(stepLow % 7UL);
remainder8 = (byte)(divisorLow & 7UL); step11 = (byte)(stepLow % 11UL);
remainder3 = (byte)(divisorLow % 3UL);
remainder7 = (byte)(divisorLow % 7UL); remainder10 = (byte)(divisorLow % 10UL);
remainder11 = (byte)(divisorLow % 11UL); remainder8 = (byte)(divisorLow & 7UL);
remainder3 = (byte)(divisorLow % 3UL);
return CheckDivisors64( remainder7 = (byte)(divisorLow % 7UL);
prime, remainder11 = (byte)(divisorLow % 11UL);
stepLow,
limit.Low, return CheckDivisors64(
divisorLow, prime,
decimalMask, stepLow,
step10, limit.Low,
step8, divisorLow,
step3, decimalMask,
step7, step10,
step11, step8,
remainder10, step3,
remainder8, step7,
remainder3, step11,
remainder7, remainder10,
remainder11, remainder8,
out processedAll); remainder3,
} remainder7,
remainder11,
ulong divisorHigh = divisor.High; out processedAll,
step10 = (byte)((((stepHigh % 10UL) * 6UL) + (stepLow % 10UL)) % 10UL); out foundDivisor);
step8 = (byte)(stepLow % 8UL); }
step3 = (byte)(((stepHigh % 3UL) + (stepLow % 3UL)) % 3UL);
step7 = (byte)((((stepHigh % 7UL) * 2UL) + (stepLow % 7UL)) % 7UL); ulong divisorHigh = divisor.High;
step11 = (byte)((((stepHigh % 11UL) * 5UL) + (stepLow % 11UL)) % 11UL); step10 = (byte)((((stepHigh % 10UL) * 6UL) + (stepLow % 10UL)) % 10UL);
step8 = (byte)(stepLow % 8UL);
remainder10 = (byte)((((divisorHigh % 10UL) * 6UL) + (divisorLow % 10UL)) % 10UL); step3 = (byte)(((stepHigh % 3UL) + (stepLow % 3UL)) % 3UL);
remainder8 = (byte)(divisorLow % 8UL); step7 = (byte)((((stepHigh % 7UL) * 2UL) + (stepLow % 7UL)) % 7UL);
remainder3 = (byte)(((divisorHigh % 3UL) + (divisorLow % 3UL)) % 3UL); step11 = (byte)((((stepHigh % 11UL) * 5UL) + (stepLow % 11UL)) % 11UL);
remainder7 = (byte)((((divisorHigh % 7UL) * 2UL) + (divisorLow % 7UL)) % 7UL);
remainder11 = (byte)((((divisorHigh % 11UL) * 5UL) + (divisorLow % 11UL)) % 11UL); remainder10 = (byte)((((divisorHigh % 10UL) * 6UL) + (divisorLow % 10UL)) % 10UL);
remainder8 = (byte)(divisorLow % 8UL);
while (divisor.CompareTo(limit) <= 0) remainder3 = (byte)(((divisorHigh % 3UL) + (divisorLow % 3UL)) % 3UL);
{ remainder7 = (byte)((((divisorHigh % 7UL) * 2UL) + (divisorLow % 7UL)) % 7UL);
bool passesSmallModuli = remainder3 != 0 && remainder7 != 0 && remainder11 != 0; remainder11 = (byte)((((divisorHigh % 11UL) * 5UL) + (divisorLow % 11UL)) % 11UL);
if (passesSmallModuli && (remainder8 == 1 || remainder8 == 7) && ((decimalMask >> remainder10) & 1) != 0)
{ var gpu = PrimeOrderCalculatorAccelerator.Rent(1);
ulong candidate = divisor.Low;
MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(candidate); while (divisor.CompareTo(limit) <= 0)
ulong divisorCycle; {
// Divisors generated from 2 * k * p + 1 exceed the small-cycle snapshot when p >= 138,000,000, so the short path below never runs. bool passesSmallModuli = remainder3 != 0 && remainder7 != 0 && remainder11 != 0;
// if (candidate <= PerfectNumberConstants.MaxQForDivisorCycles) if (passesSmallModuli && (remainder8 == 1 || remainder8 == 7) && ((decimalMask >> remainder10) & 1) != 0)
// { {
// divisorCycle = cycleCache.GetCycleLength(candidate); ulong candidate = divisor.Low;
// } MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(candidate);
// else ulong divisorCycle;
{ // Divisors generated from 2 * k * p + 1 exceed the small-cycle snapshot when p >= 138,000,000, so the short path below never runs.
if (!MersenneDivisorCycles.TryCalculateCycleLengthForExponentCpu( // if (candidate <= PerfectNumberConstants.MaxQForDivisorCycles)
candidate, // {
prime, // divisorCycle = cycleCache.GetCycleLength(candidate);
divisorData, // }
out ulong computedCycle, // else
out bool primeOrderFailed) || computedCycle == 0UL) {
{ if (!MersenneDivisorCycles.TryCalculateCycleLengthForExponentCpu(
// Divisors produced by 2 * k * p + 1 always exceed PerfectNumberConstants.MaxQForDivisorCycles gpu,
// for the exponents scanned here, so skip the unused cache fallback and compute directly. candidate,
divisorCycle = MersenneDivisorCycles.CalculateCycleLength( prime,
candidate, divisorData,
divisorData, out ulong computedCycle,
skipPrimeOrderHeuristic: primeOrderFailed); out bool primeOrderFailed) || computedCycle == 0UL)
} {
else // Divisors produced by 2 * k * p + 1 always exceed PerfectNumberConstants.MaxQForDivisorCycles
{ // for the exponents scanned here, so skip the unused cache fallback and compute directly.
divisorCycle = computedCycle; divisorCycle = MersenneDivisorCycles.CalculateCycleLength(
} candidate,
} divisorData,
skipPrimeOrderHeuristic: primeOrderFailed);
if (divisorCycle == prime) }
{ else
// A cycle equal to the tested exponent (which is prime in this path) guarantees that the candidate divides {
// the corresponding Mersenne number because the order of 2 modulo the divisor is exactly p. divisorCycle = computedCycle;
processedAll = true; }
return true; }
}
if (divisorCycle == prime)
if (divisorCycle == 0UL) {
{ // A cycle equal to the tested exponent (which is prime in this path) guarantees that the candidate divides
Console.WriteLine($"Divisor cycle was not calculated for {prime}"); // the corresponding Mersenne number because the order of 2 modulo the divisor is exactly p.
} foundDivisor = candidate;
} processedAll = true;
PrimeOrderCalculatorAccelerator.Return(gpu);
divisor.Add(step); return true;
remainder10 = AddMod10(remainder10, step10); }
remainder8 = AddMod8(remainder8, step8);
remainder3 = AddMod3(remainder3, step3); if (divisorCycle == 0UL)
remainder7 = AddMod7(remainder7, step7); {
remainder11 = AddMod11(remainder11, step11); Console.WriteLine($"Divisor cycle was not calculated for {prime}");
} }
processedAll = true; }
return false;
} divisor.Add(step);
remainder10 = AddMod10(remainder10, step10);
private static bool CheckDivisors64( remainder8 = AddMod8(remainder8, step8);
ulong prime, remainder3 = AddMod3(remainder3, step3);
ulong step, remainder7 = AddMod7(remainder7, step7);
ulong limit, remainder11 = AddMod11(remainder11, step11);
ulong divisor, }
ushort decimalMask,
byte step10, PrimeOrderCalculatorAccelerator.Return(gpu);
byte step8, processedAll = true;
byte step3, foundDivisor = 0UL;
byte step7, return false;
byte step11, }
byte remainder10,
byte remainder8, private static bool CheckDivisors64(
byte remainder3, ulong prime,
byte remainder7, ulong step,
byte remainder11, ulong limit,
out bool processedAll) ulong divisor,
{ ushort decimalMask,
processedAll = false; byte step10,
byte step8,
bool canAdvance = step <= limit; byte step3,
if (!canAdvance) byte step7,
{ byte step11,
if (divisor <= limit) byte remainder10,
{ byte remainder8,
if (remainder3 != 0 && remainder7 != 0 && remainder11 != 0 && (remainder8 == 1 || remainder8 == 7) && ((decimalMask >> remainder10) & 1) != 0) byte remainder3,
{ byte remainder7,
MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor); byte remainder11,
ulong divisorCycle; out bool processedAll,
if (!MersenneDivisorCycles.TryCalculateCycleLengthForExponentCpu( out ulong foundDivisor)
divisor, {
prime, processedAll = false;
divisorData, foundDivisor = 0UL;
out ulong computedCycle,
out bool primeOrderFailed) || computedCycle == 0UL) bool canAdvance = step <= limit;
{ var gpu = PrimeOrderCalculatorAccelerator.Rent(1);
divisorCycle = MersenneDivisorCycles.CalculateCycleLength( if (!canAdvance)
divisor, {
divisorData, if (divisor <= limit)
skipPrimeOrderHeuristic: primeOrderFailed); {
} if (remainder3 != 0 && remainder7 != 0 && remainder11 != 0 && (remainder8 == 1 || remainder8 == 7) && ((decimalMask >> remainder10) & 1) != 0)
else {
{ MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor);
divisorCycle = computedCycle; ulong divisorCycle;
} if (!MersenneDivisorCycles.TryCalculateCycleLengthForExponentCpu(
gpu,
if (divisorCycle == prime) divisor,
{ prime,
processedAll = true; divisorData,
return true; out ulong computedCycle,
} out bool primeOrderFailed) || computedCycle == 0UL)
{
if (divisorCycle == 0UL) divisorCycle = MersenneDivisorCycles.CalculateCycleLength(
{ divisor,
Console.WriteLine($"Divisor cycle was not calculated for {prime}"); divisorData,
} skipPrimeOrderHeuristic: primeOrderFailed);
} }
} else
{
processedAll = true; divisorCycle = computedCycle;
return false; }
}
if (divisorCycle == prime)
if (divisor > limit) {
{ foundDivisor = divisor;
processedAll = true; processedAll = true;
return false; PrimeOrderCalculatorAccelerator.Return(gpu);
} return true;
}
ulong remainingIterations = ((limit - divisor) / step) + 1UL;
if (divisorCycle == 0UL)
while (true) {
{ Console.WriteLine($"Divisor cycle was not calculated for {prime}");
bool passesSmallModuli = remainder3 != 0 && remainder7 != 0 && remainder11 != 0; }
if (passesSmallModuli && (remainder8 == 1 || remainder8 == 7) && ((decimalMask >> remainder10) & 1) != 0) }
{ }
MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor);
ulong divisorCycle; processedAll = true;
if (!MersenneDivisorCycles.TryCalculateCycleLengthForExponentCpu( foundDivisor = 0UL;
divisor, return false;
prime, }
divisorData,
out ulong computedCycle, if (divisor > limit)
out bool primeOrderFailed) || computedCycle == 0UL) {
{ processedAll = true;
divisorCycle = MersenneDivisorCycles.CalculateCycleLength( foundDivisor = 0UL;
divisor, return false;
divisorData, }
skipPrimeOrderHeuristic: primeOrderFailed);
} ulong remainingIterations = ((limit - divisor) / step) + 1UL;
else
{ while (true)
divisorCycle = computedCycle; {
} bool passesSmallModuli = remainder3 != 0 && remainder7 != 0 && remainder11 != 0;
if (passesSmallModuli && (remainder8 == 1 || remainder8 == 7) && ((decimalMask >> remainder10) & 1) != 0)
if (divisorCycle == prime) {
{ MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor);
processedAll = true; ulong divisorCycle;
return true; if (!MersenneDivisorCycles.TryCalculateCycleLengthForExponentCpu(
} gpu,
divisor,
if (divisorCycle == 0UL) prime,
{ divisorData,
Console.WriteLine($"Divisor cycle was not calculated for {prime}"); out ulong computedCycle,
} out bool primeOrderFailed) || computedCycle == 0UL)
} {
divisorCycle = MersenneDivisorCycles.CalculateCycleLength(
remainingIterations--; divisor,
if (remainingIterations == 0UL) divisorData,
{ skipPrimeOrderHeuristic: primeOrderFailed);
processedAll = true; }
return false; else
} {
divisorCycle = computedCycle;
divisor += step; }
remainder10 = AddMod10(remainder10, step10);
remainder8 = AddMod8(remainder8, step8); if (divisorCycle == prime)
remainder3 = AddMod3(remainder3, step3); {
remainder7 = AddMod7(remainder7, step7); foundDivisor = divisor;
remainder11 = AddMod11(remainder11, step11); processedAll = true;
} PrimeOrderCalculatorAccelerator.Return(gpu);
} return true;
}
private static byte CheckDivisor(ulong prime, ulong divisorCycle, in MontgomeryDivisorData divisorData)
{ if (divisorCycle == 0UL)
ulong residue = prime.Pow2MontgomeryModWithCycleCpu(divisorCycle, divisorData); {
return residue == 1UL ? (byte)1 : (byte)0; Console.WriteLine($"Divisor cycle was not calculated for {prime}");
} }
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static byte AddMod3(byte value, byte delta) remainingIterations--;
{ if (remainingIterations == 0UL)
const int Modulus = 3; {
int sum = value + delta; processedAll = true;
foundDivisor = 0UL;
if (sum >= Modulus) return false;
{ }
sum -= Modulus;
} divisor += step;
remainder10 = AddMod10(remainder10, step10);
return (byte)sum; remainder8 = AddMod8(remainder8, step8);
} remainder3 = AddMod3(remainder3, step3);
remainder7 = AddMod7(remainder7, step7);
remainder11 = AddMod11(remainder11, step11);
[MethodImpl(MethodImplOptions.AggressiveInlining)] }
private static byte AddMod7(byte value, byte delta) }
{
const int Modulus = 7; private static byte CheckDivisor(ulong prime, ulong divisorCycle, in MontgomeryDivisorData divisorData)
int sum = value + delta; {
ulong residue = prime.Pow2MontgomeryModWithCycleCpu(divisorCycle, divisorData);
if (sum >= Modulus) return residue == 1UL ? (byte)1 : (byte)0;
{ }
sum -= Modulus;
} [MethodImpl(MethodImplOptions.AggressiveInlining)]
private static byte AddMod3(byte value, byte delta)
return (byte)sum; {
} const int Modulus = 3;
int sum = value + delta;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static byte AddMod10(byte value, byte delta) if (sum >= Modulus)
{ {
const int Modulus = 10; sum -= Modulus;
int sum = value + delta; }
if (sum >= Modulus) return (byte)sum;
{ }
sum -= Modulus;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
return (byte)sum; private static byte AddMod7(byte value, byte delta)
} {
const int Modulus = 7;
[MethodImpl(MethodImplOptions.AggressiveInlining)] int sum = value + delta;
private static byte AddMod11(byte value, byte delta)
{ if (sum >= Modulus)
const int Modulus = 11; {
int sum = value + delta; sum -= Modulus;
}
if (sum >= Modulus)
{ return (byte)sum;
sum -= Modulus; }
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
return (byte)sum; private static byte AddMod10(byte value, byte delta)
} {
const int Modulus = 10;
[MethodImpl(MethodImplOptions.AggressiveInlining)] int sum = value + delta;
private static byte AddMod8(byte value, byte delta)
{ if (sum >= Modulus)
return (byte)((value + delta) & 7); {
} sum -= Modulus;
}
private static ulong ComputeDivisorLimitFromMaxPrime(ulong maxPrime)
{ return (byte)sum;
// The by-divisor CPU configuration only supplies primes greater than 1, so the guard below never trips. }
// if (maxPrime <= 1UL)
// { [MethodImpl(MethodImplOptions.AggressiveInlining)]
// return 0UL; private static byte AddMod11(byte value, byte delta)
// } {
if (maxPrime - 1UL >= 64UL) const int Modulus = 11;
{ int sum = value + delta;
return ulong.MaxValue;
} if (sum >= Modulus)
{
return (1UL << (int)(maxPrime - 1UL)) - 1UL; sum -= Modulus;
} }
private static ulong ComputeAllowedMaxDivisor(ulong prime, ulong divisorLimit) return (byte)sum;
{ }
// Production by-divisor scans only handle primes, so inputs never fall below 2.
// if (prime <= 1UL) [MethodImpl(MethodImplOptions.AggressiveInlining)]
// { private static byte AddMod8(byte value, byte delta)
// return 0UL; {
// } return (byte)((value + delta) & 7);
if (prime - 1UL >= 64UL) }
{
return divisorLimit; private static ulong ComputeDivisorLimitFromMaxPrime(ulong maxPrime)
} {
// The by-divisor CPU configuration only supplies primes greater than 1, so the guard below never trips.
return Math.Min((1UL << (int)(prime - 1UL)) - 1UL, divisorLimit); // if (maxPrime <= 1UL)
} // {
// return 0UL;
// }
if (maxPrime - 1UL >= 64UL)
{
return ulong.MaxValue;
}
return (1UL << (int)(maxPrime - 1UL)) - 1UL;
}
private static ulong ComputeAllowedMaxDivisor(ulong prime, ulong divisorLimit)
{
// Production by-divisor scans only handle primes, so inputs never fall below 2.
// if (prime <= 1UL)
// {
// return 0UL;
// }
if (prime - 1UL >= 64UL)
{
return divisorLimit;
}
return Math.Min((1UL << (int)(prime - 1UL)) - 1UL, divisorLimit);
}
} }
+315 -267
View File
@@ -1,329 +1,377 @@
using System.Buffers;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
using PerfectNumbers.Core.Gpu; using PerfectNumbers.Core.Gpu;
using PerfectNumbers.Core.Gpu.Accelerators;
namespace PerfectNumbers.Core; namespace PerfectNumbers.Core;
public sealed class DivisorCycleCache public sealed class DivisorCycleCache
{ {
private const byte ByteZero = 0; private const byte ByteZero = 0;
private const byte ByteOne = 1; private const byte ByteOne = 1;
private const int StackBufferThreshold = 128; private const int StackBufferThreshold = 128;
private readonly Accelerator _accelerator;
private readonly int _acceleratorIndex;
private readonly Action<AcceleratorStream, Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>> _gpuKernel;
private volatile ulong[] _snapshot;
private volatile bool _useGpuGeneration = true;
private readonly int _divisorCyclesBatchSize;
private static int _sharedDivisorCyclesBatchSize = GpuConstants.GpuCycleStepsPerInvocation;
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>>> _gpuKernelCache = new(AcceleratorReferenceComparer.Instance); private const int CycleCacheTrackingLimit = 500_000;
private volatile ulong[] _snapshot; private static readonly HashSet<ulong> CycleCacheTrackedDivisors = new();
private volatile bool _useGpuGeneration = true; private static readonly object CycleCacheTrackingLock = new();
private readonly int _divisorCyclesBatchSize; private static ulong CycleCacheHitCount;
private static int _sharedDivisorCyclesBatchSize = GpuConstants.GpuCycleStepsPerInvocation; private static bool CycleCacheTrackingDisabled;
private const int CycleCacheTrackingLimit = 500_000; public static void SetDivisorCyclesBatchSize(int divisorCyclesBatchSize)
private static readonly HashSet<ulong> CycleCacheTrackedDivisors = new(); {
private static readonly object CycleCacheTrackingLock = new(); _sharedDivisorCyclesBatchSize = Math.Max(1, divisorCyclesBatchSize);
private static ulong CycleCacheHitCount; }
private static bool CycleCacheTrackingDisabled;
public static void SetDivisorCyclesBatchSize(int divisorCyclesBatchSize) public static DivisorCycleCache Shared { get; } = new DivisorCycleCache(_sharedDivisorCyclesBatchSize);
{
_sharedDivisorCyclesBatchSize = Math.Max(1, divisorCyclesBatchSize);
}
public static DivisorCycleCache Shared { get; } = new DivisorCycleCache(_sharedDivisorCyclesBatchSize); public int PreferredBatchSize => _divisorCyclesBatchSize;
public int PreferredBatchSize => _divisorCyclesBatchSize; // private static readonly Accelerator[] _accelerators;
private DivisorCycleCache(int divisorCyclesBatchSize) private DivisorCycleCache(int divisorCyclesBatchSize)
{ {
_divisorCyclesBatchSize = Math.Max(1, divisorCyclesBatchSize); _divisorCyclesBatchSize = Math.Max(1, divisorCyclesBatchSize);
_snapshot = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
} _acceleratorIndex = acceleratorIndex;
Accelerator accelerator = AcceleratorPool.Shared.Accelerators[acceleratorIndex];
_accelerator = accelerator;
_gpuKernel = LoadKernel(accelerator);
_snapshot = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot();
}
public void ConfigureGeneratorDevice(bool useGpu) public void ConfigureGeneratorDevice(bool useGpu)
{ {
_useGpuGeneration = useGpu; _useGpuGeneration = useGpu;
} }
public void RefreshSnapshot() public void RefreshSnapshot()
{ {
_snapshot = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot(); _snapshot = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot();
} }
public ulong GetCycleLength(ulong divisor, bool skipPrimeOrderHeuristic = false) public ulong GetCycleLength(ulong divisor, bool skipPrimeOrderHeuristic = false)
{ {
if (!skipPrimeOrderHeuristic) if (!skipPrimeOrderHeuristic)
{ {
Span<ulong> result = stackalloc ulong[1]; Span<ulong> result = stackalloc ulong[1];
ReadOnlySpan<ulong> singleDivisor = stackalloc ulong[1] { divisor }; ReadOnlySpan<ulong> singleDivisor = stackalloc ulong[1] { divisor };
GetCycleLengths(singleDivisor, result); GetCycleLengths(singleDivisor, result);
return result[0]; return result[0];
} }
// The skipPrimeOrderHeuristic path is only used for divisors produced by the CPU by-divisor scanner, // The skipPrimeOrderHeuristic path is only used for divisors produced by the CPU by-divisor scanner,
// which always generates values greater than one. Keep this guard disabled here to avoid repeating the // which always generates values greater than one. Keep this guard disabled here to avoid repeating the
// validation already performed by the general GetCycleLengths path. // validation already performed by the general GetCycleLengths path.
// if (divisor <= 1UL) // if (divisor <= 1UL)
// { // {
// throw new InvalidDataException("Divisor must be > 1"); // throw new InvalidDataException("Divisor must be > 1");
// } // }
ulong[] snapshot = _snapshot; ulong[] snapshot = _snapshot;
if (divisor < (ulong)snapshot.Length) if (divisor < (ulong)snapshot.Length)
{ {
ulong cached = snapshot[divisor]; ulong cached = snapshot[divisor];
if (cached == 0UL) if (cached == 0UL)
{ {
throw new InvalidDataException($"Divisor cycle is missing for {divisor}"); throw new InvalidDataException($"Divisor cycle is missing for {divisor}");
} }
// ObserveCycleCacheDivisor(divisor); return cached;
return cached; }
}
MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor); MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor);
return MersenneDivisorCycles.CalculateCycleLength(divisor, divisorData, skipPrimeOrderHeuristic: true); return MersenneDivisorCycles.CalculateCycleLength(divisor, divisorData, skipPrimeOrderHeuristic: true);
} }
public void GetCycleLengths(ReadOnlySpan<ulong> divisors, Span<ulong> cycles) public void GetCycleLengths(ReadOnlySpan<ulong> divisors, Span<ulong> cycles)
{ {
ulong[] snapshot = _snapshot; ulong[] snapshot = _snapshot;
int length = divisors.Length; int length = divisors.Length;
int[]? rentedMissing = null; int[]? rentedMissing = null;
Span<int> missingBuffer = length <= StackBufferThreshold Span<int> missingBuffer = length <= StackBufferThreshold
? stackalloc int[length] ? stackalloc int[length]
: new Span<int>(rentedMissing = ArrayPool<int>.Shared.Rent(length), 0, length); : new Span<int>(rentedMissing = ThreadStaticPools.IntPool.Rent(length), 0, length);
int missingCount = 0; int missingCount = 0;
for (int i = 0; i < length; i++) for (int i = 0; i < length; i++)
{ {
ulong divisor = divisors[i]; ulong divisor = divisors[i];
if (divisor <= 1UL) if (divisor <= 1UL)
{ {
throw new InvalidDataException("Divisor must be > 1"); throw new InvalidDataException("Divisor must be > 1");
} }
if (divisor < (ulong)snapshot.Length) if (divisor < (ulong)snapshot.Length)
{ {
ulong cached = snapshot[divisor]; ulong cached = snapshot[divisor];
if (cached == 0UL) if (cached == 0UL)
{ {
throw new InvalidDataException($"Divisor cycle is missing for {divisor}"); throw new InvalidDataException($"Divisor cycle is missing for {divisor}");
} }
// ObserveCycleCacheDivisor(divisor); cycles[i] = cached;
cycles[i] = cached; }
} else
else {
{ missingBuffer[missingCount++] = i;
missingBuffer[missingCount++] = i; }
} }
}
if (missingCount != 0) if (missingCount != 0)
{ {
ReadOnlySpan<int> missingIndices = missingBuffer[..missingCount]; ReadOnlySpan<int> missingIndices = missingBuffer[..missingCount];
if (_useGpuGeneration) if (_useGpuGeneration)
{ {
ComputeCyclesGpu(divisors, cycles, missingIndices); ComputeCyclesGpu(divisors, cycles, missingIndices);
} }
else else
{ {
ComputeCyclesCpu(divisors, cycles, missingIndices); ComputeCyclesCpu(divisors, cycles, missingIndices);
} }
} }
if (rentedMissing is not null) if (rentedMissing is not null)
{ {
ArrayPool<int>.Shared.Return(rentedMissing, clearArray: false); ThreadStaticPools.IntPool.Return(rentedMissing, clearArray: false);
} }
} }
private static void ObserveCycleCacheDivisor(ulong divisor) private static void ObserveCycleCacheDivisor(ulong divisor)
{ {
bool logHit = false; bool logHit = false;
ulong hitNumber = 0UL; ulong hitNumber = 0UL;
bool trackingDisabledNow = false; bool trackingDisabledNow = false;
lock (CycleCacheTrackingLock) lock (CycleCacheTrackingLock)
{ {
if (CycleCacheTrackingDisabled) if (CycleCacheTrackingDisabled)
{ {
return; return;
} }
if (!CycleCacheTrackedDivisors.Add(divisor)) if (!CycleCacheTrackedDivisors.Add(divisor))
{ {
CycleCacheHitCount++; CycleCacheHitCount++;
hitNumber = CycleCacheHitCount; hitNumber = CycleCacheHitCount;
logHit = true; logHit = true;
} }
else if (CycleCacheTrackedDivisors.Count >= CycleCacheTrackingLimit) else if (CycleCacheTrackedDivisors.Count >= CycleCacheTrackingLimit)
{ {
CycleCacheTrackingDisabled = true; CycleCacheTrackingDisabled = true;
CycleCacheTrackedDivisors.Clear(); CycleCacheTrackedDivisors.Clear();
trackingDisabledNow = true; trackingDisabledNow = true;
} }
} }
if (logHit) if (logHit)
{ {
Console.WriteLine($"Cycle cache hit for divisor {divisor} ({hitNumber})"); Console.WriteLine($"Cycle cache hit for divisor {divisor} ({hitNumber})");
} }
else if (trackingDisabledNow) else if (trackingDisabledNow)
{ {
Console.WriteLine("Cycle cache hit tracking disabled after exceeding the tracking limit."); Console.WriteLine("Cycle cache hit tracking disabled after exceeding the tracking limit.");
} }
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
private static void ComputeCyclesCpu(ReadOnlySpan<ulong> divisors, Span<ulong> cycles, ReadOnlySpan<int> indices) private static void ComputeCyclesCpu(ReadOnlySpan<ulong> divisors, Span<ulong> cycles, ReadOnlySpan<int> indices)
{ {
for (int i = 0; i < indices.Length; i++) for (int i = 0; i < indices.Length; i++)
{ {
int targetIndex = indices[i]; int targetIndex = indices[i];
ulong divisor = divisors[targetIndex]; ulong divisor = divisors[targetIndex];
MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor); MontgomeryDivisorData divisorData = MontgomeryDivisorData.FromModulus(divisor);
cycles[targetIndex] = MersenneDivisorCycles.CalculateCycleLength(divisor, divisorData); cycles[targetIndex] = MersenneDivisorCycles.CalculateCycleLength(divisor, divisorData);
} }
} }
private void ComputeCyclesGpu(ReadOnlySpan<ulong> divisors, Span<ulong> cycles, ReadOnlySpan<int> indices) private void ComputeCyclesGpu(ReadOnlySpan<ulong> divisors, Span<ulong> cycles, ReadOnlySpan<int> indices)
{ {
int count = indices.Length; int count = indices.Length;
ulong[]? rentedDivisors = null; ulong[]? rentedDivisors = null;
Span<ulong> workDivisors = count <= StackBufferThreshold Span<ulong> workDivisors = count <= StackBufferThreshold
? stackalloc ulong[count] ? stackalloc ulong[count]
: new Span<ulong>(rentedDivisors = ArrayPool<ulong>.Shared.Rent(count), 0, count); : new Span<ulong>(rentedDivisors = ThreadStaticPools.UlongPool.Rent(count), 0, count);
ulong[]? rentedResults = null; ulong[]? rentedResults = null;
Span<ulong> workResults = count <= StackBufferThreshold Span<ulong> workResults = count <= StackBufferThreshold
? stackalloc ulong[count] ? stackalloc ulong[count]
: new Span<ulong>(rentedResults = ArrayPool<ulong>.Shared.Rent(count), 0, count); : new Span<ulong>(rentedResults = ThreadStaticPools.UlongPool.Rent(count), 0, count);
for (int i = 0; i < count; i++) for (int i = 0; i < count; i++)
{ {
workDivisors[i] = divisors[indices[i]]; workDivisors[i] = divisors[indices[i]];
} }
ComputeCyclesGpuCore(workDivisors, workResults); ComputeCyclesGpuCore(workDivisors, workResults);
for (int i = 0; i < count; i++) for (int i = 0; i < count; i++)
{ {
cycles[indices[i]] = workResults[i]; cycles[indices[i]] = workResults[i];
} }
if (rentedDivisors is not null) if (rentedDivisors is not null)
{ {
ArrayPool<ulong>.Shared.Return(rentedDivisors, clearArray: false); ThreadStaticPools.UlongPool.Return(rentedDivisors, clearArray: false);
ArrayPool<ulong>.Shared.Return(rentedResults!, clearArray: false); ThreadStaticPools.UlongPool.Return(rentedResults!, clearArray: false);
} }
} }
private void ComputeCyclesGpuCore(ReadOnlySpan<ulong> divisors, Span<ulong> destination) private const int DefaultCapacity = 128;
{
int length = divisors.Length;
var gpuLease = GpuKernelPool.GetKernel(useGpuOrder: true);
Accelerator accelerator = gpuLease.Accelerator; [ThreadStatic]
var kernel = _gpuKernelCache.GetOrAdd(accelerator, LoadKernel); private static MemoryBuffer1D<ulong, Stride1D.Dense>? divisorBuffer;
using MemoryBuffer1D<ulong, Stride1D.Dense> divisorBuffer = accelerator.Allocate1D<ulong>(length); [ThreadStatic]
using MemoryBuffer1D<ulong, Stride1D.Dense> powBuffer = accelerator.Allocate1D<ulong>(length); private static MemoryBuffer1D<ulong, Stride1D.Dense>? powBuffer;
using MemoryBuffer1D<ulong, Stride1D.Dense> orderBuffer = accelerator.Allocate1D<ulong>(length);
using MemoryBuffer1D<ulong, Stride1D.Dense> resultBuffer = accelerator.Allocate1D<ulong>(length);
using MemoryBuffer1D<byte, Stride1D.Dense> statusBuffer = accelerator.Allocate1D<byte>(length);
ulong[]? rentedPow = null; [ThreadStatic]
Span<ulong> powSpan = length <= StackBufferThreshold private static MemoryBuffer1D<ulong, Stride1D.Dense>? orderBuffer;
? stackalloc ulong[length]
: new Span<ulong>(rentedPow = ArrayPool<ulong>.Shared.Rent(length), 0, length);
ulong[]? rentedOrder = null; [ThreadStatic]
Span<ulong> orderSpan = length <= StackBufferThreshold private static MemoryBuffer1D<ulong, Stride1D.Dense>? resultBuffer;
? stackalloc ulong[length]
: new Span<ulong>(rentedOrder = ArrayPool<ulong>.Shared.Rent(length), 0, length);
ulong[]? rentedResult = null; [ThreadStatic]
Span<ulong> resultSpan = length <= StackBufferThreshold private static MemoryBuffer1D<byte, Stride1D.Dense>? statusBuffer;
? stackalloc ulong[length]
: new Span<ulong>(rentedResult = ArrayPool<ulong>.Shared.Rent(length), 0, length);
byte[]? rentedStatus = null; private void EnsureCapacity(int requiredCapacity)
Span<byte> statusSpan = length <= StackBufferThreshold {
? stackalloc byte[length] if (requiredCapacity <= (divisorBuffer?.Length ?? 0))
: new Span<byte>(rentedStatus = ArrayPool<byte>.Shared.Rent(length), 0, length); {
return;
}
ref ulong divisorRef = ref MemoryMarshal.GetReference(divisors); if (divisorBuffer != null)
divisorBuffer.View.CopyFromCPU(ref divisorRef, length); {
divisorBuffer.Dispose();
powBuffer!.Dispose();
orderBuffer!.Dispose();
resultBuffer!.Dispose();
statusBuffer!.Dispose();
}
for (int i = 0; i < length; i++) Accelerator accelerator = _accelerator;
{ lock(accelerator)
ulong divisor = divisors[i]; {
ulong initialPow = 2UL; divisorBuffer = accelerator.Allocate1D<ulong>(requiredCapacity);
if (initialPow >= divisor) powBuffer = accelerator.Allocate1D<ulong>(requiredCapacity);
{ orderBuffer = accelerator.Allocate1D<ulong>(requiredCapacity);
initialPow -= divisor; resultBuffer = accelerator.Allocate1D<ulong>(requiredCapacity);
} statusBuffer = accelerator.Allocate1D<byte>(requiredCapacity);
}
}
powSpan[i] = initialPow; private void ComputeCyclesGpuCore(ReadOnlySpan<ulong> divisors, Span<ulong> destination)
orderSpan[i] = 1UL; {
resultSpan[i] = 0UL; int length = divisors.Length;
statusSpan[i] = ByteZero;
}
powBuffer.View.CopyFromCPU(ref MemoryMarshal.GetReference(powSpan), length); EnsureCapacity(length);
orderBuffer.View.CopyFromCPU(ref MemoryMarshal.GetReference(orderSpan), length);
resultBuffer.View.CopyFromCPU(ref MemoryMarshal.GetReference(resultSpan), length);
statusBuffer.View.CopyFromCPU(ref MemoryMarshal.GetReference(statusSpan), length);
int pending; ulong[]? rentedPow = null;
do Span<ulong> powSpan = length <= StackBufferThreshold
{ ? stackalloc ulong[length]
kernel(length, _divisorCyclesBatchSize, divisorBuffer.View, powBuffer.View, orderBuffer.View, resultBuffer.View, statusBuffer.View); : new Span<ulong>(rentedPow = ThreadStaticPools.UlongPool.Rent(length), 0, length);
accelerator.Synchronize();
statusBuffer.View.CopyToCPU(ref MemoryMarshal.GetReference(statusSpan), length); ulong[]? rentedOrder = null;
pending = 0; Span<ulong> orderSpan = length <= StackBufferThreshold
for (int i = 0; i < length; i++) ? stackalloc ulong[length]
{ : new Span<ulong>(rentedOrder = ThreadStaticPools.UlongPool.Rent(length), 0, length);
if (statusSpan[i] == ByteZero)
{
pending++;
}
}
}
while (pending > 0);
resultBuffer.View.CopyToCPU(ref MemoryMarshal.GetReference(resultSpan), length); ulong[]? rentedResult = null;
resultSpan.CopyTo(destination); Span<ulong> resultSpan = length <= StackBufferThreshold
if (rentedPow is not null) ? stackalloc ulong[length]
{ : new Span<ulong>(rentedResult = ThreadStaticPools.UlongPool.Rent(length), 0, length);
ArrayPool<ulong>.Shared.Return(rentedPow, clearArray: false);
ArrayPool<ulong>.Shared.Return(rentedOrder!, clearArray: false);
ArrayPool<ulong>.Shared.Return(rentedResult!, clearArray: false);
ArrayPool<byte>.Shared.Return(rentedStatus!, clearArray: false);
}
gpuLease.Dispose(); byte[]? rentedStatus = null;
} Span<byte> statusSpan = length <= StackBufferThreshold
? stackalloc byte[length]
: new Span<byte>(rentedStatus = ThreadStaticPools.BytePool.Rent(length), 0, length);
private static Action<Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>> LoadKernel(Accelerator accelerator) for (int i = 0; i < length; i++)
{ {
return accelerator.LoadAutoGroupedStreamKernel<Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>>(DivisorCycleKernels.GpuAdvanceDivisorCyclesKernel); ulong divisor = divisors[i];
} ulong initialPow = 2UL;
if (initialPow >= divisor)
{
initialPow -= divisor;
}
private sealed class AcceleratorReferenceComparer : IEqualityComparer<Accelerator> powSpan[i] = initialPow;
{ orderSpan[i] = 1UL;
internal static AcceleratorReferenceComparer Instance { get; } = new(); resultSpan[i] = 0UL;
statusSpan[i] = ByteZero;
}
public bool Equals(Accelerator? x, Accelerator? y) => ReferenceEquals(x, y); // GpuPrimeWorkLimiter.Acquire();
int pending;
var kernel = _gpuKernel;
var acceleratorIndex = _acceleratorIndex;
var stream = AcceleratorStreamPool.Rent(acceleratorIndex);
public int GetHashCode(Accelerator obj) => RuntimeHelpers.GetHashCode(obj); divisorBuffer!.View.CopyFromCPU(stream, divisors);
} powBuffer!.View.CopyFromCPU(stream, powSpan);
orderBuffer!.View.CopyFromCPU(stream, orderSpan);
resultBuffer!.View.CopyFromCPU(stream, resultSpan);
statusBuffer!.View.CopyFromCPU(stream, statusSpan);
do
{
kernel(stream, length, _divisorCyclesBatchSize, divisorBuffer.View, powBuffer.View, orderBuffer.View, resultBuffer.View, statusBuffer.View);
statusBuffer.View.CopyToCPU(stream, statusSpan);
stream.Synchronize();
pending = 0;
for (int i = 0; i < length; i++)
{
if (statusSpan[i] == ByteZero)
{
pending++;
}
}
}
while (pending > 0);
resultBuffer.View.CopyToCPU(stream, resultSpan);
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
resultSpan.CopyTo(destination);
if (rentedPow is not null)
{
ThreadStaticPools.UlongPool.Return(rentedPow, clearArray: false);
ThreadStaticPools.UlongPool.Return(rentedOrder!, clearArray: false);
ThreadStaticPools.UlongPool.Return(rentedResult!, clearArray: false);
ThreadStaticPools.BytePool.Return(rentedStatus!, clearArray: false);
}
// GpuPrimeWorkLimiter.Release();
}
private static Action<AcceleratorStream, Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>> LoadKernel(Accelerator accelerator)
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>>(DivisorCycleKernels.GpuAdvanceDivisorCyclesKernel);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>>>();
}
private sealed class AcceleratorReferenceComparer : IEqualityComparer<Accelerator>
{
internal static AcceleratorReferenceComparer Instance { get; } = new();
public bool Equals(Accelerator? x, Accelerator? y) => ReferenceEquals(x, y);
public int GetHashCode(Accelerator obj) => RuntimeHelpers.GetHashCode(obj);
}
} }
-2
View File
@@ -1,5 +1,3 @@
using System;
using System.Collections.Generic;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
namespace PerfectNumbers.Core; namespace PerfectNumbers.Core;
@@ -0,0 +1,61 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu
{
public struct AcceleratorPool
{
internal readonly Context Context;
internal readonly Device Device;
internal readonly Accelerator[] Accelerators;
private int _index;
public int Rent()
{
Atomic.CompareExchange(ref _index, Capacity, 0);
var index = Atomic.Add(ref _index, 1);
if (index >= Capacity)
{
index -= Capacity;
}
return index;
}
public void Return(int acceleratorIndex)
{
// Intentionally left empty - there's nothing to do here. We're not really renting anything.
}
public static readonly AcceleratorPool Shared = new(PerfectNumberConstants.RollingAccelerators);
private readonly int Capacity;
public AcceleratorPool(int capacity)
{
Console.WriteLine("Creating accelerators");
Capacity = capacity;
var context = Context.CreateDefault();
var device = context.GetPreferredDevice(false);
Context = context;
Device = device;
int acceleratorsInContext = 0;
int acceleratorIndex = 0;
Accelerators = [.. Enumerable.Range(0, capacity).Select(_ =>
{
Console.WriteLine($"Preparing accelerator {acceleratorIndex++}");
Accelerator accelerator = device.CreateAccelerator(context);
acceleratorsInContext++;
if (acceleratorsInContext >= 128)
{
context = Context.CreateDefault();
device = context.GetPreferredDevice(false);
acceleratorsInContext = 0;
}
return accelerator;
})];
}
}
}
@@ -0,0 +1,13 @@
using System.Runtime.CompilerServices;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu;
internal sealed class AcceleratorReferenceComparer : IEqualityComparer<Accelerator>
{
internal static AcceleratorReferenceComparer Instance { get; } = new();
public bool Equals(Accelerator? x, Accelerator? y) => ReferenceEquals(x, y);
public int GetHashCode(Accelerator obj) => RuntimeHelpers.GetHashCode(obj);
}
@@ -0,0 +1,34 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
public static class AcceleratorExtensions
{
public static MemoryBuffer1D<ulong, Stride1D.Dense> CopySpanToDevice(this Accelerator accelerator, AcceleratorStream stream, in ulong[] span)
{
MemoryBuffer1D<ulong, Stride1D.Dense>? buffer;
buffer = accelerator.Allocate1D(stream, span);
return buffer;
}
public static MemoryBuffer1D<ulong, Stride1D.Dense> CopyUintSpanToDevice(this Accelerator accelerator, AcceleratorStream stream, ReadOnlySpan<uint> span)
{
MemoryBuffer1D<ulong, Stride1D.Dense>? buffer;
buffer = accelerator.Allocate1D<ulong>(span.Length);
if (!span.IsEmpty)
{
var converted = new ulong[span.Length];
for (int i = 0; i < span.Length; i++)
{
converted[i] = span[i];
}
buffer.View.CopyFromCPU(stream, converted);
}
return buffer;
}
}
@@ -0,0 +1,42 @@
using System.Collections.Concurrent;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
public static class AcceleratorStreamPool
{
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
private static readonly SemaphoreSlim[] _locks = new SemaphoreSlim[PerfectNumberConstants.RollingAccelerators];
private static readonly ConcurrentQueue<AcceleratorStream>[] _streams = new ConcurrentQueue<AcceleratorStream> [PerfectNumberConstants.RollingAccelerators];
private static SemaphoreSlim CreateLock() => new(PerfectNumberConstants.ThreadsByAccelerator);
// private static int _rented;
public static AcceleratorStream Rent(int acceleratorIndex)
{
var streamLock = _locks[acceleratorIndex];
var queue = _streams[acceleratorIndex];
streamLock.Wait();
return queue.TryDequeue(out var stream)
? stream
: _accelerators[acceleratorIndex].CreateStream();
}
public static void Return(int acceleratorIndex, AcceleratorStream stream)
{
var streamLock = _locks[acceleratorIndex];
var queue = _streams[acceleratorIndex];
queue.Enqueue(stream);
streamLock.Release();
}
public static void WarmUp(int acceleratorIndex)
{
_streams[acceleratorIndex] = new();
_locks[acceleratorIndex] = CreateLock();
}
}
@@ -0,0 +1,69 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
internal readonly struct HeuristicCombinedGpuTables
{
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
private static readonly HeuristicCombinedGpuTables[] _sharedTables = new HeuristicCombinedGpuTables[PerfectNumberConstants.RollingAccelerators];
internal static void WarmUp(int acceleratorIndex, AcceleratorStream stream) => _sharedTables[acceleratorIndex] = new(_accelerators[acceleratorIndex], stream);
internal static HeuristicCombinedGpuTables GetStaticTables(int acceleratorIndex) => _sharedTables[acceleratorIndex];
internal HeuristicCombinedGpuTables(Accelerator accelerator, AcceleratorStream stream)
{
var combinedEnding1 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding1;
HeuristicCombinedDivisorsEnding1 = accelerator.CopySpanToDevice(stream, combinedEnding1);
var combinedSquaresEnding1 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding1Squares;
HeuristicCombinedDivisorSquaresEnding1 = accelerator.CopySpanToDevice(stream, combinedSquaresEnding1);
var combinedEnding3 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding3;
HeuristicCombinedDivisorsEnding3 = accelerator.CopySpanToDevice(stream, combinedEnding3);
var combinedSquaresEnding3 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding3Squares;
HeuristicCombinedDivisorSquaresEnding3 = accelerator.CopySpanToDevice(stream, combinedSquaresEnding3);
var combinedEnding7 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding7;
HeuristicCombinedDivisorsEnding7 = accelerator.CopySpanToDevice(stream, combinedEnding7);
var combinedSquaresEnding7 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding7Squares;
HeuristicCombinedDivisorSquaresEnding7 = accelerator.CopySpanToDevice(stream, combinedSquaresEnding7);
var combinedEnding9 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding9;
HeuristicCombinedDivisorsEnding9 = accelerator.CopySpanToDevice(stream, combinedEnding9);
var combinedSquaresEnding9 = HeuristicCombinedPrimeTester.CombinedDivisorsEnding9Squares;
HeuristicCombinedDivisorSquaresEnding9 = accelerator.CopySpanToDevice(stream, combinedSquaresEnding9);
}
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorsEnding1;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorSquaresEnding1;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorsEnding3;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorSquaresEnding3;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorsEnding7;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorSquaresEnding7;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorsEnding9;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicCombinedDivisorSquaresEnding9;
internal HeuristicCombinedGpuViews CreateViews() => new(
HeuristicCombinedDivisorsEnding1.View,
HeuristicCombinedDivisorsEnding3.View,
HeuristicCombinedDivisorsEnding7.View,
HeuristicCombinedDivisorsEnding9.View,
HeuristicCombinedDivisorSquaresEnding1.View,
HeuristicCombinedDivisorSquaresEnding3.View,
HeuristicCombinedDivisorSquaresEnding7.View,
HeuristicCombinedDivisorSquaresEnding9.View);
internal void Dispose()
{
HeuristicCombinedDivisorsEnding1.Dispose();
HeuristicCombinedDivisorSquaresEnding1.Dispose();
HeuristicCombinedDivisorsEnding3.Dispose();
HeuristicCombinedDivisorSquaresEnding3.Dispose();
HeuristicCombinedDivisorsEnding7.Dispose();
HeuristicCombinedDivisorSquaresEnding7.Dispose();
HeuristicCombinedDivisorsEnding9.Dispose();
HeuristicCombinedDivisorSquaresEnding9.Dispose();
}
}
@@ -0,0 +1,45 @@
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
public readonly struct HeuristicCombinedGpuViews(
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding1,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding3,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding7,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding9,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding1,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding3,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding7,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding9)
{
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding1 = combinedDivisorsEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding3 = combinedDivisorsEnding3;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding7 = combinedDivisorsEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding9 = combinedDivisorsEnding9;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding1 = combinedDivisorSquaresEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding3 = combinedDivisorSquaresEnding3;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding7 = combinedDivisorSquaresEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding9 = combinedDivisorSquaresEnding9;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectDivisors(byte ending) => ending switch
{
1 => CombinedDivisorsEnding1,
3 => CombinedDivisorsEnding3,
7 => CombinedDivisorsEnding7,
9 => CombinedDivisorsEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectDivisorSquares(byte ending) => ending switch
{
1 => CombinedDivisorSquaresEnding1,
3 => CombinedDivisorSquaresEnding3,
7 => CombinedDivisorSquaresEnding7,
9 => CombinedDivisorSquaresEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
@@ -0,0 +1,79 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
internal sealed class HeuristicGroupABGpuTables
{
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
private static readonly HeuristicGroupABGpuTables[] _sharedTables = new HeuristicGroupABGpuTables[PerfectNumberConstants.RollingAccelerators];
private static readonly SemaphoreSlim[] _locks = [..Enumerable.Range(1, PerfectNumberConstants.RollingAccelerators).Select(_ => new SemaphoreSlim(1))];
internal static HeuristicGroupABGpuTables EnsureStaticTables(int acceleratorIndex, AcceleratorStream stream)
{
var @lock = _locks[acceleratorIndex];
@lock.Wait();
if (_sharedTables[acceleratorIndex] is {} existing)
{
@lock.Release();
return existing;
}
var accelerator = _accelerators[acceleratorIndex];
existing = new(accelerator, stream);
_sharedTables[acceleratorIndex] = existing;
@lock.Release();
return existing;
}
internal HeuristicGroupABGpuTables(Accelerator accelerator, AcceleratorStream stream)
{
var heuristicGroupA = HeuristicPrimeSieves.GroupADivisorsStorage;
HeuristicGroupADivisors = accelerator.CopySpanToDevice(stream, heuristicGroupA);
var heuristicGroupASquares = HeuristicPrimeSieves.GroupADivisorSquaresStorage;
HeuristicGroupADivisorSquares = accelerator.CopySpanToDevice(stream, heuristicGroupASquares);
HeuristicGroupBDivisorsEnding1 = accelerator.CopyUintSpanToDevice(stream, DivisorGenerator.SmallPrimesLastOneWithoutLastThree);
HeuristicGroupBDivisorSquaresEnding1 = accelerator.CopySpanToDevice(stream, DivisorGenerator.SmallPrimesPow2LastOneWithoutLastThree);
HeuristicGroupBDivisorsEnding7 = accelerator.CopyUintSpanToDevice(stream, DivisorGenerator.SmallPrimesLastSevenWithoutLastThree);
HeuristicGroupBDivisorSquaresEnding7 = accelerator.CopySpanToDevice(stream, DivisorGenerator.SmallPrimesPow2LastSevenWithoutLastThree);
HeuristicGroupBDivisorsEnding9 = accelerator.CopyUintSpanToDevice(stream, DivisorGenerator.SmallPrimesLastNineWithoutLastThree);
HeuristicGroupBDivisorSquaresEnding9 = accelerator.CopySpanToDevice(stream, DivisorGenerator.SmallPrimesPow2LastNineWithoutLastThree);
}
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupADivisors;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupADivisorSquares;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorsEnding1;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorSquaresEnding1;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorsEnding7;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorSquaresEnding7;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorsEnding9;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorSquaresEnding9;
internal HeuristicGroupABGpuViews CreateViews() => new(
HeuristicGroupADivisors.View,
HeuristicGroupADivisorSquares.View,
HeuristicGroupBDivisorsEnding1.View,
HeuristicGroupBDivisorSquaresEnding1.View,
HeuristicGroupBDivisorsEnding7.View,
HeuristicGroupBDivisorSquaresEnding7.View,
HeuristicGroupBDivisorsEnding9.View,
HeuristicGroupBDivisorSquaresEnding9.View);
internal void Dispose()
{
HeuristicGroupADivisors.Dispose();
HeuristicGroupADivisorSquares.Dispose();
HeuristicGroupBDivisorsEnding1.Dispose();
HeuristicGroupBDivisorSquaresEnding1.Dispose();
HeuristicGroupBDivisorsEnding7.Dispose();
HeuristicGroupBDivisorSquaresEnding7.Dispose();
HeuristicGroupBDivisorsEnding9.Dispose();
HeuristicGroupBDivisorSquaresEnding9.Dispose();
}
}
@@ -0,0 +1,63 @@
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
public readonly struct HeuristicGroupABGpuViews(
ArrayView1D<ulong, Stride1D.Dense> groupADivisors,
ArrayView1D<ulong, Stride1D.Dense> groupADivisorSquares,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorsEnding1,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorSquaresEnding1,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorsEnding7,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorSquaresEnding7,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorsEnding9,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorSquaresEnding9)
{
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupADivisors = groupADivisors;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupADivisorSquares = groupADivisorSquares;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorsEnding1 = groupBDivisorsEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorSquaresEnding1 = groupBDivisorSquaresEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorsEnding7 = groupBDivisorsEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorSquaresEnding7 = groupBDivisorSquaresEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorsEnding9 = groupBDivisorsEnding9;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorSquaresEnding9 = groupBDivisorSquaresEnding9;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectGroupB(byte ending) => ending switch
{
1 => GroupBDivisorsEnding1,
7 => GroupBDivisorsEnding7,
9 => GroupBDivisorsEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectGroupBSquares(byte ending) => ending switch
{
1 => GroupBDivisorSquaresEnding1,
7 => GroupBDivisorSquaresEnding7,
9 => GroupBDivisorSquaresEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectDivisors(HeuristicGpuDivisorTableKind kind) => kind switch
{
HeuristicGpuDivisorTableKind.GroupA => GroupADivisors,
HeuristicGpuDivisorTableKind.GroupBEnding1 => GroupBDivisorsEnding1,
HeuristicGpuDivisorTableKind.GroupBEnding7 => GroupBDivisorsEnding7,
HeuristicGpuDivisorTableKind.GroupBEnding9 => GroupBDivisorsEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectDivisorSquares(HeuristicGpuDivisorTableKind kind) => kind switch
{
HeuristicGpuDivisorTableKind.GroupA => GroupADivisorSquares,
HeuristicGpuDivisorTableKind.GroupBEnding1 => GroupBDivisorSquaresEnding1,
HeuristicGpuDivisorTableKind.GroupBEnding7 => GroupBDivisorSquaresEnding7,
HeuristicGpuDivisorTableKind.GroupBEnding9 => GroupBDivisorSquaresEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
@@ -0,0 +1,174 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
internal sealed class HeuristicGroupABPrimeTesterAccelerator
{
#region Pool
[ThreadStatic]
private static Queue<HeuristicGroupABPrimeTesterAccelerator>? Pool;
internal static void WarmUp()
{
Accelerator[]? accelerators = AcceleratorPool.Shared.Accelerators;
int acceleratorCount = accelerators.Length;
for (var i = 0; i < acceleratorCount; i++)
{
Console.WriteLine($"Preparing heuristic accelerator {i}...");
var accelerator = accelerators[i];
AcceleratorStream stream = accelerator.CreateStream();
_ = HeuristicGroupABGpuTables.EnsureStaticTables(i, stream);
// _ = GpuKernelPool.GetOrAddKernels(accelerator, stream, KernelType.None);
// KernelContainer kernels = GpuKernelPool.GetOrAddKernels(accelerator, stream);
// GpuStaticTableInitializer.EnsureStaticTables(accelerator, kernels, stream);
stream.Synchronize();
stream.Dispose();
Console.WriteLine($"Heuristic accelerator {i} is ready");
}
}
internal static HeuristicGroupABPrimeTesterAccelerator Rent(int minBufferCapacity = 1)
{
var pool = Pool ??= new();
if (!pool.TryDequeue(out var lease))
{
lease = new HeuristicGroupABPrimeTesterAccelerator(minBufferCapacity);
}
else
{
lease.EnsureCapacity(minBufferCapacity);
}
return lease;
}
internal static void Return(HeuristicGroupABPrimeTesterAccelerator lease)
{
var pool = Pool ??= new();
pool.Enqueue(lease);
}
internal static void DisposeAll()
{
if (Pool != null)
{
while (Pool.TryDequeue(out var lease))
{
lease.DisposeResources();
}
}
}
internal static void Clear()
{
if (Pool != null)
{
var retained = new List<HeuristicGroupABPrimeTesterAccelerator>();
while (Pool.TryDequeue(out var lease))
{
lease.DisposeResources();
}
}
}
#endregion
public readonly HeuristicGroupABGpuViews DivisorTables;
internal readonly Context Context;
public readonly Accelerator Accelerator;
[ThreadStatic]
public static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>>>? SmallPrimeSieveKernel;
internal static Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>> GetSmallPrimeSieveKernel(Accelerator accelerator)
{
var pool = SmallPrimeSieveKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
cached = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>>(PrimeTesterKernels.SmallPrimeSieveKernel)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>>>();
pool[accelerator] = cached;
return cached;
}
public readonly Kernel HeuristicTrialDivisionKernel;
public MemoryBuffer1D<ulong, Stride1D.Dense>? Input = null;
public MemoryBuffer1D<byte, Stride1D.Dense>? Output = null;
public MemoryBuffer1D<int, Stride1D.Dense>? HeuristicFlag = null!;
public int BufferCapacity;
private HeuristicGroupABGpuTables _sharedTables;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupADivisors => _sharedTables.HeuristicGroupADivisors;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupADivisorSquares => _sharedTables.HeuristicGroupADivisorSquares;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorsEnding1 => _sharedTables.HeuristicGroupBDivisorsEnding1;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorSquaresEnding1 => _sharedTables.HeuristicGroupBDivisorSquaresEnding1;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorsEnding7 => _sharedTables.HeuristicGroupBDivisorsEnding7;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorSquaresEnding7 => _sharedTables.HeuristicGroupBDivisorSquaresEnding7;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorsEnding9 => _sharedTables.HeuristicGroupBDivisorsEnding9;
public MemoryBuffer1D<ulong, Stride1D.Dense> HeuristicGroupBDivisorSquaresEnding9 => _sharedTables.HeuristicGroupBDivisorSquaresEnding9;
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
internal readonly int AcceleratorIndex;
internal HeuristicGroupABPrimeTesterAccelerator(int minBufferCapacity)
{
var acceleratorIndex = AcceleratorPool.Shared.Rent();
AcceleratorIndex = acceleratorIndex;
Accelerator accelerator = _accelerators[acceleratorIndex];
Context = accelerator.Context;
Accelerator = accelerator;
EnsureCapacity(minBufferCapacity);
HeuristicTrialDivisionKernel = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<int>, ulong, ulong, HeuristicGpuDivisorTableKind, HeuristicGroupABGpuViews>(PrimeTesterKernels.HeuristicTrialGroupABDivisionKernel));
AcceleratorStream stream = AcceleratorStreamPool.Rent(acceleratorIndex);
// GpuStaticTableInitializer.EnsureStaticTables(accelerator, kernels, stream);
_sharedTables = HeuristicGroupABGpuTables.EnsureStaticTables(acceleratorIndex, stream);
DivisorTables = _sharedTables.CreateViews();
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
}
public void EnsureCapacity(int minCapacity)
{
if (minCapacity <= BufferCapacity)
{
return;
}
Input?.Dispose();
Output?.Dispose();
HeuristicFlag?.Dispose();
Input = Accelerator.Allocate1D<ulong>(minCapacity);
Output = Accelerator.Allocate1D<byte>(minCapacity);
HeuristicFlag = Accelerator.Allocate1D<int>(minCapacity);
BufferCapacity = minCapacity;
}
public void Dispose()
{
Return(this);
}
internal void DisposeResources()
{
Input?.Dispose();
Output?.Dispose();
HeuristicFlag?.Dispose();
// ReleaseSharedTables(_sharedTables);
// These resources are shared between GPU leases
// Accelerator.Dispose();
// AcceleratorContext.Dispose();
}
}
@@ -0,0 +1,58 @@
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu.Accelerators;
internal readonly struct LastDigitGpuTables
{
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
private static readonly LastDigitGpuTables[] _sharedTables = new LastDigitGpuTables[PerfectNumberConstants.RollingAccelerators];
[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static LastDigitGpuTables GetStaticTables(int acceleratorIndex) => _sharedTables[acceleratorIndex];
[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static void WarmUp(int acceleratorIndex, AcceleratorStream stream) => _sharedTables[acceleratorIndex] = new LastDigitGpuTables(_accelerators[acceleratorIndex], stream);
internal LastDigitGpuTables(Accelerator accelerator, AcceleratorStream stream)
{
var primesLastOne = DivisorGenerator.SmallPrimesLastOne;
var primesLastSeven = DivisorGenerator.SmallPrimesLastSeven;
var primesLastThree = DivisorGenerator.SmallPrimesLastThree;
var primesLastNine = DivisorGenerator.SmallPrimesLastNine;
var primesPow2LastOne = DivisorGenerator.SmallPrimesPow2LastOne;
var primesPow2LastSeven = DivisorGenerator.SmallPrimesPow2LastSeven;
var primesPow2LastThree = DivisorGenerator.SmallPrimesPow2LastThree;
var primesPow2LastNine = DivisorGenerator.SmallPrimesPow2LastNine;
DevicePrimesLastOne = accelerator.Allocate1D(stream, primesLastOne);
DevicePrimesLastSeven = accelerator.Allocate1D(stream, primesLastSeven);
DevicePrimesLastThree = accelerator.Allocate1D(stream, primesLastThree);
DevicePrimesLastNine = accelerator.Allocate1D(stream, primesLastNine);
DevicePrimesPow2LastOne = accelerator.Allocate1D(stream, primesPow2LastOne);
DevicePrimesPow2LastSeven = accelerator.Allocate1D(stream, primesPow2LastSeven);
DevicePrimesPow2LastThree = accelerator.Allocate1D(stream, primesPow2LastThree);
DevicePrimesPow2LastNine = accelerator.Allocate1D(stream, primesPow2LastNine);
}
internal readonly MemoryBuffer1D<uint, Stride1D.Dense> DevicePrimesLastOne;
internal readonly MemoryBuffer1D<uint, Stride1D.Dense> DevicePrimesLastSeven;
internal readonly MemoryBuffer1D<uint, Stride1D.Dense> DevicePrimesLastThree;
internal readonly MemoryBuffer1D<uint, Stride1D.Dense> DevicePrimesLastNine;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> DevicePrimesPow2LastOne;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> DevicePrimesPow2LastSeven;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> DevicePrimesPow2LastThree;
internal readonly MemoryBuffer1D<ulong, Stride1D.Dense> DevicePrimesPow2LastNine;
internal void Dispose()
{
DevicePrimesLastOne.Dispose();
DevicePrimesLastSeven.Dispose();
DevicePrimesLastThree.Dispose();
DevicePrimesLastNine.Dispose();
DevicePrimesPow2LastOne.Dispose();
DevicePrimesPow2LastSeven.Dispose();
DevicePrimesPow2LastThree.Dispose();
DevicePrimesPow2LastNine.Dispose();
}
}
@@ -0,0 +1,300 @@
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
using PerfectNumbers.Core.Gpu.Kernels;
namespace PerfectNumbers.Core.Gpu.Accelerators;
public sealed class PrimeOrderCalculatorAccelerator
{
#region Pool
[ThreadStatic]
private static Queue<PrimeOrderCalculatorAccelerator>? _pool;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static PrimeOrderCalculatorAccelerator Rent(int primeTesterCapacity)
{
var pool = _pool ??= [];
if (!pool.TryDequeue(out var gpu))
{
int acceleratorIndex = AcceleratorPool.Shared.Rent();
gpu = new(acceleratorIndex, PerfectNumberConstants.DefaultFactorsBuffer, primeTesterCapacity, PerfectNumberConstants.DefaultSmallPrimeFactorSlotCount, PerfectNumberConstants.DefaultSpecialMaxFactorCapacity);
}
else
{
gpu.EnsureCapacity(PerfectNumberConstants.DefaultFactorsBuffer, primeTesterCapacity);
}
return gpu;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static void Return(PrimeOrderCalculatorAccelerator gpu) => _pool!.Enqueue(gpu);
internal static void Clear()
{
Queue<PrimeOrderCalculatorAccelerator>? pool = _pool;
if (pool != null)
{
while (pool.TryDequeue(out var gpu))
{
gpu.Dispose();
}
}
}
internal static void DisposeAll()
{
Queue<PrimeOrderCalculatorAccelerator>? pool = _pool;
if (pool != null)
{
while (pool.TryDequeue(out var gpu))
{
gpu.Dispose();
}
}
}
internal static void WarmUp()
{
Accelerator[] accelerators = AcceleratorPool.Shared.Accelerators;
int acceleratorCount = accelerators.Length;
for (var i = 0; i < acceleratorCount; i++)
{
Console.WriteLine($"Preparing accelerator {i}...");
var accelerator = accelerators[i];
AcceleratorStreamPool.WarmUp(i);
// Don't take this from the pool as quick uploads of data to the accelerator consumes much of GPU's memory and throws.
AcceleratorStream stream = accelerator.CreateStream();
// TODO: Review which tables are needed when final execution path is defined.
LastDigitGpuTables.WarmUp(i, stream);
HeuristicCombinedGpuTables.WarmUp(i, stream);
// SmallPrimeFactorGpuTables.WarmUp(i, stream);
// SharedHeuristicGpuTables.EnsureStaticTables(accelerator, stream);
// _ = GpuKernelPool.GetOrAddKernels(accelerator, stream, KernelType.None);
// KernelContainer kernels = GpuKernelPool.GetOrAddKernels(accelerator, stream);
// GpuStaticTableInitializer.EnsureStaticTables(accelerator, kernels, stream);
stream.Synchronize();
stream.Dispose();
accelerator.Synchronize();
}
}
#endregion
#region Static Tables
public readonly ArrayView1D<uint, Stride1D.Dense> DevicePrimesLastOne;
public readonly ArrayView1D<uint, Stride1D.Dense> DevicePrimesLastSeven;
public readonly ArrayView1D<uint, Stride1D.Dense> DevicePrimesLastThree;
public readonly ArrayView1D<uint, Stride1D.Dense> DevicePrimesLastNine;
public readonly ArrayView1D<ulong, Stride1D.Dense> DevicePrimesPow2LastOne;
public readonly ArrayView1D<ulong, Stride1D.Dense> DevicePrimesPow2LastSeven;
public readonly ArrayView1D<ulong, Stride1D.Dense> DevicePrimesPow2LastThree;
public readonly ArrayView1D<ulong, Stride1D.Dense> DevicePrimesPow2LastNine;
public readonly HeuristicCombinedGpuViews DivisorTables;
public readonly ArrayView1D<uint, Stride1D.Dense> SmallPrimeFactorPrimes;
public readonly ArrayView1D<ulong, Stride1D.Dense> SmallPrimeFactorSquares;
#endregion
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
public readonly Accelerator Accelerator;
public readonly int AcceleratorIndex;
public MemoryBuffer1D<ulong, Stride1D.Dense> Input;
public MemoryBuffer1D<ulong, Stride1D.Dense> OutputUlong;
public MemoryBuffer1D<byte, Stride1D.Dense> OutputByte;
public MemoryBuffer1D<int, Stride1D.Dense> OutputInt;
public MemoryBuffer1D<KeyValuePair<ulong, int>, Stride1D.Dense> Pow2ModEntriesToTestOnDevice;
public readonly MemoryBuffer1D<int, Stride1D.Dense> SmallPrimeFactorCountSlot;
public readonly MemoryBuffer1D<ulong, Stride1D.Dense> SpecialMaxResult;
public Dictionary<ulong, int> Pow2ModEntriesToTestOnHost = [];
public readonly Action<AcceleratorStream, int, ulong, ArrayView1D<KeyValuePair<ulong, int>, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>> CheckFactorsKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ArrayView1D<ulong, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>> ConvertToStandardKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, byte, ArrayView<int>, ulong, ulong, HeuristicCombinedGpuViews> HeuristicCombinedTrialDivisionKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ArrayView1D<ulong, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>> KeepMontgomeryKernelLauncher;
public readonly Action<AcceleratorStream, ulong, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> PollardRhoKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ArrayView1D<GpuUInt128, Stride1D.Dense>, GpuUInt128, ArrayView1D<GpuUInt128, Stride1D.Dense>> Pow2ModWideKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<byte>> SharesFactorKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> SmallPrimeFactorKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>> SmallPrimeSieveKernelLauncher;
public readonly Action<AcceleratorStream, Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> SpecialMaxKernelLauncher;
public void EnsureCapacity(int factorsCount, int primeTesterCapacity)
{
Accelerator accelerator = Accelerator;
MemoryBuffer1D<KeyValuePair<ulong, int>, Stride1D.Dense> pow2ModEntriesToTestOnDevice = Pow2ModEntriesToTestOnDevice;
if (pow2ModEntriesToTestOnDevice.Length < factorsCount)
{
lock (accelerator)
{
pow2ModEntriesToTestOnDevice.Dispose();
Pow2ModEntriesToTestOnDevice = accelerator.Allocate1D<KeyValuePair<ulong, int>>(factorsCount);
}
}
MemoryBuffer1D<ulong, Stride1D.Dense> input = Input;
if (input.Length < primeTesterCapacity)
{
lock (accelerator)
{
input.Dispose();
Input = accelerator.Allocate1D<ulong>(primeTesterCapacity);
}
}
MemoryBuffer1D<byte, Stride1D.Dense> outputByte = OutputByte;
if (outputByte.Length < primeTesterCapacity)
{
lock (accelerator)
{
outputByte.Dispose();
OutputByte = accelerator.Allocate1D<byte>(primeTesterCapacity);
}
}
MemoryBuffer1D<int, Stride1D.Dense> outputInt = OutputInt;
if (outputInt.Length < primeTesterCapacity)
{
lock (accelerator)
{
outputInt.Dispose();
OutputInt = accelerator.Allocate1D<int>(primeTesterCapacity);
}
}
}
public void EnsureSmallPrimeFactorSlotsCapacity(int newSize)
{
// Console.WriteLine($"Resizing GPU scratch buffer from pool ({buffer.SmallPrimeFactorPrimeSlots.Length} / {smallPrimeFactorSlotCount}), ({buffer.SpecialMaxFactors.Length}/{specialMaxFactorCapacity})");
var accelerator = Accelerator;
MemoryBuffer1D<ulong, Stride1D.Dense> outputUlong = OutputUlong;
if (outputUlong.Length < newSize)
{
lock (accelerator)
{
outputUlong.Dispose();
OutputUlong = accelerator.Allocate1D<ulong>(newSize);
}
}
MemoryBuffer1D<int, Stride1D.Dense> outputInt = OutputInt;
if (outputInt.Length < newSize)
{
lock (accelerator)
{
outputInt.Dispose();
OutputInt = accelerator.Allocate1D<int>(newSize);
}
}
}
public void EnsureSpecialMaxFactorsCapacity(int newSize)
{
var accelerator = Accelerator;
MemoryBuffer1D<ulong, Stride1D.Dense> input = Input;
if (input.Length < newSize)
{
lock (accelerator)
{
input.Dispose();
Input = accelerator.Allocate1D<ulong>(newSize);
}
}
MemoryBuffer1D<ulong, Stride1D.Dense> outputUlong = OutputUlong;
if (outputUlong.Length < newSize)
{
lock (accelerator)
{
outputUlong.Dispose();
OutputUlong = accelerator.Allocate1D<ulong>(newSize);
}
}
}
public PrimeOrderCalculatorAccelerator(int acceleratorIndex, int factorsCount, int primeTesterCapacity, int smallPrimeFactorSlotCount, int specialMaxFactorCapacity)
{
var accelerator = _accelerators[acceleratorIndex];
Accelerator = accelerator;
AcceleratorIndex = acceleratorIndex;
lock (accelerator)
{
Input = accelerator.Allocate1D<ulong>(Math.Max(specialMaxFactorCapacity, primeTesterCapacity));
OutputByte = Accelerator.Allocate1D<byte>(primeTesterCapacity);
OutputInt = Accelerator.Allocate1D<int>(Math.Max(primeTesterCapacity, smallPrimeFactorSlotCount));
OutputUlong = accelerator.Allocate1D<ulong>(Math.Max(specialMaxFactorCapacity, smallPrimeFactorSlotCount));
Pow2ModEntriesToTestOnDevice = accelerator.Allocate1D<KeyValuePair<ulong, int>>(factorsCount);
SmallPrimeFactorCountSlot = accelerator.Allocate1D<int>(1);
SpecialMaxResult = accelerator.Allocate1D<ulong>(1);
LastDigitGpuTables lastDigitSharedTables = LastDigitGpuTables.GetStaticTables(acceleratorIndex);
DevicePrimesLastOne = lastDigitSharedTables.DevicePrimesLastOne.View;
DevicePrimesLastSeven = lastDigitSharedTables.DevicePrimesLastSeven.View;
DevicePrimesLastThree = lastDigitSharedTables.DevicePrimesLastThree.View;
DevicePrimesLastNine = lastDigitSharedTables.DevicePrimesLastNine.View;
DevicePrimesPow2LastOne = lastDigitSharedTables.DevicePrimesPow2LastOne.View;
DevicePrimesPow2LastSeven = lastDigitSharedTables.DevicePrimesPow2LastSeven.View;
DevicePrimesPow2LastThree = lastDigitSharedTables.DevicePrimesPow2LastThree.View;
DevicePrimesPow2LastNine = lastDigitSharedTables.DevicePrimesPow2LastNine.View;
DivisorTables = HeuristicCombinedGpuTables
.GetStaticTables(acceleratorIndex)
.CreateViews();
var smallPrimeFactorTables = SmallPrimeFactorGpuTables.GetStaticTables(acceleratorIndex);
if (smallPrimeFactorTables.HasValue)
{
SmallPrimeFactorGpuTables tables = smallPrimeFactorTables.Value;
SmallPrimeFactorPrimes = tables.Primes.View;
SmallPrimeFactorSquares = tables.Squares.View;
}
CheckFactorsKernelLauncher = KernelUtil.GetKernel(accelerator.LoadStreamKernel<int, ulong, ArrayView1D<KeyValuePair<ulong, int>, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>>(PrimeOrderGpuHeuristics.CheckFactorsKernel)).CreateLauncherDelegate<Action<AcceleratorStream, int, ulong, ArrayView1D<KeyValuePair<ulong, int>, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>>>();
ConvertToStandardKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView1D<ulong, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>>(Pow2MontgomeryKernels.Pow2MontgomeryKernelConvertToStandard)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView1D<ulong, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>>>();
HeuristicCombinedTrialDivisionKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, byte, ArrayView<int>, ulong, ulong, HeuristicCombinedGpuViews>(PrimeTesterKernels.HeuristicTrialCombinedDivisionKernel)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, byte, ArrayView<int>, ulong, ulong, HeuristicCombinedGpuViews>>();
KeepMontgomeryKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView1D<ulong, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>>(Pow2MontgomeryKernels.Pow2MontgomeryKernelKeepMontgomery)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView1D<ulong, Stride1D.Dense>, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>>>();
PollardRhoKernelLauncher = KernelUtil.GetKernel(accelerator.LoadStreamKernel<ulong, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2MontgomeryKernels.TryPollardRhoKernel)).CreateLauncherDelegate<Action<AcceleratorStream, ulong, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<byte, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>();
Pow2ModWideKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView1D<GpuUInt128, Stride1D.Dense>, GpuUInt128, ArrayView1D<GpuUInt128, Stride1D.Dense>>(PrimeOrderGpuHeuristics.Pow2ModKernelWide)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView1D<GpuUInt128, Stride1D.Dense>, GpuUInt128, ArrayView1D<GpuUInt128, Stride1D.Dense>>>();
SharesFactorKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<byte>>(PrimeTesterKernels.SharesFactorKernel)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<byte>>>();
SmallPrimeFactorKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(SmallPrimeFactorKernels.SmallPrimeFactorKernelScan)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>();
SmallPrimeSieveKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>>(PrimeTesterKernels.SmallPrimeSieveKernel)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<uint>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<ulong>, ArrayView<byte>>>();
SpecialMaxKernelLauncher = KernelUtil.GetKernel(accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(PrimeOrderGpuHeuristics.EvaluateSpecialMaxCandidatesKernel)).CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>();
}
}
public void Dispose()
{
Input.Dispose();
OutputByte.Dispose();
OutputInt.Dispose();
OutputUlong.Dispose();
Pow2ModEntriesToTestOnDevice.Dispose();
SmallPrimeFactorCountSlot.Dispose();
SpecialMaxResult.Dispose();
// These resources are shared between GPU leases
// Stream.Dispose();
// Accelerator.Dispose();
// Context.Dispose();
}
}
@@ -0,0 +1,75 @@
using System.Runtime.CompilerServices;
using ILGPU;
namespace PerfectNumbers.Core.Gpu;
public static class ThreadStaticDeterministicRandomGpu
{
[ThreadStatic]
private static DeterministicRandomGpu? _exclusive;
internal static DeterministicRandomGpu Exclusive => _exclusive ?? new DeterministicRandomGpu();
}
internal struct DeterministicRandomGpu
{
private static long s_seedCounter;
private ulong _state;
public readonly ulong State
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
get
{
return _state;
}
}
public DeterministicRandomGpu()
{
_state = CreateThreadState();
}
public DeterministicRandomGpu(ulong initialState)
{
_state = initialState;
}
public ulong NextUInt64()
{
ulong state = _state;
state ^= state >> 12;
state ^= state << 25;
state ^= state >> 27;
_state = state;
return state * 2685821657736338717UL;
}
public GpuUInt128 NextUInt128()
{
GpuUInt128 high = (GpuUInt128)NextUInt64();
ReadOnlyGpuUInt128 low = (ReadOnlyGpuUInt128)NextUInt64();
high.ShiftLeft(64);
high.Add(low);
return high;
}
public void SetState(ulong state)
{
_state = state;
}
private static ulong CreateThreadState()
{
ulong seed = (ulong)Atomic.Add(ref s_seedCounter, 1);
seed += 0x9E3779B97F4A7C15UL;
seed = (seed ^ (seed >> 30)) * 0xBF58476D1CE4E5B9UL;
seed = (seed ^ (seed >> 27)) * 0x94D049BB133111EBUL;
seed ^= seed >> 31;
if (seed == 0UL)
{
seed = 0x9E3779B97F4A7C15UL;
}
return seed;
}
}
@@ -1,5 +1,4 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using PerfectNumbers.Core;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -24,17 +23,14 @@ internal struct ExponentRemainderStepperGpu
_currentResidue = GpuUInt128.One; _currentResidue = GpuUInt128.One;
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public bool MatchesDivisor(in GpuDivisorPartialData divisor) public readonly bool MatchesDivisor(in GpuDivisorPartialData divisor) => _modulus == divisor.Modulus;
{
return _modulus == divisor.Modulus;
}
/// <summary> /// <summary>
/// Clears the cached residue so the stepper can be reused for another divisor. /// Clears the cached residue so the stepper can be reused for another divisor.
/// Callers must reinitialize the stepper before consuming any residues after a reset. /// Callers must reinitialize the stepper before consuming any residues after a reset.
/// </summary> /// </summary>
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public void Reset() public void Reset()
{ {
PreviousExponent = 0UL; PreviousExponent = 0UL;
-127
View File
@@ -1,127 +0,0 @@
using System.Collections.Concurrent;
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu;
public static class GpuContextPool
{
// The pool intentionally skips pre-loading ProcessEightBitWindows kernels so accelerator initialization
// stays stable; GpuKernelPool JIT-compiles them on first use alongside the small-cycle uploads.
// Default device preference for generic GPU kernels (prime scans, NTT, etc.)
internal sealed class PooledContext
{
public readonly Context Context;
public readonly Accelerator Accelerator;
public PooledContext(KernelContainer kernels)
{
Context = SharedGpuContext.Context;
Accelerator = SharedGpuContext.Accelerator;
GpuStaticTableInitializer.EnsureStaticTables(kernels, Accelerator);
kernels.Dispose();
// NOTE: Avoid loading/compiling any kernel here to prevent implicit
// CL stream/queue creation during accelerator construction.
// Some OpenCL drivers are fragile when a queue is created immediately
// after device init. Real kernels will be JIT-loaded at first use.
}
public void Dispose()
{
// Ensure all cached GPU buffers for this accelerator are released.
NttGpuMath.ClearCaches(Accelerator);
// Release PrimeTester GPU state for this accelerator (kernel/device primes)
// Clear any per-accelerator cached resources to avoid releasing
// after the accelerator is destroyed.
PrimeTester.ClearGpuCaches(Accelerator);
// These resources are shared between GPU leases
// Accelerator.Dispose();
// Context.Dispose();
}
}
private static readonly ConcurrentQueue<PooledContext> GpuPool = new();
private static int WarmedGpuContextCount;
// Allows callers to choose CPU/GPU per use-case, decoupled from ForceCpu.
public static GpuContextLease Rent()
{
if (GpuPool.TryDequeue(out var gpu))
{
return new GpuContextLease(gpu);
}
// Create a new accelerator when the pool does not have one available.
return new GpuContextLease(new PooledContext(GpuKernelPool.GetKernels()));
}
public static void WarmUpPool(int threadCount)
{
if (threadCount <= WarmedGpuContextCount)
{
return;
}
int toCreate = threadCount - WarmedGpuContextCount;
Console.WriteLine($"Warming up GPU context pool to {toCreate} contexts...");
var contexts = new PooledContext[toCreate];
var pool = GpuPool;
PooledContext context;
for (int i = 0; i < toCreate; i++)
{
KernelContainer kernels = GpuKernelPool.GetKernels();
context = new PooledContext(kernels);
contexts[i] = context;
pool.Enqueue(context);
}
WarmedGpuContextCount = threadCount;
}
public static void DisposeAll()
{
while (GpuPool.TryDequeue(out var gpu))
{
gpu.Dispose();
}
PrimeTester.DisposeGpuContexts();
WarmedGpuContextCount = 0;
}
private static void Return(PooledContext ctx)
{
ctx.Accelerator.Synchronize();
GpuPool.Enqueue(ctx);
}
public struct GpuContextLease
{
private PooledContext? _ctx;
private bool _disposed;
internal GpuContextLease(PooledContext ctx)
{
_ctx = ctx;
_disposed = false;
}
public Context Context => _ctx!.Context;
public Accelerator Accelerator => _ctx!.Accelerator;
public void Dispose()
{
if (_disposed)
{
return;
}
_disposed = true;
Return(_ctx!);
_ctx = null;
}
}
}
-165
View File
@@ -1,165 +0,0 @@
using System;
using System.Collections.Concurrent;
using ILGPU;
using ILGPU.Runtime;
using PerfectNumbers.Core;
using static PerfectNumbers.Core.Gpu.GpuContextPool;
namespace PerfectNumbers.Core.Gpu;
public sealed class GpuKernelLease
{
private static readonly ConcurrentQueue<GpuKernelLease> Pool = new();
private GpuContextLease? _gpu;
public KernelContainer Kernels;
private AcceleratorStream? _stream;
private GpuKernelLease(KernelContainer kernels)
{
Kernels = kernels;
}
internal static GpuKernelLease Rent(GpuContextLease gpu, KernelContainer kernels)
{
if (!Pool.TryDequeue(out var lease))
{
lease = new GpuKernelLease(kernels);
}
lease._gpu = gpu;
lease.Kernels = kernels;
lease._stream = null;
return lease;
}
public Accelerator Accelerator => _gpu?.Accelerator ?? throw new NullReferenceException("GPU context is already released");
public AcceleratorStream Stream
{
get
{
if (_stream is { } stream)
{
return stream;
}
// It feels that we can combine these into 1 return statement, but that increases the max stack frame size to 3 from 2
stream = Accelerator.CreateStream();
_stream = stream;
return stream;
}
}
public Action<AcceleratorStream, Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>> OrderKernel
{
get
{
var accel = Accelerator; // avoid capturing 'this' in lambda
return KernelContainer.InitOnce(ref Kernels!.Order, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>>(OrderKernels.OrderKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>>>();
});
}
}
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> Pow2ModKernel
{
get
{
var accel = Accelerator;
return KernelContainer.InitOnce(ref Kernels!.Pow2Mod, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2ModKernels.Pow2ModKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
}
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>> IncrementalKernel
{
get
{
var accel = Accelerator;
return KernelContainer.InitOnce(ref Kernels!.Incremental, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>>(IncrementalKernels.IncrementalKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
}
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>> IncrementalOrderKernel
{
get
{
var accel = Accelerator;
return KernelContainer.InitOnce(ref Kernels!.IncrementalOrder, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>(IncrementalKernels.IncrementalOrderKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
}
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>> Pow2ModOrderKernel
{
get
{
var accel = Accelerator;
return KernelContainer.InitOnce(ref Kernels!.Pow2ModOrder, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2ModKernels.Pow2ModOrderKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
}
public Action<AcceleratorStream, Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ushort, Stride1D.Dense>> SpecialMaxKernel
{
get
{
var accel = Accelerator;
return KernelContainer.InitOnce(ref Kernels!.SpecialMax, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ushort, Stride1D.Dense>>(PrimeOrderGpuHeuristics.EvaluateSpecialMaxCandidatesKernel);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ushort, Stride1D.Dense>>>();
});
}
}
public Action<AcceleratorStream, Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> SmallPrimeFactorKernel
{
get
{
var accel = Accelerator;
return KernelContainer.InitOnce(ref Kernels!.SmallPrimeFactor, () =>
{
var loaded = accel.LoadAutoGroupedStreamKernel<Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(SmallPrimeFactorKernels.SmallPrimeFactorKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
}
public void Dispose()
{
_stream?.Dispose();
_stream = null;
_gpu?.Dispose();
_gpu = null;
Kernels?.Dispose();
GpuPrimeWorkLimiter.Release();
Pool.Enqueue(this);
}
}
+183 -273
View File
@@ -1,189 +1,188 @@
using PerfectNumbers.Core;
using System.Collections.Concurrent;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
using static PerfectNumbers.Core.Gpu.GpuContextPool; using PerfectNumbers.Core.Gpu.Accelerators;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
public readonly struct ResidueAutomatonArgs
{
public readonly ulong Q0M10;
public readonly ulong Step10;
public readonly ulong Q0M8;
public readonly ulong Step8;
public readonly ulong Q0M3;
public readonly ulong Step3;
public readonly ulong Q0M5;
public readonly ulong Step5;
public ResidueAutomatonArgs(ulong q0m10, ulong step10, ulong q0m8, ulong step8, ulong q0m3, ulong step3, ulong q0m5, ulong step5)
{
Q0M10 = q0m10; Step10 = step10; Q0M8 = q0m8; Step8 = step8; Q0M3 = q0m3; Step3 = step3; Q0M5 = q0m5; Step5 = step5;
}
}
public sealed class KernelContainer
{
// Serializes first-time initialization of kernels/buffers per accelerator.
public Action<AcceleratorStream, Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>>? Order;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>>? Incremental;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>,
ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>,
ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>? Pow2Mod;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>? IncrementalOrder;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>? Pow2ModOrder;
public Action<AcceleratorStream, Index1D, ulong, uint, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, int, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<int, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>? SmallPrimeFactor;
public Action<AcceleratorStream, Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ushort, Stride1D.Dense>>? SpecialMax;
// Optional device buffer with small divisor cycles (<= 4M). Index = divisor, value = cycle length.
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallCycles;
public MemoryBuffer1D<uint, Stride1D.Dense>? SmallPrimeFactorsPrimes;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimeFactorsSquares;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimeFactorPrimeSlots;
public MemoryBuffer1D<int, Stride1D.Dense>? SmallPrimeFactorExponentSlots;
public MemoryBuffer1D<int, Stride1D.Dense>? SmallPrimeFactorCountSlot;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimeFactorRemainingSlot;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SpecialMaxFactors;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SpecialMaxCandidates;
public MemoryBuffer1D<ushort, Stride1D.Dense>? SpecialMaxResult;
public MemoryBuffer1D<uint, Stride1D.Dense>? SmallPrimesLastOne;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimesPow2LastOne;
public MemoryBuffer1D<uint, Stride1D.Dense>? SmallPrimesLastSeven;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimesPow2LastSeven;
public static T InitOnce<T>(ref T? slot, Func<T> factory) where T : class
{
var current = Volatile.Read(ref slot);
if (current is not null)
{
return current;
}
current = Volatile.Read(ref slot);
if (current is null)
{
current = factory();
Volatile.Write(ref slot, current);
}
return current;
}
public void Dispose() => GpuKernelPool.Return(this);
}
public readonly struct ResiduePrimeViews(
ArrayView1D<uint, Stride1D.Dense> lastOne,
ArrayView1D<uint, Stride1D.Dense> lastSeven,
ArrayView1D<ulong, Stride1D.Dense> lastOnePow2,
ArrayView1D<ulong, Stride1D.Dense> lastSevenPow2)
{
public readonly ArrayView1D<uint, Stride1D.Dense> LastOne = lastOne;
public readonly ArrayView1D<uint, Stride1D.Dense> LastSeven = lastSeven;
public readonly ArrayView1D<ulong, Stride1D.Dense> LastOnePow2 = lastOnePow2;
public readonly ArrayView1D<ulong, Stride1D.Dense> LastSevenPow2 = lastSevenPow2;
}
public readonly struct SmallPrimeFactorTables(
MemoryBuffer1D<uint, Stride1D.Dense> primes,
MemoryBuffer1D<ulong, Stride1D.Dense> squares,
int count)
{
public readonly MemoryBuffer1D<uint, Stride1D.Dense> Primes = primes;
public readonly MemoryBuffer1D<ulong, Stride1D.Dense> Squares = squares;
public readonly int Count = count;
public ArrayView1D<uint, Stride1D.Dense> PrimesView => Primes.View;
public ArrayView1D<ulong, Stride1D.Dense> SquaresView => Squares.View;
}
public readonly struct SmallPrimeFactorScratch(
MemoryBuffer1D<ulong, Stride1D.Dense> primeSlots,
MemoryBuffer1D<int, Stride1D.Dense> exponentSlots,
MemoryBuffer1D<int, Stride1D.Dense> countSlot,
MemoryBuffer1D<ulong, Stride1D.Dense> remainingSlot)
{
public readonly MemoryBuffer1D<ulong, Stride1D.Dense> PrimeSlots = primeSlots;
public readonly MemoryBuffer1D<int, Stride1D.Dense> ExponentSlots = exponentSlots;
public readonly MemoryBuffer1D<int, Stride1D.Dense> CountSlot = countSlot;
public readonly MemoryBuffer1D<ulong, Stride1D.Dense> RemainingSlot = remainingSlot;
public void Clear()
{
PrimeSlots.MemSetToZero();
ExponentSlots.MemSetToZero();
CountSlot.MemSetToZero();
RemainingSlot.MemSetToZero();
}
}
public readonly struct SpecialMaxScratch(
MemoryBuffer1D<ulong, Stride1D.Dense> factorValues,
MemoryBuffer1D<ulong, Stride1D.Dense> candidateValues,
MemoryBuffer1D<ushort, Stride1D.Dense> resultSlot)
{
public readonly MemoryBuffer1D<ulong, Stride1D.Dense> FactorValues = factorValues;
public readonly MemoryBuffer1D<ulong, Stride1D.Dense> CandidateValues = candidateValues;
public readonly MemoryBuffer1D<ushort, Stride1D.Dense> ResultSlot = resultSlot;
public ArrayView1D<ulong, Stride1D.Dense> FactorsView => FactorValues.View;
public ArrayView1D<ulong, Stride1D.Dense> CandidatesView => CandidateValues.View;
public ArrayView1D<ushort, Stride1D.Dense> ResultView => ResultSlot.View;
}
public class GpuKernelPool public class GpuKernelPool
{ {
private static readonly ConcurrentQueue<KernelContainer> KernelCache = new(); // TODO: Replace this concurrent map with a simple accelerator-indexed lookup once kernel launchers are prewarmed during startup so we can drop the thread-safe wrapper entirely. [ThreadStatic]
private static KernelContainer[]? _pool;
public static KernelContainer GetKernels() => KernelCache.TryDequeue(out var container) public static T InitOnce<T>(ref T? target, Func<T> valueFactory) where T : class
? container
: new KernelContainer();
public static void Return(KernelContainer container) => KernelCache.Enqueue(container);
internal static void PreloadStaticTables(KernelContainer kernels, Accelerator accelerator)
{ {
EnsureSmallCyclesOnDevice(kernels, accelerator); if (target is { } value)
EnsureSmallPrimesOnDevice(kernels, accelerator); {
EnsureSmallPrimeFactorTables(kernels, accelerator); return value;
}
var newValue = valueFactory();
return Interlocked.CompareExchange(ref target, newValue, null) ?? newValue;
} }
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
internal static KernelContainer GetOrAddKernels(int acceleratorIndex, AcceleratorStream stream, KernelType kernelType)
{
var pool = _pool ??= [];
if (pool[acceleratorIndex] is not { } kernels)
{
kernels = new();
}
Accelerator accelerator = _accelerators[acceleratorIndex];
if (kernelType.HasFlag(KernelType.OrderKernelScan))
{
InitOrderKernelScan(accelerator, kernels);
}
if (kernelType.HasFlag(KernelType.Pow2ModKernelScan))
{
InitPow2ModKernelScan(accelerator, kernels);
}
if (kernelType.HasFlag(KernelType.IncrementalKernelScan))
{
InitIncrementalKernelScan(accelerator, kernels);
}
if (kernelType.HasFlag(KernelType.IncrementalOrderKernelScan))
{
IncrementalOrderKernelScan(accelerator, kernels);
}
if (kernelType.HasFlag(KernelType.Pow2ModOrderKernelScan))
{
InitPow2ModOrderKernelScan(accelerator, kernels);
}
PreloadStaticTables(accelerator, kernels, stream, kernelType);
pool[acceleratorIndex] = kernels;
return kernels;
static void InitOrderKernelScan(Accelerator accelerator, KernelContainer kernels)
{
InitOnce(ref kernels.Order, () =>
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>>(OrderKernels.OrderKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>>>();
});
}
static void InitPow2ModKernelScan(Accelerator accelerator, KernelContainer kernels)
{
InitOnce(ref kernels.Pow2Mod, () =>
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2ModKernels.Pow2ModKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
static void InitIncrementalKernelScan(Accelerator accelerator, KernelContainer kernels)
{
InitOnce(ref kernels.Incremental, () =>
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>>(IncrementalKernels.IncrementalKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
static void IncrementalOrderKernelScan(Accelerator accelerator, KernelContainer kernels)
{
InitOnce(ref kernels.IncrementalOrder, () =>
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>(IncrementalKernels.IncrementalOrderKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte,
ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
static void InitPow2ModOrderKernelScan(Accelerator accelerator, KernelContainer kernels)
{
InitOnce(ref kernels.Pow2ModOrder, () =>
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2ModKernels.Pow2ModOrderKernelScan);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>>();
});
}
}
internal static void PreloadStaticTables(Accelerator accelerator, KernelContainer kernels, AcceleratorStream stream, KernelType kernelType)
{
if (kernelType.HasFlag(KernelType.SmallCycles))
{
EnsureSmallCyclesOnDevice(accelerator, kernels, stream);
}
if (kernelType.HasFlag(KernelType.SmallPrimes))
{
EnsureSmallPrimesOnDevice(accelerator, kernels, stream);
}
}
public static ArrayView1D<ulong, Stride1D.Dense> GetSmallCyclesOnDevice(KernelContainer kernels) => kernels.SmallCycles!;
// Ensures the small cycles table is uploaded to the device for the given accelerator. // Ensures the small cycles table is uploaded to the device for the given accelerator.
// Returns the ArrayView to pass into kernels (when kernels are extended to accept it). // Returns the ArrayView to pass into kernels (when kernels are extended to accept it).
public static ArrayView1D<ulong, Stride1D.Dense> EnsureSmallCyclesOnDevice(KernelContainer kernels, Accelerator accelerator) private static void EnsureSmallCyclesOnDevice(Accelerator accelerator, KernelContainer kernels, AcceleratorStream stream)
{ {
if (kernels.SmallCycles is { } buffer) if (kernels.SmallCycles is { })
{ {
return buffer.View; return;
} }
var host = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot(); // TODO: Preload this device buffer during startup and keep it immutable so we can delete the lock above in favor of the preloaded snapshot. var host = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot();
var device = accelerator.Allocate1D<ulong>(host.Length); MemoryBuffer1D<ulong, Stride1D.Dense>? device;
device.View.CopyFromCPU(host);
lock (accelerator)
{
device = accelerator.Allocate1D<ulong>(host.Length);
}
device.View.CopyFromCPU(stream, host);
kernels.SmallCycles = device; kernels.SmallCycles = device;
return device.View;
} }
public static ResiduePrimeViews EnsureSmallPrimesOnDevice(KernelContainer kernels, Accelerator accelerator) public static ResiduePrimeViews GetSmallPrimesOnDevice(KernelContainer kernels) => new(
kernels.SmallPrimesLastOne!,
kernels.SmallPrimesLastSeven!,
kernels.SmallPrimesPow2LastOne!,
kernels.SmallPrimesPow2LastSeven!
);
private static void EnsureSmallPrimesOnDevice(Accelerator accelerator, KernelContainer kernels, AcceleratorStream stream)
{ {
if (kernels.SmallPrimesLastOne is { } lastOne && if (kernels.SmallPrimesLastOne is { } &&
kernels.SmallPrimesLastSeven is { } lastSeven && kernels.SmallPrimesLastSeven is { } &&
kernels.SmallPrimesPow2LastOne is { } lastOnePow2 && kernels.SmallPrimesPow2LastOne is { } &&
kernels.SmallPrimesPow2LastSeven is { } lastSevenPow2) kernels.SmallPrimesPow2LastSeven is { })
{ {
return new ResiduePrimeViews(lastOne.View, lastSeven.View, lastOnePow2.View, lastSevenPow2.View); return;
} }
if (kernels.SmallPrimesLastOne is { } existingLastOne && if (kernels.SmallPrimesLastOne is { } &&
kernels.SmallPrimesLastSeven is { } existingLastSeven && kernels.SmallPrimesLastSeven is { } &&
kernels.SmallPrimesPow2LastOne is { } existingLastOnePow2 && kernels.SmallPrimesPow2LastOne is { } &&
kernels.SmallPrimesPow2LastSeven is { } existingLastSevenPow2) kernels.SmallPrimesPow2LastSeven is { })
{ {
return new ResiduePrimeViews(existingLastOne.View, existingLastSeven.View, existingLastOnePow2.View, existingLastSevenPow2.View); return;
} }
var hostLastOne = PrimesGenerator.SmallPrimesLastOne; var hostLastOne = PrimesGenerator.SmallPrimesLastOne;
@@ -191,118 +190,27 @@ public class GpuKernelPool
var hostLastOnePow2 = PrimesGenerator.SmallPrimesPow2LastOne; var hostLastOnePow2 = PrimesGenerator.SmallPrimesPow2LastOne;
var hostLastSevenPow2 = PrimesGenerator.SmallPrimesPow2LastSeven; var hostLastSevenPow2 = PrimesGenerator.SmallPrimesPow2LastSeven;
var deviceLastOne = accelerator.Allocate1D<uint>(hostLastOne.Length); MemoryBuffer1D<uint, Stride1D.Dense>? deviceLastOne;
deviceLastOne.View.CopyFromCPU(hostLastOne); MemoryBuffer1D<uint, Stride1D.Dense>? deviceLastSeven;
var deviceLastSeven = accelerator.Allocate1D<uint>(hostLastSeven.Length); MemoryBuffer1D<ulong, Stride1D.Dense>? deviceLastOnePow2;
deviceLastSeven.View.CopyFromCPU(hostLastSeven); MemoryBuffer1D<ulong, Stride1D.Dense>? deviceLastSevenPow2;
var deviceLastOnePow2 = accelerator.Allocate1D<ulong>(hostLastOnePow2.Length);
deviceLastOnePow2.View.CopyFromCPU(hostLastOnePow2);
var deviceLastSevenPow2 = accelerator.Allocate1D<ulong>(hostLastSevenPow2.Length);
deviceLastSevenPow2.View.CopyFromCPU(hostLastSevenPow2);
lock (accelerator)
{
deviceLastOne = accelerator.Allocate1D<uint>(hostLastOne.Length);
deviceLastSeven = accelerator.Allocate1D<uint>(hostLastSeven.Length);
deviceLastOnePow2 = accelerator.Allocate1D<ulong>(hostLastOnePow2.Length);
deviceLastSevenPow2 = accelerator.Allocate1D<ulong>(hostLastSevenPow2.Length);
}
deviceLastOne.View.CopyFromCPU(stream, hostLastOne);
deviceLastSeven.View.CopyFromCPU(stream, hostLastSeven);
deviceLastOnePow2.View.CopyFromCPU(stream, hostLastOnePow2);
deviceLastSevenPow2.View.CopyFromCPU(stream, hostLastSevenPow2);
kernels.SmallPrimesLastOne = deviceLastOne; kernels.SmallPrimesLastOne = deviceLastOne;
kernels.SmallPrimesLastSeven = deviceLastSeven; kernels.SmallPrimesLastSeven = deviceLastSeven;
kernels.SmallPrimesPow2LastOne = deviceLastOnePow2; kernels.SmallPrimesPow2LastOne = deviceLastOnePow2;
kernels.SmallPrimesPow2LastSeven = deviceLastSevenPow2; kernels.SmallPrimesPow2LastSeven = deviceLastSevenPow2;
return new ResiduePrimeViews(deviceLastOne.View, deviceLastSeven.View, deviceLastOnePow2.View, deviceLastSevenPow2.View);
}
public static SmallPrimeFactorTables EnsureSmallPrimeFactorTables(KernelContainer kernels, Accelerator accelerator)
{
if (kernels.SmallPrimeFactorsPrimes is { } primeBuffer && kernels.SmallPrimeFactorsSquares is { } squareBuffer)
{
return new SmallPrimeFactorTables(primeBuffer, squareBuffer, (int)primeBuffer.Length);
}
if (kernels.SmallPrimeFactorsPrimes is { } existingPrimes && kernels.SmallPrimeFactorsSquares is { } existingSquares)
{
return new SmallPrimeFactorTables(existingPrimes, existingSquares, (int)existingPrimes.Length);
}
var hostPrimes = PrimesGenerator.SmallPrimes;
var hostSquares = PrimesGenerator.SmallPrimesPow2;
var devicePrimes = accelerator.Allocate1D<uint>(hostPrimes.Length);
devicePrimes.View.CopyFromCPU(hostPrimes);
var deviceSquares = accelerator.Allocate1D<ulong>(hostSquares.Length);
deviceSquares.View.CopyFromCPU(hostSquares);
kernels.SmallPrimeFactorsPrimes = devicePrimes;
kernels.SmallPrimeFactorsSquares = deviceSquares;
return new SmallPrimeFactorTables(devicePrimes, deviceSquares, hostPrimes.Length);
}
public static SmallPrimeFactorScratch EnsureSmallPrimeFactorScratch(KernelContainer kernels, Accelerator accelerator, int slotCount)
{
MemoryBuffer1D<ulong, Stride1D.Dense>? primeSlots = kernels.SmallPrimeFactorPrimeSlots;
if (primeSlots is null || primeSlots.Length < slotCount)
{
primeSlots?.Dispose();
primeSlots = accelerator.Allocate1D<ulong>(slotCount);
kernels.SmallPrimeFactorPrimeSlots = primeSlots;
}
MemoryBuffer1D<int, Stride1D.Dense>? exponentSlots = kernels.SmallPrimeFactorExponentSlots;
if (exponentSlots is null || exponentSlots.Length < slotCount)
{
exponentSlots?.Dispose();
exponentSlots = accelerator.Allocate1D<int>(slotCount);
kernels.SmallPrimeFactorExponentSlots = exponentSlots;
}
MemoryBuffer1D<int, Stride1D.Dense>? countSlot = kernels.SmallPrimeFactorCountSlot;
if (countSlot is null)
{
countSlot = accelerator.Allocate1D<int>(1);
kernels.SmallPrimeFactorCountSlot = countSlot;
}
MemoryBuffer1D<ulong, Stride1D.Dense>? remainingSlot = kernels.SmallPrimeFactorRemainingSlot;
if (remainingSlot is null)
{
remainingSlot = accelerator.Allocate1D<ulong>(1);
kernels.SmallPrimeFactorRemainingSlot = remainingSlot;
}
return new SmallPrimeFactorScratch(primeSlots!, exponentSlots!, countSlot!, remainingSlot!);
}
public static SpecialMaxScratch EnsureSpecialMaxScratch(KernelContainer kernels, Accelerator accelerator, int factorCapacity)
{
MemoryBuffer1D<ulong, Stride1D.Dense>? factorValues = kernels.SpecialMaxFactors;
if (factorValues is null || factorValues.Length < factorCapacity)
{
factorValues?.Dispose();
factorValues = accelerator.Allocate1D<ulong>(factorCapacity);
kernels.SpecialMaxFactors = factorValues;
}
MemoryBuffer1D<ulong, Stride1D.Dense>? candidateValues = kernels.SpecialMaxCandidates;
if (candidateValues is null || candidateValues.Length < factorCapacity)
{
candidateValues?.Dispose();
candidateValues = accelerator.Allocate1D<ulong>(factorCapacity);
kernels.SpecialMaxCandidates = candidateValues;
}
MemoryBuffer1D<ushort, Stride1D.Dense>? resultSlot = kernels.SpecialMaxResult;
if (resultSlot is null)
{
resultSlot = accelerator.Allocate1D<ushort>(1);
kernels.SpecialMaxResult = resultSlot;
}
return new SpecialMaxScratch(factorValues!, candidateValues!, resultSlot!);
}
public static GpuKernelLease GetKernel(bool useGpuOrder)
{
GpuPrimeWorkLimiter.Acquire();
var gpu = Rent();
var kernels = GetKernels();
return GpuKernelLease.Rent(gpu, kernels);
} }
/// <summary> /// <summary>
@@ -311,12 +219,14 @@ public class GpuKernelPool
/// <param name="action">Action to run with (Accelerator, Stream).</param> /// <param name="action">Action to run with (Accelerator, Stream).</param>
public static void Run(Action<Accelerator, AcceleratorStream> action) public static void Run(Action<Accelerator, AcceleratorStream> action)
{ {
var lease = GetKernel(useGpuOrder: true); // GpuPrimeWorkLimiter.Acquire();
var accelerator = lease.Accelerator; var acceleratorIndex = AcceleratorPool.Shared.Rent();
var stream = accelerator.CreateStream(); var accelerator = _accelerators[acceleratorIndex];
var stream = AcceleratorStreamPool.Rent(acceleratorIndex);
action(accelerator, stream); action(accelerator, stream);
stream.Synchronize();
stream.Dispose(); stream.Dispose();
lease.Dispose(); accelerator.Dispose();
// GpuPrimeWorkLimiter.Release();
} }
} }
@@ -1,15 +1,12 @@
using System;
using System.Threading;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
public static class GpuPrimeWorkLimiter public static class GpuPrimeWorkLimiter
{ {
private static SemaphoreSlim _semaphore = new(1, int.MaxValue); private static SemaphoreSlim _semaphore = new(1, int.MaxValue);
private static int _currentLimit = 1; private static int _currentLimit = 1;
public static void Acquire() => _semaphore.Wait(); public static void Acquire() { } // => _semaphore.Wait();
public static void Release() => _semaphore.Release(); public static void Release() { }// => _semaphore.Release();
public static void SetLimit(int value) public static void SetLimit(int value)
{ {
@@ -1,39 +0,0 @@
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
using PerfectNumbers.Core;
namespace PerfectNumbers.Core.Gpu;
internal static class GpuStaticTableInitializer
{
private static readonly ConcurrentDictionary<Accelerator, byte> Initialized = new(AcceleratorReferenceComparer.Instance);
internal static void EnsureStaticTables(KernelContainer kernels, Accelerator accelerator)
{
Initialized.GetOrAdd(accelerator, acc =>
{
PrimeTester.PrimeTesterGpuContextPool.EnsureStaticTables(kernels, acc);
GpuKernelPool.PreloadStaticTables(kernels, acc);
PrimeOrderGpuHeuristics.PreloadStaticTables( acc);
return 0;
});
}
private sealed class AcceleratorReferenceComparer : IEqualityComparer<Accelerator>
{
internal static AcceleratorReferenceComparer Instance { get; } = new();
public bool Equals(Accelerator? x, Accelerator? y)
{
return ReferenceEquals(x, y);
}
public int GetHashCode(Accelerator obj)
{
return RuntimeHelpers.GetHashCode(obj);
}
}
}
+47 -17
View File
@@ -261,13 +261,13 @@ public struct GpuUInt128 : IComparable<GpuUInt128>, IEquatable<GpuUInt128>
public override readonly int GetHashCode() => HashCode.Combine(High, Low); public override readonly int GetHashCode() => HashCode.Combine(High, Low);
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public void Add(in ReadOnlyGpuUInt128 other) => AddInternal(other.High, other.Low); public void Add(in ReadOnlyGpuUInt128 other) => Add(other.High, other.Low);
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public void Add(GpuUInt128 other) => AddInternal(other.High, other.Low); public void Add(GpuUInt128 other) => Add(other.High, other.Low);
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
private void AddInternal(ulong otherHigh, ulong otherLow) public void Add(ulong otherHigh, ulong otherLow)
{ {
ulong originalLow = Low; ulong originalLow = Low;
// We're reusing originalLow for carry to avoid an extra local. // We're reusing originalLow for carry to avoid an extra local.
@@ -316,7 +316,7 @@ public struct GpuUInt128 : IComparable<GpuUInt128>, IEquatable<GpuUInt128>
return; return;
} }
AddInternal(valueHigh, valueLow); Add(valueHigh, valueLow);
if (CompareTo(in modulus) >= 0) if (CompareTo(in modulus) >= 0)
{ {
Sub(in modulus); Sub(in modulus);
@@ -589,9 +589,6 @@ public struct GpuUInt128 : IComparable<GpuUInt128>, IEquatable<GpuUInt128>
[MethodImpl(MethodImplOptions.AggressiveInlining)] [MethodImpl(MethodImplOptions.AggressiveInlining)]
public static GpuUInt128 BinaryGcd(GpuUInt128 u, GpuUInt128 v) public static GpuUInt128 BinaryGcd(GpuUInt128 u, GpuUInt128 v)
{ {
// TODO: Replace this scalar binary GCD with the branchless reduction from
// GpuUInt128BinaryGcdBenchmarks so CPU fallbacks stay aligned with the GPU kernel
// performance when resolving large divisor residues.
if (u.IsZero) if (u.IsZero)
{ {
return v; return v;
@@ -602,26 +599,50 @@ public struct GpuUInt128 : IComparable<GpuUInt128>, IEquatable<GpuUInt128>
return u; return u;
} }
int shift = TrailingZeroCount(new GpuUInt128(u.High | v.High, u.Low | v.Low)); ulong combinedLow = u.Low | v.Low;
int shift;
if (combinedLow != 0UL)
{
shift = BitOperations.TrailingZeroCount(combinedLow);
}
else
{
ulong combinedHigh = u.High | v.High;
shift = 64 + BitOperations.TrailingZeroCount(combinedHigh);
}
int zu = TrailingZeroCount(u); int zu = TrailingZeroCount(u);
u >>= zu; u >>= zu;
do GpuUInt128 currentV = v;
while (true)
{ {
int zv = TrailingZeroCount(v); int zv = TrailingZeroCount(currentV);
v >>= zv; currentV >>= zv;
if (u > v)
bool swap = u > currentV;
GpuUInt128 min = ConditionalSelect(u, currentV, swap);
GpuUInt128 max = ConditionalSelect(currentV, u, swap);
max.Sub(min);
if (max.IsZero)
{ {
(u, v) = (v, u); return min << shift;
} }
v -= u; u = min;
currentV = max;
} }
while (!v.IsZero);
return u << shift;
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static GpuUInt128 ConditionalSelect(in GpuUInt128 left, in GpuUInt128 right, bool useRight)
{
ulong mask = useRight ? ulong.MaxValue : 0UL;
return new GpuUInt128(
left.High ^ ((left.High ^ right.High) & mask),
left.Low ^ ((left.Low ^ right.Low) & mask));
}
public void Sub(ulong value) public void Sub(ulong value)
{ {
@@ -1529,6 +1550,15 @@ public struct GpuUInt128 : IComparable<GpuUInt128>, IEquatable<GpuUInt128>
high = hi; high = hi;
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public void Pow2()
{
ulong operand = Low;
Low = operand * operand;
ulong highProduct = operand * High;
High = highProduct + MulHigh(operand, operand);
}
} }
// TODO: Check if the TODO below is still relevant. // TODO: Check if the TODO below is still relevant.
@@ -1,146 +0,0 @@
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu;
internal readonly struct HeuristicGpuDivisorTables
{
public static readonly HeuristicGpuDivisorTables Shared = default;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding3;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorsEnding9;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding3;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> CombinedDivisorSquaresEnding9;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupADivisors;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupADivisorSquares;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorsEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorSquaresEnding1;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorsEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorSquaresEnding7;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorsEnding9;
public readonly ArrayView1D<ulong, Stride1D.Dense> GroupBDivisorSquaresEnding9;
public HeuristicGpuDivisorTables(
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding1,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding3,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding7,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorsEnding9,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding1,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding3,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding7,
ArrayView1D<ulong, Stride1D.Dense> combinedDivisorSquaresEnding9,
ArrayView1D<ulong, Stride1D.Dense> groupADivisors,
ArrayView1D<ulong, Stride1D.Dense> groupADivisorSquares,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorsEnding1,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorSquaresEnding1,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorsEnding7,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorSquaresEnding7,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorsEnding9,
ArrayView1D<ulong, Stride1D.Dense> groupBDivisorSquaresEnding9)
{
CombinedDivisorsEnding1 = combinedDivisorsEnding1;
CombinedDivisorsEnding3 = combinedDivisorsEnding3;
CombinedDivisorsEnding7 = combinedDivisorsEnding7;
CombinedDivisorsEnding9 = combinedDivisorsEnding9;
CombinedDivisorSquaresEnding1 = combinedDivisorSquaresEnding1;
CombinedDivisorSquaresEnding3 = combinedDivisorSquaresEnding3;
CombinedDivisorSquaresEnding7 = combinedDivisorSquaresEnding7;
CombinedDivisorSquaresEnding9 = combinedDivisorSquaresEnding9;
GroupADivisors = groupADivisors;
GroupADivisorSquares = groupADivisorSquares;
GroupBDivisorsEnding1 = groupBDivisorsEnding1;
GroupBDivisorSquaresEnding1 = groupBDivisorSquaresEnding1;
GroupBDivisorsEnding7 = groupBDivisorsEnding7;
GroupBDivisorSquaresEnding7 = groupBDivisorSquaresEnding7;
GroupBDivisorsEnding9 = groupBDivisorsEnding9;
GroupBDivisorSquaresEnding9 = groupBDivisorSquaresEnding9;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectCombined(byte ending)
{
return ending switch
{
1 => CombinedDivisorsEnding1,
3 => CombinedDivisorsEnding3,
7 => CombinedDivisorsEnding7,
9 => CombinedDivisorsEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectCombinedSquares(byte ending)
{
return ending switch
{
1 => CombinedDivisorSquaresEnding1,
3 => CombinedDivisorSquaresEnding3,
7 => CombinedDivisorSquaresEnding7,
9 => CombinedDivisorSquaresEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectGroupB(byte ending)
{
return ending switch
{
1 => GroupBDivisorsEnding1,
7 => GroupBDivisorsEnding7,
9 => GroupBDivisorsEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectGroupBSquares(byte ending)
{
return ending switch
{
1 => GroupBDivisorSquaresEnding1,
7 => GroupBDivisorSquaresEnding7,
9 => GroupBDivisorSquaresEnding9,
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectDivisors(HeuristicGpuDivisorTableKind kind, byte ending)
{
return kind switch
{
HeuristicGpuDivisorTableKind.GroupA => GroupADivisors,
HeuristicGpuDivisorTableKind.GroupBEnding1 => GroupBDivisorsEnding1,
HeuristicGpuDivisorTableKind.GroupBEnding7 => GroupBDivisorsEnding7,
HeuristicGpuDivisorTableKind.GroupBEnding9 => GroupBDivisorsEnding9,
HeuristicGpuDivisorTableKind.Combined => SelectCombined(ending),
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public ArrayView1D<ulong, Stride1D.Dense> SelectDivisorSquares(HeuristicGpuDivisorTableKind kind, byte ending)
{
return kind switch
{
HeuristicGpuDivisorTableKind.GroupA => GroupADivisorSquares,
HeuristicGpuDivisorTableKind.GroupBEnding1 => GroupBDivisorSquaresEnding1,
HeuristicGpuDivisorTableKind.GroupBEnding7 => GroupBDivisorSquaresEnding7,
HeuristicGpuDivisorTableKind.GroupBEnding9 => GroupBDivisorSquaresEnding9,
HeuristicGpuDivisorTableKind.Combined => SelectCombinedSquares(ending),
_ => ArrayView1D<ulong, Stride1D.Dense>.Empty,
};
}
internal static void InitializeShared(in HeuristicGpuDivisorTables tables)
{
Unsafe.AsRef(in Shared) = tables;
}
}
@@ -0,0 +1,58 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu;
public sealed class KernelContainer
{
// Serializes first-time initialization of kernels/buffers per accelerator.
public Action<AcceleratorStream, Index1D, ulong, ulong, ArrayView<GpuUInt128>, ArrayView<ulong>>? Order;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ulong, ulong, ulong, ulong, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>>? Incremental;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>,
ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>,
ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>? Pow2Mod;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>? IncrementalOrder;
public Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong,
ResidueAutomatonArgs, ArrayView<int>, ArrayView1D<ulong, Stride1D.Dense>>? Pow2ModOrder;
public Kernel? SmallPrimeFactor;
public Action<AcceleratorStream, Index1D, ulong, ArrayView1D<ulong, Stride1D.Dense>, int, MontgomeryDivisorData, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>? SpecialMax;
// Optional device buffer with small divisor cycles (<= 4M). Index = divisor, value = cycle length.
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallCycles;
public MemoryBuffer1D<uint, Stride1D.Dense>? SmallPrimesLastOne;
public MemoryBuffer1D<uint, Stride1D.Dense>? SmallPrimesLastSeven;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimesPow2LastOne;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimesPow2LastSeven;
public MemoryBuffer1D<uint, Stride1D.Dense>? SmallPrimeFactorsPrimes;
public MemoryBuffer1D<ulong, Stride1D.Dense>? SmallPrimeFactorsSquares;
public void Dispose()
{
// Order = null;
// Incremental = null;
// Pow2Mod = null;
// IncrementalOrder = null;
// Pow2ModOrder = null;
// SmallPrimeFactor = null;
// SpecialMax = null;
// SmallCycles?.Dispose();
// SmallPrimesLastOne?.Dispose();
// SmallPrimesLastSeven?.Dispose();
// SmallPrimesPow2LastOne?.Dispose();
// SmallPrimesPow2LastSeven?.Dispose();
// SmallPrimeFactorsPrimes?.Dispose();
// SmallPrimeFactorsSquares?.Dispose();
}
}
+14
View File
@@ -0,0 +1,14 @@
namespace PerfectNumbers.Core.Gpu;
[Flags]
public enum KernelType
{
None,
SmallCycles,
SmallPrimes,
Pow2ModKernelScan,
OrderKernelScan,
IncrementalKernelScan,
IncrementalOrderKernelScan,
Pow2ModOrderKernelScan
}
@@ -1,7 +1,4 @@
using ILGPU; using ILGPU;
using ILGPU.Algorithms;
using ILGPU.Runtime;
using PerfectNumbers.Core;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -1,6 +1,5 @@
using ILGPU; using ILGPU;
using ILGPU.Algorithms; using ILGPU.Algorithms;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -1,5 +1,4 @@
using ILGPU; using ILGPU;
using ILGPU.Algorithms;
using ILGPU.Runtime; using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -1,7 +1,5 @@
using System;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using ILGPU; using ILGPU;
using ILGPU.Runtime;
using UInt128 = System.UInt128; using UInt128 = System.UInt128;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -305,7 +303,7 @@ internal static class LucasLehmerKernels
states[idx] = s; states[idx] = s;
} }
public static void Kernel(Index1D index, ulong exponent, GpuUInt128 modulus, ArrayView<GpuUInt128> state) public static void Kernel(Index1D _, ulong exponent, GpuUInt128 modulus, ArrayView<GpuUInt128> state)
{ {
// Lucas–Lehmer iteration in the field GF(2^p-1). // Lucas–Lehmer iteration in the field GF(2^p-1).
// For p < 128, use a Mersenne-specific squaring + reduction to avoid // For p < 128, use a Mersenne-specific squaring + reduction to avoid
@@ -1,5 +1,6 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using ILGPU; using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -32,7 +33,7 @@ internal static class NttBarrettKernels
} }
// Barrett-based squaring: v = v^2 mod n for 128-bit n // Barrett-based squaring: v = v^2 mod n for 128-bit n
public static void SquareBarrett128Kernel(Index1D index, ArrayView<GpuUInt128> data, ulong modHigh, ulong modLow, ulong muHigh, ulong muLow) public static void SquareBarrett128Kernel(Index1D index, AcceleratorStream stream, ArrayView<GpuUInt128> data, ulong modHigh, ulong modLow, ulong muHigh, ulong muLow)
{ {
var v = data[index]; var v = data[index];
Mul128Full(v.High, v.Low, v.High, v.Low, out var z3, out var z2, out var z1, out var z0); Mul128Full(v.High, v.Low, v.High, v.Low, out var z3, out var z2, out var z1, out var z0);
@@ -1,5 +1,6 @@
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using ILGPU; using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -43,7 +44,7 @@ internal static class NttMontgomeryKernels
data[index] = new GpuUInt128(0UL, v); data[index] = new GpuUInt128(0UL, v);
} }
public static void SquareMont64Kernel(Index1D index, ArrayView<GpuUInt128> data, ulong modulus, ulong nPrime) public static void SquareMont64Kernel(Index1D index, AcceleratorStream stream, ArrayView<GpuUInt128> data, ulong modulus, ulong nPrime)
{ {
ulong aR = data[index].Low; ulong aR = data[index].Low;
ulong v = MontMul64(aR, aR, modulus, nPrime); ulong v = MontMul64(aR, aR, modulus, nPrime);
@@ -1,5 +1,4 @@
using ILGPU; using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -1,5 +1,4 @@
using ILGPU; using ILGPU;
using ILGPU.Algorithms;
using ILGPU.Runtime; using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -1,26 +1,80 @@
using System.Runtime.CompilerServices;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
using PerfectNumbers.Core;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu.Kernels;
internal static class Pow2MontgomeryKernels internal static class Pow2MontgomeryKernels
{ {
public static void Pow2MontgomeryKernelKeepMontgomery( [MethodImpl(MethodImplOptions.AggressiveInlining)]
Index1D index, private static GpuUInt128 AdvancePolynomialGpu(GpuUInt128 x, ulong c, ulong modulus)
ArrayView1D<ulong, Stride1D.Dense> exponents, {
MontgomeryDivisorData divisor, x.Pow2();
ArrayView1D<ulong, Stride1D.Dense> results) x.AddMod(c, modulus);
{ return x;
results[index] = ULongExtensions.Pow2MontgomeryModWindowedGpuKeepMontgomery(divisor, exponents[index]); }
}
public static void Pow2MontgomeryKernelConvertToStandard( public static void Pow2MontgomeryKernelKeepMontgomery(
Index1D index, Index1D index,
ArrayView1D<ulong, Stride1D.Dense> exponents, ArrayView1D<ulong, Stride1D.Dense> exponents,
MontgomeryDivisorData divisor, MontgomeryDivisorData divisor,
ArrayView1D<ulong, Stride1D.Dense> results) ArrayView1D<ulong, Stride1D.Dense> results)
{ {
results[index] = ULongExtensions.Pow2MontgomeryModWindowedGpuConvertToStandard(divisor, exponents[index]); results[0] = ULongExtensions.Pow2MontgomeryModWindowedGpuKeepMontgomery(divisor, exponents[0]);
} }
public static void Pow2MontgomeryKernelConvertToStandard(
Index1D index,
ArrayView1D<ulong, Stride1D.Dense> exponents,
MontgomeryDivisorData divisor,
ArrayView1D<ulong, Stride1D.Dense> results)
{
results[0] = ULongExtensions.Pow2MontgomeryModWindowedGpuConvertToStandard(divisor, exponents[0]);
}
public static void TryPollardRhoKernel(ulong n, int limit, ArrayView1D<ulong, Stride1D.Dense> randomState, ArrayView1D<byte, Stride1D.Dense> factored, ArrayView1D<ulong, Stride1D.Dense> factor)
{
factored[0] = 0;
if ((n & 1UL) == 0UL)
{
factor[0] = 2UL;
factored[0] = 1;
return;
}
int attempt = 0;
GpuUInt128 nGpuUInt128 = (GpuUInt128)n;
DeterministicRandomGpu random = new(randomState[0]);
while (attempt < limit)
{
ulong c = (random.NextUInt64() % (n - 1UL)) + 1UL;
GpuUInt128 x = (random.NextUInt64() % (n - 2UL)) + 2UL;
GpuUInt128 y = x;
GpuUInt128 d = GpuUInt128.One;
while (d.CompareTo(GpuUInt128.One) == 0)
{
x = AdvancePolynomialGpu(x, c, modulus: n);
y = AdvancePolynomialGpu(y, c, n);
y = AdvancePolynomialGpu(y, c, n);
GpuUInt128 diff = x.CompareTo(y) > 0 ? x - y : y - x;
d = GpuUInt128.BinaryGcd(diff, nGpuUInt128);
}
if (d != n)
{
if (d.High == 0)
{
factor[0] = d.Low;
factored[0] = 1;
return;
}
}
attempt++;
}
randomState[0] = random.State;
}
} }
File diff suppressed because it is too large. Load diff
@@ -1,6 +1,5 @@
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
using PerfectNumbers.Core;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
@@ -13,7 +12,7 @@ internal static partial class PrimeOrderGpuHeuristics
int factorCount, int factorCount,
MontgomeryDivisorData divisor, MontgomeryDivisorData divisor,
ArrayView1D<ulong, Stride1D.Dense> candidates, ArrayView1D<ulong, Stride1D.Dense> candidates,
ArrayView1D<ushort, Stride1D.Dense> resultOut) ArrayView1D<ulong, Stride1D.Dense> resultOut)
{ {
if (index != 0) if (index != 0)
{ {
@@ -62,7 +61,7 @@ internal static partial class PrimeOrderGpuHeuristics
var stepper = new ExponentRemainderStepperGpu(divisorPartial); var stepper = new ExponentRemainderStepperGpu(divisorPartial);
if (stepper.InitializeIsUnityGpu(candidates[0])) if (stepper.InitializeIsUnityGpu(candidates[0]))
{ {
resultOut[0] = 0; Atomic.Exchange(ref resultOut[0], 0);
return; return;
} }
@@ -70,11 +69,11 @@ internal static partial class PrimeOrderGpuHeuristics
{ {
if (stepper.ComputeNextIsUnityGpu(candidates[i])) if (stepper.ComputeNextIsUnityGpu(candidates[i]))
{ {
resultOut[0] = 0; Atomic.Exchange(ref resultOut[0], 0);
return; return;
} }
} }
resultOut[0] = 1; Atomic.Exchange(ref resultOut[0], 1);
} }
} }
@@ -2,41 +2,39 @@ using System.Runtime.CompilerServices;
using ILGPU; using ILGPU;
using ILGPU.Algorithms; using ILGPU.Algorithms;
using ILGPU.Runtime; using ILGPU.Runtime;
using PerfectNumbers.Core.Gpu.Accelerators;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
internal enum HeuristicGpuDivisorTableKind : byte public enum HeuristicGpuDivisorTableKind : byte
{ {
GroupA = 0, GroupA = 0,
GroupBEnding1 = 1, GroupBEnding1 = 1,
GroupBEnding7 = 7, GroupBEnding7 = 7,
GroupBEnding9 = 9, GroupBEnding9 = 9,
Combined = 255,
} }
internal static class PrimeTesterKernels internal static class PrimeTesterKernels
{ {
// GPU kernel: small-prime sieve only. Returns 1 if passes sieve (probable prime), 0 otherwise. // GPU kernel: small-prime sieve only. Returns 1 if passes sieve (probable prime), 0 otherwise.
public static void SmallPrimeSieveKernel( public static void SmallPrimeSieveKernel(
Index1D index, Index1D index,
ArrayView<ulong> numbers, ArrayView<ulong> numbers,
ArrayView<uint> smallPrimes, ArrayView<uint> smallPrimesLastOne,
ArrayView<uint> smallPrimesLastOne, ArrayView<uint> smallPrimesLastSeven,
ArrayView<uint> smallPrimesLastSeven, ArrayView<uint> smallPrimesLastThree,
ArrayView<uint> smallPrimesLastThree, ArrayView<uint> smallPrimesLastNine,
ArrayView<uint> smallPrimesLastNine, ArrayView<ulong> smallPrimesPow2LastOne,
ArrayView<ulong> smallPrimesPow2, ArrayView<ulong> smallPrimesPow2LastSeven,
ArrayView<ulong> smallPrimesPow2LastOne, ArrayView<ulong> smallPrimesPow2LastThree,
ArrayView<ulong> smallPrimesPow2LastSeven, ArrayView<ulong> smallPrimesPow2LastNine,
ArrayView<ulong> smallPrimesPow2LastThree, ArrayView<byte> results)
ArrayView<ulong> smallPrimesPow2LastNine,
ArrayView<byte> results)
{ {
ulong n = numbers[index]; ulong n = numbers[index];
byte result = 1; byte result = 1;
ArrayView<uint> primes = smallPrimes; ArrayView<uint> primes;
ArrayView<ulong> primeSquares = smallPrimesPow2; ArrayView<ulong> primeSquares;
ulong lastDigit = n % 10UL; ulong lastDigit = n % 10UL;
switch (lastDigit) switch (lastDigit)
{ {
@@ -55,7 +53,9 @@ internal static class PrimeTesterKernels
case 9UL: case 9UL:
primes = smallPrimesLastNine; primes = smallPrimesLastNine;
primeSquares = smallPrimesPow2LastNine; primeSquares = smallPrimesPow2LastNine;
break; break;
default:
return;
} }
int length = (int)primes.Length; int length = (int)primes.Length;
@@ -78,17 +78,49 @@ internal static class PrimeTesterKernels
results[index] = result; results[index] = result;
} }
public static void HeuristicTrialDivisionKernel( public static void HeuristicTrialCombinedDivisionKernel(
Index1D index,
byte nMod10,
ArrayView<int> resultFlag,
ulong n,
ulong maxDivisorSquare,
HeuristicCombinedGpuViews tables)
{
// byte nMod10 = (byte)(n % 10UL);
ArrayView1D<ulong, Stride1D.Dense> divisors = tables.SelectDivisors(nMod10);
ArrayView1D<ulong, Stride1D.Dense> divisorSquares = tables.SelectDivisorSquares(nMod10);
int divisorLength = (int)divisorSquares.Length;
int threadIndex = index;
if (threadIndex >= divisorLength)
{
return;
}
ulong divisorSquare = divisorSquares[threadIndex];
if (divisorSquare > maxDivisorSquare)
{
return;
}
ulong divisor = divisors[threadIndex];
divisor = n % divisor;
if (divisor == 0UL)
{
resultFlag[0] = 1;
}
}
public static void HeuristicTrialGroupABDivisionKernel(
Index1D index, Index1D index,
ArrayView<int> resultFlag, ArrayView<int> resultFlag,
ulong n, ulong n,
ulong maxDivisorSquare, ulong maxDivisorSquare,
HeuristicGpuDivisorTableKind tableKind, HeuristicGpuDivisorTableKind tableKind,
HeuristicGpuDivisorTables tables) HeuristicGroupABGpuViews tables)
{ {
byte nMod10 = (byte)(n % 10UL); ArrayView1D<ulong, Stride1D.Dense> divisors = tables.SelectDivisors(tableKind);
ArrayView1D<ulong, Stride1D.Dense> divisors = tables.SelectDivisors(tableKind, nMod10); ArrayView1D<ulong, Stride1D.Dense> divisorSquares = tables.SelectDivisorSquares(tableKind);
ArrayView1D<ulong, Stride1D.Dense> divisorSquares = tables.SelectDivisorSquares(tableKind, nMod10);
int divisorLength = (int)divisorSquares.Length; int divisorLength = (int)divisorSquares.Length;
int threadIndex = index; int threadIndex = index;
@@ -121,35 +153,37 @@ internal static class PrimeTesterKernels
private static ulong BinaryGcdGpu(ulong u, ulong v) private static ulong BinaryGcdGpu(ulong u, ulong v)
{ {
// TODO: Replace this inline GPU binary GCD with the kernel extracted from
// GpuUInt128BinaryGcdBenchmarks via GpuKernelPool so device callers reuse the
// fully unrolled ladder instead of this branchy fallback.
bool uIsZero = u == 0UL; bool uIsZero = u == 0UL;
bool vIsZero = v == 0UL; bool vIsZero = v == 0UL;
bool eitherZero = uIsZero | vIsZero; bool eitherZero = uIsZero | vIsZero;
ulong zeroResult = uIsZero ? v : u; ulong zeroResult = uIsZero ? v : u;
ulong combined = u | v | 1UL; ulong combined = (u | v) | 1UL;
int shift = XMath.TrailingZeroCount(combined); int shift = XMath.TrailingZeroCount(combined);
ulong currentU = u >> XMath.TrailingZeroCount(u | 1UL);
ulong currentV = v >> XMath.TrailingZeroCount(v | 1UL);
ulong normalizedU = u >> XMath.TrailingZeroCount(u | 1UL); while (true)
ulong normalizedV = v >> XMath.TrailingZeroCount(v | 1UL);
ulong currentU = normalizedU;
ulong currentV = normalizedV;
bool loopCondition = currentV != 0UL;
while (loopCondition)
{ {
currentV >>= XMath.TrailingZeroCount(currentV); currentV >>= XMath.TrailingZeroCount(currentV);
ulong minValue = XMath.Min(currentU, currentV); bool swap = currentU > currentV;
ulong maxValue = XMath.Max(currentU, currentV); ulong minValue = ConditionalSelect(currentU, currentV, swap);
currentU = minValue; ulong maxValue = ConditionalSelect(currentV, currentU, swap);
currentV = maxValue - minValue; currentV = maxValue - minValue;
loopCondition = currentV != 0UL; currentU = minValue;
if (currentV == 0UL)
{
ulong gcd = currentU << shift;
return eitherZero ? zeroResult : gcd;
}
} }
ulong gcd = currentU << shift;
return eitherZero ? zeroResult : gcd;
} }
[MethodImpl(MethodImplOptions.AggressiveInlining)]
private static ulong ConditionalSelect(ulong left, ulong right, bool useRight)
{
ulong mask = useRight ? ulong.MaxValue : 0UL;
return left ^ ((left ^ right) & mask);
}
} }
File diff suppressed because it is too large. Load diff
@@ -1,4 +1,3 @@
using System.Collections.Concurrent;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
@@ -9,127 +8,157 @@ namespace PerfectNumbers.Core.Gpu;
/// </summary> /// </summary>
public sealed class MersenneNumberDivisorGpuTester public sealed class MersenneNumberDivisorGpuTester
{ {
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ulong, ReadOnlyGpuUInt128, ArrayView<byte>>> _kernelCache = new(); [ThreadStatic]
private readonly ConcurrentDictionary<Accelerator, MemoryBuffer1D<byte, Stride1D.Dense>> _resultBuffers = new(); private static Dictionary<Accelerator, Queue<MemoryBuffer1D<byte, Stride1D.Dense>>>? _resultBuffers;
private Action<Index1D, ulong, ReadOnlyGpuUInt128, ArrayView<byte>> GetKernel(Accelerator accelerator) => private static Queue<MemoryBuffer1D<byte, Stride1D.Dense>> GetQueue(Accelerator accelerator)
_kernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ulong, ReadOnlyGpuUInt128, ArrayView<byte>>(DivisorKernels.Kernel)); {
var pool = _resultBuffers ??= [];
if (!pool.TryGetValue(accelerator, out var bufferQueue))
{
bufferQueue = [];
pool[accelerator] = bufferQueue;
}
public static void BuildDivisorCandidates() return bufferQueue;
{ }
ulong[] snapshot = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot();
(ulong divisor, uint cycle)[] list = new (ulong divisor, uint cycle)[snapshot.Length / 2];
ulong cycle;
int count = 0, i, snapshotLength = snapshot.Length;
for (i = 3; i < snapshotLength; i += 2)
{
cycle = snapshot[i];
if (cycle == 0U)
{
continue;
}
list[count++] = ((ulong)i, (uint)cycle); private static Action<AcceleratorStream, Index1D, ulong, ReadOnlyGpuUInt128, ArrayView<byte>> GetDivisorKernel(Accelerator accelerator)
} {
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, ReadOnlyGpuUInt128, ArrayView<byte>>(DivisorKernels.Kernel);
_divisorCandidates = count == 0 ? [] : list[..count]; var kernel = KernelUtil.GetKernel(loaded);
}
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ReadOnlyGpuUInt128, ArrayView<byte>>>();
}
public static void BuildDivisorCandidates()
{
ulong[] snapshot = MersenneDivisorCycles.Shared.ExportSmallCyclesSnapshot();
(ulong divisor, uint cycle)[] list = new (ulong divisor, uint cycle)[snapshot.Length / 2];
ulong cycle;
int count = 0, i, snapshotLength = snapshot.Length;
for (i = 3; i < snapshotLength; i += 2)
{
cycle = snapshot[i];
if (cycle == 0U)
{
continue;
}
list[count++] = ((ulong)i, (uint)cycle);
}
_divisorCandidates = count == 0 ? [] : list[..count];
}
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
public bool IsDivisible(ulong exponent, in ReadOnlyGpuUInt128 divisor)
{
// var accelerator = SharedGpuContext.CreateAccelerator();
var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = _accelerators[acceleratorIndex];
var stream = accelerator.CreateStream();
Queue<MemoryBuffer1D<byte, Stride1D.Dense>> resultBuffers = GetQueue(accelerator);
if (!resultBuffers.TryDequeue(out var resultBuffer))
{
// lock(accelerator)
{
resultBuffer = accelerator.Allocate1D<byte>(1);
}
}
public bool IsDivisible(ulong exponent, in ReadOnlyGpuUInt128 divisor)
{
var gpu = GpuContextPool.Rent();
var accelerator = gpu.Accelerator;
var kernel = GetKernel(accelerator);
var resultBuffer = _resultBuffers.GetOrAdd(accelerator, acc => acc.Allocate1D<byte>(1));
// There is no point in clearing this buffer. We always override item [0] and never use it beyond item [0] // There is no point in clearing this buffer. We always override item [0] and never use it beyond item [0]
// resultBuffer.MemSetToZero(); // resultBuffer.MemSetToZero(stream);
kernel(1, exponent, divisor, resultBuffer.View); var divisorKernel = GetDivisorKernel(accelerator);
accelerator.Synchronize(); divisorKernel(stream, 1, exponent, divisor, resultBuffer.View);
Span<byte> result = stackalloc byte[1]; byte result = 0;
resultBuffer.View.CopyToCPU(ref result[0], 1); resultBuffer.View.CopyToCPU(stream, ref result, 1);
bool divisible = result[0] != 0; stream.Synchronize();
result[0] = 0; bool divisible = result != 0;
resultBuffer.View.CopyFromCPU(ref result[0], 1); resultBuffers.Enqueue(resultBuffer);
accelerator.Synchronize(); stream.Dispose();
gpu.Dispose(); // AcceleratorPool.Shared.Return(accelerator);
return divisible; // accelerator.Dispose();
} return divisible;
}
private static (ulong divisor, uint cycle)[]? _divisorCandidates = Array.Empty<(ulong divisor, uint cycle)>(); private static (ulong divisor, uint cycle)[]? _divisorCandidates = [];
public bool IsPrime(ulong p, UInt128 d, ulong divisorCyclesSearchLimit, out bool divisorsExhausted) public bool IsPrime(ulong p, UInt128 d, ulong divisorCyclesSearchLimit, out bool divisorsExhausted)
{ {
ReadOnlyGpuUInt128 readOnlyDivisor; ReadOnlyGpuUInt128 readOnlyDivisor;
if (d != UInt128.Zero) if (d != UInt128.Zero)
{ {
readOnlyDivisor = new ReadOnlyGpuUInt128(d); readOnlyDivisor = new ReadOnlyGpuUInt128(d);
if (IsDivisible(p, in readOnlyDivisor)) if (IsDivisible(p, in readOnlyDivisor))
{ {
divisorsExhausted = true; divisorsExhausted = true;
return false; return false;
} }
divisorsExhausted = false; divisorsExhausted = false;
return true; return true;
} }
if (_divisorCandidates is { Length: > 0 } candidates) if (_divisorCandidates is { Length: > 0 } candidates)
{ {
int candidateCount = candidates.Length; int candidateCount = candidates.Length;
for (int index = 0; index < candidateCount; index++) for (int index = 0; index < candidateCount; index++)
{ {
(ulong candidateDivisor, uint cycle) = candidates[index]; (ulong candidateDivisor, uint cycle) = candidates[index];
if (p % cycle != 0UL) if (p % cycle != 0UL)
{ {
continue; continue;
} }
readOnlyDivisor = new ReadOnlyGpuUInt128(candidateDivisor); // Reusing readOnlyDivisor for candidate divisors. readOnlyDivisor = new ReadOnlyGpuUInt128(candidateDivisor); // Reusing readOnlyDivisor for candidate divisors.
if (IsDivisible(p, in readOnlyDivisor)) if (IsDivisible(p, in readOnlyDivisor))
{ {
divisorsExhausted = true; divisorsExhausted = true;
return false; return false;
} }
} }
} }
UInt128 kMul2; UInt128 kMul2;
var divisorCycles = MersenneDivisorCycles.Shared; var divisorCycles = MersenneDivisorCycles.Shared;
UInt128 maxK2 = UInt128.MaxValue / ((UInt128)p << 1); UInt128 maxK2 = UInt128.MaxValue / ((UInt128)p << 1);
ulong limit = divisorCyclesSearchLimit; ulong limit = divisorCyclesSearchLimit;
if ((UInt128)limit > maxK2) if ((UInt128)limit > maxK2)
{ {
limit = (ulong)maxK2; limit = (ulong)maxK2;
} }
for (ulong k2 = 1UL; k2 <= limit; k2++) for (ulong k2 = 1UL; k2 <= limit; k2++)
{ {
kMul2 = (UInt128)k2 << 1; kMul2 = (UInt128)k2 << 1;
UInt128 candidate = checked(kMul2 * p); UInt128 candidate = checked(kMul2 * p);
d = checked(candidate + UInt128.One); d = checked(candidate + UInt128.One);
if (p < 64UL && d == ((UInt128)1 << (int)p) - UInt128.One) if (p < 64UL && d == ((UInt128)1 << (int)p) - UInt128.One)
{ {
continue; continue;
} }
UInt128 cycle128 = MersenneDivisorCycles.GetCycle(d); UInt128 cycle128 = MersenneDivisorCycles.GetCycle(d);
if ((UInt128)p % cycle128 != UInt128.Zero) if ((UInt128)p % cycle128 != UInt128.Zero)
{ {
continue; continue;
} }
readOnlyDivisor = new ReadOnlyGpuUInt128(d); // Reusing readOnlyDivisor for generated divisors. readOnlyDivisor = new ReadOnlyGpuUInt128(d); // Reusing readOnlyDivisor for generated divisors.
if (IsDivisible(p, in readOnlyDivisor)) if (IsDivisible(p, in readOnlyDivisor))
{ {
divisorsExhausted = true; divisorsExhausted = true;
return false; return false;
} }
} }
divisorsExhausted = (UInt128)divisorCyclesSearchLimit >= maxK2; divisorsExhausted = (UInt128)divisorCyclesSearchLimit >= maxK2;
return true; return true;
} }
} }
@@ -12,36 +12,41 @@ public class MersenneNumberIncrementalGpuTester(GpuKernelType kernelType, bool u
private readonly GpuKernelType _kernelType = kernelType; private readonly GpuKernelType _kernelType = kernelType;
private readonly bool _useGpuOrder = useGpuOrder; private readonly bool _useGpuOrder = useGpuOrder;
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
public void Scan(ulong exponent, UInt128 twoP, LastDigit lastDigit, UInt128 maxK, ref bool isPrime) public void Scan(ulong exponent, UInt128 twoP, LastDigit lastDigit, UInt128 maxK, ref bool isPrime)
{ {
throw new NotImplementedException($"GPU incremental scanning requires the device cycle heuristics implementation (kernel: {_kernelType}, GPU order: {_useGpuOrder})."); throw new NotImplementedException($"GPU incremental scanning requires the device cycle heuristics implementation (kernel: {_kernelType}, GPU order: {_useGpuOrder}).");
var gpuLease = GpuKernelPool.GetKernel(_useGpuOrder); // GpuPrimeWorkLimiter.Acquire();
var accelerator = gpuLease.Accelerator; var acceleratorIndex = AcceleratorPool.Shared.Rent();
var stream = gpuLease.Stream; var accelerator = _accelerators[acceleratorIndex];
var stream = accelerator.CreateStream();
int batchSize = GpuConstants.ScanBatchSize; // large batch improves GPU occupancy int batchSize = GpuConstants.ScanBatchSize; // large batch improves GPU occupancy
UInt128 kStart = 1UL; UInt128 kStart = 1UL;
ulong divMul = (ulong)((((UInt128)1 << 64) - UInt128.One) / exponent) + 1UL; ulong divMul = (ulong)((((UInt128)1 << 64) - UInt128.One) / exponent) + 1UL;
byte last = lastDigit == LastDigit.Seven ? (byte)1 : (byte)0; // ILGPU kernels do not support bool parameters byte last = lastDigit == LastDigit.Seven ? (byte)1 : (byte)0; // ILGPU kernels do not support bool parameters
var pow2Kernel = gpuLease.Pow2ModKernel; KernelContainer kernels = GpuKernelPool.GetOrAddKernels(acceleratorIndex, stream, KernelType.Pow2ModKernelScan | KernelType.IncrementalKernelScan);
var incKernel = gpuLease.IncrementalKernel; var pow2Kernel = kernels.Pow2Mod!;
var incKernel = kernels.Incremental!;
ulong step10 = (exponent.Mod10() << 1).Mod10(); ulong step10 = (exponent.Mod10() << 1).Mod10();
ulong step8 = ((exponent & 7UL) << 1) & 7UL; ulong step8 = ((exponent & 7UL) << 1) & 7UL;
ulong step3 = ((exponent % 3UL) << 1) % 3UL; ulong step3 = ((exponent % 3UL) << 1) % 3UL;
ulong step5 = ((exponent % 5UL) << 1) % 5UL; ulong step5 = ((exponent % 5UL) << 1) % 5UL;
GpuUInt128 twoPGpu = (GpuUInt128)twoP; GpuUInt128 twoPGpu = (GpuUInt128)twoP;
var smallCyclesView = GpuKernelPool.EnsureSmallCyclesOnDevice(gpuLease.Kernels, accelerator); var smallCyclesView = GpuKernelPool.GetSmallCyclesOnDevice(kernels);
ResiduePrimeViews primeViews = default; ResiduePrimeViews primeViews = default;
if (_kernelType == GpuKernelType.Pow2Mod) if (_kernelType == GpuKernelType.Pow2Mod)
{ {
primeViews = GpuKernelPool.EnsureSmallPrimesOnDevice(gpuLease.Kernels, accelerator); primeViews = GpuKernelPool.GetSmallPrimesOnDevice(kernels);
} }
var orderBuffer = accelerator.Allocate1D<ulong>(batchSize); var orderBuffer = accelerator.Allocate1D<ulong>(batchSize);
// Avoid giant stack allocations that can trigger StackOverflow when batchSize is large. // Avoid giant stack allocations that can trigger StackOverflow when batchSize is large.
// Rent a reusable array from the shared pool instead. // Rent a reusable array from the shared pool instead.
ulong[] orderArray = ArrayPool<ulong>.Shared.Rent(batchSize); ArrayPool<ulong> ulongPool = ThreadStaticPools.UlongPool;
ulong[] orderArray = ulongPool.Rent(batchSize);
UInt128 remaining; UInt128 remaining;
int currentSize; int currentSize;
int i; int i;
@@ -92,8 +97,8 @@ public class MersenneNumberIncrementalGpuTester(GpuKernelType kernelType, bool u
smallCyclesView); smallCyclesView);
} }
orderBuffer.View.CopyToCPU(stream, ref MemoryMarshal.GetReference(orders), currentSize);
stream.Synchronize(); stream.Synchronize();
orderBuffer.View.CopyToCPU(ref MemoryMarshal.GetReference(orders), currentSize);
if (_kernelType == GpuKernelType.Pow2Mod) if (_kernelType == GpuKernelType.Pow2Mod)
{ {
for (i = 0; i < currentSize; i++) for (i = 0; i < currentSize; i++)
@@ -120,8 +125,9 @@ public class MersenneNumberIncrementalGpuTester(GpuKernelType kernelType, bool u
kStart += (UInt128)currentSize; kStart += (UInt128)currentSize;
} }
ArrayPool<ulong>.Shared.Return(orderArray); stream.Dispose();
ulongPool.Return(orderArray);
orderBuffer.Dispose(); orderBuffer.Dispose();
gpuLease.Dispose(); // GpuPrimeWorkLimiter.Release();
} }
} }
@@ -8,32 +8,125 @@ namespace PerfectNumbers.Core.Gpu;
public class MersenneNumberLucasLehmerGpuTester public class MersenneNumberLucasLehmerGpuTester
{ {
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>>> KernelCache = new(); [ThreadStatic]
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ulong, ArrayView<GpuUInt128>>> AddSmallKernelCache = new(); private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>>>? _lucasLehmerKernel;
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ulong, ArrayView<GpuUInt128>>> SubSmallKernelCache = new();
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ulong, ArrayView<GpuUInt128>>> ReduceKernelCache = new();
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ArrayView<byte>>> IsZeroKernelCache = new();
private readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>>> BatchKernelCache = new();
private Action<Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>> GetKernel(Accelerator accelerator) [ThreadStatic]
{ private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>>>? _addSmallKernel;
return KernelCache.GetOrAdd(accelerator, accel => accel.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>>(LucasLehmerKernels.Kernel));
}
private Action<Index1D, ulong, ArrayView<GpuUInt128>> GetAddSmallKernel(Accelerator accelerator) => [ThreadStatic]
AddSmallKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView<GpuUInt128>>(LucasLehmerKernels.AddSmallKernel)); private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>>>? _subSmallKernel;
private Action<Index1D, ulong, ArrayView<GpuUInt128>> GetSubSmallKernel(Accelerator accelerator) => [ThreadStatic]
SubSmallKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView<GpuUInt128>>(LucasLehmerKernels.SubtractSmallKernel)); private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>>>? _reduceKernel;
private Action<Index1D, ulong, ArrayView<GpuUInt128>> GetReduceKernel(Accelerator accelerator) => [ThreadStatic]
ReduceKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView<GpuUInt128>>(LucasLehmerKernels.ReduceModMersenneKernel)); private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<byte>>>? _isZeroKernel;
private Action<Index1D, ArrayView<GpuUInt128>, ArrayView<byte>> GetIsZeroKernel(Accelerator accelerator) => [ThreadStatic]
IsZeroKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<byte>>(LucasLehmerKernels.IsZeroKernel)); private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>>>? _lucasLehmerBatchKernel;
private Action<Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>> GetBatchKernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>> GetLucasLehmerKernel(Accelerator accelerator)
BatchKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>>(LucasLehmerKernels.KernelBatch)); {
var pool = _lucasLehmerKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>>(LucasLehmerKernels.Kernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, GpuUInt128, ArrayView<GpuUInt128>>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>> GetAddSmallKernel(Accelerator accelerator)
{
var pool = _addSmallKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView<GpuUInt128>>(LucasLehmerKernels.AddSmallKernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>> GetSubSmallKernel(Accelerator accelerator)
{
var pool = _subSmallKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView<GpuUInt128>>(LucasLehmerKernels.SubtractSmallKernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>> GetReduceKernel(Accelerator accelerator)
{
var pool = _reduceKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, ArrayView<GpuUInt128>>(LucasLehmerKernels.ReduceModMersenneKernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ulong, ArrayView<GpuUInt128>>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<byte>> GetIsZeroKernel(Accelerator accelerator)
{
var pool = _isZeroKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<byte>>(LucasLehmerKernels.IsZeroKernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<byte>>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>> GetLucasLehmerBatchKernel(Accelerator accelerator)
{
var pool = _lucasLehmerBatchKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>>(LucasLehmerKernels.KernelBatch);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<ulong>, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>>>();
pool[accelerator] = cached;
return cached;
}
// Configurable LL slice size to keep kernels short. Default 32. // Configurable LL slice size to keep kernels short. Default 32.
public int SliceSize = 32; public int SliceSize = 32;
@@ -54,13 +147,11 @@ public class MersenneNumberLucasLehmerGpuTester
return false; return false;
} }
GpuPrimeWorkLimiter.Acquire();
bool result; bool result;
if (exponent >= 128UL) if (exponent >= 128UL)
{ {
if (!TryGetNttParameters(exponent, out var nttMod, out var primitiveRoot)) if (!TryGetNttParameters(exponent, out var nttMod, out var primitiveRoot))
{ {
GpuPrimeWorkLimiter.Release();
throw new NotSupportedException("NTT parameters for the given exponent are not available."); throw new NotSupportedException("NTT parameters for the given exponent are not available.");
} }
@@ -68,19 +159,22 @@ public class MersenneNumberLucasLehmerGpuTester
} }
else else
{ {
var gpu = GpuContextPool.Rent(); // GpuPrimeWorkLimiter.Acquire();
var accelerator = gpu.Accelerator; var acceleratorIndex = AcceleratorPool.Shared.Rent();
var kernel = GetKernel(accelerator); var accelerator = _accelerators[acceleratorIndex];
var stream = accelerator.CreateStream();
var modulus = new GpuUInt128(((UInt128)1 << (int)exponent) - 1UL); // TODO: Cache these Mersenne moduli per exponent so LL GPU runs skip rebuilding them every launch. var modulus = new GpuUInt128(((UInt128)1 << (int)exponent) - 1UL); // TODO: Cache these Mersenne moduli per exponent so LL GPU runs skip rebuilding them every launch.
var buffer = accelerator.Allocate1D<GpuUInt128>(1); var buffer = accelerator.Allocate1D<GpuUInt128>(1);
kernel(1, exponent, modulus, buffer.View); var lucasLehmerKernel = GetLucasLehmerKernel(accelerator);
accelerator.Synchronize(); lucasLehmerKernel(stream, 1, exponent, modulus, buffer.View);
stream.Synchronize();
// TODO: Replace this with .CopyToCPU with pre-allocated buffer using stream. Synchronize on the stream after copy.
result = buffer.GetAsArray1D()[0].IsZero; result = buffer.GetAsArray1D()[0].IsZero;
buffer.Dispose(); buffer.Dispose();
gpu.Dispose(); stream.Dispose();
} }
GpuPrimeWorkLimiter.Release(); // GpuPrimeWorkLimiter.Release();
return result; return result;
} }
@@ -100,16 +194,18 @@ public class MersenneNumberLucasLehmerGpuTester
} }
} }
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var kernel = GetBatchKernel(accelerator); var stream = accelerator.CreateStream();
var expBuffer = accelerator.Allocate1D<ulong>(count); var expBuffer = accelerator.Allocate1D<ulong>(count);
ulong[] expArray = ArrayPool<ulong>.Shared.Rent(count); ArrayPool<ulong> ulongPool = ThreadStaticPools.UlongPool;
ulong[] expArray = ulongPool.Rent(count);
exponents.CopyTo(expArray); exponents.CopyTo(expArray);
expBuffer.View.CopyFromCPU(ref expArray[0], count); expBuffer.View.CopyFromCPU(stream, ref expArray[0], count);
GpuUInt128[] modulusArray = ArrayPool<GpuUInt128>.Shared.Rent(count); ArrayPool<GpuUInt128> gpuUInt128Pool = ThreadStaticPools.GpuUInt128Pool;
GpuUInt128[] modulusArray = gpuUInt128Pool.Rent(count);
for (int i = 0; i < count; i++) for (int i = 0; i < count; i++)
{ {
// TODO: Replace this per-exponent shift with a small shared table (one entry per supported exponent < 128) // TODO: Replace this per-exponent shift with a small shared table (one entry per supported exponent < 128)
@@ -118,19 +214,20 @@ public class MersenneNumberLucasLehmerGpuTester
} }
var modBuffer = accelerator.Allocate1D<GpuUInt128>(count); var modBuffer = accelerator.Allocate1D<GpuUInt128>(count);
modBuffer.View.CopyFromCPU(ref modulusArray[0], count); modBuffer.View.CopyFromCPU(stream, ref modulusArray[0], count);
var stateBuffer = accelerator.Allocate1D<GpuUInt128>(count); var stateBuffer = accelerator.Allocate1D<GpuUInt128>(count);
kernel(count, expBuffer.View, modBuffer.View, stateBuffer.View); var lucasLehmerBatchKernel = GetLucasLehmerBatchKernel(accelerator);
accelerator.Synchronize(); lucasLehmerBatchKernel(stream, count, expBuffer.View, modBuffer.View, stateBuffer.View);
stateBuffer.View.CopyToCPU(ref residues[0], count); stateBuffer.View.CopyToCPU(stream, ref residues[0], count);
stream.Synchronize();
stateBuffer.Dispose(); stateBuffer.Dispose();
modBuffer.Dispose(); modBuffer.Dispose();
expBuffer.Dispose(); expBuffer.Dispose();
ArrayPool<GpuUInt128>.Shared.Return(modulusArray); gpuUInt128Pool.Return(modulusArray);
ArrayPool<ulong>.Shared.Return(expArray); ulongPool.Return(expArray);
gpu.Dispose(); stream.Dispose();
} }
public void IsMersennePrimeBatch(ReadOnlySpan<ulong> exponents, Span<bool> results) public void IsMersennePrimeBatch(ReadOnlySpan<ulong> exponents, Span<bool> results)
@@ -141,23 +238,27 @@ public class MersenneNumberLucasLehmerGpuTester
throw new ArgumentException("Result span too small", nameof(results)); throw new ArgumentException("Result span too small", nameof(results));
} }
GpuUInt128[] buffer = ArrayPool<GpuUInt128>.Shared.Rent(count); ArrayPool<GpuUInt128> gpuUInt128Pool = ThreadStaticPools.GpuUInt128Pool;
GpuUInt128[] buffer = gpuUInt128Pool.Rent(count);
ComputeResidues(exponents, buffer.AsSpan(0, count)); ComputeResidues(exponents, buffer.AsSpan(0, count));
for (int i = 0; i < count; i++) for (int i = 0; i < count; i++)
{ {
results[i] = buffer[i].IsZero; results[i] = buffer[i].IsZero;
} }
ArrayPool<GpuUInt128>.Shared.Return(buffer); gpuUInt128Pool.Return(buffer);
} }
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
private bool IsMersennePrimeNtt(ulong exponent, GpuUInt128 nttMod, GpuUInt128 primitiveRoot, bool runOnGpu) private bool IsMersennePrimeNtt(ulong exponent, GpuUInt128 nttMod, GpuUInt128 primitiveRoot, bool runOnGpu)
{ {
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var addKernel = GetAddSmallKernel(accelerator); var stream = accelerator.CreateStream();
var subKernel = GetSubSmallKernel(accelerator); var subKernel = GetSubSmallKernel(accelerator);
var reduceKernel = GetReduceKernel(accelerator); var reduceKernel = GetReduceKernel(accelerator);
var zeroKernel = GetIsZeroKernel(accelerator); var isZeroKernel = GetIsZeroKernel(accelerator);
int limbCount = (int)((exponent + 127UL) / 128UL); int limbCount = (int)((exponent + 127UL) / 128UL);
int target = limbCount * 2; int target = limbCount * 2;
@@ -167,8 +268,9 @@ public class MersenneNumberLucasLehmerGpuTester
transformLength <<= 1; transformLength <<= 1;
} }
var stateBuffer = accelerator.Allocate1D<GpuUInt128>(transformLength); var stateBuffer = accelerator.Allocate1D<GpuUInt128>(transformLength);
stateBuffer.MemSetToZero(); stateBuffer.MemSetToZero(stream);
addKernel(1, 4UL, stateBuffer.View); var addSmallKernel = GetAddSmallKernel(accelerator);
addSmallKernel(stream, 1, 4UL, stateBuffer.View);
// TODO(LL-SLICE): Slice Lucas–Lehmer iterations into short batches to // TODO(LL-SLICE): Slice Lucas–Lehmer iterations into short batches to
// avoid long-running kernels and TDR. Example: process 8–64 iterations // avoid long-running kernels and TDR. Example: process 8–64 iterations
@@ -183,27 +285,26 @@ public class MersenneNumberLucasLehmerGpuTester
for (int s = 0; s < iter; s++) for (int s = 0; s < iter; s++)
{ {
// Square in NTT domain with staged kernels (short runtime per kernel). // Square in NTT domain with staged kernels (short runtime per kernel).
NttGpuMath.SquareDevice(accelerator, stateBuffer.View, nttMod, primitiveRoot); NttGpuMath.SquareDevice(accelerator, stream, stateBuffer.View, nttMod, primitiveRoot);
subKernel(1, 2UL, stateBuffer.View); subKernel(stream, 1, 2UL, stateBuffer.View);
reduceKernel(1, exponent, stateBuffer.View); reduceKernel(stream, 1, exponent, stateBuffer.View);
} }
// Synchronize between slices to yield to scheduler and avoid TDR. // Synchronize between slices to yield to scheduler and avoid TDR.
accelerator.Synchronize();
i += (ulong)iter; i += (ulong)iter;
} }
reduceKernel(1, exponent, stateBuffer.View); reduceKernel(stream, 1, exponent, stateBuffer.View);
var resultBuffer = accelerator.Allocate1D<byte>(1);
isZeroKernel(stream, 1, stateBuffer.View, resultBuffer.View);
Span<byte> result = stackalloc byte[1];
resultBuffer.View.CopyToCPU(stream, result);
stream.Synchronize();
var resultBuffer = accelerator.Allocate1D<byte>(1);
zeroKernel(1, stateBuffer.View, resultBuffer.View);
accelerator.Synchronize();
byte[] result = new byte[1];
resultBuffer.View.CopyToCPU(result);
bool isPrime = result[0] != 0; bool isPrime = result[0] != 0;
resultBuffer.Dispose(); resultBuffer.Dispose();
stateBuffer.Dispose(); stateBuffer.Dispose();
gpu.Dispose(); stream.Dispose();
return isPrime; return isPrime;
} }
@@ -472,6 +573,4 @@ public class MersenneNumberLucasLehmerGpuTester
return result; return result;
} }
} }
@@ -7,36 +7,46 @@ public class MersenneNumberOrderGpuTester(GpuKernelType kernelType, bool useGpuO
private readonly GpuKernelType _kernelType = kernelType; private readonly GpuKernelType _kernelType = kernelType;
private readonly bool _useGpuOrder = useGpuOrder; private readonly bool _useGpuOrder = useGpuOrder;
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
public void Scan(ulong exponent, UInt128 twoP, LastDigit lastDigit, UInt128 maxK, ref bool isPrime) public void Scan(ulong exponent, UInt128 twoP, LastDigit lastDigit, UInt128 maxK, ref bool isPrime)
{ {
var gpuLease = GpuKernelPool.GetKernel(_useGpuOrder); // GpuPrimeWorkLimiter.Acquire();
var accelerator = gpuLease.Accelerator; // var accelerator = SharedGpuContext.Accelerator;
var stream = gpuLease.Stream; var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = _accelerators[acceleratorIndex];
var stream = accelerator.CreateStream();
int batchSize = GpuConstants.ScanBatchSize; // large batch improves GPU occupancy and avoids TDR int batchSize = GpuConstants.ScanBatchSize; // large batch improves GPU occupancy and avoids TDR
UInt128 kStart = UInt128.One; UInt128 kStart = UInt128.One;
ulong divMul = (ulong)((((UInt128)1 << 64) - UInt128.One) / exponent) + 1UL; ulong divMul = (ulong)((((UInt128)1 << 64) - UInt128.One) / exponent) + 1UL;
byte last = lastDigit == LastDigit.Seven ? (byte)1 : (byte)0; // ILGPU kernels do not support bool parameters byte last = lastDigit == LastDigit.Seven ? (byte)1 : (byte)0; // ILGPU kernels do not support bool parameters
KernelContainer kernels = GpuKernelPool.GetOrAddKernels(acceleratorIndex, stream, KernelType.Pow2ModOrderKernelScan | KernelType.IncrementalOrderKernelScan);
var kernel = _kernelType switch var kernel = _kernelType switch
{ {
GpuKernelType.Pow2Mod => gpuLease.Pow2ModOrderKernel, GpuKernelType.Pow2Mod => kernels.Pow2ModOrder!,
_ => gpuLease.IncrementalOrderKernel, _ => kernels.IncrementalOrder!,
}; };
var foundBuffer = accelerator.Allocate1D<int>(1); // TODO: Replace this allocation with the pooled device buffer from GpuOrderKernelBenchmarks so repeated scans reuse the pinned staging memory instead of allocating per run. MemoryBuffer1D<int, ILGPU.Stride1D.Dense>? foundBuffer;
// lock (accelerator)
{
foundBuffer = accelerator.Allocate1D<int>(1); // TODO: Replace this allocation with the pooled device buffer from GpuOrderKernelBenchmarks so repeated scans reuse the pinned staging memory instead of allocating per run.
}
exponent.Mod10_8_5_3Steps(out ulong step10, out ulong step8, out ulong step5, out ulong step3); exponent.Mod10_8_5_3Steps(out ulong step10, out ulong step8, out ulong step5, out ulong step3);
UInt128 remaining; UInt128 remaining;
int currentSize; int currentSize;
int found = 0; int found = 0;
var smallCyclesView = GpuKernelPool.EnsureSmallCyclesOnDevice(gpuLease.Kernels, accelerator); var smallCyclesView = GpuKernelPool.GetSmallCyclesOnDevice(kernels);
while (kStart <= maxK && Volatile.Read(ref isPrime)) while (kStart <= maxK && Volatile.Read(ref isPrime))
{ {
remaining = maxK - kStart + 1UL; remaining = maxK - kStart + 1UL;
currentSize = remaining > (UInt128)batchSize ? batchSize : (int)remaining; currentSize = remaining > (UInt128)batchSize ? batchSize : (int)remaining;
// This clear is required because the result may not always be set. Remove it after the kernels are modified to always set the result. // This clear is required because the result may not always be set. Remove it after the kernels are modified to always set the result.
foundBuffer.MemSetToZero(); foundBuffer.MemSetToZero(stream);
// Precompute residue automaton bases for this batch // Precompute residue automaton bases for this batch
UInt128 q0 = twoP * kStart + 1UL; UInt128 q0 = twoP * kStart + 1UL;
// TODO: Replace these modulo recomputations with the divisor-cycle stepping deltas captured in // TODO: Replace these modulo recomputations with the divisor-cycle stepping deltas captured in
@@ -46,11 +56,13 @@ public class MersenneNumberOrderGpuTester(GpuKernelType kernelType, bool useGpuO
var kernelArgs = new ResidueAutomatonArgs(q0m10, step10, q0m8, step8, q0m3, step3, q0m5, step5); var kernelArgs = new ResidueAutomatonArgs(q0m10, step10, q0m8, step8, q0m3, step3, q0m5, step5);
// Ensure device has small cycles table for early rejections in order kernels // Ensure device has small cycles table for early rejections in order kernels
kernel(stream, currentSize, exponent, (GpuUInt128)twoP, (GpuUInt128)kStart, last, divMul, kernel(
kernelArgs, foundBuffer.View, smallCyclesView); stream, currentSize, exponent, (GpuUInt128)twoP, (GpuUInt128)kStart,
last, divMul, kernelArgs, foundBuffer.View, smallCyclesView
);
foundBuffer.View.CopyToCPU(stream, ref found, 1);
stream.Synchronize(); stream.Synchronize();
foundBuffer.View.CopyToCPU(ref found, 1);
if (found != 0) if (found != 0)
{ {
Volatile.Write(ref isPrime, false); Volatile.Write(ref isPrime, false);
@@ -60,7 +72,8 @@ public class MersenneNumberOrderGpuTester(GpuKernelType kernelType, bool useGpuO
kStart += (UInt128)currentSize; kStart += (UInt128)currentSize;
} }
stream.Dispose();
foundBuffer.Dispose(); foundBuffer.Dispose();
gpuLease.Dispose(); // GpuPrimeWorkLimiter.Release();
} }
} }
@@ -1,9 +1,4 @@
using System;
using System.Buffers;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Runtime.InteropServices; using System.Runtime.InteropServices;
using System.Threading;
using System.Runtime.CompilerServices; using System.Runtime.CompilerServices;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
@@ -12,187 +7,211 @@ namespace PerfectNumbers.Core.Gpu;
public class MersenneNumberResidueGpuTester(bool useGpuOrder) public class MersenneNumberResidueGpuTester(bool useGpuOrder)
{ {
private readonly bool _useGpuOrder = useGpuOrder; private readonly bool _useGpuOrder = useGpuOrder;
private readonly ConcurrentBag<GpuContextPool.GpuContextLease> _acceleratorPool = new();
private readonly ConcurrentDictionary<Accelerator, Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>> _pow2ModKernelCache = new(AcceleratorReferenceComparer.Instance);
private readonly ConcurrentDictionary<Accelerator, ConcurrentBag<ResidueResources>> _resourcePools = new(AcceleratorReferenceComparer.Instance);
// TODO: Integrate MersenneDivisorCycles.Shared to consult cycle lengths for small q (<= 4M) and fast-reject [ThreadStatic]
// candidates without launching heavy kernels. Consider a device-visible constant buffer per-accelerator private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>>? _pow2ModKernel;
// via GpuKernelPool to avoid host round-trips.
// TODO: When cycles are missing for larger q values, compute only the single required cycle on the device
// requested by the caller, skip queuing additional block generation, and keep the snapshot cache untouched.
// GPU residue variant: check 2^p % q == 1 for q = 2*p*k + 1. [ThreadStatic]
public void Scan(ulong exponent, UInt128 twoP, LastDigit lastDigit, UInt128 maxK, ref bool isPrime) private static Queue<ResidueResources>? _resourcePools;
{
GpuPrimeWorkLimiter.Acquire();
var gpuLease = RentAccelerator();
var accelerator = gpuLease.Accelerator;
// Monitor.Enter(gpuLease.ExecutionLock); // TODO: Integrate MersenneDivisorCycles.Shared to consult cycle lengths for small q (<= 4M) and fast-reject
// candidates without launching heavy kernels. Consider a device-visible constant buffer per-accelerator
// via GpuKernelPool to avoid host round-trips.
// TODO: When cycles are missing for larger q values, compute only the single required cycle on the device
// requested by the caller, skip queuing additional block generation, and keep the snapshot cache untouched.
var stream = accelerator.CreateStream(); private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
var kernel = GetPow2ModKernel(accelerator);
// GPU residue variant: check 2^p % q == 1 for q = 2*p*k + 1.
public void Scan(ulong exponent, UInt128 twoP, LastDigit lastDigit, UInt128 maxK, ref bool isPrime)
{
// GpuPrimeWorkLimiter.Acquire();
var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = _accelerators[acceleratorIndex];
// Monitor.Enter(gpuLease.ExecutionLock);
var stream = accelerator.CreateStream();
var kernel = GetPow2ModKernel(accelerator);
var resources = RentResources(accelerator, GpuConstants.ScanBatchSize); var resources = RentResources(accelerator, GpuConstants.ScanBatchSize);
var gpuKernels = GpuKernelPool.GetKernels(); var gpuKernels = GpuKernelPool.GetOrAddKernels(acceleratorIndex, stream, KernelType.SmallCycles | KernelType.SmallPrimes);
var orderBuffer = resources.OrderBuffer; var orderBuffer = resources.OrderBuffer;
ulong[] orderArray = resources.OrderArray; ulong[] orderArray = resources.OrderArray;
int batchSize = resources.Capacity; int batchSize = resources.Capacity;
// Ensure device has small cycles and primes tables for in-kernel lookup // Ensure device has small cycles and primes tables for in-kernel lookup
var smallCyclesView = GpuKernelPool.EnsureSmallCyclesOnDevice(gpuKernels, accelerator); // TODO: Pass-in the stream down the path
ResiduePrimeViews primeViews = GpuKernelPool.EnsureSmallPrimesOnDevice(gpuKernels, accelerator); var smallCyclesView = GpuKernelPool.GetSmallCyclesOnDevice(gpuKernels);
ResiduePrimeViews primeViews = GpuKernelPool.GetSmallPrimesOnDevice(gpuKernels);
UInt128 kStart = 1UL; UInt128 kStart = 1UL;
UInt128 limit = maxK + UInt128.One; UInt128 limit = maxK + UInt128.One;
byte last = lastDigit == LastDigit.Seven ? (byte)1 : (byte)0; byte last = lastDigit == LastDigit.Seven ? (byte)1 : (byte)0;
twoP.Mod10_8_5_3(out ulong step10, out ulong step8, out ulong step5, out ulong step3); twoP.Mod10_8_5_3(out ulong step10, out ulong step8, out ulong step5, out ulong step3);
step10 = step10.Mod10(); step10 = step10.Mod10();
GpuUInt128 twoPGpu = (GpuUInt128)twoP; GpuUInt128 twoPGpu = (GpuUInt128)twoP;
UInt128 batchSize128 = (UInt128)batchSize; UInt128 batchSize128 = (UInt128)batchSize;
UInt128 fullBatchStep = twoP * batchSize128; UInt128 fullBatchStep = twoP * batchSize128;
UInt128 q = twoP * kStart + UInt128.One; UInt128 q = twoP * kStart + UInt128.One;
while (kStart < limit && Volatile.Read(ref isPrime)) while (kStart < limit && Volatile.Read(ref isPrime))
{ {
UInt128 remaining = limit - kStart; UInt128 remaining = limit - kStart;
int currentSize = remaining > batchSize128 ? batchSize : (int)remaining; int currentSize = remaining > batchSize128 ? batchSize : (int)remaining;
Span<ulong> orders = orderArray.AsSpan(0, currentSize); Span<ulong> orders = orderArray.AsSpan(0, currentSize);
ref ulong ordersRef = ref MemoryMarshal.GetReference(orders); ref ulong ordersRef = ref MemoryMarshal.GetReference(orders);
q.Mod10_8_5_3(out ulong q0m10, out ulong q0m8, out ulong q0m5, out ulong q0m3); q.Mod10_8_5_3(out ulong q0m10, out ulong q0m8, out ulong q0m5, out ulong q0m3);
var kernelArgs = new ResidueAutomatonArgs(q0m10, step10, q0m8, step8, q0m3, step3, q0m5, step5); var kernelArgs = new ResidueAutomatonArgs(q0m10, step10, q0m8, step8, q0m3, step3, q0m5, step5);
kernel(stream, currentSize, exponent, twoPGpu, (GpuUInt128)kStart, last, 0UL, kernel(stream, currentSize, exponent, twoPGpu, (GpuUInt128)kStart, last, 0UL,
kernelArgs, orderBuffer.View, smallCyclesView, primeViews.LastOne, primeViews.LastSeven, primeViews.LastOnePow2, primeViews.LastSevenPow2); kernelArgs, orderBuffer.View, smallCyclesView, primeViews.LastOne, primeViews.LastSeven, primeViews.LastOnePow2, primeViews.LastSevenPow2);
stream.Synchronize(); orderBuffer.View.CopyToCPU(stream, ref ordersRef, currentSize);
orderBuffer.View.CopyToCPU(ref ordersRef, currentSize); stream.Synchronize();
if (!Volatile.Read(ref isPrime)) if (!Volatile.Read(ref isPrime))
{ {
break; break;
} }
bool compositeFound = false; bool compositeFound = false;
for (int i = 0; i < currentSize; i++) for (int i = 0; i < currentSize; i++)
{ {
if (orders[i] != 0UL) if (orders[i] != 0UL)
{ {
Volatile.Write(ref isPrime, false); Volatile.Write(ref isPrime, false);
compositeFound = true; compositeFound = true;
break; break;
} }
} }
if (compositeFound) if (compositeFound)
{ {
break; break;
} }
UInt128 processed = (UInt128)currentSize; UInt128 processed = (UInt128)currentSize;
kStart += processed; kStart += processed;
if (kStart < limit) if (kStart < limit)
{ {
if (processed == batchSize128) if (processed == batchSize128)
{ {
q += fullBatchStep; q += fullBatchStep;
} }
else else
{ {
q += twoP * processed; // TODO: Swap this multiplication for the shared cycle stepping helper once residue cycles are mandatory so we can reuse the cached remainder ladder instead of performing UInt128 multiply operations. q += twoP * processed; // TODO: Swap this multiplication for the shared cycle stepping helper once residue cycles are mandatory so we can reuse the cached remainder ladder instead of performing UInt128 multiply operations.
} }
} }
} }
ReturnResources(accelerator, resources); ReturnResources(resources);
stream.Dispose(); stream.Dispose();
// Monitor.Exit(gpuLease.ExecutionLock); // Monitor.Exit(gpuLease.ExecutionLock);
gpuKernels.Dispose(); gpuKernels.Dispose();
ReturnAccelerator(gpuLease); // GpuPrimeWorkLimiter.Release();
GpuPrimeWorkLimiter.Release(); }
}
private Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> GetPow2ModKernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> GetPow2ModKernel(Accelerator accelerator)
_pow2ModKernelCache.GetOrAdd( {
accelerator,
static acc =>
{
var loaded = acc.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2ModKernels.Pow2ModKernelScan);
return (Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>)Delegate.CreateDelegate(
typeof(Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>),
loaded.Target,
loaded.Method);
});
private GpuContextPool.GpuContextLease RentAccelerator() var pool = _pow2ModKernel ?? [];
{ if (pool.TryGetValue(accelerator, out var cached))
if (_acceleratorPool.TryTake(out var lease)) {
{ return cached;
return lease; }
}
return GpuContextPool.Rent(); cached = CreatePow2ModKernel(accelerator);
} pool[accelerator] = cached;
return cached;
}
private void ReturnAccelerator(GpuContextPool.GpuContextLease lease) => _acceleratorPool.Add(lease); private static Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>> CreatePow2ModKernel(Accelerator accelerator)
{
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>(Pow2ModKernels.Pow2ModKernelScan);
private ResidueResources RentResources(Accelerator accelerator, int capacity) var kernel = (Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>)Delegate.CreateDelegate(
{ typeof(Action<AcceleratorStream, Index1D, ulong, GpuUInt128, GpuUInt128, byte, ulong, ResidueAutomatonArgs, ArrayView<ulong>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<uint, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>, ArrayView1D<ulong, Stride1D.Dense>>),
var bag = _resourcePools.GetOrAdd(accelerator, static _ => new ConcurrentBag<ResidueResources>()); loaded.Target,
while (bag.TryTake(out var resources)) loaded.Method);
{ return kernel;
if (resources.Capacity >= capacity) }
private static ResidueResources RentResources(Accelerator accelerator, int capacity)
{
var resourcePools = _resourcePools ??= new();
if (resourcePools.TryDequeue(out var bag))
{
bag.EnsureCapacity(accelerator, capacity);
return bag;
}
return new ResidueResources(accelerator, capacity);
}
private static void ReturnResources(ResidueResources resources)
{
var resourcePools = _resourcePools ??= new();
resourcePools.Enqueue(resources);
}
private sealed class ResidueResources
{
internal ResidueResources(Accelerator accelerator, int capacity)
{
Capacity = Math.Max(1, capacity);
// lock (accelerator)
{ {
// The caller always overwrites the required elements. We don't need to worry about clearing the output buffer. OrderBuffer = accelerator.Allocate1D<ulong>(Capacity);
// resources.OrderBuffer.MemSetToZero(); }
return resources;
}
resources.Dispose(); OrderArray = ThreadStaticPools.UlongPool.Rent(Capacity);
} }
var created = new ResidueResources(accelerator, capacity); internal void EnsureCapacity(Accelerator accelerator, int capacity)
// The caller always overwrites the required elements. We don't need to worry about clearing the output buffer. {
// created.OrderBuffer.MemSetToZero(); if (OrderBuffer.Length >= capacity)
return created; {
} return;
}
private void ReturnResources(Accelerator accelerator, ResidueResources resources) => OrderBuffer.Dispose();
_resourcePools.GetOrAdd(accelerator, static _ => new ConcurrentBag<ResidueResources>()).Add(resources); ThreadStaticPools.UlongPool.Return(OrderArray);
private sealed class ResidueResources // lock (accelerator)
{ {
internal ResidueResources(Accelerator accelerator, int capacity) OrderBuffer = accelerator.Allocate1D<ulong>(capacity);
{ }
Capacity = Math.Max(1, capacity);
OrderBuffer = accelerator.Allocate1D<ulong>(Capacity);
OrderArray = ArrayPool<ulong>.Shared.Rent(Capacity);
}
internal MemoryBuffer1D<ulong, Stride1D.Dense> OrderBuffer { get; } OrderArray = ThreadStaticPools.UlongPool.Rent(capacity);
Capacity = capacity;
}
internal ulong[] OrderArray { get; } internal MemoryBuffer1D<ulong, Stride1D.Dense> OrderBuffer { get; private set; }
internal int Capacity { get; } internal ulong[] OrderArray { get; private set; }
public void Dispose() internal int Capacity { get; private set; }
{
OrderBuffer.Dispose();
ArrayPool<ulong>.Shared.Return(OrderArray, clearArray: false);
}
}
private sealed class AcceleratorReferenceComparer : IEqualityComparer<Accelerator> public void Dispose()
{ {
public static AcceleratorReferenceComparer Instance { get; } = new(); OrderBuffer.Dispose();
ThreadStaticPools.UlongPool.Return(OrderArray, clearArray: false);
}
}
public bool Equals(Accelerator? x, Accelerator? y) => ReferenceEquals(x, y); private sealed class AcceleratorReferenceComparer : IEqualityComparer<Accelerator>
{
public static AcceleratorReferenceComparer Instance { get; } = new();
public int GetHashCode(Accelerator obj) => RuntimeHelpers.GetHashCode(obj); public bool Equals(Accelerator? x, Accelerator? y) => ReferenceEquals(x, y);
}
public int GetHashCode(Accelerator obj) => RuntimeHelpers.GetHashCode(obj);
}
} }
+363 -239
View File
@@ -1,4 +1,3 @@
using System.Buffers;
using System.Collections.Concurrent; using System.Collections.Concurrent;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
@@ -27,27 +26,64 @@ public static class NttGpuMath
// Global backend setting for GPU transforms. Used by ForwardGpu/InverseGpu. // Global backend setting for GPU transforms. Used by ForwardGpu/InverseGpu.
// TODO(NTT-OPT): Consider passing backend explicitly to avoid global state. // TODO(NTT-OPT): Consider passing backend explicitly to avoid global state.
public static NttBackend GpuTransformBackend { get; set; } = NttBackend.Reference; public static NttBackend GpuTransformBackend { get; set; } = NttBackend.Reference;
// Controls modular reduction strategy inside staged NTT paths. // Controls modular reduction strategy inside staged NTT paths.
public static ModReductionMode ReductionMode { get; set; } = ModReductionMode.Auto; public static ModReductionMode ReductionMode { get; set; } = ModReductionMode.Auto;
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128>> MulKernelCache = new(); // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128>> StageKernelCache = new(); // TODO: Same as above – materialize staged kernels during startup so we can drop ConcurrentDictionary usage entirely per the benchmark guidance.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128>> ScaleKernelCache = new(); // TODO: Promote to the static kernel table initialized alongside the GpuModularArithmeticBenchmarks fast path.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong>> StageMontKernelCache = new(); // TODO: Inline the Montgomery kernels into the startup table instead of using a concurrent cache now that we no longer mutate state at runtime.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>> StageBarrett128KernelCache = new(); // TODO: Same plan – reuse the precomputed kernels measured fastest in MontgomeryMultiplyBenchmarks rather than looking them up dynamically.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong>> ScaleBarrett128KernelCache = new(); // TODO: Collapse into the startup kernel array once Barrett128 constants are preloaded.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>> SquareBarrett128KernelCache = new(); // TODO: Fold these kernels into the same startup table to avoid concurrent access overhead highlighted in GpuModularArithmeticBenchmarks.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>> ToMont64KernelCache = new(); // TODO: Prebind and reuse the Montgomery conversions from MontgomeryMultiplyBenchmarks instead of storing them in a concurrent cache.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong>> FromMont64KernelCache = new(); // TODO: As above – drop ConcurrentDictionary once startup prewarms every kernel.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong>> SquareMont64KernelCache = new(); // TODO: Inline into the static kernel table established during initialization.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>> ScaleMont64KernelCache = new(); // TODO: Move to the startup kernel table so Montgomery-scaled launches skip concurrent lookups.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128>> ForwardKernelCache = new(); // TODO: Warm these forward kernels during initialization and store them in a plain array so the LucasLehmerGpuBenchmarks launch costs disappear. [ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128>>? _mulKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128>> InverseKernelCache = new(); // TODO: Same as above – replace with startup-prepared tables per LucasLehmerGpuBenchmarks guidance. [ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream,Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128>>? _stageKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
private static readonly ConcurrentDictionary<Accelerator, Action<Index1D, ArrayView<GpuUInt128>, int>> BitReverseKernelCache = new(); // TODO: Precompute bit-reversal kernels during boot so we can remove the concurrent lookup overhead flagged in the benchmarks. [ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128>>? _scaleKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
private sealed class SquareCacheEntry [ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong>>? _stageMontKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>>? _stageBarrett128Kernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>>? _toMont64Kernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong>>? _fromMont64Kernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong>>? _scaleBarrett128Kernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static readonly Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128>>? _squareMont64Kernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>>? _scaleMont64Kernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128>>? _forwardKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128>>? _inverseKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
[ThreadStatic]
private static Dictionary<Accelerator, Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int>>? _bitReverseKernel; // TODO: Replace this concurrent cache with the prewarmed accelerator-indexed tables from GpuModularArithmeticBenchmarks so kernel launches avoid dictionary lookups once the kernels are baked during startup.
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128> GetStageKernel(Accelerator accelerator)
{
var pool = _stageKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128>(NttButterflyKernels.StageKernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128>>();
pool[accelerator] = cached;
return cached;
}
private sealed class SquareCacheEntry
{ {
public MemoryBuffer1D<GpuUInt128, Stride1D.Dense> Buffer { get; } public MemoryBuffer1D<GpuUInt128, Stride1D.Dense> Buffer { get; }
public GpuUInt128 Root { get; } public GpuUInt128 Root { get; }
@@ -72,7 +108,7 @@ public static class NttGpuMath
public ulong BarrettMuHigh { get; } public ulong BarrettMuHigh { get; }
public ulong BarrettMuLow { get; } public ulong BarrettMuLow { get; }
public SquareCacheEntry(Accelerator accelerator, int length, GpuUInt128 modulus, GpuUInt128 primitiveRoot) public SquareCacheEntry(Accelerator accelerator, AcceleratorStream stream, int length, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
Buffer = accelerator.Allocate1D<GpuUInt128>(length); Buffer = accelerator.Allocate1D<GpuUInt128>(length);
Root = new GpuUInt128(primitiveRoot); Root = new GpuUInt128(primitiveRoot);
@@ -99,7 +135,7 @@ public static class NttGpuMath
// Precompute stage twiddle factors (forward and inverse) and upload to GPU. // Precompute stage twiddle factors (forward and inverse) and upload to GPU.
Bits = (int)Math.Log2(length); Bits = (int)Math.Log2(length);
int twiddleCount = length - 1; int twiddleCount = length - 1;
var pool = ArrayPool<GpuUInt128>.Shared; var pool = ThreadStaticPools.GpuUInt128Pool;
var forward = pool.Rent(twiddleCount); var forward = pool.Rent(twiddleCount);
var inverse = pool.Rent(twiddleCount); var inverse = pool.Rent(twiddleCount);
StageOffsets = new int[Bits]; StageOffsets = new int[Bits];
@@ -136,9 +172,9 @@ public static class NttGpuMath
} }
Twiddles = accelerator.Allocate1D<GpuUInt128>(twiddleCount); Twiddles = accelerator.Allocate1D<GpuUInt128>(twiddleCount);
Twiddles.View.CopyFromCPU(ref forward[0], twiddleCount); Twiddles.View.CopyFromCPU(stream, ref forward[0], twiddleCount);
TwiddlesInv = accelerator.Allocate1D<GpuUInt128>(twiddleCount); TwiddlesInv = accelerator.Allocate1D<GpuUInt128>(twiddleCount);
TwiddlesInv.View.CopyFromCPU(ref inverse[0], twiddleCount); TwiddlesInv.View.CopyFromCPU(stream, ref inverse[0], twiddleCount);
// Prepare Montgomery constants for 64-bit moduli and twiddles in Montgomery domain. // Prepare Montgomery constants for 64-bit moduli and twiddles in Montgomery domain.
if (modulus.High == 0UL && (ReductionMode == ModReductionMode.Auto || ReductionMode == ModReductionMode.Mont64)) if (modulus.High == 0UL && (ReductionMode == ModReductionMode.Auto || ReductionMode == ModReductionMode.Mont64))
@@ -166,9 +202,9 @@ public static class NttGpuMath
} }
TwiddlesMont = accelerator.Allocate1D<GpuUInt128>(twiddleCount); TwiddlesMont = accelerator.Allocate1D<GpuUInt128>(twiddleCount);
TwiddlesMont.View.CopyFromCPU(ref forwardMont[0], twiddleCount); TwiddlesMont.View.CopyFromCPU(stream, ref forwardMont[0], twiddleCount);
TwiddlesInvMont = accelerator.Allocate1D<GpuUInt128>(twiddleCount); TwiddlesInvMont = accelerator.Allocate1D<GpuUInt128>(twiddleCount);
TwiddlesInvMont.View.CopyFromCPU(ref inverseMont[0], twiddleCount); TwiddlesInvMont.View.CopyFromCPU(stream, ref inverseMont[0], twiddleCount);
pool.Return(forwardMont, clearArray: true); pool.Return(forwardMont, clearArray: true);
pool.Return(inverseMont, clearArray: true); pool.Return(inverseMont, clearArray: true);
@@ -250,7 +286,7 @@ public static class NttGpuMath
private readonly record struct SquareCacheKey(int Length, GpuUInt128 Modulus, GpuUInt128 PrimitiveRoot); private readonly record struct SquareCacheKey(int Length, GpuUInt128 Modulus, GpuUInt128 PrimitiveRoot);
private static readonly ConcurrentDictionary<Accelerator, ConcurrentDictionary<SquareCacheKey, SquareCacheEntry>> SquareCache = new(); private static readonly ConcurrentDictionary<SquareCacheKey, SquareCacheEntry> SquareCache = [];
// TODO(NTT-OPT): Add a TwiddleCache for stage-wise Cooley–Tukey NTT. // TODO(NTT-OPT): Add a TwiddleCache for stage-wise Cooley–Tukey NTT.
// Design: // Design:
@@ -258,11 +294,10 @@ public static class NttGpuMath
// - Store one GPU buffer with all stage twiddles, or per-stage slices. // - Store one GPU buffer with all stage twiddles, or per-stage slices.
// - Expose accessors to retrieve per-stage views used by butterfly kernels. // - Expose accessors to retrieve per-stage views used by butterfly kernels.
private static SquareCacheEntry GetSquareCache(Accelerator accelerator, int length, GpuUInt128 modulus, GpuUInt128 primitiveRoot) private static SquareCacheEntry GetSquareCache(Accelerator accelerator, AcceleratorStream stream, int length, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
var perAccel = SquareCache.GetOrAdd(accelerator, _ => new ConcurrentDictionary<SquareCacheKey, SquareCacheEntry>());
var key = new SquareCacheKey(length, modulus, primitiveRoot); var key = new SquareCacheKey(length, modulus, primitiveRoot);
return perAccel.GetOrAdd(key, _ => new SquareCacheEntry(accelerator, length, modulus, primitiveRoot)); return SquareCache.GetOrAdd(key, _ => new SquareCacheEntry(accelerator, stream, length, modulus, primitiveRoot));
} }
private static readonly ulong[] SmallPrimes = GenerateSmallPrimes(1000); private static readonly ulong[] SmallPrimes = GenerateSmallPrimes(1000);
@@ -330,105 +365,197 @@ public static class NttGpuMath
{ {
if (accelerator is null) if (accelerator is null)
{ {
foreach (var perAccel in SquareCache.Values) foreach (var entry in SquareCache.Values)
{ {
foreach (var entry in perAccel.Values) entry.Dispose();
{
entry.Dispose();
}
} }
SquareCache.Clear(); SquareCache.Clear();
MulKernelCache.Clear();
StageKernelCache.Clear();
ScaleKernelCache.Clear();
StageMontKernelCache.Clear();
StageBarrett128KernelCache.Clear();
ToMont64KernelCache.Clear();
FromMont64KernelCache.Clear();
SquareMont64KernelCache.Clear();
ScaleMont64KernelCache.Clear();
ForwardKernelCache.Clear();
InverseKernelCache.Clear();
// TODO(NTT-OPT): Clear TwiddleCache once introduced. // TODO(NTT-OPT): Clear TwiddleCache once introduced.
return; return;
} }
if (SquareCache.TryRemove(accelerator, out var cache)) var squareCache = SquareCache;
{ foreach (var entry in squareCache.Values)
foreach (var entry in cache.Values) {
{ entry.Dispose();
entry.Dispose(); }
}
}
MulKernelCache.TryRemove(accelerator, out _);
StageKernelCache.TryRemove(accelerator, out _);
ScaleKernelCache.TryRemove(accelerator, out _);
StageMontKernelCache.TryRemove(accelerator, out _);
StageBarrett128KernelCache.TryRemove(accelerator, out _);
ToMont64KernelCache.TryRemove(accelerator, out _);
FromMont64KernelCache.TryRemove(accelerator, out _);
SquareMont64KernelCache.TryRemove(accelerator, out _);
ScaleMont64KernelCache.TryRemove(accelerator, out _);
ForwardKernelCache.TryRemove(accelerator, out _);
InverseKernelCache.TryRemove(accelerator, out _);
// TODO(NTT-OPT): Remove per-accelerator twiddle buffers once added. // TODO(NTT-OPT): Remove per-accelerator twiddle buffers once added.
} }
private static Action<Index1D, ArrayView<GpuUInt128>, int> GetBitReverseKernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int> GetBitReverseKernel(Accelerator accelerator)
BitReverseKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int>(NttTransformKernels.BitReverseKernel)); {
var pool = _bitReverseKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
private static Action<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128> GetMulKernel(Accelerator accelerator) => var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int>(NttTransformKernels.BitReverseKernel);
MulKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128>(NttPointwiseKernels.MulKernel));
private static Action<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128> GetStageKernel(Accelerator accelerator) => var kernel = KernelUtil.GetKernel(loaded);
StageKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, GpuUInt128>(NttButterflyKernels.StageKernel));
private static Action<Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128> GetScaleKernel(Accelerator accelerator) => cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int>>();
ScaleKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128>(NttPointwiseKernels.ScaleKernel)); pool[accelerator] = cached;
return cached;
}
private static Action<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong> GetStageMontKernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128> GetForwardKernel(Accelerator accelerator)
StageMontKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong>(NttMontgomeryKernels.StageMontKernel)); {
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128>(NttTransformKernels.ForwardKernel);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128>>();
}
private static Action<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong> GetStageBarrett128Kernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong> GetFromMont64Kernel(Accelerator accelerator)
StageBarrett128KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>(NttBarrettKernels.StageBarrett128Kernel)); {
var pool = _fromMont64Kernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
private static Action<Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong> GetScaleBarrett128Kernel(Accelerator accelerator) => var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong>(NttMontgomeryKernels.FromMont64Kernel);
ScaleBarrett128KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong>(NttBarrettKernels.ScaleBarrett128Kernel));
private static Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong> GetSquareBarrett128Kernel(Accelerator accelerator) => var kernel = KernelUtil.GetKernel(loaded);
SquareBarrett128KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>(NttBarrettKernels.SquareBarrett128Kernel));
private static Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong> GetToMont64Kernel(Accelerator accelerator) => cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong>>();
ToMont64KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>(NttMontgomeryKernels.ToMont64Kernel)); pool[accelerator] = cached;
return cached;
}
private static Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong> GetFromMont64Kernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128> GetInverseKernel(Accelerator accelerator)
FromMont64KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong>(NttMontgomeryKernels.FromMont64Kernel)); {
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128>(NttTransformKernels.InverseKernel);
var kernel = KernelUtil.GetKernel(loaded);
return kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128>>();
}
private static Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong> GetSquareMont64Kernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128> GetMulKernel(Accelerator accelerator)
SquareMont64KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong>(NttMontgomeryKernels.SquareMont64Kernel)); {
var pool = _mulKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
private static Action<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong> GetScaleMont64Kernel(Accelerator accelerator) => var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128>(NttPointwiseKernels.MulKernel);
ScaleMont64KernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>(NttMontgomeryKernels.ScaleMont64Kernel));
private static Action<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128> GetForwardKernel(Accelerator accelerator) => var kernel = KernelUtil.GetKernel(loaded);
ForwardKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128>(NttTransformKernels.ForwardKernel));
// TODO(NTT-OPT): Introduce stage-wise Cooley–Tukey kernels and a cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, GpuUInt128>>();
// per-(accelerator,length,modulus,root) twiddle cache. The plan: pool[accelerator] = cached;
// - Precompute twiddle factors for all stages once and cache on GPU. return cached;
// - Replace the current O(n^2) ForwardKernel with O(n log n) butterflies. }
// - Provide accessors like GetStageKernel(...) and GetTwiddleBuffer(...).
private static Action<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128> GetInverseKernel(Accelerator accelerator) => private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128> GetScaleKernel(Accelerator accelerator)
InverseKernelCache.GetOrAdd(accelerator, acc => acc.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ArrayView<GpuUInt128>, int, GpuUInt128, GpuUInt128, GpuUInt128>(NttTransformKernels.InverseKernel)); {
var pool = _scaleKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
// TODO(NTT-OPT): Mirror the forward stage-wise design for the inverse var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128>(NttPointwiseKernels.ScaleKernel);
// transform. Precompute inverse twiddles and a normalization factor
// (n^-1 mod m). Launch one short kernel per stage to avoid TDR.
public static void BitReverse(Span<GpuUInt128> values) var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, GpuUInt128, GpuUInt128>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong> GetScaleBarrett128Kernel(Accelerator accelerator)
{
var pool = _scaleBarrett128Kernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong>(NttBarrettKernels.ScaleBarrett128Kernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, GpuUInt128, ulong, ulong, ulong, ulong>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong> GetScaleMont64Kernel(Accelerator accelerator)
{
var pool = _scaleMont64Kernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>(NttMontgomeryKernels.ScaleMont64Kernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong> GetStageBarrett128Kernel(Accelerator accelerator)
{
var pool = _stageBarrett128Kernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>(NttBarrettKernels.StageBarrett128Kernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong, ulong, ulong>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong> GetStageMontKernel(Accelerator accelerator)
{
var pool = _stageMontKernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong>(NttMontgomeryKernels.StageMontKernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, int, int, int, ArrayView<GpuUInt128>, ulong, ulong>>();
pool[accelerator] = cached;
return cached;
}
private static Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong> GetToMont64Kernel(Accelerator accelerator)
{
var pool = _toMont64Kernel ??= [];
if (pool.TryGetValue(accelerator, out var cached))
{
return cached;
}
var loaded = accelerator.LoadAutoGroupedStreamKernel<Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>(NttMontgomeryKernels.ToMont64Kernel);
var kernel = KernelUtil.GetKernel(loaded);
cached = kernel.CreateLauncherDelegate<Action<AcceleratorStream, Index1D, ArrayView<GpuUInt128>, ulong, ulong, ulong>>();
pool[accelerator] = cached;
return cached;
}
// TODO(NTT-OPT): Mirror the forward stage-wise design for the inverse
// transform. Precompute inverse twiddles and a normalization factor
// (n^-1 mod m). Launch one short kernel per stage to avoid TDR.
public static void BitReverse(Span<GpuUInt128> values)
{ {
int n = values.Length; int n = values.Length;
int bits = (int)Math.Log2(n); int bits = (int)Math.Log2(n);
@@ -446,21 +573,22 @@ public static class NttGpuMath
{ {
int n = values.Length; int n = values.Length;
int bits = (int)Math.Log2(n); int bits = (int)Math.Log2(n);
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var kernel = GetBitReverseKernel(accelerator); var pool = ThreadStaticPools.GpuUInt128Pool;
var pool = ArrayPool<GpuUInt128>.Shared;
var array = pool.Rent(n); var array = pool.Rent(n);
values.CopyTo(array); values.CopyTo(array);
var buffer = accelerator.Allocate1D<GpuUInt128>(n); var buffer = accelerator.Allocate1D<GpuUInt128>(n);
buffer.View.CopyFromCPU(ref array[0], n); var stream = accelerator.CreateStream();
kernel(n, buffer.View, bits); buffer.View.CopyFromCPU(stream, ref array[0], n);
accelerator.Synchronize(); var bitReverseKernel = GetBitReverseKernel(accelerator);
buffer.View.CopyToCPU(ref array[0], n); bitReverseKernel(stream, n, buffer.View, bits);
buffer.View.CopyToCPU(stream, ref array[0], n);
stream.Synchronize();
array.AsSpan(0, n).CopyTo(values); array.AsSpan(0, n).CopyTo(values);
pool.Return(array, clearArray: true); pool.Return(array, clearArray: true);
buffer.Dispose(); buffer.Dispose();
gpu.Dispose(); stream.Dispose();
} }
@@ -508,30 +636,30 @@ public static class NttGpuMath
private static void ForwardGpuReference(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) private static void ForwardGpuReference(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
int n = (int)values.Length; int n = (int)values.Length;
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var kernel = GetForwardKernel(accelerator); var stream = accelerator.CreateStream();
var pool = ArrayPool<GpuUInt128>.Shared; var pool = ThreadStaticPools.GpuUInt128Pool;
var inputArray = pool.Rent(n); var inputArray = pool.Rent(n);
var outputArray = pool.Rent(n); var outputArray = pool.Rent(n);
values.CopyTo(inputArray); values.CopyTo(inputArray);
var inputBuffer = accelerator.Allocate1D<GpuUInt128>(n); var inputBuffer = accelerator.Allocate1D<GpuUInt128>(n);
var outputBuffer = accelerator.Allocate1D<GpuUInt128>(n); var outputBuffer = accelerator.Allocate1D<GpuUInt128>(n);
inputBuffer.View.CopyFromCPU(ref inputArray[0], n); inputBuffer.View.CopyFromCPU(stream, ref inputArray[0], n);
UInt128 modValue = modulus; UInt128 modValue = modulus;
UInt128 expBase = (modValue - 1UL) / (ulong)n; UInt128 expBase = (modValue - 1UL) / (ulong)n;
var root = new GpuUInt128(primitiveRoot); var root = new GpuUInt128(primitiveRoot);
root.ModPow((ulong)expBase, modulus); root.ModPow((ulong)expBase, modulus);
kernel(n, inputBuffer.View, outputBuffer.View, n, modulus, root); var forwardKernel = GetForwardKernel(accelerator);
accelerator.Synchronize(); forwardKernel(stream, n, inputBuffer.View, outputBuffer.View, n, modulus, root);
outputBuffer.View.CopyToCPU(ref outputArray[0], n); outputBuffer.View.CopyToCPU(stream, ref outputArray[0], n);
stream.Synchronize();
outputArray.AsSpan(0, n).CopyTo(values); outputArray.AsSpan(0, n).CopyTo(values);
pool.Return(inputArray, clearArray: true); pool.Return(inputArray, clearArray: true);
pool.Return(outputArray, clearArray: true); pool.Return(outputArray, clearArray: true);
outputBuffer.Dispose(); outputBuffer.Dispose();
inputBuffer.Dispose(); inputBuffer.Dispose();
gpu.Dispose(); stream.Dispose();
} }
public static void Inverse(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) public static void Inverse(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
@@ -605,16 +733,16 @@ public static class NttGpuMath
private static void InverseGpuReference(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) private static void InverseGpuReference(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
int n = values.Length; int n = values.Length;
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var kernel = GetInverseKernel(accelerator); var stream = accelerator.CreateStream();
var pool = ArrayPool<GpuUInt128>.Shared; var pool = ThreadStaticPools.GpuUInt128Pool;
var inputArray = pool.Rent(n); var inputArray = pool.Rent(n);
var outputArray = pool.Rent(n); var outputArray = pool.Rent(n);
values.CopyTo(inputArray); values.CopyTo(inputArray);
var inputBuffer = accelerator.Allocate1D<GpuUInt128>(n); var inputBuffer = accelerator.Allocate1D<GpuUInt128>(n);
var outputBuffer = accelerator.Allocate1D<GpuUInt128>(n); var outputBuffer = accelerator.Allocate1D<GpuUInt128>(n);
inputBuffer.View.CopyFromCPU(ref inputArray[0], n); inputBuffer.View.CopyFromCPU(stream, ref inputArray[0], n);
UInt128 modValue = modulus; UInt128 modValue = modulus;
UInt128 expBase = (modValue - 1UL) / (ulong)n; UInt128 expBase = (modValue - 1UL) / (ulong)n;
var root = new GpuUInt128(primitiveRoot); var root = new GpuUInt128(primitiveRoot);
@@ -630,78 +758,78 @@ public static class NttGpuMath
} }
var nInv = new GpuUInt128((UInt128)n); var nInv = new GpuUInt128((UInt128)n);
if (modulus.High == 0UL) if (modulus.High == 0UL)
{ {
nInv.ModInv(modulus.Low); nInv.ModInv(modulus.Low);
} }
else else
{ {
nInv.ModInv(modulus); nInv.ModInv(modulus);
} }
kernel(n, inputBuffer.View, outputBuffer.View, n, modulus, rootInv, nInv);
accelerator.Synchronize(); var inverseKernel = GetInverseKernel(accelerator);
outputBuffer.View.CopyToCPU(ref outputArray[0], n); inverseKernel(stream, n, inputBuffer.View, outputBuffer.View, n, modulus, rootInv, nInv);
outputBuffer.View.CopyToCPU(stream, ref outputArray[0], n);
stream.Synchronize();
outputArray.AsSpan(0, n).CopyTo(values); outputArray.AsSpan(0, n).CopyTo(values);
pool.Return(inputArray, clearArray: true); pool.Return(inputArray, clearArray: true);
pool.Return(outputArray, clearArray: true); pool.Return(outputArray, clearArray: true);
outputBuffer.Dispose(); outputBuffer.Dispose();
inputBuffer.Dispose(); inputBuffer.Dispose();
gpu.Dispose(); stream.Dispose();
} }
private static void ForwardGpuStaged(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) private static void ForwardGpuStaged(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
int n = values.Length; int n = values.Length;
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var stream = accelerator.CreateStream();
var buffer = accelerator.Allocate1D<GpuUInt128>(n); var buffer = accelerator.Allocate1D<GpuUInt128>(n);
buffer.View.CopyFromCPU(ref values[0], n); buffer.View.CopyFromCPU(stream, ref values[0], n);
var cache = GetSquareCache(accelerator, n, modulus, primitiveRoot); var cache = GetSquareCache(accelerator, stream, n, modulus, primitiveRoot);
ForwardDevice(accelerator, buffer.View, n, modulus, cache); ForwardDevice(accelerator, stream, buffer.View, n, modulus, cache);
buffer.View.CopyToCPU(ref values[0], n); buffer.View.CopyToCPU(stream, ref values[0], n);
stream.Synchronize();
stream.Dispose();
buffer.Dispose(); buffer.Dispose();
gpu.Dispose();
} }
private static void InverseGpuStaged(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) private static void InverseGpuStaged(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
int n = values.Length; int n = values.Length;
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var stream = accelerator.CreateStream();
var buffer = accelerator.Allocate1D<GpuUInt128>(n); var buffer = accelerator.Allocate1D<GpuUInt128>(n);
buffer.View.CopyFromCPU(ref values[0], n); buffer.View.CopyFromCPU(stream, ref values[0], n);
var cache = GetSquareCache(accelerator, n, modulus, primitiveRoot); var cache = GetSquareCache(accelerator, stream, n, modulus, primitiveRoot);
InverseDevice(accelerator, buffer.View, n, modulus, cache); InverseDevice(accelerator, stream, buffer.View, n, modulus, cache);
buffer.View.CopyToCPU(ref values[0], n); buffer.View.CopyToCPU(stream, ref values[0], n);
stream.Synchronize();
stream.Dispose();
buffer.Dispose(); buffer.Dispose();
gpu.Dispose();
} }
private static void ForwardDevice(Accelerator accelerator, ArrayView<GpuUInt128> data, int length, GpuUInt128 modulus, SquareCacheEntry cache) private static void ForwardDevice(Accelerator accelerator, AcceleratorStream stream, ArrayView<GpuUInt128> data, int length, GpuUInt128 modulus, SquareCacheEntry cache)
{ {
var bitKernel = GetBitReverseKernel(accelerator); var bitReverseKernel = GetBitReverseKernel(accelerator);
bitKernel(length, data, cache.Bits); bitReverseKernel(stream, length, data, cache.Bits);
accelerator.Synchronize();
if (cache.UseMontgomery64) if (cache.UseMontgomery64)
{ {
var toMont = GetToMont64Kernel(accelerator); var toMont64Kernel = GetToMont64Kernel(accelerator);
toMont(length, data, cache.ModulusLow, cache.MontNPrime64, cache.MontR2Mod64); toMont64Kernel(stream, length, data, cache.ModulusLow, cache.MontNPrime64, cache.MontR2Mod64);
accelerator.Synchronize();
var stageKernel = GetStageMontKernel(accelerator); var stageKernel = GetStageMontKernel(accelerator);
int butterflies = length >> 1; int butterflies = length >> 1;
for (int s = 0; s < cache.Bits; s++) for (int s = 0; s < cache.Bits; s++)
{ {
int len = 1 << (s + 1); int len = 1 << (s + 1);
int half = len >> 1; int half = len >> 1;
stageKernel(butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesMont!.View, cache.ModulusLow, cache.MontNPrime64); stageKernel(stream, butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesMont!.View, cache.ModulusLow, cache.MontNPrime64);
accelerator.Synchronize(); }
} var fromMont64Kernel = GetFromMont64Kernel(accelerator);
var fromMont = GetFromMont64Kernel(accelerator); fromMont64Kernel(stream, length, data, cache.ModulusLow, cache.MontNPrime64);
fromMont(length, data, cache.ModulusLow, cache.MontNPrime64);
} }
else if (cache.UseBarrett128) else if (cache.UseBarrett128)
{ {
@@ -711,8 +839,7 @@ public static class NttGpuMath
{ {
int len = 1 << (s + 1); int len = 1 << (s + 1);
int half = len >> 1; int half = len >> 1;
stageKernel(butterflies, data, len, half, cache.StageOffsets[s], cache.Twiddles.View, cache.ModulusHigh, cache.ModulusLow, cache.BarrettMuHigh, cache.BarrettMuLow); stageKernel(stream, butterflies, data, len, half, cache.StageOffsets[s], cache.Twiddles.View, cache.ModulusHigh, cache.ModulusLow, cache.BarrettMuHigh, cache.BarrettMuLow);
accelerator.Synchronize();
} }
} }
else else
@@ -723,72 +850,67 @@ public static class NttGpuMath
{ {
int len = 1 << (s + 1); int len = 1 << (s + 1);
int half = len >> 1; int half = len >> 1;
stageKernel(butterflies, data, len, half, cache.StageOffsets[s], cache.Twiddles.View, modulus); stageKernel(stream, butterflies, data, len, half, cache.StageOffsets[s], cache.Twiddles.View, modulus);
accelerator.Synchronize();
} }
} }
accelerator.Synchronize(); stream.Synchronize();
} }
private static void InverseDevice(Accelerator accelerator, ArrayView<GpuUInt128> data, int length, GpuUInt128 modulus, SquareCacheEntry cache) private static void InverseDevice(Accelerator accelerator, AcceleratorStream stream, ArrayView<GpuUInt128> data, int length, GpuUInt128 modulus, SquareCacheEntry cache)
{ {
var bitKernel = GetBitReverseKernel(accelerator); var bitReverseKernel = GetBitReverseKernel(accelerator);
bitKernel(length, data, cache.Bits); bitReverseKernel(stream, length, data, cache.Bits);
accelerator.Synchronize();
if (cache.UseMontgomery64) if (cache.UseMontgomery64)
{ {
var toMont = GetToMont64Kernel(accelerator); var toMont64Kernel = GetToMont64Kernel(accelerator);
toMont(length, data, cache.ModulusLow, cache.MontNPrime64, cache.MontR2Mod64); toMont64Kernel(stream, length, data, cache.ModulusLow, cache.MontNPrime64, cache.MontR2Mod64);
accelerator.Synchronize();
var stageKernel = GetStageMontKernel(accelerator); var stageKernel = GetStageMontKernel(accelerator);
int butterflies = length >> 1; int butterflies = length >> 1;
for (int s = 0; s < cache.Bits; s++) for (int s = 0; s < cache.Bits; s++)
{ {
int len = 1 << (s + 1); int len = 1 << (s + 1);
int half = len >> 1; int half = len >> 1;
stageKernel(butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesInvMont!.View, cache.ModulusLow, cache.MontNPrime64); stageKernel(stream, butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesInvMont!.View, cache.ModulusLow, cache.MontNPrime64);
accelerator.Synchronize(); }
}
var scaleKernel = GetScaleMont64Kernel(accelerator); var scaleMont64Kernel = GetScaleMont64Kernel(accelerator);
scaleKernel(length, data, cache.ModulusLow, cache.MontNPrime64, cache.MontNInvR64); scaleMont64Kernel(stream, length, data, cache.ModulusLow, cache.MontNPrime64, cache.MontNInvR64);
accelerator.Synchronize();
var fromMont = GetFromMont64Kernel(accelerator); var fromMont64Kernel = GetFromMont64Kernel(accelerator);
fromMont(length, data, cache.ModulusLow, cache.MontNPrime64); fromMont64Kernel(stream, length, data, cache.ModulusLow, cache.MontNPrime64);
} }
else if (cache.UseBarrett128) else if (cache.UseBarrett128)
{ {
var stageKernel = GetStageBarrett128Kernel(accelerator); var stageKernel = GetStageBarrett128Kernel(accelerator);
int butterflies = length >> 1; int butterflies = length >> 1;
for (int s = 0; s < cache.Bits; s++) for (int s = 0; s < cache.Bits; s++)
{ {
int len = 1 << (s + 1); int len = 1 << (s + 1);
int half = len >> 1; int half = len >> 1;
stageKernel(butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesInv.View, cache.ModulusHigh, cache.ModulusLow, cache.BarrettMuHigh, cache.BarrettMuLow); stageKernel(stream, butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesInv.View, cache.ModulusHigh, cache.ModulusLow, cache.BarrettMuHigh, cache.BarrettMuLow);
accelerator.Synchronize(); }
}
var scaleKernel = GetScaleBarrett128Kernel(accelerator); var scaleBarrett128Kernel = GetScaleBarrett128Kernel(accelerator);
scaleKernel(length, data, cache.NInv, cache.ModulusHigh, cache.ModulusLow, cache.BarrettMuHigh, cache.BarrettMuLow); scaleBarrett128Kernel(stream, length, data, cache.NInv, cache.ModulusHigh, cache.ModulusLow, cache.BarrettMuHigh, cache.BarrettMuLow);
accelerator.Synchronize();
} }
else else
{ {
var stageKernel = GetStageKernel(accelerator); var stageKernel = GetStageKernel(accelerator);
int butterflies = length >> 1; int butterflies = length >> 1;
for (int s = 0; s < cache.Bits; s++) for (int s = 0; s < cache.Bits; s++)
{ {
int len = 1 << (s + 1); int len = 1 << (s + 1);
int half = len >> 1; int half = len >> 1;
stageKernel(butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesInv.View, modulus); stageKernel(stream, butterflies, data, len, half, cache.StageOffsets[s], cache.TwiddlesInv.View, modulus);
accelerator.Synchronize(); }
}
var scaleKernel = GetScaleKernel(accelerator); var scaleKernel = GetScaleKernel(accelerator);
scaleKernel(length, data, cache.NInv, modulus); scaleKernel(stream, length, data, cache.NInv, modulus);
accelerator.Synchronize();
} }
accelerator.Synchronize(); stream.Synchronize();
} }
public static void Convolve(Span<GpuUInt128> left, Span<GpuUInt128> right, GpuUInt128 modulus, GpuUInt128 primitiveRoot) public static void Convolve(Span<GpuUInt128> left, Span<GpuUInt128> right, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
@@ -800,7 +922,7 @@ public static class NttGpuMath
n <<= 1; n <<= 1;
} }
var pool = ArrayPool<GpuUInt128>.Shared; var pool = ThreadStaticPools.GpuUInt128Pool;
var aBuffer = pool.Rent(n); var aBuffer = pool.Rent(n);
var bBuffer = pool.Rent(n); var bBuffer = pool.Rent(n);
var a = aBuffer.AsSpan(0, n); var a = aBuffer.AsSpan(0, n);
@@ -837,7 +959,7 @@ public static class NttGpuMath
n <<= 1; n <<= 1;
} }
var pool = ArrayPool<GpuUInt128>.Shared; var pool = ThreadStaticPools.GpuUInt128Pool;
var aBuffer = pool.Rent(n); var aBuffer = pool.Rent(n);
var bBuffer = pool.Rent(n); var bBuffer = pool.Rent(n);
var a = aBuffer.AsSpan(0, n); var a = aBuffer.AsSpan(0, n);
@@ -871,41 +993,43 @@ public static class NttGpuMath
public static void SquareGpu(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) => public static void SquareGpu(Span<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) =>
ConvolveGpu(values, values, modulus, primitiveRoot); ConvolveGpu(values, values, modulus, primitiveRoot);
public static void SquareDevice(Accelerator accelerator, ArrayView<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot) public static void SquareDevice(Accelerator accelerator, AcceleratorStream stream, ArrayView<GpuUInt128> values, GpuUInt128 modulus, GpuUInt128 primitiveRoot)
{ {
var cache = GetSquareCache(accelerator, (int)values.Length, modulus, primitiveRoot); var cache = GetSquareCache(accelerator, stream, (int)values.Length, modulus, primitiveRoot);
ForwardDevice(accelerator, values, (int)values.Length, modulus, cache); ForwardDevice(accelerator, stream, values, (int)values.Length, modulus, cache);
var mul = GetMulKernel(accelerator); var mulKernel = GetMulKernel(accelerator);
mul((int)values.Length, values, values, modulus); mulKernel(stream, (int)values.Length, values, values, modulus);
InverseDevice(accelerator, values, (int)values.Length, modulus, cache); InverseDevice(accelerator, stream, values, (int)values.Length, modulus, cache);
} }
private static readonly Accelerator[] _accelerators = AcceleratorPool.Shared.Accelerators;
public static void PointwiseMultiply(Span<GpuUInt128> left, Span<GpuUInt128> right, GpuUInt128 modulus) public static void PointwiseMultiply(Span<GpuUInt128> left, Span<GpuUInt128> right, GpuUInt128 modulus)
{ {
var gpu = GpuContextPool.Rent(); var acceleratorIndex = AcceleratorPool.Shared.Rent();
var accelerator = gpu.Accelerator; var accelerator = _accelerators[acceleratorIndex];
var kernel = GetMulKernel(accelerator); var stream = accelerator.CreateStream();
var pool = ArrayPool<GpuUInt128>.Shared; var pool = ThreadStaticPools.GpuUInt128Pool;
var leftArray = pool.Rent(left.Length); var leftArray = pool.Rent(left.Length);
var rightArray = pool.Rent(right.Length); var rightArray = pool.Rent(right.Length);
left.CopyTo(leftArray); left.CopyTo(leftArray);
right.CopyTo(rightArray); right.CopyTo(rightArray);
var leftBuffer = accelerator.Allocate1D<GpuUInt128>(left.Length); var leftBuffer = accelerator.Allocate1D<GpuUInt128>(left.Length);
var rightBuffer = accelerator.Allocate1D<GpuUInt128>(right.Length); var rightBuffer = accelerator.Allocate1D<GpuUInt128>(right.Length);
leftBuffer.View.CopyFromCPU(ref leftArray[0], left.Length); leftBuffer.View.CopyFromCPU(stream, ref leftArray[0], left.Length);
rightBuffer.View.CopyFromCPU(ref rightArray[0], right.Length); rightBuffer.View.CopyFromCPU(stream, ref rightArray[0], right.Length);
kernel(left.Length, leftBuffer.View, rightBuffer.View, modulus); var mulKernel = GetMulKernel(accelerator);
accelerator.Synchronize(); mulKernel(stream, left.Length, leftBuffer.View, rightBuffer.View, modulus);
leftBuffer.View.CopyToCPU(ref leftArray[0], left.Length); leftBuffer.View.CopyToCPU(stream, ref leftArray[0], left.Length);
rightBuffer.View.CopyToCPU(ref rightArray[0], right.Length); rightBuffer.View.CopyToCPU(stream, ref rightArray[0], right.Length);
stream.Synchronize();
leftArray.AsSpan(0, left.Length).CopyTo(left); leftArray.AsSpan(0, left.Length).CopyTo(left);
rightArray.AsSpan(0, right.Length).CopyTo(right); rightArray.AsSpan(0, right.Length).CopyTo(right);
pool.Return(leftArray, clearArray: true); pool.Return(leftArray, clearArray: true);
pool.Return(rightArray, clearArray: true); pool.Return(rightArray, clearArray: true);
rightBuffer.Dispose(); rightBuffer.Dispose();
leftBuffer.Dispose(); leftBuffer.Dispose();
gpu.Dispose(); stream.Dispose();
} }
private static ulong[] GenerateSmallPrimes(int limit) private static ulong[] GenerateSmallPrimes(int limit)
@@ -0,0 +1,94 @@
using System.Runtime.CompilerServices;
using ILGPU;
using ILGPU.Runtime;
using PerfectNumbers.Core.Gpu.Accelerators;
namespace PerfectNumbers.Core.Gpu;
public static class Pow2MontgomeryGpuCalculator
{
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong CalculateKeep(ulong exponent, in MontgomeryDivisorData divisor)
{
// GpuPrimeWorkLimiter.Acquire();
ulong result = 0UL;
var gpu = PrimeOrderCalculatorAccelerator.Rent(1);
int acceleratorIndex = gpu.AcceleratorIndex;
Accelerator? accelerator = gpu.Accelerator;
MemoryBuffer1D<ulong, Stride1D.Dense> exponentBuffer = gpu.Input;
MemoryBuffer1D<ulong, Stride1D.Dense> resultBuffer = gpu.OutputUlong;
AcceleratorStream stream = AcceleratorStreamPool.Rent(acceleratorIndex);
exponentBuffer.View.CopyFromCPU(stream, ref exponent, 1);
// We don't need to worry about any left-overs here.
// resultBuffer.MemSetToZero(stream);
var kernelLauncher = gpu.KeepMontgomeryKernelLauncher;
kernelLauncher(stream, 1, exponentBuffer.View, divisor, resultBuffer.View);
resultBuffer.View.CopyToCPU(stream, ref result, 1);
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
// GpuPrimeWorkLimiter.Release();
PrimeOrderCalculatorAccelerator.Return(gpu);
return result;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong CalculateConvert(ulong exponent, in MontgomeryDivisorData divisor)
{
// GpuPrimeWorkLimiter.Acquire();
ulong result = 0UL;
var gpu = PrimeOrderCalculatorAccelerator.Rent(1);
MemoryBuffer1D<ulong, Stride1D.Dense> exponentBuffer = gpu.Input;
MemoryBuffer1D<ulong, Stride1D.Dense> resultBuffer = gpu.OutputUlong;
int acceleratorIndex = gpu.AcceleratorIndex;
AcceleratorStream stream = AcceleratorStreamPool.Rent(acceleratorIndex);
exponentBuffer.View.CopyFromCPU(stream, ref exponent, 1);
// We don't need to worry about any left-overs here.
// resultBuffer.MemSetToZero(stream);
var kernelLauncher = gpu.ConvertToStandardKernelLauncher!;
kernelLauncher(stream, 1, exponentBuffer.View, divisor, resultBuffer.View);
resultBuffer.View.CopyToCPU(stream, ref result, 1);
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
// GpuPrimeWorkLimiter.Release();
PrimeOrderCalculatorAccelerator.Return(gpu);
return result;
}
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public static ulong CalculateConvert(PrimeOrderCalculatorAccelerator gpu, ulong exponent, in MontgomeryDivisorData divisor)
{
ulong result = 0UL;
MemoryBuffer1D<ulong, Stride1D.Dense> exponentBuffer = gpu.Input;
MemoryBuffer1D<ulong, Stride1D.Dense> resultBuffer = gpu.OutputUlong;
int acceleratorIndex = gpu.AcceleratorIndex;
AcceleratorStream stream = AcceleratorStreamPool.Rent(acceleratorIndex);
exponentBuffer.View.CopyFromCPU(stream, ref exponent, 1);
// We don't need to worry about any left-overs here.
// resultBuffer.MemSetToZero(stream);
var accelerator = gpu.Accelerator;
var kernelLauncher = gpu.ConvertToStandardKernelLauncher;
kernelLauncher(stream, 1, exponentBuffer.View, divisor, resultBuffer.View);
resultBuffer.View.CopyToCPU(stream, ref result, 1);
stream.Synchronize();
AcceleratorStreamPool.Return(acceleratorIndex, stream);
return result;
}
}
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,15 @@
namespace PerfectNumbers.Core.Gpu;
public readonly struct ResidueAutomatonArgs(
ulong q0m10, ulong step10, ulong q0m8, ulong step8, ulong q0m3, ulong step3, ulong q0m5, ulong step5
)
{
public readonly ulong Q0M10 = q0m10;
public readonly ulong Step10 = step10;
public readonly ulong Q0M8 = q0m8;
public readonly ulong Step8 = step8;
public readonly ulong Q0M3 = q0m3;
public readonly ulong Step3 = step3;
public readonly ulong Q0M5 = q0m5;
public readonly ulong Step5 = step5;
}
@@ -0,0 +1,16 @@
using ILGPU;
using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu;
public readonly struct ResiduePrimeViews(
ArrayView1D<uint, Stride1D.Dense> lastOne,
ArrayView1D<uint, Stride1D.Dense> lastSeven,
ArrayView1D<ulong, Stride1D.Dense> lastOnePow2,
ArrayView1D<ulong, Stride1D.Dense> lastSevenPow2)
{
public readonly ArrayView1D<uint, Stride1D.Dense> LastOne = lastOne;
public readonly ArrayView1D<uint, Stride1D.Dense> LastSeven = lastSeven;
public readonly ArrayView1D<ulong, Stride1D.Dense> LastOnePow2 = lastOnePow2;
public readonly ArrayView1D<ulong, Stride1D.Dense> LastSevenPow2 = lastSevenPow2;
}
+19 -9
View File
@@ -1,19 +1,29 @@
using System; using System.Runtime.CompilerServices;
using System.Threading;
using ILGPU; using ILGPU;
using ILGPU.Runtime; using ILGPU.Runtime;
namespace PerfectNumbers.Core.Gpu; namespace PerfectNumbers.Core.Gpu;
internal static class SharedGpuContext public static class SharedGpuContext
{ {
private static readonly Lazy<Context> s_context = new(() => ILGPU.Context.CreateDefault(), LazyThreadSafetyMode.ExecutionAndPublication); public static readonly Context Context = Context.CreateDefault();
private static readonly Lazy<Device> s_device = new(() => s_context.Value.GetPreferredDevice(false), LazyThreadSafetyMode.ExecutionAndPublication); public static readonly Device Device = Context.GetPreferredDevice(false);
private static readonly Lazy<Accelerator> s_accelerator = new(() => s_device.Value.CreateAccelerator(s_context.Value), LazyThreadSafetyMode.ExecutionAndPublication);
internal static Context Context => s_context.Value; [MethodImpl(MethodImplOptions.AggressiveInlining)]
public static Accelerator CreateAccelerator() => Device.CreateAccelerator(Context);
}
internal static Device Device => s_device.Value; public readonly struct GpuContext
{
public readonly Context Context;
public readonly Device Device;
internal static Accelerator Accelerator => s_accelerator.Value; public GpuContext()
{
var context = Context.CreateDefault();
Context = context;
Device = context.GetPreferredDevice(false);
}
public Accelerator CreateAccelerator() => Device.CreateAccelerator(Context);
} }
Loaded 100 of 124 files, more files were not shown because too many files have changed in this diff. Show more