Is your feature request related to a problem? Please describe.
Online HSTU requests have variable history length, candidate count, cache residency, and compute cost. FIFO or static batching can combine poorly matched requests, increase padding and cache movement, and create high tail latency. The repository needs an HSTU-aware scheduling policy evaluated against a realistic online benchmark.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context, code examples, or references to existing implementations about the feature request here.
By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.
Is your feature request related to a problem? Please describe.
Online HSTU requests have variable history length, candidate count, cache residency, and compute cost. FIFO or static batching can combine poorly matched requests, increase padding and cache movement, and create high tail latency. The repository needs an HSTU-aware scheduling policy evaluated against a realistic online benchmark.
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context, code examples, or references to existing implementations about the feature request here.
By submitting this issue, you agree to follow our code of conduct and our contributing guidelines.