Search results

[Feature]: AutoDeploy: load_hook hierarchy

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

Task

#10348 In NVIDIA/TensorRT-LLM;

· lucaslie opened on Dec 30, 2025

[Bug][AutoDeploy]: model fails on torch.nonzero()

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

bugSomething isn't workingSomething isn't working

Status: Open.

#10346 In NVIDIA/TensorRT-LLM;

· tcherckez-nvidia opened on Dec 30, 2025

[AutoDeploy] Nemo SuperV3: Enable multistream MoE

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

Status: Open.

#10345 In NVIDIA/TensorRT-LLM;

· galagam opened on Dec 30, 2025

[Usage]: Question about the progress of "Eagle3 static draft tree"

questionFurther information is requestedFurther information is requested

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#10343 In NVIDIA/TensorRT-LLM;

· XiaoXuan42 opened on Dec 30, 2025

[Feature]: Conditional Disaggregation Based on the KV Availability and/or ISL limits

Disaggregated serving<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.<NV>Deploying with separated, distributed components (params, kv-cache, compute). Arch & perf.

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Status: Open.

#10342 In NVIDIA/TensorRT-LLM;

· okdimok opened on Dec 30, 2025

[Bug]: Cutlass NVFP4 MoE produces the wrong results

bugSomething isn't workingSomething isn't working

Customized kernels<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.<NV>Specialized/modified CUDA kernels in TRTLLM for LLM ops, beyond standard TRT. Dev & perf.

Status: Open.

Bug

#10331 In NVIDIA/TensorRT-LLM;

· nzmora-nvidia opened on Dec 29, 2025

[Usage]: what's the different of ModelRunner ModelRunnerCPP PyExecutor ?

LLM API<NV>High-level LLM Python API & tools (e.g., trtllm-llmapi-launch) for TRTLLM inference/workflows.<NV>High-level LLM Python API & tools (e.g., trtllm-llmapi-launch) for TRTLLM inference/workflows.

questionFurther information is requestedFurther information is requested

Status: Open.

#10326 In NVIDIA/TensorRT-LLM;

· zh794390558 opened on Dec 29, 2025

[Bug]: Eagle static chain-like draft token tree raise Exception

bugSomething isn't workingSomething isn't working

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#10325 In NVIDIA/TensorRT-LLM;

· XiaoXuan42 opened on Dec 29, 2025

[Bug]: Eagle static tree drafting raise Exception when the depth of tree is 1

bugSomething isn't workingSomething isn't working

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#10323 In NVIDIA/TensorRT-LLM;

· XiaoXuan42 opened on Dec 29, 2025

[AutoDeploy][Bug]: Nemotron SSM sharding error

AutoDeploy<NV> AutoDeploy Backend<NV> AutoDeploy Backend

bugSomething isn't workingSomething isn't working

Status: Open.

#10318 In NVIDIA/TensorRT-LLM;

· greg-kwasniewski1 opened on Dec 28, 2025

[Bug]: No MTP support for Qwen3 Next

bugSomething isn't workingSomething isn't working

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#10313 In NVIDIA/TensorRT-LLM;

· nutriarch opened on Dec 27, 2025

[Bug]: Qwen3 + Eagle3 generates different result with greedy decoding compared to "no eagle version"

bugSomething isn't workingSomething isn't working

Decoding<NV>Token sampling algorithms in TRTLLM for text gen (top-k, top-p, beam).<NV>Token sampling algorithms in TRTLLM for text gen (top-k, top-p, beam).

Speculative Decoding<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter<NV>MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

Status: Open.

#10309 In NVIDIA/TensorRT-LLM;

· BigRabbit71 opened on Dec 26, 2025