brk-code

sglang の API リファレンス

sglang (sgl-project/sglang) の公開 API 400 件 —— クラス 199、関数 125、メソッド 76。実際のソースを静的解析して抽出した正確なシグネチャを掲載しています。

リポジトリ: sgl-project/sglang

種別件数
クラス199
関数125
メソッド76

API 一覧

classpython.sglang._mps_stub.Event
Minimal stand-in for ``torch.cuda.Event``.
classpython.sglang._mps_stub.Stream
Minimal stand-in for ``torch.cuda.Stream``.
classpython.sglang._mps_stub.StreamContext
Minimal stand-in for ``torch.cuda.StreamContext``.
funcpython.sglang._mps_stub.current_device() -> int
Return the index of the current MPS device (always 0).
funcpython.sglang._mps_stub.current_stream(device:Any=None) -> Stream
Return the default (and only) MPS stream.
funcpython.sglang._mps_stub.device_count() -> int
Return the number of available MPS devices (always 1).
funcpython.sglang._mps_stub.get_device_properties(device:Any=0) -> _MPSDeviceProperties
Return the properties of the MPS device.
funcpython.sglang._mps_stub.install() -> None
Patch ``torch.mps`` with the stubs above.
funcpython.sglang._mps_stub.set_device(device:Any) -> None
Set the current device.
funcpython.sglang._mps_stub.stream(s:Any) -> Stream
Return a context manager that is a no-op on MPS.
funcpython.sglang._triton_stub.install() -> None
Register a mock ``triton`` package in *sys.modules*.
classpython.sglang.check_env.BaseEnv
Base class for environment check
methodpython.sglang.check_env.BaseEnv.get_info() -> dict
Get CUDA-related information if available.
methodpython.sglang.check_env.BaseEnv.get_package_versions() -> dict
Get versions of specified packages.
classpython.sglang.check_env.GPUEnv
Environment checker for Nvidia GPU
methodpython.sglang.check_env.GPUEnv.get_topology()
Get GPU topology information.
classpython.sglang.check_env.HIPEnv
Environment checker for ROCm/HIP
classpython.sglang.check_env.MPSEnv
Environment checker for Apple Silicon MPS
classpython.sglang.check_env.MUSAEnv
Environment checker for MThreads GPU
methodpython.sglang.check_env.MUSAEnv.get_topology()
Get GPU topology information.
classpython.sglang.check_env.NPUEnv
Environment checker for Ascend NPU
funcpython.sglang.cli.utils.get_is_diffusion_model(model_path:str) -> bool
Detect whether model_path points to a diffusion model.
classpython.sglang.global_config.GlobalConfig
Store some global constants.
funcpython.sglang.kernel_api_logging.debug_torch_op(op_func:Callable, op_name:str, *namespace:str='sglang') -> Callable
NOTE: For internal use.
funcpython.sglang.kernels.aot.python.sgl_kernel.elementwise.rmsnorm(input:torch.Tensor, weight:torch.Tensor, eps:float=1e-06, out:Optional[torch.Tensor]=None, enable_pdl:Optional[bool]=None) -> torch.Tensor
Root mean square normalization.
funcpython.sglang.kernels.aot.python.sgl_kernel.spatial.create_greenctx_stream_by_value(SM_a:int, SM_b:int, device_id:int=None) -> tuple[ExternalStream, ExternalStream]
Create two streams for greenctx.
funcpython.sglang.kernels.aot.python.sgl_kernel.spatial.get_sm_available(device_id:int=None) -> int
Get the SMs available on the device.
funcpython.sglang.kernels.jit.utils.arch.get_jit_cuda_arch() -> ArchInfo
Get the current CUDA architecture info.
classpython.sglang.kernels.ops.activation.ReLU2Op
``out = relu(input) ** 2`` (single-input, not gated).
funcpython.sglang.kernels.ops.activation.gelu_and_mul(input:torch.Tensor, out:Optional[torch.Tensor]=None) -> torch.Tensor
``out = gelu(input[..., :d]) * input[..., d:]``.
funcpython.sglang.kernels.ops.activation.gelu_quick(input:torch.Tensor, out:Optional[torch.Tensor]=None) -> torch.Tensor
``out = input * sigmoid(1.702 * input)``.
funcpython.sglang.kernels.ops.activation.relu2(input:torch.Tensor, out:Optional[torch.Tensor]=None) -> torch.Tensor
``out = relu(input) ** 2``.
funcpython.sglang.kernels.ops.attention.dsa.tilelang_kernel.fp8_index(q:torch.Tensor, q_s:torch.Tensor, k:torch.Tensor, k_s:torch.Tensor) -> torch.Tensor
Perform index score using FP8 precision.
funcpython.sglang.kernels.ops.attention.dsv4.elementwise.fused_q_indexer_rope_first_quant(q_input:torch.Tensor, weight:torch.Tensor, weight_scale:float, cos_sin_cache:torch.Tensor, positions:torch.Tensor) -> Tuple[torch.Tensor, torch.Tensor]
DeepSeek-V3.2 only.
funcpython.sglang.kernels.ops.attention.fla.kda.cdiv(a:int, b:int) -> int
Ceiling division.
funcpython.sglang.kernels.ops.attention.fla.kda.next_power_of_2(n:int) -> int
The next power of 2 (inclusive)
classpython.sglang.kernels.ops.attention.flash_attn.cute.cache_utils.JITCache
In-memory cache for compiled functions.
methodpython.sglang.kernels.ops.attention.flash_attn.cute.cache_utils.JITCache.clear() -> None
Clear in-memory cache of compiled functions
funcpython.sglang.kernels.ops.attention.flash_attn.cute.cache_utils.get_jit_cache(name:str | None=None) -> JITCache
JIT cache factory.
funcpython.sglang.kernels.ops.attention.flash_attn.cute.cute_dsl_ptxas.patch()
Install system ptxas hook.
funcpython.sglang.kernels.ops.attention.flash_attn.cute.fa_logging.set_fa_log_level(level:int | str) -> None
Set the FA log level programmatically.
classpython.sglang.kernels.ops.attention.flash_attn.cute.mask.Sm100MaskEnum
Enumeration of mask types for FMHA operations.
funcpython.sglang.kernels.ops.attention.flash_attn.cute.pipeline.make_pipeline_state(type:PipelineUserType, stages:int)
Creates a pipeline state.
funcpython.sglang.kernels.ops.attention.flash_attn.cute.softmax.floor_if_packed(q_idx, qhead_per_kvhead:cutlass.Constexpr[int]) -> cute.Tensor
Convert q_idx to packed format for Pack-GQA.
funcpython.sglang.kernels.ops.attention.position.compute_position_triton(extend_prefix_lens:torch.Tensor, extend_seq_lens:torch.Tensor, extend_seq_lens_sum)
Compute positions.
funcpython.sglang.kernels.ops.diffusion.residual_gate_add(residual:torch.Tensor, update:torch.Tensor, gate:torch.Tensor) -> torch.Tensor
Fused ``residual + gate * update``.
funcpython.sglang.kernels.ops.diffusion.triton.mps_fallback.triton_one_pass_rms_norm_native(x:torch.Tensor, w:torch.Tensor, eps:float=1e-06) -> torch.Tensor
MLX-accelerated triton_one_pass_rms_norm.
funcpython.sglang.kernels.ops.diffusion.triton.sana_wm_gdn.fused_qk_inv_rms(qkv:torch.Tensor, eps:float=1e-05) -> tuple[torch.Tensor, torch.Tensor]
Single-pass Triton fused Q+K inverse-RMS.
funcpython.sglang.kernels.ops.diffusion.usp_relayout.usp_merge_heads(x:torch.Tensor) -> torch.Tensor
Merge Ulysses output heads with an exact eager fallback.
funcpython.sglang.kernels.ops.elementwise.elementwise.fused_sigmoid_mul(attn_output:torch.Tensor, gate:torch.Tensor, inplace:bool=False) -> torch.Tensor
Fused sigmoid-mul for attention output gating.
classpython.sglang.kernels.ops.gemm.cutedsl_bf16_gemm.WorkTileInfo
Which output tile this CTA processes.
funcpython.sglang.kernels.ops.gemm.dsv3_fused_a_gemm(mat_a:torch.Tensor, mat_b:torch.Tensor, output:Optional[torch.Tensor]=None) -> torch.Tensor
DeepSeek-V3 fused QKV-A GEMM.
funcpython.sglang.kernels.ops.gemm.kv_b_lora_absorbed.step_a_q_fwd(q_nope:torch.Tensor, B_buf:torch.Tensor, batch_info:LoRABatchInfo, full_K_per_head:int) -> torch.Tensor
Step A of the q-side correction.
funcpython.sglang.kernels.ops.gemm.kv_b_lora_absorbed.step_a_v_fwd(attn_output:torch.Tensor, A_buf:torch.Tensor, batch_info:LoRABatchInfo) -> torch.Tensor
Step A of the v-side correction.
funcpython.sglang.kernels.ops.gemm.trtllm_lora_temp.kv_b_lora_absorbed.step_a_q_fwd(q_nope:torch.Tensor, B_buf:torch.Tensor, batch_info:LoRABatchInfo, full_K_per_head:int) -> torch.Tensor
Step A of the q-side correction.
funcpython.sglang.kernels.ops.gemm.trtllm_lora_temp.kv_b_lora_absorbed.step_a_v_fwd(attn_output:torch.Tensor, A_buf:torch.Tensor, batch_info:LoRABatchInfo) -> torch.Tensor
Step A of the v-side correction.
funcpython.sglang.kernels.ops.kimi_k3.all_reduce.register_comm(comm:Communicator, *pull_sem_mc_ptr:int=0) -> None
Register the CustomAllReduceV2 storage plane.
funcpython.sglang.kernels.ops.kimi_k3.attn_res_hip.supports_attn_res_hip(hidden_size:int, nvb:int) -> bool
Whether this shape fits the register budget.
funcpython.sglang.kernels.ops.kimi_k3.gemm_ar.fits(x:torch.Tensor) -> bool
Whether this o_proj input can take the fused GEMM+AR path.
funcpython.sglang.kernels.ops.layernorm.gemma4_fused_ops.gemma4_fused_routing(gating_output:torch.Tensor, per_expert_scale:torch.Tensor, topk:int) -> tuple[torch.Tensor, torch.Tensor]
One-launch Gemma4 router.
funcpython.sglang.kernels.ops.lplb.cuda_solver.solve_ipm(A:torch.Tensor, b:torch.Tensor, c:torch.Tensor, num_iters:int=DEFAULT_NUM_ITERS, result:torch.Tensor | None=None) -> torch.Tensor
Run the fused single-SM IPM kernel.
funcpython.sglang.kernels.ops.lplb.shmem_budget.assert_fits(nc:int, nv:int, gpu:str='h100') -> None
Raise if the fused kernel will not fit on the target GPU.
funcpython.sglang.kernels.ops.lplb.shmem_budget.max_nc_for_nv(nv:int, gpu:str='h100') -> int
Largest NC that fits for a given NV.
funcpython.sglang.kernels.ops.lplb.shmem_budget.report(nc:int, nv:int, gpu:str='h100') -> str
Human-readable summary — used by kernels on init for logging.
classpython.sglang.kernels.ops.mamba.triton_ops.ssu_dispatch.TritonSSUBackend
Triton-based selective-state-update backend.
funcpython.sglang.kernels.ops.moe.ep_moe_kernels.get_tma_aligned_size(x:int, element_size:int) -> int
Global memory address of TMA must be 16-byte aligned.
funcpython.sglang.kernels.ops.moe.moe_front.get_front_strategy(num_tokens:int, device) -> str
Return the measured front strategy for this exact workload.
funcpython.sglang.kernels.ops.sampling.murmur_hash.fmix32(h:tl.uint32) -> tl.uint32
final mix of 32-bit hash value for MurmurHash
funcpython.sglang.kernels.ops.sampling.murmur_hash.murmur3_mix(h:tl.uint32, k:tl.uint32) -> tl.uint32
Mixes a 32-bit key into the hash state.
funcpython.sglang.kernels.ops.sampling.murmur_hash.rotl32(x, r:tl.constexpr) -> tl.uint32
rotate left 32-bit integer x by r bits e.g.
funcpython.sglang.kernels.ops.sampling.top_k_renorm_probs(probs:torch.Tensor, top_k:Union[torch.Tensor, int]) -> torch.Tensor
Renormalize ``probs`` by top-k thresholding.
funcpython.sglang.kernels.ops.sampling.top_p_renorm_probs(probs:torch.Tensor, top_p:Union[torch.Tensor, float]) -> torch.Tensor
Renormalize ``probs`` by top-p thresholding.
funcpython.sglang.kernels.registry.register_kernel(spec:KernelSpec) -> KernelSpec
Register ``spec`` in the process-wide :data:`registry`.
funcpython.sglang.kernels.selector.clear_cache() -> None
Drop the resolved-callable cache (used by tests).
funcpython.sglang.kernels.selector.get_kernel(op:str, backend:Optional[KernelBackend]=None) -> Callable
Resolve ``op`` to a callable kernel and cache it.
classpython.sglang.kernels.spec.DeviceType
Accelerator device family a kernel can run on.
classpython.sglang.kernels.spec.FormatSignature
A light description of a kernel's data contract.
classpython.sglang.kernels.spec.KernelSpec
A single callable kernel implementation and its metadata.
methodpython.sglang.kernels.spec.KernelSpec.is_available(platform:PlatformInfo) -> bool
Whether this backend can run on ``platform`` (metadata-only check).
methodpython.sglang.kernels.spec.KernelSpec.load() -> Callable
Import and return the backing callable.
classpython.sglang.kernels.spec.PlatformInfo
A minimal snapshot of the runtime accelerator platform.
funcpython.sglang.kernels.spec.capabilities_satisfied(capabilities:Union[FrozenSet[CapabilityRequirement], Tuple[CapabilityRequirement, ...], CapabilityRequirement], platform:PlatformInfo) -> bool
OR over ``capabilities`` (empty = unrestricted).
classpython.sglang.lang.backend.crusoe.Crusoe
SGLang backend for Crusoe managed inference.
classpython.sglang.lang.backend.runtime_endpoint.Runtime
A wrapper for the HTTP server.
classpython.sglang.lang.interpreter.ProgramState
The state of an SGL program.
classpython.sglang.multimodal_gen.apps.ComfyUI_SGLDiffusion.executors.flux.FluxExecutor
Executor for Flux models in ComfyUI.
classpython.sglang.multimodal_gen.apps.ComfyUI_SGLDiffusion.executors.zimage.ZImageExecutor
Executor for ZImage models in ComfyUI.
methodpython.sglang.multimodal_gen.configs.models.base.ModelConfig.update_model_arch(source_model_dict:dict[str, Any]) -> None
Update arch_config with source_model_dict
classpython.sglang.multimodal_gen.configs.models.dits.cosmos3video.Cosmos3VideoConfig
DiT config wrapper for Cosmos3 Video model.
classpython.sglang.multimodal_gen.configs.models.dits.ltx_2.LTX2Config
Configuration for LTX-2 Video Transformer.
classpython.sglang.multimodal_gen.configs.models.encoders.qwen3.Qwen3TextArchConfig
Architecture config for Qwen3 text encoder.
classpython.sglang.multimodal_gen.configs.models.encoders.qwen3.Qwen3TextConfig
Top-level config for Qwen3 text encoder.
classpython.sglang.multimodal_gen.configs.models.encoders.qwen3vl.Qwen3VLConfig
Configuration for Qwen3-VL text encoder.
classpython.sglang.multimodal_gen.configs.models.vaes.hunyuan3d.Hunyuan3DVAEArchConfig
Architecture config for Hunyuan3D VAE.
classpython.sglang.multimodal_gen.configs.models.vaes.hunyuan3d.Hunyuan3DVAEConfig
VAE configuration for Hunyuan3D.
classpython.sglang.multimodal_gen.configs.models.vaes.stablediffusion3.StableDiffusion3VAEConfig
Configuration for StableDiffusion3 VAE.
classpython.sglang.multimodal_gen.configs.pipeline_configs.base.PipelineConfig
The base configuration class for a generation pipeline.
classpython.sglang.multimodal_gen.configs.pipeline_configs.base.STA_Mode
STA (Sliding Tile Attention) modes.
classpython.sglang.multimodal_gen.configs.pipeline_configs.base.SlidingTileAttnConfig
Configuration for sliding tile attention.
classpython.sglang.multimodal_gen.configs.pipeline_configs.base.SpatialImagePipelineConfig
Base config for spatial image pipelines (e.g.
funcpython.sglang.multimodal_gen.configs.pipeline_configs.base.parse_int_list(value:str) -> list[int]
Parse a comma-separated string of integers into a list.
classpython.sglang.multimodal_gen.configs.pipeline_configs.cosmos3.Cosmos3Config
Cosmos3 unified pipeline config.
classpython.sglang.multimodal_gen.configs.pipeline_configs.flux.FluxPipelineConfig
Configuration for the FLUX pipeline.
classpython.sglang.multimodal_gen.configs.pipeline_configs.glm_image.GlmImagePipelineConfig
Configuration for the GlmImage pipeline.
classpython.sglang.multimodal_gen.configs.pipeline_configs.helios.HeliosT2VConfig
Configuration for the Helios T2V pipeline.
classpython.sglang.multimodal_gen.configs.pipeline_configs.krea2.Krea2PipelineConfig
Krea-2 single-stream MMDiT, text-to-image.
classpython.sglang.multimodal_gen.configs.pipeline_configs.ltx_2.LTX23PipelineConfig
Configuration overrides for LTX-2.3.
classpython.sglang.multimodal_gen.configs.pipeline_configs.ltx_2.LTX2PipelineConfig
Configuration for LTX-Video pipeline.
classpython.sglang.multimodal_gen.configs.pipeline_configs.qwen_image.QwenImagePipelineConfig
Configuration for the QwenImage pipeline.
funcpython.sglang.multimodal_gen.configs.sample.glmimage.align_glm_image_dimension(value:int) -> int
Round a GLM-Image dimension up to a supported multiple.
classpython.sglang.multimodal_gen.configs.sample.joy_echo.JoyEchoSamplingParams
Sampling parameters for JoyEcho DMD inference.
classpython.sglang.multimodal_gen.configs.sample.ltx_2.LTX2SamplingParams
Sampling parameters for LTX-2.
classpython.sglang.multimodal_gen.configs.sample.sampling_params.SamplingParams
Sampling parameters for generation.
methodpython.sglang.multimodal_gen.configs.sample.sampling_params.SamplingParams.add_cli_args(parser:Any) -> Any
Add CLI arguments for SamplingParam fields
classpython.sglang.multimodal_gen.configs.sample.sana.SanaSamplingParams
Defaults for SANA 1.5 1024px variant.
classpython.sglang.multimodal_gen.configs.sample.wan.Wan2_2_Base_SamplingParams
Sampling parameters for Wan2.2 TI2V 5B model.
classpython.sglang.multimodal_gen.configs.sample.wan.Wan2_2_TI2V_5B_SamplingParam
Sampling parameters for Wan2.2 TI2V 5B model.
funcpython.sglang.multimodal_gen.registry.get_non_diffusers_pipeline_name(model_path:str) -> Optional[str]
Get the pipeline name for a known non-diffusers model.
funcpython.sglang.multimodal_gen.registry.get_pipeline_config_classes(pipeline_class_name:str) -> Tuple[Type[PipelineConfig], Type[Any]] | None
Get the configuration classes for a pipeline.
classpython.sglang.multimodal_gen.runtime.cache.cache_dit_integration.CacheDitConfig
Configuration for cache-dit integration.
classpython.sglang.multimodal_gen.runtime.cache.spectrum.SpectrumForecaster
Chebyshev + discrete Taylor blend forecaster.
funcpython.sglang.multimodal_gen.runtime.disaggregation.roles.get_module_role(module_name:str) -> 'RoleType | None'
Classify a module name to its primary role.
funcpython.sglang.multimodal_gen.runtime.disaggregation.transport.protocol.encode_transfer_msg(msg:Any) -> list[bytes]
Encode as [TRANSFER_MAGIC, json_payload_bytes].
funcpython.sglang.multimodal_gen.runtime.distributed.group_coordinator.get_local_torch_device() -> torch.device
Return the torch device for the current rank.
funcpython.sglang.multimodal_gen.runtime.distributed.parallel_state.destroy_model_parallel() -> None
Set the groups to none and destroy them.
funcpython.sglang.multimodal_gen.runtime.distributed.parallel_state.get_dp_rank() -> int
Return my rank for the data parallel group.
funcpython.sglang.multimodal_gen.runtime.distributed.parallel_state.get_world_rank() -> int
Return my rank for the world group.
funcpython.sglang.multimodal_gen.runtime.distributed.parallel_state.get_world_size() -> int
Return world size for the world group.
classpython.sglang.multimodal_gen.runtime.entrypoints.cli.cli_types.CLISubcommand
Base class for CLI subcommands
methodpython.sglang.multimodal_gen.runtime.entrypoints.cli.cli_types.CLISubcommand.validate(args:argparse.Namespace) -> None
Validate the arguments for this command
funcpython.sglang.multimodal_gen.runtime.entrypoints.http_server.model_info_endpoint(request:Request)
Get model information.
funcpython.sglang.multimodal_gen.runtime.entrypoints.http_server.server_info_endpoint(request:Request)
Get server information.
classpython.sglang.multimodal_gen.runtime.entrypoints.openai.common_api.ModelCard
Model cards.
funcpython.sglang.multimodal_gen.runtime.entrypoints.openai.common_api.available_models()
Show available models.
funcpython.sglang.multimodal_gen.runtime.entrypoints.openai.common_api.retrieve_model(model:str)
Retrieve a model instance.
classpython.sglang.multimodal_gen.runtime.entrypoints.openai.realtime.generate_session.GenerateSession
A realtime generation session
funcpython.sglang.multimodal_gen.runtime.entrypoints.openai.utils.build_sampling_params(request_id:str, **kwargs) -> SamplingParams
Build SamplingParams from request parameters.
funcpython.sglang.multimodal_gen.runtime.launch_server.parse_url_string(url_str:str) -> list[str]
Parse a semicolon-separated URL string into a list.
classpython.sglang.multimodal_gen.runtime.layers.activation.GeluAndMul
An activation function for GeGLU.
classpython.sglang.multimodal_gen.runtime.layers.activation.SiluAndMul
An activation function for SwiGLU.
methodpython.sglang.multimodal_gen.runtime.layers.activation.SiluAndMul.forward_native(x:torch.Tensor) -> torch.Tensor
PyTorch-native implementation equivalent to forward().
funcpython.sglang.multimodal_gen.runtime.layers.activation.get_act_fn(act_fn_name:str) -> nn.Module
Get an activation function by name.
classpython.sglang.multimodal_gen.runtime.layers.attention.backends.attention_backend.AttentionBackend
Abstract class for attention backends.
classpython.sglang.multimodal_gen.runtime.layers.attention.layer.LocalAttention
Attention layer.
classpython.sglang.multimodal_gen.runtime.layers.attention.turbo_layer.DistributedAttention
Initialization.
classpython.sglang.multimodal_gen.runtime.layers.custom_op.CustomOp
Base class for custom ops.
classpython.sglang.multimodal_gen.runtime.layers.layernorm.RMSNorm
Root mean square normalization.
classpython.sglang.multimodal_gen.runtime.layers.linear.ColumnParallelLinear
Linear layer with column parallelism.
classpython.sglang.multimodal_gen.runtime.layers.linear.LinearBase
Base linear layer.
classpython.sglang.multimodal_gen.runtime.layers.linear.LinearMethodBase
Base class for different (maybe quantized) linear methods.
methodpython.sglang.multimodal_gen.runtime.layers.linear.LinearMethodBase.apply(layer:torch.nn.Module, x:torch.Tensor, bias:torch.Tensor | None=None) -> torch.Tensor
Apply the weights in layer to the input tensor.
classpython.sglang.multimodal_gen.runtime.layers.linear.MergedColumnParallelLinear
Packed linear layers with column parallelism.
classpython.sglang.multimodal_gen.runtime.layers.linear.ReplicatedLinear
Replicated linear layer.
classpython.sglang.multimodal_gen.runtime.layers.linear.RowParallelLinear
Linear layer with row parallelism.
classpython.sglang.multimodal_gen.runtime.layers.linear.UnquantizedLinearMethod
Linear method without quantization.
classpython.sglang.multimodal_gen.runtime.layers.mlp.FeedForward
A feed-forward layer.
classpython.sglang.multimodal_gen.runtime.layers.mlp.MLP
MLP for DiT blocks, NO gated linear units
funcpython.sglang.multimodal_gen.runtime.layers.parallel_conv.gather_height_sizes(x:torch.Tensor) -> list[int]
gather heights of sharded feature_maps from peers
classpython.sglang.multimodal_gen.runtime.layers.quantization.configs.base_config.QuantizationConfig
Base class for quantization configs.
classpython.sglang.multimodal_gen.runtime.layers.quantization.configs.base_config.QuantizeMethodBase
Base class for different quantized methods.
methodpython.sglang.multimodal_gen.runtime.layers.quantization.configs.base_config.QuantizeMethodBase.process_weights_after_loading(layer:nn.Module) -> None
Process the weight after loading.
classpython.sglang.multimodal_gen.runtime.layers.quantization.fp8.Fp8Config
Config class for FP8.
classpython.sglang.multimodal_gen.runtime.layers.quantization.fp8.Fp8LinearMethod
Linear method for FP8.
classpython.sglang.multimodal_gen.runtime.layers.quantization.modelopt_quant.ModelOptFp4Config
Config class for NVFP4.
classpython.sglang.multimodal_gen.runtime.layers.rotary_embedding.base.RotaryEmbedding
Original rotary positional embedding.
classpython.sglang.multimodal_gen.runtime.layers.rotary_embedding.mrope.NDRotaryEmbedding
N-dimensional rotary positional embedding.
classpython.sglang.multimodal_gen.runtime.layers.rotary_embedding.mrope.OneDRotaryEmbedding
1D rotary positional embedding with caching.
classpython.sglang.multimodal_gen.runtime.layers.visual_embedding.ModulateProjection
Modulation layer for DiT blocks.
classpython.sglang.multimodal_gen.runtime.layers.vocab_parallel_embedding.UnquantizedEmbeddingMethod
Unquantized method for embeddings.
funcpython.sglang.multimodal_gen.runtime.layers.vocab_parallel_embedding.pad_vocab_size(vocab_size:int, pad_to:int=DEFAULT_VOCAB_PADDING_SIZE) -> int
Pad the vocab size to the given value.
classpython.sglang.multimodal_gen.runtime.loader.component_loaders.component_loader.AutoProcessorLoader
Loader for auto processor.
classpython.sglang.multimodal_gen.runtime.loader.component_loaders.component_loader.ImageProcessorLoader
Loader for image processor.
classpython.sglang.multimodal_gen.runtime.loader.component_loaders.component_loader.TokenizerLoader
Loader for tokenizers.
classpython.sglang.multimodal_gen.runtime.loader.component_loaders.scheduler_loader.SchedulerLoader
Loader for scheduler.
classpython.sglang.multimodal_gen.runtime.loader.component_loaders.text_encoder_loader.TextEncoderLoader
Loader for text encoders.
classpython.sglang.multimodal_gen.runtime.loader.component_loaders.text_encoder_loader.TextEncoderLoader.Source
A source for weights.
funcpython.sglang.multimodal_gen.runtime.loader.weight_utils.default_weight_loader(param:torch.Tensor, loaded_weight:torch.Tensor) -> None
Default weight loader.
funcpython.sglang.multimodal_gen.runtime.loader.weight_utils.maybe_remap_kv_scale_name(name:str, params_dict:dict) -> str | None
Remap the name of FP8 k/v_scale parameters.
funcpython.sglang.multimodal_gen.runtime.managers.forward_context.get_forward_context() -> 'ForwardContext'
Get the current forward context.
classpython.sglang.multimodal_gen.runtime.managers.gpu_worker.GPUWorker
A worker that executes the model on a single GPU.
methodpython.sglang.multimodal_gen.runtime.managers.gpu_worker.GPUWorker.execute_forward(batch:List[Req], return_req:bool=False) -> OutputBatch | Req
Execute a forward pass.
methodpython.sglang.multimodal_gen.runtime.managers.gpu_worker.GPUWorker.merge_lora_weights(target:str='all', strength:float=1.0) -> OutputBatch
Merge LoRA weights.
methodpython.sglang.multimodal_gen.runtime.managers.gpu_worker.GPUWorker.release_realtime_session(session_id:str) -> OutputBatch
release the session of a realtime connection
methodpython.sglang.multimodal_gen.runtime.managers.gpu_worker.GPUWorker.unmerge_lora_weights(target:str='all') -> OutputBatch
Unmerge LoRA weights.
classpython.sglang.multimodal_gen.runtime.managers.scheduler.Scheduler
Runs the main event loop for the rank 0 worker.
classpython.sglang.multimodal_gen.runtime.models.bridges.mova_dual_tower.PerFrameAttentionPooling
Per-frame multi-head attention pooling.
methodpython.sglang.multimodal_gen.runtime.models.bridges.mova_dual_tower.PerFrameAttentionPooling.forward(x:torch.Tensor, grid_size:Tuple[int, int, int]) -> torch.Tensor
Forward pass.
funcpython.sglang.multimodal_gen.runtime.models.dits.common.modulate(x:torch.Tensor, shift:torch.Tensor | None=None, scale:torch.Tensor | None=None) -> torch.Tensor
Modulate by shift and scale.
classpython.sglang.multimodal_gen.runtime.models.dits.cosmos3video.Cosmos3GatedMLP
Gated MLP (SwiGLU-style) for Cosmos3.
classpython.sglang.multimodal_gen.runtime.models.dits.cosmos3video.Cosmos3OmniTransformer
Cosmos3 Omni transformer.
classpython.sglang.multimodal_gen.runtime.models.dits.flux.FluxTransformer2DModel
The Transformer model introduced in Flux.
classpython.sglang.multimodal_gen.runtime.models.dits.flux_2.Flux2Transformer2DModel
The Transformer model introduced in Flux 2.
classpython.sglang.multimodal_gen.runtime.models.dits.glm_image.GlmImageKVCache
Container for all layers' KV caches.
classpython.sglang.multimodal_gen.runtime.models.dits.glm_image.GlmImageLayerKVCache
KV cache for GlmImage model.
classpython.sglang.multimodal_gen.runtime.models.dits.joy_image.ModulateWan
Modulation layer for WanX.
funcpython.sglang.multimodal_gen.runtime.models.dits.joy_image.fused_add_gate(residual:torch.Tensor, x:torch.Tensor, gate:torch.Tensor) -> torch.Tensor
Fused residual addition with gate.
classpython.sglang.multimodal_gen.runtime.models.dits.mova_audio_dit.Conv1dLocalIsland
Inherits from Conv1d and overrides forward.
classpython.sglang.multimodal_gen.runtime.models.dits.mova_video_dit.CrossAttention
Cross-Attention module for MOVA DiT.
classpython.sglang.multimodal_gen.runtime.models.dits.qwen_image.QwenImageTransformer2DModel
The Transformer model introduced in Qwen.
classpython.sglang.multimodal_gen.runtime.models.dits.sana.GLUMBConv
Gated Linear Unit with Multi-Branch Convolution.
classpython.sglang.multimodal_gen.runtime.models.dits.sana_wm.SanaWMBlock
One transformer block of SANA-WM.
classpython.sglang.multimodal_gen.runtime.models.dits.sana_wm.SanaWMTransformer3DModel
SANA-WM 2.6B TI2V world model.
classpython.sglang.multimodal_gen.runtime.models.dits.sana_wm_components.TimestepEmbedder
Upstream ``TimestepEmbedder``.
classpython.sglang.multimodal_gen.runtime.models.dits.sana_wm_refiner_transformer.SanaWMRefinerBlock
Video-only LTX-2 transformer block.
classpython.sglang.multimodal_gen.runtime.models.encoders.gemma2.Gemma2Model
Gemma2 text encoder model for SANA pipeline.
classpython.sglang.multimodal_gen.runtime.models.encoders.gemma_3.Gemma3MultiModalProjector
Projector for Gemma3 multimodal.
classpython.sglang.multimodal_gen.runtime.models.encoders.qwen3.Qwen3DecoderLayer
Qwen3 transformer decoder layer.
classpython.sglang.multimodal_gen.runtime.models.encoders.t5.AttentionType
Attention type.
classpython.sglang.multimodal_gen.runtime.models.parameter.BasevLLMParameter
Base parameter for vLLM linear layers.
classpython.sglang.multimodal_gen.runtime.models.parameter.ModelWeightParameter
Parameter class for linear layer weights.
classpython.sglang.multimodal_gen.runtime.models.schedulers.scheduling_flow_match_euler_discrete.FlowMatchEulerDiscreteScheduler
Euler scheduler.
classpython.sglang.multimodal_gen.runtime.models.vaes.autoencoder_kl_qwenimage.QwenImageDecoder3d
A 3D decoder module.
classpython.sglang.multimodal_gen.runtime.models.vaes.autoencoder_kl_qwenimage.QwenImageEncoder3d
A 3D encoder module.
classpython.sglang.multimodal_gen.runtime.models.vaes.autoencoder_kl_qwenimage.QwenImageRMS_norm
A custom RMS normalization layer.
classpython.sglang.multimodal_gen.runtime.models.vaes.autoencoder_kl_qwenimage.QwenImageResidualBlock
A custom residual block module.
classpython.sglang.multimodal_gen.runtime.models.vaes.hunyuan3d_vae.Latent2MeshOutput
Container for mesh output from VAE decoder.
classpython.sglang.multimodal_gen.runtime.models.vaes.hunyuan3d_vae.MCSurfaceExtractor
Marching Cubes surface extractor.
classpython.sglang.multimodal_gen.runtime.models.vaes.hunyuan3d_vae.SurfaceExtractor
Base class for surface extraction algorithms.
classpython.sglang.multimodal_gen.runtime.models.vaes.hunyuan3d_vae.VectsetVAE
Base VAE class for vector set encoding.
methodpython.sglang.multimodal_gen.runtime.models.vaes.hunyuan3d_vae.VectsetVAE.latents2mesh(latents:torch.FloatTensor, **kwargs)
Convert latents to mesh.
classpython.sglang.multimodal_gen.runtime.models.vaes.ltx_2_audio.LTX2AudioAudioPatchifier
Patchifier for spectrogram/audio latents.
classpython.sglang.multimodal_gen.runtime.models.vaes.ltx_2_vae.AutoencoderKLCausalLTX2Video
Streaming causal LTX-2 VAE.
classpython.sglang.multimodal_gen.runtime.models.vaes.ltx_2_vae.LTX2VideoDownBlock3D
Down block used in the LTXVideo model.
classpython.sglang.multimodal_gen.runtime.models.vaes.ltx_2_vae.LTX2VideoMidBlock3d
A middle block used in the LTXVideo model.
classpython.sglang.multimodal_gen.runtime.models.vaes.ltx_2_vae.LTX2VideoUpBlock3d
Up block used in the LTXVideo model.
funcpython.sglang.multimodal_gen.runtime.models.vaes.minimax_h3_audio_vae.alias_free.sinc(x:torch.Tensor)
Implementation of sinc, i.e.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanAttentionBlock
Causal self-attention with a single head.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanDecoder3d
A 3D decoder module.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanEncoder3d
A 3D encoder module.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanMidBlock
Middle block for WanVAE encoder and decoder.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanRMS_norm
A custom RMS normalization layer.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanResample
A custom resampling module for 2D and 3D data.
classpython.sglang.multimodal_gen.runtime.models.vaes.wanvae.WanResidualBlock
A custom residual block module.
classpython.sglang.multimodal_gen.runtime.pipelines.hunyuan3d_pipeline.Hunyuan3D2Pipeline
Hunyuan3D 2.0 image-to-mesh pipeline.
classpython.sglang.multimodal_gen.runtime.pipelines.ltx_2_pipeline.LTX2TwoStageResidencyController
LTX-2.3 two-stage residency controller.
classpython.sglang.multimodal_gen.runtime.pipelines.mova_pipeline.MOVAPipeline
MOVA pipeline with SGLang stage orchestration.
methodpython.sglang.multimodal_gen.runtime.pipelines.mova_pipeline.MOVAPipeline.initialize_pipeline(server_args:ServerArgs) -> None
Initialize the pipeline.
classpython.sglang.multimodal_gen.runtime.pipelines.sana_wm_pipeline.SanaWMPipeline
SANA-WM TI2V pipeline (single-stage).
classpython.sglang.multimodal_gen.runtime.pipelines.sana_wm_realtime_pipeline.SanaWMRealtimePipeline
SANA-WM realtime interactive pipeline.
classpython.sglang.multimodal_gen.runtime.pipelines.wan_pipeline.WanPipeline
Wan video diffusion pipeline with LoRA support.
funcpython.sglang.multimodal_gen.runtime.pipelines_core.build_pipeline(server_args:ServerArgs) -> PipelineWithLoRA
Only works with valid hf diffusers configs.
classpython.sglang.multimodal_gen.runtime.pipelines_core.schedule_batch.OutputBatch
Final output (after pipeline completion)
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.base.PipelineStage
Abstract base class for all pipeline stages.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.base.PipelineStage.device() -> torch.device
Get the device for this stage.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.base.PipelineStage.forward(batch:Req, server_args:ServerArgs) -> Req
Forward pass of the stage's processing.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.base.PipelineStage.verify_input(batch:Req, server_args:ServerArgs) -> VerificationResult
Verify the input for the stage.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.base.PipelineStage.verify_output(batch:Req, server_args:ServerArgs) -> VerificationResult
Verify the output for the stage.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.causal_denoising.CausalDMDDenoisingStage
Denoising stage for causal diffusion.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.causal_denoising.CausalDMDDenoisingStage.verify_input(batch:Req, server_args:ServerArgs) -> VerificationResult
Verify denoising stage inputs.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.denoising_dmd.DmdDenoisingStage
Denoising stage for DMD.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.denoising_dmd.DmdDenoisingStage.forward(batch:Req, server_args:ServerArgs) -> Req
Run the denoising loop.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.cosmos3.Cosmos3DecodingStage
VAE decoding stage for Cosmos3.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.cosmos3.Cosmos3TokenizationStage
Tokenization stage for Cosmos3.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.hunyuan3d.paint.DDIMSolver
DDIM solver for fast sampling.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.minimax_h3.task_profiles.MiniMaxH3TaskProfile
One row of the task table.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.mova.MOVADenoisingStage
Run MOVA dual-tower denoising loop.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.mova.MOVADenoisingStage.verify_input(batch:Req, server_args:ServerArgs) -> VerificationResult
Verify denoising stage inputs.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.mova.MOVADenoisingStage.verify_output(batch:Req, server_args:ServerArgs) -> VerificationResult
Verify denoising stage outputs.
funcpython.sglang.multimodal_gen.runtime.pipelines_core.stages.model_specific_stages.sana_wm.parity_probe.dump_obj(dirpath:Path | str | None, name:str, obj) -> None
Save a picklable object (e.g.
funcpython.sglang.multimodal_gen.runtime.pipelines_core.stages.progressive_resolution.spectral_ops.dct_1d(x:torch.Tensor, norm:str='ortho') -> torch.Tensor
1-D DCT-II via torch.fft.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators
Common validators for pipeline stages.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.bool_value(value:Any) -> bool
Check if value is a boolean.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.divisible_by(value:Any, divisor:int) -> bool
Check if value is divisible by divisor.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.is_list(value:Any) -> bool
Check if value is a list (can be empty).
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.is_tuple(value:Any) -> bool
Check if value is a tuple.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.list_length(value:Any, length:int) -> bool
Check if list has specific length.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.list_not_empty(value:Any) -> bool
Check if value is a non-empty list.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.non_negative_float(value:Any) -> bool
Check if value is a non-negative float.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.non_negative_int(value:Any) -> bool
Check if value is a non-negative float.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.none_or_list(value:Any) -> bool
Check if value is None or a list.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.none_or_positive_int(value:Any) -> bool
Check if value is None or a positive integer.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.not_none(value:Any) -> bool
Check if value is not None.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.positive_float(value:Any) -> bool
Check if value is a positive float.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.positive_int(value:Any) -> bool
Check if value is a positive integer.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.string_not_empty(value:Any) -> bool
Check if value is a non-empty string.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.StageValidators.string_not_none(value:Any) -> bool
Check if value is a non-empty string.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.ValidationFailure
Details about a specific validation failure.
classpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.VerificationResult
Wrapper class for stage verification results.
methodpython.sglang.multimodal_gen.runtime.pipelines_core.stages.validators.VerificationResult.is_valid() -> bool
Check if all validations passed.
funcpython.sglang.multimodal_gen.runtime.platforms.cpu_platform_plugin() -> str | None
Detect if CPU platform should be used.
methodpython.sglang.multimodal_gen.runtime.platforms.interface.Platform.get_current_memory_usage(device:torch.types.Device | None=None) -> float
Return the memory usage in bytes.
methodpython.sglang.multimodal_gen.runtime.platforms.interface.Platform.get_device_name(device_id:int=0) -> str
Get the name of a device.
methodpython.sglang.multimodal_gen.runtime.platforms.interface.Platform.get_device_total_memory(device_id:int=0) -> int
Get the total memory of a device in bytes.
methodpython.sglang.multimodal_gen.runtime.platforms.interface.Platform.get_device_uuid(device_id:int=0) -> str
Get the uuid of a device, e.g.
methodpython.sglang.multimodal_gen.runtime.platforms.interface.Platform.seed_everything(seed:int | None=None) -> None
Set the seed of each random module.
funcpython.sglang.multimodal_gen.runtime.platforms.xpu_platform_plugin() -> str | None
Detect if Intel XPU platform is available.
classpython.sglang.multimodal_gen.runtime.postprocess.realesrgan_upscaler.ImageUpscaler
Lazy-loaded Real-ESRGAN upscaler.
methodpython.sglang.multimodal_gen.runtime.postprocess.realesrgan_upscaler.ImageUpscaler.upscale(frames:list[np.ndarray]) -> list[np.ndarray]
Upscale a list of HWC uint8 frames.
classpython.sglang.multimodal_gen.runtime.postprocess.realesrgan_upscaler.RRDB
Residual in Residual Dense Block.
methodpython.sglang.multimodal_gen.runtime.postprocess.rife_interpolator.Model.load_model(path:str, strip_module_prefix:bool=True) -> None
Load weights from {path}/flownet.pkl.
funcpython.sglang.multimodal_gen.runtime.postprocess.rife_interpolator.warp(tenInput:torch.Tensor, tenFlow:torch.Tensor) -> torch.Tensor
Warp tenInput by tenFlow using grid_sample.
classpython.sglang.multimodal_gen.runtime.realtime.session.BaseRealtimeState
per-session state owned by pipeline stages
classpython.sglang.multimodal_gen.runtime.server_args.server_args.Backend
Enumeration for different model backends.
methodpython.sglang.multimodal_gen.runtime.server_args.server_args.Backend.from_string(value:str) -> 'Backend'
Convert string to Backend enum.
funcpython.sglang.multimodal_gen.runtime.server_warmup.get_first_generation_req(req_or_group:Any) -> Req | None
Extract the first req
funcpython.sglang.multimodal_gen.runtime.server_warmup.is_realtime_serving(server_args:ServerArgs) -> bool
Synthetic warmup has no realtime session state.
funcpython.sglang.multimodal_gen.runtime.server_warmup.is_warmup_req(req_or_group:Any) -> bool
either server-based or req-based
funcpython.sglang.multimodal_gen.runtime.utils.common.add_prefix(name:str, prefix:str) -> str
Add a weight path prefix to a module name.
funcpython.sglang.multimodal_gen.runtime.utils.hf_diffusers_utils.check_gguf_file(model:str | os.PathLike) -> bool
Check if the file is a GGUF model.
methodpython.sglang.multimodal_gen.runtime.utils.logging_utils.ColoredFormatter.format(record:logging.LogRecord) -> str
Adds color to the log
classpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.ImageProcessorV2
Image processor for Hunyuan3D single-view input.
classpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.MVImageProcessorV2
Multi-view image processor for Hunyuan3D.
funcpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.export_to_trimesh(mesh_output:Any) -> Any
Convert mesh output to trimesh format.
funcpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.get_perspective_projection_matrix(fovy:float, aspect_wh:float, near:float, far:float) -> np.ndarray
Compute perspective projection matrix.
funcpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.mesh_uv_wrap(mesh:Any) -> Any
Apply UV unwrapping to mesh.
funcpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.stride_from_shape(shape:Tuple[int, ...]) -> List[int]
Compute stride from shape for scatter operations.
funcpython.sglang.multimodal_gen.runtime.utils.mesh3d_utils.transform_pos(mtx:Union[np.ndarray, torch.Tensor], pos:torch.Tensor, keepdim:bool=False) -> torch.Tensor
Transform positions by a matrix.
classpython.sglang.multimodal_gen.runtime.utils.trace_wrapper.DiffStage
Named trace stages for the diffusion pipeline.
funcpython.sglang.multimodal_gen.runtime.utils.vision.load_image(image:str | bytes | PIL.Image.Image, convert_method:Callable[[PIL.Image.Image], PIL.Image.Image] | None=None) -> PIL.Image.Image
Loads `image` to a PIL Image.
funcpython.sglang.multimodal_gen.runtime.utils.vision.normalize(images:np.ndarray | torch.Tensor) -> np.ndarray | torch.Tensor
Normalize an image array to [-1,1].
funcpython.sglang.multimodal_gen.runtime.utils.vision.numpy_to_pt(images:np.ndarray) -> torch.Tensor
Convert a NumPy image to a PyTorch tensor.
funcpython.sglang.multimodal_gen.runtime.utils.vision.resize(image:PIL.Image.Image | np.ndarray | torch.Tensor, height:int, width:int, resize_mode:str='default', resample:str='lanczos') -> PIL.Image.Image | np.ndarray | torch.Tensor
Resize image.
funcpython.sglang.multimodal_gen.runtime.vla.observation.tensor_fingerprint(tensor:torch.Tensor) -> str
Hash tensor content with SRT's CPU/CUDA implementation.
funcpython.sglang.multimodal_gen.utils.get_mixed_precision_state() -> MixedPrecisionState
Get the current mixed precision state.
funcpython.sglang.multimodal_gen.utils.resolve_obj_by_qualname(qualname:str) -> Any
Resolve an object by its fully qualified name.
funcpython.sglang.multimodal_gen.utils.warn_for_unimplemented_methods(cls:type[T]) -> type[T]
A replacement for `abc.ABC`.
funcpython.sglang.srt.arg_groups.deepseek_v4_hook.validate_deepseek_v4_cp(server_args:ServerArgs) -> None
Validate DeepSeek V4 context-parallel configuration.
funcpython.sglang.srt.arg_groups.overrides.run_post_process_pass(server_args:Any, fn:Callable[..., dict]) -> None
Invoke one pass at its legacy handler slot.
funcpython.sglang.srt.batch_invariant_ops.batch_invariant_ops.log_softmax(input:torch.Tensor, dim:int=-1) -> torch.Tensor
Compute log_softmax using Triton kernel.
funcpython.sglang.srt.batch_invariant_ops.batch_invariant_ops.rms_norm(input:torch.Tensor, weight:torch.Tensor, eps:float=1e-06) -> torch.Tensor
Compute RMS normalization using Triton kernel.
funcpython.sglang.srt.compilation.backend.set_model_tag(tag:str)
Context manager to set the model tag.
funcpython.sglang.srt.compilation.inductor_pass.get_pass_context() -> PassContext
Get the current pass context.
classpython.sglang.srt.compilation.pass_manager.PostGradPassManager
The pass manager for post-grad passes.
classpython.sglang.srt.configs.dbrx.DbrxAttentionConfig
Configuration class for Dbrx Attention.
classpython.sglang.srt.configs.dbrx.DbrxConfig
Configuration class for Dbrx.
classpython.sglang.srt.configs.dbrx.DbrxFFNConfig
Configuration class for Dbrx FFN.
classpython.sglang.srt.configs.embedding_model_spec.EmbeddingExecution
The model path that produces an embedding.
classpython.sglang.srt.configs.internvl.InternLM2Tokenizer
Construct a InternLM2 tokenizer.
methodpython.sglang.srt.configs.internvl.InternLM2Tokenizer.get_vocab()
Returns vocab as a dict
methodpython.sglang.srt.configs.internvl.InternLM2Tokenizer.vocab_size()
Returns vocab size
classpython.sglang.srt.configs.kimi_k25.KimiK25Config
K2-VL model configuration.
methodpython.sglang.srt.configs.kimi_k25.KimiK25Config.vocab_size() -> int
Get vocab size from text config for compatibility.
classpython.sglang.srt.configs.lfm2.Lfm2Config
SGLang configuration for LFM2 models.
methodpython.sglang.srt.configs.lfm2.Lfm2Config.mamba_chunk_size() -> int
Return chunk size for Mamba2 backend.
classpython.sglang.srt.configs.lfm2_vl.Lfm2VlConfig
SGLang configuration for LFM2-VL models.
methodpython.sglang.srt.configs.lfm2_vl.Lfm2VlConfig.mamba_chunk_size() -> int
Return chunk size for Mamba2 backend.
methodpython.sglang.srt.configs.model_config.ModelConfig.get_num_kv_heads(tensor_parallel_size) -> int
Returns the number of KV heads per GPU.
methodpython.sglang.srt.configs.model_config.ModelConfig.get_total_num_kv_heads() -> int
Returns the total number of KV heads.
classpython.sglang.srt.connector.azure.AzureBlobConnector
File connector for Azure Blob Storage.
methodpython.sglang.srt.connector.serde.serde.Deserializer.from_bytes(bs:bytes) -> torch.Tensor
Deserialize a pytorch tensor from bytes.
methodpython.sglang.srt.connector.serde.serde.Serializer.to_bytes(t:torch.Tensor) -> bytes
Serialize a pytorch tensor to bytes.
funcpython.sglang.srt.connector.utils.parse_model_name(url:str) -> str
Parse the model name from the url.
classpython.sglang.srt.debug_utils.comparator.aligner.token_aligner.smart.types.TokenAlignerPlan
Token alignment plan.
funcpython.sglang.srt.debug_utils.comparator.dims_spec.tensor_naming.find_dim_index(dim_specs:list[DimSpec], name:str) -> Optional[int]
Find index by name.
funcpython.sglang.srt.debug_utils.comparator.dims_spec.tensor_naming.get_dim_names(tensor:torch.Tensor) -> tuple[Optional[str], ...]
Get dimension names attached to a tensor.
methodpython.sglang.srt.debug_utils.comparator.dims_spec.types.DimSpec.sub_dims() -> list[str]
Sub-dim names.
funcpython.sglang.srt.debug_utils.comparator.entrypoint.parse_args(argv:list[str]) -> argparse.Namespace
Parse CLI arguments from an argv list.
classpython.sglang.srt.debug_utils.comparator.report_sink.ReportSink
Unified entry point for all record output.
funcpython.sglang.srt.debug_utils.comparator.utils.calc_per_token_rel_diff(x:torch.Tensor, y:torch.Tensor, *seq_dim:int) -> torch.Tensor
Cosine-distance-like metric per token position.
classpython.sglang.srt.debug_utils.source_patcher.types.PatchApplicationError
match text not found or not unique in source.
classpython.sglang.srt.disaggregation.base.conn.BaseKVManager
Base class for managing transfer states
classpython.sglang.srt.disaggregation.common.staging_handler.DecodeStagingContext
Staging-specific context for decode mode.
methodpython.sglang.srt.disaggregation.common.staging_handler.DecodeStagingHandler.create(kv_manager, scheduler, tp_rank:int) -> DecodeStagingHandler
Factory: create handler.
methodpython.sglang.srt.disaggregation.common.staging_handler.DecodeStagingHandler.is_staging_room(room:int) -> bool
Check if a room is registered for staging scatter.
methodpython.sglang.srt.disaggregation.common.staging_handler.DecodeStagingHandler.submit_last_scatter_async(room:int) -> bool
Record all-ranks Success.
classpython.sglang.srt.disaggregation.common.staging_handler.PrefillStagingContext
Staging-specific context for prefill mode.
classpython.sglang.srt.disaggregation.decode.DecodePreallocQueue
Store the requests that are preallocating.
methodpython.sglang.srt.disaggregation.decode.DecodePreallocQueue.add(req:Req, is_retracted:bool=False, is_rebootstrap:bool=False) -> None
Add a request to the pending queue.
methodpython.sglang.srt.disaggregation.decode.DecodePreallocQueue.extend(reqs:List[Req], is_retracted:bool=False) -> None
Add a request to the pending queue.
classpython.sglang.srt.disaggregation.decode.DecodeTransferQueue
Store the requests that is polling kv
classpython.sglang.srt.disaggregation.encode_grpc_server.SGLangEncoderServer
gRPC service implementation for SGLang encoder.
funcpython.sglang.srt.disaggregation.encode_receiver.extract_original_req_id(part_req_id:str) -> str
Extract the original request ID from a part request ID.
funcpython.sglang.srt.disaggregation.encode_receiver.video_meta_attrs_for(model_type:Optional[str]) -> tuple
Video-meta attrs for model_type.
classpython.sglang.srt.disaggregation.kv_events.KVCacheEvent
Base class for all KV cache-related events
classpython.sglang.srt.disaggregation.kv_events.KVEventsConfig
Configuration for KV event publishing.
methodpython.sglang.srt.disaggregation.kv_events.KVEventsConfig.from_cli(cli_value:str) -> 'KVEventsConfig'
Parse the CLI value for the event publisher config.
classpython.sglang.srt.disaggregation.kv_events.NullEventPublisher
No-op implementation (default when disabled).
classpython.sglang.srt.disaggregation.kv_events.StorageMedium
Storage tier for KV cache events.
classpython.sglang.srt.disaggregation.nixl.conn.TransferInfo
Contains indices for a transfer, sent by KVReceiver.
classpython.sglang.srt.disaggregation.nixl.conn.TransferStatus
Used by KV Receiver to know when a transfer is done.
classpython.sglang.srt.disaggregation.prefill.PrefillBootstrapQueue
Store the requests in bootstrapping
methodpython.sglang.srt.disaggregation.prefill.PrefillBootstrapQueue.finalize_bootstrap(req:Req) -> bool
Initialize the sender after bootstrap completes.
funcpython.sglang.srt.disaggregation.prefill.should_force_retry(req:Req) -> bool
Test hook to force a request into optimistic prefill retry.
funcpython.sglang.srt.distributed.device_communicators.custom_all_reduce_utils.gpu_p2p_access_check(src:int, tgt:int) -> bool
Check if GPU src can access GPU tgt.
funcpython.sglang.srt.distributed.utils.get_global_tcp_store() -> Optional[TCPStore]
Get the existing global TCPStore.
funcpython.sglang.srt.distributed.utils.get_pp_indices(num_hidden_layers:int, pp_rank:int, pp_size:int) -> Tuple[int, int]
Try to evenly distribute layers across partitions.
classpython.sglang.srt.dllm.mixin.scheduler.DllmManager
Manager for Diffusion LLM request scheduling.
methodpython.sglang.srt.dllm.mixin.scheduler.DllmManager.add_waiting_reqs(reqs:Union[Req, List[Req]]) -> None
Add requests to waiting queue with redundancy check.
methodpython.sglang.srt.dllm.mixin.scheduler.DllmManager.any_staging_reqs() -> bool
Check if there are requests in staging queue.
methodpython.sglang.srt.dllm.mixin.scheduler.DllmManager.filter_finished_reqs() -> None
Remove finished requests from both queues.
methodpython.sglang.srt.dllm.mixin.scheduler.DllmManager.get_decode_requests() -> List[Req]
Get all decode requests from waiting queue.
methodpython.sglang.srt.dllm.mixin.scheduler.DllmManager.get_prefill_requests() -> List[Req]
Get all prefill requests from waiting queue.
funcpython.sglang.srt.elastic_ep.elastic_ep.try_admit_scale_ranks(global_ranks:List[int]) -> bool
Admit append-only ranks into the expandable WORLD group.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicBashTool
Anthropic ``bash_*`` server tool family.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicComputerTool
Anthropic ``computer_*`` server tool family.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicCountTokensRequest
Anthropic count_tokens API request.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicCountTokensResponse
Anthropic count_tokens API response.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicError
Error structure for Anthropic API.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicErrorResponse
Error response structure for Anthropic API.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicMessageEndDelta
Delta carried on ``message_delta`` events.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicMessagesRequest
Anthropic Messages API request.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicMessagesResponse
Anthropic Messages API response.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicOutputConfig
Claude 4.7 ``output_config`` block.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicToolChoice
Tool choice strategy.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicUsage
Token usage information.
classpython.sglang.srt.entrypoints.anthropic.protocol.AnthropicWebSearchTool
Anthropic ``web_search_*`` server tool family.
classpython.sglang.srt.entrypoints.anthropic.serving.AnthropicServing
Handler for Anthropic Messages API requests.
funcpython.sglang.srt.entrypoints.elastic_ep.scale_elastic_ep(raw_request:Request)
Request an asynchronous EP scale-up.
classpython.sglang.srt.entrypoints.engine.Engine
The entry point to the inference engine.
methodpython.sglang.srt.entrypoints.engine.Engine.close_session(session_id:str) -> None
Close a session and release its resources.

この情報について

掲載しているシグネチャは sgl-project/sglang の公開ソースコードを Python の ast モジュールで静的解析し、引数名・デフォルト値・ 型注釈・戻り値型をそのまま抽出したものです。実装コードは保存していません。 詳しくは仕組みの解説をご覧ください。

収録ライブラリ一覧（全 805 件）へ戻る