[Graph Optimization] SOT+CUDAGraph support ERNIE4.5T VL 28B / 424B

[DrRyanHuang]

Motivation

SOT+CUDAGraph 跑通 ERNIE4.5T VL 28B / 424B
前置PR: #4610

Modifications

• fastdeploy/model_executor/models/ernie4_5_vl/ernie4_5_vl_moe.py
• 添加 SOT+CUDAGraph 运行 ERNIE4.5T VL 28B 的单测

Usage or Command

MODEL=/workspace/EB_MODELS/ERNIE-4.5-VL-28B-A3B-Paddle
MODEL=/workspace/EB_MODELS/ERNIE-4.5-VL-424B-A47B-Paddle
rm -rf log/*

export FLAGS_cuda_graph_blacklist="custom_op.static_op_append_attention_with_output_"
export CUDA_VISIBLE_DEVICES=0,1,2,3
export PORT=39905

python -m fastdeploy.entrypoints.openai.api_server \
  --model $MODEL \
  --metrics-port 39717 \
  --port $PORT \
  --engine-worker-queue-port 39719 \
  --tensor-parallel-size 4 \
  --max-model-len 32768 \
  --max-num-seqs 128 \
  --quantization wint4 \
  --enable-mm \
  --graph-optimization-config '{"graph_opt_level": 1, "use_cudagraph": true, "full_cuda_graph": false}' 

Accuracy Tests

NULL

Checklist

• Add at least a tag in the PR title.
  • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
  • You can add new tags based on the PR content, but the semantics must be clear.
• Format your code, run pre-commit before commit.
• Add unit tests. Please write the reason in this PR if no unit tests.
• Provide accuracy results.
• If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

[paddle-bot]

Thanks for your contribution!

[EmmonsCurse]

You may need to update your branch to resolve errors.

[gongshaotian]

LGTM

[gongshaotian]

这里和xpu那个修改会冲突吗

[DrRyanHuang]

XPU 的 #4636 合入后，这个再合入就没问题

[gongshaotian]

LGTM