Git commit
master-769-cc73429
Operating System & Version
Debian 13
GGML backends
CUDA
Command-line arguments used
./sd-cli -p "a cat" --backend "diffusion=cuda0&cuda1" --diffusion-model genImg/ZIT/z_image_turbo-Q5_0.gguf --vae VAE/flux1vae.safetensors --llm Clip/Z-Image-AbliteratedV1.Q6_K.gguf
Steps to reproduce
Build stable-diffusion.cpp with CUDA support and run this
What you expected to happen
I expected it to work.
What actually happened
the program crashed with this error
[ERROR] ggml_extend.hpp:72 - CUDA error: the function failed to launch on the GPU
[ERROR] ggml_extend.hpp:72 - current device: 0, in function ggml_cuda_op_mul_mat_cublas at stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1804
[ERROR] ggml_extend.hpp:72 - cublasSgemm_v2(ctx.cublas_handle(id), CUBLAS_OP_T, CUBLAS_OP_N, row_diff, src1_ncols, ne10, &alpha, src0_ddf_i, ne00, src1_ddf1_i, ne10, &beta, dst_dd_i, ldc)
stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
Logs / error messages / stack trace
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 19941 MiB):
Device 0: NVIDIA CMP 50HX, compute capability 7.5, VMM: yes, VRAM: 9970 MiB
Device 1: NVIDIA CMP 50HX, compute capability 7.5, VMM: yes, VRAM: 9970 MiB
[INFO ] stable-diffusion.cpp:627 - loading diffusion model from 'genImg/ZIT/z_image_turbo-Q5_0.gguf'
[INFO ] model_loader.cpp:236 - load genImg/ZIT/z_image_turbo-Q5_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:689 - loading llm from 'Clip/Z-Image-AbliteratedV1.Q6_K.gguf'
[INFO ] model_loader.cpp:236 - load Clip/Z-Image-AbliteratedV1.Q6_K.gguf using gguf format
[INFO ] stable-diffusion.cpp:703 - loading vae from 'VAE/flux1vae.safetensors'
[INFO ] model_loader.cpp:239 - load VAE/flux1vae.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:754 - Version: Z-Image
[INFO ] stable-diffusion.cpp:809 - Weight type stat: f32: 640 | q5_0: 180 | q8_0: 22 | q6_K: 253
[INFO ] stable-diffusion.cpp:810 - Conditioner weight type stat: f32: 145 | q6_K: 253
[INFO ] stable-diffusion.cpp:811 - Diffusion model weight type stat: f32: 251 | q5_0: 180 | q8_0: 22
[INFO ] stable-diffusion.cpp:812 - VAE weight type stat: f32: 244
[INFO ] stable-diffusion.cpp:482 - Diffusion model graph-cut layer split: deferring 453 tensors across 2 runtime backends until first graph
[INFO ] stable-diffusion.cpp:1245 - using VAE for encoding / decoding
[INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128
[INFO ] stable-diffusion.cpp:1469 - total params memory size = 8818.96MB (VRAM 8818.96MB, RAM 0.00MB): text_encoders 4326.88MB(VRAM), diffusion_model 4332.08MB(VRAM), vae 160.00MB(VRAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A)
[INFO ] stable-diffusion.cpp:1578 - running in FLOW mode
[INFO ] stable-diffusion.cpp:4967 - generate_image 512x512
[INFO ] denoiser.hpp:1026 - get_sigmas with discrete scheduler
[INFO ] stable-diffusion.cpp:3845 - sampling using Euler method
|## | 13/386 - 0.00MB/s�[K
[INFO ] model_loader.cpp:1247 - loading tensors completed, taking 28.09s (read: 25.61s, memcpy: 0.00s, convert: 0.06s, copy_to_backend: 0.21s)
[INFO ] stable-diffusion.cpp:4665 - get_learned_condition completed, taking 28.45s
[INFO ] stable-diffusion.cpp:5014 - generating image: 1/1 - seed 42
[INFO ] ggml_graph_cut.cpp:938 - z_image build cached graph cut plan done (taking 3 ms)
[INFO ] layer_split_partition.cpp:238 - z_image graph-cut layer split: CUDA0 <- segments [0, 28), 357 tensors, 3496.4 MB
[INFO ] layer_split_partition.cpp:238 - z_image graph-cut layer split: CUDA1 <- segments [28, 36), 95 tensors, 835.7 MB
|## | 15/452 - 12.21MB/s�[K
[INFO ] model_loader.cpp:1247 - loading tensors completed, taking 37.35s (read: 36.10s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.21s)
[INFO ] ggml_graph_cut.cpp:938 - z_image build cached graph cut plan done (taking 2 ms)
[ERROR] ggml_extend.hpp:72 - CUDA error: the function failed to launch on the GPU
[ERROR] ggml_extend.hpp:72 - current device: 0, in function ggml_cuda_op_mul_mat_cublas at stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1804
[ERROR] ggml_extend.hpp:72 - cublasSgemm_v2(ctx.cublas_handle(id), CUBLAS_OP_T, CUBLAS_OP_N, row_diff, src1_ncols, ne10, &alpha, src0_ddf_i, ne00, src1_ddf1_i, ne10, &beta, dst_dd_i, ldc)
stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
[New LWP 83268]
[New LWP 83267]
[New LWP 83154]
[New LWP 83153]
[New LWP 83146]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fc951ca69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#0 0x00007fc951ca69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fc951c9b668 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007fc951c9b6ad in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007fc951d067c7 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x000055594db69c6b in ggml_print_backtrace ()
#5 0x000055594db69db8 in ggml_abort ()
#6 0x000055594cf6a343 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) ()
#7 0x000055594cf79f80 in ggml_cuda_op_mul_mat_cublas(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st*) ()
#8 0x000055594cf7b7b2 in ggml_cuda_op_mul_mat(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, void ()(ggml_backend_cuda_context&, ggml_tensor const, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st*), void ()(float const, int const*, void*, ggml_type, long, long, long, long, long, long, long, long, CUstream_st*)) ()
#9 0x000055594cf81fbf in ggml_backend_cuda_graph_compute(ggml_backend*, ggml_cgraph*) ()
#10 0x000055594db8a961 in ggml_backend_sched_compute_splits(ggml_backend_sched*) ()
#11 0x000055594db8b63b in ggml_backend_sched_graph_compute ()
#12 0x000055594cdbbd5d in std::optional<sd::Tensor > GGMLRunner::execute_graph(ggml_cgraph*, int, bool, bool, bool, bool, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const*) ()
#13 0x000055594cc2f69f in std::optional<sd::Tensor > GGMLRunner::compute(std::function<ggml_cgraph* ()>, int, bool, bool, bool, bool) [clone .constprop.0] ()
#14 0x000055594cdd9d5f in ZImage::ZImageRunner::compute(int, DiffusionParams const&) ()
#15 0x000055594cc5d6da in StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&)::{lambda(sd::Tensor const&, float, int)#1}::operator()(sd::Tensor const&, float, int) const::{lambda(SDCondition const&, sd::Tensor const*, std::vector<int, std::allocator > const*, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const*)#1}::operator()(SDCondition const&, sd::Tensor const*, std::vector<int, std::allocator > const*, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const*) const ()
#16 0x000055594cdd5d8b in StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&)::{lambda(sd::Tensor const&, float, int)#1}::operator()(sd::Tensor const&, float, int) const ()
#17 0x000055594cdd7c06 in std::_Function_handler<sd::guidance::GuiderOutput (sd::Tensor const&, float, int), StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&)::{lambda(sd::Tensor const&, float, int)#1}>::_M_invoke(std::_Any_data const&, sd::Tensor const&, float&&, int&&) ()
#18 0x000055594cd49d5a in StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&) ()
#19 0x000055594cc344a3 in generate_image ()
#20 0x000055594ca6ffa3 in main ()
[Inferior 1 (process 83145) detached]
Additional context / environment details
It is quite possible that the problem is on my side.
Git commit
master-769-cc73429
Operating System & Version
Debian 13
GGML backends
CUDA
Command-line arguments used
./sd-cli -p "a cat" --backend "diffusion=cuda0&cuda1" --diffusion-model genImg/ZIT/z_image_turbo-Q5_0.gguf --vae VAE/flux1vae.safetensors --llm Clip/Z-Image-AbliteratedV1.Q6_K.gguf
Steps to reproduce
Build stable-diffusion.cpp with CUDA support and run this
What you expected to happen
I expected it to work.
What actually happened
the program crashed with this error
[ERROR] ggml_extend.hpp:72 - CUDA error: the function failed to launch on the GPU
[ERROR] ggml_extend.hpp:72 - current device: 0, in function ggml_cuda_op_mul_mat_cublas at stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1804
[ERROR] ggml_extend.hpp:72 - cublasSgemm_v2(ctx.cublas_handle(id), CUBLAS_OP_T, CUBLAS_OP_N, row_diff, src1_ncols, ne10, &alpha, src0_ddf_i, ne00, src1_ddf1_i, ne10, &beta, dst_dd_i, ldc)
stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
Logs / error messages / stack trace
ggml_cuda_init: found 2 CUDA devices (Total VRAM: 19941 MiB):
Device 0: NVIDIA CMP 50HX, compute capability 7.5, VMM: yes, VRAM: 9970 MiB
Device 1: NVIDIA CMP 50HX, compute capability 7.5, VMM: yes, VRAM: 9970 MiB
[INFO ] stable-diffusion.cpp:627 - loading diffusion model from 'genImg/ZIT/z_image_turbo-Q5_0.gguf'
[INFO ] model_loader.cpp:236 - load genImg/ZIT/z_image_turbo-Q5_0.gguf using gguf format
[INFO ] stable-diffusion.cpp:689 - loading llm from 'Clip/Z-Image-AbliteratedV1.Q6_K.gguf'
[INFO ] model_loader.cpp:236 - load Clip/Z-Image-AbliteratedV1.Q6_K.gguf using gguf format
[INFO ] stable-diffusion.cpp:703 - loading vae from 'VAE/flux1vae.safetensors'
[INFO ] model_loader.cpp:239 - load VAE/flux1vae.safetensors using safetensors format
[INFO ] stable-diffusion.cpp:754 - Version: Z-Image
[INFO ] stable-diffusion.cpp:809 - Weight type stat: f32: 640 | q5_0: 180 | q8_0: 22 | q6_K: 253
[INFO ] stable-diffusion.cpp:810 - Conditioner weight type stat: f32: 145 | q6_K: 253
[INFO ] stable-diffusion.cpp:811 - Diffusion model weight type stat: f32: 251 | q5_0: 180 | q8_0: 22
[INFO ] stable-diffusion.cpp:812 - VAE weight type stat: f32: 244
[INFO ] stable-diffusion.cpp:482 - Diffusion model graph-cut layer split: deferring 453 tensors across 2 runtime backends until first graph
[INFO ] stable-diffusion.cpp:1245 - using VAE for encoding / decoding
[INFO ] auto_encoder_kl.hpp:527 - vae decoder: ch = 128
[INFO ] stable-diffusion.cpp:1469 - total params memory size = 8818.96MB (VRAM 8818.96MB, RAM 0.00MB): text_encoders 4326.88MB(VRAM), diffusion_model 4332.08MB(VRAM), vae 160.00MB(VRAM), controlnet 0.00MB(N/A), extensions 0.00MB(N/A)
[INFO ] stable-diffusion.cpp:1578 - running in FLOW mode
[INFO ] stable-diffusion.cpp:4967 - generate_image 512x512
[INFO ] denoiser.hpp:1026 - get_sigmas with discrete scheduler
[INFO ] stable-diffusion.cpp:3845 - sampling using Euler method
|## | 13/386 - 0.00MB/s�[K
[INFO ] model_loader.cpp:1247 - loading tensors completed, taking 28.09s (read: 25.61s, memcpy: 0.00s, convert: 0.06s, copy_to_backend: 0.21s)
[INFO ] stable-diffusion.cpp:4665 - get_learned_condition completed, taking 28.45s
[INFO ] stable-diffusion.cpp:5014 - generating image: 1/1 - seed 42
[INFO ] ggml_graph_cut.cpp:938 - z_image build cached graph cut plan done (taking 3 ms)
[INFO ] layer_split_partition.cpp:238 - z_image graph-cut layer split: CUDA0 <- segments [0, 28), 357 tensors, 3496.4 MB
[INFO ] layer_split_partition.cpp:238 - z_image graph-cut layer split: CUDA1 <- segments [28, 36), 95 tensors, 835.7 MB
|## | 15/452 - 12.21MB/s�[K
[INFO ] model_loader.cpp:1247 - loading tensors completed, taking 37.35s (read: 36.10s, memcpy: 0.00s, convert: 0.00s, copy_to_backend: 0.21s)
[INFO ] ggml_graph_cut.cpp:938 - z_image build cached graph cut plan done (taking 2 ms)
[ERROR] ggml_extend.hpp:72 - CUDA error: the function failed to launch on the GPU
[ERROR] ggml_extend.hpp:72 - current device: 0, in function ggml_cuda_op_mul_mat_cublas at stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:1804
[ERROR] ggml_extend.hpp:72 - cublasSgemm_v2(ctx.cublas_handle(id), CUBLAS_OP_T, CUBLAS_OP_N, row_diff, src1_ncols, ne10, &alpha, src0_ddf_i, ne00, src1_ddf1_i, ne10, &beta, dst_dd_i, ldc)
stable-diffusion.cpp/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
[New LWP 83268]
[New LWP 83267]
[New LWP 83154]
[New LWP 83153]
[New LWP 83146]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fc951ca69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#0 0x00007fc951ca69ee in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007fc951c9b668 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x00007fc951c9b6ad in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#3 0x00007fc951d067c7 in wait4 () from /lib/x86_64-linux-gnu/libc.so.6
#4 0x000055594db69c6b in ggml_print_backtrace ()
#5 0x000055594db69db8 in ggml_abort ()
#6 0x000055594cf6a343 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) ()
#7 0x000055594cf79f80 in ggml_cuda_op_mul_mat_cublas(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st*) ()
#8 0x000055594cf7b7b2 in ggml_cuda_op_mul_mat(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, void ()(ggml_backend_cuda_context&, ggml_tensor const, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st*), void ()(float const, int const*, void*, ggml_type, long, long, long, long, long, long, long, long, CUstream_st*)) ()
#9 0x000055594cf81fbf in ggml_backend_cuda_graph_compute(ggml_backend*, ggml_cgraph*) ()
#10 0x000055594db8a961 in ggml_backend_sched_compute_splits(ggml_backend_sched*) ()
#11 0x000055594db8b63b in ggml_backend_sched_graph_compute ()
#12 0x000055594cdbbd5d in std::optional<sd::Tensor > GGMLRunner::execute_graph(ggml_cgraph*, int, bool, bool, bool, bool, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits, std::allocator >, std::hash<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits, std::allocator > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits, std::allocator > > > const*) ()
#13 0x000055594cc2f69f in std::optional<sd::Tensor > GGMLRunner::compute(std::function<ggml_cgraph* ()>, int, bool, bool, bool, bool) [clone .constprop.0] ()
#14 0x000055594cdd9d5f in ZImage::ZImageRunner::compute(int, DiffusionParams const&) ()
#15 0x000055594cc5d6da in StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&)::{lambda(sd::Tensor const&, float, int)#1}::operator()(sd::Tensor const&, float, int) const::{lambda(SDCondition const&, sd::Tensor const*, std::vector<int, std::allocator > const*, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const*)#1}::operator()(SDCondition const&, sd::Tensor const*, std::vector<int, std::allocator > const*, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const*) const ()
#16 0x000055594cdd5d8b in StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&)::{lambda(sd::Tensor const&, float, int)#1}::operator()(sd::Tensor const&, float, int) const ()
#17 0x000055594cdd7c06 in std::_Function_handler<sd::guidance::GuiderOutput (sd::Tensor const&, float, int), StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&)::{lambda(sd::Tensor const&, float, int)#1}>::_M_invoke(std::_Any_data const&, sd::Tensor const&, float&&, int&&) ()
#18 0x000055594cd49d5a in StableDiffusionGGML::sample(std::shared_ptr const&, bool, sd::Tensor const&, sd::Tensor, SDCondition const&, SDCondition const&, SDCondition const&, sd::Tensor const&, float, sd_guidance_params_t const&, float, int, sample_method_t, bool, char const*, std::vector<float, std::allocator > const&, std::vector<sd::Tensor, std::allocator<sd::Tensor > > const&, bool, sd::Tensor const&, sd::Tensor const&, float, int, float, sd_cache_params_t const*, sd::Tensor const&) ()
#19 0x000055594cc344a3 in generate_image ()
#20 0x000055594ca6ffa3 in main ()
[Inferior 1 (process 83145) detached]
Additional context / environment details
It is quite possible that the problem is on my side.