Dataset Preview
The full dataset viewer is not available (click to read why). Only showing a preview of the rows.
The dataset generation failed because of a cast error
Error code: DatasetGenerationCastError
Exception: DatasetGenerationCastError
Message: An error occurred while generating the dataset
All the data files must have the same columns, but at some point there are 9 new columns ({'memory (gbs)', 'pipeline_cls', 'num_inference_steps', 'run_compile', 'model_cpu_offload', 'actual_gpu_memory (gbs)', 'ckpt_id', 'batch_size', 'time (secs)'}) and 10 missing columns ({'time_plain_s', 'flops_G', 'scenario', 'num_params_B', 'mem_compile_GB', 'time_compile_s', 'mem_plain_GB', 'mode', 'model_cls', 'fullgraph'}).
This happened while the csv dataset builder was generating data using
hf://datasets/diffusers/benchmarks/collated_results_old.csv (at revision 4eefb41be3cea4a11d53fe704e699e48b2664c43)
Please either edit the data files to have matching columns, or separate them into different configurations (see docs at https://hf.co/docs/hub/datasets-manual-configuration#multiple-configurations)
Traceback: Traceback (most recent call last):
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/builder.py", line 1871, in _prepare_split_single
writer.write_table(table)
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/arrow_writer.py", line 643, in write_table
pa_table = table_cast(pa_table, self._schema)
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/table.py", line 2293, in table_cast
return cast_table_to_schema(table, schema)
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/table.py", line 2241, in cast_table_to_schema
raise CastError(
datasets.table.CastError: Couldn't cast
pipeline_cls: string
ckpt_id: string
batch_size: int64
num_inference_steps: int64
model_cpu_offload: bool
run_compile: bool
time (secs): string
memory (gbs): string
actual_gpu_memory (gbs): double
github_sha: string
-- schema metadata --
pandas: '{"index_columns": [{"kind": "range", "name": null, "start": 0, "' + 1509
to
{'scenario': Value(dtype='string', id=None), 'model_cls': Value(dtype='string', id=None), 'num_params_B': Value(dtype='float64', id=None), 'flops_G': Value(dtype='float64', id=None), 'time_plain_s': Value(dtype='string', id=None), 'mem_plain_GB': Value(dtype='float64', id=None), 'time_compile_s': Value(dtype='string', id=None), 'mem_compile_GB': Value(dtype='float64', id=None), 'fullgraph': Value(dtype='bool', id=None), 'mode': Value(dtype='string', id=None), 'github_sha': Value(dtype='string', id=None)}
because column names don't match
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/src/services/worker/src/worker/job_runners/config/parquet_and_info.py", line 1436, in compute_config_parquet_and_info_response
parquet_operations = convert_to_parquet(builder)
File "/src/services/worker/src/worker/job_runners/config/parquet_and_info.py", line 1053, in convert_to_parquet
builder.download_and_prepare(
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/builder.py", line 925, in download_and_prepare
self._download_and_prepare(
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/builder.py", line 1001, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/builder.py", line 1742, in _prepare_split
for job_id, done, content in self._prepare_split_single(
File "/src/services/worker/.venv/lib/python3.9/site-packages/datasets/builder.py", line 1873, in _prepare_split_single
raise DatasetGenerationCastError.from_cast_error(
datasets.exceptions.DatasetGenerationCastError: An error occurred while generating the dataset
All the data files must have the same columns, but at some point there are 9 new columns ({'memory (gbs)', 'pipeline_cls', 'num_inference_steps', 'run_compile', 'model_cpu_offload', 'actual_gpu_memory (gbs)', 'ckpt_id', 'batch_size', 'time (secs)'}) and 10 missing columns ({'time_plain_s', 'flops_G', 'scenario', 'num_params_B', 'mem_compile_GB', 'time_compile_s', 'mem_plain_GB', 'mode', 'model_cls', 'fullgraph'}).
This happened while the csv dataset builder was generating data using
hf://datasets/diffusers/benchmarks/collated_results_old.csv (at revision 4eefb41be3cea4a11d53fe704e699e48b2664c43)
Please either edit the data files to have matching columns, or separate them into different configurations (see docs at https://hf.co/docs/hub/datasets-manual-configuration#multiple-configurations)Need help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
scenario
string | model_cls
string | num_params_B
float64 | flops_G
float64 | time_plain_s
string | mem_plain_GB
float64 | time_compile_s
string | mem_compile_GB
float64 | fullgraph
bool | mode
string | github_sha
string |
|---|---|---|---|---|---|---|---|---|---|---|
black-forest-labs/FLUX.1-dev-bf16 | FluxTransformer2DModel | 11.9 | 59,529.52 | 0.532 (0.534) | 22.61 | 0.377 (0.383) | 22.7 | true | default | a09768f2bd006dd36ce5e705e65cc87565b42702 |
black-forest-labs/FLUX.1-dev-bnb-nf4 | FluxTransformer2DModel | 5.95 | null | 0.563 | 6.7 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
black-forest-labs/FLUX.1-dev-layerwise-upcasting | FluxTransformer2DModel | 11.9 | 59,529.52 | 0.598 (0.599) | 22.18 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
black-forest-labs/FLUX.1-dev-group-offload-leaf | FluxTransformer2DModel | 11.9 | 59,529.52 | 1.897 | 0.53 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
Lightricks/LTX-Video-0.9.7-dev-bf16 | LTXVideoTransformer3DModel | 13.04 | 167,583.45 | 1.601 (1.598) | 25.21 | 1.099 (1.079) | 25.31 | true | default | a09768f2bd006dd36ce5e705e65cc87565b42702 |
Lightricks/LTX-Video-0.9.7-dev-layerwise-upcasting | LTXVideoTransformer3DModel | 13.04 | 167,583.45 | 1.661 (1.656) | 24.38 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
Lightricks/LTX-Video-0.9.7-dev-group-offload-leaf | LTXVideoTransformer3DModel | 13.04 | 167,583.45 | 2.761 (2.762) | 1.04 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
stabilityai/stable-diffusion-xl-base-1.0-bf16 | UNet2DConditionModel | 2.57 | 5,979.1 | 0.076 | 5.05 | 0.055 (0.054) | 5.24 | true | default | a09768f2bd006dd36ce5e705e65cc87565b42702 |
stabilityai/stable-diffusion-xl-base-1.0-layerwise-upcasting | UNet2DConditionModel | 2.57 | 5,979.1 | 0.162 (0.154) | 4.89 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
stabilityai/stable-diffusion-xl-base-1.0-group-offload-leaf | UNet2DConditionModel | 2.57 | 5,979.1 | 0.518 (0.524) | 0.2 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
Wan-AI/Wan2.1-T2V-14B-Diffusers-bf16 | WanTransformer3DModel | 14.29 | 785,611.67 | 10.966 (10.922) | 31.17 | 8.59 (8.456) | 31.77 | true | default | a09768f2bd006dd36ce5e705e65cc87565b42702 |
Wan-AI/Wan2.1-T2V-14B-Diffusers-layerwise-upcasting | WanTransformer3DModel | 14.29 | 785,611.67 | 10.913 (10.766) | 26.78 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
Wan-AI/Wan2.1-T2V-14B-Diffusers-group-offload-leaf | WanTransformer3DModel | 14.29 | 785,611.67 | 11.254 (11.262) | 4.48 | null | null | null | null | a09768f2bd006dd36ce5e705e65cc87565b42702 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
null | null | null | null | null | null | null | null | null | null | f4fa3beee7f49b80ce7a58f9c8002f43299175c9 |
Welcome to 🤗 Diffusers Benchmarks!
This is dataset where we keep track of the inference latency and memory information of the core pipelines in the diffusers library.
Currently, the core pipelines are the following:
- Stable Diffusion and its derivatives such as ControlNet, T2I Adapter, Image-to-Image, Inpainting
- Stable Diffusion XL and its derivatives
- SSD-1B
- Kandinsky
- Würstchen
- LCM
Note that we will continue to extend the list of core pipelines based on their API usage.
We use this GitHub Actions workflow to report the above numbers automatically. This workflow runs on a biweekly cadence.
The benchmarks are run on an A10G GPU.
- Downloads last month
- 175