Introduction to Flux.1 – The new standard for image generation

Flux.1 is a new series of models from Black Forest Labs that has set the new standard in quality and prompt adherence, and it can even render legible text. The Flux.1-Schnell version of the model generates high quality images in just 4 steps, and is released under the permissive and commercially usable Apache 2 license. In this benchmark, we measure speed and cost performance of this new model on SaladCloud.

Benchmark design

We deployed the “Flux.1-Schnell (FP8) – ComfyUI (API)” recipe on Salad, using the default configuration, but setting priority to “batch”, and requesting 10 replicas. We started the benchmark when we had at least 9/10 replicas running.

We used Postman’s collection runner feature to simulate load , first from 10 concurrent users, then ramping up to 18 concurrent users. The test ran for 1 hour. Our virtual users submit requests to generate 1 image like this:

• Prompt: photograph of a futuristic house poised on a cliff overlooking the ocean. The house is made of wood and glass. The ocean churns violently. A storm approaches. A sleek red vehicle is parked behind the house.
• Resolution: 1024×1024
• Steps: 4
• Sampler: Euler
• Scheduler: Simple

We ran this on an RTX 4090 (24GB vram) with 4 vCPU and 30GB ram.

What we measured:

• Cluster Cost: Calculated using the maximum number of replicas that were running during the benchmark. Only instances in the ”running” state are billed, so actual costs may be lower.
• Reliability: % of total requests that succeeded.
• Response Time: Total round-trip time for one request to generate an image and receive a response, as measured on my laptop.
• Throughput: The number of requests succeeding per second for the entire cluster.
• Cost Per Image: A function of throughput and cluster cost.
• Images Per $: Cost per image expressed in a different way

Deployment of Flux.1-Schnell model on SaladCloud

Log in to your portal.salad.com account. Click through the FLUX.1-Schnell recipe, available from the Container Groups interface, and set replica count to 10. Optionally, set a non-default priority, and/or enable authentication. For our benchmark, we used “Batch” priority, and did not enable authentication.

Results from the Flux.1 benchmark

Our cluster of 9 replicas showed very good overall performance, returning images in as little as 2.86s / Image, and at a cost as low as 5243 images / $.

In this test, we can see that as load increases, average round-trip time increases for requests, but throughput also increases. We did not always have the maximum requested replicas running, which is expected. Salad only bills for the running instances, so this really just means we’d want to set our desired replica count to a marginally higher number than what we actually think we need.

Interested in deploying on SaladCloud? Contact our support team today.

We saw a few failed requests that coincided with node reallocations. This is expected, and you should handle this case in your application via retries.

RTX 4090 (24gb vram)

Conclusion

The Flux.1-Schnell model is a significant advancement in AI image generation, delivering high-quality results while maintaining impressive speed and cost efficiency. Our benchmark on SaladCloud demonstrated its capability to produce images fast, achieving an average response time of just 5.45 seconds and an outstanding cost efficiency of up to 5243 images per dollar. With RTX 4090 (24GB) GPUs available on SaladCloud from just $0.18/hour, AI image generation tools can save significantly on inference cost by running Flux on SaladCloud.

These results not only highlight the model’s performance under varying loads but also underscore the potential for scalability and reliability in real-world applications. As developers and creatives seek robust tools for generating visual content, Flux.1-Schnell stands out as a compelling option.

Interested in free credits to try SaladCloud for Image Generation? Contact our support team today.