Blend cuts AI inference cost by 85% on Salad while running 3X more scale
Startups often have a unique origin story. Blend is no different. Their origins lie in a Whatsapp channel and a local fireworks seller who printed 2000 physical posters for a peculiar reason. Blend is an AI copilot for e-commerce that helps sellers create professional product photos and designs in two clicks without hiring an agency. Their mission is to help entrepreneurs & small sellers grow sales online with compelling social graphics, product photos and SEO optimized copy. Today, Blend serves thousands of sellers generating around 6000 images every hour on Salad’s distributed network. In this chat with Jamsheed Kamardeen, Chief Technology Officer (CTO) at Blend, we discuss their growth, the switch to a distributed cloud, inference cost & more. How did the idea for Blend come about? It was during Covid-19. We were in many Whatsapp groups looking for common problems faced by e-commerce sellers and we found a peculiar thing. There were many coaching sessions on how to use photo and design apps. Turns out, many of the sellers didn’t have a design team on their own but they needed to promote their products on social media, create posters, ads and the like. In fact, one of my cousins had a fireworks shop in a small village in India and 70% of his sales came from posters on Whatsapp & Instagram. He’d go to a local printing shop and use the designer there to create & print out 2000 posters just so he could get a soft copy of the design for ads/promotions. He didn’t even bother distributing the physical posters. So looking at the challenges of local sellers in promoting their products in a digital world – that’s where the idea came from. Photo editing and design apps have been out there for a long time, right? Weren’t they sufficient? Yes. There were a lot of apps & horizontal design tools which were good for designers. But the sellers aren’t designers. Plus there is the paradox of choice. Most tools had 100s of templates, colors, etc and needed a significant time commitment. Plus, often the sellers ended up with a design that looked terrible because they tweaked too much or too little. So we decided to create Blend to offload the design decision making. Just upload a picture of the product and tell us what you want the offer to be. We’ll remove the background, put the product in appropriate settings and deliver a design with text. Our goal was always to get them the final design in the fewest clicks possible. Today, you have Millions of downloads for your app. How crucial was the arrival of Generative AI in your user growth? Our initial version included background removal and adding in an appropriate background with some other features. But generative AI completely changed the game for us. For example, if a shoe store wants to do a 25% off Diwali promotion, all they had to do was upload the product photo and describe the offer and event. With Generative AI & Stable Diffusion models, we can identify it’s a shoe, have LLMs make the decision on what to paint & such, create an aesthetically pleasing urban background, automatically create appropriate text with the right color scheme and deliver the copy. All it takes is a couple of clicks. This is what led to our massive user growth. Today, 40% of our users are individual sellers, so we are introducing a separate web app for them as well. With big growth comes big cloud bills. That must have been the case for Blend as well. What infrastructure challenges did you face here? Right. Since we are an AI first design company, inference became our biggest cost factor. Plus, we needed powerful GPUs to power Diffusion models. Sourcing GPUs to keep up with surge in demand quickly became a nightmare. The existing providers didn’t have the right options for a company like us. AWS only had multi cluster A100s but there was no single cluster A100 option. GCP or Azure had them but they were expensive. So we started looking for alternatives. We found a local provider who offered A100s for a cheaper price. But that came with reliability & scalability issues. We didn’t always have enough GPUs during times of higher traffic. I started losing a lot of sleep over this GPU shortage. We’re a small team, so when the machines go down, my sleep goes away. So again, we were looking for an alternative. That’s when we found Salad. How has switching to SaladCloud impacted your cost and scaling? When we switched from the hyperscalers to A100s with a local provider, we didn’t really think the cost could go any lower. But switching to Salad was eye-opening. On Salad’s consumer GPUs, we are running 3X more scale at half the cost of A100s on our local provider and almost 85% less cost than the two major hyperscalers we were using before. Plus Salad is much more reliable. We’ve migrated all current and new workloads to Salad. I’m not losing sleep over scaling issues anymore. On Salad’s consumer GPUs, we are running 3X more scale at half the cost of A100s on our local provider and almost 85% less cost than the two major hyperscalers we were using before. I’m not losing sleep over scaling issues anymore. Jamsheed Kamardeen, Chief Technology Officer (CTO) at Blend As a CTO, making the switch to a distributed cloud is a huge decision. What was the decision making process? That’s a good question. I was very skeptical initially about the reliability of Salad. From a technical standpoint, my major question was this: Compared to data centers with reliable internet, how am I going to have reliable workloads on random people’s computers on a distributed cloud? We needed to implement some solutions to make reliability strong but it wasn’t as difficult as I initially perceived it to be. One thing that helped us was the engineering support offered by Salad which made
Introducing SSAP: Migrate to Salad GPU Cloud easily & save up to 80%
AI companies are overpaying for compute today Affordable, accessible compute is the defining challenge for many AI startups today. Recently, we’ve seen many news stories of innovative AI companies struggling with profitability or running out of cash. The reason? Exorbitant cloud bills and a race to secure high-end, AI-focused GPUs (There were even a couple startups that spent almost 50% of their fundraising on GPUs). Thankfully, the incumbent cloud/chip monopolies are being challenged by new GPU clouds bringing the power of crowdsourced, consumer GPUs for AI/ML workloads. There are three reasons this is happening: Wrong GPU choice: The mighty marketing machines at chipmakers have convinced the market that everything needs to be run on high-end, AI-focused GPUs that are hard to secure. A market made by the monopolies: The cloud monopolies have secured most of these high-end GPUs, creating a scarcity leading to expensive prices. As Chris Z. from Wing Venture Capital explains here, the rest of the market is just acting as a pipeline of capital to the chipmakers. Big margins for APIs: Managed Service Providers are much easier to first integrate with and come with low ongoing maintenance, but these come at a very high price. You are paying their margins! For example, a transcription service is priced from $0.30/hr (API provider) to $1.40/hr (A popular big cloud). Due to these reasons, many AI companies are massively overpaying for compute, especially serving inference, even though cost-effective options are available today. The urgent need for cloud migration in today’s AI landscape With profitability on top of their mind, the last year has seen many AI startups and enterprises alike take a multi-cloud approach and move production workloads to alternate clouds and consumer GPUs with lower prices and similar performance. But infrastructure migration can be a huge challenge, especially for startups with minimal resources. There’s also the not-so-minor issue of benchmarking on a new GPU provider. To tackle these challenges and help resource-strapped companies migrate seamlessly to Salad, we are introducing a new initiative – Salad Solutions Architect Program (SSAP). We know the name is a mouthful (No thanks to our marketing team here). But the service has already helped 20+ AI companies migrate from another cloud provider or API service to Salad, saving thousands of dollars per month in cloud costs. “Over time, inference will increasingly be price-performance oriented and older hardware will run some AI workloads — though inference demand will rise exponentially” – Chris Zeoli, Partner at Wing Venture Capital (https://www.linkedin.com/pulse/great-gpu-shortage-richpoor-chris-zeoli-5cs5c/) What is Salad GPU Cloud? SaladCloud is a distributed GPU cloud powered by a secure network of 1000s of individual consumer GPUs. Due to our marketplace model, our GPU prices are the lowest in the market. Salad’s fleet of RTX GPUs are powering inference at scale for many AI companies today, including some of the Top 50 most visited AI websites in the world. Our benchmarks show similar or better cost-performance with consumer GPUs for many popular AI use cases like Text to Image, Speech to Text, Text to Speech, Computer Vision and more. All of this comes at least 50% less cost compared to serving inference with high-end GPUs on big clouds. Here is feedback from a Generative AI startup founder wondering why they are paying 5x more for a V100 to get 1/2 the performance from a RTX 4090. What is the Salad Solutions Architect Program (SSAP)? “As much as we want to migrate to Salad (the pricing makes perfect business sense), we are very busy with a new product launch and frankly don’t have the resources or time to migrate. If your team can help on that end, we’d be on board right away” – Founder of a Top 50 Generative AI image generation platform The Salad Solutions Architect Program (SSAP) was born from repeated feedback similar to the one quoted above. We heard from AI startups that they were keen to migrate away from the big clouds to cut cloud costs but were hampered by two main challenges: With this feedback in mind, we designed SSAP to help companies migrate production workloads to SaladCloud easily. SSAP will essentially act as an extended team to companies helping them with benchmarking and migration. As part of the program, our team can assist with building, migrating, testing and benchmarking your workload on our highly scalable, cost performant cloud. Qualified teams will gain access to a dedicated Solutions Engineer from Salad who will assist in coding, adjusting backend architecture, configuring SaladCloud container groups, and benchmarking results across our diverse range of consumer GPUs. Once onboarded, our managed container service allows you to run stateless docker containers seamlessly across our network. How do I qualify for SSAP? To qualify, your compute requirements need to be a minimum of 10 GPUs (24GB VRAM class) running concurrently and consistently every month. Why should a company join the Salad Solutions Architect Program? For AI companies struggling with enormous cloud costs but strapped for resources, SSAP offers a way to seamlessly move production workloads to Salad’s distributed infrastructure. Some of the program benefits include: Free credits worth $5k-10k Qualifying companies get up to $10,000 in credits for a 2 month duration to test their use case on SaladCloud. SSAP allows new customers to test and integrate with SaladCloud risk free, as well as realising the cost benefits immediately. Coding done for you Developers and companies access SaladCloud via Salad Container Engine (SCE), a massively scalable orchestration engine, purpose-built to simplify container development. As long as you can containerize your model, switching to SaladCloud is a simple process. Our Solutions Architect will handle all the coding to get your containers up and running on Salad. GPU benchmarking While we have published numerous benchmarks comparing the performance of popular AI models on various consumer GPUs, we understand the need to benchmark GPU performance on your own models. More importantly, the right GPU choice for a use case could save thousands of dollars. As part of SSAP, our Solutions Architect will benchmark your
Announcing Salad’s SOC 2 Type 1 Certification
We are pleased to announce that Salad Technologies is now SOC 2 Type I Compliant, reinforcing our commitment to democratize the cloud while ensuring the highest standards of security & compliance. This certification is in accordance with American Institute of Certified Public Accountants (AICPA) standards for SOC for Service Organizations also known as SSAE 18. What is the SOC 2 Type 1 Certification? The SOC 2 Type I certification is the first level of certification within the SOC 2 framework. It gives a snapshot assessment of an organization’s security controls as of a specific point in time. SOC 2 Type I reports on the description of security & compliance controls provided by the organization and attests that the controls are suitably designed and implemented. SOC 2 Type I certification is based on five trust service principles or criteria: security, availability, processing integrity, confidentiality, and privacy. These principles define the standards for managing customer data and ensuring its protection. Why does this matter to Salad – and to our users? As the world’s largest distributed cloud, we have securely distributed business applications/workloads to anonymous hosts since 2018. With the introduction of SaladCloud to businesses in 2023, the number of B2B & B2C companies running production workloads across out network is growing tremendously. As of last count, our network welcomes more than 1 Million consumer GPUs and 100s of B2B/B2C businesses running workloads on our cloud at affordable prices. Large scale AI image generation, voice AI, large language models (LLMs), data collection, residential IP addresses, computer vision – the applications running on SaladCloud are growing and so is our network of community-sourced GPUs. To protect the data and workloads of our contributors, customers and users, our team has implemented and assiduously maintains redundant security layers across our network, organization, and distributed machine environments. This certification is confirmation that our information security practices, policies, procedures, and operations meet the SOC 2 standards for security. Security at the heart of our mission to democratize cloud computing Secure Personnel: We take the security of its data and that of its clients and customers seriously and ensures that only vetted personnel are given access to their resources. Secure Development: Our software development is conducted in line with OWASP Top 10 recommendations for web application security. Secure Testing: We deploy third party penetration testing and vulnerability scanning of all production and Internet facing systems on a regular basis. In addition, we also perform static and dynamic software application security testing of all code, including open source libraries. Secure Cloud: We provide maximum security on SaladCloud with complete customer isolation in a modern, multi-tenant cloud architecture. We leverage the native physical and network security features of the cloud service, and rely on the providers to maintain the infrastructure, services, and physical access policies and procedures. All customer cloud environments and data are isolated using our patented isolation approach. Each customer environment is stored within a dedicated trust zone to prevent any accidental or malicious co-mingling. All data is also encrypted at rest and in transmission to prevent any unauthorized access andprevent data breaches. Our entire platform is also continuously monitored by dedicated, highly trained experts. We separate each customer’s data and our own, utilizing unique encryption keys to ensure data is protected and isolated. Our data protection complies with SOC 2 standards to encrypt data in transit and at rest, ensuring customer and company data and sensitive information is protected at all times. We implement role-based access controls and the principles of least privileged access, and review revoke access as needed. Salad’s SOC 2 Type I Certification Salad Technologies, INC was audited by Prescient Assurance , a leader in security and compliance certifications for B2B, SAAS companies worldwide. Prescient Assurance is a registered public accounting in the US and Canada and provide risk management and assurance services which includes but not limited to SOC 2, PCI, ISO, NIST, GDPR, CCPA, HIPAA, CSA STAR etc. For more information about Prescient Assurance, you may reach out them at [email protected]. To learn about trust, security & compliance on Salad, visit out Trust center. Prashanth ShankaraPrashanth Shankara is an Aerospace Engineer turned Marketer with a decade of experience in Product, Content, Customer & Developer Marketing. A firm believer in the saying “It’s not rocket science. It’s just marketing”, Prashanth loves combining systematic, iterative, data-driven approaches with creative tactics in marketing. As Senior Manager, Marketing at Salad, Prashanth leads the Marketing function with a singular goal – To break the cloud monopoly and help companies fuel AI/ML innovation on the cloud at more affordable prices.
Salad Launches Kubernetes Support at KubeCon 2023
Salad is excited to announce support for Kubernetes (K8s) through the Virtual Kubelet project. The K8s integration based on the SaladCloud Virtual Kubelet Provider will be showcased at the upcoming KubeCon conference in Chicago, taking place from November 6-9, 2023. Attendees will have the opportunity to learn more about the product and its benefits firsthand by visiting Salad’s booth [E27] at KubeCon. The SaladCloud Virtual Kubelet Provider enables running K8s pods as container group deployments, unlocking a new level of flexibility and scalability for compute-hungry applications. “There is a global GPU shortage and rampant overpaying, especially for high-end, AI-focused GPUs”, said Bob Miles, CEO of Salad. “Meanwhile, millions of consumer GPUs lie unused most of the day. Salad connects the two. With this K8s integration, companies with GPU-powered applications can seamlessly access Salad’s vast network of 10,000+ consumer GPUs at the lowest market price for production deployment”. Virtual Kubelet, a Cloud Native Computing Foundation (CNCF) supported project, enables organizations to extend their Kubernetes clusters by leveraging external compute resources provided by other vendors using their existing K8s control plane and the landscape of cloud native tooling. With the new SaladCloud Virtual Kubelet Provider, customers are able to deploy containerized workloads to SaladCloud, very similar to the managed computing environments offered by major cloud providers. The containerized workloads are orchestrated across a vast network of third-party, high-end gaming PCs with advanced GPU capabilities, whose suppliers receive credits to redeem for games, gift cards & other rewards. “If you don’t have enough GPUs in your data center or you’re overpaying for cloud resources, you can use SaladCloud directly from your K8s cluster with our Virtual Kubelet Provider to access affordable, on-demand, pay-per-use GPUs at massive scale”, added Kyle Dodson, Head of Engineering at Salad. “Engineering teams already building K8s apps can easily deploy to SaladCloud, without changing existing development workflows.” To learn more about the SaladCloud Virtual Kubelet Provider and to request a demo, visit www.salad.com/virtualkubelet. Prashanth ShankaraPrashanth Shankara is an Aerospace Engineer turned Marketer with a decade of experience in Product, Content, Customer & Developer Marketing. A firm believer in the saying “It’s not rocket science. It’s just marketing”, Prashanth loves combining systematic, iterative, data-driven approaches with creative tactics in marketing. As Senior Manager, Marketing at Salad, Prashanth leads the Marketing function with a singular goal – To break the cloud monopoly and help companies fuel AI/ML innovation on the cloud at more affordable prices.
How Dreamup reduces AI image generation cloud cost with Salad
Using AI to reduce cloud costs and fund artists If you’ve spent any time on the internet in the last few months, two things will be everywhere. As with any new technology, there will always be affected parties. In Stable Diffusion’s case, it directly impacts artists by generating (incredible) art faster than the human hand can ever create. Does this mean artists will go away? Definitely not. The AI models are still trained on human-generated art (ex, image, video, voice, music, etc). There will always be a market for original art. But make no mistake – Stable Diffusion generated art will replace a big chunk of human-generated art. Now Shawn Rushefsky, Dreamup.ai’s founder, has set out to bridge this gap and give back to the community that’s both helped train Stable Diffusion (SD) models and is being affected by it: Artists. In founder Shawn’s words, “We are funneling money from this new controversial industry into the population that’s being affected by it”. In this interview, Shawn talks about Dreamup.ai, their use of the SaladCloud to reduce cloud cost, his motivation behind starting the company and more. Salad: What is DreamUp and what was your motivation to start DreamUp? Shawn: “DreamUp is a Stable Diffusion powered image-generation application that’s quickly racked up 2k+ users in 15+ countries. It originally started as just a way to make art. But soon we realized that there are so many similar platforms and we were just being one of them” Salad: That’s true. There’s a new stable diffusion platform coming out every week. So what makes DreamUp different from others? Shawn: “We recently became a non-profit organization. 30% of our revenue goes to a different public art fund every month. In Feb, it was Tone Memphis, a Black Artists Collective. In March, we are donating to the National Association of Women in Arts. Our difference is our mission – to give money back to the artistic community through Stable Diffusion. We also want to make art more accessible – and affordable – to people with conditions like Aphantasia and arthritis” Salad: We see that DreamUp offers unlimited image generation today. Can you elaborate on that decision? Shawn: Sure. It’s mostly just to make this technology easily accessible. Today, most of the SD tools use a credit model. To me, micro-transactions aren’t the right way to make art accessible to people. Every single time you have to top-up credits, it disincentivizes you from learning. That’s why we offer unlimited generation, now and forever. Salad: How is Salad helping in bringing unlimited AI-generated art to the masses? Shawn: “Your team reached out to me on Discord asking if I wanted to lower my cloud bills. Every generative AI developer today wants to, as these services can be very expensive to operate. Salad’s GPU pricing is much better than everywhere else. The next best option was not even close. That’s the real winner for me. Salad affords us to keep DreamUp free and unlimited”. Salad: That’s always music to our ears. What models are you running on Salad? Shawn: Every time there’s a new model, we work to bring it into DreamUp. Stable Diffusion is standard but there are so many new ones – like Nitro Diffusion, Foto Assisted Diffusion, wooletize where every image is made of yarn, etc. Salad makes it more realistic to keep up with deploying these new models. We might never deploy most of them if we had to pay AWS cost for them” Salad: Anything else you like or we can improve upon? Shawn: Of course. The team is extremely responsive on your Discord. The Beta version is missing some production-readiness features like observability today. But for how much we save on cost on Salad, we’re happy to run on Salad in its current state knowing new features are coming in the next few weeks. Salad: You mentioned Recipes of popular models. As an open-source developer yourself, what’s your take on that? Shawn: Every developer I know wishes they could spend more time on Open Source Development (OSD). The entire modern world is built on it. But open source devs not getting paid is definitely a problem. It would be good to have more decentralized methods of funding OSD, versus relying on large corporate sponsors as in projects like React. I think your model of paying open source devs whenever their code gets deployed on Salad Cloud is pretty cool. It would definitely make open source development a viable way of living. About DreamUp.ai DreamUp.ai is a project by the Foundation for Technology in the Arts, and we believe that AI-generated art is art, pure and simple. And we believe all artists should have access to the tools they need to create it. Our mission is to make Generative AI accessible to all, so everyone can unleash their creativity and bring more art into the world. We’re a nonprofit that operates on a pay-what-you-want basis, and we donate 30% of our proceeds to charities that support the arts. Prashanth ShankaraPrashanth Shankara is an Aerospace Engineer turned Marketer with a decade of experience in Product, Content, Customer & Developer Marketing. A firm believer in the saying “It’s not rocket science. It’s just marketing”, Prashanth loves combining systematic, iterative, data-driven approaches with creative tactics in marketing. As Senior Manager, Marketing at Salad, Prashanth leads the Marketing function with a singular goal – To break the cloud monopoly and help companies fuel AI/ML innovation on the cloud at more affordable prices.