On Wednesday, a San Francisco-based developer named Liu Liu released Draw Things: AI Generation, a free app available in the App Store that lets iPhone owners run the popular static diffusion AI image generator. Type in a description and the app will generate an image in minutes. This is a remarkable step towards bringing image synthesis to a wider audience, with the added privacy of running it on your own hardware.
Introduced in August, Stable Diffusion (SD) is an AI image generator model that creates new images from text descriptions (called “prompts”). Typically, people run SD locally through a commercial DreamStudio service, on a remote cloud machine with rented compute time, or using a custom open source implementation. When running locally, SD requires a fairly beefy GPU to generate images quickly, but some developers have optimized the model to run on older GPUs with less VRAM (if you don’t mind waiting longer to see results).
In the same vein, Liu Liu has managed to optimize Stable Diffusion to run on the iPhone, a somewhat difficult process described by the developer in a blog post. “The main challenge is running apps on 6GiB RAM iPhone devices,” writes Liu Liu. “6GiB sounds like a lot, but iOS will start killing your app if you use more than 2.8GiB on a 6GiB device and more than 2GiB on a 4GiB device.”
After first running Draw Things, the app downloads several necessary files—including a static diffusion 1.4 model—to your iPhone. To use it, type a prompt at the top of the screen, then tap “Generate.” When generating an image, tap the number in the top center of the screen to randomize the seed, which is a number that partially guides the creation of the image.
On our iPhone 11 Pro, it took a little over two minutes to create a 384×384 image. It’s faster on the iPhone 14 Pro, producing images in about a minute, according to Liu Liu. Either way, SD is computationally intensive. Generation after generation, our iPhone became noticeably warmer to the touch.
It’s worth noting that with static diffusion, 384×384 images often produce relatively poor, low-detail results because the makers of SD trained the model using 512×512 images. When trying to generate a 512×512 image on our iPhone 11 Pro, we got a warning and proceeded anyway, but the app crashed to a black screen.
In addition to regular image creation duties, Draw Things also supports inpainting, which lets you replace a portion of an image with AI-generated imagery, and load additional image synthesis models, such as the unofficial “Modern Disney Diffusion” model (which is Disney-generated). visible characters) and the anime-driven “waifu diffusion” model (we’ve tested and it’s possible to create NSFW content using the app, so beware). The inclusion of those capabilities means that Draw Things may not stay in the App Store for long if it becomes popular, as the content it generates may violate Apple’s terms of service.
Whatever its fate, Draw Things seems like an important proof of concept, showing that stable diffusion can run natively on the iPhone, albeit slowly. If the app sticks around—perhaps with some filters or improvements—Liu Liu describes room for potential future optimizations that could speed things up: “I’ve probably left 50% of the performance on the table.”
This isn’t the first time Stable Diffusion has run on an iPhone. In September, developers Matt Waller ran Steady local distribution on its iPhone XS, but it hasn’t provided an App Store app that would allow others to replicate the feat. We’re likely to see more local AI image synthesis on smartphones as devices continue to grow in computing power.
Go to discussion…