漫画工厂ai-comic-factory 文字2漫画

demo入口https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory

最终展示

大概流程：

1. 选漫画分格
2. 输入需要将啥故事X
3. X 通过Llama2 70B 生成具体的每个分割图的描述Y
4. Y 通过SDXL 生成图

LLM: llama-2 is used to generate the captions of 4 comic panels (prompt source code)

Stable Diffusion:

• I run SDXL 1.0 (no fine-tuning, no LoRA) 4 times, one for each panel (prompt source code)

• 25 inference steps

• various resolutions to change the aspect ratio (1024x768, 768x1024, also did some testing with 1024x512, 512x1024)

其中核心部分就是prompt生成

https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory/blob/main/src/app/queries/getStory.ts#L17-L32

参考：

GitHub - jbilcke-hf/ai-comic-factory: Generate comic panels using a LLM + SDXL. Powered by Hugging Face 🤗

https://www.reddit.com/r/StableDiffusion/comments/163ikmd/wip_comic_factory_a_web_app_to_generate_comic/

https://huggingface.co/spaces/jbilcke-hf/ai-comic-factory/blob/main/src/app/queries/getStory.ts#L17-L32