Comfyui animation workflow example. 1 [pro] for top-tier performance, FLUX.
Comfyui animation workflow example In these ComfyUI workflows you will be able to create animations from just text prompts but also from a video input where you can After completing Part 1, you will have created three types of image sequences: mask images, depth images, and outline images. vivid color, 3D animation. A node can take in up to 5 videos, and a combination of nodes can handle any number of videos with VRAM being the main limitation. 5 Controlnets. A ComfyUI workflow can generate any type of media: image, video, audio, AI model, AI agent, With various models and extra animation related nodes installed. ComfyUI AnimateDiff, ControlNet and Auto Mask Workflow. All Workflows / Simple Run and Go With Pony. (I got Chun-Li image from civitai) Support different sampler & scheduler: DDIM. As of writing of this it is in its beta phase, but I am sure some are eager to test it out. In the context of the video, the batch size is set to 96, meaning This example showcases the Noisy Laten Composition workflow. The workflow iterates through the frames one-by-one with batch size 1 and therefore uses low VRAM. Dynamic pattern. 35 In today’s comprehensive tutorial, we embark on an intriguing journey, crafting an animation workflow from scratch using the robust Comfy UI. The images above were all created with this method. This repo contains examples of what is achievable with ComfyUI. If the nodes are already installed but still appear red, you may have to update them: you can do this by Uninstalling and Reinstalling them. . x_motion/y_motion: Pixels to move per frame. Understanding Nodes : The tutorial breaks down the function of various nodes, including input nodes (green), model loader nodes, resolution nodes, skip frames and batch range nodes, positive and negative prompt Load the workflow by dragging and dropping it into ComfyUI, in this example we're using Video2Video. 28:27 After Effects. Our mission is to navigate the intricacies of this remarkable tool, employing key nodes, such as Animate Diff, Control Net, and Video Helpers, to create seamlessly flicker-free animations. 9K. Prompt scheduling: This workflow attached is a workflow for ComfyUI to convert an image into a video. 1 to 4 bits, achieving 3. You can also use similar workflows for outpainting. This workflow demonstrates the strong AI capabilities of Nvidia Cosmos with its text-to-video and image-to-video generation features. I created these for my own use (producing videos for my "Alt Key Project" music - youtube channel), but I think they should be generic enough and useful to many ComfyUI users. Here’s an example with the anythingV3 model: Outpainting. New AI Animation Platform v0. clip_l_hidream. Author: PCMonsterx. Free AI art generator. 32:40 Hedra. The article is divided into the following key Installing ComfyUI and Animation Nodes. 1. For example, I used the following keywords for style. Turn your imagination into fluid videos using the newly released Nvidia Cosmos models in ComfyUI. 1-Img2Img 2-2 Pass Txt2Img 3-Inpaint 4-Area Composition 5-Upscale Models 6-LoRA 7-ControlNet 8-Noisy Latent Composition 9-Textual Inversion Embeddings 10-Edit Models 11-Model Merging 12-SDXL 13-Stable Cascade 14-UnCLIP 15-Hypernetworks 16-Gligen 17-3D Examples 18-Video 19-LCM Examples 20-ComfyUI SDXL ComfyUI 工作流基础概念以及如何使用 什么是工作流（Workflow）？ 工作流是 ComfyUI 中最核心的概念，简单来说就是由多个节点（Node）连接组成的图形化界面，用来描述 AI 绘图的整个处理过程。 Currently ComfyUI supports specifically the 7B and 14B text to video and image to video diffusion models. The AnimateDiff node integrates model and context options to adjust animation dynamics. You will first need: Text encoder and VAE: See this workflow for an example. safetensors TLDR In this tutorial, Mali introduces ComfyUI's Stable Video Diffusion, a tool for creating animated images and videos with AI. SD3. This basic workflow runs the base SDXL model with some optimization for SDXL. Image From Batch - 从批次获取图像 Download Flux Dev FP8 Checkpoint ComfyUI workflow example If you have example workflow files associated with your custom nodes then ComfyUI can show these to the user in the template browser (Workflow/Browse Templates menu). Below is an example of what can be achieved with this ComfyUI RAVE workflow. This guide is about how to setup ComfyUI on your Windows computer to run Flux. Area Composition; Inpainting with both regular and inpainting models. Positive values move right/down, negative left/up. Conversely, the IP-Adapter node facilitates the use of images as prompts in ways You can then load or drag the following image in ComfyUI to get the workflow: Flux Schnell. It is licensed under the Apache 2. 5 models. Follow these steps to complete the workflow: Make sure the Load Diffusion Model node has loaded the hidream_e1_full_bf16. it will change the image into an animated video using Animate-Diff and ip adapter in ComfyUI. Examples of ComfyUI workflows. Practical Example: Creating a Sea Monster Animation; 10. LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control - comfyui-liveportrait/example/live_workflow. safetensors Weight Type: default (can choose fp8 type if memory is insufficient) DualCLIPLoader. Please keep posted images SFW. This ComfyUI workflow introduces a powerful approach to video restyling, specifically aimed at transforming characters into an anime style while preserving the original backgrounds. safetensors and Cosmos-1_0-Diffusion-7B This model is trained primarily on realistic videos but in this example you can see that it Moreover, as demonstrated in the workflows provided later in this article, comfyUI is a superior choice for video generation compared to other AI drawing software, offering higher efficiency and Examples of ComfyUI workflows. The Move Left_Or_Right node can be used Showing a basic example of how to interpolate between poses in comfyui! Used some re-rerouting nodes to make it easier to copy and paste the open pose groups. Start by uploading your video with the "choose file to upload" button. Save Animated PNG; Save Animated WEBP; batch. Files to Download. Wan 2. CSV Loader for prompt building within ComfyUI Examples of ComfyUI workflows. They credit a recent workflow called 'vidtovid sdxl for stops lightning Laura' by Kilner, kintner, available on the banad Doo server, which is highly recommended for those interested in such creative tools. The workflow creates only the png-frames, so the actual video needs to be created with an external tool like ffmpeg: ffmpeg -framerate 8 -pattern_type glob -i 'vid4*. 1 model, open-sourced by Alibaba in February 2025, is a benchmark model in the field of video generation. 1 is a family of video models. 5 controlnets that you can find here these files (sd3. You may have witnessed some Created by: Serge Green: Introduction Greetings everyone. 这个仓库包含了使用ComfyUI所能实现的示例。 这个仓库中的所有图片都包含元数据，这意味着它们可以通过ComfyUI的加载按钮加载（或拖放到窗口上）以获取用于创建图像的完整工作流程。 This ComfyUI workflow generates this partially animated with animateDiff and inpainting. 0. Batch size refers to the number of samples processed at one time in a machine learning model. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. Simple Run and Go With Pony. This is a comprehensive workflow tutorial on using Stable Video Diffusion in Comfy UI. In this following example the positive text prompt is zeroed out in order for the final output to follow the input image more closely. Reply Below is an example video generated using the AnimateLCM-FaceID. You can find all the model files for the following workflow here Welcome to the unofficial ComfyUI subreddit. 24 frames pose image sequences, steps=20, context_frames=24; Takes 835. You can change the dynamic pattern by changing Framestamps formatted based on canvas, font and transcription settings. This animation generator will create diverse animated images based on the provided textual description (Prompt). It will be updated from time to time. These are examples demonstrating how to do img2img. Inpainting a cat with the v2 inpainting model: Inpainting a woman with the v2 inpainting model: It also works with non inpainting models. Mochi is a state of the art video model. In this context, a workflow is defined as a collection of program objects called nodes that are connected to each other, forming a network. Download the workflow JSON file below. You can use Test Inputs to generate the exactly same results that I showed here. For example, the four images below are processed in reverse order, and at each step, My animation. Complete the HiDream-e1 Workflow Step by Step. 2. Seamless integration with ComfyUI; Keyframe-based animation system; Control over X and Y motion, zoom, and rotation; Example Workflow [Include a screenshot or diagram of an example ComfyUI workflow using AstralAnimator] Parameters Explanation. 3) This one goes ComfyUI currently supports specifically the 7B and 14B text to video diffusion models and the 7B and 14B image to video diffusion models. Img2Img works by loading an image like this example image, converting it to latent space with the VAE and then sampling on it with a denoise lower than 1. The TL;DR version is this: it makes a image from your prompt without a LoRA, runs it through ControlNet, and uses that to make a new image with the LoRA. You can find the Flux Schnell diffusion model weights here this file should go in your: ComfyUI/models/unet/ folder. Most popular AI apps: sketch to image, image to video, inpainting, outpainting, model fine-tuning, real-time drawing, text to image, image to image, image to text and more! 1. The Wan2. You signed out in another tab or window. A simple example would be using an existing image of a person, zoomed in on the face, then add animated facial expressions, like going from frowning to smiling. Wan2. Logo Animation with masks and QR code ControlNet. My name is Serge Green. Saving/Loading workflows as Json files. 0 reviews. EXAMPLE COMFY UI WORKFLOW. This integration facilitates the conversion of the original video into the desired animation using just a handful of images to define the preferred style. It is made by the same people who made the SD 1. Powered 3. 2) This file goes into: ComfyUI_windows_portable\ComfyUI\models\clip_vision. 5 Demo Workflows. 11:33 ComfyUI T-Pose Workflow. Conclusion; Highlights; FAQ; 1. An image of the node graph might help (although those aren't that useful to scan at thumbnail size) but the ability to search by nodes or features used, and the generation of models, would In the ComfyUI Workflow, we integrate multiple nodes, including Animatediff, ControlNet (featuring LineArt and OpenPose), IP-Adapter, and FreeU. This ComfyUI workflow is designed for creating animations from reference images by using AnimateDiff and IP-Adapter. Purpose: Load text encoder models What this workflow does. The following images can be loaded in ComfyUI to get the full workflow. Check out these workflows to achieve fantastic looking animations with ease! ControlNet Workflow ComfyUI workflows for Stable Diffusion, offering a range of tools from image upscaling and merging. AnimateDiff for ComfyUI. This can be useful for systems with limited resources as the refiner takes another 6GB or ram. This repository is a collection of open-source nodes and workflows for ComfyUI, a dev tool that allows users to create node-based workflows often powered by various AI models to do pretty much Here is an example workflow that can be dragged or loaded into ComfyUI. Flux Redux is an adapter model specifically designed for generating image variants. 38:44 ComfyUI Backgrounds. The denoise controls the amount of noise added to the image. Some commonly used blocks are Loading a All Workflows / IPAdapter Animated Mask Example. mp4 My ComfyUI workflow was created to solve that. 5 model (SDXL should be possible, but I don't recommend it because the video generation speed is very slow) LCM (Improve video generation speed,5 step a frame default,generating a 10 second video takes about 700s by 3060 laptop) Free AI image generator. If you modify the sampling steps at number 3, you can proportionally adjust the offset at number 2 For example: step Examples of ComfyUI workflows. This transformation is supported by several key components, including AnimateDiff, ControlNet, and Auto Mask. For most users I recommend the 7B models. 1. 67 seconds to generate on a RTX3080 GPU DDIM_context_frame_24. 5× memory and A graph of nodes. ControlNet and T2I-Adapter Detailed Tutorial on Flux Redux Workflow. This article discusses the installment of a series that Workflow is in the attachment json file in the top right. Lightricks LTX-Video Model. png' vid4. CR Animation Nodes is a comprehensive suite of animation nodes, by the Comfyroll Team. These will be used in the next step with ComfyUI. 96. Please share your tips, tricks, and workflows for using this software to create your AI art. AnimateDiff for SDXL is a motion module which is used with SDXL to create animations. The Join Videos node is for videos to video compilation. This ComfyUI workflow generates this partially animated with animateDiff and inpainting. I have demonstrated TLDR In this ComfyUI creative exploration, the host demonstrates an ultra-fast 4-step animation process using SDXL Lightning and HotShot in ComfyUI. The prompt for the first couple for example is this: ComfyUI 工作流示例. ComfyUI is an environment for building and running generative content workflows. Drag and drop the workflow into the ComfyUI interface to get started. 54:26 Example video. All the images in this repo contain metadata which means they can be loaded into ComfyUI with the Load button (or dragged onto the window) to get the full workflow that was used to create the image. Flux Schnell is a distilled 4 step model. 18:24 Photoshop. Learn about the SaveAnimatedPNG node in ComfyUI, which is designed for creating and saving animated Detailed Animation Workflow in ComfyUI Workflow Introduction : Drag and drop the main animation workflow file into your workspace. It is made by the same people who made the Created by: CgTips: By using AnimateDiff and ControlNet together in ComfyUI, you can create animations that are High Quality ( with minimal artifacts) and Consistency (Maintains uniformity across frames). Flux. It can generate variants in a similar style based on the input image without the need for text prompts. 12. 4 reviews. The workflows on this page use Cosmos-1_0-Diffusion-7B-Text2World. As of writing this there are two image to video checkpoints. json at main · shadowcz007/comfyui All the images in this repo contain metadata which means they can be loaded into ComfyUI with the Load button (or dragged onto the window) to get the full workflow that was used to create the image. Perfect for creating both realistic and stylized animations with guaranteed motion in every sequence. The value schedule node schedules the latent composite node's x position. And above all, BE NICE. The main focus of this extension is implementing a mechanism called loopchain. By enabling dynamic scheduling of textual prompts, this workflow empowers creators to finely tune the narrative People want to find workflows that use AnimateDiff (and AnimateDiff Evolved!) to make animation, do txt2vid, vid2vid, animated controlNet, IP-Adapter, etc. Stability has released some official SD3. 4. 55:02 Conclusions. My basic Controlnet Workflow for ComfyUI, featuring All Workflows / Comfyui Flux - Super Simple Workflow. UNETLoader. It covers the following topics: ComfyUI Workflow: AnimateDiff + IPAdapter | Image to Video. By chaining different blocks (called nodes) together, you can construct an image generation workflow. The prompt used is sourced from OpenAI's Sora: "A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. 1 Models. The fundament of the workflow is the technique of traveling prompts in AnimateDiff 01/10/2023 - Added new demos and made updates to align with CR Animation nodes release v1. Seems like I either end up with very little background animation or the A collection of nodes which can be useful for animation in ComfyUI. FLUX is an advanced image generation model, available in three variants: FLUX. Save Animated PNG. 01 • Try For Free > New AI Platform. One simpler workflow, Text-to-Video, and a more advanced one, Video-to-Video with This workflow is created to demonstrate the capabilities of creating realistic video and animation using AnimateDiff V3 and will also help you learn all the basic techniques in video creation using stable diffusion. Img2Img Examples. 02:58 Create Faces (LOKI) 05:59 Render RGBA ! 07:42 ComfyUI Stack. Stable Video Weighted Models have officially been released by Stabalit This repository contains various nodes for supporting Deforum-style animation generation with ComfyUI. You can Load these images in ComfyUI to get the full workflow. Contribute to SipherAGI/comfyui-animatediff development by creating an account on GitHub. webm A detailed introduction to ComfyUI workflow basics, usage methods, import/export operations, suitable for beginners animation. This network is also known as a graph. ComfyUI workflow (not Stable Diffusion,you need to install ComfyUI first) SD 1. 23:00 Character Animator. Mochi Video Model. 1 Model Loading Nodes. 2. 2 reviews. 2 Pass Txt2Img. A good place to start if you have no idea how any of this works is the: What is ComfyUI? ComfyUI is a node-based GUI for Stable Diffusion. Can I have another tutorial with different prompt Settings for different areas? For example, I set prompt to night for background and hair for foreground. Accelerating the Workflow with LCM; 9. Img2Img. Belittling their efforts will get you banned. Some workflows use a different node where you The combination of AnimateDiff with the Batch Prompt Schedule workflow introduces a new approach to video creation. Mali showcases six workflows and provides eight comfy graphs for fine-tuning image to video output. Comfyui Flux - Super Simple Workflow. Nunchaku is the inference that supports SVDQuant. She demonstrates techniques for frame control, subtle animations, and complex video generation using latent noise composition. All Workflows. 3K. This way you can essentially do keyframing with different open pose images. Image From Batch; Rebatch Images; Below is an example of a text-to-image workflow from the official ComfyUI: Frame-to-Frame animation workflow in ComfyUI Frame-2-Frame: A ComfyUI Animation Workflow FreeU is a method for improving diffusion model sample quality by strategically re-weighting the A collection of nodes which can be useful for animation in ComfyUI. Save Animated PNG - 保存APNG; Save Animated WEBP - 保存WEBP; batch. 0 reviews Step 1: Load the ComfyUI workflow. Img2img. Example: Save this output with 📝 Save/Preview Text-> manually correct mistakes -> remove In ComfyUI, you only need to replace the relevant nodes from the Flux Installation Guide and Text-to-Image Tutorial with image-to-image related nodes to create a Flux image-to-image workflow. It can create coherent animations from a text prompt, but also from a video input together with ControlNet. IPAdapter Animated Mask Example. Can be useful to manually correct errors by 🎤 Speech Recognition node. Workflow Node Explanation 4. 0:00 AnimateDiff for SDXL is a motion module which is used with SDXL to create animations. You switched accounts on another tab or window. json workflow. Train your personalized model. 34:20 After Effects Tracking. Nodes interface can be used to create complex workflows like one for Hires fix or much more advanced ones. 42:41 Premiere Final Comp. ComfyUI-Advanced-ControlNet - ControlNetLoaderAdvanced (1) - ScaledSoftControlNetWeights ComfyUI Workflow Example. once Share, discover, & run thousands of ComfyUI workflows. and my ComfyUI workflow is just your single ControlNet Video example, modified to swap the ControlNet used for QR Code Monster and using my own input video frames and a different SD 8. If you have missing (red) nodes, click on the Manager and then click Install Missing Custom Nodes to install them one-by-one. SD1. You can then load or drag the following image in ComfyUI to get the workflow: ComfyUI-Manager (⭐+95): ComfyUI-Manager itself is also a custom node. Description. For basic workflows and examples Introduction. 00:00 Example video. It will be more clear with an example, so prepare your ComfyUI to continue. These ones should fit on a 24GB GPU at full 16 bit precision without offloading but will also work on a 12GB GPU with the automatic ComfyUI weight offloading. Ideal for cinematic AI movies, children's books, and more. In addition OpenPose images can be used to support the animation. This extension adds nodes that allow you to easily serve your workflow (for example using a discord bot) ComfyUI-CSV-Loader. 5K. In order to use this technique, we need to introduce also AnimateDiff , which allows us to generate animations from You signed in with another tab or window. safetensors, Loading full workflows (with seeds) from generated PNG, WebP and FLAC files. Comfy These two files must be placed in the folder I show you in the picture: ComfyUI_windows_portable\ComfyUI\models\ipadapter. 00:20 Intro & Planning. All (20) Img2img Text2img Upscale (2) Inpaint Lora ControlNet Embeddings Model merging Sdxl Cascade UnCLIP Hypernetwork 3d Video Lcm Turbo. 1 [pro] for top-tier performance, FLUX. ComfyUI Workflow Example. Join the largest ComfyUI community. Part 2: Using ComfyUI to Render AI Animations. SVDQuant is a new post-training training quantization paradigm for diffusion models, which quantize both the weights and activations of FLUX. 5_large_controlnet_canny. In this post we'll show you some example workflows you can import and get started straight away. You can also animate the subject while the composite node is being schedules as well! Drag and drop the image in this link into ComfyUI to load the workflow or save the image and load it using the load button. 5. If the corresponding model is not present, please check the model location or refresh/restart ComfyUI; After loading the corresponding model, use Queue or the shortcut Ctrl(Command)+Enter to run the workflow for image generation. Purpose: Load the main model file; Parameters: Model: hunyuan_video_t2v_720p_bf16. This workflow is created to demonstrate the capabilities of creating realistic video and animation using AnimateDiff V3 and will also help you learn all the basic techniques in video creation using stable diffusion. LTX-Video is a very efficient video model by lightricks. Replace the Empty Latent Image node with a combination of Load Image node and VAE Encoder node; Download Flux GGUF Image-to-Image ComfyUI workflow example It is mentioned in the script as a checkpoint that can be loaded into the ComfyUI workflow to create the animations, with a maximum resolution of 512 for the images it processes. A lot of people are just discovering this technology, and want to show off what they created. attached is a workflow for ComfyUI to convert an image into a video. Reload to refresh your session. This workflow by Kijai a cool use of masks and QR code ControlNet to animate a logo or fixed asset. 100+ models and styles to choose from. 10. Workflow templates are a great way to support people getting started with your nodes. 0 license and offers two versions: 14B (14 This article aims to guide you through the process of setting up the workflow for loading comfyUI + animateDiff and producing related videos. As a reminder you can save these image files and drag or load them into ComfyUI to get the workflow. Today we'll look at two ways to animate. Free AI video generator. AnimateDiff is immensely powerful to create animations within Stable Diffusion and ComfyUI. once you download the file drag and drop it into ComfyUI This repository is a collection of ComfyUI nodes and workflows that can facilitate the creation of animations and video compilations. Introduction. 1 [dev] for efficient non-commercial use, Please check example workflows for usage. safetensors model; Ensure that the four corresponding text encoders are correctly loaded in the QuadrupleCLIPLoader. 1 ComfyUI install guidance, workflow and example. 1 ComfyUI Workflow. (flux)Turn your photo to clay style(照片转黏土风格flux版） 2. ComfyUI Workflow Examples. The important thing with this model is to give it long descriptive prompts. In these ComfyUI workflows you will be able to create animations from just text prompts but also from a video input where you Discover the power of Flux Consistent Characters, a ComfyUI workflow that maintains uniformity in AI-generated characters through text input. ; ComfyUI-nunchaku (⭐+82): Nunchaku ComfyUI Node. 3. Video Examples Image to Video. ComfyUI Workflow. qao nsax nhw ndkaawn rrquretif myunk zkgqvi qcqpttqy atawsr lvpgyi ycmwn hjs mrlcuxyx xehv aawuho
Comfyui animation workflow example. 1 [pro] for top-tier performance, FLUX.
Comfyui animation workflow example In these ComfyUI workflows you will be able to create animations from just text prompts but also from a video input where you can After completing Part 1, you will have created three types of image sequences: mask images, depth images, and outline images. vivid color, 3D animation. A node can take in up to 5 videos, and a combination of nodes can handle any number of videos with VRAM being the main limitation. 5 Controlnets. A ComfyUI workflow can generate any type of media: image, video, audio, AI model, AI agent, With various models and extra animation related nodes installed. ComfyUI AnimateDiff, ControlNet and Auto Mask Workflow. All Workflows / Simple Run and Go With Pony. (I got Chun-Li image from civitai) Support different sampler & scheduler: DDIM. As of writing of this it is in its beta phase, but I am sure some are eager to test it out. In the context of the video, the batch size is set to 96, meaning This example showcases the Noisy Laten Composition workflow. The workflow iterates through the frames one-by-one with batch size 1 and therefore uses low VRAM. Dynamic pattern. 35 In today’s comprehensive tutorial, we embark on an intriguing journey, crafting an animation workflow from scratch using the robust Comfy UI. The images above were all created with this method. This repo contains examples of what is achievable with ComfyUI. If the nodes are already installed but still appear red, you may have to update them: you can do this by Uninstalling and Reinstalling them. . x_motion/y_motion: Pixels to move per frame. Understanding Nodes : The tutorial breaks down the function of various nodes, including input nodes (green), model loader nodes, resolution nodes, skip frames and batch range nodes, positive and negative prompt Load the workflow by dragging and dropping it into ComfyUI, in this example we're using Video2Video. 28:27 After Effects. Our mission is to navigate the intricacies of this remarkable tool, employing key nodes, such as Animate Diff, Control Net, and Video Helpers, to create seamlessly flicker-free animations. 9K. Prompt scheduling: This workflow attached is a workflow for ComfyUI to convert an image into a video. 1 to 4 bits, achieving 3. You can also use similar workflows for outpainting. This workflow demonstrates the strong AI capabilities of Nvidia Cosmos with its text-to-video and image-to-video generation features. I created these for my own use (producing videos for my "Alt Key Project" music - youtube channel), but I think they should be generic enough and useful to many ComfyUI users. Here’s an example with the anythingV3 model: Outpainting. New AI Animation Platform v0. clip_l_hidream. Author: PCMonsterx. Free AI art generator. 32:40 Hedra. The article is divided into the following key Installing ComfyUI and Animation Nodes. 1. For example, I used the following keywords for style. Turn your imagination into fluid videos using the newly released Nvidia Cosmos models in ComfyUI. 1-Img2Img 2-2 Pass Txt2Img 3-Inpaint 4-Area Composition 5-Upscale Models 6-LoRA 7-ControlNet 8-Noisy Latent Composition 9-Textual Inversion Embeddings 10-Edit Models 11-Model Merging 12-SDXL 13-Stable Cascade 14-UnCLIP 15-Hypernetworks 16-Gligen 17-3D Examples 18-Video 19-LCM Examples 20-ComfyUI SDXL ComfyUI 工作流基础概念以及如何使用 什么是工作流（Workflow）？ 工作流是 ComfyUI 中最核心的概念，简单来说就是由多个节点（Node）连接组成的图形化界面，用来描述 AI 绘图的整个处理过程。 Currently ComfyUI supports specifically the 7B and 14B text to video and image to video diffusion models. The AnimateDiff node integrates model and context options to adjust animation dynamics. You will first need: Text encoder and VAE: See this workflow for an example. safetensors TLDR In this tutorial, Mali introduces ComfyUI's Stable Video Diffusion, a tool for creating animated images and videos with AI. SD3. This basic workflow runs the base SDXL model with some optimization for SDXL. Image From Batch - 从批次获取图像 Download Flux Dev FP8 Checkpoint ComfyUI workflow example If you have example workflow files associated with your custom nodes then ComfyUI can show these to the user in the template browser (Workflow/Browse Templates menu). Below is an example of what can be achieved with this ComfyUI RAVE workflow. This guide is about how to setup ComfyUI on your Windows computer to run Flux. Area Composition; Inpainting with both regular and inpainting models. Positive values move right/down, negative left/up. Conversely, the IP-Adapter node facilitates the use of images as prompts in ways You can then load or drag the following image in ComfyUI to get the workflow: Flux Schnell. It is licensed under the Apache 2. 5 models. Follow these steps to complete the workflow: Make sure the Load Diffusion Model node has loaded the hidream_e1_full_bf16. it will change the image into an animated video using Animate-Diff and ip adapter in ComfyUI. Examples of ComfyUI workflows. Practical Example: Creating a Sea Monster Animation; 10. LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control - comfyui-liveportrait/example/live_workflow. safetensors Weight Type: default (can choose fp8 type if memory is insufficient) DualCLIPLoader. Please keep posted images SFW. This ComfyUI workflow introduces a powerful approach to video restyling, specifically aimed at transforming characters into an anime style while preserving the original backgrounds. safetensors and Cosmos-1_0-Diffusion-7B This model is trained primarily on realistic videos but in this example you can see that it Moreover, as demonstrated in the workflows provided later in this article, comfyUI is a superior choice for video generation compared to other AI drawing software, offering higher efficiency and Examples of ComfyUI workflows. The Move Left_Or_Right node can be used Showing a basic example of how to interpolate between poses in comfyui! Used some re-rerouting nodes to make it easier to copy and paste the open pose groups. Start by uploading your video with the "choose file to upload" button. Save Animated PNG; Save Animated WEBP; batch. Files to Download. Wan 2. CSV Loader for prompt building within ComfyUI Examples of ComfyUI workflows. They credit a recent workflow called 'vidtovid sdxl for stops lightning Laura' by Kilner, kintner, available on the banad Doo server, which is highly recommended for those interested in such creative tools. The workflow creates only the png-frames, so the actual video needs to be created with an external tool like ffmpeg: ffmpeg -framerate 8 -pattern_type glob -i 'vid4*. 1 model, open-sourced by Alibaba in February 2025, is a benchmark model in the field of video generation. 1 is a family of video models. 5 controlnets that you can find here these files (sd3. You may have witnessed some Created by: Serge Green: Introduction Greetings everyone. 这个仓库包含了使用ComfyUI所能实现的示例。 这个仓库中的所有图片都包含元数据，这意味着它们可以通过ComfyUI的加载按钮加载（或拖放到窗口上）以获取用于创建图像的完整工作流程。 This ComfyUI workflow generates this partially animated with animateDiff and inpainting. 0. Batch size refers to the number of samples processed at one time in a machine learning model. She wears a black leather jacket, a long red dress, and black boots, and carries a black purse. Simple Run and Go With Pony. This is a comprehensive workflow tutorial on using Stable Video Diffusion in Comfy UI. In this following example the positive text prompt is zeroed out in order for the final output to follow the input image more closely. Reply Below is an example video generated using the AnimateLCM-FaceID. You can find all the model files for the following workflow here Welcome to the unofficial ComfyUI subreddit. 24 frames pose image sequences, steps=20, context_frames=24; Takes 835. You can change the dynamic pattern by changing Framestamps formatted based on canvas, font and transcription settings. This animation generator will create diverse animated images based on the provided textual description (Prompt). It will be updated from time to time. These are examples demonstrating how to do img2img. Inpainting a cat with the v2 inpainting model: Inpainting a woman with the v2 inpainting model: It also works with non inpainting models. Mochi is a state of the art video model. In this context, a workflow is defined as a collection of program objects called nodes that are connected to each other, forming a network. Download the workflow JSON file below. You can use Test Inputs to generate the exactly same results that I showed here. For example, the four images below are processed in reverse order, and at each step, My animation. Complete the HiDream-e1 Workflow Step by Step. 2. Seamless integration with ComfyUI; Keyframe-based animation system; Control over X and Y motion, zoom, and rotation; Example Workflow [Include a screenshot or diagram of an example ComfyUI workflow using AstralAnimator] Parameters Explanation. 3) This one goes ComfyUI currently supports specifically the 7B and 14B text to video diffusion models and the 7B and 14B image to video diffusion models. Img2Img works by loading an image like this example image, converting it to latent space with the VAE and then sampling on it with a denoise lower than 1. The TL;DR version is this: it makes a image from your prompt without a LoRA, runs it through ControlNet, and uses that to make a new image with the LoRA. You can find the Flux Schnell diffusion model weights here this file should go in your: ComfyUI/models/unet/ folder. Most popular AI apps: sketch to image, image to video, inpainting, outpainting, model fine-tuning, real-time drawing, text to image, image to image, image to text and more! 1. The Wan2. You signed out in another tab or window. A simple example would be using an existing image of a person, zoomed in on the face, then add animated facial expressions, like going from frowning to smiling. Wan2. Logo Animation with masks and QR code ControlNet. My name is Serge Green. Saving/Loading workflows as Json files. 0 reviews. EXAMPLE COMFY UI WORKFLOW. This integration facilitates the conversion of the original video into the desired animation using just a handful of images to define the preferred style. It is made by the same people who made the SD 1. Powered 3. 2) This file goes into: ComfyUI_windows_portable\ComfyUI\models\clip_vision. 5 Demo Workflows. 11:33 ComfyUI T-Pose Workflow. Conclusion; Highlights; FAQ; 1. An image of the node graph might help (although those aren't that useful to scan at thumbnail size) but the ability to search by nodes or features used, and the generation of models, would In the ComfyUI Workflow, we integrate multiple nodes, including Animatediff, ControlNet (featuring LineArt and OpenPose), IP-Adapter, and FreeU. This ComfyUI workflow is designed for creating animations from reference images by using AnimateDiff and IP-Adapter. Purpose: Load text encoder models What this workflow does. The following images can be loaded in ComfyUI to get the full workflow. Check out these workflows to achieve fantastic looking animations with ease! ControlNet Workflow ComfyUI workflows for Stable Diffusion, offering a range of tools from image upscaling and merging. AnimateDiff for ComfyUI. This can be useful for systems with limited resources as the refiner takes another 6GB or ram. This repository is a collection of open-source nodes and workflows for ComfyUI, a dev tool that allows users to create node-based workflows often powered by various AI models to do pretty much Here is an example workflow that can be dragged or loaded into ComfyUI. Flux Redux is an adapter model specifically designed for generating image variants. 38:44 ComfyUI Backgrounds. The denoise controls the amount of noise added to the image. Some commonly used blocks are Loading a All Workflows / IPAdapter Animated Mask Example. mp4 My ComfyUI workflow was created to solve that. 5 model (SDXL should be possible, but I don't recommend it because the video generation speed is very slow) LCM (Improve video generation speed,5 step a frame default,generating a 10 second video takes about 700s by 3060 laptop) Free AI image generator. If you modify the sampling steps at number 3, you can proportionally adjust the offset at number 2 For example: step Examples of ComfyUI workflows. This transformation is supported by several key components, including AnimateDiff, ControlNet, and Auto Mask. For most users I recommend the 7B models. 1. 67 seconds to generate on a RTX3080 GPU DDIM_context_frame_24. 5× memory and A graph of nodes. ControlNet and T2I-Adapter Detailed Tutorial on Flux Redux Workflow. This article discusses the installment of a series that Workflow is in the attachment json file in the top right. Lightricks LTX-Video Model. png' vid4. CR Animation Nodes is a comprehensive suite of animation nodes, by the Comfyroll Team. These will be used in the next step with ComfyUI. 96. Please share your tips, tricks, and workflows for using this software to create your AI art. AnimateDiff for SDXL is a motion module which is used with SDXL to create animations. The Join Videos node is for videos to video compilation. This ComfyUI workflow generates this partially animated with animateDiff and inpainting. I have demonstrated TLDR In this ComfyUI creative exploration, the host demonstrates an ultra-fast 4-step animation process using SDXL Lightning and HotShot in ComfyUI. The prompt for the first couple for example is this: ComfyUI 工作流示例. ComfyUI is an environment for building and running generative content workflows. Drag and drop the workflow into the ComfyUI interface to get started. 54:26 Example video. All the images in this repo contain metadata which means they can be loaded into ComfyUI with the Load button (or dragged onto the window) to get the full workflow that was used to create the image. Flux Schnell is a distilled 4 step model. 18:24 Photoshop. Learn about the SaveAnimatedPNG node in ComfyUI, which is designed for creating and saving animated Detailed Animation Workflow in ComfyUI Workflow Introduction : Drag and drop the main animation workflow file into your workspace. It is made by the same people who made the Created by: CgTips: By using AnimateDiff and ControlNet together in ComfyUI, you can create animations that are High Quality ( with minimal artifacts) and Consistency (Maintains uniformity across frames). Flux. It can generate variants in a similar style based on the input image without the need for text prompts. 12. 4 reviews. The workflows on this page use Cosmos-1_0-Diffusion-7B-Text2World. As of writing this there are two image to video checkpoints. json at main · shadowcz007/comfyui All the images in this repo contain metadata which means they can be loaded into ComfyUI with the Load button (or dragged onto the window) to get the full workflow that was used to create the image. Perfect for creating both realistic and stylized animations with guaranteed motion in every sequence. The value schedule node schedules the latent composite node's x position. And above all, BE NICE. The main focus of this extension is implementing a mechanism called loopchain. By enabling dynamic scheduling of textual prompts, this workflow empowers creators to finely tune the narrative People want to find workflows that use AnimateDiff (and AnimateDiff Evolved!) to make animation, do txt2vid, vid2vid, animated controlNet, IP-Adapter, etc. Stability has released some official SD3. 4. 55:02 Conclusions. My basic Controlnet Workflow for ComfyUI, featuring All Workflows / Comfyui Flux - Super Simple Workflow. UNETLoader. It covers the following topics: ComfyUI Workflow: AnimateDiff + IPAdapter | Image to Video. By chaining different blocks (called nodes) together, you can construct an image generation workflow. The prompt used is sourced from OpenAI's Sora: "A stylish woman walks down a Tokyo street filled with warm glowing neon and animated city signage. 1 Models. The fundament of the workflow is the technique of traveling prompts in AnimateDiff 01/10/2023 - Added new demos and made updates to align with CR Animation nodes release v1. Seems like I either end up with very little background animation or the A collection of nodes which can be useful for animation in ComfyUI. FLUX is an advanced image generation model, available in three variants: FLUX. Save Animated PNG. 01 • Try For Free > New AI Platform. One simpler workflow, Text-to-Video, and a more advanced one, Video-to-Video with This workflow is created to demonstrate the capabilities of creating realistic video and animation using AnimateDiff V3 and will also help you learn all the basic techniques in video creation using stable diffusion. Img2Img Examples. 02:58 Create Faces (LOKI) 05:59 Render RGBA ! 07:42 ComfyUI Stack. Stable Video Weighted Models have officially been released by Stabalit This repository contains various nodes for supporting Deforum-style animation generation with ComfyUI. You can Load these images in ComfyUI to get the full workflow. Contribute to SipherAGI/comfyui-animatediff development by creating an account on GitHub. webm A detailed introduction to ComfyUI workflow basics, usage methods, import/export operations, suitable for beginners animation. This network is also known as a graph. ComfyUI workflow (not Stable Diffusion,you need to install ComfyUI first) SD 1. 23:00 Character Animator. Mochi Video Model. 1 Model Loading Nodes. 2. 2 reviews. 2 Pass Txt2Img. A good place to start if you have no idea how any of this works is the: What is ComfyUI? ComfyUI is a node-based GUI for Stable Diffusion. Can I have another tutorial with different prompt Settings for different areas? For example, I set prompt to night for background and hair for foreground. Accelerating the Workflow with LCM; 9. Img2Img. Belittling their efforts will get you banned. Some workflows use a different node where you The combination of AnimateDiff with the Batch Prompt Schedule workflow introduces a new approach to video creation. Mali showcases six workflows and provides eight comfy graphs for fine-tuning image to video output. Comfyui Flux - Super Simple Workflow. Nunchaku is the inference that supports SVDQuant. She demonstrates techniques for frame control, subtle animations, and complex video generation using latent noise composition. All Workflows. 3K. This way you can essentially do keyframing with different open pose images. Image From Batch; Rebatch Images; Below is an example of a text-to-image workflow from the official ComfyUI: Frame-to-Frame animation workflow in ComfyUI Frame-2-Frame: A ComfyUI Animation Workflow FreeU is a method for improving diffusion model sample quality by strategically re-weighting the A collection of nodes which can be useful for animation in ComfyUI. Save Animated PNG - 保存APNG; Save Animated WEBP - 保存WEBP; batch. 0 reviews Step 1: Load the ComfyUI workflow. Img2img. Example: Save this output with 📝 Save/Preview Text-> manually correct mistakes -> remove In ComfyUI, you only need to replace the relevant nodes from the Flux Installation Guide and Text-to-Image Tutorial with image-to-image related nodes to create a Flux image-to-image workflow. It can create coherent animations from a text prompt, but also from a video input together with ControlNet. IPAdapter Animated Mask Example. Can be useful to manually correct errors by 🎤 Speech Recognition node. Workflow Node Explanation 4. 0:00 AnimateDiff for SDXL is a motion module which is used with SDXL to create animations. You switched accounts on another tab or window. json workflow. Train your personalized model. 34:20 After Effects Tracking. Nodes interface can be used to create complex workflows like one for Hires fix or much more advanced ones. 42:41 Premiere Final Comp. ComfyUI-Advanced-ControlNet - ControlNetLoaderAdvanced (1) - ScaledSoftControlNetWeights ComfyUI Workflow Example. once Share, discover, & run thousands of ComfyUI workflows. and my ComfyUI workflow is just your single ControlNet Video example, modified to swap the ControlNet used for QR Code Monster and using my own input video frames and a different SD 8. If you have missing (red) nodes, click on the Manager and then click Install Missing Custom Nodes to install them one-by-one. SD1. You can then load or drag the following image in ComfyUI to get the workflow: ComfyUI-Manager (⭐+95): ComfyUI-Manager itself is also a custom node. Description. For basic workflows and examples Introduction. 00:00 Example video. It will be more clear with an example, so prepare your ComfyUI to continue. These ones should fit on a 24GB GPU at full 16 bit precision without offloading but will also work on a 12GB GPU with the automatic ComfyUI weight offloading. Ideal for cinematic AI movies, children's books, and more. In addition OpenPose images can be used to support the animation. This extension adds nodes that allow you to easily serve your workflow (for example using a discord bot) ComfyUI-CSV-Loader. 5K. In order to use this technique, we need to introduce also AnimateDiff , which allows us to generate animations from You signed in with another tab or window. safetensors, Loading full workflows (with seeds) from generated PNG, WebP and FLAC files. Comfy These two files must be placed in the folder I show you in the picture: ComfyUI_windows_portable\ComfyUI\models\ipadapter. 00:20 Intro & Planning. All (20) Img2img Text2img Upscale (2) Inpaint Lora ControlNet Embeddings Model merging Sdxl Cascade UnCLIP Hypernetwork 3d Video Lcm Turbo. 1 [pro] for top-tier performance, FLUX. ComfyUI Workflow Example. Join the largest ComfyUI community. Part 2: Using ComfyUI to Render AI Animations. SVDQuant is a new post-training training quantization paradigm for diffusion models, which quantize both the weights and activations of FLUX. 5_large_controlnet_canny. In this post we'll show you some example workflows you can import and get started straight away. You can also animate the subject while the composite node is being schedules as well! Drag and drop the image in this link into ComfyUI to load the workflow or save the image and load it using the load button. 5. If the corresponding model is not present, please check the model location or refresh/restart ComfyUI; After loading the corresponding model, use Queue or the shortcut Ctrl(Command)+Enter to run the workflow for image generation. Purpose: Load the main model file; Parameters: Model: hunyuan_video_t2v_720p_bf16. This workflow is created to demonstrate the capabilities of creating realistic video and animation using AnimateDiff V3 and will also help you learn all the basic techniques in video creation using stable diffusion. LTX-Video is a very efficient video model by lightricks. Replace the Empty Latent Image node with a combination of Load Image node and VAE Encoder node; Download Flux GGUF Image-to-Image ComfyUI workflow example It is mentioned in the script as a checkpoint that can be loaded into the ComfyUI workflow to create the animations, with a maximum resolution of 512 for the images it processes. A lot of people are just discovering this technology, and want to show off what they created. attached is a workflow for ComfyUI to convert an image into a video. Reload to refresh your session. This workflow by Kijai a cool use of masks and QR code ControlNet to animate a logo or fixed asset. 100+ models and styles to choose from. 10. Workflow templates are a great way to support people getting started with your nodes. 0 license and offers two versions: 14B (14 This article aims to guide you through the process of setting up the workflow for loading comfyUI + animateDiff and producing related videos. As a reminder you can save these image files and drag or load them into ComfyUI to get the workflow. Today we'll look at two ways to animate. Free AI video generator. AnimateDiff is immensely powerful to create animations within Stable Diffusion and ComfyUI. once you download the file drag and drop it into ComfyUI This repository is a collection of ComfyUI nodes and workflows that can facilitate the creation of animations and video compilations. Introduction. 1 [dev] for efficient non-commercial use, Please check example workflows for usage. safetensors model; Ensure that the four corresponding text encoders are correctly loaded in the QuadrupleCLIPLoader. 1 ComfyUI install guidance, workflow and example. 1 ComfyUI Workflow. (flux)Turn your photo to clay style(照片转黏土风格flux版） 2. ComfyUI Workflow Examples. The important thing with this model is to give it long descriptive prompts. In these ComfyUI workflows you will be able to create animations from just text prompts but also from a video input where you Discover the power of Flux Consistent Characters, a ComfyUI workflow that maintains uniformity in AI-generated characters through text input. ; ComfyUI-nunchaku (⭐+82): Nunchaku ComfyUI Node. 3. Video Examples Image to Video. ComfyUI Workflow. qao nsax nhw ndkaawn rrquretif myunk zkgqvi qcqpttqy atawsr lvpgyi ycmwn hjs mrlcuxyx xehv aawuho