Beta 2 -

fixed save location for pose and line art

fixed batching and re-batching for SAM custom masks

added a default project folder with a default video its 400+ frames original so limit the frames if you have a lower vram card to use the default. Sheesh Dance, example in previews

Put project folder inside comfy ui output folder to load default workflow

V5 beta

- reworked interface

- added simple masking creation and save

- added dual mask loading

- added depth, pose, lineart generation and save

- Reworked loading controls

- Reworked prompting

- Simple Prompting, Scheduled Prompting, Regional Prompting

- added more custom masking options

- using constants to prevent line spam

- updated all video combines

Fixes

- not using clipvison the wrong way

Removed

-Depth lineart

-Face Masking

-Face Mesh

-Face Controls

-Background scroll/pan

Looking to get feedback if this is better, worse or whatever.

Leave a comment if you think its worse for some reason, or a thumb up i you like it

Not sure if its better than v4 but its a lot cleaner.

Updated workflow to fix the spin1 example, it now has the needed files, sorry.

Starting to go through this to optimize and fix anything comment on anything you would like added or changed.

masking 4.1

- added Depth Anything v2

- added dense pose saving for testing

- added proper saving locations, now all control net files are saved to the projects folder inside the chosen video folder for easier sorting

note - all files still need to be renamed to remove the 00001 extensions.

- added proper switch to change the save directory name

V4Simplified - this version is cut down to only offer less confusing options with a clear path.

- I dont know many more things to make this easier. Some advanced things removed, and UI is now a walkthrough.

- Removed RGB masks \ multi prompting (use advanced version for that)

- Added Step groups to help guild new users.

- Added new notes

All 4 parts included in 1

Masking

Videos

Upscale

Extras

• 2 example projects

• 1 spin as usual

• 1 anime dance that was requested.

• a few old projects to show examples, they are problably out of date but should show you settings i used.

If You tried this workflow before and thought it was too complicated, this version is for you. This took a while to put together.

May 24th - bug - downscaling masks can cause preview problems. Will fix for release

V4 - preview - 1 workflow set for the spin animation with 6 different ways to do it. PNG examples

- Large VRAM savings, usage down about 50% I think due to IP adaptors and prompting combines. 6gb ram may be useable for smaller videos, especially in painting.

- removed cropping (do that in masking flow if needed).

- masked prompting basic has been removed because it was redundant with the RGB masks doing the same things.

- RGB masks now default to black and white masks as normal. Can just enable any color after that.

- All IP adaptor loading is now unified... big error in the betas was the loading of the IP adaptors, causing much higher memory use than should have been when using more than one.

- finished... kinda the scroll/pan/zoom BG generations.

- Added new notes for things, more needed

- Cleaned up interface for release

beta 4 - i know i said the last one was the last but i need this tested before i call it ready.

- full prompting rework to make it as simple or as complex as you want.

- no masking path simplified

- Black and white masks path for subject/background simplified

- full RGB mask path, up to 3 people and background added (complex)

- On/off controls for all prompting options.

- custom controlnet masks cleaned up.

- added switch for animate diff context options choice

- added FPS selector for anyone importing image folder for the source video instead of mp4

- Full pipe network to get rid of node connection spam.

masking beta2 included without the dumb batching.

Masking v4beta

Features

- batching system

- Depth map saving

- Open Pose saving

- Animal pose saving

- Mask Generations, combining, editing, and coloring

- Coco masking

- SAM SEG masking

- Depth Masking

- all mixable

- Person separation color masking

- SAM Face Masks

- Pose Face Masks

- Media pipe face mesh

beta 3 - scheduled prompting uses interpolation between keyframes you make.

probably the last beta before 4 comes with upscale

- unrats nest almost complete. Most nodes now run through a bus

- rearranged previews to the start of the flow, rearanged middle for my sanity.

- added combine masks to combine video masks into one mask. Lower vram but much less accurate.

- added custom masking preview to all custom mask areas.

- added custom mask mix with subject masks

- added masking connections everywhere to allow subject/background/faces/custom masks to be used anywhere.

- added line art from depth as an option with line masks.

- added video preview for numbered frames

- added background scroller option to make scrolling backgrounds. (testing)

- removed bg depth/line groups. Now just generated when you connect it. no prevews.

- remove/add to masks should be easier with the combine masks preview.

- tried to remove unneeded things to not confuse people.

- mask previews now show lineart of the masked area to display what the control net will be working off of.

- added repeat video for those small .gif files that are small number of frames and can be looped.

It appears when i talked to fizznodes creator, he or I misunderstood each other. Im pretty sure the scheduling nodes are still using interpolation. He advised me to use string schedule to avoid the prompt weights. I thought it was working but it has become clear it is not now. Not sure if it was an update.

You can change this like before if you want

Otherwise, know that the scheduler interpolates between prompts again.

v4beta 2

- Fixed disable masks node set to properly disable masks. This probably caused problems for people who wanted to disable masking.

- added subject mask add/remove groups to allow slight modifications to the masks.

- added face mesh creation if you did not save

- added face mesh conversion into face masks if you can get a good face mesh set.

- Changed the separate prompting to custom mask prompting. Can be used the same but more precise masking

- Changed the order of ip adaptor and prompt lora loading.

- added many custom mask group options

- added optional ip adaptor background option.

- continued to un rats nest the flow.

Beta v4 - Looking for input on any problems or questions so i can make things more clear for people in the flow.

Flow UI changed to a fully left to right directed flow. Start on the left and move right.

Contains the workflow files for the singing animation in the previews, 12gb rec for it.

Also has an upscale png, its an early version of 4.0 upscale if you want to try it as well.

Please comment any problems, questions.

3.6 - Next version is 4.0 . This is just to add the loading of new controls before the 4.0 change of interface. This version is fairly undocumented. 4.0 will be a more left to right interface.

- added face mask loading both openpose/SAM masks

- added face mesh loading

- added controls for face masked line art and face mesh for both base and upscale pass

- added custom mask control (this allows you to mask an area to use full lineart control over) enabled for both base and upscale pass

- added hand masking option... works about as well as you can get the detector to work..... tip it rarely works.

- added Line art background option for all 3 background replacements and original frames

- added resize cutout to the placement background options to further control location/size

masking 3.5

- added SAM face mask saving

- added open pose face mask saving

- added media pipe face mesh saving

- few small other fixes

Videos 3.6 will use these as options

Videos 3.5

- Separated groups more, hopefully improving readability of the flow, I like compact flows, but they are hard to follow for most people.

- Added Main IP adaptor

- added Split prompts for 2 persons prompting.

- added Split IP adaptors for 2 person IP adaptor references.

- added Options for upscale prompt / replace prompts

- added depth contrast adjustments

- added lineart for upscale use.

- added face masking to use with lineart for face animations.

- This flow comes with a few project folders with png workflows showing a few of the different ways to use the settings.

-- these pngs may be slightly different from the main json workflow. While i was creating them i changed/added things in the flow. Its best to use the main json workflow when starting your own video. The pngs are there to show you settings needed to copy different videos.

-- each comes with the controlnet files used, png workflow, step1 and step2 example videos, and a readme.

Single workflows

- Spin workflow - depth/admotion

- walking workflow - depth/openpose

- hairflip workflow - depth/line/pose

Split workflows

- 2 person separation open pose IP adaptors for dance scene

- 2 person separation lineart/depth IP adaptors for kissing scene

Upscale 3.5

• added ip adaptors

• added seperated prompts.

• added lineart stabilzer

some possible nodes needed

https://github.com/ltdrdata/ComfyUI-Manager

https://github.com/Kosinkadink/ComfyUI-VideoHelperSuite

https://github.com/crystian/ComfyUI-Crystools

https://github.com/mav-rik/facerestore_cf

https://github.com/ltdrdata/ComfyUI-Impact-Pack

https://github.com/WASasquatch/was-node-suite-comfyui

https://github.com/Nourepide/ComfyUI-Allor

https://github.com/Suzie1/ComfyUI_Comfyroll_CustomNodes

https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet

https://github.com/cubiq/ComfyUI_essentials

https://github.com/Fannovel16/ComfyUI-Frame-Interpolation

https://github.com/SLAPaper/ComfyUI-Image-Selector

https://github.com/FizzleDorf/ComfyUI_FizzNodes

https://github.com/LonicaMewinsky/ComfyUI-MakeFrame

Controlnet Models used

Think these2 are the same, or they produce very similar results.

(unknown name) adMotion - https://huggingface.co/crishhh/animatediff_controlnet/resolve/main/controlnet_checkpoint.ckpt?download=true

Animatediff Controlnet - https://civitai.com/models/232617/animatediff-controlnet-models

Media Pipe Face Mesh - https://huggingface.co/CrucibleAI/ControlNetMediaPipeFace/tree/main

Outfit to Outfit - https://civitai.com/models/191956/outfit-to-outfit-controlnet-outfit2outfit

pix2pix - https://civitai.com/models/38784/controlnet-11-models