[SD 1.5][LORA]The Idolmaster Cinderella Girls LoRA Lite (style+9 chars from U149) v1-linear

0xSoul

A LoRA (Hadamard Product) based on Cinderella Girls Cards and Starlight Stage Cards of the 9 characters in the upcoming U149 anime. A small test before training for many more characters in Starlight Stage.

Usage

This Hadamard Product LoRA requires installing an extra extension to be used in the Web UI.

The style itself doesn't have a trigger word. For the characters, below are the character tags and respective top 20 related tags.

古賀小春,
short hair, brown hair, 1girl, smile, brown eyes, solo, open mouth, bow, dress, looking at viewer, hair bow, hairband, blush, sitting, frills, light brown hair, sky, :d, gloves, blonde hair

赤城みりあ,
short hair, black hair, smile, brown eyes, two side up, open mouth, looking at viewer, 1girl, solo, blush, hair ornament, :d, brown hair, multiple girls, skirt, jewelry, one eye closed, closed eyes, gloves, shorts

結城晴,
orange hair, smile, 1girl, solo, long hair, open mouth, shorts, looking at viewer, blush, sky, day, brown eyes, gloves, outdoors, multiple girls, brown hair, midriff, navel, hat, shirt

龍崎薫,
short hair, hair ornament, hairclip, smile, brown hair, open mouth, 1girl, yellow eyes, solo, orange hair, shorts, looking at viewer, closed eyes, hat, brown eyes, blush, food, multiple girls, flower, skirt

的場梨沙,
black hair, long hair, yellow eyes, twintails, smile, jewelry, 1girl, ribbon, solo, hair ribbon, open mouth, necklace, looking at viewer, blush, skirt, animal print, bangs, midriff, one eye closed, sitting

橘ありす,
brown eyes, brown hair, long hair, 1girl, solo, bow, looking at viewer, smile, hair bow, dress, open mouth, blush, ribbon, food, flower, sitting, fruit, jewelry, hair ornament, strawberry

佐々木千枝,
short hair, black hair, smile, black eyes, hair ornament, 1girl, rabbit hair ornament, solo, open mouth, hairclip, looking at viewer, blush, skirt, dress, brown hair, jewelry, holding, multiple girls, sitting, indoors

櫻井桃華,
blonde hair, green eyes, short hair, smile, 1girl, looking at viewer, solo, blush, wavy hair, dress, open mouth, frills, hairband, flower, hair ornament, bow, gloves, jewelry, rose, sitting

市原仁奈,
long hair, brown hair, smile, open mouth, blush, 1girl, brown eyes, solo, blunt bangs, cosplay, animal costume, kigurumi, :d, bow, bangs, looking at viewer, closed eyes, gloves, outdoors, hat

Training info

Trained solely on 164 640x800 frameless Cinderella Girls Cards and 273 1280x824 Starlight Stage Cards collected from アイマス シンデレラガールズ カードギャラリー without balancing and tag cleaning. Near-duplicates were found (minor differences like closed/open eyes), but they were not cleaned. As the jpgs are highly compressed, they are passed through RealESRGAN_x4plus_anime_6B with --outscale=1 in order to remove the artifacts. Tags generated with SmilingWolf/wd-v1-4-swinv2-tagger-v2 with a threshold of 0.35. Caption template was "[char_name], [filewords]".

Originally trained with LoRA on convolution blocks ("LoCon"), but it was found that removing the convolution weights from the output doesn't alter the results significantly. The original is available as v1-full

and the conv-pruned version is v1-linear

Training cost: ~4.5 T4-hours

Data distribution:

Character    Total    640     1280
---------------------------------
櫻井桃華        60      36      24
市原仁奈        56      38      18
橘ありす        54      34      20
赤城みりあ      53      37      16
佐々木千枝      52      30      22
龍崎薫          50      34      16
結城晴          44      26      18
的場梨沙        40      22      18
古賀小春        28      16      12

LoRA meta dump:

{'ss_cache_latents': 'False',
 'ss_caption_dropout_every_n_epochs': '0',
 'ss_caption_dropout_rate': '0.0',
 'ss_caption_tag_dropout_rate': '0.0',
 'ss_clip_skip': '2',
 'ss_dataset_dirs': '{"meta_lat.json": {"n_repeats": 1, "img_count": 437}}',
 'ss_datasets': '[{"is_dreambooth": false, "batch_size_per_device": 3, "num_train_images": 437, "num_reg_images": 0, "resolution": [768, 768], "enable_bucket": true, "min_bucket_reso": null, "max_bucket_reso": null, "tag_frequency": {"meta_lat.json": {}}, "bucket_info": {"buckets": {"0": {"resolution": [640, 768], "count": 273}, "1": {"resolution": [896, 576], "count": 164}}, "mean_img_ar_error": 0.0}, "subsets": [{"img_count": 437, "num_repeats": 1, "color_aug": false, "flip_aug": false, "random_crop": false, "shuffle_caption": true, "keep_tokens": 0, "image_dir": "train_data", "metadata_file": "meta_lat.json"}]}]',
 'ss_epoch': '45',
 'ss_face_crop_aug_range': 'None',
 'ss_full_fp16': 'False',
 'ss_gradient_accumulation_steps': '1',
 'ss_gradient_checkpointing': 'False',
 'ss_learning_rate': '0.0001',
 'ss_lowram': 'True',
 'ss_lr_scheduler': 'polynomial',
 'ss_lr_warmup_steps': '100',
 'ss_max_grad_norm': '1.0',
 'ss_max_token_length': '225',
 'ss_max_train_steps': '7300',
 'ss_mixed_precision': 'fp16',
 'ss_network_alpha': '16',
 'ss_network_dim': '16',
 'ss_network_module': 'lycoris.kohya',
 'ss_new_sd_model_hash': '89d59c3dde4c56c6d5c41da34cc55ce479d93b4007046980934b14db71bdb2a8',
 'ss_noise_offset': 'None',
 'ss_num_batches_per_epoch': '146',
 'ss_num_epochs': '50',
 'ss_num_reg_images': '0',
 'ss_num_train_images': '437',
 'ss_optimizer': 'bitsandbytes.optim.adamw.AdamW8bit',
 'ss_output_name': 'u149',
 'ss_prior_loss_weight': '1.0',
 'ss_sd_model_hash': '925997e9',
 'ss_sd_model_name': 'Animefull-final-pruned.ckpt',
 'ss_sd_scripts_commit_hash': '5b506b14a3f0cda56c0012bb502590e785f5ff21',
 'ss_seed': 'None',
 'ss_session_id': '505064346',
 'ss_tag_frequency': '{"meta_lat.json": {}}',
 'ss_text_encoder_lr': '5e-05',
 'ss_training_comment': 'None',
 'ss_training_finished_at': '1678689998.9046714',
 'ss_training_started_at': '1678673431.1014886',
 'ss_unet_lr': '0.0001',
 'ss_v2': 'False',
 'sshs_legacy_hash': '65cdfd7c',
 'sshs_model_hash': '6a126d6984ae0903678f7a149e18a0b2ce34bf6e9ee4bb10311c3f1b82ea2901'}