XTTSv2 model size growth when finetuning/training.

[erew123]

When finetuning a model, say the Xttsv2 2.0.3 model, it goes from being 1.8GB in size to something closer to 5GB's. Is there a way to compact this down again, or is there a setting I am missing?

I can see people asking this question #2678

and possible references to code with other model types #1890 (comment)

but I cant find either a block of working code or a solid answer anywhere.

I knocked together and tried the following. It does remove the discriminator, but.........

import torch
from TTS.tts.configs.xtts_config import XttsConfig
from TTS.tts.models.xtts import Xtts

# Load the XTTS model
config = XttsConfig()
config.load_json("/path/to/xtts/config.json")
model = Xtts.init_from_config(config)
model.load_checkpoint(config, checkpoint_dir="/path/to/xtts/", use_deepspeed=True)
model.cuda()

# Remove the discriminator part
model.disc = None

# Save the generator-only model
save_checkpoint = "/path/to/xtts/model_without_discriminator.pth"
torch.save(model.state_dict(), save_checkpoint)

........it looks like the finetuning process changes the keys e.g. "gpt.gpt.h.0.ln_1.weight", "gpt.gpt.h.0.ln_1.bias", etc., while the keys in the checkpoint include keys like "gpt.gpt_inference.transformer.h.0.norm_w", "gpt.gpt_inference.transformer.h.0.norm_b"

How do Coqui do it when they fine tune their models? Could you share any code or suggestions?

Thanks

[HobisPL]

If you want to reduce the weight from 5GB to 1.8GB.
You need to remove it from the model optimizer.

import os
import glob
import torch

def find_latest_best_model(folder_path):
    search_path = "/content/drive/MyDrive/tts/GPT_XTTS_FT-December-27-2023_03+32PM-0000000/checkpoint_9000.pth"
    files = glob.glob(search_path, recursive=True)
    latest_file = max(files, key=os.path.getctime, default=None)
    return latest_file

model_path = find_latest_best_model("/content/drive/MyDrive/tts")
checkpoint = torch.load(model_path, map_location=torch.device("cpu"))
del checkpoint["optimizer"]
for key in list(checkpoint["model"].keys()):
    if "dvae" in key:
        del checkpoint["model"][key]
torch.save(checkpoint, "model.pth")
model_dir = os.path.dirname(model_path)

[erew123]

If I could give you 2 thumbs up or 5 stars, I would! Thats perfect! Thankyou so much!! :)