Improving Generative Model Self-Training with Geometrically Modified Outputs

What happened

To address model collapse and model autophagy degradation that occur during generative model self-training amid scarce high-quality training data, the research team proposed the Geometrically Modified Outputs (GMO) method. The method reweights the singular values of the generator’s input-output Jacobian matrix to amplify the influence of dominant singular directions, strengthening the mode-seeking characteristics and distortion effects of standard generative outputs to provide stronger, more precise negative signals for negative-guidance self-training. Testing across multiple types of one-step generative models shows that GMO consistently improves the performance of negative guidance methods such as Neon and SIMS.

Key facts

Method Name

Geometrically Modified Outputs (GMO)

Technical Principle

Reweights singular values of the generator’s input-output Jacobian matrix to amplify the influence of dominant singular directions

Compatible Methods

Negative-guidance self-training methods including Neon and SIMS

Model Compatibility Range

Multiple categories of one-step generative models

Background

As high-quality training data becomes increasingly scarce, self-training approaches that let generative models iterate using their own outputs have grown in value, but directly fine-tuning on model-generated samples leads to degenerative issues including model collapse and autophagy disruption. Existing negative-guidance self-training methods use standard model outputs directly as negative signals, without targeted enhancement or optimization of these negative signals.

Why it matters

For generative model researchers and developers, this method eliminates the need for additional real-world annotated data collection: it enhances negative-guidance self-training performance solely via geometric modification of model outputs, reducing annotation costs for generative model iteration. For end users who rely on AI-generated content, improved self-training performance means existing generative models will deliver continuously better output quality without additional human annotation investment.

What to watch

Future work should track GMO’s extended adaptation performance on non-one-step generative architectures such as diffusion models.