SkillPE: Cinematic Prompt Framework for Text-to-Video

What happened

To address the problem that prompts from non-expert users often lack professional narrative and creative design when generating cinematic videos, researchers proposed the SkillPE prompt engineering framework. It evolves reusable filmmaking skills from expert-written seed prompts, with fine-grained representation of creative professional elements including shot logic, composition, lighting, and sound design. The framework optimizes skills by matching and classifying reference cinematic materials, and builds a skill library by generating candidate videos across four dimensions: prompt fidelity, cinematic quality, narrative appeal, and creativity. The related code has been open-sourced.

Key facts

Framework Type

Creativity-oriented text-to-video prompt engineering framework

Evaluation Benchmarks

StoryEval, VBench

Improvement Over Strongest External Baseline

Up to 1.40 point gain on a 7-point scale across four dimensions

Improvement Over Initial Seed Skills

Up to 0.51 point gain on a 7-point scale across four dimensions

Code Repository

https://github.com/Ais0n/SkillPE

Background

Currently, generating high-quality cinematic footage from text remains a high-bar task. Prompts written by non-professional users often lack professional narrative and creative design elements, making it difficult to consistently produce creative video content that meets expectations.

Why it matters

For the text-to-video generation industry, this framework lowers the barrier to creating professional-grade cinematic videos, and provides a viable path for skill evolution in prompt engineering that balances fidelity and creativity. For developers, the open-source framework can be directly integrated into existing text-to-video workflows to quickly optimize generation results. For general users, it enables creating narrative, creative video content via prompts without mastering professional filmmaking knowledge.

What to watch

Future attention can be paid to the framework's adaptation performance and real-world implementation across more mainstream text-to-video large models.