Video Agent Loop Demo: Edited Keyframe Repair

Demo directory: examples/agnes-agent-loop-demo-chef-edited-keyframe-v1

核心链路： 用户 prompt 生成 V0 视频；observer 抽帧并输出约束 ledger；controller 决定修复；从 V0 选一张参考帧；Agnes Image 2.1 Flash 先把参考帧编辑正确；再用编辑后的关键帧作为 reference image 调用 Agnes Video V2.0 生成 V1。

Step 1. User Prompt -> Initial Video V0

Input prompt

A realistic cinematic video in a modern kitchen. One chef wearing a white jacket stands centered behind a wooden table. Gentle steam rises from a small pot in the background. Three red tomatoes are on the left side of the table and two blue bowls are on the right side.

V0 video

V0: text-to-video output reused from the previous run.

Sampled frames from V0

V0 frame 2, selected as the best reference frame

Step 2. Observer: Constraint Ledger for V0

Qwen observer 对 V0 抽帧逐帧打分，然后聚合为约束 ledger。

Constraint	Status	Score	Evidence
modern kitchen scene	Satisfied	1.0	Modern kitchen with stainless appliances and range hood.
exactly one chef	Satisfied	1.0	Exactly one chef is visible in sampled frames.
white chef jacket	Satisfied	1.0	The chef wears a white chef jacket.
chef centered behind table	Failed	0.6	Some frames place the chef slightly to the right of center.
steam rising	Satisfied	1.0	Steam rises from the pot in the background.
three red tomatoes on the left	Failed	0.0	Only two red tomatoes are visible.
two blue bowls on the right	Satisfied	1.0	Two blue bowls are visible on the right side.

Step 3. Controller: Repair Plan

Controller 决定继续修复，而不是 stop。它保留已满足约束，只修 chef 居中和三个番茄。

Controller diagnosis

The chef is not perfectly centered behind the table in all frames, and only two red tomatoes are visible on the left side instead of the required three.

Repair prompt for next video step

A realistic cinematic video in a modern kitchen. One chef wearing a white jacket stands exactly centered behind a wooden table. Gentle steam rises from a small pot in the background. Exactly three red tomatoes are clearly visible on the left side of the table, and exactly two blue bowls are visible on the right side. Make every required object visible in every frame. No extra objects. Keep counts, colors, side placement, and visibility stable.

Step 4. Image Editing: Repair the Keyframe First

为了避免把错误参考帧直接送入 I2V，系统先调用 Agnes Image 2.1 Flash 修复单张关键帧。修复目标是：保留场景、厨师、蒸汽、蓝碗，同时补出第三个红番茄并改善居中构图。

Reference frame before editing — Before image edit: selected V0 reference frame, only two tomatoes.

Edited reference keyframe — After image edit: edited keyframe, three tomatoes and centered chef.

Image edit instruction

Single-image edit only; do not create a video frame sequence. Edit this image minimally and preserve the chef, white jacket, kitchen, wooden table, lighting, camera, steam, and already-correct blue bowls. If the image currently shows only two red tomatoes, add one additional red tomato on the left side of the table so exactly three red tomatoes are clearly visible. Recenter the chef behind the wooden table while keeping the same chef and pose.

Step 5. Edited Keyframe -> Re-generated Video V1

Agnes Video V2.0 使用编辑后的关键帧作为 reference image 执行图生视频。这个阶段不再从纯文本重新生成，而是围绕修复后的视觉状态生成 V1。

V1: image-to-video output conditioned on the edited keyframe.

Sampled frames from V1

Step 6. Observer: Constraint Ledger for V1

V1 的核心修复成功：三个番茄、两个蓝碗、厨师居中、蒸汽都满足。

Constraint	Status	Score	Evidence
modern kitchen scene	Satisfied	1.0	Modern kitchen remains visible.
white chef jacket	Satisfied	1.0	The main chef wears a white jacket.
chef centered behind table	Satisfied	1.0	The main chef is centered behind the table.
steam rising	Satisfied	1.0	Steam remains visible from the pot.
three red tomatoes on the left	Satisfied	1.0	Exactly three red tomatoes appear on the left side.
two blue bowls on the right	Satisfied	1.0	Exactly two blue bowls appear on the right side.

Takeaway

This demo shows why a video agent loop should not rely only on prompt rewriting. The loop first repairs a keyframe with an image editing model, then uses the corrected keyframe as a visual anchor for image-to-video generation. The main repaired constraints succeeded, producing a compact edited-keyframe repair loop for the demo.