Modify Video Overview
What is Modify Video?
Luma's new Modify Video tool lets you take a real piece of footage, live action, animation, performance, or even something shot on your phone, and completely transform it into something new without losing the original motion or camera dynamics.
Available with all paid Dream Machine Subscriptions on web. iOS coming soon!
What can I achieve with Modify Video?
Preserve full-body motion and facial performance, including choreography, lip sync, and nuanced expression
Restyle or retexture the entire shot, from turning live-action into CG or stylized animation, to changing wardrobe, props, or the overall aesthetic
Swap environments or time periods, giving you control over background, location, or even weather
Edit at the element level, like isolating just the outfit, face, or prop — or adding generative FX (smoke, fire, water, etc.) on top
What do “Strength Levels” do?
The "Strength Level" setting controls how closely the modified video adheres to the source material.
a) Adhere — The output adheres very closely to the source video. This is ideal for subtle enhancements, minor retexturing, or applying a light stylistic filter without drastically altering the original content.
b) Flex — The output flexibly adheres somewhat to the shapes, characters, and details of the source video. This range allows for more significant stylistic changes while still maintaining recognizable elements from the original footage.
c) Reimagine — The output adheres much more loosely to the source video. This setting is best for fundamentally changing the world and style to something entirely new, even transforming the characters or objects into completely different forms.
What's the maximum length for videos uploaded to Modify Video?
Currently, the maximum length for videos uploaded to Modify Video is 10 seconds. Video length created up to 30s.
Does Modify Video work in relaxed mode?
Yes, Modify Video does work in Relaxed Mode. However, please be aware that processing times will be significantly longer compared to using it in Fast mode.
If I modify a video will it keep the audio?
Yes, Modified videos retain their original audio. You can also add new audio by clicking on a generation and choosing Audio.
Which formats are compatible with Modify Video?
Modify Video (v2v) is compatible with a wide range of common video formats, including but not limited to: .mp4, .mov, and .wmv. For the best results, we recommend using high-quality source footage.
How many credits do Modify Video use?
As a general guideline, a typical 5-second uses 400 credits, a 10-second Modify Video uses 800 credits.
What kind of source footage works best with Modify Video?
Clear and in-focus: Sharp details allow the AI to perform more precise edits, especially when you're aiming for element-level modifications like changing an outfit or a prop.
With clear subject separation (if applicable): If you intend to isolate and modify a specific subject, having good contrast between the subject and the background can improve the results. Simple backgrounds are often easier for the AI to work with.
With intentional background setup: When planning to modify just one part of the frame—like replacing the background while keeping the subject consistent—it helps to film against a clean, minimal backdrop. Shooting on a blank wall, greenscreen, or cycloramic setup provides better separation and leads to more accurate results. In these cases, traditional filmmaking techniques still play a key role in setting up footage for successful modification.
Which model is currently available with Modify Video?
Currently, the Ray2 model is available for use with Modify Video.
Are there any limitations to what Modify Video can do?
Yes, like any advanced AI tool, Modify Video has certain limitations, and the quality of your source footage significantly impacts the results. You might encounter challenges with:
Excessive Motion or Instability: Footage with very high dynamic motion, extreme shakiness, or chaotic camera work (e.g., handheld in a very active scene without stabilization) can make it difficult for the AI to maintain consistency and track elements accurately. This can lead to artifacts, warping, or a loss of detail.
Blurriness or Low Resolution: Blurry, out-of-focus, or very low-resolution source videos provide less information for the AI to work with. This can result in less precise transformations, a loss of fine details, or a "muddy" appearance in the modified output.
Fast Action or Busy Scenes: Chaotic scenes like fight sequences with multiple fast-moving actors can sometimes lead to motion blur artifacts, "phantom limbs," merging figures, or lost subjects during rapid transitions. It often works best with one or two key moving elements.
Does the input video and reference image need to have matching aspect ratios?
No, your input video and reference image do not need to have matching aspect ratios. For example, your uploaded footage can be 9:16 (vertical), while your reference image is 1:1 (square).
Important: You have control over the final output! Make sure to select your desired output aspect ratio in the settings within the prompt box before processing. This allows you to tailor the final video to your intended platform or display.
How do I get started with Modify Video?
Getting started with Modify Video is quick and intuitive!
Follow these steps:
Access the Modify Tab: Within the main prompt box, click on the "Modify" Tab.
Select Output Resolution: Choose your desired output resolution by selecting either 720p or 1080p.
Upload Your Video: Press the "+" box icon located above the prompt input area. This will allow you to upload your source video footage.
Add Your Creative Instructions:
Option A (Prompt only): Enter your descriptive prompt in the text box detailing the desired transformation. Then, simply click "Generate".
Option B (Prompt + Reference Image): After uploading your video, a new "+" box icon will appear. Click this to upload a reference image that further guides the visual style or content of your modification.
Once your reference image is uploaded, enter your prompt and then click "Generate".
What if I want to change only a small part of the video?
Modify Video allows you to target specific elements within your footage through precise natural language prompting; simply describe what you want to change in your positive prompt. The AI is designed to understand and isolate these elements while maintaining the rest of the scene.
Remember to use positive prompting – describe what you want to see, rather than what you don't want.
Pro Tip: You can use external tools like Photoshop to prep your first frame before bringing it into Dream Machine. For example, if you want to swap out or modify a specific element, try using inpainting or manual edits to make that change in the still frame—then use that edited image as your starting point in Modify Video for more controlled results.
This can sometimes lead to a better result than prompting a change in Dream Machine.
Here are some examples of how to achieve element-level changes:
Changing an outfit:
Prompt: "A person wearing a sleek futuristic silver jumpsuit"
Original Video: Shows a person in a regular t-shirt and jeans.
Result: The person's clothes are transformed into a silver jumpsuit, while their motion and the background remain consistent.
Modifying a prop:
Prompt: "A classic vintage red car, on a busy street"
Original Video: Features a modern white sedan driving down a street.
Result: The white sedan is restyled into a classic vintage red car, with the street and surrounding environment largely unchanged.
Adding generative effects to a specific area:
Prompt: "A dragon breathing fire over a medieval castle"
Original Video: Shows a dragon flying near a castle.
Result: Fire is generated from the dragon's mouth, aimed towards the castle, integrating naturally into the scene.
Altering facial features or expressions subtly:
Prompt: "A person with a soft smile and glowing eyes"
Original Video: Shows a person with a neutral expression.
Result: The person's expression subtly shifts to a smile, and their eyes have a gentle glow, maintaining their identity.
Changing only the background/environment:
Prompt: "A busy city street at night with neon lights and rain"
Original Video: Shows a person walking down a quiet street during the day.
Result: The street behind the person transforms into a bustling night scene with neon lights and rain, while the person's appearance and motion are preserved.
By being specific and descriptive in your prompts, you can guide Modify Video to make precise, localized changes without affecting the entire shot.