Modify Video Overview
What is Modify Video?
Luma's new Modify Video tool lets you take a real piece of footage, live action, animation, performance, or even something shot on your phone, and completely transform it into something new without losing the original motion or camera dynamics.
Available with all paid Dream Machine Subscriptions on web. iOS coming soon!
Feature Info
Included with → All paid Dream Machine subscriptions
Available on → Web. (rolling out on iOS soon!)
Output formats → 720p and 1080p video
Max video length → 10 seconds upload, video length up to 30s
Model → Ray2
Relaxed Mode → Available with longer processing times
Web & iOS → 5-second video = around 400 credits
10-second video = around 800 credits
Can't wait to try? Head over to our How To Section and check out our Modify Video Tutorials!
What can I achieve with Modify Video?
Preserve full-body motion and facial performance, including choreography, lip sync, and nuanced expression
Restyle or retexture the entire shot, from turning live-action into CG or stylized animation, to changing wardrobe, props, or the overall aesthetic
Swap environments or time periods, giving you control over background, location, or even weather
Edit at the element level, like isolating just the outfit, face, or prop — or adding generative FX (smoke, fire, water, etc.) on top
How do I get started with Modify Video?
Getting started with Modify Video is quick and intuitive!
Follow these steps:
Access the Modify Tab:
Within the main prompt box, click on the "Modify" Tab.Select Output Resolution:
Choose your desired output resolution by selecting either 720p or 1080p.Upload Your Video:
Press the "+" box icon located above the prompt input area. This will allow you to upload your source video footage.Add Your Creative Instructions:
Option A (Prompt only):
Enter your descriptive prompt in the text box detailing the desired transformation. Then, simply click "Generate".Option B (Prompt + Reference Image):
After uploading your video, a new "+" box icon will appear. Click this to upload a reference image that further guides the visual style or content of your modification.
Once your reference image is uploaded, enter your prompt and then click "Generate".
What are Style and Start Frame?
Modify Video lets you specify the image as either a Style or a Start Frame.
If set to Start Frame, the video will follow the motion of the source video, extending from the frame you provide.
If set to Style, the image will be used purely as a Style Reference, not as the literal first frame of the video.
However, you can click Preview to view a range of start frames generated from your style reference.
What do Strength Levels do?
The "Strength Level" setting controls how closely the modified video adheres to the source material.
a) Adhere — The output adheres very closely to the source video. This is ideal for subtle enhancements, minor retexturing, or applying a light stylistic filter without drastically altering the original content.
b) Flex — The output flexibly adheres somewhat to the shapes, characters, and details of the source video. This range allows for more significant stylistic changes while still maintaining recognizable elements from the original footage.
c) Reimagine — The output adheres much more loosely to the source video. This setting is best for fundamentally changing the world and style to something entirely new, even transforming the characters or objects into completely different forms.
What's the max length for videos uploaded to Modify Video?
Currently, the maximum length for videos uploaded to Modify Video is 10 seconds. Video length created up to 30s.
Does Modify Video work in relaxed mode?
Yes, Modify Video does work in Relaxed Mode. However, please be aware that processing times will be significantly longer compared to using it in Fast mode.
If I modify a video will it keep the audio?
Yes, Modified videos retain their original audio. You can also add new audio by clicking on a generation and choosing Audio.
Which formats are compatible with Modify Video?
Modify Video (v2v) is compatible with a wide range of common video formats, including but not limited to: .mp4, .mov, and .wmv. For the best results, we recommend using high-quality source footage.
How many credits do Modify Video use?
As a general guideline, a typical 5-second uses 400 credits, a 10-second Modify Video uses 800 credits.
What kind of source footage works best with Modify Video?
Clear and in-focus: Sharp details allow the AI to perform more precise edits, especially when you're aiming for element-level modifications like changing an outfit or a prop.
With clear subject separation (if applicable): If you intend to isolate and modify a specific subject, having good contrast between the subject and the background can improve the results. Simple backgrounds are often easier for the AI to work with.
With intentional background setup: When planning to modify just one part of the frame—like replacing the background while keeping the subject consistent—it helps to film against a clean, minimal backdrop. Shooting on a blank wall, greenscreen, or cycloramic setup provides better separation and leads to more accurate results. In these cases, traditional filmmaking techniques still play a key role in setting up footage for successful modification.
Which model is currently available with Modify Video?
Currently, the Ray2 model is available for use with Modify Video.
Are there any limitations to what Modify Video can do?
Yes, like any advanced AI tool, Modify Video has certain limitations, and the quality of your source footage significantly impacts the results. You might encounter challenges with:
Excessive Motion or Instability: Footage with very high dynamic motion, extreme shakiness, or chaotic camera work (e.g., handheld in a very active scene without stabilization) can make it difficult for the AI to maintain consistency and track elements accurately. This can lead to artifacts, warping, or a loss of detail.
Blurriness or Low Resolution: Blurry, out-of-focus, or very low-resolution source videos provide less information for the AI to work with. This can result in less precise transformations, a loss of fine details, or a "muddy" appearance in the modified output.
Fast Action or Busy Scenes: Chaotic scenes like fight sequences with multiple fast-moving actors can sometimes lead to motion blur artifacts, "phantom limbs," merging figures, or lost subjects during rapid transitions. It often works best with one or two key moving elements.
Does the input video and reference image need to have matching aspect ratios?
Your input video and reference image do not need to have matching aspect ratios. For example, your uploaded footage can be 9:16 (vertical), while your reference image is 1:1 (square).
IMPORTANT
You can use any aspect ratio you like, though, we recommend keeping your style reference image the same as your input video.
You have control over the final output. Make sure to select your desired output aspect ratio in the settings within the prompt box before processing. This allows you to tailor the final video to your intended platform or display.
What if I want to change only a small part of the video?
Modify Video allows you to target specific elements within your footage through precise natural language prompting; simply describe what you want to change in your positive prompt. The AI is designed to understand and isolate these elements while maintaining the rest of the scene.
Remember to use positive prompting – describe what you want to see, rather than what you don't want.
Pro Tip: You can use external tools like Photoshop to prep your first frame before bringing it into Dream Machine. For example, if you want to swap out or modify a specific element, try using inpainting or manual edits to make that change in the still frame—then use that edited image as your starting point in Modify Video for more controlled results.
This can sometimes lead to a better result than prompting a change in Dream Machine.
NOTE: By being specific and descriptive in your prompts, you can guide Modify Video to make precise, localized changes without affecting the entire shot.