Auto reframe: horizontal to vertical video with AI speaker tracking
First video free up to 60 min, then 20 min a month, watermarked, no card. No watermark from $12/month, or self-host free under MIT.
No link? Upload the file in the app instead. Sources need at least 45 seconds of material.
A 16:9 frame is 1.78 times wider than it is tall; a 9:16 one is 0.56. Converting landscape to portrait throws away about two thirds of the width, and the only question is which two thirds. A static crop answers once for the whole video. Auto reframe answers scene by scene, and here it answers with more than one shape: a tracked crop for one speaker, two stacked halves for two, a full-width screen with the presenter under it.
Paste a long horizontal video and you get vertical clips back with the framing already decided, plus an editor to overrule it on the scenes where it chose wrong.
Free plan: your first video clipped whole up to 60 minutes, then 20 minutes a month, with a watermark and no credit card. Paid plans start at $12/month with no watermark, and the self-hosted edition is free under MIT.
How auto reframe works here
1. Paste a link or upload the video
Any horizontal recording works: an interview, a podcast with two cameras, a screen tutorial, a stream with a facecam. Paste a public link or upload the file; it needs at least 45 seconds of material.
2. AI cuts the clips and decides the frame
The best moments are picked from the transcript, then each clip is reframed scene by scene: faces are tracked, two people in one shot are stacked, screens are kept readable and a corner webcam is enlarged.
3. Check, adjust, export
Every clip comes back 9:16 with captions. Where the AI framed a scene wrong, draw the crop yourself for that scene in the reframe editor, then download or post to TikTok, YouTube Shorts and Instagram.
Auto reframe versus a static crop
There are two ways to make a horizontal video vertical. The first is a fixed treatment: keep a vertical slice at one position, or shrink the whole picture into the middle over a blurred copy or black bars. That is what the free video to 9:16 converter does, in your browser, with no upload. It is the right tool for a short clip where the subject stays in one place, a slide deck, or a wide shot you want to keep whole.
The second is to look at the picture. A person who walks across the stage, two people taking turns, a presenter who shares a screen halfway through: each of those needs the frame to change with the content, and no single crop position can do it. Auto reframe does the looking. The difference shows up most on exactly the footage people most want to clip, interviews and podcasts, where a fixed centre crop either cuts the guest out or shows an empty chair. Church services are the other common case: a preacher who walks the stage through a whole sermon, which the sermon clip maker handles with the same tracking.
Real OpenShorts output, not a mock-up. A two-person podcast: both speakers stacked in half-frames, captions on the seam where they cover nobody. Source: Turn the Tables with Dan and Shoshana Jordan, Heritage of Faith, CC BY.
The layouts, and when each one is used
The layout is decided per scene, not once per video, because one video mixes shots: an edited podcast cuts between close-ups and a wide two-shot, a tutorial cuts between the camera and the screen.
Tracked crop (one speaker)
A single subject is followed with MediaPipe face detection, with a YOLOv8 person detector as the fallback when the face is lost. The camera is damped by a stabiliser that holds still while the subject stays inside a safe zone and only moves when they leave it, so the clip looks operated rather than jittery. A detection that jumps further than the safe zone in one frame has to repeat before the camera follows, which stops a passing face from yanking the frame.
Stacked split (two people in one shot)
When two faces are visible in the same frame for at least half of the sampled frames, the scene is stacked, one person per half. That condition is what separates a real two-shot from shot and reverse shot, where stacking would show the same person twice. The captions on a stacked stretch sit on the seam between the halves, where they cover nobody.
Screen and presenter
For tutorials and demos each shot is classified from one still: a screen, a screen beside the presenter, or a camera shot. A full screen keeps its width, cropped to the area actually being read; a presenter beside their content is stacked under the content columns; a webcam window floating over the screen is boxed and enlarged below it.
Wide content
When what matters spans more of the frame than any crop can keep (a group on a stage, a map, a chase), the scene keeps its full width over a blurred backdrop instead of being cropped to a random strip.
Facecam inset
A stream or recording with a webcam composited into a corner is detected by geometry: a subject that is small, off centre and still. The clip renders the main picture across the top and the webcam enlarged below.
Speaker cuts and punch-ins
Two more options are available on the API (layouts:
speaker_cut, punch_in): hard cuts to whoever is
talking, decided from mouth activity normalised per speaker, and a small push
in on the beats of the audio.
The layout choice itself comes from twelve stills of the video, sent to Gemini at 1024 pixels with a closed question, rather than the whole video: on a hand-labelled set of 48 clips that gets the layout right 92 to 96% of the time, and costs the same whether the source is ten minutes or three hours.
When the AI frames a scene wrong
Automatic framing is right most of the time and occasionally wrong in a way nobody would accept: the crop on the interviewer while the guest talks, a face in the audience instead of on stage. The reframe editor fixes that scene by scene. Each scene plays with the uncropped picture and its crop rectangle over it, so you can judge it with motion and sound; drag the rectangle, or split it into two stacked regions, and only the scenes you touched are re-rendered. The rest stay automatic.
Sources that are already vertical, and output size
A source shot vertical is passed through untouched: every layout above exists to win back width that a crop throws away, and a vertical video has none to give. Horizontal sources are delivered at least 1080 pixels wide, so a 1080p recording becomes 1080x1920. A smaller crop is scaled up to that floor; a larger source is never scaled down.
Next to the auto reframe in video editors
Desktop and mobile editors such as Premiere Pro and CapCut have an auto reframe feature too, and it is good at what it does: it reframes a sequence you have already cut, inside your edit. The difference here is where it sits. Reframing is one stage of turning a long recording into clips, after the moments are picked and before the captions are burned, and it can change the shape of the frame (stacked halves, screen over presenter), not only the crop position. If you already have a finished horizontal edit and only need it vertical, an editor's auto reframe, or the free converter for short files, is the shorter path.
Free tier compared
Competitor plans checked 5 October 2026 on their public pricing pages.
| Tool | Free plan | Paid from |
|---|---|---|
| OpenShorts Cloud | First video up to 60 min, then 20 min/month. Watermark, no credit card | $12/month, 100 min, no watermark |
| OpenShorts self-hosted | Free under MIT. No watermark, no cap; your hardware and API keys | $0 |
| 2short.ai | 30 min/mo. No watermark, YouTube links only | $9.90/month |
| Klap | 1 video. To try it, no ongoing free tier | $14/month billed yearly |
| Opus Clip | 60 min/mo. Watermarked, export within 3 days | $15/month |
| Submagic | None. A trial with no credit card | $12/month billed yearly |
Every tool here reframes with face tracking. The differences are the layouts beyond a single crop: Klap lists split-screen, screencast and gaming layouts, 2short.ai keeps one centred speaker, Submagic is a caption editor first.
What it does not do
- It does not reframe a whole long video into one vertical file. It reframes the clips it cuts, each up to 180 seconds.
- It does not output square or 4:5. The target is 9:16.
- It does not take sources under 45 seconds. For a short file, use the free converter.
- It does not track objects or animals as subjects: tracking is for faces and people.
- It cannot invent picture. When the content is wider than any crop, it keeps the width over a blurred backdrop rather than guessing.
- Free clips carry a watermark. A paid plan removes it, also from clips already made.
Common questions
- What does auto reframe mean?
- Converting a horizontal video to a vertical (or other) aspect ratio by moving the crop to follow what matters in the shot, usually a face, instead of cutting a fixed slice out of the middle. Here it is one stage of turning a long video into vertical clips.
- Is the auto reframe free?
- On the hosted free plan your first video is clipped whole up to 60 minutes, then 20 minutes a month, with a watermark and no credit card; paid plans start at $12/month with no watermark. The self-hosted edition is free under MIT. For a quick static crop of a short file, the free video to 9:16 converter runs in your browser with no account and no watermark.
- How is it different from the free 9:16 converter?
- The converter applies one fixed treatment to the whole file (crop at a position you choose, blurred background or black bars) and never looks at what is in the frame. Auto reframe tracks faces, decides a layout per scene and cuts the video into clips. Use the converter for a short clip you have already cut; use auto reframe for a long recording with people in it.
- Can it reframe a whole video into one vertical video?
- No. It reframes the clips it cuts, each up to 180 seconds when you raise the maximum. To convert a whole short video in one piece with a fixed crop, use the free 9:16 converter, which takes up to 5 minutes in the browser.
- What happens with two people in the shot?
- When both faces are visible in the same frame for at least half of the sampled frames, the scene is stacked: one person in each half of the vertical frame, with the captions on the seam between them. Shot and reverse shot (one person at a time) stays a single tracked crop, so nobody appears twice.
- Does it work on screen recordings?
- Yes. With the layout on auto or screencast, each shot is classified as screen, screen beside a presenter, or camera. A screen keeps its full width, cropped to the area being read; a presenter beside the screen is stacked under the content; a webcam floating over the screen is enlarged below it.
- What if my video is already vertical?
- It is passed through untouched. Every layout exists to win back width a crop throws away, and a vertical source has none to throw away.
- Can I correct the framing?
- Yes. The reframe editor plays each scene with the uncropped picture and the crop rectangle over it, so you can move the rectangle or split it into two stacked regions. Scenes you do not touch stay automatic.
- What resolution is the output?
- At least 1080 pixels wide: a 1080p source gives 1080x1920. A smaller crop is scaled up to that floor; a larger source is never scaled down.
- How does it compare with the auto reframe in Opus Clip or 2short.ai?
- Both face-track to 9:16. 2short.ai keeps the speaker centred with one tracked crop and has an unwatermarked free plan of 30 minutes, YouTube links only (checked 5 October 2026). OpenShorts adds stacked two-speaker, screencast and webcam-inset layouts, a per-scene manual override, and an open source, self-hostable edition.
Sources
- Layout picker accuracy (12 stills at 1024 px, 92 to 96% on a hand-labelled 48-clip set), split and screencast rules, and the vertical pass-through:
layout_picker.py,split_layout.py,screencast_layout.py,reframe_v2.pyat github.com/mutonby/openshorts. - Competitor plans: our comparison data, checked 5 October 2026. See 2short AI.