0 Fork 0

Files

T

lucianoandClaude Fable 5 21ad969b10 Initial commit: InTono — app KMP per l'apprendimento musicale

App Kotlin Multiplatform (Android/iOS/Desktop) con Compose Multiplatform:
controllo intonazione (YIN), pratica guidata, metronomo, ear training,
sfide, scansione spartiti via AI, spartiti bundled e pentagramma con
glifi tipografici, travature, battute e spaziatura proporzionale.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-07-22 10:06:59 +02:00

3.2 KiB

Raw Blame History

name, description, metadata

name

description

metadata

voiceover

Adding AI-generated voiceover to Remotion compositions using ElevenLabs TTS

tags
voiceover, audio, elevenlabs, tts, speech, calculateMetadata, dynamic duration

Adding AI voiceover to a Remotion composition

Use ElevenLabs TTS to generate speech audio per scene, then use calculateMetadata to dynamically size the composition to match the audio.

Prerequisites

An ElevenLabs API key is required (ELEVENLABS_API_KEY environment variable).

MUST ask the user for their ElevenLabs API key if ELEVENLABS_API_KEY is not set. MUST NOT fall back to other TTS tools.

Ensure the environment variable is available when running the generation script:

node --strip-types generate-voiceover.ts

Generating audio with ElevenLabs

Create a script that reads the config, calls the ElevenLabs API for each scene, and writes MP3 files to the public/ directory so Remotion can access them via staticFile().

The core API call for a single scene:

const response = await fetch(
  `https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`,
  {
    method: "POST",
    headers: {
      "xi-api-key": process.env.ELEVENLABS_API_KEY!,
      "Content-Type": "application/json",
      Accept: "audio/mpeg",
    },
    body: JSON.stringify({
      text: "Welcome to the show.",
      model_id: "eleven_multilingual_v2",
      voice_settings: {
        stability: 0.5,
        similarity_boost: 0.75,
        style: 0.3,
      },
    }),
  },
);

const audioBuffer = Buffer.from(await response.arrayBuffer());
writeFileSync(`public/voiceover/${compositionId}/${scene.id}.mp3`, audioBuffer);

Dynamic composition duration with calculateMetadata

Use calculateMetadata to measure the audio durations and set the composition length accordingly.

import { CalculateMetadataFunction, staticFile } from "remotion";
import { getAudioDuration } from "./get-audio-duration";

const FPS = 30;

const SCENE_AUDIO_FILES = [
  "voiceover/my-comp/scene-01-intro.mp3",
  "voiceover/my-comp/scene-02-main.mp3",
  "voiceover/my-comp/scene-03-outro.mp3",
];

export const calculateMetadata: CalculateMetadataFunction<Props> = async ({
  props,
}) => {
  const durations = await Promise.all(
    SCENE_AUDIO_FILES.map((file) => getAudioDuration(staticFile(file))),
  );

  const sceneDurations = durations.map((durationInSeconds) => {
    return durationInSeconds * FPS;
  });

  return {
    durationInFrames: Math.ceil(sceneDurations.reduce((sum, d) => sum + d, 0)),
  };
};

The computed sceneDurations are passed into the component via a voiceover prop so the component knows how long each scene should be.

If the composition uses <TransitionSeries>, subtract the overlap from total duration: ./transitions.md#calculating-total-composition-duration

Rendering audio in the component

See audio.md for more information on how to render audio in the component.

Delaying audio start

See audio.md#delaying for more information on how to delay the audio start.

3.2 KiB Raw Blame History