SW Agent runs models on your device

The workspaces that use an AI model run it on your own device, and SW Agent can now run each of them for you from plain words: write and time the words of a song; split a song into vocals, drums, bass and other; translate lyrics, each line keeping its time; take a picture's background out; and find how far away each part of a picture is, for Generate visualizer. It starts the same job the workspace's own button starts, so it works the same way and gives the same result.

Attach the song or the picture (the paperclip, or drop it in) and ask, for example "write the lyrics for this song", "split the vocals out", "take the background out of this photo" or "translate these lyrics to Spanish".

• Write and time the words of a song (transcribe_lyrics): Fast (tiny) at 41 MB, Balanced (base) at 77 MB and Accurate (small) at 249 MB.
• Split a song into vocals, drums, bass and other (split_stems): Vocals at 166 MB, Drums at 166 MB, Bass at 166 MB and Other at 166 MB.
• Translate lyrics, each line keeping its time (translate_lyrics): English to Spanish at 80 MB, Spanish to English at 80 MB, English to French at 80 MB, French to English at 80 MB, English to German at 80 MB, German to English at 80 MB, English to Chinese at 80 MB, Chinese to English at 80 MB, Japanese to English at 80 MB, English to Vietnamese at 80 MB and Vietnamese to English at 80 MB.
• Take a picture's background out (remove_background): People (MODNet) at 7 MB, Any subject (ORMBG) at 45 MB and Best quality (BiRefNet Lite) at 99 MB.
• Find how far away each part of a picture is, for Generate visualizer (picture_depth): Find the depth at 27 MB.

Before a model's first run SW Agent asks, with its size: "Write and time the lyrics of Night Drive? This downloads the 77 MB lyrics model once and keeps it on this device; your song never leaves it." Press Download and run, choose the smaller or the larger model where there is one (41 MB or 249 MB for lyrics), or press Not now, which downloads nothing.

A model that is already on your device asks only to run. Nothing downloads until you press the button, and when your browser says it is on a metered connection or saving data, the ask says so too.

Some jobs use more than one model: isolating the vocals before writing the lyrics also uses the 166 MB vocal stem model, and the best four-stem split uses all four stem models. The ask names the total.

The job's progress shows under the reply, and Stop stops it at once: a stopped run changes nothing and claims nothing.

What it made comes back as files in the chat (the lyrics as SRT, LRC and WebVTT, each stem as a WAV, the cut-out as a PNG, the depth as a picture) and in the workspace: in Classic the lyrics go straight into the project's captions (Undo takes them back); otherwise your song waits in Lyric videos or the Stem Splitter, your picture in the Image editor, and the look in Generate visualizer. The next step is offered with them, such as translating the lyrics or writing the lyrics from the vocals.

Make captions (make_captions) and Send to the Image editor's cut-out ask the same way before their models download.

SW Agent reads what your device reports before it asks. Where a model cannot run, it says why and offers the smaller model that can: on a phone the 249 MB lyrics model is offered as the 77 MB one.

The stem models run on a graphics card the browser can use with 16-bit shader support, as the Stem Splitter needs; without one SW Agent says so and downloads nothing, since a processor version would take hours. Background removal chooses the model your device can run, as the Image editor does.

Every model is downloaded once from its pinned source and kept in your browser, so the next run starts at once. 22 models can be kept this way. Settings lists the ones kept on this device, each with its size and a Remove button, under Preferences, Downloaded models (and in SW Agent's settings); a removed model downloads again the next time you use it.

Ask "what models can you run" (list_models) for every model with its size, whether it is kept here and whether this device can run it, beside every workspace's settings, actions and models that SW Agent can reach for you.

• Downloaded models in Settings
• Open SW Agent

Your songs, pictures and lyrics stay on your device: the models run in your browser, and nothing you give them is uploaded. The jobs are transcribe_lyrics, split_stems, translate_lyrics, remove_background and picture_depth, each listed in SW Agent's Functions panel.