Be part of our each day and weekly newsletters for the newest updates and unique content material on industry-leading AI protection. Be taught Extra
ByteDance researchers have developed an AI system that transforms single pictures into practical movies of individuals talking, singing and transferring naturally — a breakthrough that might reshape digital leisure and communications.
The brand new system, referred to as OmniHuman, generates full-body movies that present individuals gesturing and transferring in ways in which match their speech, surpassing earlier AI fashions that might solely animate faces or higher our bodies.
How OmniHuman makes use of 18,700 hours of coaching information to create practical movement
“Finish-to-end human animation has undergone notable developments lately,” the ByteDance researchers wrote in a paper printed on arXiv. “Nevertheless, present strategies nonetheless battle to scale up as giant basic video technology fashions, limiting their potential in actual functions,”
The workforce educated OmniHuman on greater than 18,700 hours of human video information utilizing a novel strategy that mixes a number of sorts of inputs — textual content, audio and physique actions. This “omni-conditions” coaching technique permits the AI to study from a lot bigger and extra various datasets than earlier strategies.
AI video technology breakthrough reveals full-body motion and pure gestures
“Our key perception is that incorporating a number of conditioning indicators, similar to textual content, audio and pose, throughout coaching can considerably cut back information wastage,” the analysis workforce defined.
The expertise marks a major advance in AI-generated media, demonstrating capabilities that vary from creating movies of individuals delivering speeches to depicting topics enjoying musical devices. In testing, OmniHuman outperformed present programs throughout a number of high quality benchmarks.
Tech giants race to develop next-generation video AI programs
The event emerges amid intensifying competitors in AI video technology, with firms like Google, Meta and Microsoft pursuing comparable applied sciences. ByteDance’s breakthrough may give its TikTok father or mother firm a bonus on this quickly evolving area.
Trade consultants say such expertise may remodel leisure manufacturing, academic content material creation and digital communications. Nevertheless, it additionally raises considerations about potential misuse in creating artificial media for misleading functions.
The researchers will current their findings at an upcoming pc imaginative and prescient convention, though they haven’t but specified when or which one.
