Think about a boombox that tracks your each transfer and suggests music to match your private dance fashion. That’s the concept behind “Be the Beat,” one in all a number of tasks from MIT course 4.043/4.044 (Interplay Intelligence), taught by Marcelo Coelho within the Division of Structure, that had been introduced on the thirty eighth annual NeurIPS (Neural Info Processing Techniques) convention in December 2024. With over 16,000 attendees converging in Vancouver, NeurIPS is a aggressive and prestigious convention devoted to analysis and science within the area of synthetic intelligence and machine studying, and a premier venue for showcasing cutting-edge developments.
The course investigates the rising area of massive language objects, and the way synthetic intelligence will be prolonged into the bodily world. Whereas “Be the Beat” transforms the artistic potentialities of dance, different pupil submissions span disciplines corresponding to music, storytelling, vital pondering, and reminiscence, creating generative experiences and new types of human-computer interplay. Taken collectively, these tasks illustrate a broader imaginative and prescient for synthetic intelligence: one which goes past automation to catalyze creativity, reshape schooling, and reimagine social interactions.
Be the Beat
“Be the Beat,” by Ethan Chang, an MIT mechanical engineering and design pupil, and Zhixing Chen, an MIT mechanical engineering and music pupil, is an AI-powered boombox that implies music from a dancer’s motion. Dance has historically been guided by music all through historical past and throughout cultures, but the idea of dancing to create music isn’t explored.
“Be the Beat” creates an area for human-AI collaboration on freestyle dance, empowering dancers to rethink the standard dynamic between dance and music. It makes use of PoseNet to explain actions for a big language mannequin, enabling it to research dance fashion and question APIs to seek out music with comparable fashion, power, and tempo. Dancers interacting with the boombox reported having extra management over inventive expression and described the boombox as a novel strategy to discovering dance genres and choreographing creatively.
A Thriller for You
“A Thriller for You,” by Mrinalini Singha SM ’24, a latest graduate within the Artwork, Tradition, and Know-how program, and Haoheng Tang, a latest graduate of the Harvard College Graduate Faculty of Design, is an academic recreation designed to domesticate vital pondering and fact-checking abilities in younger learners. The sport leverages a big language mannequin (LLM) and a tangible interface to create an immersive investigative expertise. Gamers act as citizen fact-checkers, responding to AI-generated “information alerts” printed by the sport interface. By inserting cartridge combos to immediate follow-up “information updates,” they navigate ambiguous eventualities, analyze proof, and weigh conflicting data to make knowledgeable choices.
This human-computer interplay expertise challenges our news-consumption habits by eliminating touchscreen interfaces, changing perpetual scrolling and skim-reading with a haptically wealthy analog machine. By combining the affordances of sluggish media with new generative media, the sport promotes considerate, embodied interactions whereas equipping gamers to raised perceive and problem right this moment’s polarized media panorama, the place misinformation and manipulative narratives thrive.
Memorscope
“Memorscope,” by MIT Media Lab analysis collaborator Keunwook Kim, is a tool that creates collective reminiscences by merging the deeply human expertise of face-to-face interplay with superior AI applied sciences. Impressed by how we use microscopes and telescopes to look at and uncover hidden and invisible particulars, Memorscope permits two customers to “look into” one another’s faces, utilizing this intimate interplay as a gateway to the creation and exploration of their shared reminiscences.
The machine leverages AI fashions corresponding to OpenAI and Midjourney, introducing totally different aesthetic and emotional interpretations, which ends up in a dynamic and collective reminiscence house. This house transcends the constraints of conventional shared albums, providing a fluid, interactive atmosphere the place reminiscences should not simply static snapshots however dwelling, evolving narratives, formed by the continuing relationship between customers.
Narratron
“Narratron,” by Harvard Graduate Faculty of Design college students Xiying (Aria) Bao and Yubo Zhao, is an interactive projector that co-creates and co-performs kids’s tales by means of shadow puppetry utilizing massive language fashions. Customers can press the shutter to “seize” protagonists they wish to be within the story, and it takes hand shadows (corresponding to animal shapes) as enter for the principle characters. The system then develops the story plot as new shadow characters are launched. The story seems by means of a projector as a backdrop for shadow puppetry whereas being narrated by means of a speaker as customers flip a crank to “play” in actual time. By combining visible, auditory, and bodily interactions in a single system, the venture goals to spark creativity in shadow play storytelling and allow multi-modal human-AI collaboration.
Excellent Syntax
“Excellent Syntax,” by Karyn Nakamura ’24, is a video artwork piece analyzing the syntactic logic behind movement and video. Utilizing AI to control video fragments, the venture explores how the fluidity of movement and time will be simulated and reconstructed by machines. Drawing inspiration from each philosophical inquiry and inventive observe, Nakamura’s work interrogates the connection between notion, expertise, and the motion that shapes our expertise of the world. By reimagining video by means of computational processes, Nakamura investigates the complexities of how machines perceive and characterize the passage of time and movement.
