Credit: Unsplash
It’s like AutoTune for dancing.
If there’s one obvious disparity between music and dancing, it’s that music can more or less be recognized and created by an algorithm. Dancing, on the other hand, seems to lack a concrete definition. For some, dancing is act of expression where you flail your limbs wildly in whatever direction the music moves you. For others, if you move your arms and legs even a little bit, then you’re not dancing. Still, technological advancement is nothing if not the tackling of obtuse problems, which is why Adobe has decided to try and make an AI that can recognize dancing.
Adobe showed off their latest AI experiments powered by Adobe Sensei, their proprietary machine learning algorithm. One of the presentations, titled “On the Beat,” featured researcher Yang Zhou who, on the urgings of his daughter, attempted to learn one of the latest TikTok dance crazes. Unfortunately, while he could mimic the basic moves, his timing was all over the place. To aid himself, Zhou created a program using Adobe Sensei to monitor his moves and help him get the timing down.
Watching through a webcam, the program creates a skeletal mockup of Zhou’s body as he moves to the music, with large orange circles signifying the major beats. When his steps synchronize with the backing track, he gets a small “Perfect~!” message to let him know, not unlike a video game. Utilizing this program, Zhou was able to train himself to master the dance for his daughter’s amusement.
Interestingly, the program also works in reverse; Zhou was able to gather up a bunch of random clips of children playing in disconnected ways, and the program automatically chose a song to sync all of them together nearly perfectly. If someone thought about it, there’s probably a practical application for this particular program, but frankly, if Adobe released it as a dancing aid app for parents, they could probably make some good money.
