November 4, 2015

System automatically converts 2-D video to 3-D

Illustration: Christine Daniloff/MIT

(November 4, 2015) Exploiting video game software yields broadcast-quality 3-D video of soccer games in real time.

By exploiting the graphics-rendering software that powers sports video games, researchers at MIT and the Qatar Computing Research Institute (QCRI) have developed a system that automatically converts 2-D video of soccer games into 3-D.

The converted video can be played back over any 3-D device — a commercial 3-D TV, Google’s new Cardboard system, which turns smartphones into 3-D displays, or special-purpose displays such as Oculus Rift.

The researchers presented the new system last week at the Association for Computing Machinery’s Multimedia conference.

“Any TV these days is capable of 3-D,” says Wojciech Matusik, an associate professor of electrical engineering and computer science at MIT and one of the system’s co-developers. “There’s just no content. So we see that the production of high-quality content is the main thing that should happen. But sports is very hard. With movies, you have artists who paint the depth map. Here, there is no luxury of hiring 100 artists to do the conversion. This has to happen in real-time.”
The system is one result of a collaboration between QCRI and MIT’s Computer Science and Artificial Intelligence Laboratory. Joining Matusik on the conference paper are Kiana Calagari, a research associate at QCRI and first author; Alexandre Kaspar, an MIT graduate student in electrical engineering and computer science; Piotr Didyk, who was a postdoc in Matusik’s group and is now a researcher at the Max Planck Institute for Informatics; Mohamed Hefeeda, a principal scientist at QCRI; and Mohamed Elgharib, a QCRI postdoc. QCRI also helped fund the project.

Zeroing in

In the past, researchers have tried to develop general-purpose systems for converting 2-D video to 3-D, but they haven’t worked very well and have tended to produce odd visual artifacts that detract from the viewing experience.

read entire press  release >>