An interesting solution.
The listed formats, 16: 9 and 4: 3, usually do not fit on smartphone screens. For this reason, the frames of many movies are simply cut for these devices. As Google determines, in this case, relevant content is manually identified in each video frame and the transitions between them are tracked to properly select the areas that should be cropped in each frame. This process consumes time and effort, which can be used in a better way, and to make matters worse, the results vary.
The solution to the problem is to be Google's open source tool called Autoflip. Thanks to him, movies cut in the wrong or strange way are to become a relic of the past.
Autoflip is based on two types of input data
The first is the film, shot amateur or professional, and the second is the selected format to which the material is to be cut. The tool automatically analyzes video content in search of relevant elements, develops optimal strategies for framing and tracking transitions between frames, and then creates an output film of the same duration as the input film, but with the desired ratio of length to frame height.
"Autoflip is a fully automatic solution that allows you to intelligently modify the size of frames in movies. It is a solution that uses the latest detection and tracking technologies, based on machine learning, which allows it to intelligently understand video content. ", we read in the Google message.
Thanks to the technologies listed Autoflip creates changed format materialswhose individual frames fit together much better than after manually trimming those frames. Thus, it looks like the tool can be extremely useful. I just hope that no one will try to use it to cut cinematic films or TV series, where the opportunity to admire the full original frame is usually of great importance.
Materials with manually cropped frames (placed higher) versus materials trimmed with the help of the Autoflip tool (located lower). | Source: Google AI Blog
You will learn more about the technologies used to create the Autoflip tool and how it works dedicated to artificial intelligence Google AI Blog. You can also look directly at the source code of the tool, which was made available in GitHub.
Source: Google AI Blog