Black Technology automatically generates super slow motion, and NVIDIA's AI is so cool.
Slow motion is a technique often used in today's film and television works. He can better attract the audience, let people see the details that are not visible under normal speed, bring visual impact; bring people into the mind's mind better, pass emotion.
With the advancement of technology, today's mobile phones can also shoot slow motion. If you just send a circle of friends, Instagram, Slow-Motion in the iPhone camera is absolutely enough!
But what if you want to shoot professional-grade slow motion? You often need a fairly high-end camera. However, the average slow-motion camera starts at 1000fps (frames per second), and thousands of dollars are normal; professional grades are up to 3000fps or even tens of thousands of fps, and the price is often tens of thousands of dollars.
Not everyone can be The Slow Mo Guys...
The picture above is from The Slow Mo Guys. Look, it’s not the same with a slow-motion camera with hundreds of thousands of knives...
If you switch to a slag frame rate camera, it becomes like this??
Why are they not coherent? Because the principle of slow motion is to shoot at a high frame rate, the higher the frame rate, the smoother it will be when played at a normal frame rate of 60 or 120 fps.
The camera itself has a low frame rate. Do you have to force it to slow down?
But what if you want it?
Introducing a new technology recently released by NVIDIA: machine learning automatically generates slow motion:
Slow motion by algorithm
The traditional slow motion is because the camera really captures so many frames of pictures.
And I believe you can understand, NVIDIA's black technology does not need so many frames: it uses algorithms to add more frames to standard video, which turns it into a high frame rate video, which has slow motion. .
What does it mean? Let's take the example below. For example, if the normal camera captures 10 frames of the following group, NVIDIA Black Technology uses the calculation "forced" to generate additional frames, which are inserted in the middle of the existing frames.
These newly generated images, in the end, should be modeled by algorithms. In the past, there were similar algorithms generated, and the effect was not good. From the video just now, you also saw that NVIDIA has far exceeded the degree of realism - this is really a slow motion!
The name of this black technology is called "Variable-Length Multi-Frame Interpolation".
It can generate new "false frames" based on standard video with unimaginable accuracy, which turns into smooth and clear slow motion video. The frame rate of the original video must be at or above 30fps.
The most powerful thing is that this slow motion, how slow you want, you can turn it into 60fps, that is, you can insert more than one frame every two frames; you can also change to 900 frames, that is, every two Insert 29 frames between frames...
Nvidia calls this technology visually: super slow motion
And Nvidia said that this algorithm "is better than the most cutting-edge methods available" in the test.
For example, the following figure shows six similar algorithms. It can be seen that the first five have different degrees of image distortion, and the trueness of f is the best, which is also the technology of NVIDIA.
How much does the AI brain make up?
This so-called multi-frame interpolation technique is not really new.
If you still remember watching TV when you were young, watching the game feels very smooth. Watching the TV drama feels like a milky voice, soft and disappointing, then congratulations: You have already experienced a technique called dynamic interpolation...
This is because TV has this ability long ago, it can copy and evolve two frames of pictures to a certain extent, generate a new frame, and then insert it.
NVIDIA's new algorithm is also between two frames, but the name "variable multi-frame" means that interpolation can be done between any two frames, where interpolation and occlusion reasoning are simultaneously built on an end-to-end convolutional neural network. mold.
For example, in the 1, 2, and 3 frames of the original video, Nvidia's black technology can generate a new one between 1 and 3, and compare it with the real 2 frames to optimize the algorithm.
Everyone can do this, why did Nvidia win? The answer is obvious: they are sitting on a lot of graphics cards...
During the training algorithm, NVIDIA used the Tesla V100 GPU and the improved PyTorch deep learning framework. NVIDIA used a total of 1132 videos with a total of 376,000 independent video frames.
In order to show their results more intuitively, NVIDIA has really cooperated with The Slow Mo Guys. The video at the beginning shows Nvidia's "slower-motion" video, which was further generated by the slow-motion video they sent online. The effect is quite amazing.
More importantly behind AI's breakthrough is that it gives slow-motion production the possibility of getting rid of hardware constraints.
NVIDIA said in the paper that they hope to complete most of the processing in the cloud in the future, so that this technology can be popularized on consumer devices.
The meaning of this sentence is: in the future you can pat on the phone, the server automatically gives you a super slow lens...
Is it worth looking forward to?
Only then, we may have another problem:
When more than half of the frames in the video are forced to be "edited" by AI, is it still a real video?
Follow Me
——END——
With the advancement of technology, today's mobile phones can also shoot slow motion. If you just send a circle of friends, Instagram, Slow-Motion in the iPhone camera is absolutely enough!
But what if you want to shoot professional-grade slow motion? You often need a fairly high-end camera. However, the average slow-motion camera starts at 1000fps (frames per second), and thousands of dollars are normal; professional grades are up to 3000fps or even tens of thousands of fps, and the price is often tens of thousands of dollars.
Not everyone can be The Slow Mo Guys...
The picture above is from The Slow Mo Guys. Look, it’s not the same with a slow-motion camera with hundreds of thousands of knives...
If you switch to a slag frame rate camera, it becomes like this??
Why are they not coherent? Because the principle of slow motion is to shoot at a high frame rate, the higher the frame rate, the smoother it will be when played at a normal frame rate of 60 or 120 fps.
The camera itself has a low frame rate. Do you have to force it to slow down?
But what if you want it?
Introducing a new technology recently released by NVIDIA: machine learning automatically generates slow motion:
Slow motion by algorithm
The traditional slow motion is because the camera really captures so many frames of pictures.
And I believe you can understand, NVIDIA's black technology does not need so many frames: it uses algorithms to add more frames to standard video, which turns it into a high frame rate video, which has slow motion. .
What does it mean? Let's take the example below. For example, if the normal camera captures 10 frames of the following group, NVIDIA Black Technology uses the calculation "forced" to generate additional frames, which are inserted in the middle of the existing frames.
These newly generated images, in the end, should be modeled by algorithms. In the past, there were similar algorithms generated, and the effect was not good. From the video just now, you also saw that NVIDIA has far exceeded the degree of realism - this is really a slow motion!
The name of this black technology is called "Variable-Length Multi-Frame Interpolation".
It can generate new "false frames" based on standard video with unimaginable accuracy, which turns into smooth and clear slow motion video. The frame rate of the original video must be at or above 30fps.
The most powerful thing is that this slow motion, how slow you want, you can turn it into 60fps, that is, you can insert more than one frame every two frames; you can also change to 900 frames, that is, every two Insert 29 frames between frames...
Nvidia calls this technology visually: super slow motion
And Nvidia said that this algorithm "is better than the most cutting-edge methods available" in the test.
For example, the following figure shows six similar algorithms. It can be seen that the first five have different degrees of image distortion, and the trueness of f is the best, which is also the technology of NVIDIA.
How much does the AI brain make up?
This so-called multi-frame interpolation technique is not really new.
If you still remember watching TV when you were young, watching the game feels very smooth. Watching the TV drama feels like a milky voice, soft and disappointing, then congratulations: You have already experienced a technique called dynamic interpolation...
This is because TV has this ability long ago, it can copy and evolve two frames of pictures to a certain extent, generate a new frame, and then insert it.
NVIDIA's new algorithm is also between two frames, but the name "variable multi-frame" means that interpolation can be done between any two frames, where interpolation and occlusion reasoning are simultaneously built on an end-to-end convolutional neural network. mold.
For example, in the 1, 2, and 3 frames of the original video, Nvidia's black technology can generate a new one between 1 and 3, and compare it with the real 2 frames to optimize the algorithm.
Everyone can do this, why did Nvidia win? The answer is obvious: they are sitting on a lot of graphics cards...
During the training algorithm, NVIDIA used the Tesla V100 GPU and the improved PyTorch deep learning framework. NVIDIA used a total of 1132 videos with a total of 376,000 independent video frames.
In order to show their results more intuitively, NVIDIA has really cooperated with The Slow Mo Guys. The video at the beginning shows Nvidia's "slower-motion" video, which was further generated by the slow-motion video they sent online. The effect is quite amazing.
More importantly behind AI's breakthrough is that it gives slow-motion production the possibility of getting rid of hardware constraints.
NVIDIA said in the paper that they hope to complete most of the processing in the cloud in the future, so that this technology can be popularized on consumer devices.
The meaning of this sentence is: in the future you can pat on the phone, the server automatically gives you a super slow lens...
Is it worth looking forward to?
Only then, we may have another problem:
When more than half of the frames in the video are forced to be "edited" by AI, is it still a real video?
Follow Me
Link:Tenco
——END——
评论
发表评论