This AI Learned How To Generate Human Appearance


Dear Fellow Scholars, this is Two Minute Papers
with Károly Zsolnai-Fehér. In this series, we often discuss that neural
networks are extraordinarily useful for classification tasks. This means that if we give them an image,
they can tell us what’s on it, which is great for self-driving cars, image search, and a
variety of other applications. However, fewer people know that they can also
be used for image generation. We’ve seen many great examples of this where
NVIDIA’s AI was able to dream up high-resolution images of imaginary celebrities. This was done using a generative adversarial
network, an architecture where two neural networks battle each other. However, these methods don’t work too well
if we have too much variation in our datasets. For instance, they are great for faces, but
not for synthesizing an entire human body. This particular technique uses a different
architecture, and as a result, can synthesize an entire human body, and is also able to
synthesize both shape and appearance. You will see in a moment that because of that,
it can do magical things. For instance, in this example, all we have
is one low-quality image of a test subject as an input, and we can give it a photo of
a different person. What happens now is that the algorithm runs
pose estimation on this input, and transforms our test subject into that pose. The crazy thing about this is that it even
creates views for new angles we didn’t even have access to! In this other experiment, we have one image
on the left. What we can do here is that we specify not
a person, but draw the pose directly, indicating that we wish to see our test subject in this
pose, and the algorithm is also able to create an appropriate new image. And again, it works for angles that require
information that we don’t have access to. These new angles show that the technique understands
the concept of shorts or trousers…although it seems to forget to put on socks sometimes. Truth be told, I don’t blame it. What is even cooler is that it seems to behave
very similarly for a variety of different inputs. This is non-trivial as this property doesn’t
just emerge out of thin air, and will be a great selling point for this new method. It also supports a feature where we need to
give a crude drawing to the algorithm, and it will transform it into a photorealistic
image. However, it is clear that there are many-many
ways to fill this drawing with information, so how do we tell the algorithm what appearance
we are looking for? Well, worry not, because this technique can
also perform appearance transfer. This means that we can exert artistic control
over the output by providing a photo of a different object, and it will transfer the
style of this photo to our input. No artistic skills needed, but good taste
is still as much of a necessity as ever. Yet another AI that will empower both experts
and novice users alike. And while we are enjoying these amazing results,
or even better, if you have already built up an addiction for the papers, you can keep
it in check by supporting us on Patreon and in return, getting access to these videos
earlier. You can find us through Patreon.com/TwoMinutePapers. There is a link to it in the video description,
and as always, to the paper as well. Thanks for watching and for your generous
support, and I’ll see you next time!

100 Comments

Add a Comment

Your email address will not be published. Required fields are marked *