Deep learning is being used artificially increase the resolution of textures in old games using ESRGAN.
ESRGAN stands for Enhanced Super Resolution Generative Adversarial Networks and is an advanced deep learning technique to increase the visual resolution of an image using AI.
While this algorithm is quite impressive, it does have the side effect of creating artifacts. Before I get into the wow let me give a brief explanation of wtf I'm talking about.
Deep learning is the process of training a computer to learn about a subject matter similar to how the human brain works. You feed a neural network with data you want to learn to identify or manipulate and the neurons of the network start to form paths and learn the data. Once you have gone through enough sample data, you then have weights that have been adjusted thousands or even millions of times until it has a very high degree of accuracy (typically 70-98%). You can then take this model and plug it into almost any system with very little overhead. The training part of deep learning is extremely time consuming and expensive, but the final model can typically run on a Raspberry Pi.
ESRGAN uses what is called a Generative Adversarial Network (GANS), this is basically two deep learning networks fighting against each other to improve their performance. One network acts as a generator to create output based on the input. The other network is a discriminator to learn the original data set. They are then set against each other, one acting as a forger trying to create data that looks like the original data set and the other trying to determine how likely it is the original data. Over time both get better at their job and in the end, you build a model that can create data very accurately.
A good example of this technology is to create higher resolution images from low-resolution source.
In the above example, you can see how well the network guessed the original image with what looks like very little information to work on.
There are now texture packs being developed for older games using this technology and the results are pretty impressive. What's really great about this technology is you don't need to hire artists to go over every texture, one a network is trained, it can be used for many projects.
Let me show you some examples.
Original
Using ESRGAN
Original
ESRGAN
Original
ESRGAN
The results are very impressive and you can usually expect roughly 4x resolution bump using the technique. Below you will find video footage of ESRGANS created texture packs in real gameplay.
CES 2019 is happening right now and is the largest consumer technology convention in the world. This year 8K TVs are being showcased and many of them are using ESRGANS or something similar to automatically upscale content to higher resolutions. If you ever had a Bluray player in the early days, one of the features was upscaling which would take the 480p DVD content and make it look like 1080p on HD TVs. While it varied dramatically how successful it was, using ESGANS you can actually achieve a reasonable 4K upscaling from similar DVDs.
Imagine the Playstation 5 with 8K resolution using this technology. Even taking Playstation 3 games and making them look like they were native to the Playstation 4.
It's an exciting time to be alive!
