January 12, 2023
Before diving into the headlining Face Recovery feature improvement of Gigapixel AI v6.1, I’d like to point out that users who have computers containing NVIDIA GPUs that are 30, 20, or 10 Series will experience notable performance improvements. We optimized the library used for NVIDIA GPUs from DirectML to TensorRT. While this will require NVIDIA GPU users to re-download the Gigapixel AI models, the performance boost will make the task totally worth it. Here is a benchmark test comparing the NVIDIA RTX 3090 performance using DirectML and TensorRT. As you can see, the performance gains are notable.
Despite what you may have read, not all upscaling methods produce the same results. Increasing the resolution of a photo involves a lot more than just changing its pixel dimensions. In virtually every case, it’s as important—if not more so—to preserve the details of the subject(s) within the photo you’re enlarging and perhaps there is no detail more important to preserve than the facial features of people in a photo.
That is why we are making a significant update to the Face Recovery model with Gigapixel AI v6.1, which was first introduced as Face Refinement with Gigapixel AI v5.8. Before I dive into how your upscaled photos will benefit from Face Recovery, I’d first like to discuss upscaling photos in a broader sense and illustrate why Gigapixel AI is the best solution available. I’d also like to share the links to download Gigapixel AI v6.1 and the official release notes.
When discussing upscaling in terms of digital photography, we most commonly refer to the process of increasing the resolution—or dimensions—of a photo, and there are several reasons why you’d need to upscale one. Perhaps the original version of the photo is already low resolution and you need to increase it for a large poster print. Another common scenario is that you had to apply a heavy crop to the photo, which effectively reduces the resolution, and you want to increase it again. Regardless of why you may need to upscale a photo, Gigapixel AI will provide superior results while also preserving (and enhancing) important details.
Here’s a practical example of why you’d need to upscale a photo and why using Gigapixel AI is the smart choice for that task. Earlier this year, I went on a trip to photograph bald eagles that routinely nest and feed in the area during the winter. Despite using a 100-400mm telephoto lens with a 1.4x teleconverter, this was the closest I could focus on the bird.
As much as I like this composition, I don’t think it’s ideal to share on Instagram (which is one of the most common use cases for digital photos). Unfortunately, the only version of the photo that I had access to at the time was 2000 pixels on the long end, which I embedded above. After applying a square crop to isolate the eagle, the resulting resolution was 611 x 611 pixels, translating to about 0.37 megapixels, and does not meet Instagram’s suggested resolution of 1080 x 1080 pixels.
At this point, the best solution would be to upscale the photo to something more suitable. Because I am never certain what I’ll ultimately need an upscaled photo for, I tend to use a 6x, or 600%, factor which will yield a resolution of 3666 x 3666 pixels, or 13.4 megapixels. And this is where choosing the right software makes all the difference.
One of the most common applications that Gigapixel AI is compared to is Adobe Photoshop and we’ve already covered how it produces more detailed and natural results. However, here is a comparison of a 600% upscale using Gigapixel AI vs Adobe Photoshop 2022. As you can see, the improvements from the Gigapixel AI results are immediately noticeable.
Now that we’ve established how Gigapixel AI can benefit you with your upscaling requirements, I want to dive into a major improvement with the Face Recovery model introduced with Gigapixel AI v6.1.
There is a certain built-in quality threshold that users expect when upscaling a low-resolution photo and it varies based on the subject that is being upscaled. For example, you may have one threshold for a low-resolution photo of a building that is being upscaled. As long as the details are sharp and discernible, we tend to be satisfied with the results.
That threshold is much higher when it comes to upscaling a photo with people in it. Part of the reason could be because of the personal connection with the person (or people) in the photo, especially if you are in it. The other reason is that subpar upscaling of facial features causes the photo to “fall apart” at a much higher rate than, say, a building. Humans are especially good at determining when something about a person’s face just “doesn’t look quite right,” as if it has fallen into the uncanny valley.
That’s why it was so important for us to further improve the Face Refinement model that we first introduced with Gigapixel AI v5.8. Before diving into how we improved the Face Refinement model, I’d like to show you the improvements with a portrait photo when comparing the Face Refinement model (using Gigapixel AI v6.0) to the new Face Recovery model (using Gigapixel AI v6.1). At first glance, you could argue that the improvements made when upscaling this portrait photo are commendable, especially when you compare it to the original image. Gigapixel AI handily removed all of the distracting noise while recovering most of the facial details. However, there is something about the right eye and teeth that would cause a viewer to do a double-take because something “doesn’t look quite right.”
Now, when you compare the exact same image using the exact same settings in Gigapixel AI v6.1 along with the new Face Recovery model, the improvements are immediately noticeable. More importantly, a viewer will look at the upscaled photo and never think to question whether something “doesn’t look quite right.” The results are far more natural compared to the former Face Refinement model and is in a league of its when compared to Adobe Photoshop or any other comparable photo upscaling software.
In addition to improving the overall Face Recovery model, we also added a new Face Recovery Strength slider that allows you to fine-tune the strength of the model. At lower settings, faces will have less denoising and sharpening applied when compared to the surrounding area. At higher strength, faces will be noticeably sharper and stand out more clearly. I recommend starting at 100 and reducing the strength as needed. Let’s compare the Face Recovery Strength slider at a few levels to see the improvements.
The incremental improvements to the details of the face become more apparent as you increase the value of the Face Recovery Strength slider, especially around the eyes and teeth, so be sure to experiment with it whenever you use the Face Recovery model.
The primary purpose of the Face Recovery model is to recover a wide variety of facial features that may have been negatively impacted from low-quality photos, such as those suffering from being highly compressed, noisy, artifacted, shaky, or blurred. Fortunately, we were able to improve the former Face Refinement model by adopting a brand new deep learning process and training the new Face Recovery model by using a large library of high quality faces.
When you activate Face Recovery, each detected face is resized to 512 x 512 pixels, then processed by the model with object enhancements (eyes, teeth, etc), and finally resized, blended, and positioned accordingly. These object enhancements are determined based on the training using the library of high quality faces mentioned above.
The improvements to the recovered facial features are often presented with more defined details to the pupils, eyelashes, hair, and skin textures. When combined with additional noise reduction and sharpening, the overall results of the upscaled photo are often significantly improved when compared to the original.
When you look back at all the photo examples I shared in this article, do you see anything that they have in common? Each of them is a low-resolution photo of a person or people whose face is relatively small within the overall composition. Just look at the size of the white focus box in the navigator view (top right of the UI) for each photo above. Also, some of the faces suffer from excess noise or other quality degradations due to the low resolution.
When upscaling a high-resolution or high-quality photo with a face that takes up a majority of the frame, we recommend disabling Face Refinement and select the most appropriate base super-resolution model to increase pixel size. The Comparison View makes this task especially easy because it allows you to compare four models simultaneously.
Let’s use the following portrait photo as an example. As you can see, the girl’s face takes up most of the frame. You can tell this by looking at the full image in the navigator in the top right of the UI. In this example with Face Refinement enabled, you’ll notice that the model over-smoothes the girl’s skin and eyes.
When you compare the same photo upscaled using the Low Resolution model and Face Recovery disabled, you will immediately see a marked improved throughout the face. The skin looks natural and the eyes have the perfect amount of sharpness added.
As we continue to train the Face Recovery model using photos with larger faces, we will improve our ability to provide superior results.
To recap, we recommend using the Face Recovery model with low-resolution photos containing faces affected by obvious quality issues such as being highly compressed, noisy, artifacted, shaky, or blurred. For faces that already have a satisfactory amount of details, we recommend disabling this tool and using one of the five base super-resolution models. We also recommend setting the Face Recovery Strength slider to 100 and reducing the strength as needed.
There is an undeniable need to upscale portrait photos so that they look natural regardless of the source resolution. The improved Face Recovery model makes it that much easier to achieve sharp and realistic upscaled portrait photos. And Gigapixel AI’s five purpose-built models will handle all your other upscaling needs with high quality results, especially when compared to Adobe Photoshop and other comparable software.
Be sure to download a free trial of Gigapixel AI today to see the difference for yourself.
© 2005 - 2024 Topaz Labs. All rights reserved.
