A.I. Gigapixel – An Inside Story
I still vividly remember the day I was blown away when I discovered an enlarged photo similar to the one above. I was reading a paper about deep-learning based super-resolution. In the paper, 400% enlarged photos had crisp edges, few artifacts, and — never seen before — rich detail!
As the first company to use super-resolution technology in commercial products, we keep track of all major research in this area. Theoretically, there is no way to perfectly recreate a high-resolution image from only a low-resolution image. The amazing breakthrough of this particular paper is that it uses artificial intelligence (A.I.) to fill in those missing pieces that cannot be directly computed. A neural network is exposed to a large number of high-resolution and low-resolution image pairs. This network gradually learns to synthesize plausible detail in the enlarged image based what it has seen.
Everybody was excited. We wanted to let our users enjoy this revolutionary development. But wait, in this age of good digital cameras, does anyone even need more pixels? It turns out many people do. People that develop large prints want more DPI. Photos from drones or phone cameras can be improved.
We immediately put a team together and planned to develop a product quickly. Within weeks, Chris, our youngest developer, had an app prototype and Dr. Acharjee developed the initial neural network. However, things started to go off-track. Weeks turned into months. We had a great challenge on our hands.
First, the published method was great for small, high-quality test images, but failed on real camera photos. We had to develop a method robust enough for real digital camera raw/jpeg images.
Then there is the issue of speed — or rather, the lack of it. It took many hours to enlarge just one raw image since over 4 million calculations were needed to enlarge just one pixel. We had to find a new neural network architecture that not only produced the high-quality result but required much less computation.
Even so, a regular PC would still take a few hours to enlarge a large image. Dr. Acharjee had to develop a customized GPU neural network engine to take advantage of the computational power of your graphics card. Now it takes a laptop (with integrated graphics) 20 minutes, or high-end desktop GPU a few minutes per image. We started to see the possibilities unravel before our eyes.
In the end, Chris made it into an image batch processor so that it could run in the background. Since there is no need for parameter tuning, batch processing is actually a better workflow for image enlargement.
Over a year after that day, with countless hours of frustration and joy, we present A.I. Gigapixel. It is still very slow on most laptops, but we are making it available so that you can enjoy the latest A.I. technology without delay. We are still tweaking and training new variations of our neural networks as I am writing this (it takes at least a week for us to know if the tweak is better or worse), and we will continue to release updates whenever better results are achieved. (Edit 10/04/2018 – We has since released A.I. Gigapixel V2 which had increased processing speed 3 to 5 time!)
I hope you find A.I. Gigapixel useful. There is so much hype nowadays about A.I., but Topaz Labs is the only company that has actually delivered A.I. based desktop photo processing application(*). We’ve already released A.I. Remix to change photos into paintings, A.I. Clear to remove photo noise, and now we hope A.I. Gigapixel can help you a little more in your pursuit of artistry.
Thank you for reading. Please give your feedback in the comments section. If you have any questions you can reach out to our support system here.
– Albert Yang, CEO Topaz Labs
(*) there are other deep-learning based photo processing products, all of them run on cloud servers instead of on laptop/desktop computers.
About the Author
About Albert Yang
Albert Yang founded Topaz Labs over 10 years ago, to form a company that adopts and implements the latest technology to introduce cutting-edge tools to the Photo market. With over 30 years of programming experience, he’s proud to offer his technical expertise to our users as the primary developer of our latest tools.