Anyone who takes pictures with a digital camera is probably familiar with the phenomenon of “noise”, also known as tingling: colorless dots that cause a picture to lose its clarity and sharpness. Many photographers make adjustments to the camera lens or pay attention to the angle of light when taking pictures to reduce unwanted spots in images. However, these tingles exist not only in photographs, but also in computers and graphics. We will now take a closer look at the “denoising” process, which is a life-saving technique.
What is Denoising?
Black dots occur with random brightness and color changes that are not part of the original image. The removal of this tingling, which is common in the field of image processing, is known as “denoising”.
In the denoising process, advanced algorithms are used to remove unwanted points from graphics and renders. With these algorithms, the quality of the images can change drastically. Without denoising technology, it would not be possible to create and use photorealistic images.
Images used in computer graphics can have both pure visuals and tingles. As we mentioned, these dots reduce the sharpness of images, the sharpness disappears. After the purification process is completed, the clear image we want remains in the final, the unwanted spots disappear. It is also important to preserve visual details and components such as edges, corners, textures and other sharp structures when denoising an image.
Three points in the image are important to reduce tingling while not affecting visual details:
- Clutter: Diffused lighting that reflects in all directions.
- Specular or reflections: Illuminations that reflect in a particular direction.
- Endless light source shadows: Finally, sunlight, shadows, and other visible light sources.
In order to obtain the clearest image, it is necessary to use thousands of beams in the directions following the scattered and specular signals. Real-time ray tracing scenarios, on the other hand, typically only use one ray per pixel or less.
How Does the Denoising Process Work?
Image cleaning generally relies on three techniques: spatial filtering, temporal deposition, and machine learning/deep learning.
Spatial filtering replaces parts of an image by reusing similar neighboring pixels. The advantage of this technique is that it does not create delays under varying flow conditions. On the downside, it brings with it blur and muddy appearance, as well as temporal instability, which represents flickering and visual imperfections in the image.
Temporal accumulation reuses data from the previous frame to determine if there are any artifacts or visual anomalies in the current frame that can be corrected. Temporal accumulation causes temporal delay, but not blurring. Instead, it adds temporal stability to reduce jitter and artifacts across multiple frames.
Machine learning and deep learning technique uses a neural network to reconstruct the signal. The neural network is trained using various pitting and reference signals. Although the reconstructed signal for a single frame may appear complete, it can become temporally unstable over time, so some form of temporal stabilization is required.
Denoising in Graphics
Denoising provides instant visual feedback to users. So people can see and interact with graphics and designs. It can change variables such as light, materials, viewing angle and shadows.
Solutions like NVIDIA Real-Time Denoisers (NRD) make denoising techniques more accessible for developers to integrate into their pipelines. NRD is offered as a spatio-temporal denoising library that is independent of application programming interfaces and is designed to work with low beams per pixel.
With NRD, developers can achieve real-time results using a limited per-pixel beam stack. In the video above you can see the work NRD does in real time to remove image noise. Popular games like Dying Light 2 and Hitman III use NRD for denoising.
