119 reads

Beeble Researchers Develop AI That Can Make Any Photo Look Perfectly Lit—Even in the Darkest Room

by Auto Encoder: How to Ignore the Signal NoiseDecember 21st, 2024

Too Long; Didn't Read

Researchers at Beeble AI have developed a method for improving how light and shadows can be applied to human portraits in digital images.

featured image - Beeble Researchers Develop AI That Can Make Any Photo Look Perfectly Lit—Even in the Darkest Room

Read by Dr. One voice-avatar

Listen to this story

Authors:

(1) Hoon Kim, Beeble AI, and contributed equally to this work;

(2) Minje Jang, Beeble AI, and contributed equally to this work;

(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;

(4) Jisoo Lee, Beeble AI, and contributed equally to this work;

(5) Donghyun Na, Beeble AI, and contributed equally to this work;

(6) Sanghyun Woo, New York University, and contributed equally to this work.

Editor's Note: This is Part 6 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below.

Table of Links

Appendix

3.4. Objectives

We supervise both intrinsic image attributes and relit images using their corresponding ground truths, obtained from the lightstage. We employ a combination of reconstruction, perceptual [24], adversarial [22], and specular [34] losses.

Figure 4. Neural Render Enhancement. Using the CookTorrance model, diffuse and specular renders are computed, which are then composited into a physically-based rendering. Subsequently, a neural network enhances this PBR render, improving aspects such as brightness and specular details.

Final Loss. The SwitchLight is trained in an end-to-end manner using the weighted sum of the above losses:

We empirically determined the weighting coefficients.

Figure 5. Dynamic Masking Strategies. We have generalized the MAE masks to include overlapping patches of varying sizes, as well as outpainting and free-form masks.

This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license.

L O A D I N G
. . . comments & more!

About Author

Auto Encoder: How to Ignore the Signal Noise@autoencoder

Research & publications on Auto Encoders, revolutionizing data compression and feature learning techniques.

Read my stories Learn More

TOPICS

machine-learning #self-supervised-learning #relighting #human-portrait-relighting #physics-guided-architecture #cook-torrance-model #light-surface-interactions #switchlight-framework #self-supervised-pre-training

THIS ARTICLE WAS FEATURED IN...

Latest technology trends. Customized Experience. Curated Stories. Publish Your Ideas

Beeble Researchers Develop AI That Can Make Any Photo Look Perfectly Lit—Even in the Darkest Room

Auto Encoder: How to Ignore the Signal Noise

@autoencoder

Too Long; Didn't Read

Auto Encoder: How to Ignore the Signal Noise

STORY’S CREDIBILITY

Academic Research Paper

Table of Links