Purely AI News: For AI professionals in a hurry
July 19, 2020
Microsoft is developing AI to improve camera-in-display technology for natural perspectives and clearer visuals in video calls
Spatial factors also affect conversational dynamics but are not taken into account by current videoconferencing systems. One of the outcomes of this research is repositioning of speakers to make the conversation feel more natural. Credits: Microsoft Research.
With this project, Microsoft Research is exploring how machine learning can help to solve some of the problems of image loss associated with putting cameras behind the screen, which can help frame remote conversations in a more realistic space setting. The two most significant problems associated with camera-in-display technology today are of perspective (for making eye contact in video conferencing) and the diffraction caused by OLED screens.

Using transparent OLED (T-OLED) displays, we can put a camera behind the screen, potentially solving the issue of perspective. Yet since the glass is not fully clear, it degrades image quality by adding diffraction and noise when looking through it. To address this, Microsoft used a U-Net neural-network structure to compensate for the image loss inherent in photographing through a T-OLED screen, which both increase the signal-to-noise ratio and de-blurs the image.

On the issue of perspective, both the arrangement of the participants relative to each other and their distance (proxemics) are meaningful aspects of nonverbal communication but are not taken into account by current videoconferencing systems. The solution to this, according to Microsoft Research, lies in first identifying the speakers using image segmentation achieved with a custom CNN model. It then becomes possible to scale the incoming video after identifying the speaker in the remote view, so that the remote participant can appear on the local display in a lifelike image. One way to accomplish this would be to zoom the whole image and re-center it on the speaker. However, for this project, the researchers went a step further. They extracted and scaled the individual independently of the background, as seen in the image shown above.

Their research concludes, "Human interaction in videoconferences can be made more natural by correcting gaze, scale, and position by using convolutional neural network segmentation together with cameras embedded in a partially transparent display. The diffraction and noise resulting from placing the camera behind the screen can be effectively removed using U-Net neural network. Segmentation of live video also makes it possible to combine the speaker with a choice of background content."
LinkedIn



Aug. 2, 2020

Sample Factory, a new training framework for Reinforcement Learning slashes the level of compute required for state-of-the-art results

23
July 31, 2020

Intel joins hands with researchers from MIT and Georgia Tech to work on a code improvement recommendation system, develops "An End-to-End Neural Code Similarity System"

22
July 25, 2020

Google's tensorflow-lite framework for deep learning is now more than 2x faster on average, using operator fusion and optimizations for additional CPU instruction sets

21
July 23, 2020

Fawkes: An AI system that puts an 'invisibility cloak' on images so that facial recognition algorithms are not able to reveal identities of people without permission

20
July 22, 2020

Researchers from Austria propose an AI system that reads sheet music from raw images and aligns that to a given audio accurately

19
July 21, 2020

WordCraft: A Reinforcement Learning environment for enabling common-sense based agents

18
July 20, 2020

A designer who worked on over 20 commercial projects for a year turns out to be an AI built by the Russian design firm Art. Lebedev Studio

17
July 18, 2020

Microsoft and Zhajiang Univ. researchers create AI Model that can sing in several languages including both Chinese and English

15
July 18, 2020

New event-based learning algorithm 'E-Prop' inspired by the Human brain is more efficient than conventional Deep Learning

14
July 17, 2020

Scientists from the University of California address the false-negative problem of MRI Reconstruction Networks using adversarial techniques

13
July 16, 2020

A new technique of exposing DeepFakes uses the classical signal processing technique of frequency analysis

12
July 16, 2020

New AI model By Facebook researchers can recognize five different voices speaking simultaneously, pushes state-of-the-art forward

11
July 15, 2020

Researchers from Columbia Univ. and DeepMind propose a new framework for Taylor Expansion Policy Optimization (TayPO)

10
July 14, 2020

Federated Learning is finally here; Presagen's new algorithm creates higher performing AI than traditional centralized learning

9
July 14, 2020

Fujitsu designed a new Deep Learning based method for dimensionality reduction inspired by compression technology

8
July 12, 2020

Databricks donates its immensely popular MLflow framework to the Linux Foundation

7
July 12, 2020

Microsoft Research restores old photos that suffer from severe degradation with a new deep learning based approach

6
July 12, 2020

Amazon launches a new AI based automatic code review service named CodeGuru

5
July 12, 2020

IBM launches new Deep Learning project: Verifiably Safe Reinforcement Learning (VSRL) framework

4
July 12, 2020

DevOps for ML get an upgrade with new open-source CI/CD library, "Continuous Machine Learning (CML)"

3
July 12, 2020

DeepMind's new open-sourced Reinforcement-Learning library, dm_control, packs a simple interface to common RL utilities

2
July 12, 2020

Learning to learn: Google's AutoML-Zero learns to evolve new ML algorithms from scratch

1