OPPO Reno4 Series: AI-enhanced Smart Sensor, a new era in smartphone technology

Population density and population mobility is increasing daily all around the world and more people are becoming tech savvy with each passing day. Internet and information technology have gradually and consistently penetrated into our lifestyles. Staying always connected through the internet has made privacy protection one of the trickiest problems for smartphone users. On the other hand, for smartphone brands, the ability to continuously update technologies and create products with excellent user experience are also growing challenges.

Reno4 is equipped with a new AI-enhanced smart sensor, which is an attempt made by OPPO for building products in the AI-smart era. This smart sensor can identify the owner of the mobile phone and intelligently decide whether to show the detailed content in a notification, while enabling touchless operation of the mobile phone. With the new AI-enhanced smart sensor, Reno4 realizes the integrated functions of N-in-one such as AI assistant, privacy security and touchless gesture operation. It supports the original intention of the OPPO brand and its Research & Development team to make mobile phones smarter and maximize the experience and the technology.

Smart sensors, an answer to users’ demand of better privacy protection

For some time in the recent years, OPPO has been cultivating the Southeast Asian market, since it first entered the Indonesian market in 2012. According to a survey by Euromonitor International, comparing the market share of OPPO in 2019 and 2014, Thailand increased to 4.8 times, Indonesia 4.7 times and Vietnam 2.9 times. With further expansion of market share, OPPO realized that mobile phone users in the Southeast Asian market are also increasingly demanding privacy protection. Simultaneously, the population density in this market is increasing and people are living a more connected lifestyle. Mobile phones, as the most important social interaction tools, are becoming more and more vulnerable to threats against information privacy for the owners. For instance, your friends may accidently catch a glimpse of critical or private content on your phone.

After performing a market research, OPPO realized that the functions which the smart sensor provides could help bridge the gaps in market demand, especially for privacy protection. In the past, there were two solutions in the industry. The first was the use of radar, which can effectively reduce the power consumption of mobile phones; yet this also means that the footprint of the sensor is increased and the screen ratio is reduced accordingly. Also, the radar itself has limited recognition capabilities, thus the scenarios to be implemented are restricted. The second was to use an ordinary camera for real-time shooting to recognize motions and faces, being able to achieve smart control, but in this case the power consumption brought by the function is huge. Both solutions have their own limitations, OPPO had to find another way.

● Smart Spying Prevention
Based on that, OPPO decided to apply the AI-enhanced smart sensor to the field of smart spying prevention, which is constructed by utilizing a low power consumption sensor in combination with an intelligent recognition algorithm. The sensor could not only be used in privacy protection, but also could extend the scenarios in Smart AirControl, Smart Rotation, and Smart Always On. Many high-end smart home devices have equipped smart spying prevention technology as an auxiliary function. For instance, high-end home security monitoring equipment leverages DVS technology to capture only the outlines of people in its range - helping protect the privacy in living spaces. Similar with that, when your Reno4 receives a message, AI-enhanced smart sensor on the phone could identify whether only the mobile phone's owner is watching the screen. If not so, the content of the notification would be hidden to protect the user’s privacy.

To be specific, the setting of the entire recognition logic makes the spying prevention more humane and comprehensive, which is a great companion to its advanced hardware. The sensor will first probe how many faces are in the scale.
• If only 1 face is detected, the main camera will be turned on to identify whether the face is of the owner him/herself; if it is, then it displays the entire message.
• If the sensor detected that there is more than one person (or no person) facing the display, then the notification will still be shown but the content will not be shown.

A smart sensor that can do much more – innovation improving user experience

The AI-enhanced smart sensor technology also introduced advanced AI capability to Reno4, which enables some futuristic features such as touch-free control and smart rotations. In recent years, gesture recognition has become another popular way of human-computer interaction and has been widely used in many fields, especially in the automotive field in smart cars. With the continued trend of marginal innovation in smartphone industry, gesture control may become an inevitable function of smartphones. OPPO joins this race with the AI-enhanced smart sensor. OPPO believes that such innovative and optimal functional user experiences are inspirational to the mobile phone AI assistant race.

• Smart AirControl
Reno4 is the first OPPO phone to bring full-screen gestures to their line-up. The Smart AirControl allows you to interact with Reno4 without touching the display and could perform tasks such as scroll up and down the pages on some popular apps like YouTube, Facebook, Instagram and TikTok. More conveniently, in a short period of time, the owner does not need to do a “starting up” gesture every time he or she scrolls the screen. It is also worth mentioning that this feature allows you to answer the call without touching your phone, which brings great convenience to those who work with plastic gloves most of the time.

• Smart Rotation
For most of us, especially the ones who prefer operating their phone while lying in bed, it is inconvenient when the content on the phone automatically flips to vertical when the Auto Rotate function is on. Based on this user insight, OPPO also introduces Smart Rotation to Reno4 with AI-enhanced smart sensor. When the phone and your face is rotated simultaneously, the AI-enhanced Smart Sensor and pose-recognition algorithm intelligently detect the orientation of your face and the phone to help determine whether to rotate the screen. There are some of the rotation scenes you often encounter:

When the phone is rotated from vertical to horizontal：
• If the face stays still - for example if you are watching a video and rotate your Reno4 - the screen will rotate.
• If the face rotated with the phone (such as lay in bed), the screen will not be rotated Surely this function could offer users a more immersive experience in various lifestyle scenarios in a smart manner.

• Smart Always On Display
Making full use of the characteristics of this AI-enhanced Smart Sensor, OPPO also applies functions to enrich user experience, such as “Smart Always On Display”. This feature can identify whether a user is continuously watching the mobile phone, so that the screen will not turn off. This feature makes Reno4 more intelligent and provides better using experience.

In addition, on software level, a special brightening algorithm will make the sensor work better in low light. Even when the level of luminance is lower than 1 lux, Reno4’s AI-enhanced Smart Sensor can function well.

The future of technology - convergent products

The development of mobile phones can be roughly divided into the feature phone era and the smartphone era. In recent years, it has entered the evolutionary stage of artificial intelligence era. What sets two eras apart is the ability to solve problems using new and improved solutions. AI is a very broad concept, and R&D for AI in smartphones means to identify and develop new solutions that provide human beings with more convenience in terms of communication.

Digital experiences enabled by AI assistance, especially virtual assistance, is considered to be one of the major recent technological advancement and most promising consumer trend. The form we are most familiar with is virtual assistant, such as Microsoft’s Xiaoice or Cortana. The AI-enhanced smart sensor and its Smart AirControl and privacy protection functions have brought new possibilities to smartphone AI assistant. More recently, the future in smartphone features - AI assistants - may become more invisible and subtle. Finding a balance between easy operation and false touch, Smart AirControl is relatively easy to operate, avoiding false touch, it is an attempt and exploration in the field of human-device interaction. Even you could be like Iron Man and have your very own “Jarvis” or “Friday”. A futuristic AIOT form could be foreseen, to replace operating the devices with your phone or remote control.

To further enhance user experience, OPPO put large amount of effort in the high-end hardware and software. Strategically, with rapid development of emerging technology, OPPO has stepped into the Era of Convergence, in which OPPO is not placed only as a mobile device company. It will also deep dive into the whole technology world to create convergent products for all.