ISSUE

Schneider Electric’s Innovation Division is in charge of innovating, as it name states, by researching a variety of promising technologies. With this in mind, Schneider Electric looked into the subject of artificial intelligence, augmented reality, and the use of smartphones. The aim was to find out how the company could use and combine these technologies. Very quickly, a project around the identification of products marketed by Schneider Electric emerged.

CUSTOMER

SCHNEIDER ELECTRIC INDUSTRIES Grenoble(FRANCE)

SOLUTION PROVIDED

Neovision has designed and developed a mobile application using a technology combining Deep Learning, Computer Vision, and Augmented Reality. This application enables the detection and identification of Schneider Electric circuit breakers on a video stream in real time. After having recognised and positioned the different drawings of the room, an image detector locates the circuit breakers and our OCR applies itself to read the reference on the label. Following the video analysis of an installation, SE has a list of the products installed.

CUSTOMER BENEFIT

Schneider Electric’s Innovation Division has a multifunctional mobile application now. Incorporating an object detector and OCR technology, the uses linked to this application are as numerous as they are varied: creation of digital twins, automatic entry of references, visual search in the product catalogue, better knowledge of the stock and recommendation of suitable products… Only our imagination seems to be able to restrict its uses.

REALIZATION

Schneider Electric is an international group, with 135,000 employees, and above all, the embodiment of a desire to innovate on a global scale. A leading specialist in its market, its activity is based on products relating to electrical distribution and automation. As a forward-thinking company, it has been quick to commit to energy efficiency and sustainability. Preferring to act rather than react, Schneider Electric is making a strategic shift, moving increasingly towards a digital services business.

In this logic of developing new digital services, all related technologies have a place of choice. This is of course the case with the Smartphone, Artificial Intelligence, and Augmented Reality.

These questions were answered when Romain Gassion, Innovation Project Manager at Schneider Electric, discovered Neovision through the Grenoble startup ecosystem. Having already worked on other projects with Schneider Electric, Neovision was known to the company and the first contacts were made very quickly.

At the time, Romain and his team wanted to experiment. To experiment with the use of artificial intelligence to recognise the products manufactured and marketed by Schneider Electric. The challenge was well within Neovision’s grasp. However, a first constraint quickly emerged: the solution had to take the form of a mobile application. This was quite logical, since many people, if not everyone, has a smartphone, and aiming for a mobile deployment would simplify the production of this digital service. The second was a bit more challenging. Our technology had to be embedded directly into the application and therefore run on the smartphone, as connectivity is not always present in the field. Product detection and recognition must therefore be done in real time on the video stream without the need for a network connection. Although experimental, the foundations of the project were laid and it was taking shape. The Neovision team could get to work!

Algorithms and models

With the deployment targeted and the database built, the Neovision team could work on the famous model to be integrated into the application. As a reminder, the task at hand is the detection and recognition of Schneider Electric circuit breakers while ensuring good 3D tracking. Thus, our experts set their sights on a model from the Pytorch framework with a custom backbone via learning parameters. This model, which is as fast as it is powerful, looks to define the targeted objects in a single point, and this is its main interest. While some circuit breakers are isolated and large, others are much smaller and arranged next to each other: the Acti9. It was therefore necessary to be able to differentiate between them. By defining it with a single dot, detection is more precise.

Once trained, the model delivered its results, and to say the least, they were quite conclusive, even more when the generated images were included in the training set. The addition of these images improved the detection score of the different classes from 3 to 26 points!

This was a new challenge faced by the Neovision team. Running Deep Learning models directly on a smartphone requires real expertise. First of all, the right model must be created, but also, and above all, optimising the latter, without the competent skills, would be impossible to execute.

OCR integration

With the detection and tracking model ready, all that remained was to integrate Neovision’s OCR technology. To do this, a new algorithmic step allows us to isolate an area of interest on the image: the label. It is then, on this new image of the label, that our OCR extracts the reference of the circuit breaker to formally identify it. Learn more about our OCR technology.

Schneider Electric has now a mobile application that includes an intelligence solution combining several technologies: 3D tracking, object detection, and automatic character reading (OCR). Moreover, and this is where the interest of the project lies, the uses (automatic reference entry, visual search to access technical data sheets, recommendations of similar products, creation of digital twins, etc.) are as numerous as the potential users (technicians, sales representatives, customers, etc.)

TECHNOLOGIES & ASSOCIATED EXPERTISE

DEEP LEARNING, COMPUTER VISION, EDGE COMPUTING, SMARTPHONE, AUGMENTED REALITY (AR)

TESTIMONIAL

« As part of an innovation process, we asked Neovision to help us develop an algorithm that would allow us to recognize our equipment using photos. Throughout the project, Neovision used its expertise in the field of image recognition to help us define our needs, and then developed a solution that would allow us to demonstrate technical feasibility. The prototype we ended up with allowed us to convince internally of the relevance of the subject and triggered a ‘wow’ effect. We really appreciated our collaboration with the Neovision team because they quickly identified our needs and were able to react to the different changes we had to make. All this was done within the framework of a very friendly and rewarding relationship. »

Romain Gassion, Innovation Project Manager at Schneider Electric