ISSUE

The recent trend is to embed deep learning algorithms. Following this trend, STMicroelectronics wanted to demonstrate that it is possible to embed such algorithms in its products, including a component dedicated to neural network acceleration. To this end, the objective was to develop a functional embedded AI technology. While taking into account many constraints: limited memory, low computing power, adaptation of a recent model, acquisition of a large image base but also very limited time …

CUSTOMER

STMICROELECTRONICS  Grenoble (FRANCE)

SOLUTION PROVIDED

After advising STMicroelectronics in the design of a neural network gas pedal, Neovision developed an algorithm for detecting and counting people embedded on a low-power microcontroller. The solution was then demonstrated at Mobile World Congress 2018 in Barcelona at the STMicroelectronics booth.

CUSTOMER BENEFIT

Throughout the project, Neovision and STMicroelectronics collaborated to develop a working ecosystem where the complementary expertise of Neovision and STMicroelectronics was put to best use. That’s why the final demonstration at the MWC was a success. As a result, ST was able to demonstrate the performance of its product, which is capable of embedding deep learning algorithms for real-world applications. This project then gave rise to a partnership between the two companies. 

REALIZATION

STMicroelectronics is a leading global manufacturer of innovative semiconductor components. Sensing a paradigm shift looming, they wanted to develop a product that could embed deep-learning algorithms. But the goal was to be able to run a concrete and real demonstration through their hardware innovation. Nevertheless, many constraints had to be taken into account.

The first issue consisted in gathering a large database that could be used by deep learning algorithms. Indeed, the available databases did not target infrared. To solve this problem, Neovision provided its expertise and advice on the method of infrared image acquisition. The latter would then be used to learn the model. In addition, STMicroelectronics provided the human and material resources needed to produce a diverse, high-quality database.

The deep learning model selected was based on the latest advances in science. Indeed, it used the latest algorithms from YOLO. An algorithm recognized for its speed of calculation applied to the detection of humans in the visible domain. However, Neovision had to use a “transfer label” technique. This technique allows to transpose a label from RGB images to infrared images. This model was thus able to process infrared images with the embedded constraints (little memory available, limited computing power).

Once the model was trained on the basis of captured images, it was converted into an OpenVx graph. It was then used to describe image processing algorithms in STMicroelectronics’ microcontroller framework.

Even so, the demonstrator was ready for the MWC 2018, and the MWC 2018 made quite an impact and garnered praise.

Building on this successful first collaboration, the relationship between STMicroelectronics and Neovision was set to grow even stronger. STMicroelectronics recognized the added value of Neovision and included Neovision in its partner network. Neovision became an Authorized Partner of STMicroelectronics.

ASSOCIATED TECHNOLOGIES AND EXPERTISE

COMPUTER VISION, DEEP LEARNING, EMBEDDED AI, R&D

TESTIMONIAL

« Embedded AI will allow the realization of real-time applications, make objects autonomous, and allow for better preservation of confidential information by not exposing them in the cloud. Embedded AI will offer fantastic prospects in all areas, and now with the silicon integration capabilities offered by current technological processes, semiconductor manufacturers, including STMicroelectronics, are able to offer processors that enable very significant computing power to be embedded with very attractive energy balances. However, processors are only part of the solution, and the vast majority of customers wishing to develop an intelligent application lack an essential skill, which not only allows them to define the right algorithms, but also to know how to optimize them by making the most of the available computing resources: this is the skill that Neovision offers, and which you and your colleagues have demonstrated by helping us implement, in record time, a Smart Office demonstration that we wanted to carry out to present the capabilities of our configurable IP enabling the implementation of neural networks. This demonstration was indeed a success, in particular because it demystified the subject and showed how a particular problem could find an accessible solution! Neovision certainly has a key role to play in the development of the ecosystem that will enable the deployment of embedded AI, and ST welcomes the collaboration initiated at MWC18 »

Thierry Bauchon, Director of the STMicroelectronics site in Grenoble, France