Snapdragon Ride SDK: a premium platform for developing customizable ADAS applications

Almost everything is becoming more intelligent and more connected - and automobiles are no exception. The Snapdragon Ride Platformis at the forefront of this intelligent and connected automotive transformation. Since its introduction at CES 2020, Snapdragon Ride has gained momentum with global automakers and Tier 1 suppliers worldwide. At our recent annual Snapdragon Tech Summitin Hawaii, we announced new collaborations with BMW and Cadillac - two luxury automotive companies poised to lead the intelligently connected automotive revolution.

The expanded product roadmap for Snapdragon Ride makes it a truly advanced solution - a scalable, fully customizable ADAS platform - ready to transform the automotive industry. And now, the Snapdragon Ride Software Developer Kit (SDK) helps automotive OEMs and developers build safety-certified applications for perception and drive policy on top of the Snapdragon Ride Platform.

Introduced in January 2020, the Snapdragon Ride platform is a car-to-cloud connected ADAS infrastructure that inlcudes:

• low-power SoC platforms comprising of high performance compute and AI engines with functional safety built in;
• out-of-the-box flexible vision solutions supporting front and surround camera;
• a rich tool set to enable simulation and continuous learning frameworks. The DNN used in the device can be uploaded to the cloud for faster training. Afterwhich, the newly retrained networks can be redeployed to the device from the cloud.

Snapdragon Ride allows for rapid creation of customizable and ASIL-compliant ADAS solutions across the entire product line from entry level (L1) to premium (L3) with direct support from Qualcomm Technologies.

The Snapdragon Ride SDK is designed to provide modules that deliver abstractions to the underlying Snapdragon HW blocks, as well as the driver software that maximizes KPIs like latency, memory, and bandwidth usage. This allows application developers to focus on custom application logic, and quickly develop and deploy applications with modules that are pre-optimized for performance and latency.

Qualcomm ADAS SDK and the Snapdragon Ride hardware platform

Click to see a larger image

A key ingredient of the Snapdragon Ride hardware platform is the Snapdragon Ride SoC. Designed to address the needs of the ADAS ecosystem, it has hardware blocks that enable ADAS applications such as:

• machine learning and vision processing blocks for neural network-based perception
• a vision accelerator for image pre- and post-processing applications
• graphic accelerators for visualization and parallel processing GP-GPU support
• a video processor for streaming media support
• a system safety manager to ensure ISO26262 compliance for applications
• multi-core ARM-based CPU for running general purpose drive policy applications

Click to see a larger image.

The Snapdragon Ride hardware platform comes with extensive hardware and driver support for the most common sensors used in ADAS platforms including cameras, lidar, and radar.

The Snapdragon Ride SDK software package provides optimized libraries and software that are tuned to work with the ADAS SOC and hardware.

SDK Software package

Click to see a larger image

The Snapdragon Ride SDK ships with a comprehensive set of tooling, documentation, libraries, and sample code that allow developers to leverage the Snapdragon Ride hardware and SoC quickly and develop custom applications.

Developers should be able to install the docker image on an x86 host and build out the target board support package that contains the modules, libraries, and application software relevant to their deployments.

Board support package (BSP)

The Snapdragon Ride board support package ships with a base set of sensor drivers for cameras and other specialized sensors/hardware blocks for ADAS that are part of the Snapdragon Ride hardware platform. These drivers are abstracted out into libraries for application use into a set of SDKs/Interfaces, allowing for application developers to focus on core application development. The platform supports various middleware, including Adaptive Autosar for safety monitoring, message passing, and more. Application developers are encouraged to use Adaptive Autosar-based interfaces so that applications are safety compliant.

Click to see a larger image

Automotive Imaging Systems Camera SDK

The Automotive Imaging Systems (AIS) Camera SDK provides abstractions over the camera drivers, allowing the capture of camera frames from a camera serial interface (CSI) input, into memory buffers (DDR). It is designed to abstract the underlying hardware blocks and provide a programmable interface for application developers.

Key features include:

• multi-camera and multi-client support
• timestamp and synchronization handling
• error handling and propagation
• extendable camera module plugin
• extendable driver APIs
• support for ISP processing functions like image scaling and color conversion

AI SDK

The AI SDK provides comprehensive tooling for compiling deep neural network graphs and running them efficiently on the GPU, CPU, or the Qualcomm Hexagon Tensor Processor (HTP) cores. It supports operators from multiple frameworks like pytorch, tensorflow, and caffe2 and network weights frozen in formats like ONNX and Frozen pb. The QNN compiler can take these files as input and generate a binary that can be compiled to run on the CPU, GPU or HTP. The QNN compiler supports compilation into INT8 precision mode for maximum efficiency and lowest latency on the HTP but mixed precision and FP16/FP32 modes are also supported on GPU and CPU. Plus, it supports extensibility using custom operators or compound operators where multiple simple operators can be combined. The AI model efficiency toolkit provides a library of techniques to combine and quantize trained models. Its primary purpose is to speed up model execution by conversion into INT8.

Click to see a larger image.

FastADAS APIs

Developer tooling

The Snapdragon Ride SDK includes a comprehensive toolkit designed to make it easier to develop applications rapidly and efficiently, so you can deploy and market applications in a short amount of time. The rich tooling support includes enhanced diagnostic and tracing capabilities to identify any bottlenecks quickly; profiling support to analyze and benchmark CPU, GPU, memory, power, thermal and network performance; scheduling tools to determine the best compute engines to run the applications; calibration tools to configure various sensors; and much more.

Snapdragon Ride SDK Developer Portal

The Snapdragon Ride SDK Development Portal provides a single web interface that provides users all the information needed to start development on the Snapdragon Ride platform. The Portal includes documentation for a platform user guide, SDK development guides, sample applications with source code, and tooling. In addition to documentation, the portal includes a downloadable package for developers that features a simple hands-on experience on the Snapdragon Ride platform, allowing you to quickly bring-up the platform, run sample code/apps, create, and run your own pipelines, benchmark different load scenarios for neural networks, and generate KPI reports.

How developers can leverage the SDK

The Snapdragon Ride SDK also comes with a comprehensive set of sample applications that allows developers to use in their own applications. While most of the sample applications use the optimized SDK libraries that are tuned to our hardware, in most cases, developers should be able to find sample applications that fits their needs and start with it as a baseline while developing custom applications. Some of the sample applications include:

1. sample application to run various complied neural networks on the platform and profile for KPIs like latency and utilization;
2. a single camera pipeline application to run sample lane or object detection networks that allows developers to use both AIS camera and the AI SDK toolkit and visualize outputs;
3. a multi-camera pipeline application that allows developers to work with multiple cameras and apply multiple networks in parallel;
4. a lidar pipeline sample with point cloud visualization output;
5. multiple sample applications that allow developers to test out various camera, lidar, and radar sensors that are supported by the platform;
6. sample applications that showcase post processing using Fast ADAS libraries.

Developers can leverage the SDK sample applications by replacing the sample neural network definitions (that are part of the sample applications) with their own custom neural network definitions and add custom post-processing logic to develop their own applications.

Snapdragon Ride SDK allows developers to truly focus on creating applications without worrying about optimizing or rewriting fundamental components found in common ADAS algorithms.

Snapdragon Ride hardware and SDK is currently available to customers for evaluating, prototyping, developing and finally productizing their next-generation ADAS platforms. Contact your local Qualcomm Technologies sales team for further information.

Redefining the car of the future

The era of the intelligently connected, autonomous driving car is upon us. The Snapdragon Ride SDK assists automotive suppliers and automakers to deploy virtually all the autonomous driving features they want now - with the ability to scale in the future. Our full suite of automotive products was developed with more than 30 years of mobile leadership and 20 years of automotive experience. The Snapdragon Ride Platform and SDK are supporting automakers to build cars that are connected to networks, the cloud, other cars, pedestrians, and traffic and infrastructure systems. Simply put: Snapdragon Ride is redefining the car of the future.

Learn more about Qualcomm's extensive automotive product portfolio.

Learn how Snapdragon Ride is moving autonomous driving forward.