Application overview
Bird's Eye View (BEV) is one of the fundamental building blocks for L2/L2+ sensor fusion systems in autonomous driving applications. Modern advanced driver assistance systems (ADAS) and autonomous vehicles require a comprehensive understanding of the vehicle's surroundings. Traditional approaches run separate perception pipelines for each camera and merge results after inference. This process compounds latency, increases memory overhead and can also introduce inconsistencies at camera boundaries. The BEV approach instead fuses multiple camera inputs into a single unified spacial representation before object detection which allows a single neural network inference with a lower overall system cost.
Real-time BEV requires substantial processing power to handle multi-camera input, perform geometric transformations, execute neural network inference for object detection and maintain low latency. TDA processors are designed specifically for these types of multicamera workloads and are equipped with the C7™ NPU and the interfaces needed to implement BEV with high performance.
The processor SDK includes the FastBEV demo which can be run out of the box as the initial step during evaluation before developing a custom solution. The demo enables the fusion of the image feeds from 6 cameras and performs concurrent object detection, creating a unified "bird's-eye" perspective. Importantly, BEV perception here is purely vision-based. Lidar point cloud data is used exclusively as a ground truth reference for validating detected objects, not as a live input to the perception pipeline.
Starting evaluation
Data collection
Sensor fusion BEV systems for L2/L2+ applications require calibrated multi-camera setups with full coverage around the vehicle.
Data collection should cover various driving scenarios including different lighting conditions (daytime, nighttime, shadows), weather conditions, road types, traffic densities and the presence of various vehicles, pedestrians and obstacles relevant to autonomous driving.
Lidar point cloud data serves as ground truth for validating detected objects in the BEV representation during development and evaluation, it is not a required input to the deployed perception pipeline. If lidar ground truth is used, synchronized capture of camera frames and lidar scans is essential for accurate validation.
For AI-enhanced BEV systems with concurrent object detection, labeled training data should identify vehicles, pedestrians, cyclists, lane markings, road boundaries and other relevant features critical for autonomous driving functions.
Data quality assessment
Camera calibration accuracy is critical for the perspective-to-BEV view transform. This relies on two categories of parameters:
Intrinsic parameters: Describe the internal optical properties of each camera sensor and lens. This includes things like focal length, principal point and distortion coefficients. These govern how a 3D point in the world projects onto a 2D image plane within a single camera.
Extrinsic parameters: Describe the position and orientation of each camera relative to a shared BEV coordinate system.
Both are fundamental to accurately mapping depth and spatial position from 2D image frames. Errors in either will propagate directly into the spatial accuracy of detected objects in the BEV representation.
Build and train your model
Pre-trained BEV models are available in the TI Model Zoo that are ready for evaluation and deployment.
Developers may also wish to prepare their own BEV model. If so, one could leverage edgeai-mmdetection3d, a unified framework for the development, training and evaluation of various BEV 3D detection models, built on top of mmdetection3d. Alternatively, advanced developers may choose to use their own custom training environments.
Regardless of how the model is trained, before deployment it must be compiled for TIDL to run on the C7™ NPU. Models in ONNX or LiteRT formats can be compiled using edgeai-tidl-tools which will optimize the network for TI's deep learning accelerators.
Find the right model for your needs
Model selection should balance detection accuracy with latency constraints. The TI Model Zoo and benchmark tools provide performance data to guide selection based on target hardware.
The model used in TI's FastBEV demo uses a ResNet-18 backbone + FPN architecture and is based on the open source Fast-BEV model, which was specifically designed to be capable of performing faster BEV perception on the on-vehicle chips. The model was trained and validated with the nuScenes dataset.
Deploying your model
The FastBEV demo showcases the typical deployment of a BEV model using the Vision Apps framework included as part of the SDK for TDA processors.
Core perception pipeline: The OpenVX-based pipeline integrates 6-camera capture, MSC hardware scaling, A72 pre-processing, TIDL inference on C7™ NPU and A72 post-processing.
Lidar ground truth integration: In addition to the core pipeline, lidar point cloud data is ingested in parallel and overlaid on the BEV output as a spatial reference. This allows developers to visually compare detected objects against a ground truth during evaluation.
Additional pipeline features: The demo also includes real-time performance monitoring and a dual-view display rendering detected objects in both the individual camera image and the top-down BEV perspectives.
Choosing the right device for you
Device selection for L2/L2+ Birds eye view (BEV) systems depends on the number of cameras, resolution, frame rate requirements and concurrent AI processing needs. These applications benefit from automotive-grade processors in the TDA4x family with high memory bandwidth, dedicated vision accelerators (VPAC), multiple C7™ NPU cores for AI acceleration and the capability to split networks across cores for optimal performance.
| Product number | Processing core | NPU performance | Birds eye view benchmarks | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
fastbev_plus_nms_r18_f1 (256x704) Performance |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| TDA4VH-Q1 | 8x Arm® Cortex®-A72 + 4x C7™ NPU | 32 TOPS | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| TDA4VPE-Q1 | 4x Arm® Cortex®-A72 + 2x C7™ NPU | 16 TOPS | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| TDA4VL-Q1 | 2x Arm® Cortex®-A72 + 1x C7™ NPU | 8 TOPS | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
FPS (Frames per second)
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
All the hardware, software and resources you’ll need to get started
Hardware
TDA4VH-Q1
The TDA4VH is a high-performace, functional safety-compliant, System-on-Chip (SoC) designed for advanced ADAS, autonomous vehicles and robotics. It features 32 TOPS of AI performance, quad C7x DSPs, eight Arm Cortex A72 cores, dedicated vision accelerators and high-speed networking.
J784S4XEVM
The J784S4 evaluation module (EVM) is a platform for evaluating the TDA4VH-Q1 SoC in vision analytics and networking applications throughout automotive and industrial markets. These processors perform particularly well in multicamera, sensor fusion and advanced driver-assistance system (ADAS) domain-control applications.
Software & development tools
Command-Line tools
Command-line tools like edgeai-modelmaker for training and edgeai-tidl-tools for custom-models can be found from the main MPU developer tool pages on the TI GitHub.
PROCESSOR-SDK-J784S4
The processor SDK RTOS and either processor SDK Linux can be used together to implement various use cases in robotics, vision and automotive advanced driver-assistance systems (ADAS).
CCStudio™ Edge AI Studio
This tool contains tools for training, compiling and deploying a model to TI edge AI processors. A model selection tool is available to view pre-generated benchmarks of popular models.
Supporting resources
The Vision Apps framework included in the Processor SDK provides the FastBEV demo as a reference implementation. This user guide provides a brief overview and instructions for running the FastBEV demo.
Development, training and optimization of bird's eye view 3D detection models for TI devices.
A unified framework for the development, training and evaluation of various BEV 3D detection models, built on top of mmdetection3d.
Industrial | Vision
Find and localize specific objects, people in real-time at high framerate with AI-accelerated processors and industry-standard software
Industrial | Vision
Detect fine-grain obstacles and pathways from imagery in real-time using C7™ NPU equipped processors