3 Smart Camera Use Cases

Applications of smart cameras and computer vision span many domains including industrial, automotive, consumer, and public security. The application requirements and constraints depend on the use case, many of which include reporting an event, such as a security breach or intrusion, over the network to a cloud server. Computing at the edge reduces the impact on the network, yet these applications still often require video encoders, for example,H.265, to limit bandwidth usage when there is need to uplink video data.

Choosing the appropriate image sensor, such that, camera, is important to developing a robust smart system. Key parameters include resolution, frame rate, bit depth, pixel-size, etc. A home security camera may use a 5MP sensor giving 30 frames per second (fps) with rolling shutter whereas an infrastructure camera for tollways may require 2MP with 60fps and global shutter to capture plates on fast-moving vehicles, and a machine vision camera may need 2MP greyscale with 90 fps and global shutter to identiy defects on parts rapidly moving along an assembly line. Image sensors can include an image sensor processor (ISP) that internally preprocess images into a typical format like JPEG or YUV. However, many sensors do not include an ISP to reduce sensor cost and provide freedom in selecting an external ISP that can be more readily tuned than one within the sensor. Selecting a processor with its own integrated ISP, like those in the AM6xA family, gives the benefit of an external ISP, yet with simplified PCB design, lower BOM cost, lower end-to-end latency, and reduced DDR usage.

Applications also have varied computation requirements due to computer vision and machine learning complexity. Machine learning tasks like image classification requires fewer resources than object or keypoint detection, which identifies specific objects like people or vehicles, and their locations. More complex tasks like pixel-level segmentation require even more resources as each pixel is classified as part of an object or region ,such as the current lane for a lane detection algorithm in ADAS applications. Some applications may require multiple models. Increasing the resolution also dramatically increases the processing requirements. AM6xA processors include deep learning acceleration hardware to offload these compute-intensve tasks.

Table 3-1 lists end equipments for AM62A with ranges for required specs/features (min camera resolution, FPS range, low-med-high ML complexity, video enc/dec).