ISP SDK
The topic provides basic information about the Image Signal Processing (ISP) block in JH-7110 and the associated ISP drivers developed under the Linux OS.
Function Introduction
The ISP processor provides algorithms and real-time image information feedback. Since some parameters of the algorithms are initially uncertain, the algorithms must work with adaptive control methods in firmware. The adaptive control method obtains the image information, recalculates related parameter values such as the exposure time and gain, and estimates the parameter values required by the next frame to control the sensor, and ISP logic. This ensures that the image quality is automatically adjusted.
The light signals onto the photosensitive area of the sensor. After photoelectric conversion, the sensor transmits Bayer raw images to the ISP. The ISP processes the images based on related algorithms and outputs the images in the Bayer/YUV spatial domain to the firmware and display module. During this process, the ISP controls the sensor by using the firmware and implementing the functions including Automatic Exposure (AE), and Automatic White Balance (AWB). The firmware is driven by the interrupts of the ISP module. The StarFive Tuning Tools adjust the quality of the online images of the ISP over the Ethernet port.
ISP SDK Architecture
| Component | Description |
|---|---|
| IOCTL Library | Provides the ISP register read/write and buffer
allocate/release functions to service the module unit. IOCTRL objects are contained in the libCI_User.a file. |
| Device Library | Provides the basic device operations. For example, open,
close, ioctl, initiate, allocate buffer, and release
buffer. Device objects are contained in the libISPC.a file. |
| Control Library | Includes the 3A algorithm and other basic algorithms for
adaptive control. For example, AE/AWB/SAT. These adaptive
controls are based on the ISO and color temperature. Control objects are contained in the libISPC.a file. |
| Module Library | Provides the parameter conversion and the ISP register
program. Module objects are contained in the libISPC.a file. |
| Sensor Library | Provides the exposure time and sensor gain calculation and
program. Sensor objects are contained in the libSensorapi.a file. |
| Pipeline Library | Provides the device/control/module management. Pipeline objects are contained in the libISPC.a file. |
| ISP Control Application (stf_isp_ctrl) | Monitors the video device stream on/off and establishes/tears down the pipeline for the ISP adaptive control. |
ISP Control Application Workflow
The ISP Control application invokes the ISP SDK API to make the ISP hardware keep processing every stream automatically. It uses the SDK library to operate the ISP mainly in the following steps:
- Fetch the statistical collection data.
- Pass the statistics data to registered controls for the adaptive control.
- Loop registered modules to convert the high-level parameters to ISP register values.
- Send these register values to kernel driver through the IOCTL command.
- Program the ISP.
- Return the statistics buffer to video device after completing the registers update.
