GMSL2 Camera
Hardware
AutomatePro can be ordered with 0, 2, or 4 GMSL2 (Gigabit Multimedia Serial Link) camera inputs. GMSL2 is a high-speed, low-latency camera link commonly used in automotive applications. It supports cable lengths up to 15 m and Power over Coax (PoC). AutomatePro can supply up to 12 W total to downstream GMSL2 cameras.
Recommended cameras are listed in the camera accessories. The number of populated FAKRA Z connectors depends on the ordered device configuration. The AutomatePro software supports up to two cameras, cam1 and cam2.
Onboard GMSL2 Specifications
| Parameter | Value |
|---|---|
| Onboard IC | MAX9296AGT |
| Standard | GMSL2 |
| Max cable length | 15 m |
| Max data rate | 4.5 Gbps |
| PoC voltage | 12 VDC |
| FAKRA type | Type Z (Water Blue) |
| Max power | 12 W total |
ROS API
Nodes: automatepro_cam1_node, automatepro_cam2_node
Publishers
Replace <camera_name> with the configured camera name, usually cam1 or cam2.
| Topic | Type | Description |
|---|---|---|
/camera/<camera_name>/image_raw | sensor_msgs/msg/Image | Raw camera frames in bgr8 encoding. |
/camera/<camera_name>/camera_info | sensor_msgs/msg/CameraInfo | Camera calibration and projection information. |
/camera/<camera_name>/h264/video | foxglove_msgs/msg/CompressedVideo | H.264 compressed video frames. |
/camera/<camera_name>/h264/calib | foxglove_msgs/msg/CameraCalibration | Calibration metadata for the compressed stream. |
Debug Service
| Service | Type | Description |
|---|---|---|
/automatepro_cam<N>_node/disable_camera_inputs | std_srvs/srv/SetBool | Debug-only service for simulating camera input loss and recovery behavior. |
Verify Cameras
Run these checks from AutomatePro over SSH:
source /opt/ros/humble/setup.bash
ros2 node list | grep automatepro_cam
ros2 topic list | grep '/camera/'
For a single-frame check:
ros2 topic echo /camera/cam1/camera_info --once
If camera compression is enabled, verify the H.264 stream:
ros2 topic hz /camera/cam1/h264/video
Configuration
Cameras are exposed as V4L2 devices such as /dev/video0 and /dev/video1. The AutomatePro camera driver uses these devices through GStreamer and publishes ROS 2 image and compressed-video topics.
Driver: automatepro_camera_driver
Container: automatepro-core-driver
Runtime ROS configuration is stored in /opt/automatepro/config/ros. Packaged defaults are stored in /opt/automatepro/defconfig/ros.
Edit the camera parameter file:
sudo nano /opt/automatepro/config/ros/camera_params.yaml
Important parameter groups:
| Parameter | Default | Description |
|---|---|---|
enable | true | Not used by the AutomatePro services; ROS_DRIVER_CAM_1 and ROS_DRIVER_CAM_2 in /opt/automatepro/.env select the camera nodes. |
camera_name | cam1, cam2 | Name used in camera topic namespaces. |
frame_id | cam1, cam2 | Frame ID stamped on camera messages. |
resolution | [640, 512] | ROS output resolution. |
qos_history_depth | 1 | Queue depth for raw image publishers. |
qos_reliability | reliable | Raw image QoS reliability, either reliable or best_effort. |
qos_durability | volatile | Raw image QoS durability, either volatile or transient_local. |
camera_info.* | Identity calibration by default | Calibration values published on camera_info. |
compression.enable | true | Enables H.264 output. |
compression.encoder | auto | Selects the encoder: auto uses the NVIDIA hardware video encoder (NVENC) when available and falls back to CPU encoding, nvenc forces nvv4l2h264enc and fails to configure without it, and x264 forces x264enc. |
compression.bitrate_kbps | 5000 | Target bitrate in kbps. Set to 0 for automatic bitrate calculation. |
compression.bits_per_pixel | 0.1 | Used for automatic bitrate calculation when bitrate_kbps is 0. |
compression.resolution | [640, 512] | Compressed stream resolution. |
compression.frame_rate_fps | 30 | Compressed stream frame rate. |
compression.qos_* | 1, best_effort, volatile | QoS settings for the compressed-video publishers. |
rtp.enable | false | Sends the H.264 stream over RTP/UDP when enabled. |
rtp.ip | 192.168.20.100 | RTP destination IP address. |
rtp.port | 5000 / 5001 | RTP destination UDP port. |
watchdog.timeout_ms | 2000 | Time in milliseconds without frames before recovery starts. |
watchdog.cycle_time_ms | 500 | Watchdog check interval in milliseconds. |
recovery.gmsl2_escalation_threshold | 1 | Pipeline recovery attempts before GMSL2 restart escalation. |
recovery.gmsl2_restart_wait_s | 10 | Wait time in seconds after a GMSL2 restart. |
recovery.window_s | 60 | Time window in seconds for counting recovery attempts. |
recovery.max_total_attempts | 5 | Recovery attempts before the camera node gives up. 0 means unlimited. |
recovery.retry_delay_s | 5, 7 | Seconds between recovery attempts; the first attempt is immediate. cam2 waits longer so both cameras do not recover at the same time. |
logging.level | info | Driver log level. |
logging.throttle_ms | 10000 | Throttle interval for repeated debug logs. |
publish_rate_hz | 0.0 | Raw image publish rate limit in Hz. 0.0 publishes every frame; H.264 output rate is controlled by compression.frame_rate_fps. |
sensor.device | /dev/video0, /dev/video1 | V4L2 device. Normally fixed by hardware. |
sensor.resolution | [1920, 1536] | Sensor capture resolution. Normally fixed by hardware. |
sensor.frame_rate_fps | 30 | Sensor capture frame rate. |
sensor.format | UYVY | Sensor pixel format. Normally fixed by hardware. |
sensor.presence_check.enable | true | Configures the camera only when its GMSL2 serializer answers on the link. |
sensor.presence_check.i2c_bus | /dev/i2c-9 | I2C bus of the serializer check. |
sensor.presence_check.serializer_address | 96, 98 | Serializer I2C address (0x60 for cam1, 0x62 for cam2). |
sensor.presence_check.timeout_ms | 2000 | Time in milliseconds the serializer is probed before the camera counts as absent. |
sensor.unfilled_frame_check.enable | true | Drops frames whose last row is entirely pixel_bgr, which marks an unwritten capture buffer. |
sensor.unfilled_frame_check.pixel_bgr | [0, 136, 0] | Pixel value of an unwritten capture buffer. |
Default Config
The default file includes one section for each camera node. The most commonly edited values are shown below:
automatepro_cam1_node:
ros__parameters:
enable: true
camera_name: "cam1"
frame_id: "cam1"
resolution: [640, 512]
camera_info:
distortion_model: "plumb_bob"
d: [0.0, 0.0, 0.0, 0.0, 0.0]
k: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
r: [1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0]
p: [1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0]
compression:
enable: true
encoder: "auto"
bitrate_kbps: 5000
bits_per_pixel: 0.1
keyframe_interval_frames: 30
resolution: [640, 512]
frame_rate_fps: 30
rtp:
enable: false
ip: "192.168.20.100"
port: 5000
watchdog:
timeout_ms: 2000
cycle_time_ms: 500
recovery:
gmsl2_restart_wait_s: 10
gmsl2_escalation_threshold: 1
window_s: 60
publish_rate_hz: 0.0
# Do not change the sensor block unless Lemvos support instructs you to.
sensor:
device: "/dev/video0"
resolution: [1920, 1536]
frame_rate_fps: 30
format: "UYVY"
automatepro_cam2_node:
ros__parameters:
enable: true
camera_name: "cam2"
frame_id: "cam2"
resolution: [640, 512]
# Same structure as cam1. Defaults use /dev/video1 and RTP port 5001.
sensor:
device: "/dev/video1"
resolution: [1920, 1536]
frame_rate_fps: 30
format: "UYVY"
Restart automatepro-core-driver after changing the configuration. Configuration values are loaded when the driver container starts and are only applied after the service restarts.
sudo systemctl restart automatepro-core-driver
Enable or Disable Camera Drivers
To disable the AutomatePro camera driver and use a different V4L2, OpenCV, GStreamer, or custom camera stack, edit /opt/automatepro/.env:
sudo nano /opt/automatepro/.env
Set the relevant flags to 0:
ROS_DRIVER_CAM_1=0
ROS_DRIVER_CAM_2=0
The same file sets the camera type with GMSL2_CAM_TYPE: SG3S-ISX031C-GMSL2F, the packaged default, uses the 3 Gbps link rate, and SG3S-ISX031C-GMSL2 does not. automatepro-gmsl2.service reads it when it starts.
Apply the change:
sudo systemctl restart automatepro-core-driver
Now you can use the cameras with any standard V4L2 driver, OpenCV, GStreamer, or another camera driver. For more details on hardware acceleration and GStreamer, see the NVIDIA Jetson multimedia documentation.
Troubleshooting
Camera Topics Are Missing
-
Check whether the camera driver is enabled:
grep ROS_DRIVER_CAM /opt/automatepro/.env -
Check the services:
systemctl status automatepro-gmsl2 automatepro-core-driver -
Inspect recent logs:
sudo journalctl -u automatepro-gmsl2 -n 100 --no-pager
sudo journalctl -u automatepro-core-driver -n 100 --no-pager
Restart Camera Hardware
If the camera hardware needs to be reinitialized:
sudo systemctl stop automatepro-core-driver
sudo systemctl restart automatepro-gmsl2-reload
sudo systemctl start automatepro-core-driver
Automatic Recovery
The camera driver includes a watchdog. If frames stop arriving, it first attempts driver-level recovery. If hardware-level recovery is required, the driver sets recovery.reset_fail=true; the AutomatePro Driver Manager watches that parameter and triggers GMSL2 restart through the AutomatePro IPC path and systemd services.
Verify that the driver manager is running:
ros2 node list | grep automatepro_driver_manager
Automatic GMSL2 hardware recovery requires ROS_DRIVER_MANAGER=1 in /opt/automatepro/.env and the automatepro_driver_manager node must be running.
Developer Notes: launch and recovery internals
automatepro_bringup starts camera nodes with core_driver.launch.py. Camera 1 and Camera 2 are controlled by the launch arguments enable_cam1 and enable_cam2.
The driver manager discovers camera nodes matching /automatepro_cam.*, checks lifecycle state, reads the recovery.reset_fail parameter, and creates a restart request under /opt/automatepro/ipc/ when a camera node is unconfigured with recovery.reset_fail=true. The automatepro-gmsl2-ipc.path unit triggers automatepro-gmsl2-reactor.service, which restarts automatepro-gmsl2-reload.service.