Automotive LiDAR and RADAR Integration with MIPI CSI-2

Image courtesy: MIPI Alliance/Various

LiDAR and RADAR can use the MIPI Camera Serial Interface 2 (CSI-2) because CSI-2 is, at heart, a packet protocol with generic and user-defined data types, not only a pixel pipe. The sensor's front end or controller packs its samples, detections or point cloud into CSI-2 long packets, sends them over a D-PHY or C-PHY link (or over a SerDes such as MIPI A-PHY when the sensor is metres away from the processor), and the SoC receives them through the same CSI-2 receiver it uses for cameras. In production cars this is most common for imaging RADAR front ends that stream raw ADC data, while most automotive LiDARs today still ship their point clouds over Automotive Ethernet.

The MIPI CSI-2 protocol, developed by the MIPI Alliance, was designed for high-speed serial links between cameras and host processors in mobile, automotive and IoT devices. This post explains how LiDAR and RADAR work, how their data maps onto CSI-2, how much bandwidth they really need, and what system designers should watch for.

Key takeaways

• CSI-2 carries any data that can be framed as packets. Non-image sensors use the generic long packet types or the user-defined data types, and virtual channels let several streams share one link.
• Configuration does not travel over the CSI-2 lanes. It uses the Camera Control Interface (CCI), which is I2C compatible, or I3C in newer versions.
• A processed LiDAR point cloud needs far less bandwidth than a modern camera: a few hundred Mbps against several Gbps. Raw ADC streams from RADAR and raw LiDAR waveform data are the heavy cases.
• D-PHY and C-PHY are short-reach PHYs. Across a vehicle, CSI-2 is carried by a serialiser and deserialiser pair (MIPI A-PHY, or proprietary links such as GMSL and FPD-Link).
• The main engineering work is in packet formatting, timestamps and synchronisation across sensors, and driver support on the SoC side.

Overview of LiDAR and RADAR operation

LiDAR (Light Detection and Ranging)

LiDAR measures distance with laser light. In the common pulsed form it fires short pulses, typically in the near infrared at around 905 nm or 1550 nm, and times how long the reflection takes to return (time of flight). Distance is half the round trip at the speed of light, so a 1 microsecond round trip corresponds to about 150 m. Frequency modulated continuous wave (FMCW) LiDAR is an alternative that measures distance and radial velocity from a frequency shift instead of a pulse time.

By sweeping many beams across the scene, a LiDAR builds a 3D point cloud: each point carries X, Y, Z coordinates (or range and angles), an intensity value and often a timestamp. A high-resolution automotive LiDAR produces from hundreds of thousands to a few million points per second.

Applications: autonomous vehicles, robotics, mapping and surveying.

RADAR (Radio Detection and Ranging)

RADAR uses radio waves to detect objects and measure their range, radial velocity (from the Doppler effect) and angle. Radar as a whole spans a wide range of frequencies, from the microwave bands used in weather and air traffic radar to millimetre waves. Automotive radar today operates mainly in the 76 to 81 GHz band, a wavelength of about 4 mm. Most automotive radars use FMCW chirps: the sensor transmits a frequency ramp, mixes the echo with the transmitted signal, and the resulting beat frequency gives the range. A sequence of chirps gives the velocity.

A conventional radar outputs a list of detections (range, speed and azimuth). A 4D imaging radar adds elevation and outputs a sparse point cloud. Radar resolution is coarser than LiDAR, so its point clouds are smaller, typically hundreds to a few thousand points per frame against tens or hundreds of thousands for LiDAR, though both figures vary widely by product.

Applications: automotive ADAS, weather monitoring and air traffic control.

CSI-2 protocol overview

CSI-2 is a high-speed, scalable, low-power serial interface for moving image and sensor data into a processor. It is a protocol layer that runs on top of a physical layer:

• MIPI D-PHY: one clock lane plus one to four data lanes in typical sensors. Each lane has a differential high-speed (HS) mode for data and a single-ended low-power (LP) mode for control states and lane turnaround. Older D-PHY versions run up to about 2.5 Gbps per lane, later versions up to 4.5 Gbps, and MIPI states that D-PHY v3.0 doubles the standard channel rate to 9 Gbps.
• MIPI C-PHY: three-wire "trios" using a three-phase symbol code, with higher throughput per wire than D-PHY.
• MIPI A-PHY: a long-reach automotive SerDes PHY. CSI-2 is carried over it through a protocol adaptation layer, and MIPI cites links of up to 15 m.

Packets, data types and virtual channels

CSI-2 data moves in packets. A short packet signals events such as frame start, frame end, line start and line end. A long packet carries the payload: a header with a data identifier (virtual channel and data type), a 16-bit word count and an error-correcting code, then the payload, then a 16-bit CRC footer.

• Data types tell the receiver how to interpret the payload: RAW8 to RAW20 and beyond, YUV, RGB, generic types such as embedded data, and a block of user-defined 8-bit types. Non-image sensors normally use the user-defined or generic types.
• Virtual channels let several logical streams share one physical link. Early CSI-2 versions offered four; current versions support 16 on D-PHY and 32 on C-PHY. A radar front end can, for example, send ADC samples on one channel and chirp metadata on another.

Control: CCI, I2C and I3C

Sensor configuration (scan rate, chirp profiles, resolution, test modes) is done over a separate sideband bus, the Camera Control Interface (CCI). CCI is compatible with I2C, and newer CSI-2 versions support I3C as well. It does not run on the CSI-2 data lanes. Some radar and LiDAR devices use SPI for configuration instead, but that is the sensor vendor's choice, not part of CSI-2.

How LiDAR works with CSI-2

1. Data generation and processing

The LiDAR emits laser pulses and records the time of flight and intensity of each return. An onboard controller, usually an FPGA or ASIC, turns those measurements into points and formats them for the host link. Depending on the design it may send finished points, range images (one range value per beam and angle step, which looks a lot like a pixel image), or raw waveform or histogram data from the detector for processing on the SoC.

2. Mapping to CSI-2

Point data is not pixel data, so it goes into user-defined or generic long packets. One practical pattern is to treat one scan line, or one firing of all channels, as one CSI-2 line, and one full rotation or frame as one CSI-2 frame. The receiver then writes each "frame" to memory just as it would an image, and software parses the buffer. Range images map even more naturally, since they can be sent as a RAW16 style image with a user-defined data type.

Timestamps matter for sensor fusion. They are usually carried inside the payload or in an embedded data line, and the sensor and SoC share a time base through a common trigger or a time synchronisation protocol.

3. Where LiDAR really connects today

Most automotive and robotics LiDARs on the market, including spinning sensors from the Velodyne and Ouster line and long-range sensors such as Luminar's, deliver their point clouds over Ethernet (often Automotive Ethernet in vehicles) using UDP packets. CSI-2 is more relevant where a LiDAR or depth sensor sits close to an SoC and is designed with a camera-style interface, or where its data is bridged into a camera SerDes so it can share the camera aggregation hardware in the central computer.

4. Challenges

• Non-standard data: because CSI-2 is camera-centric, custom firmware, receiver configuration and drivers are needed to handle point cloud data.
• Raw data volume: processed points are light, but raw waveform or histogram data from SPAD-based LiDARs can be very heavy, and that is where multiple lanes or C-PHY are needed.
• Power: high-speed transmission adds power, which matters in battery-powered robots and handheld devices.

How RADAR works with CSI-2

1. Data generation and processing

A radar front end transmits chirps, receives echoes on several antennas, mixes them down and digitises them. The result is a stream of ADC samples per receive channel per chirp. A radar system on chip then runs FFTs for range, Doppler and angle and outputs a detection list or a 4D point cloud. A radar front end without its own processor instead streams the raw ADC samples to an external processor.

2. Mapping to CSI-2

This raw ADC case is where CSI-2 is genuinely used in radar. Texas Instruments' AWR1243 front end, for example, streams raw ADC data over a four-lane CSI-2 D-PHY interface, with per-lane rates from 150 to 600 Mbps according to its datasheet, and its payload interleaves chirp profile information, the chirp number and the ADC samples of all four receive channels. A typical mapping sends each chirp as one CSI-2 line and each radar frame as one CSI-2 frame. Processed detection lists, being small, are more often sent over CAN-FD, SPI or Ethernet.

Configuration (chirp profiles, frequency band, frame timing) is done over SPI or I2C, separately from the CSI-2 lanes.

3. Example use case

In an ADAS design, one or more radar front ends stream raw ADC data over CSI-2 (directly, or through a serialiser when the sensor is at a bumper corner) to a central processor with radar acceleration. The processor does the FFT processing and fuses the result with camera and LiDAR data for features such as adaptive cruise control and collision avoidance.

4. Challenges

• Radar resolution is coarse, so object classification depends on fusion with cameras, which brings its own synchronisation work.
• The radar link and its cable must be designed for automotive electromagnetic compatibility; a high-speed differential link next to a radar module needs care with shielding and layout.
• Many radar sensors use CAN-FD or Ethernet, so adapting them to CSI-2 may need bridge hardware.

Worked example: how much bandwidth does each sensor need?

The figures below are illustrative calculations to show the method, not the specification of any product.

Stream	Assumptions	Data rate
Camera, 8 MP	8.3 million pixels, RAW12, 30 frames per second	about 3.0 Gbps
LiDAR point cloud	1.3 million points per second, 16 bytes per point (X, Y, Z, intensity as 32-bit values)	about 166 Mbps
High-resolution LiDAR	2.6 million points per second, 16 bytes per point	about 333 Mbps
Radar raw ADC, during a chirp	4 receive channels, 10 Msps, 16-bit I plus 16-bit Q per sample	1.28 Gbps burst
Radar raw ADC, frame average	256 samples per chirp, 128 chirps, 4 channels, 32 bits, 20 frames per second	about 84 Mbps

Three points come out of this. First, a processed LiDAR point cloud fits comfortably on a single D-PHY lane, so LiDAR bandwidth alone rarely strains CSI-2; the original claim that LiDAR's data rate strains CSI-2 holds only for raw waveform data or many sensors aggregated on one link. Second, radar raw data is bursty: the link must be sized for the burst during active chirps, not for the frame average, unless the sensor buffers the data. In the example, 1.28 Gbps burst needs at least three lanes at 600 Mbps. Third, a single modern camera needs more bandwidth than the LiDAR and radar together, which is why camera links set the size of the SoC's CSI-2 receivers and SerDes.

Key considerations for using CSI-2 with LiDAR and RADAR

1. Bandwidth and lanes

Size the link for the peak rate, including packet overhead and blanking, and leave margin. LiDAR raw data and multi-sensor aggregation are the demanding cases; processed radar output is light.

2. Reach

D-PHY and C-PHY are designed for short connections inside a module or across a board. In a vehicle the sensors may be several metres from the central computer, so CSI-2 is carried over a SerDes: MIPI A-PHY, the Automotive SerDes Alliance (ASA) Motion Link, or proprietary links such as Analog Devices GMSL and TI FPD-Link. See the Automotive SerDes Alliance post and the automotive SerDes and bridge chip report for more.

3. Data type compatibility

The CSI-2 receiver and its driver must accept the chosen data type and pass the payload through untouched, without trying to demosaic or colour process it. Check this early on the target SoC; some image signal processor paths only accept image formats.

4. Sensor fusion and timing

Using CSI-2 for cameras, LiDAR and radar lets one receiver and one software framework handle all streams, which helps fusion. Fusion still needs every sample timestamped against a common clock, and frame triggers aligned where the sensors allow it.

5. Real-time performance

CSI-2 adds little latency itself, but the host must keep up: memory bandwidth, interrupt handling and the processing pipeline usually set end-to-end latency.

6. Hardware requirements

The sensor needs a CSI-2 transmitter, built into the sensor chip or provided by an FPGA or bridge. This can add cost and design effort compared with Ethernet or LVDS interfaces the sensor may already have.

7. Environmental robustness

Radar's radio waves are less affected by fog and rain than LiDAR's laser light. The wired link is not immune to the environment: it must survive automotive temperature ranges and meet EMC limits, and functional safety rules (ISO 26262) apply to the data path. Recent CSI-2 versions and A-PHY add features aimed at these needs.

Comparison of LiDAR and RADAR with CSI-2

Aspect	LiDAR	RADAR
Output sent to host	Point cloud, range image or raw waveform data	Raw ADC samples, or detection list / 4D point cloud
Typical bandwidth	Hundreds of Mbps for points; much more for raw data	Tens of Mbps average, bursts above 1 Gbps for raw ADC
CSI-2 data type	User-defined or generic long packets, or RAW style range images	User-defined packets per chirp, metadata on a separate virtual channel or embedded line
Usual alternative interface	Automotive Ethernet	CAN-FD, SPI, Ethernet, LVDS
Weather sensitivity	Higher (fog, rain, spray)	Lower

Practical examples

Autonomous vehicles: robotaxi and ADAS platforms combine many cameras, several radars and one or more LiDARs. Cameras almost always reach the central computer as CSI-2 over a SerDes; radar front ends with raw ADC output can share that path; LiDARs more often use Ethernet. Central computers such as NVIDIA DRIVE platforms provide both camera SerDes inputs and Ethernet ports for this reason. Operators do not generally publish which interface each sensor uses.

Robotics: low-cost 2D LiDARs such as the Slamtec RPLIDAR A1 connect over a UART, often through a USB adapter, not CSI-2. Depth cameras with an image-style output, on the other hand, can feed the CSI-2 camera port of a Raspberry Pi or NVIDIA Jetson module directly.

Consumer devices: the LiDAR scanner Apple introduced in the iPad Pro and iPhone 12 Pro in 2020 provides depth data for augmented reality and camera focus. Apple has not published the internal interface it uses, so any statement that it is CSI-2 should be treated as an assumption.

For a wider view of the sensor suite in an autonomous vehicle, see this overview of autonomous vehicle sensors: cameras, LiDAR and radar.

Common mistakes

• Sizing the link for the average rate. Raw radar data arrives in bursts during chirps.
• Running D-PHY across the vehicle. It is a short-reach PHY; use a SerDes for metre-scale links.
• Assuming the SoC will accept any data type. Some receiver paths only pass image formats to memory.
• Forgetting timestamps. Without a shared time base, fusion of LiDAR, radar and camera data is unreliable.
• Treating CCI as part of the CSI-2 lanes. Control needs its own I2C, I3C or SPI wiring, or a SerDes that tunnels it.

Frequently asked questions

Can CSI-2 carry data that is not an image?

Yes. CSI-2 has generic and user-defined data types, and the receiver writes the payload to memory without interpreting it, provided the SoC driver is set up for that data type.

Is CSI-2 used for LiDAR in production cars?

It is possible and some designs bridge LiDAR into camera SerDes links, but most automotive LiDARs today use Automotive Ethernet. CSI-2 is more common for radar front ends that stream raw ADC data.

What is the difference between CSI-2 and A-PHY?

CSI-2 is the protocol layer. A-PHY is a long-reach physical layer for automotive use that carries CSI-2 (and DSI-2) through adaptation layers, so a sensor several metres away can still appear to the SoC as a CSI-2 source.

How is a CSI-2 sensor configured?

Over the Camera Control Interface, which is I2C compatible, or I3C in newer versions. Some radar and LiDAR parts use SPI. None of these use the CSI-2 data lanes.

Conclusion

LiDAR and RADAR can use CSI-2 by packing their data (point clouds or range images for LiDAR, raw ADC samples or detection lists for radar) into user-defined or generic packets on a link that SoCs already support for cameras. Raw radar ADC streaming is the most established use. Processed LiDAR point clouds are lighter than often assumed and fit easily within CSI-2, though in vehicles they still mostly travel over Ethernet.

Sharing the camera interface simplifies sensor fusion, since one receiver and one software framework can handle every stream. The work lies in custom data formatting, timestamps, long-reach SerDes links and robust integration in a noisy automotive environment. For best results, match each sensor's peak data rate to the lane count and PHY, and confirm early that the host processor's CSI-2 receiver and drivers will accept the data.

Related reading

• The evolution of cameras in ADAS and sensor fusion
• The Automotive SerDes Alliance
• Automotive SerDes and bridge chip report
• The Semiconductor Hub glossary entries for MIPI CSI-2, LiDAR, RADAR and sensor fusion