Modern ADAS and autonomous vehicles fuse cameras, LiDAR and radar — radar being uniquely robust in rain, fog and darkness, and able to measure velocity directly via Doppler. Deep networks now classify radar returns, but they are black boxes. For a safety-critical system that must be certified (ISO 26262 / SOTIF), engineers need to know why a detection was made — that is the role of Explainable AI (XAI).
Working principle
Radar produces range–Doppler–angle data cubes that a CNN turns into object detections. XAI methods add an interpretation layer: saliency / Grad-CAM highlights which regions of the range–Doppler map drove the decision; SHAP attributes the output to input features with game-theoretic consistency; surrogate models approximate the network locally. The explanation lets developers spot spurious cues (e.g. clutter) and build the safety argument.
| Method | Type | What it shows |
|---|---|---|
| Grad-CAM | Saliency map | Influential input regions |
| SHAP | Attribution | Per-feature contribution |
| LIME | Local surrogate | Locally faithful explanation |
| Attention | Built-in | Where the model focuses |
CautionExplanations must be faithful to the model, not just plausible to humans. A convincing-but-wrong explanation is dangerous in a safety case — evaluating explanation fidelity is itself a research problem.
Applications
- Validating ADAS object detection and false-alarm analysis
- Safety certification evidence (ISO 26262 / SOTIF)
- Debugging sensor-fusion and corner-case behaviour
References & further reading
- Selvaraju et al., “Grad-CAM: Visual Explanations from Deep Networks,” ICCV 2017.
- Lundberg & Lee, “A Unified Approach to Interpreting Model Predictions (SHAP),” NeurIPS 2017.
- ISO 26262 / ISO 21448 (SOTIF) functional-safety standards.