Project Context: Remote driving and safety-critical teleoperation require strict Glass-to-Glass (G2G) latency guarantees. Traditional protocols (like standard WebRTC) are limited by a physical latency floor (acquisition + encoding + network + decoding). Our research introduces Extrapolation-based Latency Control (ELC), a novel framework that combines adaptive video rate control (VP9/WebRTC) at the transmitter with neural video frame extrapolation at the receiver to actively compensate for network delays and achieve sub-physical apparent latency.

• Context & Motivation: Evaluating extrapolated video streams requires going beyond standard full-frame PSNR, as background pixels dilute errors occurring on moving obstacles (e.g., pedestrians or vehicles). Furthermore, neural frame extrapolation faces physical limits during sudden scene changes or abrupt object appearances.

• Main Objectives:

  1. Dataset Preparation: Select and curate diverse driving/teleoperation video sequences with varying spatial and temporal complexity, isolating critical "edge-case" clips (e.g., sudden appearance of new objects/pedestrians, occlusions).

  2. Advanced QoE Pipeline: Develop an automated offline evaluation suite in Python to compute perceptual and region-aware metrics from simulator outputs, including VMAF, Masked PSNR / Masked SSIM (isolating moving Regions of Interest via motion masks), and Temporal Stability / Jerkiness ($\Delta h$).

  3. Failure-Mode & Recovery Analysis: Stress-test the ELC simulator under sudden object appearances and packet-loss events, quantifying the transient quality drop and the exact recovery time once real frames are received.