Autonomous cars, drones cheerfully obey prompt injection by road sign

Use of VLMs/LLMs in Real Autonomous Vehicles

• Strong disagreement over whether “real” self‑driving stacks use VLMs or only classic perception + planning.
• One side claims “no serious AV would touch VLMs for control”; others counter with Waymo blog posts and papers describing Gemini‑based multimodal models that feed into trajectory prediction and world models.
• Clarification: these models are not (yet) pure end‑to‑end controllers; they provide semantic signals that are then distilled or combined with traditional systems.
• Consensus: production stacks are layered and conservative; any VLM output is (or should be) treated as untrusted input, not directly wired to steering/brakes.

Human vs Machine Susceptibility to ‘Prompt Injection’

• Many note that temporary and handwritten signs (“accident ahead,” construction paddles) must influence AV behavior; otherwise the system is unusable.
• A key criticism of the demo: the model obeys a “PROCEED” sign even when pedestrians are visibly in the crosswalk—behavior humans would not (and legally must not) copy.
• Others argue humans can also be “prompt injected” by confident workers in vests, though most agree people still prioritize “don’t hit anyone.”

Validity and Relevance of the Research / Article

• Several commenters see the paper as an obvious, almost trivial demonstration: of course a naive VLM prompted via text on a sign can be misled.
• Strong criticism of the article’s framing as implying current cars and drones actually behave this way; viewed as clickbait and misrepresentation by omission.
• Some point out the paper explicitly targets “a new class of systems,” not today’s deployed robo‑taxis.

AV Architecture, Safety Priorities, and Attack Surface

• Well‑designed AVs are described as multiple subsystems with ordered priorities: avoid collisions, stay on drivable surfaces, behave predictably, then obey signs/laws, then optimize route.
• A malicious sign should only corrupt the “follow signs” layer; higher‑priority safety layers (obstacle avoidance, road boundaries) should still prevent crashes.
• Suggested mitigations: HD maps to cross‑check new signs; flagging unusual signage for human review; conservative behavior when inputs conflict.
• Some worry that end‑to‑end VLM‑based robotics (future “vision‑language agents”) will inherit similar prompt‑injection and poisoning vulnerabilities unless new defenses are built.

Broader Skepticism and Social Response

• A subset views fully unattended self‑driving in mixed city traffic as a “pipe dream,” expecting remote operators and restricted lanes instead.
• Others report that existing robo‑taxis already coexist reasonably well with human drivers.
• Story of people deliberately cutting off AVs sparks debate over “Luddite”‑style resistance: some see it as symbolic labor protest; others as recklessly endangering passengers.

Side Discussions

• Tangents on 4‑way stops vs roundabouts and “smart” signalization highlight that human road design itself often confuses both people and potential AVs.
• Several note that any system which must respond to real‑world signage inherently creates an attack surface; the core challenge is bounding the damage when that layer is fooled.