Two percent of the runs noticed anything wrong↗ (external link)

A skill file, in the paper's definition, is "a self-contained package consisting of a SKILL.md specification — comprising YAML metadata and instructions — alongside optional executable scripts and reference assets." It is the thing you drop into a repository so the agent knows how to do something. It is also, as of this week, a measured attack surface.

The authors took 471 real shell commands, had six LLMs across four families rewrite each of them as a benign-sounding natural-language skill, and ended up with 2,826 adversarial skill files mapped to 11 MITRE ATT&CK tactics. They ran those against Gemini CLI and Qwen Code — 5,629 completed runs.

Gemini CLI executed the hostile instruction in 95.5–96.1% of runs. Qwen Code, 71.6–74.0%. Explicit safety recognition — the agent saying anything at all about the request being suspect — appeared in 1.99% of runs.

The rate was, in their phrase, "nearly invariant to the generating model." It did not matter much which LLM wrote the malicious skill. That is the part I would underline: this is not an adversarial-prompting speciality that a red team develops. Any competent model will produce these, because dressing a shell command up in helpful prose is not a hard writing task.

Two honest caveats, because the headline number is doing more work than it can carry.

First, both agents were "set to run with the delegated privileges, meaning every action will be auto-approved." That is the worst case by construction. The finding is about what a fully-YOLO agent does, not what your agent does — unless you run it that way, which, be honest, a lot of us do.

Second, success is defined generously: evidence of "(a) intent to execute a command or (b) confirmation of execution." The attack counts as successful when the agent runs the command, not when the command achieves anything. Ninety-six percent is a compliance rate, not a breach rate.

Even discounted for both, the shape holds, and the per-tactic breakdown is more interesting than the total. Initial Access 91.2%, Defense Evasion 90.4%, Privilege Escalation 86.6%, Discovery 86.5%. The floor is Exfiltration at 67.2% — and that is where the two agents diverge hardest, Gemini at 95.5% against Qwen's 38.5%. Something in Qwen's training baulks at outbound traffic in a way it does not baulk at privilege escalation. Neither vendor, I suspect, could tell you why.

This is the second time in a week I have linked a story where the agent's own configuration is the payload — the npm compromise wrote a SessionStart hook into .claude/settings.json and waited for someone to open the repo. The pattern is the same. We built a convention where an agent reads instructions out of the working tree and follows them, and we did not build any of the machinery — signing, review, permissions, provenance — that we would demand of any other executable in that tree.

Their recommendation is a tiered permission model where high-risk actions escape blanket auto-approval. Fine, and it will be ignored by exactly the people running --yolo today. The cheaper habit, available now: read the skill files you did not write. They are short. That is the whole point of them.