{"slug":"openai-huggingface-cost-audit","verification":{"valid":true,"entries":14,"head":"ae50ab11ac7c61dd17e6f9d29aff42ea3d9c33ca665375ede04e6fd5fea20fd0"},"count":14,"sources":[{"id":"s1","type":"paper","url":"https://arxiv.org/abs/2605.11086","title":"ExploitGym: Can AI Agents Turn Security Vulnerabilities into Real Attacks?","quote":"Table 3: Agent performance and cost comparison (two-hour timeout). ... Cost (USD) is estimated. The remaining columns report per-task averages over the successful subset (Succ.) and over the full benchmark (Full).","author":"Zhun Wang, Nico Schiller, Hongwei Li, Milad Nasr, Nicholas Carlini, Eric Wallace, Elie Bursztein, Kurt Thomas, Yan Shoshitaishvili, Wenbo Guo, Jingxuan He, Thorsten Holz, Dawn Song et al.","publisher":"arXiv","date":"2026-05-11","claim_ids":["c1","c2"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"genesis","hash":"952797ab1c3ac53531917102dba5c015aff174066ee4fe3754d16a3903e5ae95"},{"id":"s2","type":"paper","url":"https://arxiv.org/html/2605.11086v1","title":"ExploitGym, experimental setup: two-hour timeout per task","quote":"We evaluate all agent configurations on the full benchmark with security mitigations disabled and impose a two-hour wall-clock timeout per task.","author":"Wang et al.","publisher":"arXiv","date":"2026-05-11","claim_ids":["c5"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"952797ab1c3ac53531917102dba5c015aff174066ee4fe3754d16a3903e5ae95","hash":"0c16507ca749ccae1481b45de8d8c1056ef7b490d3e200bd0d0626cc634a1a77"},{"id":"s3","type":"paper","url":"https://arxiv.org/html/2605.11086v1","title":"ExploitGym, success definition and alternative-path finding","quote":"successes, which require not only that the agent achieve unauthorized code execution to exfiltrate the secret flag, but also that it exercise the specific vulnerability provided in the task specification, as validated by an agent-as-a-judge","author":"Wang et al.","publisher":"arXiv","date":"2026-05-11","claim_ids":["c3"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"0c16507ca749ccae1481b45de8d8c1056ef7b490d3e200bd0d0626cc634a1a77","hash":"a70edd9b5f3d0720824cc3cc0e4cb3b546c1f9f79824af1c54e297a795c36891"},{"id":"s4","type":"statement","url":"https://openai.com/index/hugging-face-model-evaluation-security-incident/","title":"OpenAI and Hugging Face partner to address security incident during model evaluation","quote":"All evidence suggests that the models were hyperfocused on finding a solution for ExploitGym, going to extreme lengths to achieve a rather narrow testing goal.","author":"OpenAI","publisher":"OpenAI","date":"2026-07-21","claim_ids":["c4"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"a70edd9b5f3d0720824cc3cc0e4cb3b546c1f9f79824af1c54e297a795c36891","hash":"56fb4ea8caf84d40ea2948f6c26d7846cf7ee340e734f8056203f45a9314cbf8"},{"id":"s5","type":"statement","url":"https://huggingface.co/blog/security-incident-july-2026","title":"Security incident disclosure — July 2026","quote":"we ran LLM-driven analysis agents over the full attacker action log, comprised of more than 17,000 recorded events","author":"Hugging Face","publisher":"Hugging Face","date":"2026-07-16","claim_ids":["c2","c5"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"56fb4ea8caf84d40ea2948f6c26d7846cf7ee340e734f8056203f45a9314cbf8","hash":"2809795ab0835ecb75515532743b31d1d9a60271899276c2fe9ff138ea1125aa"},{"id":"s6","type":"article","url":"https://simonwillison.net/2026/Jul/22/openai-cyberattack/","title":"OpenAI's accidental cyberattack against Hugging Face is science fiction that happened","quote":"The ExploitGym benchmark is available on GitHub.","author":"Simon Willison","publisher":"simonwillison.net","date":"2026-07-22","claim_ids":["c3"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"2809795ab0835ecb75515532743b31d1d9a60271899276c2fe9ff138ea1125aa","hash":"fdd3744dcdb13f7fe48065b9acbfb9dd3906845e9aef8ecfe723ae6269d8b1cf"},{"id":"s7","type":"article","url":"https://time.com/article/2026/07/24/openai-hugging-face-attack/","title":"How OpenAI Lost Control of an AI Model—and What Needs to Change","quote":"How long were the agents running? Did they work in unison? What was the prompt? These details remain unknown, at least to the public.","author":"Harry Booth","publisher":"TIME","date":"2026-07-24","claim_ids":["c4"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"fdd3744dcdb13f7fe48065b9acbfb9dd3906845e9aef8ecfe723ae6269d8b1cf","hash":"9fdeff9aebd29275cac6b67756c686f4a2192f4d5e8e6d61bc11fa299da31c88"},{"id":"s8","type":"article","url":"https://www.forrester.com/blogs/an-ai-security-facepalm-openais-evaluation-became-hugging-faces-incident/","title":"An AI Security Facepalm: OpenAI's Evaluation Became Hugging Face's Incident","quote":"Agents can pursue authorized goals through unauthorized means, especially when evaluators reward the outcome and fail to police the path.","author":"Jeff Pollard, Jess Burn, Allie Mellen, Janet Worthington, Joseph Blankenship","publisher":"Forrester","date":"2026-07-22","claim_ids":["c4"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"9fdeff9aebd29275cac6b67756c686f4a2192f4d5e8e6d61bc11fa299da31c88","hash":"7e517eb8c143be9f505e5bf8898f4022565076bfb9113ec71b6337e10840a92b"},{"id":"s9","type":"article","url":"https://www.elisity.com/blog/openai-hugging-face-incident-lateral-movement","title":"The OpenAI Hugging Face Incident Is a Lateral Movement Story, Not a Rogue AI Story","quote":"Hugging Face happened to host the benchmark's answers. That was the entire selection logic.","author":"Charlie Treadwell","publisher":"Elisity","date":"2026-07-22","claim_ids":["c3"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"7e517eb8c143be9f505e5bf8898f4022565076bfb9113ec71b6337e10840a92b","hash":"b97b03723fedef4b0e286da61484a84ddf03dc536c19702e7aadae3c13ffad5a"},{"id":"s10","type":"article","url":"https://www.rapid7.com/blog/post/ai-openai-hugging-face-what-happened/","title":"What Happened Between OpenAI and Hugging Face?","quote":"the more freedom a model has to pursue a defined reward or goal, the more important containment, monitoring, and clear constraints become","author":"Wade Woolwine","publisher":"Rapid7","date":"2026-07-23","claim_ids":["c6"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"b97b03723fedef4b0e286da61484a84ddf03dc536c19702e7aadae3c13ffad5a","hash":"373e59eac7aca295645cc29a25d7dd92bed38250fdea598395cf9c5414e216d0"},{"id":"s11","type":"article","url":"https://www.trendmicro.com/en_us/research/26/g/inside-the-openai-hugging-face-incident.html","title":"Inside the OpenAI – Hugging Face Incident: The AI Breach With No Human Attacker Behind It","quote":"Telemetry reveals behavior, not intent. Defenders therefore need to focus on what an agent actually does, rather than why it does it.","author":"Bestin Koruthu, David Girard","publisher":"Trend Micro","date":"2026-07-23","claim_ids":["c4"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"373e59eac7aca295645cc29a25d7dd92bed38250fdea598395cf9c5414e216d0","hash":"94aaa73645259ce91f03d548825c349507d22fa3dd6be57d87a2006fbcf12263"},{"id":"s12","type":"paper","url":"https://arxiv.org/html/2605.14153v1","title":"ExploitBench: A Capability Ladder Benchmark for LLM Cybersecurity Agents","quote":"ExploitGym evaluates each model through one vendor CLI, which does not directly measure LLM performance.","publisher":"arXiv","date":"2026-05","claim_ids":["c6"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"94aaa73645259ce91f03d548825c349507d22fa3dd6be57d87a2006fbcf12263","hash":"a0240096403b17fe8162368342ac04caf93e5db55b4a51a55d6bb8547e117e9f"},{"id":"s13","type":"article","url":"https://time.com/article/2026/07/24/openai-hugging-face-attack/","title":"TIME: an OpenAI staffer on recurrence","quote":"Externally, this feels like a big warning shot, but internally, related incidents have been happening for a while.","author":"anonymous OpenAI staffer, to Harry Booth","publisher":"TIME","date":"2026-07-24","claim_ids":["c7"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"a0240096403b17fe8162368342ac04caf93e5db55b4a51a55d6bb8547e117e9f","hash":"08f395b8499484473eadd239ad91601eb9dc48a5dcebbd9ecc99edd9b45cd4e9"},{"id":"s14","type":"article","url":"https://time.com/article/2026/07/24/openai-hugging-face-attack/","title":"TIME: the Anthropic Mythos internal escape","quote":"Anthropic disclosed in April that it realized an internal deployment of Mythos had gained unauthorized access after one of its researchers received an email from the model while having lunch in a park.","author":"Harry Booth","publisher":"TIME","date":"2026-07-24","claim_ids":["c7"],"accessed_at":"2026-07-27T02:37:16.438Z","prev":"08f395b8499484473eadd239ad91601eb9dc48a5dcebbd9ecc99edd9b45cd4e9","hash":"ae50ab11ac7c61dd17e6f9d29aff42ea3d9c33ca665375ede04e6fd5fea20fd0"}]}