«

在人工智能风险加剧的背景下,Anthropic出台了新规

qimuai 发布于 阅读:14 一手编译


在人工智能风险加剧的背景下,Anthropic出台了新规

内容来源:https://www.theverge.com/news/760080/anthropic-updated-usage-policy-dangerous-ai-landscape

内容总结:

人工智能公司Anthropic升级Claude安全政策 明确禁止协助开发生化核武器

为应对日益增长的AI安全风险,美国人工智能公司Anthropic近日更新了其聊天机器人Claude的使用政策,重点强化对高危武器开发与网络安全风险的管控。

新版政策首次明确禁止用户利用Claude开发生物、化学、放射性或核武器(CBRN),并新增对高能爆炸物研发的禁令。此前条款虽禁止协助开发武器,但未具体列出武器类别。这一调整与5月推出的"AI安全等级3"防护措施相呼应,该机制旨在防止模型被破解后协助CBRN武器开发。

针对AI代理工具(如可操控用户电脑的"Computer Use"和集成至开发者终端的"Claude Code")可能引发的规模化滥用风险,Anthropic新增"不得危害计算机及网络系统"条款,明文禁止利用Claude发现系统漏洞、制作恶意软件或开发拒绝服务攻击工具。

值得注意的是,公司对政治内容限制有所放宽,不再全面禁止竞选游说类内容,仅限制"破坏民主进程或涉及选民定向操控"的用途。同时明确"高风险用例"规范仅适用于面向消费者的场景,不涉及商业用途。

此次政策调整反映出AI行业在技术迭代与安全伦理之间的平衡探索。随着大模型能力边界的扩展,头部企业正通过细化规则应对潜在滥用风险。

中文翻译:

面对日益增长的安全隐忧,人工智能公司Anthropic更新了其Claude聊天机器人的使用政策。除引入更严格的网络安全规则外,该政策首次明确禁止用户利用Claude开发特定高危武器系统。

Anthropic针对危险升级的AI环境制定新规
这家AI初创企业在新政策中特别强调,禁止使用Claude协助开发生物武器、化学武器、放射性武器或核武器。虽然该公司在政策更新公告中未重点提及武器条款修订,但对比新旧版本可发现显著变化。此前政策仅笼统禁止将Claude用于"生产、改装、设计、营销或分销武器、爆炸物、危险材料及其他旨在致人伤亡的系统",而更新版本则具体将高爆炸药与生物、核、化学及放射性(CBRN)武器纳入禁止清单。

今年五月,Anthropic在推出Claude Opus 4模型时同步实施"AI安全三级防护"机制。该防护体系旨在增强模型防破解能力,并阻止其协助CBRN武器研发。

公告同时承认智能代理工具的风险隐患,包括可操控用户电脑的"计算机控制"功能,以及能让开发者直接在终端嵌入Claude的编程工具。"这些强大功能可能引发规模化滥用、恶意软件制作及网络攻击等新型风险,"Anthropic在声明中指出。

为此,该公司在政策中新增"禁止危害计算机及网络系统"章节,明文禁止利用Claude进行漏洞探测、恶意软件制作、拒绝服务攻击工具开发等行为。

值得注意的是,Anthropic反而放宽了政治内容限制。新政策不再全面禁止政治竞选与游说相关内容创作,仅限制"欺骗性、破坏民主进程或涉及选民/竞选精准定位"的应用场景。公司同时澄清,当用户通过Claude向个人或客户提供建议时,所有"高风险"使用场景的管控要求仅适用于消费者端,不涉及商业用途。

英文来源:

Anthropic has updated the usage policy for its Claude AI chatbot in response to growing concerns about safety. In addition to introducing stricter cybersecurity rules, Anthropic now specifies some of the most dangerous weapons that people should not develop using Claude.
Anthropic has new rules for a more dangerous AI landscape
The AI startup’s new policy now specifically bans using Claude to help develop biological, chemical, radiological, or nuclear weapons.
Anthropic doesn’t highlight the tweaks made to its weapons policy in the post summarizing its changes, but a comparison between the company’s old usage policy and its new one reveals a notable difference. Though Anthropic previously prohibited the use of Claude to “produce, modify, design, market, or distribute weapons, explosives, dangerous materials or other systems designed to cause harm to or loss of human life,” the updated version expands on this by specifically prohibiting the development of high-yield explosives, along with biological, nuclear, chemical, and radiological (CBRN) weapons.
In May, Anthropic implemented “AI Safety Level 3” protection alongside the launch of its new Claude Opus 4 model. The safeguards are designed to make the model more difficult to jailbreak, as well as to help prevent it from assisting with the development of CBRN weapons.
In its post, Anthropic also acknowledges the risks posed by agentic AI tools, including Computer Use, which lets Claude take control of a user’s computer, as well as Claude Code, a tool that embeds Claude directly into a developer’s terminal. “These powerful capabilities introduce new risks, including potential for scaled abuse, malware creation, and cyber attacks,” Anthropic writes.
The AI startup is responding to these potential risks by folding a new “Do Not Compromise Computer or Network Systems” section into its usage policy. This section includes rules against using Claude to discover or exploit vulnerabilities, create or distribute malware, develop tools for denial-of-service attacks, and more.
Additionally, Anthropic is loosening its policy around political content. Instead of banning the creation of all kinds of content related to political campaigns and lobbying, Anthropic will now only prohibit people from using Claude for “use cases that are deceptive or disruptive to democratic processes, or involve voter and campaign targeting.” The company also clarified that its requirements for all its “high-risk” use cases, which come into play when people use Claude to make recommendations to individuals or customers, only apply to consumer-facing scenarios, not for business use.

ThevergeAI大爆炸

文章目录


    扫描二维码,在手机上阅读