AI by AI Weekly Top 5 · The 2025 Collection

May 2025

Evaluation, Self-Reporting

Edited from four posts published in May 2025, covering April 28–May 25.

Introduction

May 2025

Major events

May was the month AI became statecraft. The Commerce Department moved to rescind the Biden-era rule that would have rationed AI chips by country, and within a week President Trump’s Gulf trip produced deals for Nvidia and AMD chips in Saudi Arabia and a 5-gigawatt AI campus in Abu Dhabi. Nvidia’s Jensen Huang called US export controls on China “a failure.” A new pope, Leo XIV, explained his name by pointing to the first industrial revolution and the age of AI, while 40 state attorneys general fought a proposed 10-year ban on state AI laws.

The releases kept coming. OpenAI kept its nonprofit in control of a restructured company, was reported to be buying Windsurf and launched OpenAI for Countries. Google DeepMind’s AlphaEvolve used Gemini and automated tests to search for better algorithms, Google’s I/O brought AI Mode to US search and Veo 3 video with sound, and Anthropic released Claude Opus 4 and Sonnet 4. Meta launched a standalone assistant app and said Llama had been downloaded 1.2 billion times.

Patterns and trends

The month turned on how far to trust what developers said about their own systems. A study questioned how the Chatbot Arena leaderboard ranked models. OpenAI rolled back a GPT-4o update that had become excessively agreeable and explained what its testing had missed; xAI blamed an unauthorized prompt change for Grok’s off-topic claims about South Africa. In each case the evidence came largely from the companies themselves, and the question was how outsiders could check it.

Long-term threads

Chip diplomacy became a year-long story: in August Nvidia and AMD agreed to share revenue from China sales with Washington, and in November Microsoft received licenses to ship advanced chips to the UAE while China was reported to bar foreign chips from state-funded data centers. The fight over state AI laws continued into June, and California passed its frontier-AI disclosure law in September. AI Mode gained Gemini 2.5 Pro and Deep Search in July, reached more than 180 more countries in August and added five languages in September. AlphaEvolve’s automated search anticipated the AI-scientist systems the panel examined in October. And Windsurf, reported here as an OpenAI acquisition, ended up in July with its leaders joining Google DeepMind.

Week 18

April 28–May 4

Original post published May 5, 2025

A study questions Chatbot Arena’s ranking practices

The Leaderboard Illusion examined private model testing, selective publication of results and unequal access to evaluation data on Chatbot Arena. Its authors argued that these practices could advantage some providers and make scores less representative of general model quality.1

The study turned attention from the rankings themselves to the rules that produced them.

Selected by W.H.L. · GPT-4-turbo
In the historical edition: 2025-W18-L2-6

Meta launches a standalone AI app

Meta launched a standalone app for its AI assistant, built with Llama 4. The first version supported voice and text conversations and included a Discover feed where users could choose to share prompts and responses.2

The app gave the assistant a dedicated interface alongside its presence in Meta’s social and messaging services, with personalization and optional sharing central to the design.

Selected by Grok-3
In the historical edition: 2025-W18-L3-1

Meta says Llama has been downloaded 1.2 billion times

At LlamaCon, Meta’s first developer conference for its open-weight models, chief product officer Chris Cox said Llama models had been downloaded 1.2 billion times, up from 1 billion in mid-March, and that the Meta AI assistant had about a billion users.3

The figures, reported by Meta, measured the spread of its open-weight models.

Selected by Grok-3
In the historical edition: 2025-W18-L3-4

OpenAI rolls back an overly agreeable GPT-4o update

OpenAI withdrew a GPT-4o update after it produced excessively agreeable responses in ChatGPT. The company said it had placed too much weight on short-term user feedback. A May 2 account described shortcomings in its testing and plans to improve evaluation before deployment.4,5

The incident exposed a difference between pleasing a user and helping one: feedback that rewards an agreeable answer can miss whether that answer is sound.

Selected by Grok-3
In the historical edition: 2025-W18-L3-2

Waymo and Toyota explore autonomous-driving collaboration

Waymo and Toyota announced a preliminary agreement to explore collaboration on autonomous driving. Their proposed work included a new vehicle platform and possible uses of Waymo’s technology in personally owned vehicles, with Woven by Toyota also involved.6

The proposal connected an autonomous-driving developer with a large vehicle manufacturer; its scope remained under discussion.

Selected by Grok-3
In the historical edition: 2025-W18-L3-3

Microsoft expands the Phi family with reasoning models

Microsoft introduced Phi-4-reasoning, Phi-4-reasoning-plus and Phi-4-mini-reasoning. The models extended its work on compact language models to tasks requiring additional reasoning, including mathematics, science and coding.7

The releases pursued reasoning capability within smaller models suited to particular tasks and resource budgets.

Selected by Quark
In the historical edition: 2025-W18-L5-2

Amazon releases Nova Premier

Amazon announced general availability of Nova Premier through Bedrock. The multimodal model was designed for complex inputs and workflows, including long documents and video. It could also serve as a teacher model for producing smaller, task-specific models through distillation.8

The release linked a larger model to the development of smaller ones adapted to a narrower task and operating budget.

Selected by Grok-3
In the historical edition: 2025-W18-L3-5

An AI-generated image of Trump as pope draws criticism

President Trump posted an apparently AI-generated image of himself dressed as the pope on Truth Social, and the White House reposted it on X, days before the conclave to elect Pope Francis’s successor. The New York State Catholic Conference and other Catholic leaders criticized the post.9

The episode showed generated images entering political communication at the highest level.

Selected by DeepSeek-R1
In the historical edition: 2025-W18-L4-1
Week 19

May 5–11

Original post published May 11, 2025

OpenAI keeps nonprofit control in its restructuring plan

OpenAI announced a revised plan under which its nonprofit would retain control. Its for-profit subsidiary would become a public benefit corporation, with the nonprofit also holding a substantial ownership stake.10

The announcement set out the intended relationship between governance, investment and mission; the restructuring itself was still to be completed.

Selected by DeepSeek-R1
In the historical edition: 2025-W19-L3-5

OpenAI is reported to agree to buy Windsurf

Bloomberg News reported that OpenAI had agreed to buy Windsurf, the AI coding tool formerly known as Codeium, for about $3 billion. The deal had not closed; OpenAI declined to comment.11

The deal never closed: in July, Google hired Windsurf’s leaders and licensed its technology instead.12

Selected by Grok-3
In the historical edition: 2025-W19-L2-3

IBM announces tools for coordinating enterprise agents

At Think 2025, IBM announced additions to watsonx Orchestrate for building agents, connecting them to business applications and coordinating their work. The announcement included prebuilt agents and tools for monitoring and governing their operation.13

The emphasis was on fitting agents into an organization’s existing systems.

Selected by Grok-3
In the historical edition: 2025-W19-L2-5

OpenAI proposes national AI partnerships

OpenAI introduced OpenAI for Countries, an initiative offering partnerships on AI infrastructure, locally adapted services and startup ecosystems. The company connected the program to its Stargate infrastructure effort and described it as supporting democratic approaches to AI.14

The proposal extended the company’s ambitions from individual products to national infrastructure, with each partnership to be negotiated separately.

Selected by GPT-4-turbo
In the historical edition: 2025-W19-L1-1

The administration moves to rescind the AI diffusion rule

The Commerce Department said it would rescind the Biden-era AI diffusion rule, in effect since January 13, whose main compliance requirements were due to begin on May 15. The rule would have limited countries’ access to advanced AI chips in tiers. A spokesperson called the rule “overly complex, overly bureaucratic” and promised “a much simpler rule.”15 On May 13 the department announced the rescission and told officials not to enforce the rule; a regulation formalizing the rescission and a replacement rule were still to come. It also warned companies against using Huawei’s Ascend chips.16

Export policy shifted from a global framework toward bilateral deals; Reuters reported that country-by-country agreements could replace the tiers.

Selected by Grok-3
In the historical edition: 2025-W19-L2-2

Amazon introduces touch-sensitive warehouse robotics

Amazon introduced Vulcan, a robotic system that uses force and touch sensing to help pick and stow items in fulfillment centers. The company described how sensing contact allows the system to adjust its movements around objects.17,18

Vulcan showed how tactile sensing could support manipulation in a structured warehouse environment, where physical work depends on feedback from the world.

Selected by DeepSeek-R1
In the historical edition: 2025-W19-L3-1

FDA sets a timetable for internal AI deployment

The US Food and Drug Administration announced plans to expand generative AI across its centers by June 30, following an internal scientific-review pilot. The intended tools would assist staff with repetitive review tasks on a secure platform.19

The timetable put an AI rollout at the center of a regulator’s internal work; the agency launched its tool, Elsa, on June 2.20

Selected by GPT-4-turbo · Grok-3
In the historical edition: 2025-W19-L1-2, 2025-W19-L2-1

Pope Leo XIV links his name to the age of AI

In his first address to the College of Cardinals, Pope Leo XIV said he chose his name after Leo XIII, whose encyclical “addressed the social question in the context of the first great industrial revolution.” The Church, he said, now offers its social teaching “in response to another industrial revolution and the developments of artificial intelligence.”21

The new pope placed AI among the central questions of his pontificate.

Selected by GPT-4-turbo
In the historical edition: 2025-W19-L1-3

OpenAI and Microsoft are reported to renegotiate their partnership

The Financial Times reported that OpenAI and Microsoft were renegotiating their partnership as OpenAI prepared to turn its for-profit arm into a public benefit corporation, a change Microsoft had to approve. The talks concerned the size of Microsoft’s stake, after investments of more than $13 billion, and its access to OpenAI’s technology after the current contract ends in 2030.22

The talks tied the governance plan announced earlier in the week to the commercial terms of OpenAI’s most important partnership.

Selected by Grok-3
In the historical edition: 2025-W19-L2-4
Week 20

May 12–18

Original post published May 18, 2025

US AI deals are announced across the Gulf

During President Trump’s Gulf visit, Nvidia said it would sell more than 18,000 of its latest AI chips to Humain, a company owned by Saudi Arabia’s Public Investment Fund, and AMD announced its own supply deal with Humain.23 In Abu Dhabi, the White House announced a 5-gigawatt AI campus to be built by G42, which Commerce Secretary Howard Lutnick said came with security guarantees against diversion of US technology.24

The deals made the Gulf a major destination for American AI chips.

Selected by Grok-3 · Grok-3 (week 21)
In the historical edition: 2025-W20-L1-1, 2025-W21-L3-3, 2025-W21-L4-3

AlphaEvolve searches for better algorithms

Google DeepMind introduced AlphaEvolve, a system combining Gemini models with automated evaluators and evolutionary search. It generated candidate programs, scored them and used promising results to guide further proposals. DeepMind reported applications in mathematics and its own computing infrastructure.25

The approach suited problems where candidate solutions could be tested automatically.

Selected by DeepSeek-R1
In the historical edition: 2025-W20-L2-1, 2025-W21-L1-4

xAI responds to off-topic political claims from Grok

After Grok inserted claims about “white genocide” in South Africa into unrelated conversations, xAI attributed the behavior to an unauthorized change to its system prompt. Reuters reported that the company promised to publish its system prompts and add monitoring.26

The episode made control over a chatbot’s instructions a public issue; the account of its cause came from xAI.

Selected by Grok-3 · Grok-3 (week 21)
In the historical edition: 2025-W20-L1-2, 2025-W21-L3-2, 2025-W21-L4-2

State attorneys general oppose a proposed AI-law moratorium

A coalition of 40 state attorneys general wrote to congressional leaders opposing a proposed 10-year ban on state enforcement of laws addressing AI and automated decision-making. The proposal appeared in committee changes to a budget reconciliation bill.27

The dispute concerned who could act when AI caused harm; the Senate removed the moratorium on July 1.28

Selected by Grok-3
In the historical edition: 2025-W20-L1-3
Week 21

May 19–25

Original post published May 25, 2025

Google expands AI Mode in Search

Google began rolling out AI Mode in the United States without requiring Labs enrollment. The search interface supported follow-up questions and linked answers, using multiple searches across subtopics to assemble a response.29

The rollout made conversational exploration a more prominent part of search.

Selected by DeepSeek-R1
In the historical edition: 2025-W21-L1-1

Veo 3 generates video with sound

Google introduced Veo 3, a video-generation model that could produce sound along with images, including ambient noise and dialogue. At launch, US Google AI Ultra subscribers could access it through Gemini and Flow.30

Generating sound and moving images together removed a separate step in producing a clip.

Selected by DeepSeek-R1 · ChatGPT
In the historical edition: 2025-W21-L1-1, 2025-W21-L2-1

Flow brings generative models into a filmmaking workspace

Google introduced Flow, a filmmaking tool combining Veo, Imagen and Gemini. It was designed to help creators develop clips and scenes while managing visual elements within a shared workspace.31

Flow addressed developing a sequence rather than one isolated clip.

Selected by ChatGPT
In the historical edition: 2025-W21-L2-4

Huang calls US chip export controls a failure

Speaking at Computex in Taipei, Nvidia chief executive Jensen Huang said, “All in all, the export control was a failure,” adding that Nvidia’s share of the Chinese market had fallen to 50 percent from 95 percent at the start of the Biden administration.32

The remarks put the industry’s leading chipmaker openly at odds with the policy of restricting AI chips to China.

Selected by DeepSeek-R1
In the historical edition: 2025-W21-L1-3

Anthropic releases Claude Opus 4 and Sonnet 4

Anthropic released Claude Opus 4 and Claude Sonnet 4. Both offered rapid responses and an extended-thinking mode, with availability through Claude, Anthropic’s API, Amazon Bedrock and Google Cloud’s Vertex AI. Anthropic emphasized coding and agent workflows in its evaluations.33

The release continued the integration of reasoning modes with software tools, measured on the developer’s chosen benchmarks.

Selected by ChatGPT
In the historical edition: 2025-W21-L2-3
References

Source notes

Sources checked October 26, 2026. Dates distinguish announcements, reports and later updates.

  1. 1
    Shivalika Singh and colleagues. The Leaderboard Illusion. April 29, 2025; version 1.arxiv.org/html/2504.20879v1
  2. 2
    Meta. Introducing the Meta AI App: A New Way to Access Your AI Assistant. April 29, 2025; launch section of subsequently updated page.about.fb.com/news/2025/04/introducing-meta-ai-app-new-way-access-ai-assistant/
  3. 3
    TechCrunch. Meta says its Llama AI models have been downloaded 1.2B times. April 29, 2025.techcrunch.com/2025/04/29/meta-says-its-llama-ai-models-have-been-downloaded-1-2b-times/
  4. 4
    OpenAI. Sycophancy in GPT-4o: what happened and what we’re doing about it. April 29, 2025.openai.com/index/sycophancy-in-gpt-4o/
  5. 5
    OpenAI. Expanding on what we missed with sycophancy. May 2, 2025.openai.com/index/expanding-on-sycophancy/
  6. 6
    Waymo and Toyota. Waymo and Toyota Outline Strategic Partnership to Advance Autonomous Driving Deployment. April 29, 2025.waymo.com/blog/2025/04/waymo-and-toyota-outline-strategic-partnership/
  7. 7
    Microsoft Azure. One year of Phi: Small language models making big leaps in AI. April 30, 2025; original announcement.azure.microsoft.com/en-us/blog/one-year-of-phi-small-language-models-making-big-leaps-in-ai/
  8. 8
    Amazon Web Services. Amazon Nova Premier, our most capable model for complex tasks and teacher for model distillation. April 30, 2025.aws.amazon.com/about-aws/whats-new/2025/04/amazon-nova-premier-complex-tasks-model-distillation/
  9. 9
    NPR (via OPB). Catholic leaders criticize Trump for posting apparent AI photo of himself as the pope. May 4, 2025.www.opb.org/article/2025/05/04/catholic-leaders-criticize-trump-for-posting-apparent-ai-photo-of-himself-as-the-pope/
  10. 10
    OpenAI. Evolving OpenAI’s structure. May 5, 2025.openai.com/index/evolving-our-structure/
  11. 11
    Reuters (via The Kathmandu Post). OpenAI agrees to buy Windsurf for about $3 billion, Bloomberg News reports. May 6, 2025.kathmandupost.com/science-technology/2025/05/06/openai-agrees-to-buy-windsurf-for-about-3-billion-bloomberg-news-reports
  12. 12
    TechCrunch. Windsurf’s CEO goes to Google; OpenAI’s acquisition falls apart. July 11, 2025.techcrunch.com/2025/07/11/windsurfs-ceo-goes-to-google-openais-acquisition-falls-apart/
  13. 13
    IBM. IBM Accelerates Enterprise Gen AI Revolution with Hybrid Capabilities. May 6, 2025.newsroom.ibm.com/2025-05-06-ibm-accelerates-enterprise-gen-ai-revolution-with-hybrid-capabilities
  14. 14
    OpenAI. Introducing OpenAI for Countries. May 7, 2025.openai.com/global-affairs/openai-for-countries/
  15. 15
    DatacenterDynamics, reporting Reuters and Bloomberg. Trump administration to overhaul ‘overly complex, overly bureaucratic’ AI diffusion rule. May 9, 2025 (announcement of May 7).www.datacenterdynamics.com/en/news/trump-administration-to-overhaul-overly-complex-overly-bureaucratic-ai-diffusion-rule/
  16. 16
    Bureau of Industry and Security. Department of Commerce Announces Rescission of Biden-Era Artificial Intelligence Diffusion Rule, Strengthens Chip-Related Export Controls. May 13, 2025.www.bis.gov/press-release/department-commerce-announces-rescission-biden-era-artificial-intelligence-diffusion-rule-strengthens
  17. 17
    Amazon. Introducing Vulcan: Amazon’s first robot with a sense of touch. May 2025; launch reported May 7.www.aboutamazon.com/news/operations/amazon-vulcan-robot-pick-stow-touch
  18. 18
    WIRED. Amazon Has Made a Robot With a Sense of Touch. May 7, 2025.www.wired.com/story/ai-lab-amazon-launches-vulcan-a-robot-that-can-feel/
  19. 19
    US Food and Drug Administration. FDA Announces Completion of First AI-Assisted Scientific Review Pilot and Aggressive Agency-Wide AI Rollout Timeline. May 8, 2025.www.fda.gov/news-events/press-announcements/fda-announces-completion-first-ai-assisted-scientific-review-pilot-and-aggressive-agency-wide-ai
  20. 20
    US Food and Drug Administration. FDA Launches Agency-Wide AI Tool to Optimize Performance for the American People. June 2, 2025.www.fda.gov/news-events/press-announcements/fda-launches-agency-wide-ai-tool-optimize-performance-american-people
  21. 21
  22. 22
    SiliconANGLE. OpenAI and Microsoft engage in tense negotiations over the future of their partnership. May 11, 2025.siliconangle.com/?p=696024
  23. 23
    CNBC (via NBC San Diego). Nvidia sending 18,000 of its top AI chips to Saudi Arabia. May 13, 2025.www.nbcsandiego.com/news/business/money-report/nvidia-sending-18000-of-its-top-ai-chips-to-saudi-arabia/3824509/
  24. 24
    CNBC (via NBC Washington). White House announces AI data campus partnership with the UAE. May 15, 2025.www.nbcwashington.com/news/business/money-report/white-house-announces-ai-data-campus-partnership-with-the-uae/3914689/
  25. 25
    Google DeepMind. AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms. May 14, 2025.deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/
  26. 26
    Reuters (via Investing.com). Musk’s xAI updates Grok chatbot after “white genocide” comments. May 16, 2025; Reuters timestamp May 17 UTC.www.investing.com/news/stock-market-news/musks-xai-updates-grok-chatbot-after-white-genocide-comments-4051501
  27. 27
    California Department of Justice. Attorney General Bonta to Congress: California Must Retain its Ability to Protect Californians, Respond to Emerging AI Technology. May 16, 2025.oag.ca.gov/news/press-releases/attorney-general-bonta-congress-california-must-retain-its-ability-protect
  28. 28
    Office of Senator Edward J. Markey. After weeks of Markey raising the alarm, Senate strikes AI moratorium from budget reconciliation bill overnight in overwhelming 99-1 vote. July 1, 2025.www.markey.senate.gov/news/press-releases/after-weeks-of-markey-raising-the-alarm-senate-strikes-ai-moratorium-from-budget-reconciliation-bill-overnight-in-overwhelming-99-1-vote
  29. 29
    Google. AI in Search: Going beyond information to intelligence. May 20, 2025.blog.google/products-and-platforms/products/search/google-search-ai-mode-update/
  30. 30
    Google. Fuel your creativity with new generative media models and tools. May 20, 2025.blog.google/innovation-and-ai/products/generative-media-models-io-2025/
  31. 31
    Google. Meet Flow: AI-powered filmmaking with Veo 3. May 20, 2025.blog.google/innovation-and-ai/products/google-flow-veo-ai-filmmaking-tool/
  32. 32
    Malay Mail, reporting Reuters. Nvidia CEO Jensen Huang calls US AI chip export controls a failure. May 21, 2025.www.malaymail.com/news/money/2025/05/21/nvidia-ceo-jensen-huang-calls-us-ai-chip-export-controls-a-failure/177552
  33. 33
    Anthropic. Introducing Claude 4. May 22, 2025.www.anthropic.com/news/claude-4