Gemini 4 Carbon: a new top-1? A review of the available benchmarks

This article covers all benchmarks available so far (AA, Vals, Arena, and data from Google - I combined everything into a convenient chart and compared Carbon with Opus 5.5, Astra 6, and Argon - click if that's what you came for), as well as the key details of Google's new model, and also summarizes how the community received the release. First we'll find out what Google itself claims, then look at the benchmarks, and at the end check the community's opinion.

In brief: the new model Gemini 4 Carbon beats Opus 5.5 on benchmarks from Google, loses on benchmarks from Arena AI (where users do the voting), and competes with varying success in other tests. It wins on cost per task (i.e., it's cheaper), and is possibly better in cybersecurity. Without real user experience, a final assessment is premature; you can read about this separately in the section on how the community received the release. So, let's begin.

On October 28, 2026, Google unveiled Gemini 4 Carbon, a new model for complex programming, long-running tasks, and cyber defense. Access will be opened in stages: in the week of the announcement, a small group of cybersecurity specialists will start getting it, then Google will expand the circle of users through the Fairwind program, with no general launch date yet. For comparison: general access to Argon was opened 13 days after its announcement, almost two weeks.

By the way, Google CEO Sundar Pichai, in his post about Carbon, reminded people of the company's intention to release models periodically and emphasized the pace of updates: Carbon was announced 15 days after the broad launch of Argon.

What Gemini 4 Carbon can do

Carbon is built on Gemini 4 Argon. In Google's announcement, the main emphasis is on protecting software systems and working on complex tasks that require many actions. Here is exactly what the company claims about the new model.

Cyber defense: from finding a vulnerability to fixing it

Google tests Carbon at three stages of a security specialist's work: finding a vulnerability in code, checking whether it can actually be exploited, and preparing a fix. The second stage helps assess how dangerous a finding is and decide which problems to fix first.

In the tests Google presented, Carbon scored 88.2% on vulnerability discovery in SEC-bench Pro, 54.8% on exploitation verification in ExploitGym, and 71% on fixes in CWE-bench. These tests were run without the model's protective restrictions; the results and conditions are discussed in more detail below.

The company also reports that in its internal tests, Carbon successfully wrote secure code more than twice as often as Argon. The Chrome team, according to Google, got better fixes from Carbon than from one of the large models from another lab. The name of that model and the details of this comparison are not given in the announcement, so it is currently impossible to evaluate it independently.

Long-running work on complex code

Carbon is designed for tasks where an agent has to study files in sequence, look for the cause of a bug, try fixes, and keep going if the first approach doesn't work. Google gives the example of finding and fixing elusive bugs in large codebases and low-level systems.

According to the company, its teams watched Carbon work through such multi-step tasks over several days. This is Google's internal experience, not a guarantee that the public version will be able to run any project autonomously for the same length of time. The outcome depends on both the agent's tools and the constraints of the working environment.

Fewer fabricated results and going beyond the task

Google specifically trained Carbon to behave well in difficult situations: if data is insufficient or an action is blocked, the model should report this rather than present unverified output as completed work. In its internal tests, the company recorded roughly half as many unconfirmed claims as Argon.

The company also reports that, in its evaluations, Carbon went beyond the assignment 26% less often and under-delivered on it 29% less often. These are results from Google's specific tests, not a guarantee of an absence of errors. You cannot independently verify them on your own project yet without access to the model.

How Carbon differs from Argon

Argon can already be connected to an application through the public API. Carbon is currently in limited launch: Google has described the model's improvements but has not yet published a detailed developer specification. As a result, some of the new model's technical characteristics remain unknown.

Argon specifications come from the Gemini API documentation, and Carbon details come from the Google announcement. "Not yet specified" means the data is absent from the published specification, not that the feature is missing. Context, formats, and tools from Argon cannot be carried over to Carbon without confirmation.

The main announced change in Carbon concerns long-running agent work. Google states that the model handles complex engineering tasks better, produces unconfirmed claims less often, and adheres more precisely to the boundaries of an assignment. These are improvements based on the company's internal observations; how they will show up in the public version remains to be tested.

Google's Carbon API will cost $2 per million input tokens and $10 per million output tokens at the announced introductory pricing. After the introductory period, prices will rise to $4 per million input and $20 per million output tokens. Cached input tokens under the introductory pricing will be 95% cheaper than regular input. These are the same rates Google announced for Argon. Google did not specify an end date for Carbon's introductory period (did not specify).

Independent test results: Arena, Artificial Analysis and Vals

WebDev Arena

WebDev Arena compares models on web development tasks based on user ratings. In the October 27 snapshot of the ranking, Gemini 4 Carbon in High mode takes fourth place with 1768 points.

Ahead of Carbon are Claude Opus 5.5 Max with 1810 points, GPT-6 Astra Max with 1773 and Claude Sonnet 5.5 xHigh with 1770. Behind it are GPT-6.1 Sol Max with 1751 and Claude Fable 5.1 Max with 1746. Its predecessor, Gemini 4 Argon High, takes 11th place with 1679 points.

Carbon's result is preliminary: the screenshot accounts for 2230 votes, and its possible ranking range is from second to sixth place. Points and places change as new ratings come in; Google's announcement cites 1769 points for Carbon.

In head-to-head comparisons, Claude Opus 5.5 Max beats Gemini 4 Carbon High in 69% of duels in the WebDev Arena snapshot shown. The matrix below shows other pairs as well: the value in a model's row reflects its result against the model in the corresponding column.

The ranking table above also shows the estimated median cost per task: $1.62 for Carbon, $9.01 for Opus 5.5 Max and $9.38 for GPT-6 Astra Max. This is an Arena metric, not the price of a task in GPTunneL. You should check your own cost once you have access to Carbon.

Agent Arena: results on user tasks

In Agent Arena, models are rated using signals from real agent mode sessions. In the October 28 snapshot, Carbon High takes second place with a Net Improvement of +16.04 pp, behind Opus 5.5 Max at +20.94 pp. For comparison: Astra Max is fifth at +12.96 pp, and Argon High is seventh at +10.73 pp. Carbon's result is marked as preliminary; its possible place range is from second to fourth.

The table lists all ten models from the snapshot; values and intervals are given in percentage points. Arena shows 95% confidence intervals: close results with overlapping intervals cannot be reliably ranked by a single point estimate.

According to Arena's methodology, Net Improvement reflects the estimated change relative to the average model across several signals, including confirmation of task completion, user reactions, and the ability to account for corrections. This is not the share of solved tasks. The individual metrics Confirmed Success and Praise vs Complaint in the ranking also express deviation from a baseline level, not an absolute success or praise percentage.

On this snapshot, the median task cost is $1.74 for Carbon, $6.70 for Opus, $5.25 for Astra, and $1.09 for Argon. These are task costs in Agent Arena; they are not equal to the WebDev Arena metric and are not a GPTunneL tariff. Carbon is based on 26,511 sessions; the whole ranking contains 2,875,897 sessions; the number of observations for individual signals may differ.

Artificial Analysis Index and Google benchmarks

In the Artificial Analysis Intelligence Index v5.0, Carbon in High mode scored 59 points and took first place among the evaluated models. Next come Claude Opus 5.5 with 58 and GPT-6 Astra with 57 points, while Argon scored 50. This is the set of results Google presented in the comparison table below.

The table shared by Pichai collects different benchmarks, not a breakdown of a single Artificial Analysis index. The comparison was prepared by Google DeepMind; the Terminal-Bench 4.0 and Terminal-Bench-Science 0.1 results are marked as AA evaluations. For example, in Terminal-Bench 4.0 from AA, Carbon scored 65.2% versus 57.1% for Argon and 59.6% for Claude Opus 5.5. In FrontierSWE 2, Carbon scored 61.2% and fell behind GPT-6 Astra at 65.5% and Opus 5.5 at 62.3%.

The index combines nine tests, including programming, reasoning, mathematics, and scientific tasks. Artificial Analysis states that it conducts the evaluations itself. This is a useful general benchmark, but it does not replace a test on your own task: the weight of individual tests and model settings affect the outcome.

Another trap is comparing the new 59 points with numbers from the old version of the index and declaring an exact gain. Version v5.0 uses an updated methodology, so it is better to compare results obtained within the same version of the ranking.

How much the result costs. On the provided Artificial Analysis snapshot, the weighted average cost of one index task is estimated at $7.76 for Carbon High. For comparison: GPT-6 Astra Max is $6.24, and Claude Opus 5.5 Max with a fallback model is $10.04. It turns out that in this set of tests, Carbon scored more points than these two models, at a cost between them.

The horizontal axis of the chart shows task cost in dollars, on a logarithmic scale; the vertical axis shows the index score. The higher and further left the point, the better the result at lower cost. Artificial Analysis calculates cost based on token usage in individual tests and accounts for the weight of each test in the index.

This is a calculation for the benchmark tasks, not the price of an ordinary message or a GPTunneL tariff. As of the snapshot date, Carbon did not yet have a public API, so its cost here should be treated as an estimate that needs to be rechecked once access opens.

Vals Index: Carbon's strengths by area

In the Vals Index, Gemini 4 Carbon ranks second out of 29 with a score of 70.36%. Ahead of it is Claude Sonnet 5.5 at 70.85%, and behind Carbon is Claude Opus 5.5 at 70.18%. The index combines tasks in finance, programming, law, and tax, and the areas are weighted by their share of the US economy. This is therefore a score on a specific set of practical tasks, not a universal intelligence ranking.

On the snapshot of Carbon's profile on Vals, the positions in individual areas and tests look like this:

This is a snapshot of the rankings as of October 29, 2026. Positions may change as models and results are added. The indexes and individual tests measure different sets of tasks, so their percentages cannot be compared directly with one another. For example, on HLAB-Vals-1.1, Carbon trailed Argon, which scored 27.08%: the new model is not stronger than its predecessor on every test. Vals' Terminal-Bench result also differs from the Artificial Analysis score cited above, because these are separate runs by different evaluators.

Price, tokens, and time. Vals shows not only accuracy but also the cost of its tests. Below are all 11 models from the provided ranking snapshot:

In this set, Carbon achieved a result close to Sonnet's at a lower cost and in about half the time. Compared with Opus, its estimated cost is about 2.7 times lower. These are Vals figures for the index tasks, not the cost of an ordinary request or a GPTunneL tariff. The number of tokens alone does not determine the price, because input and output tokens are billed differently. As of the snapshot, Carbon does not yet have a public API, so the cost estimate should be rechecked once it opens.

How the index works. Vals takes the average of test results within each domain, then combines the domains using weights based on each corresponding industry's share of US GDP: finance and insurance — 8.0, the information sector as a proxy for programming — 5.6, legal services — 1.2, accounting and tax services — 0.5. The sum is divided by 15.3. Finance and programming thus account for most of the final score. This is a useful reference point for the chosen methodology, but it simplifies the real economic impact of AI.

In the breakdown, Carbon scored 76.5% on EMB financial modeling, 63.1% on Terminal-Bench, 89.1% on Vibe Code, and 80.1% on the Code Migration sample used by the index. For comparison, Argon scored 74.8%, 57.6%, 91.9%, and 74.4%, respectively. Carbon leads on three of these four checks, and Argon is stronger on Vibe Code. The figures are rounded as in the Vals table. The Code Migration sample used for the index differs from the full version of the test.

Why is Sonnet ahead of Opus? In this version of the index, programming tasks carry significant weight. According to the Vals breakdown, Sonnet scored 87.4% versus 80.5% for Opus on the Code Migration sample counted by the index, and 92.4% versus 90.3% on Vibe Code Bench. Meanwhile, Opus leads on Terminal-Bench and several other tests. The outcome depends on the combination of results and weights, not on the model line's name or price: Vals shows cost separately from the accuracy score.

The difference between Sonnet and Opus is only 0.67 percentage points. Vals gives standard errors of ±0.93 and ±1.00 points for their scores, respectively. One such slice is not enough to treat the small lead as proof that Sonnet is reliably stronger than Opus on all working tasks.

How the model finds and fixes vulnerabilities

Cybersecurity is the main focus of Google's announcement. The company presents results from three external tests:

Google's important caveat: these cyber tests were run without the model's protective restrictions. The results of such tests cannot be automatically transferred to a future version for all users.

In the CWE-bench table, Carbon is also listed with a 71% result on the automated pass@1 check; Argon has 68%. The test has 120 tasks, four attempts are allowed for each model, and up to one hour is given per attempt. Both Google models in this table ran through Antigravity; other participants may have used different agentic tools.

The practical meaning of the three-percentage-point difference is clear only together with the test conditions. Even a high result does not mean the model will find all vulnerabilities in your application or that its fixes can be accepted without review.

Has the model become more reliable?

Google separately writes about the agent's behavior during operation. In the company's own tests, Carbon roughly halved the number of unconfirmed claims compared with Argon, and in Google's team work environments, the number of serious behavioral mismatch signals fell by almost 80%.

Protection against indirect prompt injections

In the Gray Swan IPI test, Google reports 0% successful attacks on Carbon at a score for 15 attempts. Here, lower is better: for comparison, the chart shows 0.7% for Argon and 1% for Claude Opus 5.5. The result applies to the specific attacks tested and the test conditions.

An indirect prompt injection is an instruction that an attacker hides in external material, for example on a web page, in a document, or in an email that an agent reads. It may try to make the model ignore the user's task and follow someone else's instruction.

Protection matters most for AI agents that both read content the user does not control and can take actions: running tools, changing files, or sending data. Without resistance to such instructions, an attacker may try to control the agent through material it opened for work.

Zero on this test is a strong result, but not a guarantee of protection from all future attacks. The public version still needs to be checked in practice.

So for coding work, I would keep the usual checks: limit the agent's access to the needed folder, review the changes, run the tests, and separately verify sensitive areas. The model can speed up the search and prepare a fix, but responsibility for applying the changes stays with the team.

How Gemini 4 Carbon Was Received in the Community

It makes sense to look not only at benchmarks and vendor claims, but also at what experienced users who have been through many releases like this think. I analyzed popular posts in the community, and the following key points stand out:

1. Skepticism about bench-maxxing: commenters call Gemini 4 Carbon's high test results "paper superiority." Google has historically been criticized for optimizing models to pass specific tests (for example, SWE-bench) at the expense of everyday developer convenience. The previous model, Argon, was also presented as revolutionary but in practice fell short of the same Opus 5.5.

In general, we already see a divergence in model rankings between Artificial Analysis and Arena AI. On the first, the model leads the ranking, but on Arena it sits in 2nd to 6th place and loses 70% of pairwise duels to Claude Opus 5.5.

2. Availability problem: everyone is annoyed that Carbon is currently held for security review by the US government. Commenters joke that by the time it actually launches, Anthropic will have released the next versions of Opus and Fable.

3. Absurd naming: there is a lot of irony about how understanding Google's model lineup will soon require a chemistry PhD. The Carbon model (carbon, atomic number 6) came after Argon (argon, number 18), breaking all logic of the periodic table. Commenters joke that if the model is based on carbon-14, its half-life means it will lose its weights in 5,730 years, and if it's based on carbon-12, then it's a stable release. The naming is so confusing that someone suggested Gemini Tokyo Drift as the name for Google's next AI model 😄

4. Interface problems: many complain about Google's ecosystem. The new features can only be fully used through the Antigravity CLI tool, which many call "the worst and most awkward coding framework." People also recall their experience with Carbon: when limits run out, the Carbon chat automatically switches to the weaker 3.5-lite model, completely erasing the context history.

5. Restrained optimism: some are glad that Google is returning to the frontier model race and are willing to give the new model a chance.

To sum up: there is a lot of skepticism about bench-maxxing, and an expectation that among frontier models Opus/Fable will still remain the leader.

Summary Comparison of Google Carbon, Opus 5.5, Astra 6, Argon

Let's try to determine which frontier model is better: the new Google Carbon or Claude Opus 5.5. I also added Astra 6 and Argon for convenience, but hardly anyone expects them to be better. Spoiler: Carbon wins on Google's benchmarks, but users on Arena AI choose Opus. In other independent benchmarks, the picture is different.

I gathered the results from the breakdowns above into one table. Each row has its own source and methodology, so models can be compared within a single row, but scores from different tests should not be added together into an overall total.

The full comparison table is below. The best result among the four models in each row is highlighted in green: for scores and accuracy, higher is better; for place, cost, time, tokens and successful attacks, lower is better. This is not necessarily the leader of the overall ranking or a statistically significant advantage.

Where Carbon looks most promising. The model is worth considering for finding and fixing vulnerabilities, complex multi-step coding tasks, long-document analysis, workflow automation, and financial research. In cybersecurity and long-context tests it showed strong results; in the Coding Agent Index, the Carbon and Antigravity pairing trailed Opus with Claude Code but outperformed the Astra and Argon pairings shown. These are areas to test after general access opens: code fixes and analytical conclusions need to be verified, and on individual tasks competitors may be stronger.

Sources: Artificial Analysis, WebDev Arena, Vals Index and breakdown by test, Google announcement. Arena results are given for Carbon and Argon in High mode, and for Opus and Astra in Max mode; setups and tools may differ across evaluators. Vals scores for individual tests are rounded to one decimal place. A dash means a comparable value is not provided in the data used above. Cost and time refer to the tasks of the named evaluators, not to ordinary requests or GPTunneL pricing.

Agent Arena - snapshot from October 28: signal values are given in percentage points relative to the average model, and Carbon's result is preliminary. For example, +18.05 pp on Confirmed Success does not mean it completed 18.05% of tasks. Rankings and scores may change; score variance matters when differences are small.

Rows marked "Google table" reproduce the comparison from the Google DeepMind announcement; Terminal-Bench 4.0 and Terminal-Bench-Science 0.1 are labeled there as Artificial Analysis scores. Cybersecurity tests were run without the model's safety restrictions. GraphWalks measures long-context performance, not the size of Carbon's public context window. The Vending-Bench 2 balance is the result of an agent work simulation, not a task cost.

Coding Agent Index v2.0 evaluates model-and-agent combinations: Carbon and Argon were tested in Antigravity CLI in High mode, Opus in Claude Code in Max mode with a fallback model, and Astra in Codex in Max mode. Costs are calculated for the index tasks at API rates; Astra's result and its cost are given for Codex. Results depend on both the tool and the model.

The cost of the Vals tasks is taken from the tables of the corresponding tests: Finance Agent, EMB, Terminal-Bench, Vibe Code, Legal Research, HLAB and Tax Agent. For Code Migration, the estimated cost of the full test is given, while the accuracy in the row above is based on the index sample; the cost of Carbon for the full test is not specified in the available table. Prices do not account for result quality and are not GPTunneL tariffs.

Carbon leads among these four models on the Intelligence Index and Vals Index, but trails Opus and Astra on WebDev Arena. In individual Vals tests, it is also not first everywhere: for example, Opus is stronger on Terminal-Bench, and Argon on HLAB. The difference in the overall Vals scores between Carbon and Opus is small, so the choice is worth checking on your own tasks.

What Is Still Unknown

Google did not specify in its announcement the exact date of general availability, Carbon's context window size, or the public model identifier for the API. These cannot be automatically taken from Argon's specifications. Real-world speed, the price of a typical task, and the quality of work on your own code also still need to be verified.

For now, Carbon should be viewed as a promising model with strong early results and limited access. If you are already working with Argon, save one representative task and its result: once Carbon opens up, you can compare the models on the same prompt, with the same tools and tests.

If you need an available model from the family right now, you can look at Gemini 4 Argon and our review of its launch.