Dataproc

Note

Currently in beta (pre-v1.0), and may see breaking changes until the first stable release (v1.0).

This repository packages MCP Toolbox's prebuilt dataproc server as a plugin/extension to interact with Dataproc clusters and jobs. It can be used with various AI agents, including Antigravity, Claude Code and Codex, to manage your clusters, monitor jobs, and troubleshoot issues using natural language prompts.

Important

We Want Your Feedback! Please share your thoughts with us by filling out our feedback form. Your input is invaluable and helps us improve the project for everyone.

Table of Contents

• Why Use Dataproc?
• Prerequisites
• Getting Started
  • Configuration
  • Installation & Usage
    • Antigravity
    • Claude Code
    • Codex
• Installing via a compatible Agent Plugins client
• Usage Examples
• Available Tools
• Generating Skills Instead
• Troubleshooting

Why Use Dataproc?

• Seamless Workflow: Integrates seamlessly into your AI agent's environment. No need to constantly switch contexts for common Dataproc tasks.
• Natural Language Queries: Stop wrestling with complex gcloud commands. Manage your clusters and jobs by describing what you want in plain English.
• Full Lifecycle Control: Manage the entire lifecycle of your Dataproc resources, from listing clusters to checking job statuses.

Prerequisites

Before you begin, ensure you have the following:

• One of these AI agents installed
  • Antigravity
    • Antigravity CLI version v1.6.0 or higher
    • Antigravity 2.0 version v2.0.0 or higher.
  • Claude Code version v2.1.94 or higher.
  • Codex v0.117.0 or higher.
• Node.js — the MCP server runs via npx.
• A Google Cloud project with the Dataproc API enabled.
• Ensure Application Default Credentials are available in your environment.
• IAM Permissions:
  • Dataproc Viewer (roles/dataproc.viewer) or Dataproc Editor (roles/dataproc.editor)

Getting Started

Configuration

Please keep these env vars handy during the installation process:

• DATAPROC_PROJECT: The GCP project ID.
• DATAPROC_REGION: The region of your Dataproc resources.

Note

• Ensure Application Default Credentials are available in your environment.
• If your Cloud SQL for PostgreSQL instance uses private IPs, you must run your agent in the same Virtual Private Cloud (VPC) network.

Installation & Usage

To start interacting with your database, install the extension for your preferred AI agent, then launch the agent and use natural language to ask questions or perform tasks.

For the latest version, check the releases page.

Antigravity

You can use either of these two agents for Antigravity:

• Antigravity CLI version v1.6.0 or higher
• Antigravity 2.0 version v2.0.0 or higher.

💡 Tip — Migrating from Gemini CLI?
If you previously installed this extension with gemini extensions install, you can convert it to an Antigravity plugin instead of reinstalling from scratch:

• On first launch of Antigravity CLI, accept the Migration Options prompt to automatically convert your installed Gemini CLI extensions to Antigravity plugins.
• Or, from your terminal, run:

  agy plugin import gemini

See Migrating from Gemini CLI for details on plugins, context files (GEMINI.md / AGENTS.md), and MCP server config differences.

Antigravity 2.0 (IDE)

1. Install the plugin:

Install the plugin directly from the remote GitHub repository:

agy plugin install https://github.com/gemini-cli-extensions/dataproc

2. Set env vars: Set your environment vars as described in the configuration section.

(Tip: You can verify the MCP server is active by running the /mcp command in your active session.)

Antigravity CLI

You can install plugins directly from a remote GitHub repository.

1. Install the plugin:

agy plugin install https://github.com/gemini-cli-extensions/dataproc

2. Set env vars: Set your environment vars as described in the configuration section.

Claude Code

1. Set env vars: In your terminal, set your environment vars as described in the configuration section.

2. Start the agent:

claude

3. Install the plugin:

/plugin install dataproc@claude-plugins-official

_(Tip: Run /plugin list inside Claude Code to verify the plugin is active, or /reload-plugins if you just installed it.)

Codex

1. Install marketplace:

codex plugin marketplace add GoogleCloudPlatform/data-cloud-plugins

2. Install the plugin:

codex plugin add dataproc@data-cloud-plugins

3. Set env vars: Enter your environment vars as described in the configuration section.

4. (Optional) Update the marketplace:

codex plugin marketplace upgrade data-cloud-plugins

Installing via a compatible Agent Plugins client

Installing via a compatible Agent Plugins client

This repository is a valid Agent Plugins (v1) plugin. Any Agent Plugins–compatible client can install it directly using its own built-in plugin command — no extra tooling required — by pointing at this repository:

https://github.com/gemini-cli-extensions/dataproc

Beyond harnesses covered by the native install above, compatible clients include VS Code, Cursor, GitHub Copilot, and Kiro. See your agent's documentation for its exact install command.

Set env vars: Set your environment vars as described in the configuration section.

Usage Examples

Interact with Dataproc using natural language:

• List Clusters:
  • "List all my Dataproc clusters in us-central1."
• Check Jobs:
  • "Show me the status of the job with ID 'my-spark-job-123'."
  • "List all failed jobs in my project."
• Get Details:
  • "Get details for the cluster named 'my-cluster'."

Available Tools

The tools come from MCP Toolbox's prebuilt dataproc server, grouped into toolsets:

• dataproc_tools - Skills to interact with your Dataproc clusters and jobs.

For the full, up-to-date list, see the dataproc prebuilt config in the MCP Toolbox repository.

Generating Skills Instead

The tool-backed skills this plugin used to ship were generated from the same prebuilt toolsets. If your agent lacks deferred tool loading, or you prefer skills, regenerate them with the script in this repository:

VERSION=<toolbox version> ./.github/scripts/generate_skills.sh

Use the toolbox version pinned in mcp.json. A single toolset, without the script:

npx @toolbox-sdk/server@<toolbox version> --prebuilt dataproc skills-generate \
  --name "<skill name>" \
  --toolset "<toolset>" \
  --description "<what it is for>"

The generated scripts call the toolbox through npx, so no binary download is needed. See Generate Agent Skills in the MCP Toolbox repository.

Troubleshooting

Use the debug mode of your agent (e.g., gemini --debug) to enable debugging.

Common issues:

• "failed to find default credentials": Ensure Application Default Credentials are available in your environment.
• "cannot execute binary file": The Toolbox binary did not download correctly.