NodeAI documentation

NodeAI is a control plane, not a model provider. You connect machines running a local inference runtime (vLLM today), label how much you trust each one, and set a policy for which data sensitivity may run where. Every request is then routed to a node that satisfies the policy.

The part that is hard to build yourself is failure handling: if the node serving a reply dies mid-sentence — the process is killed, the GPU falls over, the network drops — NodeAI regenerates that reply on another eligible node inside the same response, without relaxing the trust or compliance requirements to do it. If no eligible node exists, the request fails visibly rather than quietly running somewhere it should not.

Trust tiers are labels you assign: Tier 1 for hardware you own, Tier 2 for contracted cloud, Tier 3 for anything else. Sensitivity labels the data in a conversation. A conversation marked sensitive or regulated will only ever run on Tier 1, and the boundary can tighten across turns but never loosen.

A 60-second recording: Claude asks NodeAI to run a prompt, the node serving it is killed, and NodeAI regenerates the whole reply on the second node. The session record shows one failover and both nodes.

NodeAI is an MCP server with OAuth, so Claude connects by signing you in — there is no API key to copy.

1. In Claude: Settings → Connectors → Add custom connector.
2. Enter https://nodeai.chromeo.io/api/mcp as the URL. Enter it exactly, path included.
3. Claude discovers the authorization server and sends you to NodeAI. Sign in (or create an account), review the permissions, and approve.
4. You are returned to Claude with the connector live. It appears in NodeAI under Connection → Connected apps, where you can disconnect it at any time.

For Claude Code, the same endpoint works from the CLI:

claude mcp add --transport http nodeai https://nodeai.chromeo.io/api/mcp

Claude Code runs the OAuth flow on your machine and opens the same consent screen in your browser.

NodeAI's MCP server is standards-based, so ChatGPT connects to the same endpoint over the same OAuth flow. Today that means either enabling developer mode on a paid ChatGPT plan and adding https://nodeai.chromeo.io/api/mcp as a custom MCP server, or — on Business, Enterprise or Edu — having an admin publish it to the workspace.

A listing in ChatGPT's public directory additionally requires OpenAI's review; until then the two paths above are how it is used.

Permissions are granted per connection and shown on the consent screen. A connector is offered the two it needs and never admin by default.

• read — list your nodes and their health, explain where a request would run and why, read a conversation transcript, read the audit log.
• infer — run a prompt on your fleet through the policy-aware router, including mid-reply failover. This consumes compute on your own machines and, if you rent them, costs you money.
• admin — quarantine, release or revoke a node, and change a node's trust tier. Requested only if an application explicitly asks, and never implied by the other two.

Read-only tools are marked as such, so an assistant can call them without prompting you each time; anything that changes your fleet is marked destructive and asks first.

NodeAI is its own authorization server. It implements OAuth 2.1 with mandatory PKCE (S256), RFC 9728 protected resource metadata, RFC 8414 authorization server metadata, RFC 8707 resource indicators with audience validation, RFC 9207 issuer identification, RFC 7591 dynamic client registration, RFC 7009 revocation, and Client ID Metadata Documents.

Discovery:

https://nodeai.chromeo.io/.well-known/oauth-protected-resource/api/mcp
https://nodeai.chromeo.io/.well-known/oauth-authorization-server

Endpoints:

authorization  https://nodeai.chromeo.io/oauth/authorize
token         https://nodeai.chromeo.io/api/nodeai/oauth/token
registration  https://nodeai.chromeo.io/api/nodeai/oauth/register
revocation    https://nodeai.chromeo.io/api/nodeai/oauth/revoke

Access tokens last one hour and refresh tokens thirty days, rotating on every use. Tokens are opaque and stored only as hashes, which is what lets a disconnect take effect on the very next request instead of when a token would have expired.

For direct programmatic access, create an API key in the console under Connection → API keys. Both major wire formats are supported, so most SDKs work unchanged.

OpenAI-compatible — point any OpenAI SDK at the base URL:

base_url = "https://nodeai.chromeo.io/api/nodeai/v1"
api_key  = "nai_..."

Anthropic-compatible — including Claude Code, via environment variables:

export ANTHROPIC_BASE_URL=https://nodeai.chromeo.io/api/nodeai/anthropic
export ANTHROPIC_AUTH_TOKEN=nai_...
export ANTHROPIC_MODEL=<a model your fleet serves>

Model names are not translated: use an id your own nodes serve. List them at GET /api/nodeai/v1/models. Tool calling works in both formats, and a reply that fails over mid-stream is marked in the stream so a client rendering it live can discard the abandoned text.

A node joins your fleet by running the Compute Agent next to its inference runtime. Download it from the console, then:

npm ci
npm run build
npm start -- enroll   # pairs the node with your account
npm start -- start    # keeps it connected

The agent discovers the GPU with nvidia-smi and the runtime at NODEAI_VLLM_URL (default http://127.0.0.1:18000), reports which models are served and the context window, and holds a persistent connection so work can be dispatched to it. It sends a heartbeat every 30 seconds, and while a reply is streaming it sends one every 3 seconds — which is how a node that dies silently is noticed in about ten seconds instead of a minute.

A node cannot promote itself: trust tier and compliance are set by the account owner in the console, and NODEAI_TRUST_TIER is ignored if set.

Tool calling needs a runtime that can parse tool calls. For vLLM, start it with a parser for your model family:

vllm serve <model> --enable-auto-tool-choice --tool-call-parser hermes

Use hermes for Qwen, llama3_json for Llama 3.1/3.2, mistral for Mistral. The agent probes for this and reports the result; NodeAI will not route a request carrying tools to a node that cannot serve it, and a tool-calling request already in flight fails over to one that can.

The path a request takes:

client (Claude / ChatGPT / your code)
  -> OAuth access token or API key
  -> NodeAI: identity, scopes, policy, routing
  -> your GPU node over an authenticated socket
  -> vLLM

Three credential types exist and they are deliberately not interchangeable. A browser session is you, in the console. An API key is your own code, holding the permissions you gave it. An OAuth token is a third-party application, holding only what you consented to. A node's workstation credential is a machine credential for one node and is never accepted as a user credential.

Every request re-resolves identity and permissions from storage, so revoking an application or a key stops it on the next call. Requests carrying tools are only routed to nodes that verified tool support; conversations carrying sensitive data are only routed to nodes at or above the required trust tier; and a mid-reply failover re-applies the same requirements rather than relaxing them to keep the reply alive.

Passwords, API keys, OAuth tokens and node credentials are stored only as hashes. See the Privacy Policy for what is stored and for how long — including the fact that prompts and replies are stored.

"No eligible node" / "no node satisfies this conversation". The Runtime panel gives a verdict per node with the exact reason: offline, no recent heartbeat, does not serve this model, trust tier below what the data requires, not marked compliant while your policy requires it, or tool calling not enabled.

A connector will not finish authorizing. Check the URL is the MCP endpoint exactly, https://nodeai.chromeo.io/api/mcp. You can confirm the server is advertising discovery correctly:

curl -s https://nodeai.chromeo.io/.well-known/oauth-protected-resource/api/mcp
curl -si -X POST https://nodeai.chromeo.io/api/mcp | head -3

The second should answer 401 with a WWW-Authenticate header pointing at the first. That challenge is how a client discovers where to authorize.

Tool calls fail on one node only. That runtime was started without a tool parser; see tool calling. NodeAI routes around it, so the symptom is usually slower first tokens rather than an error.

Anything else: support.