Proxy HTML API: Fetch Web Pages Without Element Mapping

Mapped scrapes convert web pages into clean, schema-stable JSON—the ideal format when feeding structured variables to RAG pipelines or AI agents. However, developers and LLM scrapers often need raw, fully rendered page HTML through stealth proxies so custom DOM parsers, BeautifulSoup scripts, or headless AI agents can handle extraction directly.

That is why we built the Proxy HTML API.

TL;DR: POST /api/v1/proxy delivers plain, fully rendered HTML through Gluedly’s rotating proxy and anti-bot bypass stack without requiring visual mapping, running /map steps, or saving data snapshots.

What Shipped

• Direct HTML Endpoint (POST /api/v1/proxy): Fetch any public URL through rotating residential proxies and receive plain HTML or structured JSON.
• Granular Scope Control (proxy_fetch): Keep operational security tight by issuing dedicated API keys with proxy_fetch permissions (isolated from execute_scrape).
• Reusable Page Configurations: Pass a page_id to automatically inherit existing JavaScript rendering and premium anti-bot settings.
• Dynamic Execution Flags: Toggle js_rendering and use_premium proxies dynamically on a per-request basis for unmapped URLs.
• Flexible Response Formats: Receive structured JSON by default (url, html, credits_spent) or stream raw HTML documents directly using Accept: text/html or ?format=html.
• Native Remote MCP Tool (gluedly_fetch_html): Seamless integration for local or remote Model Context Protocol (MCP) clients and autonomous AI agents.
• Success-Only Proxy Billing: Uses identical credit tiers as mapped scrapes (standard, JS rendering, premium). You are only charged when a valid, billable HTML body is delivered.

Note: The Proxy HTML API does not trigger visual /map steps, write Data snapshots, alter page status, or execute data delivery webhooks.

When to Use Proxy HTML vs. Mapped JSON

• Use Proxy HTML (/api/v1/proxy): When you already have custom DOM parsing logic, need to inspect raw HTML payloads behind rotating proxies, or are building LLM agents that read full web DOMs.
• Use Mapped JSON (/execute): When you need Gluedly to automatically extract clean key-value variables, Markdown, or schema-valid JSON without running local parsing scripts.

How to Call the Proxy HTML API

Step 1: Enable the proxy_fetch Scope

Navigate to your workspace dashboard, create or edit an API key, and assign the proxy_fetch permission scope (requires an active paid plan).

Step 2: Send a POST Request to /api/v1/proxy

Send an HTTP POST request containing your Bearer token in the Authorization header. Pass the target url in the JSON payload, or provide a page_id to reuse pre-configured page settings.

Step 3: Parse Your HTML Payload

Extract the html string from the default JSON response, or pass Accept: text/html to receive raw HTML along with X-Gluedly-Credits-Spent and X-Gluedly-Url response headers.

Direct URL (JSON Response)

POST /api/v1/proxy
Authorization: Bearer 
Content-Type: application/json
Accept: application/json

{
  "url": "https://example.com/products",
  "js_rendering": true,
  "use_premium": false
}

Example JSON Response:

{
  "url": "https://example.com/products",
  "html": "Products...",
  "credits_spent": 2,
  "js_rendering": true,
  "use_premium": false,
  "page_id": null
}

Reusing Pre-configured Page Settings

Pass a page_id to inherit dynamic JS rendering rules and premium anti-bot bypass settings automatically:

{
  "page_id": 42,
  "url": "https://example.com/products"
}

Note: When page_id is set, that page's js_rendering and use_premium flags win. Request-level JS/premium flags are ignored. Workflow and experimental Facebook pages are rejected.

Streaming Plain / Raw HTML Responses

To bypass JSON wrappers and receive a raw document stream directly into your parser, set your headers or query parameters:

POST /api/v1/proxy?format=html
Authorization: Bearer 
Accept: text/html

{
  "url": "https://example.com/products"
}

Returned HTTP Response Headers:

• X-Gluedly-Credits-Spent: 2
• X-Gluedly-Url: https://example.com/products

Transparent, Success-Only Proxy Billing

Proxy fetches calculate credit costs identically to mapped scrapes based on the execution features enabled (standard, JS rendering, premium). Short, empty, or failed bodies are not charged.

Documentation & AI Agent Integrations

Full contract details live in the API Documentation and our machine-readable llms-full.txt configuration.

Building with Claude, LangChain, or custom LLM frameworks? Remote MCP automatically exposes gluedly_fetch_html with the same arguments (url, optional page_id, js_rendering, use_premium).