Update .env.sample and Ansible configurations to enhance Pangolin Integration API setup. Add detailed comments for environment variables and clarify usage in README. Implement HTTP-01 challenge support in cert-manager configurations for Let's Encrypt, ensuring proper resource management for domain validation.
This commit is contained in:
10
.env.sample
10
.env.sample
@@ -13,12 +13,16 @@ NEWT_ID=
|
|||||||
NEWT_SECRET=
|
NEWT_SECRET=
|
||||||
|
|
||||||
# Optional: Pangolin Integration API — automate public HTTP resources + Traefik targets (**noble_pangolin_sync_http_resources=true** in **group_vars**; see **clusters/noble/bootstrap/newt/README.md** §4).
|
# Optional: Pangolin Integration API — automate public HTTP resources + Traefik targets (**noble_pangolin_sync_http_resources=true** in **group_vars**; see **clusters/noble/bootstrap/newt/README.md** §4).
|
||||||
# NOBLE_PANGOLIN_API_BASE=https://api.your-pangolin.example/v1
|
# NOBLE_PANGOLIN_API_BASE=https://api.example.com/v1 # Integration API — separate host from the main Pangolin UI; see clusters/noble/bootstrap/newt/README.md §4
|
||||||
# NOBLE_PANGOLIN_ORG_ID=
|
# NOBLE_PANGOLIN_ORG_ID=
|
||||||
# NOBLE_PANGOLIN_API_TOKEN=
|
# NOBLE_PANGOLIN_API_TOKEN= # **apiKeyId.apiKeySecret** (one value, dot in the middle) from Organization → API keys — **not** login password; browser cookies do not apply. Alternatively: secret only here + **NOBLE_PANGOLIN_API_KEY_ID** below.
|
||||||
# NOBLE_PANGOLIN_SITE_ID=
|
# NOBLE_PANGOLIN_API_KEY_ID= # optional; if set, **NOBLE_PANGOLIN_API_TOKEN** may be the secret half only
|
||||||
|
# NOBLE_PANGOLIN_SITE_ID= # numeric siteId, or Pangolin **niceId** (Sites UI slug, e.g. unruly-asian-badger)
|
||||||
# NOBLE_PANGOLIN_TRAEFIK_IP=192.168.50.211
|
# NOBLE_PANGOLIN_TRAEFIK_IP=192.168.50.211
|
||||||
# NOBLE_PANGOLIN_TRAEFIK_PORT=443
|
# NOBLE_PANGOLIN_TRAEFIK_PORT=443
|
||||||
|
# Self-signed Integration API TLS: either trust your CA (preferred) or homelab-only skip verify:
|
||||||
|
# NOBLE_PANGOLIN_CA_BUNDLE=/path/to/ca.pem
|
||||||
|
# NOBLE_PANGOLIN_INSECURE_SKIP_TLS_VERIFY=true
|
||||||
|
|
||||||
# Velero — when **noble_velero_install=true**, set bucket + S3 API URL and credentials (see clusters/noble/bootstrap/velero/README.md).
|
# Velero — when **noble_velero_install=true**, set bucket + S3 API URL and credentials (see clusters/noble/bootstrap/velero/README.md).
|
||||||
NOBLE_VELERO_S3_BUCKET=
|
NOBLE_VELERO_S3_BUCKET=
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ noble_skip_k8s_health_check: false
|
|||||||
|
|
||||||
# Pangolin / Newt — set true only after newt-pangolin-auth Secret exists (SOPS: clusters/noble/secrets/ or imperative — see clusters/noble/bootstrap/newt/README.md)
|
# Pangolin / Newt — set true only after newt-pangolin-auth Secret exists (SOPS: clusters/noble/secrets/ or imperative — see clusters/noble/bootstrap/newt/README.md)
|
||||||
noble_newt_install: true
|
noble_newt_install: true
|
||||||
|
noble_pangolin_sync_http_resources: true
|
||||||
|
|
||||||
# cert-manager needs Secret cloudflare-dns-api-token in cert-manager namespace before ClusterIssuers work
|
# cert-manager needs Secret cloudflare-dns-api-token in cert-manager namespace before ClusterIssuers work
|
||||||
noble_cert_manager_require_cloudflare_secret: true
|
noble_cert_manager_require_cloudflare_secret: true
|
||||||
|
|||||||
@@ -11,8 +11,7 @@ spec:
|
|||||||
privateKeySecretRef:
|
privateKeySecretRef:
|
||||||
name: letsencrypt-prod-account-key
|
name: letsencrypt-prod-account-key
|
||||||
solvers:
|
solvers:
|
||||||
# DNS-01 — works when public HTTP to Traefik is wrong (e.g. hostname proxied through Cloudflare
|
# DNS-01 — Cloudflare token covers pcenicni.dev only. Requires Secret cloudflare-dns-api-token in cert-manager.
|
||||||
# returns 404 for /.well-known/acme-challenge). Requires Secret cloudflare-dns-api-token in cert-manager.
|
|
||||||
- dns01:
|
- dns01:
|
||||||
cloudflare:
|
cloudflare:
|
||||||
apiTokenSecretRef:
|
apiTokenSecretRef:
|
||||||
@@ -21,3 +20,8 @@ spec:
|
|||||||
selector:
|
selector:
|
||||||
dnsZones:
|
dnsZones:
|
||||||
- pcenicni.dev
|
- pcenicni.dev
|
||||||
|
# HTTP-01 fallback — used for all other zones (e.g. nikflix.ca via Pangolin → Newt → Traefik).
|
||||||
|
# Requires a Pangolin HTTP resource + target for each hostname before LE can reach /.well-known/acme-challenge/.
|
||||||
|
- http01:
|
||||||
|
ingress:
|
||||||
|
ingressClassName: traefik
|
||||||
|
|||||||
@@ -19,3 +19,6 @@ spec:
|
|||||||
selector:
|
selector:
|
||||||
dnsZones:
|
dnsZones:
|
||||||
- pcenicni.dev
|
- pcenicni.dev
|
||||||
|
- http01:
|
||||||
|
ingress:
|
||||||
|
ingressClassName: traefik
|
||||||
|
|||||||
@@ -98,8 +98,41 @@ You still **link domains** in Pangolin and create **CNAME** records at your DNS
|
|||||||
|
|
||||||
Steps:
|
Steps:
|
||||||
|
|
||||||
1. In Pangolin, create an **organization API key** with permission to manage domains, resources, and targets ([Integration API](https://docs.pangolin.net/manage/integration-api)).
|
0. **Enable the Integration API** in Pangolin’s `config.yml` on the Pangolin host — it is **off by default**. Add to `config.yml`:
|
||||||
2. Add to repository **`.env`** (never commit secrets): **`NOBLE_PANGOLIN_API_BASE`**, **`NOBLE_PANGOLIN_ORG_ID`**, **`NOBLE_PANGOLIN_API_TOKEN`**, **`NOBLE_PANGOLIN_SITE_ID`** (numeric site that owns your **Newt** pair). Optionally **`NOBLE_PANGOLIN_TRAEFIK_IP`** / **`NOBLE_PANGOLIN_TRAEFIK_PORT`** — if unset, Ansible uses **`kubectl`** to read the Traefik Service **LoadBalancer** IP.
|
|
||||||
|
```yaml
|
||||||
|
flags:
|
||||||
|
enable_integration_api: true
|
||||||
|
server:
|
||||||
|
integration_port: 3003 # default; omit to keep 3003
|
||||||
|
```
|
||||||
|
|
||||||
|
Then expose it with a **Traefik route** in `config/traefik/dynamic_config.yml`. The Integration API is a *separate* process from the main Pangolin server and needs its own hostname (e.g. `api.pcenicni.dev`):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
routers:
|
||||||
|
int-api-router:
|
||||||
|
rule: "Host(`api.pcenicni.dev`)"
|
||||||
|
service: int-api-service
|
||||||
|
entryPoints: [websecure]
|
||||||
|
tls: { certResolver: letsencrypt }
|
||||||
|
int-api-router-redirect:
|
||||||
|
rule: "Host(`api.pcenicni.dev`)"
|
||||||
|
service: int-api-service
|
||||||
|
entryPoints: [web]
|
||||||
|
middlewares: [redirect-to-https]
|
||||||
|
services:
|
||||||
|
int-api-service:
|
||||||
|
loadBalancer:
|
||||||
|
servers: [{ url: "http://pangolin:3003" }]
|
||||||
|
```
|
||||||
|
|
||||||
|
After restarting Pangolin, verify: `curl https://api.pcenicni.dev/v1/` should return `{"message":"Pangolin Integration API"}`. Also add a **CNAME** for `api.pcenicni.dev` pointing at the same upstream as `pangolin.pcenicni.dev`.
|
||||||
|
|
||||||
|
> **Common mistake:** `https://pangolin.pcenicni.dev/api/v1` is the session-based **external API** — it will always return **401** to Bearer tokens. The Integration API must have its own Traefik-exposed hostname.
|
||||||
|
|
||||||
|
1. In Pangolin, create an **organization API key** ([Integration API docs](https://docs.pangolin.net/self-host/advanced/integration-api)) with permission to manage domains, resources, and targets. The API expects **`Authorization: Bearer {apiKeyId}.{apiKeySecret}`** — paste **`id.secret`** as a single string into **`NOBLE_PANGOLIN_API_TOKEN`**, or set **`NOBLE_PANGOLIN_API_KEY_ID`** + only the **secret** in **`NOBLE_PANGOLIN_API_TOKEN`**. For **`NOBLE_PANGOLIN_SITE_ID`** as a **niceId** slug, enable **Site → List Sites** (the script falls back to listing sites if **Get Site** returns **404**).
|
||||||
|
2. Add to repository **`.env`** (never commit secrets): **`NOBLE_PANGOLIN_API_BASE`** is the Integration API hostname with `/v1` suffix — e.g. **`https://api.pcenicni.dev/v1`** (not `https://pangolin.pcenicni.dev/api/v1`). Also set **`NOBLE_PANGOLIN_ORG_ID`**, **`NOBLE_PANGOLIN_API_TOKEN`** (optionally **`NOBLE_PANGOLIN_API_KEY_ID`**), **`NOBLE_PANGOLIN_SITE_ID`** (numeric **siteId** *or* **niceId**). Optionally **`NOBLE_PANGOLIN_TRAEFIK_IP`** / **`NOBLE_PANGOLIN_TRAEFIK_PORT`** — if unset, Ansible uses **`kubectl`** to read the Traefik Service **LoadBalancer** IP. TLS: **`NOBLE_PANGOLIN_CA_BUNDLE`** or **`NOBLE_PANGOLIN_INSECURE_SKIP_TLS_VERIFY=true`** for self-signed APIs
|
||||||
3. Set **`noble_pangolin_sync_http_resources: true`** in **`ansible/inventory/group_vars/all.yml`** (or pass **`-e noble_pangolin_sync_http_resources=true`**).
|
3. Set **`noble_pangolin_sync_http_resources: true`** in **`ansible/inventory/group_vars/all.yml`** (or pass **`-e noble_pangolin_sync_http_resources=true`**).
|
||||||
4. Run **`ansible-playbook playbooks/noble.yml --tags newt`** (or a full **`noble.yml`**) with **`KUBECONFIG`** pointed at the cluster.
|
4. Run **`ansible-playbook playbooks/noble.yml --tags newt`** (or a full **`noble.yml`**) with **`KUBECONFIG`** pointed at the cluster.
|
||||||
|
|
||||||
@@ -108,6 +141,8 @@ Implementation: **`clusters/noble/bootstrap/newt/scripts/sync_pangolin_http_reso
|
|||||||
|
|
||||||
The script matches each FQDN to the **longest** linked **`baseDomain`** in Pangolin, creates the HTTP resource if missing, then adds a **target** (**`siteId`** + Traefik **`ip`:`port`**, **`method`:** **`http`**) if none matches. Pangolin’s API is still evolving — if a call fails, compare with [Swagger](https://api.pangolin.net/v1/docs) for your deployment version.
|
The script matches each FQDN to the **longest** linked **`baseDomain`** in Pangolin, creates the HTTP resource if missing, then adds a **target** (**`siteId`** + Traefik **`ip`:`port`**, **`method`:** **`http`**) if none matches. Pangolin’s API is still evolving — if a call fails, compare with [Swagger](https://api.pangolin.net/v1/docs) for your deployment version.
|
||||||
|
|
||||||
|
**`.env` vs shell:** If **`NOBLE_PANGOLIN_API_TOKEN`** (or other **`NOBLE_PANGOLIN_*`**) is set in your shell to an empty or old value, older script versions could ignore **`.env`**. Current script overwrites **`os.environ`** from **`.env`** when **`--env-file`** is passed — unset stray exports if you still see **401**.
|
||||||
|
|
||||||
### Authentik on a public name
|
### Authentik on a public name
|
||||||
|
|
||||||
Use **`noble_authentik_ingress_extra_hosts`** (see **`ansible/roles/noble_authentik/README.md`**) so the Authentik Ingress (and **cert-manager** SANs) include your public FQDN, then create the Pangolin **HTTP** resource + **target** to the same Traefik **:443** endpoint as other apps. One Newt site can carry many hostnames.
|
Use **`noble_authentik_ingress_extra_hosts`** (see **`ansible/roles/noble_authentik/README.md`**) so the Authentik Ingress (and **cert-manager** SANs) include your public FQDN, then create the Pangolin **HTTP** resource + **target** to the same Traefik **:443** endpoint as other apps. One Newt site can carry many hostnames.
|
||||||
|
|||||||
Binary file not shown.
@@ -6,8 +6,13 @@ Docs: https://docs.pangolin.net/manage/integration-api
|
|||||||
Walkthrough: https://docs.pangolin.net/manage/common-api-routes
|
Walkthrough: https://docs.pangolin.net/manage/common-api-routes
|
||||||
|
|
||||||
Environment (or --env-file) can set:
|
Environment (or --env-file) can set:
|
||||||
NOBLE_PANGOLIN_API_BASE, NOBLE_PANGOLIN_ORG_ID, NOBLE_PANGOLIN_API_TOKEN,
|
NOBLE_PANGOLIN_API_BASE, NOBLE_PANGOLIN_ORG_ID,
|
||||||
NOBLE_PANGOLIN_SITE_ID, NOBLE_PANGOLIN_TRAEFIK_IP, NOBLE_PANGOLIN_TRAEFIK_PORT (default 443)
|
NOBLE_PANGOLIN_API_TOKEN — must be **apiKeyId.apiKeySecret** (one string, dot in the middle), OR
|
||||||
|
set **NOBLE_PANGOLIN_API_KEY_ID** and put only the **secret** in **NOBLE_PANGOLIN_API_TOKEN**,
|
||||||
|
NOBLE_PANGOLIN_SITE_ID (numeric siteId **or** Pangolin site **niceId**, e.g. unruly-asian-badger),
|
||||||
|
NOBLE_PANGOLIN_TRAEFIK_IP, NOBLE_PANGOLIN_TRAEFIK_PORT (default 443)
|
||||||
|
Optional TLS: NOBLE_PANGOLIN_CA_BUNDLE (path to PEM) or NOBLE_PANGOLIN_INSECURE_SKIP_TLS_VERIFY=true
|
||||||
|
(homelab self-signed Integration API only — insecure).
|
||||||
|
|
||||||
CLI overrides env. FQDNs: --fqdns a.example.com,b.example.com (required).
|
CLI overrides env. FQDNs: --fqdns a.example.com,b.example.com (required).
|
||||||
"""
|
"""
|
||||||
@@ -16,8 +21,10 @@ from __future__ import annotations
|
|||||||
import argparse
|
import argparse
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
|
import ssl
|
||||||
import sys
|
import sys
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
import urllib.request
|
import urllib.request
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -39,16 +46,61 @@ def load_dotenv(path: str) -> dict[str, str]:
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def api_request(
|
def env_truthy(raw: str | None) -> bool:
|
||||||
|
if raw is None:
|
||||||
|
return False
|
||||||
|
return raw.strip().lower() in ("1", "true", "yes", "on")
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_api_token(raw: str) -> str:
|
||||||
|
"""Strip whitespace and a single accidental ``Bearer `` prefix (`.env` / copy-paste)."""
|
||||||
|
t = str(raw).strip()
|
||||||
|
if t.lower().startswith("bearer "):
|
||||||
|
return t[7:].strip()
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
def pangolin_bearer_credential(token: str, key_id: str) -> str:
|
||||||
|
"""
|
||||||
|
Pangolin's Integration API expects ``Authorization: Bearer {apiKeyId}.{apiKeySecret}``
|
||||||
|
(see Pangolin ``verifyApiKey`` — the part after ``Bearer`` is split on the first ``.``).
|
||||||
|
"""
|
||||||
|
t = normalize_api_token(token)
|
||||||
|
kid = (key_id or "").strip()
|
||||||
|
if "." in t:
|
||||||
|
return t
|
||||||
|
if kid:
|
||||||
|
return f"{kid}.{t}"
|
||||||
|
return t
|
||||||
|
|
||||||
|
|
||||||
|
def tls_ssl_context(ca_bundle: str, insecure: bool) -> ssl.SSLContext | None:
|
||||||
|
"""Return custom SSL context, or None for default certificate verification."""
|
||||||
|
path = (ca_bundle or "").strip()
|
||||||
|
if path:
|
||||||
|
if not os.path.isfile(path):
|
||||||
|
raise SystemExit(f"NOBLE_PANGOLIN_CA_BUNDLE is not a readable file: {path!r}")
|
||||||
|
return ssl.create_default_context(cafile=path)
|
||||||
|
if insecure:
|
||||||
|
ctx = ssl.create_default_context()
|
||||||
|
ctx.check_hostname = False
|
||||||
|
ctx.verify_mode = ssl.CERT_NONE
|
||||||
|
return ctx
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def http_exchange(
|
||||||
method: str,
|
method: str,
|
||||||
url: str,
|
url: str,
|
||||||
token: str,
|
token: str,
|
||||||
body: dict[str, Any] | None = None,
|
body: dict[str, Any] | None = None,
|
||||||
) -> dict[str, Any]:
|
ssl_context: ssl.SSLContext | None = None,
|
||||||
data = None if body is None else json.dumps(body).encode("utf-8")
|
) -> tuple[int, Any]:
|
||||||
|
"""HTTP request; returns (status_code, parsed JSON body or dict with __raw on parse error)."""
|
||||||
|
data_bytes = None if body is None else json.dumps(body).encode("utf-8")
|
||||||
req = urllib.request.Request(
|
req = urllib.request.Request(
|
||||||
url,
|
url,
|
||||||
data=data,
|
data=data_bytes,
|
||||||
method=method,
|
method=method,
|
||||||
headers={
|
headers={
|
||||||
"Authorization": f"Bearer {token}",
|
"Authorization": f"Bearer {token}",
|
||||||
@@ -57,14 +109,60 @@ def api_request(
|
|||||||
},
|
},
|
||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
with urllib.request.urlopen(req, timeout=120) as resp:
|
with urllib.request.urlopen(req, timeout=120, context=ssl_context) as resp:
|
||||||
payload = resp.read().decode("utf-8")
|
payload = resp.read().decode("utf-8")
|
||||||
|
code = int(getattr(resp, "status", None) or resp.getcode() or 200)
|
||||||
except urllib.error.HTTPError as e:
|
except urllib.error.HTTPError as e:
|
||||||
detail = e.read().decode("utf-8", errors="replace")
|
code = int(e.code)
|
||||||
raise SystemExit(f"HTTP {e.code} {method} {url}\n{detail}") from e
|
raw = e.read().decode("utf-8", errors="replace")
|
||||||
if not payload:
|
try:
|
||||||
return {}
|
return code, (json.loads(raw) if raw.strip() else {})
|
||||||
return json.loads(payload)
|
except json.JSONDecodeError:
|
||||||
|
return code, {"__raw": raw[:2000]}
|
||||||
|
except urllib.error.URLError as e:
|
||||||
|
reason = str(e.reason) if getattr(e, "reason", None) is not None else str(e)
|
||||||
|
if "CERTIFICATE_VERIFY_FAILED" in reason or "certificate verify failed" in reason.lower():
|
||||||
|
raise SystemExit(
|
||||||
|
f"{e}\n"
|
||||||
|
"TLS verification failed. For a self-signed Integration API, set in .env:\n"
|
||||||
|
" NOBLE_PANGOLIN_INSECURE_SKIP_TLS_VERIFY=true\n"
|
||||||
|
"or set NOBLE_PANGOLIN_CA_BUNDLE to a PEM file that trusts that API (preferred)."
|
||||||
|
) from e
|
||||||
|
raise SystemExit(f"Request failed: {e}") from e
|
||||||
|
|
||||||
|
if not payload.strip():
|
||||||
|
return code, {}
|
||||||
|
try:
|
||||||
|
return code, json.loads(payload)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
return code, {"__raw": payload[:2000]}
|
||||||
|
|
||||||
|
|
||||||
|
def api_request(
|
||||||
|
method: str,
|
||||||
|
url: str,
|
||||||
|
token: str,
|
||||||
|
body: dict[str, Any] | None = None,
|
||||||
|
ssl_context: ssl.SSLContext | None = None,
|
||||||
|
) -> Any:
|
||||||
|
code, parsed = http_exchange(method, url, token, body, ssl_context)
|
||||||
|
if code >= 400:
|
||||||
|
detail = json.dumps(parsed) if isinstance(parsed, (dict, list)) else str(parsed)
|
||||||
|
hint = ""
|
||||||
|
if code in (401, 403):
|
||||||
|
hint = (
|
||||||
|
"\n\nPangolin expects **Authorization: Bearer {apiKeyId}.{apiKeySecret}** (id, dot, secret "
|
||||||
|
"from **Organization → API keys** when the key is created). Put **`id.secret`** in "
|
||||||
|
"**`NOBLE_PANGOLIN_API_TOKEN`**, or set **`NOBLE_PANGOLIN_API_KEY_ID`** and only the secret in "
|
||||||
|
"**`NOBLE_PANGOLIN_API_TOKEN`**. A browser tab uses **session cookies**, not this header.\n\n"
|
||||||
|
"Also check **NOBLE_PANGOLIN_API_BASE**: this must be the **Integration API** hostname "
|
||||||
|
"(e.g. `https://api.example.com/v1`), NOT the main Pangolin UI host "
|
||||||
|
"(`https://pangolin.example.com/api/v1` is the session-based external API and always "
|
||||||
|
"returns 401 to Bearer tokens). The Integration API runs on a **separate port** (default 3003) "
|
||||||
|
"and needs its own Traefik-exposed hostname. See `flags.enable_integration_api` in Pangolin `config.yml`."
|
||||||
|
)
|
||||||
|
raise SystemExit(f"HTTP {code} {method} {url}\n{detail}{hint}")
|
||||||
|
return parsed
|
||||||
|
|
||||||
|
|
||||||
def unwrap(resp: dict[str, Any]) -> Any:
|
def unwrap(resp: dict[str, Any]) -> Any:
|
||||||
@@ -108,13 +206,18 @@ def resolve_domain(
|
|||||||
return best[0], best[1]
|
return best[0], best[1]
|
||||||
|
|
||||||
|
|
||||||
def list_all_resources(api_base: str, org_id: str, token: str) -> list[dict[str, Any]]:
|
def list_all_resources(
|
||||||
|
api_base: str,
|
||||||
|
org_id: str,
|
||||||
|
token: str,
|
||||||
|
ssl_context: ssl.SSLContext | None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
out: list[dict[str, Any]] = []
|
out: list[dict[str, Any]] = []
|
||||||
page = 1
|
page = 1
|
||||||
page_size = 100
|
page_size = 100
|
||||||
while True:
|
while True:
|
||||||
url = f"{api_base.rstrip('/')}/org/{org_id}/resources?page={page}&pageSize={page_size}"
|
url = f"{api_base.rstrip('/')}/org/{org_id}/resources?page={page}&pageSize={page_size}"
|
||||||
data = unwrap(api_request("GET", url, token))
|
data = unwrap(api_request("GET", url, token, ssl_context=ssl_context))
|
||||||
if isinstance(data, list):
|
if isinstance(data, list):
|
||||||
out.extend(data)
|
out.extend(data)
|
||||||
break
|
break
|
||||||
@@ -136,6 +239,39 @@ def list_all_resources(api_base: str, org_id: str, token: str) -> list[dict[str,
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def list_all_sites(
|
||||||
|
api_base: str,
|
||||||
|
org_id: str,
|
||||||
|
token: str,
|
||||||
|
ssl_context: ssl.SSLContext | None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
out: list[dict[str, Any]] = []
|
||||||
|
page = 1
|
||||||
|
page_size = 100
|
||||||
|
while True:
|
||||||
|
url = f"{api_base.rstrip('/')}/org/{org_id}/sites?page={page}&pageSize={page_size}"
|
||||||
|
data = unwrap(api_request("GET", url, token, ssl_context=ssl_context))
|
||||||
|
if isinstance(data, list):
|
||||||
|
out.extend(data)
|
||||||
|
break
|
||||||
|
if not isinstance(data, dict):
|
||||||
|
break
|
||||||
|
batch = data.get("sites") or data.get("items") or []
|
||||||
|
if not isinstance(batch, list):
|
||||||
|
break
|
||||||
|
out.extend(batch)
|
||||||
|
pag = data.get("pagination") or {}
|
||||||
|
total = pag.get("total")
|
||||||
|
if isinstance(total, int) and len(out) >= total:
|
||||||
|
break
|
||||||
|
if len(batch) < page_size:
|
||||||
|
break
|
||||||
|
page += 1
|
||||||
|
if page > 500:
|
||||||
|
raise SystemExit("Pagination safety stop (sites >500 pages)")
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
def resource_public_fqdn(res: dict[str, Any]) -> str | None:
|
def resource_public_fqdn(res: dict[str, Any]) -> str | None:
|
||||||
fd = res.get("fullDomain")
|
fd = res.get("fullDomain")
|
||||||
if isinstance(fd, str) and fd.strip():
|
if isinstance(fd, str) and fd.strip():
|
||||||
@@ -163,9 +299,14 @@ def find_resource_for_fqdn(resources: list[dict[str, Any]], fqdn: str) -> dict[s
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def list_targets(api_base: str, resource_id: int, token: str) -> list[dict[str, Any]]:
|
def list_targets(
|
||||||
|
api_base: str,
|
||||||
|
resource_id: int,
|
||||||
|
token: str,
|
||||||
|
ssl_context: ssl.SSLContext | None,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
url = f"{api_base.rstrip('/')}/resource/{resource_id}/targets"
|
url = f"{api_base.rstrip('/')}/resource/{resource_id}/targets"
|
||||||
data = unwrap(api_request("GET", url, token))
|
data = unwrap(api_request("GET", url, token, ssl_context=ssl_context))
|
||||||
if isinstance(data, list):
|
if isinstance(data, list):
|
||||||
return data
|
return data
|
||||||
if isinstance(data, dict):
|
if isinstance(data, dict):
|
||||||
@@ -181,6 +322,63 @@ def target_matches(t: dict[str, Any], site_id: int, ip: str, port: int) -> bool:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_site_id(
|
||||||
|
api_base: str,
|
||||||
|
org_id: str,
|
||||||
|
token: str,
|
||||||
|
site_ref: str,
|
||||||
|
ssl_context: ssl.SSLContext | None,
|
||||||
|
) -> int:
|
||||||
|
"""Pangolin targets need numeric siteId. Accept digits or site **niceId** (UI slug)."""
|
||||||
|
ref = site_ref.strip()
|
||||||
|
if not ref:
|
||||||
|
raise SystemExit("NOBLE_PANGOLIN_SITE_ID is empty")
|
||||||
|
if ref.isdigit():
|
||||||
|
return int(ref)
|
||||||
|
slug = urllib.parse.quote(ref, safe="")
|
||||||
|
url = f"{api_base.rstrip('/')}/org/{org_id}/site/{slug}"
|
||||||
|
code, envelope = http_exchange("GET", url, token, None, ssl_context)
|
||||||
|
|
||||||
|
sid: int | None = None
|
||||||
|
if code == 200 and isinstance(envelope, dict):
|
||||||
|
if not envelope.get("error") and envelope.get("success") is not False:
|
||||||
|
data = envelope.get("data")
|
||||||
|
if isinstance(data, dict) and data.get("siteId") is not None:
|
||||||
|
sid = int(data["siteId"])
|
||||||
|
|
||||||
|
if sid is not None:
|
||||||
|
return sid
|
||||||
|
|
||||||
|
sites = list_all_sites(api_base, org_id, token, ssl_context)
|
||||||
|
ref_l = ref.lower()
|
||||||
|
for s in sites:
|
||||||
|
nid = s.get("niceId") or s.get("nice_id")
|
||||||
|
if isinstance(nid, str) and nid.lower() == ref_l:
|
||||||
|
out = s.get("siteId") if s.get("siteId") is not None else s.get("id")
|
||||||
|
if out is not None:
|
||||||
|
print(f"[site] resolved niceId {ref!r} via List Sites -> siteId={int(out)}")
|
||||||
|
return int(out)
|
||||||
|
|
||||||
|
if code in (401, 403):
|
||||||
|
msg = (
|
||||||
|
f"HTTP {code} on GET {url} and no site with niceId {ref!r} in List Sites. "
|
||||||
|
"Grant **Site → Get Site** and **Site → List Sites** on the organization API key, "
|
||||||
|
"or set **NOBLE_PANGOLIN_SITE_ID** to the numeric **siteId** from Pangolin **Sites**. "
|
||||||
|
"Remember: **Bearer** must be **apiKeyId.apiKeySecret**; a browser tab uses **session cookies** instead."
|
||||||
|
)
|
||||||
|
if isinstance(envelope, dict) and envelope.get("message"):
|
||||||
|
msg += f" Server message: {envelope.get('message')!r}."
|
||||||
|
raise SystemExit(msg)
|
||||||
|
|
||||||
|
detail = json.dumps(envelope) if isinstance(envelope, (dict, list)) else str(envelope)
|
||||||
|
raise SystemExit(
|
||||||
|
f"No site matched {ref!r}. GET {url} returned HTTP {code}.\n{detail}\n"
|
||||||
|
"Check **NOBLE_PANGOLIN_API_BASE** (self-hosted: often https://<pangolin-host>/api/v1 — match "
|
||||||
|
"Swagger **servers**), **NOBLE_PANGOLIN_ORG_ID**, and API key permissions. "
|
||||||
|
"Or set **NOBLE_PANGOLIN_SITE_ID** to the numeric **siteId**."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def main() -> None:
|
def main() -> None:
|
||||||
ap = argparse.ArgumentParser(description=__doc__)
|
ap = argparse.ArgumentParser(description=__doc__)
|
||||||
ap.add_argument("--env-file", help="Parse KEY=value lines (optional overrides for env)")
|
ap.add_argument("--env-file", help="Parse KEY=value lines (optional overrides for env)")
|
||||||
@@ -204,17 +402,49 @@ def main() -> None:
|
|||||||
action="store_true",
|
action="store_true",
|
||||||
help="Print actions only; do not call mutating endpoints",
|
help="Print actions only; do not call mutating endpoints",
|
||||||
)
|
)
|
||||||
|
ap.add_argument(
|
||||||
|
"--ca-bundle",
|
||||||
|
default=os.environ.get("NOBLE_PANGOLIN_CA_BUNDLE", ""),
|
||||||
|
help="Path to PEM CA bundle for the Integration API (preferred over --insecure-skip-tls-verify)",
|
||||||
|
)
|
||||||
|
ap.add_argument(
|
||||||
|
"--insecure-skip-tls-verify",
|
||||||
|
action="store_true",
|
||||||
|
help="Disable TLS certificate verification (homelab self-signed API only)",
|
||||||
|
)
|
||||||
args = ap.parse_args()
|
args = ap.parse_args()
|
||||||
|
|
||||||
if args.env_file:
|
if args.env_file:
|
||||||
for k, v in load_dotenv(args.env_file).items():
|
for k, v in load_dotenv(args.env_file).items():
|
||||||
if k.startswith("NOBLE_PANGOLIN_"):
|
if k.startswith("NOBLE_PANGOLIN_"):
|
||||||
os.environ.setdefault(k, v)
|
# Assign (not setdefault): a stale or empty **NOBLE_*** in the parent environment
|
||||||
args.api_base = args.api_base or os.environ.get("NOBLE_PANGOLIN_API_BASE", "")
|
# must not hide values from **.env** when Ansible runs this script.
|
||||||
args.org_id = args.org_id or os.environ.get("NOBLE_PANGOLIN_ORG_ID", "")
|
os.environ[k] = v
|
||||||
args.token = args.token or os.environ.get("NOBLE_PANGOLIN_API_TOKEN", "")
|
args.api_base = os.environ.get("NOBLE_PANGOLIN_API_BASE", args.api_base or "")
|
||||||
args.site_id = args.site_id or os.environ.get("NOBLE_PANGOLIN_SITE_ID", "")
|
args.org_id = os.environ.get("NOBLE_PANGOLIN_ORG_ID", args.org_id or "")
|
||||||
args.traefik_ip = args.traefik_ip or os.environ.get("NOBLE_PANGOLIN_TRAEFIK_IP", "")
|
args.token = os.environ.get("NOBLE_PANGOLIN_API_TOKEN", args.token or "")
|
||||||
|
args.site_id = os.environ.get("NOBLE_PANGOLIN_SITE_ID", args.site_id or "")
|
||||||
|
args.traefik_ip = os.environ.get("NOBLE_PANGOLIN_TRAEFIK_IP", args.traefik_ip or "")
|
||||||
|
args.ca_bundle = os.environ.get("NOBLE_PANGOLIN_CA_BUNDLE", args.ca_bundle or "")
|
||||||
|
|
||||||
|
if not str(args.ca_bundle or "").strip():
|
||||||
|
args.ca_bundle = os.environ.get("NOBLE_PANGOLIN_CA_BUNDLE", "")
|
||||||
|
if not args.insecure_skip_tls_verify:
|
||||||
|
args.insecure_skip_tls_verify = env_truthy(
|
||||||
|
os.environ.get("NOBLE_PANGOLIN_INSECURE_SKIP_TLS_VERIFY", "")
|
||||||
|
)
|
||||||
|
|
||||||
|
api_key_id = os.environ.get("NOBLE_PANGOLIN_API_KEY_ID", "").strip()
|
||||||
|
args.token = pangolin_bearer_credential(str(args.token or ""), api_key_id)
|
||||||
|
# Print the key ID (never the secret) so mismatches are easy to spot.
|
||||||
|
used_key_id = args.token.split(".")[0] if "." in args.token else "(none)"
|
||||||
|
print(f"[auth] using apiKeyId={used_key_id!r} — verify this exists in Pangolin → Organization → API keys")
|
||||||
|
if "." not in args.token:
|
||||||
|
print(
|
||||||
|
"[auth] WARNING: NOBLE_PANGOLIN_API_TOKEN has no '.' — Pangolin requires **apiKeyId.apiKeySecret**. "
|
||||||
|
"Set **NOBLE_PANGOLIN_API_KEY_ID** + secret, or paste **id.secret** as a single value in **NOBLE_PANGOLIN_API_TOKEN**.",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
|
||||||
missing = [
|
missing = [
|
||||||
n
|
n
|
||||||
@@ -237,12 +467,18 @@ def main() -> None:
|
|||||||
api_base = str(args.api_base).rstrip("/")
|
api_base = str(args.api_base).rstrip("/")
|
||||||
org_id = str(args.org_id).strip()
|
org_id = str(args.org_id).strip()
|
||||||
token = str(args.token).strip()
|
token = str(args.token).strip()
|
||||||
site_id = int(str(args.site_id).strip())
|
ssl_ctx = tls_ssl_context(str(args.ca_bundle).strip(), bool(args.insecure_skip_tls_verify))
|
||||||
|
if str(args.ca_bundle).strip():
|
||||||
|
print(f"[tls] using CA bundle {args.ca_bundle!r}")
|
||||||
|
elif args.insecure_skip_tls_verify:
|
||||||
|
print("[tls] WARNING: certificate verification disabled (NOBLE_PANGOLIN_INSECURE_SKIP_TLS_VERIFY)")
|
||||||
|
site_id = resolve_site_id(api_base, org_id, token, str(args.site_id).strip(), ssl_ctx)
|
||||||
|
print(f"[site] siteId={site_id} (NOBLE_PANGOLIN_SITE_ID={args.site_id!r})")
|
||||||
traefik_ip = str(args.traefik_ip).strip()
|
traefik_ip = str(args.traefik_ip).strip()
|
||||||
traefik_port = int(args.traefik_port)
|
traefik_port = int(args.traefik_port)
|
||||||
|
|
||||||
dom_url = f"{api_base}/org/{org_id}/domains"
|
dom_url = f"{api_base}/org/{org_id}/domains"
|
||||||
domains_raw = unwrap(api_request("GET", dom_url, token))
|
domains_raw = unwrap(api_request("GET", dom_url, token, ssl_context=ssl_ctx))
|
||||||
domains: list[dict[str, Any]] = []
|
domains: list[dict[str, Any]] = []
|
||||||
if isinstance(domains_raw, list):
|
if isinstance(domains_raw, list):
|
||||||
domains = domains_raw
|
domains = domains_raw
|
||||||
@@ -251,7 +487,7 @@ def main() -> None:
|
|||||||
if not isinstance(domains, list):
|
if not isinstance(domains, list):
|
||||||
raise SystemExit(f"Unexpected domains response: {domains_raw!r}")
|
raise SystemExit(f"Unexpected domains response: {domains_raw!r}")
|
||||||
|
|
||||||
resources = list_all_resources(api_base, org_id, token)
|
resources = list_all_resources(api_base, org_id, token, ssl_ctx)
|
||||||
|
|
||||||
for fqdn in fqdns:
|
for fqdn in fqdns:
|
||||||
domain_id, subdomain = resolve_domain(fqdn, domains)
|
domain_id, subdomain = resolve_domain(fqdn, domains)
|
||||||
@@ -273,6 +509,7 @@ def main() -> None:
|
|||||||
f"{api_base}/org/{org_id}/resource",
|
f"{api_base}/org/{org_id}/resource",
|
||||||
token,
|
token,
|
||||||
body,
|
body,
|
||||||
|
ssl_context=ssl_ctx,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
rid = int(str(created.get("resourceId", "")).strip() or 0)
|
rid = int(str(created.get("resourceId", "")).strip() or 0)
|
||||||
@@ -300,7 +537,7 @@ def main() -> None:
|
|||||||
if not rid:
|
if not rid:
|
||||||
raise SystemExit(f"Resource missing resourceId: {res!r}")
|
raise SystemExit(f"Resource missing resourceId: {res!r}")
|
||||||
|
|
||||||
targets = list_targets(api_base, rid, token)
|
targets = list_targets(api_base, rid, token, ssl_ctx)
|
||||||
if any(target_matches(t, site_id, traefik_ip, traefik_port) for t in targets):
|
if any(target_matches(t, site_id, traefik_ip, traefik_port) for t in targets):
|
||||||
print(f" -> target OK site={site_id} {traefik_ip}:{traefik_port}")
|
print(f" -> target OK site={site_id} {traefik_ip}:{traefik_port}")
|
||||||
continue
|
continue
|
||||||
@@ -318,6 +555,7 @@ def main() -> None:
|
|||||||
f"{api_base}/resource/{rid}/target",
|
f"{api_base}/resource/{rid}/target",
|
||||||
token,
|
token,
|
||||||
tbody,
|
tbody,
|
||||||
|
ssl_context=ssl_ctx,
|
||||||
)
|
)
|
||||||
print(" -> target created")
|
print(" -> target created")
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user