feat(diagnostics): add health endpoint with Grafana auto-provisioning (#54)

Implements heartbeat telemetry for WaaS dashboard monitoring:
- JSON health endpoint at /?mokowaas=health with token auth
- Database, filesystem, cache, and extension health checks
- Auto-generated API token (separate from Joomla user tokens)
- Grafana Infinity datasource auto-provisioning via REST API
- Shared dashboard with endpoint dropdown variable
- Auto-provision on plugin install/update via script.php
- Grafana plugin install via API (replaces deprecated CLI)
- Deprovisioning on disable (datasource cleanup)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jonathan Miller
2026-05-21 15:42:58 -05:00
parent a0f3e42861
commit d1e2555f00
7 changed files with 1367 additions and 9 deletions
+145 -6
View File
@@ -10,13 +10,13 @@
DEFGROUP: Joomla.Plugin
INGROUP: MokoWaaS.Guides
REPO: https://github.com/mokoconsulting-tech/mokowaas
VERSION: 02.01.08
VERSION: 02.01.22
PATH: /docs/guides/configuration-guide.md
BRIEF: Configuration guide for the MokoWaaS system plugin
NOTE: Defines plugin parameters, expected behaviors, and recommended defaults
-->
# MokoWaaS Configuration Guide (VERSION: 02.01.08)
# MokoWaaS Configuration Guide (VERSION: 02.01.22)
## 1. Objective
@@ -218,7 +218,141 @@ All restrictions apply to non-master users only. The master user always has full
Restricted components are automatically hidden from the admin menu via `onPreprocessMenuItems`.
## 9. Security Hardening (fieldset: `security`)
## 9. Diagnostics & Monitoring (fieldset: `diagnostics`)
This fieldset configures a lightweight JSON health check endpoint designed for external monitoring systems such as Grafana. The endpoint follows patterns similar to Akeeba Panopticon, providing system diagnostics without requiring Joomla authentication.
### 9.1 Enable Health Endpoint
| Property | Value |
| -------- | ----- |
| Field name | `enable_health_endpoint` |
| Type | Radio (Yes/No) |
| Default | No |
Exposes a JSON health check endpoint at `/?mokowaas=health`. When enabled for the first time, a 64-character API token is automatically generated and stored in `health_api_token`. When disabled, the token is cleared.
### 9.2 Health API Token
| Property | Value |
| -------- | ----- |
| Field name | `health_api_token` |
| Type | Text (read-only) |
| Default | (empty — auto-generated on enable) |
The bearer token used to authenticate health check requests. This is a dedicated API key separate from Joomla user API tokens. Generated automatically using `random_bytes(32)` when the health endpoint is enabled.
**Authentication methods (either works):**
* HTTP header: `Authorization: Bearer <token>`
* Query parameter: `?mokowaas=health&token=<token>`
### 9.3 Endpoint Behavior
**URL:** `https://<site>/?mokowaas=health`
The endpoint intercepts requests in `onAfterInitialise()`, before Joomla routing. This minimizes overhead for monitoring probes. HTTPS is always enforced (the `enforceHttps()` handler runs first).
**HTTP responses:**
* `200` — All checks pass or only degraded warnings
* `401` — Invalid or missing API token
* `503` — One or more checks report error status
### 9.4 Health Check Payload
The endpoint returns a JSON payload with the following structure:
```json
{
"status": "ok|degraded|error",
"timestamp": "2026-05-21T12:00:00Z",
"checks": {
"database": {
"status": "ok",
"latency_ms": 5.12,
"driver": "mysqli",
"users": 3
},
"filesystem": {
"status": "ok",
"tmp_writable": true,
"log_writable": true,
"cache_writable": true,
"free_disk_mb": 12345
},
"cache": {
"status": "ok",
"enabled": true,
"handler": "file"
},
"extensions": {
"status": "ok|degraded",
"counts": { "plugin": 25, "component": 12 },
"pending_updates": 0
}
},
"meta": {
"brand": "MokoWaaS",
"plugin_version": "02.01.22",
"joomla_version": "5.1.0",
"php_version": "8.2.0",
"server_name": "My Site",
"server_time": "2026-05-21T12:00:00Z"
}
}
```
**Overall status logic:**
* `ok` — All individual checks report `ok`
* `degraded` — At least one check is `degraded` (e.g., low disk space, pending updates)
* `error` — At least one check is `error` (e.g., database unreachable, unwritable directories)
### 9.5 Grafana URL
| Property | Value |
| -------- | ----- |
| Field name | `grafana_url` |
| Type | URL |
| Default | (empty) |
Base URL of the Grafana instance (e.g., `https://grafana.mokoconsulting.tech`). When provided together with an API key, the plugin auto-provisions an Infinity datasource and a MokoWaaS dashboard in Grafana.
### 9.6 Grafana API Key
| Property | Value |
| -------- | ----- |
| Field name | `grafana_api_key` |
| Type | Text |
| Default | (empty) |
A Grafana service account token or API key with at least **Editor** role. Required for the plugin to create/update datasources and dashboards via Grafana's REST API.
### 9.7 Auto-Provisioning Behavior
When the health endpoint is enabled and both Grafana URL and API key are set, saving the plugin config triggers automatic provisioning:
1. **Datasource creation** — An [Infinity datasource](https://grafana.com/grafana/plugins/yesoreyeram-infinity-datasource/) (`yesoreyeram-infinity-datasource`) is created in Grafana, configured with the site URL and the health API bearer token. The datasource UID is deterministic: `mokowaas-<md5(siteUrl)>`.
2. **Dashboard creation** — A MokoWaaS dashboard is created with 14 panels across three rows:
- **Row 1 (Status Overview):** Health Status (stat), DB Latency (gauge, ms), Free Disk Space (gauge, MB), Pending Updates (stat)
- **Row 2 (System Info):** Joomla Version, PHP Version, Plugin Version, Cache Status
- **Row 3 (Detail):** /tmp Writable, /logs Writable, /cache Writable, DB Driver, User Count, DB Status
3. **Idempotent updates** — Re-saving the plugin config updates the existing datasource and dashboard (PUT with `overwrite: true`). No duplicates are created.
4. **Deprovisioning** — When the health endpoint is disabled, both the datasource and dashboard are deleted from Grafana.
**Prerequisites:** The Grafana Infinity plugin must be installed on the Grafana instance. Install via: `grafana cli plugins install yesoreyeram-infinity-datasource`.
### 9.8 Manual Grafana Setup (Alternative)
If auto-provisioning is not desired (leave Grafana URL and API key empty), you can set up Grafana manually:
1. Enable the health endpoint — a token is generated automatically.
2. In Grafana, add an **Infinity** datasource pointing to the site URL.
3. Set the `Authorization: Bearer <token>` header.
4. Create panels querying `/?mokowaas=health` and extracting values from the JSON paths documented in section 9.4.
## 10. Security Hardening (fieldset: `security`)
| Field | Default | Description |
| ----- | ------- | ----------- |
@@ -231,22 +365,24 @@ Restricted components are automatically hidden from the admin menu via `onPrepro
| `upload_allowed_types` | jpg,jpeg,png,gif,webp,svg,pdf,doc,docx,xls,xlsx | Comma-separated allowed extensions |
| `upload_max_size_mb` | 10 | Maximum upload size in MB |
## 10. Configuration Change Workflow
## 11. Configuration Change Workflow
1. Document the change request.
2. Apply updates in a staging environment.
3. Validate branding, restrictions, and security settings.
4. Promote changes to production following WaaS change controls.
## 11. Troubleshooting
## 12. Troubleshooting
* **Branding not appearing:** Clear Joomla and browser cache. Verify `enable_branding` is Yes.
* **Logo not changing:** Replace files in `/media/plg_system_mokowaas/`, clear cache.
* **Emergency access not working:** Verify `$mokowaas_allowed_ips` is set in `configuration.php` and includes your IP.
* **Tenant can access restricted area:** Verify the user is not using the master username.
* **Password rejected:** Check password policy settings — all rules must pass.
* **Health endpoint returns 401:** Verify the API token matches. Regenerate by disabling then re-enabling the health endpoint.
* **Health endpoint not responding:** Confirm `enable_health_endpoint` is Yes and the URL includes `?mokowaas=health`.
## 12. Validation Checklist
## 13. Validation Checklist
* Brand name appears consistently across all admin screens
* Company name appears in login support links
@@ -260,6 +396,8 @@ Restricted components are automatically hidden from the admin menu via `onPrepro
* Emergency access works with correct IP + DB password + file verification
* Emergency access attempts visible in System > Action Logs
* Existing site language overrides are preserved
* Health endpoint returns 200 with valid token and 401 without
* Grafana datasource successfully polls the health endpoint
## Revision History
@@ -267,3 +405,4 @@ Restricted components are automatically hidden from the admin menu via `onPrepro
| -------- | ---------- | ------------------------------- | ---------------------------------------------- |
| 01.02.00 | 2025-12-11 | Jonathan Miller (@jmiller-moko) | Initial standalone configuration guide created |
| 02.01.08 | 2026-04-07 | Jonathan Miller (@jmiller-moko) | Full rewrite: WaaS access, visual branding, tenant restrictions, security, maintenance, action logs |
| 02.01.22 | 2026-05-21 | Claude Code (@claude-code) | Added Diagnostics & Monitoring fieldset: health endpoint, auto-token generation, Grafana integration |
+31 -2
View File
@@ -10,13 +10,13 @@
DEFGROUP: Joomla.Plugin
INGROUP: MokoWaaS.Guides
REPO: https://github.com/mokoconsulting-tech/mokowaas
VERSION: 02.01.08
VERSION: 02.01.22
PATH: /docs/guides/operations-guide.md
BRIEF: Operational guide for administering and managing the MokoWaaS system plugin
NOTE: Defines lifecycle, responsibilities, and operational behaviors
-->
# MokoWaaS Operations Guide (VERSION: 02.01.08)
# MokoWaaS Operations Guide (VERSION: 02.01.22)
## Introduction
@@ -90,12 +90,40 @@ Operational logging and monitoring should focus on:
* Plugin initialization or load order errors
* Warnings related to language strings or missing constants
* UI rendering anomalies reported by administrators
* Health endpoint status changes (degraded/error transitions)
Recommended monitoring sources:
* Joomla Administrator logs
* Web server and PHP error logs
* Centralized WaaS logging and observability tools where available
* **Grafana** — via the MokoWaaS health endpoint (see below)
### Grafana Health Endpoint Integration
MokoWaaS provides a built-in health check endpoint at `/?mokowaas=health` for integration with Grafana or similar monitoring platforms. This follows the Akeeba Panopticon pattern of exposing system diagnostics via a lightweight, token-authenticated JSON API.
**Automatic setup:**
1. Enable the health endpoint in plugin config — a unique API token is auto-generated.
2. Enter your Grafana URL and API key (service account token with Editor role).
3. Save the plugin — the datasource and a 14-panel dashboard are auto-provisioned in Grafana.
4. Configure alerting rules on `status` transitions to `degraded` or `error`.
**Manual setup (if auto-provisioning is not used):**
1. Enable the health endpoint and copy the generated token.
2. In Grafana, install the Infinity datasource plugin and create a datasource pointing to the site URL.
3. Set the `Authorization: Bearer <token>` header.
4. Build dashboard panels from the structured JSON response.
**Operational considerations:**
* The health endpoint bypasses Joomla routing and sessions — minimal server overhead per poll.
* HTTPS is always enforced on health requests.
* The API token is independent of Joomla user accounts — it does not expire with sessions.
* Token regeneration: disable then re-enable the health endpoint in plugin config.
* If the endpoint returns 503, at least one system check has failed — investigate immediately.
## Maintenance Lifecycle
@@ -129,3 +157,4 @@ Administrators should factor these dependencies into maintenance and upgrade pla
| Date | Author | Description |
| ---------- | ------------------------------- | ---------------------------- |
| 2025-12-11 | Jonathan Miller (@jmiller-moko) | Rewrite for version 01.03.00 |
| 2026-05-21 | Claude Code (@claude-code) | Added Grafana health endpoint monitoring section for 02.01.22 |
File diff suppressed because it is too large Load Diff
@@ -99,6 +99,19 @@ PLG_SYSTEM_MOKOWAAS_DISABLE_INSTALL_URL_DESC="Block installing extensions from U
PLG_SYSTEM_MOKOWAAS_HIDDEN_MENUS_LABEL="Hidden Menu Items"
PLG_SYSTEM_MOKOWAAS_HIDDEN_MENUS_DESC="Components to hide from admin menu for non-master users. One per line (e.g., com_installer)."
; ===== Diagnostics fieldset =====
PLG_SYSTEM_MOKOWAAS_FIELDSET_DIAGNOSTICS_LABEL="Diagnostics & Monitoring"
PLG_SYSTEM_MOKOWAAS_FIELDSET_DIAGNOSTICS_DESC="Health check endpoint for external monitoring systems (e.g. Grafana). Exposes system status via a token-authenticated JSON API."
PLG_SYSTEM_MOKOWAAS_ENABLE_HEALTH_LABEL="Enable Health Endpoint"
PLG_SYSTEM_MOKOWAAS_ENABLE_HEALTH_DESC="Expose a JSON health check endpoint at <code>/?mokowaas=health</code>. Requires a valid API token. A random token is generated automatically when enabled."
PLG_SYSTEM_MOKOWAAS_HEALTH_TOKEN_LABEL="Health API Token"
PLG_SYSTEM_MOKOWAAS_HEALTH_TOKEN_DESC="Auto-generated bearer token for the health endpoint. Use this token in your Grafana datasource configuration. Send as <code>Authorization: Bearer &lt;token&gt;</code> header or <code>&amp;token=&lt;value&gt;</code> query parameter."
PLG_SYSTEM_MOKOWAAS_GRAFANA_URL_LABEL="Grafana URL"
PLG_SYSTEM_MOKOWAAS_GRAFANA_URL_DESC="Base URL of your Grafana instance (e.g. <code>https://grafana.example.com</code>). When provided along with an API key, the plugin will auto-provision a datasource and dashboard in Grafana when the health endpoint is enabled."
PLG_SYSTEM_MOKOWAAS_GRAFANA_KEY_LABEL="Grafana API Key"
PLG_SYSTEM_MOKOWAAS_GRAFANA_KEY_DESC="Service account token or API key with Editor role in Grafana. Required for auto-provisioning the MokoWaaS datasource and dashboard."
; ===== Security fieldset =====
PLG_SYSTEM_MOKOWAAS_FIELDSET_SECURITY_LABEL="Security Hardening"
PLG_SYSTEM_MOKOWAAS_FIELDSET_SECURITY_DESC="HTTPS enforcement, session timeouts, password policy, and upload restrictions."
@@ -99,6 +99,19 @@ PLG_SYSTEM_MOKOWAAS_DISABLE_INSTALL_URL_DESC="Block installing extensions from U
PLG_SYSTEM_MOKOWAAS_HIDDEN_MENUS_LABEL="Hidden Menu Items"
PLG_SYSTEM_MOKOWAAS_HIDDEN_MENUS_DESC="Components to hide from admin menu for non-master users. One per line (e.g., com_installer)."
; ===== Diagnostics fieldset =====
PLG_SYSTEM_MOKOWAAS_FIELDSET_DIAGNOSTICS_LABEL="Diagnostics & Monitoring"
PLG_SYSTEM_MOKOWAAS_FIELDSET_DIAGNOSTICS_DESC="Health check endpoint for external monitoring systems (e.g. Grafana). Exposes system status via a token-authenticated JSON API."
PLG_SYSTEM_MOKOWAAS_ENABLE_HEALTH_LABEL="Enable Health Endpoint"
PLG_SYSTEM_MOKOWAAS_ENABLE_HEALTH_DESC="Expose a JSON health check endpoint at <code>/?mokowaas=health</code>. Requires a valid API token. A random token is generated automatically when enabled."
PLG_SYSTEM_MOKOWAAS_HEALTH_TOKEN_LABEL="Health API Token"
PLG_SYSTEM_MOKOWAAS_HEALTH_TOKEN_DESC="Auto-generated bearer token for the health endpoint. Use this token in your Grafana datasource configuration. Send as <code>Authorization: Bearer &lt;token&gt;</code> header or <code>&amp;token=&lt;value&gt;</code> query parameter."
PLG_SYSTEM_MOKOWAAS_GRAFANA_URL_LABEL="Grafana URL"
PLG_SYSTEM_MOKOWAAS_GRAFANA_URL_DESC="Base URL of your Grafana instance (e.g. <code>https://grafana.example.com</code>). When provided along with an API key, the plugin will auto-provision a datasource and dashboard in Grafana when the health endpoint is enabled."
PLG_SYSTEM_MOKOWAAS_GRAFANA_KEY_LABEL="Grafana API Key"
PLG_SYSTEM_MOKOWAAS_GRAFANA_KEY_DESC="Service account token or API key with Editor role in Grafana. Required for auto-provisioning the MokoWaaS datasource and dashboard."
; ===== Security fieldset =====
PLG_SYSTEM_MOKOWAAS_FIELDSET_SECURITY_LABEL="Security Hardening"
PLG_SYSTEM_MOKOWAAS_FIELDSET_SECURITY_DESC="HTTPS enforcement, session timeouts, password policy, and upload restrictions."
+41 -1
View File
@@ -30,7 +30,7 @@
<license>GNU General Public License version 3 or later; see LICENSE.md</license>
<authorEmail>hello@mokoconsulting.tech</authorEmail>
<authorUrl>https://mokoconsulting.tech</authorUrl>
<version>02.01.21</version>
<version>02.01.22</version>
<description>This plugin rebrands the Joomla system interface with MokoWaaS identity. It applies language overrides and ensures consistent branding across the platform.</description>
<namespace path=".">Moko\Plugin\System\MokoWaaS</namespace>
<scriptfile>script.php</scriptfile>
@@ -268,6 +268,46 @@
description="PLG_SYSTEM_MOKOWAAS_HIDDEN_MENUS_DESC"
rows="5" filter="raw" />
</fieldset>
<fieldset name="diagnostics"
label="PLG_SYSTEM_MOKOWAAS_FIELDSET_DIAGNOSTICS_LABEL"
description="PLG_SYSTEM_MOKOWAAS_FIELDSET_DIAGNOSTICS_DESC"
>
<field
name="enable_health_endpoint"
type="radio"
label="PLG_SYSTEM_MOKOWAAS_ENABLE_HEALTH_LABEL"
description="PLG_SYSTEM_MOKOWAAS_ENABLE_HEALTH_DESC"
default="0"
class="btn-group btn-group-yesno"
>
<option value="1">JYES</option>
<option value="0">JNO</option>
</field>
<field
name="health_api_token"
type="text"
label="PLG_SYSTEM_MOKOWAAS_HEALTH_TOKEN_LABEL"
description="PLG_SYSTEM_MOKOWAAS_HEALTH_TOKEN_DESC"
default=""
filter="raw"
readonly="true"
/>
<field
name="grafana_url"
type="url"
label="PLG_SYSTEM_MOKOWAAS_GRAFANA_URL_LABEL"
description="PLG_SYSTEM_MOKOWAAS_GRAFANA_URL_DESC"
default=""
/>
<field
name="grafana_api_key"
type="text"
label="PLG_SYSTEM_MOKOWAAS_GRAFANA_KEY_LABEL"
description="PLG_SYSTEM_MOKOWAAS_GRAFANA_KEY_DESC"
default=""
filter="raw"
/>
</fieldset>
<fieldset name="security"
label="PLG_SYSTEM_MOKOWAAS_FIELDSET_SECURITY_LABEL"
description="PLG_SYSTEM_MOKOWAAS_FIELDSET_SECURITY_DESC"
+133
View File
@@ -128,6 +128,7 @@ class plgSystemMokoWaaSInstallerScript implements InstallerScriptInterface
$this->updateLoginSupportUrls();
$this->updateAtumBranding();
$this->registerActionLogExtension();
$this->provisionHealthEndpoint();
$this->sendInstallNotification($type);
}
@@ -731,6 +732,138 @@ class plgSystemMokoWaaSInstallerScript implements InstallerScriptInterface
*
* @since 02.01.08
*/
/**
* Provision health endpoint with Grafana if configured.
*
* On install/update, if the health endpoint is enabled and Grafana
* credentials are set, generates a token (if missing) and triggers
* Grafana datasource provisioning via cURL. All data is passed as
* structured JSON — no shell commands are invoked.
*
* @return void
*
* @since 02.01.22
*/
private function provisionHealthEndpoint()
{
$db = Factory::getDbo();
$query = $db->getQuery(true)
->select($db->quoteName('params'))
->from($db->quoteName('#__extensions'))
->where($db->quoteName('element') . ' = '
. $db->quote('mokowaas'))
->where($db->quoteName('folder') . ' = '
. $db->quote('system'));
$db->setQuery($query);
$rawParams = $db->loadResult();
if (empty($rawParams))
{
return;
}
$params = new \Joomla\Registry\Registry($rawParams);
$changed = false;
if (!(int) $params->get('enable_health_endpoint', 0))
{
return;
}
// Auto-generate token if missing
if (empty($params->get('health_api_token', '')))
{
$params->set(
'health_api_token',
bin2hex(random_bytes(32))
);
$changed = true;
}
if ($changed)
{
$db->setQuery(
$db->getQuery(true)
->update($db->quoteName('#__extensions'))
->set($db->quoteName('params') . ' = '
. $db->quote($params->toString()))
->where($db->quoteName('element') . ' = '
. $db->quote('mokowaas'))
->where($db->quoteName('folder') . ' = '
. $db->quote('system'))
);
$db->execute();
}
// Trigger Grafana provisioning if credentials are set
$grafanaUrl = rtrim($params->get('grafana_url', ''), '/');
$grafanaKey = $params->get('grafana_api_key', '');
if (empty($grafanaUrl) || empty($grafanaKey))
{
return;
}
$siteUrl = rtrim(\Joomla\CMS\Uri\Uri::root(), '/');
$siteName = Factory::getConfig()->get('sitename', 'Joomla');
$dsUid = 'mokowaas-' . md5($siteUrl);
$token = $params->get('health_api_token', '');
// Provision datasource via Grafana REST API (cURL)
$dsPayload = json_encode([
'uid' => $dsUid,
'name' => 'MokoWaaS — ' . $siteName,
'type' => 'yesoreyeram-infinity-datasource',
'access' => 'proxy',
'url' => $siteUrl,
'jsonData' => [
'auth_method' => 'bearerToken',
'global_queries' => [],
],
'secureJsonData' => [
'bearerToken' => $token,
],
], JSON_UNESCAPED_SLASHES);
$headers = [
'Authorization: Bearer ' . $grafanaKey,
'Content-Type: application/json',
'Accept: application/json',
];
// Try PUT (update), fall back to POST (create)
$ch = curl_init($grafanaUrl . '/api/datasources/uid/' . $dsUid);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'PUT');
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $dsPayload);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
$response = curl_exec($ch);
$code = (int) curl_getinfo($ch, CURLINFO_HTTP_CODE);
curl_close($ch);
if ($code === 404)
{
$ch = curl_init($grafanaUrl . '/api/datasources');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $dsPayload);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
curl_exec($ch);
curl_close($ch);
}
Log::add(
'Health endpoint provisioned with Grafana on '
. ($code === 200 ? 'update' : 'install'),
Log::INFO,
'mokowaas'
);
}
private function registerActionLogExtension()
{
$db = Factory::getDbo();