Lighthouse Monitor for Watch My Domains SED

A native WMD SED extension that scans domains with Google Lighthouse, stores
scores in custom domain columns, and provides a browser-based report tool with
separate mobile and desktop results.


OVERVIEW

MIGRATION NOTE

The browser report tool now lives at tools/lighthouse.php.
The older tools/lighthouse/lighthouse-report.php file is no longer part of the
active deployment and should be removed from the lighthouse folder.

This extension has two deployment parts.

Runtime tools
These ship as part of the WMD SED application:

In tools/:
- lighthouse.php
  Browser report tool

In tools/lighthouse/:
- lighthouse-gather.php
  CLI data gatherer
- lighthouse-scan.js
  Node.js and Puppeteer scanner
- lighthouse-tools-readme.txt
  This file

Note:
- lighthouse-report.php is no longer used
- the active browser tool is tools/lighthouse.php

Setup script
This is downloaded separately and run once by the administrator:

- lighthouse-setup.sh
  Available from:
  learn.domainpunch.com/static/downloads/lighthouse-setup.sh


STANDARD INSTALLATION LAYOUT

For a standard WMD SED installation created by the WMD SED installer:

/home/wmdsed/
    sites/
        wmdsed60/
            tools/
                lighthouse.php
                lighthouse/
                    lighthouse-scan.js
                    lighthouse-gather.php
                    lighthouse-tools-readme.txt

    webdata/
        node_modules/
        websites.js
        lighthouse-scan.js

    logs/
        wmdsed/
            websites -> /home/wmdsed/webdata
            lh-data/
                example.com-mobile.json
                example.com-desktop.json
                anotherdomain.com-mobile.json
                anotherdomain.com-desktop.json

Notes:
- The websites folder under the log path is a symlink to /home/wmdsed/webdata.
- The setup script resolves this automatically.


REQUIREMENTS

- WMD SED already installed with shell access
- Node.js and Puppeteer already installed
  This is normally done by the WMD SED installer.
- The lighthouse npm package installed by lighthouse-setup.sh


SETUP

LINUX / DEBIAN / UBUNTU

Step 1. Download and run lighthouse-setup.sh

Download:
learn.domainpunch.com/static/downloads/lighthouse-setup.sh

Run as the wmdsed user:

bash lighthouse-setup.sh

For a standard installation, no editing is required. The script defaults to:

- WMD_ROOT=/home/wmdsed/sites/wmdsed60
- NODE_DIR=/home/wmdsed/webdata

For non-standard installations, edit the variables at the top of the script
before running it.

Run with --dry-run first to verify without making changes:

bash lighthouse-setup.sh --dry-run

The script will:

1. Run pre-flight checks on required paths
2. Install lighthouse into the Node folder if needed
3. Copy lighthouse-scan.js from tools/lighthouse/ into the Node folder if needed
4. Create the custom WMD columns by calling --setup-columns
5. Add a cron job for automatic scanning
   This is controlled by INSTALL_CRON at the top of the script


WINDOWS

There is no setup script on Windows. Complete the steps manually.

Step 1. Copy lighthouse-scan.js to the Node folder

copy "C:\wmdsed\tools\lighthouse\lighthouse-scan.js" "C:\wmdsed\logs\websites\lighthouse-scan.js"

Update the paths for your installation. If websites is a symlink, use the real
folder it points to, usually C:\wmdsed\webdata.

Step 2. Install lighthouse

cd "C:\wmdsed\logs\websites"
npm install lighthouse --save


ALL PLATFORMS

Run from the tools/lighthouse/ folder on Windows or with the full path on Linux.

Step 3. Create custom WMD columns

php lighthouse-gather.php --setup-columns

This creates the following ten custom columns:

- lh_perf_mobile      integer   Performance score 0 to 100 (mobile)
- lh_seo_mobile       integer   SEO score 0 to 100 (mobile)
- lh_a11y_mobile      integer   Accessibility score 0 to 100 (mobile)
- lh_bp_mobile        integer   Best Practices score 0 to 100 (mobile)
- lh_perf_desktop     integer   Performance score 0 to 100 (desktop)
- lh_seo_desktop      integer   SEO score 0 to 100 (desktop)
- lh_a11y_desktop     integer   Accessibility score 0 to 100 (desktop)
- lh_bp_desktop       integer   Best Practices score 0 to 100 (desktop)
- lh_status           string    pass / fail / error
- lh_scanned_at       datetime  Example: 2026-03-26 14:32:00

This command is safe to run more than once. Existing columns are skipped.

Step 4. Create the Lighthouse Test Domains category

In WMD SED, create a category named:

Lighthouse Test Domains

Assign the domains you want to scan to that category.

This is separate from WCAG Test Domains. A domain may belong to both.

Step 5. Run a dry run

php lighthouse-gather.php --dry-run

This lists the domains that would be scanned without scanning anything.

Step 6. Run the first scan

php lighthouse-gather.php
php lighthouse-gather.php --batch=2


CONFIGURATION

All constants are defined near the top of lighthouse-gather.php:

const LIGHTHOUSE_NODE_FOLDER       = 'websites';
const LIGHTHOUSE_OUTPUT_FOLDER     = 'lh-data';
const LIGHTHOUSE_DEFAULT_CATEGORY  = 'Lighthouse Test Domains';
const LIGHTHOUSE_DEVICE            = 'both';
const LIGHTHOUSE_FAIL_THRESHOLD    = 50;
const LIGHTHOUSE_FAIL_CATEGORIES   = ['performance', 'accessibility', 'best-practices', 'seo'];
const LIGHTHOUSE_SKIP_RECENT_DAYS  = 30;
const LIGHTHOUSE_TIMEOUT_MS        = 60000;
const LIGHTHOUSE_LOCK_FILE         = 'lighthouse-gather.lock';
const LIGHTHOUSE_PROTOCOL_COLUMN   = '';

Important:
LIGHTHOUSE_OUTPUT_FOLDER must match the constant of the same name in
lighthouse.php.


DEVICE MODE

LIGHTHOUSE_DEVICE controls which device modes are scanned:

- both
  Scan mobile and desktop for every domain and store separate scores
- mobile
  Scan mobile only
- desktop
  Scan desktop only

Default:
both

The --device CLI flag can override this for a single run.

Mobile mode uses Lighthouse default throttling.
Desktop mode disables throttling and uses a desktop viewport.


PASS / FAIL THRESHOLD

A domain is marked fail if any score in LIGHTHOUSE_FAIL_CATEGORIES falls below
LIGHTHOUSE_FAIL_THRESHOLD for any scanned device mode.

By default, all four categories count toward pass/fail:

- performance
- accessibility
- best-practices
- seo


PROTOCOL RESOLUTION

If LIGHTHOUSE_PROTOCOL_COLUMN is set to a custom column name such as
site_protocol, the scanner reads the protocol from that column.

If it is empty, the scanner tries https first and falls back to http.


LOADER RESOLUTION

Both PHP files use findLoader() to locate the WMD bootstrap by walking up from
__DIR__.

This allows the files to work at any folder depth inside the WMD installation
without relying on a hardcoded relative path.


CLI REFERENCE - LIGHTHOUSE-GATHER.PHP

Arguments:

--category="Name"
    Override the default category. Case-sensitive.

--domain=example.com
    Scan one domain only. Ignores the skip-recent rule.

--batch=2
    Run N scan processes in parallel. Default is 1.

--device=mobile|desktop|both
    Override LIGHTHOUSE_DEVICE for this run.

--skip-recent=14
    Override LIGHTHOUSE_SKIP_RECENT_DAYS for this run.

--force
    Re-scan all matching domains regardless of lh_scanned_at.

--setup-columns
    Create the ten lh_* custom columns and exit.

--dry-run
    Show what would be scanned. No scans and no database writes.

--debug
    Verbose output including SQL, parameters, and raw Node output.


EXAMPLES

Scan the default category and skip domains scanned in the last 30 days:
php lighthouse-gather.php

Scan a different category:
php lighthouse-gather.php --category="Client Sites"

Re-scan everything regardless of age:
php lighthouse-gather.php --force

Scan a single domain now in both modes:
php lighthouse-gather.php --domain=example.com

Scan a single domain in mobile mode only:
php lighthouse-gather.php --domain=example.com --device=mobile

Scan with 2 parallel workers:
php lighthouse-gather.php --batch=2

Show what would be scanned without doing anything:
php lighthouse-gather.php --dry-run --debug


RE-SCAN BEHAVIOUR

- If lh_scanned_at is NULL
  Always scanned

- If scanned within LIGHTHOUSE_SKIP_RECENT_DAYS
  Skipped

- If --force is present
  Always scanned

- If --domain=x is specified
  Always scanned


DOMAIN ELIGIBILITY

A domain is excluded from scanning only if all three conditions are true:

- availability = available
- registry_expiry is NULL
- registrar_expiry is NULL

Domains with availability set to Not Available or possibly available are always
included.


CONCURRENCY LOCK

lighthouse-gather.php uses a lock file named lighthouse-gather.lock in the log
folder to prevent concurrent runs.

If a second instance starts while one is already running, it exits immediately
with a warning.

Stale locks from crashed processes are detected by PID check and removed
automatically.


WHAT LIGHTHOUSE-SCAN.JS DOES

lighthouse-scan.js is called once per domain per device mode by
lighthouse-gather.php through proc_open().

It is executed from the Node folder so node_modules resolves correctly.

It does the following:

1. Launches headless Chromium through Puppeteer
2. Navigates to the domain URL
   It tries https first and falls back to http
3. Runs Lighthouse in the configured device mode across four categories
4. Outputs one JSON object to stdout
   Output is always valid JSON, even on error


OUTPUT JSON STRUCTURE

{
    "status": "scanned",
    "device": "mobile",
    "url": "https://example.com",
    "scores": {
        "performance": 87,
        "seo": 92,
        "accessibility": 74,
        "best-practices": 83
    },
    "audits": {
        "performance": [ ... ],
        "seo": [ ... ],
        "accessibility": [ ... ],
        "best-practices": [ ... ]
    },
    "incomplete": {
        "accessibility": [ ... ]
    },
    "fetchTime": "2026-03-26T14:32:00.000Z",
    "error": null
}

Status values:
- scanned
  Scores were obtained
- error
  Navigation failed and scores are null

The overall lh_status written to the domain table is determined by
lighthouse-gather.php after all device results are collected and the threshold
rules are applied.

Full JSON files are stored in:
[log folder]/lh-data/[domain]-[device].json

Example:
example.com-mobile.json

These files are loaded on demand by the browser report popup.


BROWSER REPORT - LIGHTHOUSE.PHP

Access as an authenticated WMD administrator:

https://yourserver/tools/lighthouse.php

Features:

- Summary score cards showing average score per category
- Status summary cards showing total, scanned, pass, fail, error, and not scanned
- Filterable table by domain and status
- Sortable columns
- Mobile and desktop toggle when both modes have data
- Color-coded scores
  - green: 90 and above
  - amber: 50 to 89
  - red: below 50
- Reset button per row to clear lh_scanned_at
- Row click opens a popup with full audit detail
- Export CSV for summary data
- Export JSON for full data
- Session expiry handling with a clear login prompt

Popup detail includes:

- Score circles for all four categories
- Mobile and desktop sub-tabs when both modes are available
- Per-category tabs
- Failing and warning audits with score, description, and display value
- Manual review section for items Lighthouse could not fully evaluate


AJAX ENDPOINTS

All endpoints are served by the same file.

?action=detail&domain=x&mode=mobile
    Return full JSON from disk for domain x and the selected mode

?action=reset&sid=x
    Clear lh_scanned_at for the domain with sid x

?action=export_csv
    Download summary CSV

?action=export_json
    Download summary plus full audit detail JSON


SCHEDULED SCANNING

The lock file prevents re-entry if a previous run is still in progress.

Because Lighthouse scans are heavier than WCAG scans, the recommended batch
size is 2 rather than 3.

The cron or scheduled task runs every 30 minutes by default.

Actual per-domain re-scan frequency is controlled by
LIGHTHOUSE_SKIP_RECENT_DAYS, which defaults to 30 days. Most scheduled runs
will usually find nothing to do and exit quickly.


LINUX / DEBIAN / UBUNTU - CRON

lighthouse-setup.sh adds a cron job automatically when INSTALL_CRON="true".

To add or edit manually:

crontab -e

Add:

*/30 * * * * php /home/wmdsed/sites/wmdsed60/tools/lighthouse/lighthouse-gather.php --batch=2 >> /home/wmdsed/logs/wmdsed/lighthouse-cron.log 2>&1


WINDOWS - TASK SCHEDULER

Three files are needed in tools\lighthouse\.

See:
- WINDOWS-SETUP.md in this folder
- learn.domainpunch.com/wmdsed/install/wintask/

lighthouse-gather.bat
"C:\xampp\php\php.exe" "C:\wmdsed\tools\lighthouse\lighthouse-gather.php" --batch=2 >> "C:\wmdsed\logs\lighthouse-cron.log" 2>&1

lighthouse-gather.vbs
Dim WinScriptHost
Set WinScriptHost = CreateObject("WScript.Shell")
WinScriptHost.Run Chr(34) & "C:\wmdsed\tools\lighthouse\lighthouse-gather.bat" & Chr(34), 0
Set WinScriptHost = Nothing

Import lighthouse-gather.xml into Task Scheduler with a 30-minute interval.


SECURITY NOTES

- lighthouse-gather.php is CLI only
  It exits immediately if accessed through a browser

- lighthouse.php requires a valid WMD administrator session
  Non-admin users are rejected, and expired sessions return a clear prompt

- The lh-data output folder is inside the WMD log folder
  It is outside the web root by default and not web-accessible

- lighthouse-scan.js runs headless Chromium with --no-sandbox
  Ensure the OS user running PHP has appropriate permissions
