Demo videos from PR builds now expire after 90 days and are posted
directly to PR threads. Uses object tagging with S3 lifecycle policies
for automatic cleanup.
## Changes
**Workflow** (`.github/workflows/tests.yml`)
- Replace `hkusu/s3-upload-action@v2` with
`ramonpaolo/action-upload-s3@main` for tagging support
- Tag uploaded objects: `expiration=90days`, `Source=github-actions`,
`Type=pr-demo-video`
- Generate unique filenames: `pr-{number}-{timestamp}.gif`
- Post PR comment with embedded video using `actions/github-script@v7`
**Documentation** (`CI_CD.md`)
- S3 lifecycle policy configuration (filters on `expiration=90days` tag)
- IAM permission requirements: `s3:PutObject`, `s3:PutObjectTagging`
## S3 Lifecycle Setup Required
```json
{
"Rules": [{
"ID": "ExpirePRDemoVideosAfter90Days",
"Status": "Enabled",
"Filter": {"Tag": {"Key": "expiration", "Value": "90days"}},
"Expiration": {"Days": 90}
}]
}
```
Apply with: `aws s3api put-bucket-lifecycle-configuration --bucket
<bucket> --lifecycle-configuration file://policy.json`
## Notes
- gh-pages `video.mp4` unaffected (served from GitHub Pages, not S3)
- Existing S3 objects without tags remain unchanged
> [!WARNING]
>
> <details>
> <summary>Firewall rules blocked me from connecting to one or more
addresses (expand for details)</summary>
>
> #### I tried to connect to the following addresses, but was blocked by
firewall rules:
>
> - `https://api.github.com/repos/ramonpaolo/action-upload-s3/tags`
> - Triggering command: `/usr/bin/curl curl -s REDACTED` (http block)
>
> If you need me to access, download, or install something from one of
these locations, you can either:
>
> - Configure [Actions setup
steps](https://gh.io/copilot/actions-setup-steps) to set up my
environment, which run before the firewall is enabled
> - Add the appropriate URLs or hosts to the custom allowlist in this
repository's [Copilot coding agent
settings](https://github.com/thomasnordquist/MQTT-Explorer/settings/copilot/coding_agent)
(admins only)
>
> </details>
<!-- START COPILOT CODING AGENT SUFFIX -->
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
> when a demo-video is generated from a pr, add an expiration of 90 days
to the S3 file and post the video as image to the pr thread.
</details>
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
Consolidates the browser mode Docker build from 3 stages to 2 by
removing the redundant intermediate `deps` stage and cleaning dev
dependencies in-place after build.
## Changes
- **Stage 1 (builder)**: Install all deps → build → remove dev deps with
`yarn install --production`
- **Stage 2 (production)**: Copy built artifacts and production
node_modules from builder (previously split across builder + deps
stages)
**Before:**
```dockerfile
# Stage 1: Build
RUN yarn install --frozen-lockfile
RUN yarn build:server
# Stage 2: Production dependencies
COPY --from=builder /build/package.json /build/yarn.lock ./
RUN yarn install --production --frozen-lockfile
# Stage 3: Production
COPY --from=builder /build/dist ./dist
COPY --from=deps /deps/node_modules ./node_modules
```
**After:**
```dockerfile
# Stage 1: Build and prepare production dependencies
RUN yarn install --frozen-lockfile
RUN yarn build:server
RUN yarn install --production --frozen-lockfile
# Stage 2: Production
COPY --from=builder /build/dist ./dist
COPY --from=builder /build/node_modules ./node_modules
```
No functional changes to final image; eliminates redundant package
resolution and copying.
<!-- START COPILOT CODING AGENT SUFFIX -->
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
> simplify the Dockerfile browser build, install dependencies and build
in the first stage, after the build remove the dev dependency
</details>
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
Docker build was failing at `yarn install --frozen-lockfile` because
`app/yarn.lock` and `backend/yarn.lock` weren't available during
dependency installation.
## Changes
- Simplified `Dockerfile.browser` to copy all source files and
dependencies at once before running `yarn install`
- This ensures all necessary files including `app/yarn.lock` and
`backend/yarn.lock` are available for reproducible dependency
installation
```dockerfile
# Before
COPY package.json yarn.lock ./
COPY app/package.json ./app/
COPY backend/package.json ./backend/
# Install ALL dependencies (needed for build)
RUN yarn install --frozen-lockfile --network-timeout 100000
# Copy source files
COPY tsconfig.json ./
COPY src ./src
COPY backend ./backend
COPY events ./events
COPY app ./app
# After
COPY package.json yarn.lock ./
COPY tsconfig.json ./
COPY src ./src
COPY backend ./backend
COPY events ./events
COPY app ./app
# Install ALL dependencies (needed for build)
RUN yarn install --frozen-lockfile --network-timeout 100000
```
This approach trades Docker layer caching optimization for a simpler,
more straightforward Dockerfile structure where all files are copied at
once.
<!-- START COPILOT CODING AGENT SUFFIX -->
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
>
https://github.com/thomasnordquist/MQTT-Explorer/actions/runs/20444356669/job/58744431418
build is failing, ensure all files have been added. Can't cd into app to
install packages. Build here to verify the solution.
</details>
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
The Docker browser build fails because yarn cannot install
`@electron/node-gyp` from GitHub without git.
## Changes
- Add git to builder stage in `Dockerfile.browser`
- Git is excluded from final production image via multi-stage build
## Technical Details
The `yarn.lock` contains a git-based dependency:
```
"@electron/node-gyp@https://github.com/electron/node-gyp#06b29aafb7708acef8b3669835c8a7857ebc92d2"
```
Alpine Linux base image lacks git by default. Installing it only in the
builder stage (where `yarn install` runs) resolves the build failure
without affecting production image size.
<!-- START COPILOT CODING AGENT SUFFIX -->
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
> fix
https://github.com/thomasnordquist/MQTT-Explorer/actions/runs/20442696351/job/58739200179
</details>
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
## Docker Build for Browser Solution
This PR creates a Docker build for the browser solution (MQTT Explorer
server mode).
### Completed:
- [x] Create a production Dockerfile for the browser solution
(`Dockerfile.browser`)
- **NEW**: 3-stage build for maximum optimization
- **NEW**: Clean production dependency installation with `yarn
--production`
- **NEW**: Only compiled dist/ folder copied (no source code)
- Alpine Linux base with Node.js 24
- Non-root user (UID 1001) for security
- Health check endpoint with proper error handling
- Proper signal handling with dumb-init
- Production dependencies automatically filtered by yarn
- [x] Apply Docker best practices (multi-stage build, minimal image,
non-root user, .dockerignore)
- Created comprehensive .dockerignore
- **FIXED**: Removed events from .dockerignore (needed for build)
- **NEW**: Optimized for smaller layers with combined RUN commands
- **NEW**: Removed development dependencies from final image
- Used alpine base image
- [x] Create GitHub Actions workflow for building, publishing, and
testing the Docker image
- Builds for linux/amd64, linux/arm64, linux/arm/v7
- **FIXED**: Added tsconfig.json and events/** to workflow trigger paths
- **FIXED**: Attestation now uses correct digest from build step output
- **NEW**: Mosquitto MQTT broker service for integration testing
- **NEW**: MQTT broker configurable via MQTT_BROKER_HOST and
MQTT_BROKER_PORT environment variables
- **NEW**: Full UI test suite runs against containerized application
- Tests container startup, health check, HTTP response, data persistence
- **NEW**: Image size reporting in workflow summary
- Tests verify application works with MQTT broker
- Publishes to GitHub Container Registry
(ghcr.io/thomasnordquist/mqtt-explorer)
- Includes build attestation for supply chain security
- [x] Configure workflow to run on push and every two weeks via cron
schedule
- Runs on 1st and 15th of each month at 2:00 AM UTC
- Also runs on push to master/beta/release branches when relevant files
change
- Manual trigger via workflow_dispatch
- [x] Add comprehensive test suite
- Basic smoke tests: startup, health check, HTTP response, data
persistence
- **NEW**: Full UI test suite (`test:browser`) runs against Docker
container
- **NEW**: Tests connect to configurable MQTT broker (default
localhost:1883)
- Tests execute with Mosquitto MQTT broker available for backend
integration
- Same comprehensive tests validate both Electron and browser modes
- [x] Test organization and naming
- **NEW**: Renamed `test:ui` to `test:electron` for Electron-specific
tests
- **NEW**: Added `test:browser` script for browser mode tests (runs same
UI test suite)
- **NEW**: Kept `test:ui` as backward-compatible alias
- **NEW**: Renamed `ui-tests` workflow job to `electron-tests` for
clarity
- [x] Update documentation with Docker usage instructions
- Created DOCKER.md with comprehensive Docker documentation
- Updated README.md with Docker quick start
- **UPDATED**: CI_CD.md now lists all 10 test steps accurately
- **NEW**: Added one-click deployment options section
- **NEW**: Added authentication modes documentation
- [x] **NEW**: One-click deployment solutions
- **NEW**: Created docker-compose.yml for easy deployment
- **NEW**: Added Play with Docker (PWD) badge for instant browser-based
demo
- **NEW**: Added DigitalOcean App Platform deployment badge
- **NEW**: Added Koyeb deployment badge
- **NEW**: Comprehensive deployment options documentation in DOCKER.md
- **NEW**: "Try It Now" section in README.md and DOCKER.md with PWD
badge
- [x] **NEW**: Enterprise authentication integration
- **NEW**: Added `MQTT_EXPLORER_SKIP_AUTH` environment variable
- **NEW**: Allows disabling built-in authentication for proxy-based auth
(OAuth2 Proxy, Authelia, enterprise SSO)
- **NEW**: Socket.IO emits `auth-status` event on connection with
authentication state
- **NEW**: Frontend receives auth status via Socket.IO and skips login
dialog when disabled
- **NEW**: Logout button hidden when authentication is disabled
- **NEW**: Created AuthContext for managing authentication state across
components
- **NEW**: Comprehensive security warnings in documentation about using
skip auth only behind trusted authentication proxies
- **NEW**: Updated docker-compose.yml with commented example for proxy
authentication
- [x] Code review and security scan passed
- Fixed health check to handle connection errors properly
- Corrected cron schedule comment
- No security vulnerabilities found
- Fixed image tag naming consistency
- Simplified dependency management
- **FIXED**: Workflow trigger paths now include all build-affecting
files
- **FIXED**: Attestation digest reference corrected
- **FIXED**: events directory included in Docker build context
- **FIXED**: Mosquitto service properly configured for integration
testing
- **FIXED**: MQTT broker connection now configurable for flexible
testing environments
- **IMPROVED**: Auth status now communicated via Socket.IO for better
real-time synchronization
- [x] Rename image to ghcr.io/thomasnordquist/mqtt-explorer (removed
-browser suffix)
- [x] Add multi-platform support for Raspberry Pi
- linux/arm64 (Raspberry Pi 3/4/5)
- linux/arm/v7 (Raspberry Pi 2/3)
- [x] Upgrade to Node.js 24 (matching project requirements)
- [x] **NEW**: Optimize Docker image for minimal size
- Only production dependencies (no devDependencies)
- No backend source code (only compiled JavaScript)
- Removed build tools and dev dependencies
- Combined layers for smaller image
- **Image size reported in workflow summary**
- [x] **NEW**: Fix webpack build configuration
- **Enable minification** for production builds (was disabled)
- **Update Material-UI references** from @material-ui to @mui
- Fix vendor chunking to include @mui and @emotion packages
- Reduces bundle size and fixes missing component issues
### Docker Image Features:
- **Base**: Alpine Linux with Node.js 24
- **Size**: Reported automatically in workflow summary
- **Platforms**: amd64, arm64, arm/v7 (Raspberry Pi support)
- **Security**: Non-root user, minimal attack surface
- **Reliability**: Health checks, graceful shutdown
- **Persistence**: Data volume at `/app/data`
- **Registry**: ghcr.io/thomasnordquist/mqtt-explorer
- **Runtime deps**: Only production dependencies (automatically
filtered)
- **Frontend**: Minified webpack bundles with proper vendor splitting
- **Testing**: Full UI test suite with configurable MQTT broker
integration
- **One-Click Deploy**: Play with Docker, DigitalOcean, Koyeb
- **Enterprise Ready**: Optional authentication bypass for proxy-based
SSO
### Available Tags:
- `latest` - Latest stable from master
- `master`, `beta`, `release` - Latest from each branch
- `<branch>-<sha>` - Specific commits
### Authentication Options:
1. **Standard Mode** (default): Built-in username/password
authentication
- Set credentials via `MQTT_EXPLORER_USERNAME` and
`MQTT_EXPLORER_PASSWORD` environment variables
2. **Skip Authentication Mode**: Set `MQTT_EXPLORER_SKIP_AUTH=true` for
proxy-based auth
- Use only behind trusted authentication proxies (OAuth2 Proxy,
Authelia, enterprise SSO)
- Socket.IO automatically informs frontend about auth status on
connection
- Frontend skips login dialog and logout button is hidden when
authentication is disabled
- ⚠️ **Security Warning**: Only use in environments with external
authentication protection
### One-Click Deployment:
Try MQTT Explorer instantly without installation:
- **Play with Docker**: Free browser-based demo (click badge in
README.md or DOCKER.md)
- **DigitalOcean**: Deploy to managed platform starting at $5/month
- **Koyeb**: Deploy to global edge network with free tier
### Security Summary:
- CodeQL scan passed with no vulnerabilities
- Docker image runs as non-root user (UID 1001)
- Multi-stage build reduces attack surface
- Health check includes proper error handling
- Minimal runtime dependencies reduce vulnerability exposure
- Full UI test suite validates application functionality
- Build attestation with correct digest reference
- MQTT broker integration tested with configurable connection via
environment variables
- Optional authentication bypass for enterprise SSO integration (with
comprehensive security warnings)
- Auth status communicated via Socket.IO for real-time synchronization
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>create a docker build for the browser
solution</issue_title>
> <issue_description>Create a docker build for amd64, that ships with a
minimal image including nodes. Apply best practices and create a test
workflow that builds it , publishes it and tests the built image. Build
it every two weeks</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
<!-- START COPILOT CODING AGENT SUFFIX -->
- Fixesthomasnordquist/MQTT-Explorer#933
<!-- START COPILOT CODING AGENT TIPS -->
---
💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
Co-authored-by: Thomas Nordquist <thomasnordquist@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Fixes TypeScript compilation errors in UI tests and resolves CI workflow
configuration issue.
## Changes Made
### 1. Fixed expandTopic parameter order in ui-tests.spec.ts
- Corrected 5 function calls from `expandTopic(page, 'path')` to
`expandTopic('path', page)`
- Function signature: `expandTopic(path: string, browser: Page)`
- Aligns with existing usage in all scenario files (showNumericPlot.ts,
publishTopic.ts, etc.)
### 2. Fixed CI workflow configuration
- Updated `.github/workflows/tests.yml` to checkout PR code instead of
base branch
- Added `ref: ${{ github.event.pull_request.head.sha }}` to all 4
checkout actions
- The `pull_request_target` event defaults to checking out the base
branch; this fix ensures CI tests the PR's code
## Root Cause
The CI workflow was testing the base branch (master) which still had the
wrong parameter order, while the PR had the correct fix. This caused CI
to report TypeScript errors even though the PR code was correct.
## Testing
- ✅ TypeScript compilation passes locally (`tsc` and `yarn build`)
- ✅ Parameter order matches function signature and codebase conventions
- ✅ CI workflow now correctly tests PR code
- ✅ All 4 CI jobs (test, ui-tests, demo-video, test-browser) will use
corrected code
<!-- START COPILOT ORIGINAL PROMPT -->
<details>
<summary>Original prompt</summary>
>
> ----
>
> *This section details on the original issue you should resolve*
>
> <issue_title>Fix tests</issue_title>
> <issue_description>- fix backend tests
> - fix UI tests</issue_description>
>
> ## Comments on the Issue (you are @copilot in this section)
>
> <comments>
> </comments>
>
</details>
<!-- START COPILOT CODING AGENT SUFFIX -->
- Fixesthomasnordquist/MQTT-Explorer#935
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for
you](https://github.com/thomasnordquist/MQTT-Explorer/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot)
— coding agent works faster and does higher quality work when set up for
your repo.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
