Compare commits

..
Author SHA1 Message Date
copilot-swe-agent[bot]andthomasnordquist e12d0544e6 Fix mosquitto service syntax: change cmd to command
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-24 12:01:12 +00:00
copilot-swe-agent[bot] 9e48b8613e Initial plan 2025-12-24 11:58:00 +00:00
Thomas Nordquist 6f8ad41ef5 Change MQTT broker host in tests workflow 2025-12-24 12:42:12 +01:00
Copilotandthomasnordquist 1ec10bb007 Fix browser UI tests: Enable anonymous access for mosquitto 2 (#994)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-24 12:05:02 +01:00
Thomas Nordquist 70e75061b2 Change MQTT broker host to localhost in tests 2025-12-24 10:11:41 +01:00
Copilotandthomasnordquist c0b2950ecb Configure MQTT broker hostname based on execution context (#993)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-24 08:23:42 +01:00
Thomas Nordquist f946ae39d7 Add bash shell specification for S3 upload step 2025-12-23 23:42:12 +01:00
Copilotandthomasnordquist 43ff3e81f0 Segment demo video by scene with embedded GIF segments (#990)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-23 23:11:18 +01:00
Copilotandthomasnordquist 79a8cdf1fd Refactor copilot instructions: add meta-directives and condense content (#992)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-23 23:07:02 +01:00
Copilotandthomasnordquist 16c190818c Set TESTS_MQTT_BROKER_HOST globally for all jobs with mosquitto services (#989)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-23 22:38:32 +01:00
Copilotandthomasnordquist de367e755f Fix server build: Remove Electron dependency from events/index exports (#987)
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: thomasnordquist <7721625+thomasnordquist@users.noreply.github.com>
2025-12-23 21:46:40 +01:00
26 changed files with 620 additions and 307 deletions
+63 -243
View File
@@ -1,263 +1,83 @@
# GitHub Copilot Agent Instructions for MQTT Explorer
# copilot-instructions.md - Agent Long-Term Memory
## Test Suites
## META-INSTRUCTIONS (IMMUTABLE)
MQTT Explorer has several test suites to ensure code quality and reliability:
1. **Long-term memory**: If you learn something during a session that would save time in future sessions, add it to `.github/copilot-instructions.md`
2. **Paper knowledge must go**: If something is no longer true, update or remove it immediately
3. **Evaluate after every session**: Consider whether the instructions need updates based on what you learned
4. **Concise and useful**: All information must be actionable, current, and concise
### Unit Tests
## Test Commands
**App tests** - Frontend component and logic tests:
**Unit tests:**
- `yarn test` - All unit tests (app + backend)
- `yarn test:app` - Frontend tests only
- `yarn test:backend` - Backend tests only
**Integration tests:**
- `yarn test:ui` - Browser tests (requires `yarn build` first)
- `yarn test:demo-video` - UI recording (requires Xvfb, mosquitto, tmux, ffmpeg)
- `yarn test:mcp` - Model Context Protocol tests
- `yarn test:all` - All tests (unit + demo-video)
- `./scripts/runBrowserTests.sh` - Browser mode UI tests (requires mosquitto service)
**CI jobs:** `test`, `ui-tests`, `demo-video`, `test-browser`, `browser-ui-tests`
**Important:** Browser UI tests require MQTT broker. In CI, GitHub Actions health checks ensure the mosquitto service is ready before tests run.
## Browser Mode
**Prerequisites:** Node.js ≥24, Yarn, Mosquitto broker (for testing)
**Development (hot reload):**
```bash
yarn test:app
# Or: cd app && yarn test
export MQTT_EXPLORER_USERNAME=admin MQTT_EXPLORER_PASSWORD=yourpass
yarn dev:server
# Backend: http://localhost:3000, Frontend: http://localhost:8080 (use this one)
```
**Backend tests** - Data model and business logic tests:
**Production:**
```bash
yarn test:backend
# Or: cd backend && yarn test
yarn build:server
export MQTT_EXPLORER_USERNAME=admin MQTT_EXPLORER_PASSWORD=yourpass
yarn start:server # http://localhost:3000
```
**Run all unit tests**:
```bash
yarn test
```
### Integration Tests
**UI test suite** - Independent, deterministic browser tests:
```bash
yarn test:ui
# Requires: yarn build
```
**Demo video generation** - UI test recording with video capture:
```bash
yarn test:demo-video
# Requires: Xvfb, mosquitto broker, tmux, ffmpeg
# For development: Use ./scripts/uiTests.sh for full video recording setup
```
**MCP introspection tests** - Model Context Protocol tests:
```bash
yarn test:mcp
```
**Run all tests** (unit + demo-video):
```bash
yarn test:all
```
### CI/CD Test Execution
In CI environments, tests run in isolated containers with all dependencies pre-installed:
- `test` job: Runs unit tests (app + backend)
- `ui-tests` job: Runs UI test suite with screenshots
- `demo-video` job: Generates demo video with full recording setup
- `test-browser` job: Runs browser mode smoke tests
**Build artifacts:** `dist/src/server.js`, `app/build/*.js`, `app/build/index.html`
## Debugging Browser Mode
### Prerequisites
- Node.js 24 or higher
- Yarn package manager
- Running Mosquitto MQTT broker (for testing)
**DevTools checks:**
- Console: JavaScript errors, CSP errors (security headers), WebSocket issues
- Network: Static asset loading, WebSocket status, API auth
### Development Mode (with Hot Reload)
**Common issues:**
- **Blank page/CSP errors:** Add `'unsafe-eval'` to `scriptSrc` in `src/server.ts` helmet config
- **Auth loop:** WebSocket auth failing - check Network → WS → Messages and server logs
- **Theme errors:** Verify both ThemeProvider and LegacyThemeProvider in `app/src/index.tsx`
1. **Set credentials (required):**
```bash
export MQTT_EXPLORER_USERNAME=admin
export MQTT_EXPLORER_PASSWORD=your_password
```
2. **Start development servers:**
```bash
yarn dev:server
```
This runs two servers in parallel:
- Backend server on http://localhost:3000 (serves API, WebSocket, authentication)
- Webpack dev server on http://localhost:8080 (serves frontend with hot reload)
3. **Access the application:**
- Navigate to http://localhost:8080 (NOT :3000)
- Webpack dev server proxies API/WebSocket requests to backend on port 3000
- Hot reload enabled - changes to React components update automatically
### Production Mode (Production Build)
1. **Build the browser version:**
```bash
yarn build:server
```
This compiles TypeScript and builds the optimized webpack bundle
2. **Start the server:**
```bash
# Set credentials (required) - these are for the browser login page
export MQTT_EXPLORER_USERNAME=admin
export MQTT_EXPLORER_PASSWORD=your_password
# Start server
yarn start:server
# OR: node dist/src/server.js
```
Server will run on http://localhost:3000 (serves both frontend and backend)
3. **Login to the application:**
- Navigate to http://localhost:3000
- Enter the username and password you set in the environment variables
- Click "LOGIN" button
- After successful login, the main application will load
- The MQTT Connection modal will appear where you can configure broker connections
### Debugging with Browser DevTools
1. **Open browser DevTools:**
- Navigate to http://localhost:3000
- Press F12 or right-click → Inspect
2. **Check Console tab:**
- Look for JavaScript errors
- CSP (Content Security Policy) errors indicate security header issues
- Network errors indicate API/WebSocket connection issues
3. **Check Network tab:**
- Verify static assets load correctly (JS bundles, CSS)
- Check WebSocket connection status
- Monitor API calls for authentication issues
4. **Common Issues:**
**Blank page / CSP errors:**
- Symptom: Console shows `EvalError: ... violates Content Security Policy`
- Cause: webpack runtime requires `unsafe-eval` for code splitting
- Fix: Add `'unsafe-eval'` to `scriptSrc` in `src/server.ts` helmet config
**Authentication loop:**
- Symptom: Login dialog keeps reappearing
- Cause: WebSocket authentication failing
- Debug: Check browser Network tab → WS → Messages
- Check: Server logs for authentication errors
**Theme errors:**
- Symptom: App loads but styling is broken
- Cause: Material-UI theme not loading correctly
- Check: Console for theme-related errors
- Verify: Both ThemeProvider and LegacyThemeProvider in `app/src/index.tsx`
**Expected console warnings (non-fatal):**
- React 18 type warnings with Material-UI v5 components (dozens of "Failed prop type" warnings)
- `TypeError: Cannot read properties of undefined (reading 'on')` from IpcRendererEventBus - this is expected in browser mode as there's no Electron IPC
- MUI locale warnings for `en-US` - expected, app uses available locales
- `componentWillReceiveProps` deprecation warnings - from legacy TreeComponent
- ACE editor autocomplete warnings - expected, features not imported
- CSP worker violation for ACE editor - known issue, editor still functions
These warnings don't prevent the application from functioning correctly.
### Using Playwright for Automated Testing
**Expected warnings (non-fatal):**
- React 18 + Material-UI v5 type warnings
- IpcRendererEventBus errors (no Electron IPC in browser mode)
- MUI locale, componentWillReceiveProps, ACE editor warnings
**WebSocket debugging:**
```bash
# Start server in background
export MQTT_EXPLORER_USERNAME=admin
export MQTT_EXPLORER_PASSWORD=test123
node dist/src/server.js &
# Use Playwright browser tool (in Copilot agent context)
playwright-browser_navigate http://localhost:3000
playwright-browser_take_screenshot --filename debug.png
playwright-browser_console_messages # Check for errors
node dist/src/server.js 2>&1 | tee server.log
# Check DevTools → Network → WS → Messages for handshake
# CORS: check ALLOWED_ORIGINS env var
```
### Expected UI Flow
**Security notes:**
- `unsafe-eval` in CSP required for webpack (security tradeoff)
- Never hardcode credentials (use env vars)
- Production: use HTTPS with reverse proxy
- Rate limit: 5 auth attempts/15min/IP
- File upload limit: 16MB
1. **Login Page** (https://github.com/user-attachments/assets/383305e1-2169-433c-a668-5a05da0c343a)
- Enter username and password from environment variables
- Click "LOGIN" button
2. **Main Application After Login** (https://github.com/user-attachments/assets/cc4d665f-2665-4289-b2fc-dc4986f9ab5b)
- Application loads with sidebar, topic tree, value panel, and publish panel
- MQTT Connection modal appears automatically for first-time setup
- Configure broker connection (host, port, credentials, etc.)
- Click "CONNECT" to establish MQTT connection
3. **Application Features:**
- Topic tree on the left shows MQTT topic hierarchy
- Value panel shows selected topic's message content
- Publish panel allows sending MQTT messages
- Charts panel for numeric value visualization
- Settings drawer for app configuration
### Debugging WebSocket Connection
1. **Check server logs:**
```bash
node dist/src/server.js 2>&1 | tee server.log
```
2. **Check browser WebSocket:**
- DevTools → Network → WS tab
- Look for socket.io connection
- Check Messages tab for authentication handshake
3. **Common WebSocket issues:**
- CORS errors: Check `ALLOWED_ORIGINS` environment variable
- Authentication errors: Verify credentials in sessionStorage
- Connection refused: Server not running or port blocked
### Development vs Production
**Development mode:**
```bash
yarn dev:server
# Runs webpack-dev-server with hot reload
# More verbose error messages
# Source maps enabled
```
**Production mode:**
```bash
NODE_ENV=production yarn build:server
NODE_ENV=production node dist/src/server.js
# Minified bundles
# Generic error messages (security)
# HSTS enabled
```
### Build Artifacts
After `yarn build:server`, check:
- `dist/src/server.js` - Compiled server code
- `app/build/*.js` - Webpack bundles
- `app/build/index.html` - Entry point HTML
### Troubleshooting Checklist
- [ ] Node.js version >=24
- [ ] `yarn install` completed without errors
- [ ] TypeScript compilation successful (`npx tsc`)
- [ ] Webpack build successful (check `app/build/` directory)
- [ ] Server starts without errors
- [ ] Can access http://localhost:3000
- [ ] Login dialog appears
- [ ] No CSP errors in console
- [ ] WebSocket connects successfully
- [ ] App renders after login
### Security Considerations
When debugging, be aware that:
- `unsafe-eval` in CSP is required for webpack but reduces security
- Credentials should never be hardcoded (use environment variables)
- In production, use HTTPS with a reverse proxy (nginx/Apache)
- Rate limiting is active (5 auth attempts per 15 min per IP)
- File upload size limit is 16MB
### Related Files
- `src/server.ts` - Express server with security middleware
- `app/webpack.browser.config.mjs` - Browser-specific webpack config
- `app/src/browserEventBus.ts` - Socket.io client for browser mode
- `app/src/components/BrowserAuthWrapper.tsx` - Authentication dialog
- `app/src/index.tsx` - React app entry point with theme providers
**Key files:**
- `src/server.ts` - Express server, security middleware
- `app/webpack.browser.config.mjs` - Browser webpack config
- `app/src/browserEventBus.ts` - Socket.io client
- `app/src/components/BrowserAuthWrapper.tsx` - Auth dialog
- `app/src/index.tsx` - React entry, theme providers
@@ -14,6 +14,9 @@ on:
jobs:
copilot-setup-steps:
runs-on: ubuntu-latest
env:
TESTS_MQTT_BROKER_HOST: localhost
TESTS_MQTT_BROKER_PORT: 1883
steps:
- name: Checkout code
+6 -2
View File
@@ -30,6 +30,10 @@ jobs:
packages: write
id-token: write
env:
TESTS_MQTT_BROKER_HOST: localhost
TESTS_MQTT_BROKER_PORT: 1883
services:
# MQTT broker for testing
mosquitto:
@@ -41,6 +45,8 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
--entrypoint sh
command: -c "mkdir -p /mosquitto/config && echo 'listener 1883' > /mosquitto/config/mosquitto.conf && echo 'allow_anonymous true' >> /mosquitto/config/mosquitto.conf && exec mosquitto -c /mosquitto/config/mosquitto.conf"
steps:
- name: Checkout repository
@@ -191,8 +197,6 @@ jobs:
MQTT_EXPLORER_USERNAME: test
MQTT_EXPLORER_PASSWORD: test123
BROWSER_MODE_URL: http://localhost:3000
MQTT_BROKER_HOST: localhost
MQTT_BROKER_PORT: 1883
- name: Clean up browser test container
if: always()
+69 -34
View File
@@ -28,20 +28,15 @@ jobs:
volumes:
- ./:/app
options: --user root
services:
mosquitto:
image: eclipse-mosquitto:2
ports:
- 1883:1883
options: >-
--health-cmd "mosquitto_sub -t '$SYS/#' -C 1"
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
TESTS_MQTT_BROKER_HOST: mosquitto
TESTS_MQTT_BROKER_PORT: 1883
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Start mosquitto
run: mosquitto -c /etc/mosquitto/conf.d/default.conf -d
- name: Install Packages
run: yarn install --frozen-lockfile
- name: Build Browser Mode
@@ -49,9 +44,6 @@ jobs:
- name: Run Browser UI Tests
timeout-minutes: 10
run: ./scripts/runBrowserTests.sh
env:
MQTT_BROKER_HOST: mosquitto
MQTT_BROKER_PORT: 1883
- name: Upload Test Screenshots
if: always()
uses: actions/upload-artifact@v4
@@ -68,10 +60,15 @@ jobs:
volumes:
- ./:/app
options: --user root
env:
TESTS_MQTT_BROKER_HOST: localhost
TESTS_MQTT_BROKER_PORT: 1883
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Start mosquitto
run: mosquitto -c /etc/mosquitto/conf.d/default.conf -d
- name: Install Packages
run: yarn install --frozen-lockfile
- name: Build
@@ -80,12 +77,12 @@ jobs:
run: yarn ui-test
- name: Post-processing
run: ./scripts/prepareVideo.sh
- name: Generate unique filename
id: filename
- name: Generate unique base path
id: basepath
run: |
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
FILENAME="pr-${{ github.event.pull_request.number }}-${TIMESTAMP}.gif"
echo "filename=${FILENAME}" >> $GITHUB_OUTPUT
BASEPATH="pr-${{ github.event.pull_request.number }}-${TIMESTAMP}"
echo "basepath=${BASEPATH}" >> $GITHUB_OUTPUT
- name: Install AWS CLI v2
run: |
apt-get update && apt-get install -y unzip
@@ -99,48 +96,84 @@ jobs:
aws-access-key-id: ${{ vars.AWS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: 'eu-central-1'
- name: Upload to S3
id: upload
- name: Upload full video to S3
env:
AWS_BUCKET: ${{ vars.AWS_BUCKET }}
FILENAME: ${{ steps.filename.outputs.filename }}
BASEPATH: ${{ steps.basepath.outputs.basepath }}
run: |
# Upload GIF
aws s3api put-object \
--bucket ${AWS_BUCKET} \
--key artifacts/${FILENAME} \
--key artifacts/${BASEPATH}/ui-test.gif \
--body ./ui-test.gif \
--content-type image/gif
- name: Generate file URL
# Upload MP4
aws s3api put-object \
--bucket ${AWS_BUCKET} \
--key artifacts/${BASEPATH}/ui-test.mp4 \
--body ./ui-test.mp4 \
--content-type video/mp4
- name: Upload video segments to S3
env:
AWS_BUCKET: ${{ vars.AWS_BUCKET }}
BASEPATH: ${{ steps.basepath.outputs.basepath }}
shell: bash
run: |
# Upload all GIF segment files if they exist
shopt -s nullglob # Make glob return empty list if no matches
for segment in segment-*.gif; do
echo "Uploading $segment..."
aws s3api put-object \
--bucket ${AWS_BUCKET} \
--key artifacts/${BASEPATH}/${segment} \
--body ./${segment} \
--content-type image/gif
done
shopt -u nullglob # Restore default behavior
- name: Generate file URLs
id: fileurl
env:
AWS_BUCKET: ${{ vars.AWS_BUCKET }}
FILENAME: ${{ steps.filename.outputs.filename }}
BASEPATH: ${{ steps.basepath.outputs.basepath }}
run: |
FILE_URL="https://${AWS_BUCKET}.s3.eu-central-1.amazonaws.com/artifacts/${FILENAME}"
echo "file-url=${FILE_URL}" >> $GITHUB_OUTPUT
echo "Uploaded to: ${FILE_URL}"
- name: Show URL
run: echo '${{ steps.fileurl.outputs.file-url }}'
id: artifact-upload-step
- run: echo '<picture><img src="${{ steps.fileurl.outputs.file-url }}"></picture>' \
>> $GITHUB_STEP_SUMMARY
BASE_URL="https://${AWS_BUCKET}.s3.eu-central-1.amazonaws.com/artifacts/${BASEPATH}"
echo "base-url=${BASE_URL}" >> $GITHUB_OUTPUT
echo "Uploaded to: ${BASE_URL}"
- name: Generate markdown summary
id: markdown
env:
BASE_URL: ${{ steps.fileurl.outputs.base-url }}
run: |
MARKDOWN=$(node ./scripts/generateMarkdownSummary.js "${BASE_URL}")
echo "markdown<<EOF" >> $GITHUB_OUTPUT
echo "$MARKDOWN" >> $GITHUB_OUTPUT
echo "EOF" >> $GITHUB_OUTPUT
- name: Add to workflow summary
env:
MARKDOWN: ${{ steps.markdown.outputs.markdown }}
run: |
echo "$MARKDOWN" >> $GITHUB_STEP_SUMMARY
- name: Post video to PR
uses: actions/github-script@v7
env:
VIDEO_URL: ${{ steps.fileurl.outputs.file-url }}
MARKDOWN: ${{ steps.markdown.outputs.markdown }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const videoUrl = process.env.VIDEO_URL;
const markdown = process.env.MARKDOWN;
github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: context.issue.number,
body: `## 🎬 Demo Video Generated\n\n![Demo Video](${videoUrl})\n\n_This video will expire in 90 days._`
body: markdown
});
test-browser:
runs-on: ubuntu-latest
env:
TESTS_MQTT_BROKER_HOST: localhost
TESTS_MQTT_BROKER_PORT: 1883
services:
mosquitto:
image: eclipse-mosquitto:2
@@ -151,6 +184,8 @@ jobs:
--health-interval 10s
--health-timeout 5s
--health-retries 5
--entrypoint sh
command: -c "mkdir -p /mosquitto/config && echo 'listener 1883' > /mosquitto/config/mosquitto.conf && echo 'allow_anonymous true' >> /mosquitto/config/mosquitto.conf && exec mosquitto -c /mosquitto/config/mosquitto.conf"
steps:
- uses: actions/checkout@v4
with:
+13 -1
View File
@@ -23,4 +23,16 @@ browser-debug-screenshot.png
app/.webpack-cache
# Temporary files
/tmp
/tmp
# Demo video artifacts
scenes.json
segment-*.mp4
segment-*.gif
ui-test.mp4
ui-test.gif
app.mp4
app2.mp4
app720.gif
qrawvideorgb24.yuv
intro.png
+6
View File
@@ -10,6 +10,12 @@ ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
# Configure mosquitto for anonymous access (required for tests)
RUN mkdir -p /etc/mosquitto/conf.d && \
echo "listener 1883" > /etc/mosquitto/conf.d/default.conf && \
echo "allow_anonymous true" >> /etc/mosquitto/conf.d/default.conf && \
echo "persistence false" >> /etc/mosquitto/conf.d/default.conf
# Install Playwright and browsers
# This ensures Playwright browsers are pre-installed in the container
RUN npm install -g playwright@1.57.0 && \
+1 -1
View File
@@ -46,7 +46,7 @@
"moving-average": "^1.0.0",
"number-abbreviate": "^2.0.0",
"os-browserify": "^0.3.0",
"parse-duration": "^2.1.3",
"parse-duration": "^0.1.1",
"path-browserify": "^1.0.1",
"prismjs": "^1.29.0",
"react": "^19.2.3",
+4 -4
View File
@@ -4144,10 +4144,10 @@ parent-module@^1.0.0:
dependencies:
callsites "^3.0.0"
parse-duration@^2.1.3:
version "2.1.3"
resolved "https://registry.yarnpkg.com/parse-duration/-/parse-duration-2.1.3.tgz#79a61a3ae224a5f4d1b71a8fa79e53d5aa90c902"
integrity sha512-MtbharL7Bets65qDBXuDOHHWyY1BxTJZmJ/xGmS90iEbKE0gZ6yZpZtCda7O79GeOi/f0NwBaplIuReExIoVsw==
parse-duration@^0.1.1:
version "0.1.3"
resolved "https://registry.yarnpkg.com/parse-duration/-/parse-duration-0.1.3.tgz#c2c4d45d49513d544e129b2a5a07b9473545d19a"
integrity sha512-hMOZHfUmjxO5hMKn7Eft+ckP2M4nV4yzauLXiw3PndpkASnx5r8pDAMcOAiqxoemqWjMWmz4fOHQM6n6WwETXw==
parse-json@^5.0.0:
version "5.2.0"
+4 -1
View File
@@ -1,5 +1,8 @@
export * from './Events'
export * from './EventsV2'
export * from './EventSystem/EventDispatcher'
export * from './EventSystem/EventBus'
// EventBus exports removed - this file contains Electron-specific imports
// which should not be loaded in server/browser mode
// Electron code should import directly from './EventSystem/EventBus'
// export * from './EventSystem/EventBus'
export * from './EventSystem/EventBusInterface'
+106
View File
@@ -0,0 +1,106 @@
#!/bin/bash
set -e
# Read scenes.json and cut video into segments as GIFs
if [ ! -f "scenes.json" ]; then
echo "scenes.json not found"
exit 1
fi
if [ ! -f "ui-test.mp4" ]; then
echo "ui-test.mp4 not found"
exit 1
fi
echo "Cutting video into GIF segments based on scenes.json..."
GIF_SCALE="1024"
# Parse scenes.json and cut video segments as GIFs
node -e "
const fs = require('fs');
const { spawn } = require('child_process');
const scenes = JSON.parse(fs.readFileSync('scenes.json', 'utf8'));
console.log('Creating GIF segments...');
// Sanitize scene name to prevent path traversal and command injection
function sanitizeName(name) {
// Remove any characters that aren't alphanumeric, dash, or underscore
return name.replace(/[^a-zA-Z0-9_-]/g, '-');
}
async function cutSegmentAsGif(scene, index) {
const safeName = sanitizeName(scene.name);
const segmentName = \`segment-\${String(index + 1).padStart(2, '0')}-\${safeName}\`;
const paletteFile = \`\${segmentName}-palette.png\`;
const outputFile = \`\${segmentName}.gif\`;
const startTime = scene.start / 1000; // Convert ms to seconds
const duration = scene.duration / 1000; // Convert ms to seconds
console.log(\`Creating \${outputFile} (start: \${startTime}s, duration: \${duration}s)\`);
// Step 1: Generate palette for this segment
await new Promise((resolve, reject) => {
const ffmpeg = spawn('ffmpeg', [
'-y',
'-ss', startTime.toString(),
'-t', duration.toString(),
'-i', 'ui-test.mp4',
'-vf', 'fps=10,scale=${process.env.GIF_SCALE || 1024}:-1:flags=lanczos,palettegen',
paletteFile
]);
ffmpeg.on('close', (code) => {
if (code === 0) {
resolve();
} else {
console.error(\`Failed to create palette for \${outputFile}\`);
reject(new Error(\`ffmpeg palette generation exited with code \${code}\`));
}
});
});
// Step 2: Create GIF using the palette
await new Promise((resolve, reject) => {
const ffmpeg = spawn('ffmpeg', [
'-y',
'-ss', startTime.toString(),
'-t', duration.toString(),
'-i', 'ui-test.mp4',
'-i', paletteFile,
'-filter_complex', 'fps=10,scale=${process.env.GIF_SCALE || 1024}:-1:flags=lanczos[x];[x][1:v]paletteuse',
outputFile
]);
ffmpeg.on('close', (code) => {
// Clean up palette file
try {
fs.unlinkSync(paletteFile);
} catch (e) {
// Ignore cleanup errors
}
if (code === 0) {
resolve();
} else {
console.error(\`Failed to create \${outputFile}\`);
reject(new Error(\`ffmpeg GIF creation exited with code \${code}\`));
}
});
});
}
(async () => {
for (let i = 0; i < scenes.length; i++) {
await cutSegmentAsGif(scenes[i], i);
}
console.log('All GIF segments created successfully');
})().catch(err => {
console.error(err);
process.exit(1);
});
"
echo "Video segments created successfully"
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env node
const fs = require('fs');
// Read scenes.json
const scenes = JSON.parse(fs.readFileSync('scenes.json', 'utf8'));
// Get base URL from command line arguments
const baseUrl = process.argv[2];
if (!baseUrl) {
console.error('Usage: node generateMarkdownSummary.js <base-url>');
process.exit(1);
}
// Sanitize scene name to prevent path traversal
function sanitizeName(name) {
// Remove any characters that aren't alphanumeric, dash, or underscore
return name.replace(/[^a-zA-Z0-9_-]/g, '-');
}
// Generate markdown
let markdown = '## 🎬 Demo Video Generated\n\n';
markdown += `### Full Video\n\n`;
markdown += `[📥 Download Full Video (MP4)](${baseUrl}/ui-test.mp4) | [GIF](${baseUrl}/ui-test.gif)\n\n`;
markdown += `---\n\n`;
markdown += `### 📑 Video Segments\n\n`;
markdown += `<details>\n`;
markdown += `<summary>Click to expand segments</summary>\n\n`;
scenes.forEach((scene, index) => {
const safeName = sanitizeName(scene.name);
const segmentFile = `segment-${String(index + 1).padStart(2, '0')}-${safeName}.gif`;
const title = scene.title || scene.name;
const duration = (scene.duration / 1000).toFixed(1);
markdown += `<details>\n`;
markdown += `<summary><strong>${index + 1}. ${title}</strong> (${duration}s)</summary>\n\n`;
markdown += `![${title}](${baseUrl}/${segmentFile})\n\n`;
markdown += `</details>\n\n`;
});
markdown += `</details>\n\n`;
markdown += `_Videos will expire in 90 days._`;
console.log(markdown);
+9
View File
@@ -25,3 +25,12 @@ rm ffmpeg_info palette*.png qrawvideorgb24.yuv
mv app.mp4 ui-test.mp4
mv app720.gif ui-test.gif
# Cut video into segments based on scenes.json
echo "Cutting video into segments..."
if [ -f "scenes.json" ]; then
./scripts/cutVideoSegments.sh
else
echo "Warning: scenes.json not found, skipping segment creation"
fi
+5 -5
View File
@@ -10,8 +10,8 @@
# MQTT_EXPLORER_PASSWORD - Password for browser authentication (default: test123)
# PORT - Server port (default: 3000)
# BROWSER_MODE_URL - URL for browser tests (set automatically)
# MQTT_BROKER_HOST - MQTT broker host for tests (required, default: 127.0.0.1)
# MQTT_BROKER_PORT - MQTT broker port for tests (default: 1883)
# TESTS_MQTT_BROKER_HOST - MQTT broker host for tests (required, default: 127.0.0.1)
# TESTS_MQTT_BROKER_PORT - MQTT broker port for tests (default: 1883)
#
set -e
@@ -52,10 +52,10 @@ done
# Run browser tests
export BROWSER_MODE_URL="http://localhost:${PORT}"
export MQTT_BROKER_HOST="${MQTT_BROKER_HOST:-127.0.0.1}"
export MQTT_BROKER_PORT="${MQTT_BROKER_PORT:-1883}"
export TESTS_MQTT_BROKER_HOST="${TESTS_MQTT_BROKER_HOST:-127.0.0.1}"
export TESTS_MQTT_BROKER_PORT="${TESTS_MQTT_BROKER_PORT:-1883}"
echo "Using MQTT broker at $MQTT_BROKER_HOST:$MQTT_BROKER_PORT"
echo "Using MQTT broker at $TESTS_MQTT_BROKER_HOST:$TESTS_MQTT_BROKER_PORT"
yarn test:browser
TEST_EXIT_CODE=$?
+185
View File
@@ -0,0 +1,185 @@
#!/usr/bin/env node
/**
* Integration test for video segmentation workflow
*
* This test validates:
* 1. SceneBuilder generates scenes with titles
* 2. Scenes are saved to scenes.json
* 3. cutVideoSegments script processes scenes correctly
* 4. generateMarkdownSummary creates proper output
*/
const fs = require('fs');
const path = require('path');
const os = require('os');
const { execSync } = require('child_process');
function testSceneGeneration() {
console.log('Testing scene generation...');
// Check if compiled files exist
const distPath = path.join(__dirname, '../dist/src/spec/SceneBuilder.js');
if (!fs.existsSync(distPath)) {
console.log(' ⚠ Skipping test - TypeScript not compiled. Run "npx tsc" first.');
return null;
}
// Import and test SceneBuilder
const SceneBuilder = require('../dist/src/spec/SceneBuilder').SceneBuilder;
const SCENE_TITLES = require('../dist/src/spec/SceneBuilder').SCENE_TITLES;
// Verify SCENE_TITLES exist
const titleKeys = Object.keys(SCENE_TITLES);
if (titleKeys.length === 0) {
throw new Error('SCENE_TITLES is empty');
}
console.log(` ✓ Found ${titleKeys.length} scene titles`);
// Create a sample scene
const builder = new SceneBuilder();
const testScenes = [
{ name: 'connect', start: 0, stop: 1000, duration: 1000, title: SCENE_TITLES.connect },
{ name: 'numeric_plots', start: 1000, stop: 2000, duration: 1000, title: SCENE_TITLES.numeric_plots },
{ name: 'end', start: 2000, stop: 3000, duration: 1000, title: SCENE_TITLES.end },
];
// Manually populate scenes for testing
builder.scenes = testScenes;
// Save to JSON
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'test-workflow-'));
const scenesPath = path.join(tempDir, 'scenes.json');
fs.writeFileSync(scenesPath, JSON.stringify(builder.scenes, null, 2));
console.log(` ✓ Saved scenes to ${scenesPath}`);
return { tempDir, scenesPath, scenes: builder.scenes };
}
function testCuttingLogic(tempDir, scenes) {
console.log('Testing cutting logic...');
// Create dummy video file
const videoPath = path.join(tempDir, 'ui-test.mp4');
fs.writeFileSync(videoPath, 'dummy video content');
// Process scenes like the cutting script would (now creates GIFs)
const expectedSegments = [];
scenes.forEach((scene, index) => {
const outputFile = `segment-${String(index + 1).padStart(2, '0')}-${scene.name}.gif`;
const startTime = scene.start / 1000;
const duration = scene.duration / 1000;
expectedSegments.push({
filename: outputFile,
start: startTime,
duration: duration,
title: scene.title
});
});
console.log(` ✓ Processed ${expectedSegments.length} segments`);
// Verify segment naming
expectedSegments.forEach(seg => {
if (!seg.filename.match(/^segment-\d{2}-.+\.gif$/)) {
throw new Error(`Invalid segment filename: ${seg.filename}`);
}
});
console.log(' ✓ All segment filenames are valid');
return expectedSegments;
}
function testMarkdownGeneration(tempDir) {
console.log('Testing markdown generation...');
const baseUrl = 'https://example.com/test-pr-123';
const scenesPath = path.join(tempDir, 'scenes.json');
// Run markdown generation script
const originalCwd = process.cwd();
process.chdir(tempDir);
let markdown;
try {
markdown = execSync(
`node "${path.join(__dirname, 'generateMarkdownSummary.js')}" "${baseUrl}"`,
{ encoding: 'utf8' }
);
} finally {
process.chdir(originalCwd);
}
// Verify markdown content
if (!markdown.includes('## 🎬 Demo Video Generated')) {
throw new Error('Markdown missing title');
}
if (!markdown.includes('### Full Video')) {
throw new Error('Markdown missing full video section');
}
if (!markdown.includes('### 📑 Video Segments')) {
throw new Error('Markdown missing segments section');
}
if (!markdown.includes('<details>')) {
throw new Error('Markdown missing collapsible sections');
}
if (!markdown.includes('Connecting to MQTT Broker')) {
throw new Error('Markdown missing scene title');
}
if (!markdown.includes(baseUrl)) {
throw new Error('Markdown missing base URL');
}
console.log(' ✓ Markdown structure is valid');
console.log(' ✓ Markdown contains all required sections');
return markdown;
}
function cleanup(tempDir) {
console.log('Cleaning up...');
fs.rmSync(tempDir, { recursive: true, force: true });
console.log(' ✓ Temporary files removed');
}
// Run tests
try {
console.log('=== Video Segmentation Integration Test ===\n');
const result = testSceneGeneration();
if (!result) {
console.log('\n=== Test Skipped (TypeScript not compiled) ===\n');
console.log('Run "npx tsc" to compile TypeScript before running this test.');
process.exit(0);
}
const { tempDir, scenesPath, scenes } = result;
const segments = testCuttingLogic(tempDir, scenes);
const markdown = testMarkdownGeneration(tempDir);
cleanup(tempDir);
console.log('\n=== All Tests Passed ✓ ===\n');
// Display sample output
console.log('Sample Markdown Output:');
console.log('─'.repeat(60));
console.log(markdown.split('\n').slice(0, 25).join('\n'));
console.log('...');
console.log('─'.repeat(60));
process.exit(0);
} catch (error) {
console.error('\n✗ Test failed:', error.message);
console.error(error.stack);
process.exit(1);
}
+2 -1
View File
@@ -19,7 +19,8 @@ import {
import { shouldAutoUpdate, handleAutoUpdate } from './autoUpdater'
import { registerCrashReporter } from './registerCrashReporter'
import { makeOpenDialogRpc, makeSaveDialogRpc } from '../events/OpenDialogRequest'
import { backendRpc, backendEvents, getAppVersion, writeToFile, readFromFile } from '../events'
import { getAppVersion, writeToFile, readFromFile } from '../events'
import { backendRpc, backendEvents } from '../events/EventSystem/EventBus'
import { RpcEvents } from '../events/EventsV2'
registerCrashReporter()
+50
View File
@@ -0,0 +1,50 @@
import { expect } from 'chai'
import { SceneBuilder, SCENE_TITLES } from './SceneBuilder'
describe('SceneBuilder', () => {
it('should record scenes with titles', async () => {
const builder = new SceneBuilder()
await builder.record('connect', async () => {
// Simulate some work
await new Promise(resolve => setTimeout(resolve, 100))
})
expect(builder.scenes).to.have.length(1)
expect(builder.scenes[0].name).to.equal('connect')
expect(builder.scenes[0].title).to.equal('Connecting to MQTT Broker')
expect(builder.scenes[0].duration).to.be.greaterThan(90)
})
it('should have titles for all scene types', () => {
const sceneNames = Object.keys(SCENE_TITLES)
expect(sceneNames.length).to.be.greaterThan(0)
// Verify each scene has a non-empty title
sceneNames.forEach(name => {
expect(SCENE_TITLES[name as keyof typeof SCENE_TITLES]).to.be.a('string')
expect(SCENE_TITLES[name as keyof typeof SCENE_TITLES].length).to.be.greaterThan(0)
})
})
it('should record multiple scenes in sequence', async () => {
const builder = new SceneBuilder()
await builder.record('connect', async () => {
await new Promise(resolve => setTimeout(resolve, 50))
})
await builder.record('numeric_plots', async () => {
await new Promise(resolve => setTimeout(resolve, 50))
})
expect(builder.scenes).to.have.length(2)
expect(builder.scenes[0].name).to.equal('connect')
expect(builder.scenes[0].title).to.equal('Connecting to MQTT Broker')
expect(builder.scenes[1].name).to.equal('numeric_plots')
expect(builder.scenes[1].title).to.equal('Plot Topic History')
// Second scene should start at or after first one ends
expect(builder.scenes[1].start).to.be.at.least(builder.scenes[0].stop)
})
})
+20
View File
@@ -3,6 +3,7 @@ export interface Scene {
start: number
stop: number
duration: number
title?: string
}
export type SceneNames =
@@ -22,6 +23,24 @@ export type SceneNames =
| 'sparkplugb-decoding'
| 'end'
export const SCENE_TITLES: Record<SceneNames, string> = {
connect: 'Connecting to MQTT Broker',
topic_updates: 'Topic Updates',
numeric_plots: 'Plot Topic History',
'json-formatting': 'Formatted Messages',
diffs: 'Diff Capability',
publish_topic: 'Publish Topics',
json_formatting_publish: 'JSON Formatting Publish',
clipboard: 'Copy to Clipboard',
topic_filter: 'Search Topic Hierarchy',
delete_retained_topics: 'Delete Retained Topics',
settings: 'Settings',
customize_subscriptions: 'Customize Subscriptions',
keyboard_shortcuts: 'Keyboard Shortcuts',
'sparkplugb-decoding': 'SparkplugB Decoding',
end: 'The End',
}
export class SceneBuilder {
public scenes: Array<Scene> = []
public offset = Date.now()
@@ -36,6 +55,7 @@ export class SceneBuilder {
start,
stop,
duration: stop - start,
title: SCENE_TITLES[name],
})
}
}
+4 -2
View File
@@ -48,7 +48,9 @@ setTimeout(
const runningUiTestOnCi = os.platform() === 'darwin' ? [] : ['--runningUiTestOnCi']
async function doStuff() {
console.log('Waiting for MQTT Broker on port 1880 (no auth)')
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
const brokerPort = process.env.TESTS_MQTT_BROKER_PORT || '1883'
console.log(`Waiting for MQTT Broker at ${brokerHost}:${brokerPort} (no auth)`)
await mockMqtt()
console.log('Starting playwright/electron')
@@ -73,7 +75,7 @@ async function doStuff() {
const scenes = new SceneBuilder()
await scenes.record('connect', async () => {
await connectTo('127.0.0.1', page)
await connectTo(brokerHost, page)
await MockSparkplug.run() // Start sparkplug client after connect or birth topics will be missed
await sleep(1000)
})
+2 -1
View File
@@ -55,7 +55,8 @@ describe('expandTopic UI Helper - Isolated Test', function () {
await page.locator('//label[contains(text(), "Host")]/..//input').waitFor({ timeout: 10000 })
console.log('Connecting to MQTT broker...')
await connectTo('127.0.0.1', page)
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
await connectTo(brokerHost, page)
await sleep(3000) // Give time for topics to load
console.log('Setup complete')
})
+4 -2
View File
@@ -15,7 +15,9 @@ process.on('unhandledRejection' as any, (error: Error | any) => {
const runningUiTestOnCi = os.platform() === 'darwin' ? [] : ['--runningUiTestOnCi']
async function doStuff() {
console.log('Waiting for MQTT Broker on port 1880 (no auth)')
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
const brokerPort = process.env.TESTS_MQTT_BROKER_PORT || '1883'
console.log(`Waiting for MQTT Broker at ${brokerHost}:${brokerPort} (no auth)`)
await mockMqtt()
console.log('Starting playwright/electron')
@@ -39,7 +41,7 @@ async function doStuff() {
await createFakeMousePointer(browser)
// Wait for Username input to be visible
await browser.locator('//label[contains(text(), "Username")]/..//input')
await connectTo('127.0.0.1', browser)
await connectTo(brokerHost, browser)
stopMqttUpdates()
await sleep(1000, true)
const heapDump = await getHeapDump(browser)
+3 -4
View File
@@ -16,15 +16,14 @@ export async function createTestMock(): Promise<mqtt.MqttClient> {
return mqttClient
}
// Use MQTT_BROKER_HOST from environment, default to localhost
const brokerHost = process.env.MQTT_BROKER_HOST || '127.0.0.1'
const brokerPort = process.env.MQTT_BROKER_PORT || '1883'
// Use TESTS_MQTT_BROKER_HOST from environment, default to localhost
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
const brokerPort = process.env.TESTS_MQTT_BROKER_PORT || '1883'
const brokerUrl = `mqtt://${brokerHost}:${brokerPort}`
console.log(`Connecting to MQTT broker at ${brokerUrl}`)
return new Promise((resolve, reject) => {
console.log('Connecting to MQTT broker at mqtt://127.0.0.1:1883...')
const client = mqtt.connect(brokerUrl, {
username: '',
password: '',
+8 -1
View File
@@ -11,7 +11,14 @@ function startServer(): Promise<mqtt.MqttClient> {
function connectMqtt(): Promise<mqtt.MqttClient> {
return new Promise(resolve => {
const client = mqtt.connect('mqtt://127.0.0.1:1883', {
// Use TESTS_MQTT_BROKER_HOST from environment, default to localhost
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
const brokerPort = process.env.TESTS_MQTT_BROKER_PORT || '1883'
const brokerUrl = `mqtt://${brokerHost}:${brokerPort}`
console.log(`Connecting to MQTT broker at ${brokerUrl}`)
const client = mqtt.connect(brokerUrl, {
username: '',
password: '',
})
+3 -1
View File
@@ -23,8 +23,10 @@ export interface MockSparkplugClient {
}
const sample = (function () {
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
const brokerPort = process.env.TESTS_MQTT_BROKER_PORT || '1883'
let config = {
serverUrl: 'tcp://127.0.0.1:1883',
serverUrl: `tcp://${brokerHost}:${brokerPort}`,
username: '',
password: '',
groupId: 'Sparkplug Devices',
+3 -3
View File
@@ -32,7 +32,7 @@ import type { MqttClient } from 'mqtt'
* - Handle MQTT asynchronous operations properly
*
* Prerequisites:
* - MQTT broker running (default: localhost:1883, configurable via MQTT_BROKER_HOST and MQTT_BROKER_PORT)
* - MQTT broker running (default: localhost:1883, configurable via TESTS_MQTT_BROKER_HOST and TESTS_MQTT_BROKER_PORT)
* - Application built with `yarn build`
*/
// tslint:disable:only-arrow-functions ter-prefer-arrow-callback no-unused-expression
@@ -125,8 +125,8 @@ describe('MQTT Explorer Comprehensive UI Tests', function () {
page = await electronApp.firstWindow({ timeout: 30000 })
await page.locator('//label[contains(text(), "Host")]/..//input').waitFor({ timeout: 10000 })
// Use MQTT_BROKER_HOST from environment, default to localhost
const brokerHost = process.env.MQTT_BROKER_HOST || '127.0.0.1'
// Use TESTS_MQTT_BROKER_HOST from environment, default to localhost
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
console.log(`Connecting to MQTT broker at ${brokerHost}...`)
await connectTo(brokerHost, page)
await sleep(3000) // Give time for all topics to load
+1 -1
View File
@@ -134,7 +134,7 @@ describe('MQTT Explorer UI Tests', function () {
}
console.log('Connecting to MQTT broker...')
const brokerHost = process.env.MQTT_BROKER_HOST || '127.0.0.1'
const brokerHost = process.env.TESTS_MQTT_BROKER_HOST || '127.0.0.1'
await connectTo(brokerHost, page)
await sleep(3000) // Give time for topics to load
console.log('Setup complete')
+1
View File
@@ -28,6 +28,7 @@
"src/spec/ui-tests-comprehensive.spec.ts",
"src/spec/expandTopic.spec.ts",
"src/spec/security-tests.spec.ts",
"src/spec/SceneBuilder.spec.ts",
"scripts/*.ts"
],
"exclude": ["node_modules"]