# PSAMS/PTP Delay Viewer — Project Documentation
Generated: 2026-04-30 22:08:52 UTC  

This document provides full context for AI agents interacting with
the Delay Viewer system. For live data and statistics, use `/ai`.

## AI Endpoints

| Endpoint | Method | Description |
|----------|--------|-------------|
| `/ai` | GET | Live data report: stats, anomalies, CAISO events, trends |
| `/ai/docs` | GET | This document: project architecture, API, formats |
| `/ai/reports` | GET | Read reports submitted by AI agents |
| `/ai/reports` | POST | Submit a report (JSON: agent_id, content, tags) |

## 1. Project Overview

**Goal**: Achieve PTP (IEEE 1588) synchronization ≤100 µs on PSAMS ECU1370
embedded devices, even under CAISO event load every 30 seconds.

The **PSAMS** (Power System Analogue Measurement System) devices are
embedded ARM boards (NXP i.MX8MQ, Debian Trixie) that act as PTP
Boundary Clocks between a GPS Grand Master (Tekron TTM01-G) and
downstream microcontrollers that measure electrical grid signals.

**CAISO events** are relay pulses triggered simultaneously on multiple
boards via SPI. Comparing trigger timestamps (|Δ|) measures cross-board
synchronization quality.

## 2. Architecture

```
GPS → Tekron (GM) → Switch → eth2 (HW TS) → ptp4l (slave)
  → phc2sys → CLOCK_REALTIME → eth0 (SW TS) → ptp4l (master)
  → Switch → µC/HEM → DSP → PPS → Oscilloscope (peke)
                                     └→ PPS → FX2 Logic Analyzer (peke) [ECU-1270]
```

### Devices

| Device | IP | Role | Notes |
|--------|-----|------|-------|
| peke | 192.168.204.134 | Oscilloscope + web server | R&S RTB2004, data at /var/log/osciloscope/ |
| PS1 | 192.168.204.51 | PSAMS1 (PTP BC, RAMA A) | Kernel 6.6.71, all SW optimizations |
| PS5 | 192.168.204.118 | PSAMS5 (PTP BC, RAMA B) | Kernel 6.1.55, µC firmware issues |
| Tekron | 192.168.204.115 | GPS Grand Master | TTM01-G, PTP L2 E2E |

### Network interfaces per board

| Interface | Chip | HW Timestamps | Use |
|-----------|------|---------------|-----|
| eth2 | FEC + RTL8211F | ✅ Yes | PTP slave ← Tekron |
| eth0 | RTL8119i (r8169) | ❌ No | PTP master → µC |
| eth1 | RTL8119i (r8169) | ❌ No | Spare |

## 3. Delay Viewer Web App

**Stack**: FastAPI + uvicorn (backend) · Plotly.js 2.35.2 (frontend)

### Access

| Access | URL |
|--------|-----|
| Public (Tailscale Funnel) | https://portatil-pequenyo-power.tailaae1cf.ts.net/ |
| LAN (WireGuard) | http://10.2.255.79:8090/ |
| LAN (direct) | http://192.168.204.134:8090/ |

### Features

- **scattergl** GPU-accelerated rendering of 100K+ points
- **LTTB** progressive downsampling (adapts to time window)
- **Dark/Light themes** with localStorage persistence
- **Zoom-persistent auto-refresh** (5s to 5min)
- **Zoom-aware stats bar**: µ, σ, min, max recalculated for visible range
- **Auto-unit scaling**: ns/µs/ms based on data magnitude
- **CAISO overlay**: |Δ| scatter points (green ≤50µs, yellow ≤100µs, red >100µs)
- **Unpaired markers**: ✕ green (only PS1) / ✕ blue (only PS5)
- **Collaborative annotations**: double-click to create timestamped notes
- **Channel labels**: renamable trace names via annotations
- **CSV download**: modal with time range selector

## 4. API Reference

Base URL: `http://<host>:8090`

| Endpoint | Method | Description | Key params |
|----------|--------|-------------|------------|
| `/api/data` | GET | Delay data with LTTB | `last_secs`, `max_points` |
| `/api/info` | GET | Server info and file stats | — |
| `/api/files` | GET | List available CSVs | — |
| `/api/export` | GET | Download CSV | `filename`, `last_secs` |
| `/api/tail` | GET | Last N lines of latest CSV | `n` |
| `/api/notes` | GET | List all annotations | — |
| `/api/notes` | POST | Create annotation | JSON: `text`, `timestamp_s`, `color` |
| `/api/notes/{id}` | PUT | Update annotation | JSON fields to update |
| `/api/notes/{id}` | DELETE | Delete annotation | — |
| `/api/caiso_events` | GET | CAISO triggers + pair matching | `force` |
| `/api/collector/start` | POST | Start oscilloscope collector | — |
| `/api/collector/stop` | POST | Stop oscilloscope collector | — |
| `/api/collector/status` | GET | Collector status | — |
| `/api/la` | GET | Logic Analyzer PPS delay data | `last_secs`, `max_points`, `channels` |
| `/ai` | GET | Live data report (Markdown) | `window`, `since`, `raw_samples` |
| `/ai/docs` | GET | Project documentation (this) | — |
| `/ai/reports` | GET | Agent reports | `agent_id`, `since`, `tag`, `limit` |
| `/ai/reports` | POST | Submit agent report | JSON body |

## 5. Data Formats

### Oscilloscope CSV (delay_log_*.csv)

```csv
timestamp_s,delay1_ns,delay2_ns,delay3_ns
1772000000.123,45000,-102000,83000
```

| Column | Signal | Description |
|--------|--------|-------------|
| delay1_ns | DELAY1 (d1, green) | PS1 DSP PPS vs Tekron |
| delay2_ns | DELAY2 (d2, orange) | PS5 µC PPS vs Tekron |
| delay3_ns | DELAY3 (d3, blue) | PS5 DSP PPS vs Tekron |

Values are in **nanoseconds**. The web app auto-scales to µs or ms.

### Logic Analyzer CSV (la_delays.csv)

```csv
timestamp_s,d2_ns,d3_ns,d5_ns,d6_ns
1775124600.000,2310,7182,2184,6762
```

| Column | Signal | Description |
|--------|--------|-------------|
| d2_ns | µC TOP (1270-1) | ECU-1270-1 microcontroller PPS vs Tekron |
| d3_ns | DSP TOP (1270-1) | ECU-1270-1 DSP PPS vs Tekron |
| d5_ns | µC BOT (1270-2) | ECU-1270-2 microcontroller PPS vs Tekron |
| d6_ns | DSP BOT (1270-2) | ECU-1270-2 DSP PPS vs Tekron |

Values in **nanoseconds**, 42ns resolution (24 MHz sample rate).
Timestamps are PPS-snapped to exact integer seconds (GPS-disciplined Tekron PPS as 1 Hz reference).
One row per PPS (~35 bytes/s → ~3 MB/day). Written by la_collector.py running on peke.

### CAISO event CSV (from PSAMS boards)

- 2 metadata lines + CSV header
- Column `Trigger Mark` = `"Trigger"` marks the event instant
- Column `Time nsecs` = epoch in nanoseconds

### CAISO pairs CSV (output of caiso analyze)

```csv
file_a,file_b,trigger_ns_a,trigger_ns_b,delta_ns,delta_us
```

### /api/data response (JSON)

```json
{
  "traces": [{"name":"DELAY1","x":[epoch_s,...],"y":[val,...],
              "color":"#2ecc71","points_sent":N}],
  "stats": {"DELAY1":{"mean":...,"std":...,"min":...,"max":...,"count":N}},
  "unit": "µs", "total_rows": N, "t_min": epoch, "t_max": epoch,
  "title": "Delay (µs)"
}
```

## 6. CAISO Event System

### SPI relay commands (ADAM module on boards)

```bash
# ON
printf "\x41\x44\x41\x4D\x01\x00\x01\x13\x28\x01\x00\x09\x00\x11\x00\x00\x01\x1B\x00" > /dev/spidev0.0
# OFF
printf "\x41\x44\x41\x4D\x01\x00\x01\x13\x28\x01\x00\x09\x00\x11\x00\x00\x00\x1A\x00" > /dev/spidev0.0
```

### Pair matching

- Window: 5 ms (configurable via `CAISO_PAIR_WINDOW_NS`)
- Algorithm: O(n+m) sorted merge on epoch-ns timestamps
- Cache: persistent JSON on disk + 5 min in-memory TTL
- SSH incremental: only fetches new triggers since last known

### Services on boards

- `pe-trigger-caiso-event.service` — continuous events (0.25s pulse, 60s interval)
- `caiso-relay` — ad-hoc configurable tool: `caiso-relay -p 0.5 -i 30 -n 20`

## 7. Recent Changes (Delay Viewer)

| Date | Change |
|------|--------|
| 2026-04-02 | Logic Analyzer (FX2) real-time pipeline: 24 MHz, 42ns resolution, PPS-snapped timestamps |
| 2026-04-02 | LA section in `/ai` report + machine-readable metrics |
| 2026-04-02 | `/api/la` endpoint for LA data with LTTB downsampling |
| 2026-03-02 | `/ai/docs` and `/ai/reports` endpoints for multi-agent collaboration |
| 2026-03-02 | Zoom-aware stats bar (min/max/mean/std update on zoom) |
| 2026-03-02 | PS5 IP updated .128→.121 (DHCP reassignment) |
| 2026-03-02 | CAISO unpaired markers: scatter SVG, positioned at y-range bottom |
| 2026-03-02 | CAISO time range filtering for visible zoom window |
| 2026-03-02 | pe-delay-collector deployed as systemd service |
| 2026-03-01 | `/ai` endpoint: full Markdown context report for LLMs |
| 2026-03-01 | Tailscale Funnel for public HTTPS access |
| 2026-03-01 | CAISO overlay with pair matching and incremental cache |
| 2026-02-28 | Collaborative annotations with channel labels |

## 8. Agent Reports System

AI agents can submit and read reports via `/ai/reports`.

### Submit a report (POST /ai/reports)

```json
{
  "agent_id": "copilot-jguillo-vscode",
  "agent_name": "GitHub Copilot (jguillo session)",
  "content": "Observed delay spike at 14:30 UTC...",
  "tags": ["anomaly", "delay1", "investigation"]
}
```

Required fields: `agent_id`, `content`.
Optional: `agent_name`, `tags` (list of strings).

### Read reports (GET /ai/reports)

| Param | Type | Description |
|-------|------|-------------|
| `agent_id` | string | Filter by agent |
| `since` | float | Only reports after this epoch |
| `tag` | string | Filter by tag |
| `limit` | int | Max reports (default 50) |

### Guidelines for agents

- Use a **stable `agent_id`** across sessions (e.g., `copilot-<user>-<tool>`)
- Include **`agent_name`** for human-readable identification
- Use **`tags`** for categorization: `anomaly`, `investigation`, `config-change`,
  `experiment`, `observation`, `question`, `recommendation`
- Keep content **concise but complete**: include epochs, values, conclusions
- Reports are **append-only** — no editing or deletion via API

## 9. Quick Start for AI Agents

1. **Understand the system**: Read this document (`/ai/docs`)
2. **Get live data**: Fetch `/ai` or `/ai?window=24h` for current state
3. **Check for anomalies**: Look at the Anomalies section in `/ai`
4. **Read other agents' reports**: `GET /ai/reports`
5. **Submit your findings**: `POST /ai/reports` with your observations
6. **Ask the user**: For actions requiring SSH/board access, instruct the user

### Example workflow

```
1. fetch /ai/docs                    → understand project
2. fetch /ai?window=6h               → get recent data
3. GET  /ai/reports?since=<6h_ago>   → see what others found
4. POST /ai/reports                  → share your analysis
```

## 10. Logic Analyzer (FX2 — ECU-1270)

A **WeAct LogicAnalyzerV1** (~5€, Cypress CY7C68013A FX2LP) connected
to peke captures PPS signals from 2× ECU-1270 boards at 24 MHz (42ns resolution).

### Pipeline Architecture

```
sigrok-cli (24 MHz binary) → pipe → la_collector.py (numpy) → la_delays.csv
     24 MB/s USB stream          │      973 MB/s processing     ~35 bytes/s
                                  │                                   │
                              discarded                     Delay Viewer /api/la
```

**Key insight**: 24 MB/s of raw samples flow through a Unix pipe and are
processed in-memory with numpy vectorized edge detection. Only 1 line of
CSV per second (~35 bytes) persists to disk. No intermediate storage.

### Channels

| Pin | Signal | Description |
|-----|--------|-------------|
| D0 | Tekron GM PPS | Reference 1 Hz (GPS), 50ms positive pulse |
| D2 | µC TOP (1270-1) | ECU-1270-1 microcontroller PPS |
| D3 | DSP TOP (1270-1) | ECU-1270-1 DSP PPS |
| D5 | µC BOT (1270-2) | ECU-1270-2 microcontroller PPS |
| D6 | DSP BOT (1270-2) | ECU-1270-2 DSP PPS |

### Timestamp Strategy (PPS-snapping)

The FX2 oscillator has ~0.8% drift. Instead of using wall-clock or
sample-counting timestamps, each PPS is assigned an integer-second
timestamp by snapping to the nearest second:

```
raw_ts = chunk_wall_time - (samples_to_chunk_end × 42ns)
ts = round(raw_ts)  →  exact integer second
```

The Tekron PPS fires at exact integer seconds (GPS-disciplined),
so `round()` with ~5ms error always snaps correctly (needs >500ms to fail).
Each PPS is independently synchronized — no drift accumulation.

### Resource Footprint

| Resource | Value |
|----------|-------|
| USB throughput | 24 MB/s (USB 2.0 HS) |
| RAM (Python) | ~65 MB |
| CPU | ~7% (one core) |
| Disk write | ~35 bytes/s (1 CSV line per PPS) |
| Disk per day | ~3 MB |
| Binary data on disk | 0 bytes (all via pipe) |

### Files

| File | Location | Description |
|------|----------|-------------|
| la_collector.py | peke:/tmp/ (runtime) · git:delay_viewer_v2/ | Numpy collector |
| la_delays.csv | peke:/var/log/osciloscope/ | Output CSV (only persistent data) |
| la_state.json | peke:/var/log/osciloscope/ | Collector state for webapp |
| la_start.sh | peke:/tmp/ | tmux launcher script |

### Operations

```bash
# Start (tmux session 'la' on peke)
ssh peke 'tmux new-session -d -s la "sigrok-cli -d fx2lafw \\
    --config samplerate=24m --channels D0,D2,D3,D5,D6 \\
    --continuous --output-format binary 2>/tmp/la_sigrok.log \\
    | python3 -u /tmp/la_collector.py --rate 24 \\
    2>&1 | tee /tmp/la_collector.log"'

# Stop
ssh peke 'tmux kill-session -t la'

# Status
curl http://10.2.255.79:8090/api/la?last_secs=60
curl http://10.2.255.79:8090/ai | grep -A 20 'Logic Analyzer'
```