Add VoiceNotes firmware and CDM customer service SQL project
VoiceNotes is an Arduino sketch for the Waveshare ESP32-C6-LCD-0.85 that records meetings and voice memos to microSD and turns them into notes: - Records 16 kHz mono WAV (up to 4 h) from the ES7210 dual mics, with a boosted preamp and soft-limited digital gain - Transcribes via an OpenAI-compatible speech-to-text API in ~10 min parts cut at quiet points, with retries and resume after failures - Summarizes the transcript with Claude into meeting notes (decisions, action items, topics with timestamps), streamed to and from the card - Copies the notes to Nextcloud over WebDAV - Shows recording/processing state on the 0.85" LCD and the LED ring - Works around the board's SD card failing multi-sector reads Credentials live in VoiceNotes/secrets.h, which is git-ignored; see secrets.example.h. Also adds the Microsoft CDM customer service SQL implementation (DDL, sample data, reporting queries and docs). Its README is renamed to CDM_README.md so the repository README stays at the top level. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Ravi Reddy
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
ef38a502d8
commit
3b371cdb28
@@ -0,0 +1,2 @@
|
||||
secrets.h
|
||||
build/
|
||||
@@ -0,0 +1,149 @@
|
||||
# VoiceNotes — voice recorder + AI notes for the Waveshare ESP32-C6-LCD-0.85
|
||||
|
||||
Record meetings (up to 4 hours) or voice memos to the microSD card. Then, with one
|
||||
button press, the device transcribes the newest recording and has Claude turn it into
|
||||
meeting notes: decisions, action items and topics. The notes are saved on the card.
|
||||
|
||||
## Buttons
|
||||
|
||||
The board has **two** usable buttons (BOOT and PLUS). The PWR key is wired to the
|
||||
power-switch chip, so the firmware can't read it. "Button 3" is a long-press:
|
||||
|
||||
| Action | Press | What happens |
|
||||
|---|---|---|
|
||||
| Button 1 – record | **PLUS** click | Starts a new recording |
|
||||
| Button 2 – stop | **BOOT** click | Stops and saves the WAV file |
|
||||
| Button 3 – AI notes | **PLUS** hold 1 s | Transcribes the latest recording and writes a summary |
|
||||
|
||||
If you'd rather have a real third button, wire a push-button from a free GPIO to
|
||||
GND and set `PIN_BTN_EXTRA` in `config.h`.
|
||||
|
||||
## Files on the microSD card
|
||||
|
||||
```
|
||||
/recordings/REC_2026-10-02_14-30-05.wav 16 kHz mono 16-bit WAV
|
||||
/recordings/last.txt remembers the newest recording across reboots
|
||||
/notes/REC_2026-10-02_14-30-05_transcript.txt transcript, with [hh:mm:ss] markers every ~10 min
|
||||
/notes/REC_2026-10-02_14-30-05_transcript.progress only while a transcription is unfinished (used to resume)
|
||||
/notes/REC_2026-10-02_14-30-05_summary.md Claude's notes: summary, decisions, action items, topics
|
||||
```
|
||||
|
||||
The timestamp comes from NTP over Wi-Fi (the ESP32-C6 has no battery-backed
|
||||
clock). If Wi-Fi isn't up when you record, the file is named `REC_nodate_xxxxxxxx.wav`.
|
||||
|
||||
## Nextcloud
|
||||
|
||||
When the notes are ready, the summary and transcript are also copied to your Nextcloud over
|
||||
WebDAV, into `NEXTCLOUD_FOLDER` (the folder is created if it doesn't exist). Set it up in `secrets.h`:
|
||||
|
||||
```c
|
||||
#define NEXTCLOUD_HOST "nc.example.com"
|
||||
#define NEXTCLOUD_USER "you@example.com"
|
||||
#define NEXTCLOUD_PASSWORD "xxxxx-xxxxx-xxxxx-xxxxx-xxxxx" // an app password
|
||||
#define NEXTCLOUD_FOLDER "secondbrain"
|
||||
```
|
||||
|
||||
Use an **app password**, not your login password. Create one in Nextcloud under
|
||||
**Settings → Security → Devices & sessions → Create new app password**. It works even with
|
||||
two-factor authentication on, and you can revoke it without changing your main password.
|
||||
|
||||
If the copy fails, the notes are still on the SD card. Holding PLUS again retries only the
|
||||
Nextcloud step; it doesn't redo the transcription or the summary. To regenerate a summary, delete
|
||||
its `_summary.md` from the card first.
|
||||
|
||||
## What the screen and LED ring show
|
||||
|
||||
| State | Screen | LED ring |
|
||||
|---|---|---|
|
||||
| Starting | Mic logo, "Connecting Wi-Fi…" | Amber spinner |
|
||||
| Ready | "Ready", time of the last recording | Slow blue breathing (amber if Wi-Fi is down) |
|
||||
| Recording | Blinking ● REC, timer (h:mm:ss after an hour), scrolling level meter | Red VU meter that moves with your voice |
|
||||
| Saved | ✓, date/time, length and size | Green flash |
|
||||
| Uploading | Progress bar, "Part 3 of 12" and % | Blue ring fills up with overall progress |
|
||||
| Transcribing | Cyan spinner, part number | Cyan spinning comet |
|
||||
| Summarizing | Purple spinner, characters received | Two purple comets |
|
||||
| Saving to Nextcloud | Green spinner | Green spinning comet |
|
||||
| Done | ✓ "Notes saved", file name or "Also saved to Nextcloud" | Green pulse |
|
||||
| Error | ✗ and the reason | Red blinking |
|
||||
| No microSD | Red card icon | Slow red blink |
|
||||
|
||||
The status bar shows the clock, the SD card, and Wi-Fi bars.
|
||||
|
||||
## Setup
|
||||
|
||||
1. **Arduino IDE 2.x** with the **esp32 core 3.x** by Espressif (Boards Manager).
|
||||
2. Install these libraries from Library Manager:
|
||||
*GFX Library for Arduino* (moononournation), *FastLED*, *OneButton*, *U8g2*, *ArduinoJson* (v7).
|
||||
3. Copy `secrets.example.h` to `secrets.h` and fill in:
|
||||
- your Wi-Fi SSID and password, and your timezone,
|
||||
- a speech-to-text API key (`STT_*`). Any OpenAI-compatible
|
||||
`/audio/transcriptions` endpoint works: OpenAI (`whisper-1`, `gpt-4o-transcribe`) or Groq.
|
||||
- an Anthropic API key for the summary (`ANTHROPIC_API_KEY`).
|
||||
4. Board settings (**Tools** menu):
|
||||
- Board: **ESP32C6 Dev Module**
|
||||
- USB CDC On Boot: **Enabled** (so Serial Monitor works over USB-C)
|
||||
- Flash Size: **8MB**
|
||||
- Partition Scheme: **8M with spiffs (3MB APP/1.5MB SPIFFS)**
|
||||
5. Put a **FAT32** microSD card in the slot, then upload.
|
||||
|
||||
From the command line:
|
||||
|
||||
```bash
|
||||
arduino-cli compile --upload -p /dev/cu.usbmodem* -b esp32:esp32:esp32c6:CDCOnBoot=cdc,FlashSize=8M,PartitionScheme=default_8MB .
|
||||
```
|
||||
|
||||
Open the Serial Monitor at 115200 baud to see state changes and any API errors. You can also send:
|
||||
`p` (same as holding PLUS), `n` (print the latest transcript and summary), `d` (list recordings),
|
||||
`s` (SD read test), and `u<file name>` (choose which recording `p` and `n` use).
|
||||
|
||||
## How it works
|
||||
|
||||
| File | Role |
|
||||
|---|---|
|
||||
| `VoiceNotes.ino` | Setup, buttons, state machine |
|
||||
| `audio.cpp` | ES7210 mic codec setup over I2C, I2S capture task, WAV writer |
|
||||
| `cloud.cpp` | HTTPS: streams the WAV to speech-to-text, then streams Claude's reply into `summary.md` |
|
||||
| `ui.cpp` | 128×128 screens (drawn off-screen, then pushed to the LCD) and LED ring animations |
|
||||
| `config.h` | Pin map and limits |
|
||||
|
||||
- **Audio:** the two onboard mics go through the ES7210 ADC at 16 kHz and are mixed to mono.
|
||||
A high-priority task fills a 2-second RAM buffer, and the main loop writes that buffer to the card.
|
||||
This way a slow SD write can't make the recording skip.
|
||||
- **Shared SPI bus:** the LCD and the microSD use the same SPI pins, so every access takes `spiMutex`.
|
||||
- **Long recordings:** the WAV header is updated every 10 s, so a power cut mid-meeting still leaves a
|
||||
playable file. Recording stops at 4 hours (`MAX_RECORD_SECONDS`), or earlier if the card is nearly full.
|
||||
- **Transcription in parts:** speech-to-text APIs accept at most 25 MB per file, so the recording is sent
|
||||
in ~10-minute parts (`STT_SEGMENT_SECONDS`). Each part is cut at the quietest moment in the 15 s before
|
||||
the target, so no word is split. The end of the previous part's text is sent as a hint, so names and
|
||||
spellings stay consistent. Each part is streamed straight from the card.
|
||||
- **Resume:** each part is retried up to 3 times. If it still fails (for example, Wi-Fi drops), the
|
||||
finished parts are kept, and holding PLUS again continues from where it stopped.
|
||||
- **SD reads** use 512-byte (single-sector) reads. On this board the microSD fails multi-sector reads.
|
||||
- **Summary:** `POST /v1/messages` to `claude-opus-5-5` with streaming. A 2-hour transcript (~100 KB) is too
|
||||
big for the board's RAM, so the request body is streamed from the card, and the reply is written to the
|
||||
card as it arrives.
|
||||
The request sets `fallbacks: "default"` (beta `server-side-fallback-2026-07-01`). If the model declines,
|
||||
the API retries on a fallback model instead of failing.
|
||||
- If the transcript was saved but the summary step failed, a long-press again reuses the
|
||||
saved transcript and doesn't upload the audio a second time.
|
||||
|
||||
## Limits and notes
|
||||
|
||||
- Recordings use about **115 MB per hour** (16 kHz mono WAV). A 2-hour meeting is ~230 MB.
|
||||
- Processing a 2-hour meeting takes roughly **15–25 minutes**: 12 parts, each about a minute to upload at
|
||||
~300 KB/s plus 20–40 s to transcribe, then a minute or two for Claude. The device can't record while it's
|
||||
processing.
|
||||
- For multi-hour meetings, run the device from USB power. A small battery won't last.
|
||||
- Privacy: your audio goes to the speech-to-text provider, and the transcript goes to Anthropic.
|
||||
- TLS certificates are verified with the ESP32 core's built-in CA bundle.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Fix |
|
||||
|---|---|
|
||||
| LED colours wrong (red shows green) | Set `LED_COLOR_ORDER` to `GRB` in `config.h` |
|
||||
| Recordings are quiet or noisy | Change `MIC_GAIN_STEP` in `config.h` (10 = 30 dB) |
|
||||
| "Microphone init failed" | The ES7210 didn't answer at I2C 0x40. Power-cycle the board |
|
||||
| `STT HTTP 401` / `Claude HTTP 401` | Check the API keys in `secrets.h` |
|
||||
| File names say `nodate` | Wi-Fi/NTP wasn't ready yet. Wait for the clock in the status bar |
|
||||
| Serial warns that audio was dropped | Use a faster (Class 10 / U1) microSD card |
|
||||
@@ -0,0 +1,298 @@
|
||||
// VoiceNotes - voice recorder + AI notes for the Waveshare ESP32-C6-LCD-0.85
|
||||
//
|
||||
// PLUS click -> start recording (Button 1)
|
||||
// BOOT click -> stop & save to microSD (Button 2)
|
||||
// PLUS long-press -> transcribe the latest recording, summarize it with
|
||||
// Claude, and save both to microSD (Button 3)
|
||||
//
|
||||
// Recordings: /recordings/REC_YYYY-MM-DD_HH-MM-SS.wav (16 kHz mono WAV)
|
||||
// Notes: /notes/REC_..._transcript.txt and /notes/REC_..._summary.md
|
||||
|
||||
#include <OneButton.h>
|
||||
#include <fcntl.h>
|
||||
#include <unistd.h>
|
||||
#include <SD.h>
|
||||
#include <SPI.h>
|
||||
#include <WiFi.h>
|
||||
#include "app_state.h"
|
||||
#include "audio.h"
|
||||
#include "cloud.h"
|
||||
#include "config.h"
|
||||
#include "secrets.h"
|
||||
#include "ui.h"
|
||||
|
||||
AppStatus app;
|
||||
SemaphoreHandle_t spiMutex;
|
||||
|
||||
static OneButton btnPlus(PIN_BTN_PLUS, true, true);
|
||||
static OneButton btnBoot(PIN_BTN_BOOT, true, true);
|
||||
#if PIN_BTN_EXTRA >= 0
|
||||
static OneButton btnExtra(PIN_BTN_EXTRA, true, true);
|
||||
#endif
|
||||
|
||||
void setState(AppState s, const char *detail) {
|
||||
if (detail) strlcpy(app.detail, detail, sizeof(app.detail));
|
||||
app.stateSince = millis();
|
||||
app.state = s;
|
||||
static const char *names[] = {"BOOT", "NO_SD", "IDLE", "RECORDING", "SAVED", "UPLOADING",
|
||||
"TRANSCRIBING", "SUMMARIZING", "SYNCING", "DONE", "ERROR"};
|
||||
Serial.printf("[state] %s %s\n", names[s], detail ? detail : "");
|
||||
}
|
||||
|
||||
bool timeIsValid() { return time(nullptr) > 1700000000; } // NTP has synced
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
static bool mountSD() {
|
||||
SpiLock lock;
|
||||
if (!SD.begin(PIN_SD_CS, SPI, SD_SPI_HZ) || SD.cardType() == CARD_NONE) {
|
||||
SD.end();
|
||||
return false;
|
||||
}
|
||||
SD.mkdir(DIR_RECORDINGS);
|
||||
SD.mkdir(DIR_NOTES);
|
||||
File f = SD.open(LAST_REC_FILE, FILE_READ);
|
||||
if (f) {
|
||||
String p = f.readStringUntil('\n');
|
||||
p.trim();
|
||||
f.close();
|
||||
if (SD.exists(p)) strlcpy(app.lastRecording, p.c_str(), sizeof(app.lastRecording));
|
||||
}
|
||||
Serial.printf("SD mounted, %llu MB free\n", (SD.totalBytes() - SD.usedBytes()) / 1048576);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool canStartSomething() {
|
||||
AppState s = app.state;
|
||||
return s == ST_IDLE || s == ST_SAVED || s == ST_DONE || s == ST_ERROR;
|
||||
}
|
||||
|
||||
// Button 1
|
||||
static void startRecording() {
|
||||
if (!canStartSomething()) return;
|
||||
char path[64];
|
||||
if (timeIsValid()) {
|
||||
time_t now = time(nullptr);
|
||||
strftime(path, sizeof(path), DIR_RECORDINGS "/REC_%Y-%m-%d_%H-%M-%S.wav", localtime(&now));
|
||||
} else {
|
||||
// No network time yet (the C6 has no battery-backed clock).
|
||||
snprintf(path, sizeof(path), DIR_RECORDINGS "/REC_nodate_%08lx.wav", (unsigned long)esp_random());
|
||||
}
|
||||
// Limit the length to what fits on the card, keeping 50 MB spare.
|
||||
uint64_t freeBytes;
|
||||
{
|
||||
SpiLock lock;
|
||||
freeBytes = SD.totalBytes() - SD.usedBytes();
|
||||
}
|
||||
uint64_t fitSeconds = freeBytes > 50ULL * 1048576 ? (freeBytes - 50ULL * 1048576) / (SAMPLE_RATE * 2) : 0;
|
||||
if (fitSeconds < 60) {
|
||||
setState(ST_ERROR, "microSD card is full");
|
||||
return;
|
||||
}
|
||||
app.maxSeconds = (uint32_t)min((uint64_t)MAX_RECORD_SECONDS, fitSeconds);
|
||||
if (!recorderStart(path)) {
|
||||
setState(ST_ERROR, "Can't create file on microSD");
|
||||
return;
|
||||
}
|
||||
strlcpy(app.lastRecording, path, sizeof(app.lastRecording));
|
||||
setState(ST_RECORDING, path);
|
||||
}
|
||||
|
||||
// Button 2
|
||||
static void stopRecording() {
|
||||
if (app.state != ST_RECORDING) return;
|
||||
recorderStop();
|
||||
{
|
||||
SpiLock lock;
|
||||
File f = SD.open(LAST_REC_FILE, FILE_WRITE);
|
||||
if (f) {
|
||||
f.println(app.lastRecording);
|
||||
f.close();
|
||||
}
|
||||
}
|
||||
if (app.droppedBytes)
|
||||
Serial.printf("Warning: %lu bytes of audio dropped (slow SD card?)\n", app.droppedBytes);
|
||||
setState(ST_SAVED, strrchr(app.lastRecording, '/') + 1);
|
||||
}
|
||||
|
||||
// Button 3
|
||||
static void processLatest() {
|
||||
if (!canStartSomething()) return;
|
||||
if (!app.lastRecording[0]) {
|
||||
setState(ST_ERROR, "No recording yet - press PLUS to record");
|
||||
return;
|
||||
}
|
||||
if (WiFi.status() != WL_CONNECTED) {
|
||||
setState(ST_ERROR, "Wi-Fi not connected - check secrets.h");
|
||||
return;
|
||||
}
|
||||
if (!cloudStart(app.lastRecording)) setState(ST_ERROR, "Couldn't start network task");
|
||||
}
|
||||
|
||||
// Runs on every (re)connect, after DHCP, so the DNS override survives lease renewals.
|
||||
static void onGotIp(arduino_event_id_t, arduino_event_info_t) {
|
||||
Serial.printf("[wifi] IP %s, DHCP DNS %s / %s\n", WiFi.localIP().toString().c_str(),
|
||||
WiFi.dnsIP(0).toString().c_str(), WiFi.dnsIP(1).toString().c_str());
|
||||
IPAddress d1, d2;
|
||||
if (d1.fromString(DNS_PRIMARY)) {
|
||||
if (!d2.fromString(DNS_SECONDARY)) d2 = IPAddress();
|
||||
WiFi.setDNS(d1, d2);
|
||||
Serial.printf("[wifi] using DNS %s / %s\n", WiFi.dnsIP(0).toString().c_str(), WiFi.dnsIP(1).toString().c_str());
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
void setup() {
|
||||
Serial.begin(115200);
|
||||
spiMutex = xSemaphoreCreateMutex();
|
||||
SPI.begin(PIN_SPI_SCK, PIN_SPI_MISO, PIN_SPI_MOSI); // shared by LCD and microSD
|
||||
|
||||
if (!uiBegin()) Serial.println("LCD init failed");
|
||||
setState(ST_BOOT);
|
||||
uiUpdate();
|
||||
|
||||
WiFi.onEvent(onGotIp, ARDUINO_EVENT_WIFI_STA_GOT_IP);
|
||||
WiFi.mode(WIFI_STA);
|
||||
WiFi.setAutoReconnect(true);
|
||||
WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
|
||||
configTzTime(TIMEZONE, "pool.ntp.org", "time.google.com");
|
||||
|
||||
btnPlus.attachClick(startRecording);
|
||||
btnPlus.setPressMs(BTN3_LONG_PRESS_MS);
|
||||
btnPlus.attachLongPressStart(processLatest);
|
||||
btnBoot.attachClick(stopRecording);
|
||||
#if PIN_BTN_EXTRA >= 0
|
||||
btnExtra.attachClick(processLatest);
|
||||
#endif
|
||||
|
||||
bool micOk = audioBegin();
|
||||
bool sdOk = mountSD();
|
||||
|
||||
// Give Wi-Fi + NTP up to 10 s so the first recording gets a real timestamp.
|
||||
uint32_t t0 = millis();
|
||||
while (millis() - t0 < 10000 && !timeIsValid()) {
|
||||
uiUpdate();
|
||||
delay(10);
|
||||
}
|
||||
Serial.printf("Wi-Fi %s, clock %s\n", WiFi.status() == WL_CONNECTED ? "connected" : "offline",
|
||||
timeIsValid() ? "synced" : "not set");
|
||||
|
||||
if (!micOk) setState(ST_ERROR, "Microphone init failed");
|
||||
else if (!sdOk) setState(ST_NO_SD);
|
||||
else setState(ST_IDLE);
|
||||
}
|
||||
|
||||
// Serial commands (115200 baud), handy for testing without the buttons:
|
||||
// p = transcribe + summarize the latest recording (same as holding PLUS)
|
||||
// s = read the latest recording back from the SD card and report the result
|
||||
// d = list /recordings
|
||||
// w = show Wi-Fi and DNS status
|
||||
// r / x = start / stop recording (same as PLUS / BOOT)
|
||||
// a = measure each microphone's level for 2 s
|
||||
// n = print the notes (transcript + summary) for the latest recording
|
||||
// u<file name> = choose which recording p and n use, e.g. uREC_2026-10-03_15-31-12.wav
|
||||
static void sdReadTest() {
|
||||
if (!app.lastRecording[0]) { Serial.println("[test] no recording yet"); return; }
|
||||
SpiLock lock;
|
||||
uint32_t t0 = millis();
|
||||
int fd = open((String("/sd") + app.lastRecording).c_str(), O_RDONLY);
|
||||
if (fd < 0) { Serial.printf("[test] open %s failed (errno %d)\n", app.lastRecording, errno); return; }
|
||||
static uint8_t buf[512]; // single-sector reads, see sdRead() in cloud.cpp
|
||||
size_t total = 0;
|
||||
int n;
|
||||
while ((n = read(fd, buf, sizeof(buf))) > 0) total += n;
|
||||
int e = n < 0 ? errno : 0;
|
||||
close(fd);
|
||||
Serial.printf("[test] read %u bytes of %s in %lu ms (errno %d %s)\n", (unsigned)total, app.lastRecording,
|
||||
millis() - t0, e, strerror(e));
|
||||
}
|
||||
|
||||
static void sdList() {
|
||||
SpiLock lock;
|
||||
File dir = SD.open(DIR_RECORDINGS);
|
||||
for (File f = dir.openNextFile(); f; f = dir.openNextFile())
|
||||
Serial.printf("[test] %-40s %8u bytes\n", f.name(), (unsigned)f.size());
|
||||
dir.close();
|
||||
}
|
||||
|
||||
static void printFile(const String &path) {
|
||||
SpiLock lock;
|
||||
int fd = open((String("/sd") + path).c_str(), O_RDONLY);
|
||||
if (fd < 0) { Serial.printf("[test] %s not found\n", path.c_str()); return; }
|
||||
Serial.printf("===== %s =====\n", path.c_str());
|
||||
static uint8_t buf[512];
|
||||
int n;
|
||||
while ((n = read(fd, buf, sizeof(buf))) > 0) Serial.write(buf, n);
|
||||
close(fd);
|
||||
Serial.println("\n===== end =====");
|
||||
}
|
||||
|
||||
static void printNotes() {
|
||||
if (!app.lastRecording[0]) { Serial.println("[test] no recording yet"); return; }
|
||||
String base = String(strrchr(app.lastRecording, '/') + 1);
|
||||
base.replace(".wav", "");
|
||||
printFile(String(DIR_NOTES "/") + base + "_transcript.txt");
|
||||
printFile(String(DIR_NOTES "/") + base + "_summary.md");
|
||||
}
|
||||
|
||||
static void serialCommands() {
|
||||
while (Serial.available()) {
|
||||
switch (Serial.read()) {
|
||||
case 'p': Serial.println("[test] start pipeline"); processLatest(); break;
|
||||
case 's': sdReadTest(); break;
|
||||
case 'd': sdList(); break;
|
||||
case 'n': printNotes(); break;
|
||||
case 'r': startRecording(); break;
|
||||
case 'a': audioLevelReport(2000); break;
|
||||
case 'x': stopRecording(); break;
|
||||
case 'w':
|
||||
Serial.printf("[wifi] %s, IP %s, DNS %s / %s, RSSI %d dBm\n", WiFi.isConnected() ? "connected" : "offline",
|
||||
WiFi.localIP().toString().c_str(), WiFi.dnsIP(0).toString().c_str(),
|
||||
WiFi.dnsIP(1).toString().c_str(), WiFi.RSSI());
|
||||
break;
|
||||
case 'u': {
|
||||
String name = Serial.readStringUntil('\n');
|
||||
name.trim();
|
||||
String path = String(DIR_RECORDINGS "/") + name;
|
||||
bool found;
|
||||
{
|
||||
SpiLock lock;
|
||||
found = SD.exists(path);
|
||||
}
|
||||
if (found) strlcpy(app.lastRecording, path.c_str(), sizeof(app.lastRecording));
|
||||
Serial.printf("[test] %s %s\n", found ? "using" : "not found:", path.c_str());
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void loop() {
|
||||
serialCommands();
|
||||
btnPlus.tick();
|
||||
btnBoot.tick();
|
||||
#if PIN_BTN_EXTRA >= 0
|
||||
btnExtra.tick();
|
||||
#endif
|
||||
|
||||
recorderService();
|
||||
if (app.state == ST_RECORDING && recorderSeconds() >= app.maxSeconds) {
|
||||
Serial.println("Max length reached, stopping");
|
||||
stopRecording();
|
||||
}
|
||||
|
||||
// Result screens fall back to the ready screen after a while.
|
||||
uint32_t inState = millis() - app.stateSince;
|
||||
if ((app.state == ST_SAVED && inState > 5000) ||
|
||||
((app.state == ST_DONE || app.state == ST_ERROR) && inState > 15000)) {
|
||||
setState(ST_IDLE);
|
||||
}
|
||||
|
||||
static uint32_t lastSdTry = 0;
|
||||
if (app.state == ST_NO_SD && millis() - lastSdTry > 3000) {
|
||||
lastSdTry = millis();
|
||||
if (mountSD()) setState(ST_IDLE);
|
||||
}
|
||||
|
||||
uiUpdate();
|
||||
delay(2);
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
// State shared between the main loop, the audio task and the network task.
|
||||
#pragma once
|
||||
#include <Arduino.h>
|
||||
#include <freertos/FreeRTOS.h>
|
||||
#include <freertos/semphr.h>
|
||||
|
||||
enum AppState : uint8_t {
|
||||
ST_BOOT,
|
||||
ST_NO_SD,
|
||||
ST_IDLE,
|
||||
ST_RECORDING,
|
||||
ST_SAVED,
|
||||
ST_UPLOADING, // sending the WAV to the speech-to-text service
|
||||
ST_TRANSCRIBING, // waiting for / receiving the transcript
|
||||
ST_SUMMARIZING, // Claude is writing the notes
|
||||
ST_SYNCING, // copying the notes to Nextcloud
|
||||
ST_DONE,
|
||||
ST_ERROR,
|
||||
};
|
||||
|
||||
struct AppStatus {
|
||||
volatile AppState state = ST_BOOT;
|
||||
volatile uint32_t stateSince = 0; // millis() when the state was entered
|
||||
volatile uint8_t progress = 0; // 0-100 for ST_UPLOADING
|
||||
volatile uint32_t received = 0; // bytes of transcript / summary received
|
||||
volatile float micLevel = 0; // 0..1, updated by the audio task
|
||||
volatile uint32_t recordedBytes = 0; // PCM bytes in the current recording
|
||||
volatile uint32_t droppedBytes = 0; // audio lost because the SD card stalled
|
||||
volatile uint32_t maxSeconds = 0; // recording limit, from config and free space
|
||||
volatile uint8_t partIndex = 0; // transcription part being processed (1-based)
|
||||
volatile uint8_t partCount = 0; // estimated number of parts
|
||||
char lastRecording[64] = ""; // full path of the newest .wav
|
||||
char detail[96] = ""; // file name / error text shown on screen
|
||||
};
|
||||
|
||||
extern AppStatus app;
|
||||
|
||||
// The LCD and the microSD share one SPI bus; anything touching either must hold this.
|
||||
extern SemaphoreHandle_t spiMutex;
|
||||
struct SpiLock {
|
||||
SpiLock() { xSemaphoreTake(spiMutex, portMAX_DELAY); }
|
||||
~SpiLock() { xSemaphoreGive(spiMutex); }
|
||||
};
|
||||
|
||||
void setState(AppState s, const char *detail = nullptr);
|
||||
bool timeIsValid();
|
||||
@@ -0,0 +1,244 @@
|
||||
#include "audio.h"
|
||||
#include "app_state.h"
|
||||
#include "config.h"
|
||||
#include <ESP_I2S.h>
|
||||
#include <SD.h>
|
||||
#include <Wire.h>
|
||||
#include <freertos/stream_buffer.h>
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// ES7210 4-channel mic ADC. Register sequence follows Espressif's es7210
|
||||
// driver (esp-bsp/components/es7210), configured as plain I2S slave,
|
||||
// 16-bit, TDM off, so SDOUT1 carries MIC1 (left) and MIC2 (right).
|
||||
// ---------------------------------------------------------------------------
|
||||
static bool es7210Write(uint8_t reg, uint8_t val) {
|
||||
Wire.beginTransmission(ES7210_ADDR);
|
||||
Wire.write(reg);
|
||||
Wire.write(val);
|
||||
return Wire.endTransmission() == 0;
|
||||
}
|
||||
|
||||
static bool es7210Init() {
|
||||
// MCLK = 256 x 16 kHz = 4.096 MHz. Clock coefficients from Espressif's table:
|
||||
// {4096000, 16000, ss_ds 0, adc_div 1, dll 1, doubler 1, osr 0x20, lrck 0x0100}
|
||||
const uint8_t gain = 0x10 | MIC_GAIN_STEP;
|
||||
const uint8_t seq[][2] = {
|
||||
{0x00, 0xFF}, {0x00, 0x32}, // software reset
|
||||
{0x09, 0x30}, {0x0A, 0x30}, // power-up timing
|
||||
{0x23, 0x2A}, {0x22, 0x0A}, // ADC1/2 high-pass filter
|
||||
{0x21, 0x2A}, {0x20, 0x0A}, // ADC3/4 high-pass filter
|
||||
{0x11, 0x60}, {0x12, 0x00}, // I2S format, 16-bit, TDM off
|
||||
{0x40, 0xC3}, // analog power, VMID
|
||||
{0x41, 0x70}, {0x42, 0x70}, // mic bias 2.87 V
|
||||
{0x43, gain}, {0x44, gain}, {0x45, gain}, {0x46, gain},
|
||||
{0x47, 0x08}, {0x48, 0x08}, {0x49, 0x08}, {0x4A, 0x08}, // mic power
|
||||
{0x07, 0x20}, // OSR
|
||||
{0x02, 0xC1}, // adc_div=1 | doubler<<6 | dll<<7
|
||||
{0x04, 0x01}, {0x05, 0x00}, // LRCK divider = 256
|
||||
{0x06, 0x04}, // power down DLL
|
||||
{0x4B, 0x0F}, {0x4C, 0x0F}, // mic bias, ADC, PGA on
|
||||
{0x00, 0x71}, {0x00, 0x41}, // enable
|
||||
};
|
||||
for (auto &rw : seq) {
|
||||
if (!es7210Write(rw[0], rw[1])) {
|
||||
Serial.printf("ES7210: I2C write 0x%02X failed\n", rw[0]);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Capture task: I2S -> mono PCM -> stream buffer. Runs at high priority and
|
||||
// never touches SPI, so a slow SD write can't make it miss samples. The
|
||||
// 64 KB stream buffer absorbs ~2 s of SD card latency.
|
||||
// ---------------------------------------------------------------------------
|
||||
static I2SClass i2s;
|
||||
static StreamBufferHandle_t pcmBuffer;
|
||||
static volatile bool capturing = false;
|
||||
static const size_t PCM_BUFFER_BYTES = 64 * 1024;
|
||||
|
||||
// Per-microphone level statistics for audioLevelReport().
|
||||
static volatile bool statsOn = false;
|
||||
static uint64_t statSumSq[3]; // MIC1, MIC2, recorded output
|
||||
static int32_t statPeak[3];
|
||||
static uint32_t statFrames;
|
||||
|
||||
static void audioTask(void *) {
|
||||
static int16_t stereo[512]; // 256 frames, L/R interleaved
|
||||
static int16_t mono[256];
|
||||
float level = 0;
|
||||
const float gain = powf(10.0f, DIGITAL_GAIN_DB / 20.0f);
|
||||
for (;;) {
|
||||
size_t got = i2s.readBytes((char *)stereo, sizeof(stereo));
|
||||
size_t frames = got / 4;
|
||||
if (frames == 0) continue;
|
||||
|
||||
if (statsOn) {
|
||||
for (size_t i = 0; i < frames; i++) {
|
||||
for (int ch = 0; ch < 2; ch++) {
|
||||
int32_t v = stereo[2 * i + ch];
|
||||
statSumSq[ch] += (int64_t)v * v;
|
||||
statPeak[ch] = max(statPeak[ch], abs(v));
|
||||
}
|
||||
}
|
||||
statFrames += frames;
|
||||
}
|
||||
|
||||
uint64_t sumSq = 0;
|
||||
for (size_t i = 0; i < frames; i++) {
|
||||
// Mix both mics, apply the digital gain, then a tanh soft limiter: nearly
|
||||
// linear for normal speech, rounding off peaks instead of hard clipping.
|
||||
float x = ((int32_t)stereo[2 * i] + (int32_t)stereo[2 * i + 1]) * 0.5f * gain;
|
||||
int32_t s = DIGITAL_GAIN_DB > 0 ? (int32_t)(32767.0f * tanhf(x / 32767.0f)) : (int32_t)x;
|
||||
mono[i] = (int16_t)s;
|
||||
if (statsOn) {
|
||||
statSumSq[2] += (int64_t)s * s;
|
||||
statPeak[2] = max(statPeak[2], abs(s));
|
||||
}
|
||||
sumSq += (int64_t)s * s;
|
||||
}
|
||||
|
||||
// Level meter: RMS mapped from -60..0 dBFS to 0..1, fast attack / slow decay.
|
||||
float rms = sqrtf((float)sumSq / frames);
|
||||
float db = 20.0f * log10f(rms / 32768.0f + 1e-9f);
|
||||
float target = constrain((db + 60.0f) / 60.0f, 0.0f, 1.0f);
|
||||
level = target > level ? target : level * 0.9f + target * 0.1f;
|
||||
app.micLevel = level;
|
||||
|
||||
if (capturing) {
|
||||
size_t bytes = frames * 2;
|
||||
if (xStreamBufferSpacesAvailable(pcmBuffer) >= bytes) {
|
||||
xStreamBufferSend(pcmBuffer, mono, bytes, 0);
|
||||
} else {
|
||||
app.droppedBytes += bytes;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void audioLevelReport(uint32_t ms) {
|
||||
for (int ch = 0; ch < 3; ch++) {
|
||||
statSumSq[ch] = 0;
|
||||
statPeak[ch] = 0;
|
||||
}
|
||||
statFrames = 0;
|
||||
statsOn = true;
|
||||
delay(ms);
|
||||
statsOn = false;
|
||||
delay(20);
|
||||
static const char *names[] = {"MIC1", "MIC2", "recorded"};
|
||||
for (int ch = 0; ch < 3; ch++) {
|
||||
float rms = sqrtf((float)statSumSq[ch] / max(statFrames, (uint32_t)1));
|
||||
Serial.printf("[audio] %-8s RMS %6.1f dBFS, peak %6.1f dBFS\n", names[ch], 20 * log10f(rms / 32768 + 1e-9f),
|
||||
20 * log10f(statPeak[ch] / 32768.0f + 1e-9f));
|
||||
}
|
||||
}
|
||||
|
||||
bool audioBegin() {
|
||||
pinMode(PIN_PA_CTRL, OUTPUT);
|
||||
digitalWrite(PIN_PA_CTRL, LOW); // speaker amp off; we only record
|
||||
|
||||
// Start I2S first: the ES7210 needs MCLK running while it powers up.
|
||||
i2s.setPins(PIN_I2S_BCLK, PIN_I2S_WS, -1 /* dout */, PIN_I2S_DIN, PIN_I2S_MCLK);
|
||||
if (!i2s.begin(I2S_MODE_STD, SAMPLE_RATE, I2S_DATA_BIT_WIDTH_16BIT, I2S_SLOT_MODE_STEREO)) {
|
||||
Serial.println("I2S begin failed");
|
||||
return false;
|
||||
}
|
||||
Wire.begin(PIN_I2C_SDA, PIN_I2C_SCL, 400000);
|
||||
if (!es7210Init()) return false;
|
||||
|
||||
pcmBuffer = xStreamBufferCreate(PCM_BUFFER_BYTES, 1);
|
||||
if (!pcmBuffer) return false;
|
||||
xTaskCreate(audioTask, "audio", 4096, nullptr, configMAX_PRIORITIES - 2, nullptr);
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// WAV recorder
|
||||
// ---------------------------------------------------------------------------
|
||||
static File wavFile;
|
||||
static uint32_t recStartMs = 0;
|
||||
static uint32_t lastFlushMs = 0;
|
||||
|
||||
void wavHeader(uint8_t h[44], uint32_t dataBytes) {
|
||||
const uint32_t byteRate = SAMPLE_RATE * 2;
|
||||
auto u32 = [&](int o, uint32_t v) { h[o] = v; h[o + 1] = v >> 8; h[o + 2] = v >> 16; h[o + 3] = v >> 24; };
|
||||
auto u16 = [&](int o, uint16_t v) { h[o] = v; h[o + 1] = v >> 8; };
|
||||
memcpy(h, "RIFF", 4); u32(4, 36 + dataBytes);
|
||||
memcpy(h + 8, "WAVEfmt ", 8);
|
||||
u32(16, 16); // fmt chunk size
|
||||
u16(20, 1); // PCM
|
||||
u16(22, 1); // mono
|
||||
u32(24, SAMPLE_RATE);
|
||||
u32(28, byteRate);
|
||||
u16(32, 2); // block align
|
||||
u16(34, 16); // bits per sample
|
||||
memcpy(h + 36, "data", 4); u32(40, dataBytes);
|
||||
}
|
||||
|
||||
static void writeWavHeader(File &f, uint32_t dataBytes) {
|
||||
uint8_t h[44];
|
||||
wavHeader(h, dataBytes);
|
||||
f.seek(0);
|
||||
f.write(h, sizeof(h));
|
||||
}
|
||||
|
||||
bool recorderStart(const char *path) {
|
||||
{
|
||||
SpiLock lock;
|
||||
wavFile = SD.open(path, FILE_WRITE);
|
||||
if (!wavFile) return false;
|
||||
writeWavHeader(wavFile, 0);
|
||||
}
|
||||
xStreamBufferReset(pcmBuffer);
|
||||
app.recordedBytes = 0;
|
||||
app.droppedBytes = 0;
|
||||
recStartMs = lastFlushMs = millis();
|
||||
capturing = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool recorderActive() { return capturing; }
|
||||
|
||||
uint32_t recorderSeconds() { return app.recordedBytes / (SAMPLE_RATE * 2); }
|
||||
|
||||
static void drain(bool all) {
|
||||
static uint8_t chunk[4096];
|
||||
// While recording, write only full chunks (SD cards like 512-byte multiples)
|
||||
// and cap the work per call so the UI stays responsive.
|
||||
for (int i = 0; all || i < 8; i++) {
|
||||
size_t avail = xStreamBufferBytesAvailable(pcmBuffer);
|
||||
if (avail == 0 || (!all && avail < sizeof(chunk))) break;
|
||||
size_t n = xStreamBufferReceive(pcmBuffer, chunk, sizeof(chunk), 0);
|
||||
if (n == 0) break;
|
||||
SpiLock lock;
|
||||
wavFile.write(chunk, n);
|
||||
app.recordedBytes += n;
|
||||
}
|
||||
}
|
||||
|
||||
void recorderService() {
|
||||
if (!capturing) return;
|
||||
drain(false);
|
||||
// Every 10 s, update the header's length fields and flush, so a long
|
||||
// recording interrupted by a power cut is still a valid, playable WAV.
|
||||
if (millis() - lastFlushMs > 10000) {
|
||||
SpiLock lock;
|
||||
writeWavHeader(wavFile, app.recordedBytes);
|
||||
wavFile.seek(44 + app.recordedBytes);
|
||||
wavFile.flush();
|
||||
lastFlushMs = millis();
|
||||
}
|
||||
}
|
||||
|
||||
bool recorderStop() {
|
||||
if (!capturing) return false;
|
||||
capturing = false;
|
||||
delay(40); // let the capture task finish its current block
|
||||
drain(true);
|
||||
SpiLock lock;
|
||||
writeWavHeader(wavFile, app.recordedBytes);
|
||||
wavFile.close();
|
||||
return true;
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
// Microphone capture (ES7210 over I2S) and WAV recording to the microSD card.
|
||||
#pragma once
|
||||
#include <Arduino.h>
|
||||
|
||||
bool audioBegin(); // configure codec + I2S, start the capture task
|
||||
bool recorderStart(const char *path); // open WAV file and start capturing
|
||||
void recorderService(); // call often from loop(): drains audio to SD
|
||||
bool recorderStop(); // finish header, close file
|
||||
bool recorderActive();
|
||||
uint32_t recorderSeconds();
|
||||
|
||||
// Prints each microphone's RMS and peak level (dBFS) over `ms` milliseconds.
|
||||
void audioLevelReport(uint32_t ms);
|
||||
|
||||
// Fills a 44-byte header for a 16 kHz mono 16-bit WAV with dataBytes of PCM.
|
||||
void wavHeader(uint8_t h[44], uint32_t dataBytes);
|
||||
@@ -0,0 +1,940 @@
|
||||
#include "cloud.h"
|
||||
#include "app_state.h"
|
||||
#include "audio.h"
|
||||
#include "config.h"
|
||||
#include "secrets.h"
|
||||
#include <ArduinoJson.h>
|
||||
#include <base64.h>
|
||||
#include <NetworkClientSecure.h>
|
||||
#include <SD.h>
|
||||
#include <WiFi.h>
|
||||
#include <errno.h>
|
||||
#include <fcntl.h>
|
||||
#include <sys/stat.h>
|
||||
#include <unistd.h>
|
||||
|
||||
static volatile bool busy = false;
|
||||
static char jobWav[64];
|
||||
|
||||
bool cloudBusy() { return busy; }
|
||||
|
||||
// Network diagnostics on the serial port, e.g. "[net 12.345s heap 98304] TLS connect ok"
|
||||
#define NETLOG(fmt, ...) \
|
||||
Serial.printf("[net %lu.%03lus heap %lu] " fmt "\n", millis() / 1000, millis() % 1000, \
|
||||
(unsigned long)ESP.getFreeHeap(), ##__VA_ARGS__)
|
||||
|
||||
static const char *wifiStatusName(wl_status_t s) {
|
||||
switch (s) {
|
||||
case WL_CONNECTED: return "connected";
|
||||
case WL_NO_SSID_AVAIL: return "SSID not found";
|
||||
case WL_CONNECT_FAILED: return "connect failed (wrong password?)";
|
||||
case WL_CONNECTION_LOST: return "connection lost";
|
||||
case WL_DISCONNECTED: return "disconnected";
|
||||
case WL_IDLE_STATUS: return "idle";
|
||||
default: return "unknown";
|
||||
}
|
||||
}
|
||||
|
||||
static void logTlsError(NetworkClientSecure &c, const char *what) {
|
||||
char msg[128] = "";
|
||||
int code = c.lastError(msg, sizeof(msg));
|
||||
NETLOG("%s: TLS error %d (-0x%04X) %s", what, code, code < 0 ? -code : code, msg);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Minimal HTTP/1.1 over TLS. We talk to the socket directly (rather than
|
||||
// HTTPClient) so we can stream a multi-MB WAV from the SD card, wait minutes
|
||||
// for a reply, and decode chunked / server-sent-event responses line by line.
|
||||
// ---------------------------------------------------------------------------
|
||||
static int readByteWait(NetworkClient &c, uint32_t timeoutMs) {
|
||||
uint32_t t0 = millis();
|
||||
while (!c.available()) {
|
||||
if (!c.connected() || millis() - t0 > timeoutMs) return -1;
|
||||
delay(5);
|
||||
}
|
||||
return c.read();
|
||||
}
|
||||
|
||||
static bool readRawLine(NetworkClient &c, String &line, uint32_t timeoutMs) {
|
||||
line = "";
|
||||
for (;;) {
|
||||
int b = readByteWait(c, timeoutMs);
|
||||
if (b < 0) return line.length() > 0;
|
||||
if (b == '\n') break;
|
||||
if (b != '\r') line += (char)b;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool writeAll(NetworkClient &c, const uint8_t *data, size_t len) {
|
||||
uint32_t t0 = millis();
|
||||
while (len) {
|
||||
size_t n = c.write(data, len);
|
||||
if (n == 0) {
|
||||
if (!c.connected()) {
|
||||
NETLOG("write failed: server closed the connection (%u bytes unsent)", (unsigned)len);
|
||||
return false;
|
||||
}
|
||||
if (millis() - t0 > 20000) {
|
||||
NETLOG("write failed: no progress for 20 s (%u bytes unsent, Wi-Fi %s, RSSI %d dBm)", (unsigned)len,
|
||||
wifiStatusName(WiFi.status()), WiFi.RSSI());
|
||||
return false;
|
||||
}
|
||||
delay(10);
|
||||
continue;
|
||||
}
|
||||
data += n;
|
||||
len -= n;
|
||||
t0 = millis();
|
||||
}
|
||||
return true;
|
||||
}
|
||||
static bool writeAll(NetworkClient &c, const String &s) { return writeAll(c, (const uint8_t *)s.c_str(), s.length()); }
|
||||
|
||||
struct HttpHead {
|
||||
int status = 0;
|
||||
bool chunked = false;
|
||||
long contentLength = -1;
|
||||
};
|
||||
|
||||
// Wait (up to firstByteTimeoutMs) for the server to answer, then parse the head.
|
||||
static bool readHead(NetworkClient &c, HttpHead &h, uint32_t firstByteTimeoutMs) {
|
||||
uint32_t t0 = millis();
|
||||
while (!c.available()) {
|
||||
if (!c.connected()) {
|
||||
NETLOG("no response: server closed the connection after %lu ms", millis() - t0);
|
||||
return false;
|
||||
}
|
||||
if (millis() - t0 > firstByteTimeoutMs) {
|
||||
NETLOG("no response: timed out after %lu s", firstByteTimeoutMs / 1000);
|
||||
return false;
|
||||
}
|
||||
delay(20);
|
||||
}
|
||||
NETLOG("first response byte after %lu ms", millis() - t0);
|
||||
String line;
|
||||
if (!readRawLine(c, line, 15000)) {
|
||||
NETLOG("no response: status line never arrived");
|
||||
return false;
|
||||
}
|
||||
NETLOG("< %s", line.c_str());
|
||||
int sp = line.indexOf(' ');
|
||||
if (sp < 0) return false;
|
||||
h.status = line.substring(sp + 1).toInt();
|
||||
while (readRawLine(c, line, 15000) && line.length()) {
|
||||
if (h.status >= 300) NETLOG("< %s", line.c_str()); // full headers only when something went wrong
|
||||
String lower = line;
|
||||
lower.toLowerCase();
|
||||
if (lower.startsWith("transfer-encoding:") && lower.indexOf("chunked") > 0) h.chunked = true;
|
||||
if (lower.startsWith("content-length:")) h.contentLength = lower.substring(15).toInt();
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// Reads a response body, transparently removing chunked transfer encoding.
|
||||
class BodyReader {
|
||||
public:
|
||||
BodyReader(NetworkClient &c, const HttpHead &h, uint32_t idleTimeoutMs)
|
||||
: c_(c), chunked_(h.chunked), remaining_(h.contentLength), timeout_(idleTimeoutMs) {}
|
||||
|
||||
// Returns bytes read, 0 at end of body (or on timeout / disconnect).
|
||||
size_t read(uint8_t *buf, size_t n) {
|
||||
if (done_) return 0;
|
||||
if (chunked_) {
|
||||
if (chunkLeft_ == 0) {
|
||||
String sizeLine;
|
||||
if (!readRawLine(c_, sizeLine, timeout_)) return finish();
|
||||
chunkLeft_ = strtol(sizeLine.c_str(), nullptr, 16);
|
||||
if (chunkLeft_ == 0) return finish();
|
||||
}
|
||||
if (n > chunkLeft_) n = chunkLeft_;
|
||||
} else if (remaining_ >= 0) {
|
||||
if (remaining_ == 0) return finish();
|
||||
if ((long)n > remaining_) n = remaining_;
|
||||
}
|
||||
size_t got = 0;
|
||||
uint32_t t0 = millis();
|
||||
while (got == 0) {
|
||||
int avail = c_.available();
|
||||
if (avail > 0) {
|
||||
int r = c_.read(buf, min((size_t)avail, n));
|
||||
if (r > 0) got = r;
|
||||
} else if (!c_.connected() || millis() - t0 > timeout_) {
|
||||
return finish();
|
||||
} else {
|
||||
delay(5);
|
||||
}
|
||||
}
|
||||
if (chunked_) {
|
||||
chunkLeft_ -= got;
|
||||
if (chunkLeft_ == 0) {
|
||||
String crlf;
|
||||
readRawLine(c_, crlf, timeout_);
|
||||
}
|
||||
} else if (remaining_ > 0) {
|
||||
remaining_ -= got;
|
||||
}
|
||||
return got;
|
||||
}
|
||||
|
||||
bool readLine(String &line) {
|
||||
line = "";
|
||||
for (;;) {
|
||||
if (pos_ == len_) {
|
||||
len_ = read(buf_, sizeof(buf_));
|
||||
pos_ = 0;
|
||||
if (len_ == 0) return line.length() > 0;
|
||||
}
|
||||
char ch = (char)buf_[pos_++];
|
||||
if (ch == '\n') return true;
|
||||
if (ch != '\r') line += ch;
|
||||
}
|
||||
}
|
||||
|
||||
String readAll(size_t maxLen) {
|
||||
String s;
|
||||
uint8_t b[256];
|
||||
size_t n;
|
||||
while (s.length() < maxLen && (n = read(b, sizeof(b))) > 0) s.concat((const char *)b, n);
|
||||
return s;
|
||||
}
|
||||
|
||||
private:
|
||||
size_t finish() { done_ = true; return 0; }
|
||||
NetworkClient &c_;
|
||||
bool chunked_;
|
||||
long remaining_;
|
||||
uint32_t timeout_;
|
||||
size_t chunkLeft_ = 0;
|
||||
bool done_ = false;
|
||||
uint8_t buf_[512];
|
||||
size_t pos_ = 0, len_ = 0;
|
||||
};
|
||||
|
||||
static bool connectTls(NetworkClientSecure &c, const char *host) {
|
||||
c.useBuiltinCACertBundle(); // verify the server certificate
|
||||
c.setHandshakeTimeout(30);
|
||||
NETLOG("connecting to %s:443", host);
|
||||
uint32_t t0 = millis();
|
||||
if (!c.connect(host, 443)) { // does its own (thread-safe) DNS lookup
|
||||
logTlsError(c, "connect failed (DNS or TLS)");
|
||||
return false;
|
||||
}
|
||||
NETLOG("TLS connect ok to %s in %lu ms", c.remoteIP().toString().c_str(), millis() - t0);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool ensureWifi() {
|
||||
if (WiFi.status() != WL_CONNECTED) {
|
||||
NETLOG("Wi-Fi %s, reconnecting...", wifiStatusName(WiFi.status()));
|
||||
WiFi.reconnect();
|
||||
for (int i = 0; i < 150 && WiFi.status() != WL_CONNECTED; i++) delay(100);
|
||||
}
|
||||
if (WiFi.status() != WL_CONNECTED) {
|
||||
NETLOG("Wi-Fi still %s", wifiStatusName(WiFi.status()));
|
||||
return false;
|
||||
}
|
||||
NETLOG("Wi-Fi ok: SSID \"%s\", IP %s, DNS %s, RSSI %d dBm", WiFi.SSID().c_str(),
|
||||
WiFi.localIP().toString().c_str(), WiFi.dnsIP().toString().c_str(), WiFi.RSSI());
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SD reads. On this board the card fails multi-sector reads (CMD18) with an
|
||||
// I/O error, while single-sector reads (CMD17) and all writes work. Arduino's
|
||||
// File::read() goes through stdio, which reads ahead 4 KB at a time and so
|
||||
// always triggers multi-sector reads. Calling POSIX read() with 512-byte
|
||||
// requests keeps every read to a single sector.
|
||||
// ---------------------------------------------------------------------------
|
||||
static int sdOpenRead(const char *path) { return open((String("/sd") + path).c_str(), O_RDONLY); }
|
||||
|
||||
// Fills buf with up to len bytes, 512 at a time. Returns bytes read, or -1 on error.
|
||||
static int sdRead(int fd, uint8_t *buf, size_t len) {
|
||||
size_t got = 0;
|
||||
while (got < len) {
|
||||
int n = read(fd, buf + got, min((size_t)512, len - got));
|
||||
if (n < 0) return got ? (int)got : -1;
|
||||
if (n == 0) break;
|
||||
got += n;
|
||||
}
|
||||
return got;
|
||||
}
|
||||
|
||||
static String httpErrorText(const char *service, int status, const String &body) {
|
||||
// Both OpenAI and Anthropic return {"error": {"message": "..."}}.
|
||||
NETLOG("%s HTTP %d, body: %s", service, status, body.c_str());
|
||||
JsonDocument doc;
|
||||
String msg;
|
||||
if (deserializeJson(doc, body) == DeserializationError::Ok) msg = doc["error"]["message"] | "";
|
||||
if (msg.isEmpty()) msg = body.substring(0, 120);
|
||||
return String(service) + " HTTP " + status + ": " + msg;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Step 1: speech-to-text, in parts.
|
||||
//
|
||||
// Long recordings are split into ~10-minute parts (each upload must stay
|
||||
// under 25 MB). Each part is sent as its own small WAV: a fresh 44-byte
|
||||
// header followed by a slice of the recording's PCM, streamed from the card.
|
||||
// Cuts are moved to the quietest moment just before the target, so no word is
|
||||
// split. Text from the end of the previous part goes along as a "prompt" so
|
||||
// the speech-to-text model keeps names and spelling consistent across parts.
|
||||
//
|
||||
// After each part, its text is appended to the transcript and the position is
|
||||
// saved to a .progress file. If a part fails (Wi-Fi drop, API error), holding
|
||||
// PLUS again resumes from that part instead of starting over.
|
||||
// ---------------------------------------------------------------------------
|
||||
static const uint32_t BYTES_PER_SEC = SAMPLE_RATE * 2;
|
||||
static const uint32_t PART_BYTES = STT_SEGMENT_SECONDS * BYTES_PER_SEC;
|
||||
static const uint32_t LAST_PART_SLACK = PART_BYTES / 10; // fold a short tail (<1 min) into the last part
|
||||
static const int CUT_BLOCKS = min(CUT_SEARCH_SECONDS, STT_SEGMENT_SECONDS / 4) * 10; // 0.1 s blocks searched
|
||||
static_assert(44 + PART_BYTES + LAST_PART_SLACK < MAX_UPLOAD_BYTES, "parts must stay under the upload limit");
|
||||
|
||||
static String hms(uint32_t secs) {
|
||||
char b[16];
|
||||
snprintf(b, sizeof(b), "%02lu:%02lu:%02lu", secs / 3600, secs / 60 % 60, secs % 60);
|
||||
return b;
|
||||
}
|
||||
|
||||
// Last maxBytes of s, starting on a UTF-8 character boundary.
|
||||
static String utf8Tail(const String &s, size_t maxBytes) {
|
||||
if (s.length() <= maxBytes) return s;
|
||||
size_t i = s.length() - maxBytes;
|
||||
while (i < s.length() && ((uint8_t)s[i] & 0xC0) == 0x80) i++;
|
||||
return s.substring(i);
|
||||
}
|
||||
|
||||
// Pick a cut point (file offset) in the CUT_SEARCH_SECONDS before `target`:
|
||||
// the middle of the quietest 0.4 s, measured in 0.1 s blocks.
|
||||
static uint32_t findQuietCut(int fd, uint32_t target) {
|
||||
const uint32_t BLOCK = BYTES_PER_SEC / 10;
|
||||
const int NB = CUT_BLOCKS;
|
||||
const uint32_t start = target - NB * BLOCK;
|
||||
static uint64_t energy[CUT_BLOCKS];
|
||||
static int16_t samples[BYTES_PER_SEC / 20]; // BLOCK bytes
|
||||
{
|
||||
SpiLock lock;
|
||||
if (lseek(fd, start, SEEK_SET) < 0) return target;
|
||||
for (int b = 0; b < NB; b++) {
|
||||
if (sdRead(fd, (uint8_t *)samples, BLOCK) != (int)BLOCK) return target;
|
||||
uint64_t e = 0;
|
||||
for (uint32_t i = 0; i < BLOCK / 2; i++) e += abs(samples[i]);
|
||||
energy[b] = e;
|
||||
}
|
||||
}
|
||||
int best = 0;
|
||||
uint64_t bestSum = UINT64_MAX;
|
||||
for (int b = 0; b + 4 <= NB; b++) {
|
||||
uint64_t sum = energy[b] + energy[b + 1] + energy[b + 2] + energy[b + 3];
|
||||
if (sum < bestSum) { bestSum = sum; best = b; }
|
||||
}
|
||||
return start + (best + 2) * BLOCK; // even offset: always on a sample boundary
|
||||
}
|
||||
|
||||
// Upload bytes [start, end) of the open WAV as one file; the text goes to `text`.
|
||||
static bool transcribePart(int fd, uint32_t start, uint32_t end, const String &fileName, const String &prompt,
|
||||
String &text, String &err) {
|
||||
const uint32_t len = end - start;
|
||||
const char *boundary = "----VoiceNotesFormBoundary7MA4YWxkTrZu0gW";
|
||||
String pre;
|
||||
pre += String("--") + boundary + "\r\nContent-Disposition: form-data; name=\"model\"\r\n\r\n" STT_MODEL "\r\n";
|
||||
pre += String("--") + boundary + "\r\nContent-Disposition: form-data; name=\"response_format\"\r\n\r\ntext\r\n";
|
||||
if (!prompt.isEmpty())
|
||||
pre += String("--") + boundary + "\r\nContent-Disposition: form-data; name=\"prompt\"\r\n\r\n" + prompt + "\r\n";
|
||||
pre += String("--") + boundary + "\r\nContent-Disposition: form-data; name=\"file\"; filename=\"" + fileName +
|
||||
"\"\r\nContent-Type: audio/wav\r\n\r\n";
|
||||
String post = String("\r\n--") + boundary + "--\r\n";
|
||||
uint8_t header[44];
|
||||
wavHeader(header, len);
|
||||
const size_t bodyLen = pre.length() + sizeof(header) + len + post.length();
|
||||
|
||||
NetworkClientSecure client;
|
||||
if (!connectTls(client, STT_HOST)) { err = "Can't reach " STT_HOST; return false; }
|
||||
|
||||
String req;
|
||||
req += "POST " STT_PATH " HTTP/1.1\r\n";
|
||||
req += "Host: " STT_HOST "\r\n";
|
||||
req += "Authorization: Bearer " STT_API_KEY "\r\n";
|
||||
req += String("Content-Type: multipart/form-data; boundary=") + boundary + "\r\n";
|
||||
req += "Content-Length: " + String(bodyLen) + "\r\n";
|
||||
req += "Connection: close\r\n\r\n";
|
||||
NETLOG("STT: POST %s (%s, %u bytes) model=" STT_MODEL ", key %.7s...", fileName.c_str(), hms(len / BYTES_PER_SEC).c_str(),
|
||||
(unsigned)bodyLen, STT_API_KEY);
|
||||
|
||||
bool ok = writeAll(client, req) && writeAll(client, pre) && writeAll(client, header, sizeof(header));
|
||||
if (!ok) NETLOG("STT: failed while sending request headers");
|
||||
{
|
||||
SpiLock lock;
|
||||
if (ok && lseek(fd, start, SEEK_SET) < 0) { NETLOG("STT: seek failed"); ok = false; }
|
||||
}
|
||||
static uint8_t buf[4096];
|
||||
uint32_t sent = 0, upStart = millis();
|
||||
uint8_t lastLogged = 0;
|
||||
app.progress = 0;
|
||||
while (ok && sent < len) {
|
||||
int n;
|
||||
{
|
||||
SpiLock lock;
|
||||
n = sdRead(fd, buf, min((uint32_t)sizeof(buf), len - sent));
|
||||
}
|
||||
if (n <= 0) {
|
||||
NETLOG("STT: SD read failed at offset %u (errno %d %s)", (unsigned)(start + sent), errno, strerror(errno));
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
ok = writeAll(client, buf, n);
|
||||
if (!ok) NETLOG("STT: upload stopped at byte %u of %u", (unsigned)sent, (unsigned)len);
|
||||
sent += n;
|
||||
app.progress = (uint8_t)((uint64_t)sent * 100 / len);
|
||||
if (app.progress >= lastLogged + 25) {
|
||||
lastLogged = app.progress - app.progress % 25;
|
||||
float secs = (millis() - upStart) / 1000.0f;
|
||||
NETLOG("STT: uploaded %u%% (%.0f KB/s)", app.progress, sent / 1024.0f / max(secs, 0.001f));
|
||||
}
|
||||
}
|
||||
if (ok) ok = writeAll(client, post);
|
||||
if (!ok) {
|
||||
// Servers often reject early (bad key, file too big) and close mid-upload;
|
||||
// if they sent a reply before closing, show it.
|
||||
err = "Upload failed";
|
||||
HttpHead early;
|
||||
if (readHead(client, early, 3000)) {
|
||||
BodyReader b(client, early, 3000);
|
||||
err = httpErrorText("STT", early.status, b.readAll(600));
|
||||
}
|
||||
logTlsError(client, "STT upload");
|
||||
return false;
|
||||
}
|
||||
NETLOG("STT: upload done in %.1f s, waiting for transcript", (millis() - upStart) / 1000.0f);
|
||||
|
||||
setState(ST_TRANSCRIBING);
|
||||
HttpHead head;
|
||||
if (!readHead(client, head, 5 * 60 * 1000)) { err = "No reply from STT service"; return false; }
|
||||
BodyReader body(client, head, 60000);
|
||||
if (head.status != 200) {
|
||||
err = httpErrorText("STT", head.status, body.readAll(600));
|
||||
return false;
|
||||
}
|
||||
text = body.readAll(256 * 1024); // ~10 min of speech is ~10 KB of text
|
||||
text.trim();
|
||||
NETLOG("STT: got %u bytes of text", text.length());
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool readProgress(const String &path, uint32_t &offset, uint8_t &part) {
|
||||
SpiLock lock;
|
||||
File f = SD.open(path, FILE_READ); // tiny file: a single-sector read is fine
|
||||
if (!f) return false;
|
||||
String s = f.readString();
|
||||
f.close();
|
||||
return sscanf(s.c_str(), "%lu %hhu", &offset, &part) == 2;
|
||||
}
|
||||
|
||||
static void writeProgress(const String &path, uint32_t offset, uint8_t part) {
|
||||
SpiLock lock;
|
||||
File f = SD.open(path, FILE_WRITE);
|
||||
if (f) {
|
||||
f.printf("%lu %u\n", offset, part);
|
||||
f.close();
|
||||
}
|
||||
}
|
||||
|
||||
// The last ~200 bytes of the transcript so far, used as the prompt when resuming.
|
||||
static String transcriptTail(const String &txtPath) {
|
||||
SpiLock lock;
|
||||
int fd = sdOpenRead(txtPath.c_str());
|
||||
if (fd < 0) return "";
|
||||
struct stat st;
|
||||
String s;
|
||||
if (fstat(fd, &st) == 0 && st.st_size > 0) {
|
||||
char b[257];
|
||||
lseek(fd, st.st_size > 256 ? st.st_size - 256 : 0, SEEK_SET);
|
||||
int n = sdRead(fd, (uint8_t *)b, 256);
|
||||
if (n > 0) s.concat(b, n);
|
||||
}
|
||||
close(fd);
|
||||
return utf8Tail(s, 200);
|
||||
}
|
||||
|
||||
static bool transcribe(const char *wavPath, const String &txtPath, const String &progPath, String &err) {
|
||||
int fd;
|
||||
uint32_t size;
|
||||
{
|
||||
SpiLock lock;
|
||||
fd = sdOpenRead(wavPath);
|
||||
struct stat st;
|
||||
if (fd < 0 || fstat(fd, &st) != 0) {
|
||||
NETLOG("STT: can't open %s (errno %d %s)", wavPath, errno, strerror(errno));
|
||||
if (fd >= 0) close(fd);
|
||||
err = "Can't open recording";
|
||||
return false;
|
||||
}
|
||||
size = st.st_size;
|
||||
}
|
||||
NETLOG("STT: %s, %u bytes (%s of audio)", wavPath, (unsigned)size, hms((size - 44) / BYTES_PER_SEC).c_str());
|
||||
if (size < 44 + BYTES_PER_SEC / 2) { SpiLock l; close(fd); err = "Recording too short"; return false; }
|
||||
|
||||
uint32_t offset = 44;
|
||||
uint8_t part = 0;
|
||||
String prompt;
|
||||
if (readProgress(progPath, offset, part)) {
|
||||
NETLOG("STT: resuming at part %u (%s into the recording)", part + 1, hms((offset - 44) / BYTES_PER_SEC).c_str());
|
||||
prompt = transcriptTail(txtPath);
|
||||
} else {
|
||||
SpiLock lock;
|
||||
SD.remove(txtPath);
|
||||
}
|
||||
if (offset == 44) writeProgress(progPath, offset, part);
|
||||
|
||||
String base = String(strrchr(wavPath, '/') + 1);
|
||||
base.replace(".wav", "");
|
||||
bool ok = true;
|
||||
while (ok && offset < size) {
|
||||
uint32_t end = size - offset <= PART_BYTES + LAST_PART_SLACK ? size : findQuietCut(fd, offset + PART_BYTES);
|
||||
app.partIndex = part + 1;
|
||||
app.partCount = part + 1 + (size > end ? (size - end + PART_BYTES - 1) / PART_BYTES : 0);
|
||||
setState(ST_UPLOADING);
|
||||
NETLOG("STT: part %u of ~%u: %s - %s", app.partIndex, app.partCount, hms((offset - 44) / BYTES_PER_SEC).c_str(),
|
||||
hms((end - 44) / BYTES_PER_SEC).c_str());
|
||||
|
||||
String text;
|
||||
ok = false;
|
||||
for (int attempt = 1; attempt <= STT_MAX_ATTEMPTS && !ok; attempt++) {
|
||||
if (attempt > 1) {
|
||||
NETLOG("STT: retrying part %u (attempt %d of %d) after: %s", part + 1, attempt, STT_MAX_ATTEMPTS, err.c_str());
|
||||
delay(5000 * (attempt - 1));
|
||||
setState(ST_UPLOADING);
|
||||
}
|
||||
if (!ensureWifi()) { err = "Wi-Fi not connected"; continue; }
|
||||
ok = transcribePart(fd, offset, end, base + "_part" + (part + 1) + ".wav", prompt, text, err);
|
||||
}
|
||||
if (!ok) break;
|
||||
|
||||
if (!text.isEmpty()) {
|
||||
SpiLock lock;
|
||||
File f = SD.open(txtPath, FILE_APPEND);
|
||||
if (!f) { err = "Can't write transcript"; ok = false; break; }
|
||||
f.printf("[%s]\n", hms((offset - 44) / BYTES_PER_SEC).c_str());
|
||||
f.print(text);
|
||||
f.print("\n\n");
|
||||
f.close();
|
||||
prompt = utf8Tail(text, 200);
|
||||
}
|
||||
app.received += text.length();
|
||||
offset = end;
|
||||
part++;
|
||||
writeProgress(progPath, offset, part);
|
||||
}
|
||||
{ SpiLock l; close(fd); }
|
||||
|
||||
if (!ok) {
|
||||
err = "Part " + String(app.partIndex) + "/" + String(app.partCount) + ": " + err + ". Hold PLUS to resume";
|
||||
return false;
|
||||
}
|
||||
bool haveText;
|
||||
{
|
||||
SpiLock lock;
|
||||
SD.remove(progPath);
|
||||
haveText = SD.exists(txtPath);
|
||||
}
|
||||
if (!haveText) { err = "No speech detected"; return false; }
|
||||
NETLOG("STT: transcript complete -> %s", txtPath.c_str());
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Step 2: summarize with Claude.
|
||||
//
|
||||
// A two-hour transcript is ~100 KB, too big to hold in RAM next to Wi-Fi and
|
||||
// TLS. So the JSON request body is streamed: the transcript is read from the
|
||||
// card twice, once to measure its JSON-escaped length for Content-Length,
|
||||
// then again to escape and send it in 512-byte pieces. The reply is streamed
|
||||
// too (server-sent events), and each text delta is appended to the notes file.
|
||||
// ---------------------------------------------------------------------------
|
||||
static size_t jsonEscape(const uint8_t *in, size_t n, char *out) {
|
||||
size_t o = 0;
|
||||
for (size_t i = 0; i < n; i++) {
|
||||
uint8_t ch = in[i];
|
||||
switch (ch) {
|
||||
case '"': if (out) { out[o] = '\\'; out[o + 1] = '"'; } o += 2; break;
|
||||
case '\\': if (out) { out[o] = '\\'; out[o + 1] = '\\'; } o += 2; break;
|
||||
case '\n': if (out) { out[o] = '\\'; out[o + 1] = 'n'; } o += 2; break;
|
||||
case '\r': if (out) { out[o] = '\\'; out[o + 1] = 'r'; } o += 2; break;
|
||||
case '\t': if (out) { out[o] = '\\'; out[o + 1] = 't'; } o += 2; break;
|
||||
default:
|
||||
if (ch < 0x20) {
|
||||
if (out) snprintf(out + o, 7, "\\u%04x", ch);
|
||||
o += 6;
|
||||
} else {
|
||||
if (out) out[o] = ch; // UTF-8 bytes pass through unchanged
|
||||
o += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
return o; // with out == nullptr, just measures
|
||||
}
|
||||
|
||||
static String jsonEscape(const String &s) {
|
||||
String out;
|
||||
out.reserve(s.length() + s.length() / 8 + 8);
|
||||
char tmp[6 * 64 + 1];
|
||||
for (size_t i = 0; i < s.length(); i += 64) {
|
||||
size_t n = jsonEscape((const uint8_t *)s.c_str() + i, min((size_t)64, s.length() - i), tmp);
|
||||
out.concat(tmp, n);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
static const char *SYSTEM_PROMPT =
|
||||
"You turn recordings of meetings and voice memos into clear, well-organized notes. The transcript "
|
||||
"comes from automatic speech recognition, so expect filler words, false starts and the occasional "
|
||||
"misheard word; infer the intended meaning, but don't add facts that aren't in the recording.";
|
||||
|
||||
static const char *NOTES_INSTRUCTIONS =
|
||||
"Write notes in Markdown, scaled to the length of the recording (a short memo needs only a few lines):\n"
|
||||
"- a short descriptive title as a level-1 heading\n"
|
||||
"- **Summary**: one short paragraph, or up to three for a long meeting\n"
|
||||
"- **Participants**: people who are named or clearly identifiable (omit if none)\n"
|
||||
"- **Decisions**: what was decided (omit if nothing was)\n"
|
||||
"- **Action items**: checkbox bullets with owner and due date when mentioned, or \"None\"\n"
|
||||
"- **Discussion**: for longer recordings, the main topics in order, each with its key points and the "
|
||||
"[hh:mm:ss] time it came up\n"
|
||||
"- **Open questions**: unresolved issues (omit if none)\n"
|
||||
"The transcript has no speaker labels, so only attribute a statement to someone when the transcript "
|
||||
"makes it clear who said it. Reply with the notes only.";
|
||||
|
||||
static bool summarize(const String &txtPath, const String &mdPath, const char *wavPath, String &err) {
|
||||
// Pass 1: measure the transcript and its escaped length.
|
||||
static uint8_t in[512];
|
||||
static char out[512 * 6];
|
||||
size_t rawLen = 0, escLen = 0;
|
||||
{
|
||||
SpiLock lock;
|
||||
int fd = sdOpenRead(txtPath.c_str());
|
||||
if (fd < 0) { err = "Can't open transcript"; return false; }
|
||||
int n;
|
||||
while ((n = sdRead(fd, in, sizeof(in))) > 0) {
|
||||
rawLen += n;
|
||||
escLen += jsonEscape(in, n, nullptr);
|
||||
}
|
||||
close(fd);
|
||||
}
|
||||
NETLOG("Claude: transcript %u bytes", (unsigned)rawLen);
|
||||
if (rawLen == 0) { err = "Transcript is empty"; return false; }
|
||||
|
||||
// The file name carries the start time, e.g. REC_2026-10-02_14-30-05.
|
||||
String stamp = String(strrchr(wavPath, '/') + 1);
|
||||
stamp.replace("REC_", "");
|
||||
stamp.replace(".wav", "");
|
||||
uint32_t secs = 0;
|
||||
{
|
||||
SpiLock lock;
|
||||
File w = SD.open(wavPath, FILE_READ);
|
||||
if (w) { secs = (w.size() - 44) / BYTES_PER_SEC; w.close(); }
|
||||
}
|
||||
|
||||
String intro = "Below is the transcript of a recording started " + stamp + " and lasting " + hms(secs) +
|
||||
". It was transcribed in parts; [hh:mm:ss] markers give the time since the recording "
|
||||
"began.\n\n<transcript>\n";
|
||||
String outro = String("\n</transcript>\n\n") + NOTES_INSTRUCTIONS;
|
||||
|
||||
String prefix;
|
||||
prefix += "{\"model\":\"" CLAUDE_MODEL "\",\"max_tokens\":64000,\"stream\":true,";
|
||||
prefix += "\"fallbacks\":\"default\",\"output_config\":{\"effort\":\"medium\"},";
|
||||
prefix += "\"system\":\"" + jsonEscape(SYSTEM_PROMPT) + "\",";
|
||||
prefix += "\"messages\":[{\"role\":\"user\",\"content\":\"" + jsonEscape(intro);
|
||||
String suffix = jsonEscape(outro) + "\"}]}";
|
||||
const size_t bodyLen = prefix.length() + escLen + suffix.length();
|
||||
|
||||
NetworkClientSecure client;
|
||||
if (!connectTls(client, "api.anthropic.com")) { err = "Can't reach Claude API"; return false; }
|
||||
String req;
|
||||
req += "POST /v1/messages HTTP/1.1\r\n";
|
||||
req += "Host: api.anthropic.com\r\n";
|
||||
req += "x-api-key: " ANTHROPIC_API_KEY "\r\n";
|
||||
req += "anthropic-version: 2023-06-01\r\n";
|
||||
req += "anthropic-beta: server-side-fallback-2026-07-01\r\n";
|
||||
req += "content-type: application/json\r\n";
|
||||
req += "accept: text/event-stream\r\n";
|
||||
req += "Content-Length: " + String(bodyLen) + "\r\n";
|
||||
req += "Connection: close\r\n\r\n";
|
||||
NETLOG("Claude: POST /v1/messages model=" CLAUDE_MODEL ", key %.7s..., body %u bytes", ANTHROPIC_API_KEY,
|
||||
(unsigned)bodyLen);
|
||||
|
||||
// Pass 2: send prefix, escaped transcript, suffix.
|
||||
bool ok = writeAll(client, req) && writeAll(client, prefix);
|
||||
int fd;
|
||||
{
|
||||
SpiLock lock;
|
||||
fd = sdOpenRead(txtPath.c_str());
|
||||
}
|
||||
size_t sentEsc = 0;
|
||||
while (ok && fd >= 0) {
|
||||
int n;
|
||||
{
|
||||
SpiLock lock;
|
||||
n = sdRead(fd, in, sizeof(in));
|
||||
}
|
||||
if (n <= 0) break;
|
||||
size_t m = jsonEscape(in, n, out);
|
||||
ok = writeAll(client, (const uint8_t *)out, m);
|
||||
sentEsc += m;
|
||||
}
|
||||
if (fd >= 0) { SpiLock l; close(fd); }
|
||||
if (ok && sentEsc != escLen) {
|
||||
NETLOG("Claude: transcript changed while sending (%u vs %u bytes)", (unsigned)sentEsc, (unsigned)escLen);
|
||||
ok = false;
|
||||
}
|
||||
if (ok) ok = writeAll(client, suffix);
|
||||
if (!ok) {
|
||||
logTlsError(client, "Claude send");
|
||||
err = "Send to Claude failed";
|
||||
return false;
|
||||
}
|
||||
|
||||
HttpHead head;
|
||||
if (!readHead(client, head, 120000)) { err = "No reply from Claude"; return false; }
|
||||
// Ping events arrive every few seconds while the model thinks, so a 90 s
|
||||
// idle timeout only trips if the connection has really stalled.
|
||||
BodyReader sse(client, head, 90000);
|
||||
if (head.status != 200) {
|
||||
err = httpErrorText("Claude", head.status, sse.readAll(600));
|
||||
return false;
|
||||
}
|
||||
|
||||
File md;
|
||||
{
|
||||
SpiLock lock;
|
||||
md = SD.open(mdPath, FILE_WRITE);
|
||||
if (!md) { err = "Can't write notes"; return false; }
|
||||
}
|
||||
|
||||
// Keep only the fields we use from each event; thinking signatures etc. are dropped.
|
||||
JsonDocument filter;
|
||||
filter["type"] = true;
|
||||
filter["delta"]["type"] = true;
|
||||
filter["delta"]["text"] = true;
|
||||
filter["delta"]["stop_reason"] = true;
|
||||
filter["error"]["message"] = true;
|
||||
|
||||
String line, pending, stopReason;
|
||||
bool gotStop = false;
|
||||
size_t total = 0;
|
||||
auto flush = [&]() {
|
||||
if (pending.isEmpty()) return;
|
||||
SpiLock lock;
|
||||
md.print(pending);
|
||||
pending = "";
|
||||
};
|
||||
|
||||
while (sse.readLine(line)) {
|
||||
if (!line.startsWith("data:")) continue;
|
||||
JsonDocument ev;
|
||||
if (deserializeJson(ev, line.c_str() + 5, DeserializationOption::Filter(filter))) continue;
|
||||
const char *type = ev["type"] | "";
|
||||
if (!strcmp(type, "content_block_delta") && !strcmp(ev["delta"]["type"] | "", "text_delta")) {
|
||||
const char *t = ev["delta"]["text"] | "";
|
||||
pending += t;
|
||||
total += strlen(t);
|
||||
app.received = total;
|
||||
if (pending.length() > 512) flush();
|
||||
} else if (!strcmp(type, "message_delta")) {
|
||||
stopReason = ev["delta"]["stop_reason"] | "";
|
||||
} else if (!strcmp(type, "message_stop")) {
|
||||
gotStop = true;
|
||||
break;
|
||||
} else if (!strcmp(type, "error")) {
|
||||
NETLOG("Claude stream error event: %s", line.c_str());
|
||||
err = String("Claude: ") + (ev["error"]["message"] | "stream error");
|
||||
break;
|
||||
}
|
||||
}
|
||||
flush();
|
||||
NETLOG("Claude: stream ended, %u chars, stop_reason=%s, message_stop=%s", (unsigned)total,
|
||||
stopReason.isEmpty() ? "(none)" : stopReason.c_str(), gotStop ? "yes" : "no");
|
||||
{
|
||||
SpiLock lock;
|
||||
md.printf("\n\n---\n_Recording: %s (%s) · Transcript: %s_\n", wavPath, hms(secs).c_str(), txtPath.c_str());
|
||||
md.close();
|
||||
}
|
||||
|
||||
if (!err.isEmpty()) return false;
|
||||
if (stopReason == "refusal") { err = "Claude declined this request"; return false; }
|
||||
if (!gotStop) { err = "Claude stream cut off"; return false; }
|
||||
if (total == 0) { err = "Claude returned no text"; return false; }
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Step 3: copy the notes to Nextcloud over WebDAV (HTTP PUT with Basic auth).
|
||||
// /remote.php/webdav/ maps to the logged-in user's files, so it works whether
|
||||
// NEXTCLOUD_USER is the user ID or an email login.
|
||||
// ---------------------------------------------------------------------------
|
||||
#ifndef NEXTCLOUD_HOST
|
||||
#define NEXTCLOUD_HOST ""
|
||||
#define NEXTCLOUD_USER ""
|
||||
#define NEXTCLOUD_PASSWORD ""
|
||||
#define NEXTCLOUD_FOLDER ""
|
||||
#endif
|
||||
|
||||
static bool nextcloudEnabled() { return strlen(NEXTCLOUD_HOST) > 0; }
|
||||
|
||||
static String urlEncodePath(const String &s) {
|
||||
String out;
|
||||
for (size_t i = 0; i < s.length(); i++) {
|
||||
char c = s[i];
|
||||
if (isalnum((unsigned char)c) || strchr("-_.~/", c)) {
|
||||
out += c;
|
||||
} else {
|
||||
char b[4];
|
||||
snprintf(b, sizeof(b), "%%%02X", (uint8_t)c);
|
||||
out += b;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// One WebDAV request. Streams the local file as the body when localPath is set.
|
||||
static int webdavRequest(const char *method, const String &remotePath, const char *localPath, const char *contentType,
|
||||
String &err) {
|
||||
int fd = -1;
|
||||
size_t len = 0;
|
||||
if (localPath) {
|
||||
SpiLock lock;
|
||||
fd = sdOpenRead(localPath);
|
||||
struct stat st;
|
||||
if (fd < 0 || fstat(fd, &st) != 0) {
|
||||
if (fd >= 0) close(fd);
|
||||
err = String("Can't open ") + localPath;
|
||||
return -1;
|
||||
}
|
||||
len = st.st_size;
|
||||
}
|
||||
NetworkClientSecure client;
|
||||
if (!connectTls(client, NEXTCLOUD_HOST)) {
|
||||
if (fd >= 0) { SpiLock l; close(fd); }
|
||||
err = "Can't reach " NEXTCLOUD_HOST;
|
||||
return -1;
|
||||
}
|
||||
String req = String(method) + " /remote.php/webdav/" + urlEncodePath(remotePath) + " HTTP/1.1\r\n";
|
||||
req += "Host: " NEXTCLOUD_HOST "\r\n";
|
||||
req += "Authorization: Basic " + base64::encode(String(NEXTCLOUD_USER ":" NEXTCLOUD_PASSWORD)) + "\r\n";
|
||||
if (contentType) req += String("Content-Type: ") + contentType + "\r\n";
|
||||
req += "Content-Length: " + String(len) + "\r\n";
|
||||
req += "Connection: close\r\n\r\n";
|
||||
NETLOG("Nextcloud: %s /remote.php/webdav/%s (%u bytes) as " NEXTCLOUD_USER, method, remotePath.c_str(), (unsigned)len);
|
||||
|
||||
bool ok = writeAll(client, req);
|
||||
static uint8_t buf[4096];
|
||||
size_t sent = 0;
|
||||
while (ok && fd >= 0 && sent < len) {
|
||||
int n;
|
||||
{
|
||||
SpiLock lock;
|
||||
n = sdRead(fd, buf, sizeof(buf));
|
||||
}
|
||||
if (n <= 0) { ok = false; break; }
|
||||
ok = writeAll(client, buf, n);
|
||||
sent += n;
|
||||
}
|
||||
if (fd >= 0) { SpiLock l; close(fd); }
|
||||
if (!ok) { err = "Upload to Nextcloud failed"; return -1; }
|
||||
|
||||
HttpHead head;
|
||||
if (!readHead(client, head, 60000)) { err = "No reply from Nextcloud"; return -1; }
|
||||
if (head.status >= 300) {
|
||||
BodyReader body(client, head, 5000);
|
||||
NETLOG("Nextcloud: HTTP %d, body: %s", head.status, body.readAll(400).c_str());
|
||||
}
|
||||
return head.status;
|
||||
}
|
||||
|
||||
static bool nextcloudUpload(const String &localPath, const char *contentType, String &err) {
|
||||
String name = String(strrchr(localPath.c_str(), '/') + 1);
|
||||
String remote = String(NEXTCLOUD_FOLDER) + "/" + name;
|
||||
int status = webdavRequest("PUT", remote, localPath.c_str(), contentType, err);
|
||||
if (status == 404 || status == 409) { // folder missing: create it, then retry once
|
||||
NETLOG("Nextcloud: folder " NEXTCLOUD_FOLDER " not found, creating it");
|
||||
int mk = webdavRequest("MKCOL", NEXTCLOUD_FOLDER, nullptr, nullptr, err);
|
||||
if (mk == 201 || mk == 405) status = webdavRequest("PUT", remote, localPath.c_str(), contentType, err);
|
||||
}
|
||||
if (status == 201 || status == 204) {
|
||||
NETLOG("Nextcloud: saved %s", remote.c_str());
|
||||
return true;
|
||||
}
|
||||
if (status == 401) err = "Nextcloud login failed - check the app password in secrets.h";
|
||||
else if (status == 507) err = "Nextcloud storage is full";
|
||||
else if (status > 0) err = "Nextcloud HTTP " + String(status);
|
||||
return false;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
static void netTask(void *) {
|
||||
String base = String(strrchr(jobWav, '/') + 1);
|
||||
base.replace(".wav", "");
|
||||
String txtPath = String(DIR_NOTES "/") + base + "_transcript.txt";
|
||||
String progPath = String(DIR_NOTES "/") + base + "_transcript.progress";
|
||||
String mdPath = String(DIR_NOTES "/") + base + "_summary.md";
|
||||
String err;
|
||||
|
||||
// Reuse finished work: a transcript without a .progress file is complete,
|
||||
// and an existing summary only needs copying to Nextcloud (e.g. when only
|
||||
// that step failed last time). Delete the summary on the card to redo it.
|
||||
bool transcriptDone, summaryDone;
|
||||
{
|
||||
SpiLock lock;
|
||||
transcriptDone = SD.exists(txtPath) && !SD.exists(progPath);
|
||||
summaryDone = transcriptDone && SD.exists(mdPath);
|
||||
}
|
||||
|
||||
app.progress = 0;
|
||||
app.received = 0;
|
||||
app.partIndex = 0;
|
||||
app.partCount = 0;
|
||||
setState(ST_UPLOADING, base.c_str());
|
||||
bool ok = ensureWifi();
|
||||
if (!ok) err = "Wi-Fi not connected";
|
||||
|
||||
if (ok && !transcriptDone) ok = transcribe(jobWav, txtPath, progPath, err);
|
||||
if (ok && !summaryDone) {
|
||||
app.received = 0;
|
||||
setState(ST_SUMMARIZING, base.c_str());
|
||||
ok = summarize(txtPath, mdPath, jobWav, err);
|
||||
}
|
||||
|
||||
bool synced = false;
|
||||
if (ok && nextcloudEnabled()) {
|
||||
setState(ST_SYNCING, base.c_str());
|
||||
if (strlen(NEXTCLOUD_PASSWORD) == 0) {
|
||||
err = "Notes are on the SD card. Add the Nextcloud app password to secrets.h";
|
||||
ok = false;
|
||||
} else {
|
||||
ok = nextcloudUpload(mdPath, "text/markdown; charset=utf-8", err) &&
|
||||
nextcloudUpload(txtPath, "text/plain; charset=utf-8", err);
|
||||
if (!ok) err = "Notes are on the SD card. " + err + ". Hold PLUS to retry";
|
||||
synced = ok;
|
||||
}
|
||||
}
|
||||
|
||||
if (ok) {
|
||||
NETLOG("SUCCESS: notes written to %s%s", mdPath.c_str(), synced ? " and copied to Nextcloud" : "");
|
||||
setState(ST_DONE, synced ? "Also saved to Nextcloud /" NEXTCLOUD_FOLDER : (base + "_summary.md").c_str());
|
||||
} else {
|
||||
NETLOG("FAILED: %s", err.c_str());
|
||||
setState(ST_ERROR, err.c_str());
|
||||
}
|
||||
busy = false;
|
||||
vTaskDelete(nullptr);
|
||||
}
|
||||
|
||||
bool cloudStart(const char *wavPath) {
|
||||
if (busy) return false;
|
||||
busy = true;
|
||||
strlcpy(jobWav, wavPath, sizeof(jobWav));
|
||||
// TLS handshakes need a deep stack.
|
||||
if (xTaskCreate(netTask, "net", 16384, nullptr, 2, nullptr) != pdPASS) {
|
||||
busy = false;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
// Speech-to-text + Claude summary pipeline. Runs in its own FreeRTOS task so
|
||||
// the display and LED ring keep animating while the network calls block.
|
||||
#pragma once
|
||||
|
||||
bool cloudStart(const char *wavPath); // false if a job is already running
|
||||
bool cloudBusy();
|
||||
@@ -0,0 +1,73 @@
|
||||
// Board pinout and app settings for the Waveshare ESP32-C6-LCD-0.85.
|
||||
// Pins come from Waveshare's schematic and demo code:
|
||||
// https://github.com/waveshareteam/ESP32-C6-LCD-0.85
|
||||
#pragma once
|
||||
|
||||
// ---------- Buttons (active LOW, external pull-ups on the board) ----------
|
||||
// The board has only two GPIO buttons. The PWR key drives the power switch IC
|
||||
// and is not readable from firmware.
|
||||
#define PIN_BTN_PLUS 18 // "PLUS" key -> Button 1: start recording
|
||||
#define PIN_BTN_BOOT 9 // "BOOT" key -> Button 2: stop recording
|
||||
// Button 3 (transcribe + summarize) defaults to a long-press of PLUS.
|
||||
// If you wire an external push button (to GND), set its GPIO here and it
|
||||
// will also act as Button 3 on a single click. -1 = not fitted.
|
||||
#define PIN_BTN_EXTRA -1
|
||||
#define BTN3_LONG_PRESS_MS 1000
|
||||
|
||||
// ---------- LCD: ST7735, 128x128, shares the SPI bus with the microSD ----------
|
||||
#define PIN_SPI_SCK 1
|
||||
#define PIN_SPI_MOSI 2
|
||||
#define PIN_SPI_MISO 16
|
||||
#define PIN_LCD_DC 3
|
||||
#define PIN_LCD_CS 5
|
||||
#define PIN_LCD_RST 4
|
||||
#define PIN_LCD_BL 6
|
||||
#define LCD_COL_OFFSET 2
|
||||
#define LCD_ROW_OFFSET 3
|
||||
|
||||
// ---------- microSD ----------
|
||||
#define PIN_SD_CS 17
|
||||
#define SD_SPI_HZ 20000000
|
||||
|
||||
// ---------- LED ring: 8 x WS2812B ----------
|
||||
#define PIN_LEDS 14
|
||||
#define NUM_LEDS 8
|
||||
#define LED_BRIGHTNESS 60 // 0-255; the ring is bright at close range
|
||||
#define LED_COLOR_ORDER RGB // per Waveshare's demo; use GRB if red and green look swapped
|
||||
|
||||
// ---------- Audio: ES7210 mic ADC on I2C + I2S ----------
|
||||
#define PIN_I2C_SDA 8
|
||||
#define PIN_I2C_SCL 7
|
||||
#define PIN_I2S_MCLK 19
|
||||
#define PIN_I2S_BCLK 20
|
||||
#define PIN_I2S_WS 22
|
||||
#define PIN_I2S_DIN 21 // ES7210 SDOUT1 -> ESP32
|
||||
#define PIN_PA_CTRL 15 // NS4150 speaker amp enable (kept off)
|
||||
#define ES7210_ADDR 0x40
|
||||
#define MIC_GAIN_STEP 14 // ES7210 PGA: 0..10 = 0..30 dB in 3 dB steps, 11..14 = 33/34.5/36/37.5 dB (max)
|
||||
#define DIGITAL_GAIN_DB 6 // extra software gain after the preamp, with a soft limiter (0 = off)
|
||||
|
||||
#define SAMPLE_RATE 16000 // 16 kHz mono 16-bit = 32 KB/s, ideal for speech-to-text
|
||||
|
||||
// Longest recording (also limited by free space on the card). 4 h = ~460 MB.
|
||||
#define MAX_RECORD_SECONDS (4 * 3600)
|
||||
|
||||
// Transcription APIs reject files over 25 MB (~13 min), so long recordings are
|
||||
// sent in parts of about STT_SEGMENT_SECONDS each. Every cut is moved to the
|
||||
// quietest moment in the CUT_SEARCH_SECONDS before it, so words aren't split.
|
||||
#define MAX_UPLOAD_BYTES (25UL * 1024 * 1024)
|
||||
#define STT_SEGMENT_SECONDS 600
|
||||
#define CUT_SEARCH_SECONDS 15
|
||||
#define STT_MAX_ATTEMPTS 3 // per part, before giving up (progress is kept for a later resume)
|
||||
|
||||
// ---------- Network ----------
|
||||
// DNS servers used instead of the ones the router hands out ("" = keep the router's).
|
||||
// Pinned because the board was once seen using 8.8.8.9, which doesn't answer DNS and
|
||||
// added ~8 s to every connection.
|
||||
#define DNS_PRIMARY "8.8.8.8"
|
||||
#define DNS_SECONDARY "1.1.1.1"
|
||||
|
||||
// ---------- Storage layout ----------
|
||||
#define DIR_RECORDINGS "/recordings"
|
||||
#define DIR_NOTES "/notes"
|
||||
#define LAST_REC_FILE "/recordings/last.txt"
|
||||
@@ -0,0 +1,32 @@
|
||||
// Copy this file to secrets.h and fill in your values.
|
||||
// secrets.h is git-ignored so your keys stay off GitHub.
|
||||
#pragma once
|
||||
|
||||
#define WIFI_SSID "your-wifi-name"
|
||||
#define WIFI_PASSWORD "your-wifi-password"
|
||||
|
||||
// POSIX timezone string used for the timestamp in file names.
|
||||
// Examples: "PST8PDT,M3.2.0,M11.1.0" (US Pacific), "EST5EDT,M3.2.0,M11.1.0" (US Eastern),
|
||||
// "GMT0BST,M3.5.0/1,M10.5.0" (UK), "IST-5:30" (India), "UTC0"
|
||||
#define TIMEZONE "PST8PDT,M3.2.0,M11.1.0"
|
||||
|
||||
// ---- Speech-to-text: any OpenAI-compatible /audio/transcriptions endpoint ----
|
||||
// OpenAI: host "api.openai.com", path "/v1/audio/transcriptions", model "whisper-1"
|
||||
// (or "gpt-4o-transcribe" / "gpt-4o-mini-transcribe")
|
||||
// Groq: host "api.groq.com", path "/openai/v1/audio/transcriptions", model "whisper-large-v3-turbo"
|
||||
#define STT_HOST "api.openai.com"
|
||||
#define STT_PATH "/v1/audio/transcriptions"
|
||||
#define STT_MODEL "whisper-1"
|
||||
#define STT_API_KEY "sk-..."
|
||||
|
||||
// ---- Summarization: Claude ----
|
||||
#define ANTHROPIC_API_KEY "sk-ant-..."
|
||||
#define CLAUDE_MODEL "claude-opus-5-5"
|
||||
|
||||
// ---- Nextcloud: finished notes are copied here (leave NEXTCLOUD_HOST "" to turn off) ----
|
||||
// Use an app password, not your login password:
|
||||
// Nextcloud -> Settings -> Security -> "Devices & sessions" -> Create new app password.
|
||||
#define NEXTCLOUD_HOST "" // e.g. "cloud.example.com"
|
||||
#define NEXTCLOUD_USER ""
|
||||
#define NEXTCLOUD_PASSWORD "" // paste the app password here
|
||||
#define NEXTCLOUD_FOLDER "secondbrain"
|
||||
@@ -0,0 +1,362 @@
|
||||
#include "ui.h"
|
||||
#include "app_state.h"
|
||||
#include "audio.h"
|
||||
#include "config.h"
|
||||
#include <Arduino_GFX_Library.h>
|
||||
#include <FastLED.h>
|
||||
#include <U8g2lib.h>
|
||||
#include <WiFi.h>
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Display: draw into a 128x128 off-screen canvas (32 KB) and push the whole
|
||||
// frame at once, so animations don't flicker.
|
||||
// ---------------------------------------------------------------------------
|
||||
static Arduino_DataBus *bus = new Arduino_HWSPI(PIN_LCD_DC, PIN_LCD_CS, PIN_SPI_SCK, PIN_SPI_MOSI, PIN_SPI_MISO);
|
||||
static Arduino_GFX *panel = new Arduino_ST7735(bus, PIN_LCD_RST, 0, true, 128, 128,
|
||||
LCD_COL_OFFSET, LCD_ROW_OFFSET, LCD_COL_OFFSET, LCD_ROW_OFFSET);
|
||||
static Arduino_Canvas *gfx = new Arduino_Canvas(128, 128, panel);
|
||||
|
||||
static CRGB leds[NUM_LEDS];
|
||||
|
||||
// Palette
|
||||
static const uint16_t C_BG = RGB565(0, 0, 0);
|
||||
static const uint16_t C_TEXT = RGB565(235, 235, 235);
|
||||
static const uint16_t C_DIM = RGB565(110, 110, 120);
|
||||
static const uint16_t C_FAINT = RGB565(40, 40, 48);
|
||||
static const uint16_t C_RED = RGB565(255, 60, 60);
|
||||
static const uint16_t C_GREEN = RGB565(60, 220, 110);
|
||||
static const uint16_t C_BLUE = RGB565(60, 150, 255);
|
||||
static const uint16_t C_CYAN = RGB565(40, 220, 230);
|
||||
static const uint16_t C_PURPLE = RGB565(180, 110, 255);
|
||||
static const uint16_t C_AMBER = RGB565(255, 170, 40);
|
||||
|
||||
// Fonts (U8g2 font set, rendered by Arduino_GFX)
|
||||
#define F_SMALL u8g2_font_5x8_tr
|
||||
#define F_MED u8g2_font_helvB10_tr
|
||||
#define F_BIG u8g2_font_logisoso22_tn
|
||||
|
||||
static float levelHistory[32];
|
||||
static uint8_t levelPos = 0;
|
||||
|
||||
bool uiBegin() {
|
||||
FastLED.addLeds<WS2812B, PIN_LEDS, LED_COLOR_ORDER>(leds, NUM_LEDS);
|
||||
FastLED.setBrightness(LED_BRIGHTNESS);
|
||||
FastLED.clear(true);
|
||||
|
||||
SpiLock lock;
|
||||
if (!gfx->begin()) return false;
|
||||
gfx->fillScreen(C_BG);
|
||||
gfx->flush();
|
||||
pinMode(PIN_LCD_BL, OUTPUT);
|
||||
digitalWrite(PIN_LCD_BL, HIGH);
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------- drawing helpers ----------
|
||||
static void textCentered(const char *s, int y, const uint8_t *font, uint16_t color) {
|
||||
int16_t x1, y1;
|
||||
uint16_t w, h;
|
||||
gfx->setFont(font);
|
||||
gfx->getTextBounds(s, 0, y, &x1, &y1, &w, &h);
|
||||
gfx->setTextColor(color);
|
||||
gfx->setCursor((128 - (int)w) / 2 - x1, y);
|
||||
gfx->print(s);
|
||||
}
|
||||
|
||||
// Word-wrap into lines of the 5x8 font (24 chars across), centered.
|
||||
static void textWrapped(const char *s, int y, uint16_t color, int maxLines) {
|
||||
const int cols = 24;
|
||||
char line[cols + 1];
|
||||
int len = strlen(s), start = 0;
|
||||
for (int n = 0; n < maxLines && start < len; n++) {
|
||||
while (s[start] == ' ') start++;
|
||||
int end = min(start + cols, len);
|
||||
if (end < len) {
|
||||
int sp = end;
|
||||
while (sp > start && s[sp] != ' ') sp--;
|
||||
if (sp > start) end = sp;
|
||||
}
|
||||
memcpy(line, s + start, end - start);
|
||||
line[end - start] = 0;
|
||||
textCentered(line, y + n * 10, F_SMALL, color);
|
||||
start = end;
|
||||
}
|
||||
}
|
||||
|
||||
static void drawCheck(int cx, int cy, uint16_t color) {
|
||||
gfx->fillCircle(cx, cy, 18, color);
|
||||
for (int d = -1; d <= 1; d++) {
|
||||
gfx->drawLine(cx - 9, cy + d, cx - 3, cy + 6 + d, C_BG);
|
||||
gfx->drawLine(cx - 3, cy + 6 + d, cx + 9, cy - 6 + d, C_BG);
|
||||
}
|
||||
}
|
||||
|
||||
static void drawCross(int cx, int cy, uint16_t color) {
|
||||
gfx->fillCircle(cx, cy, 18, color);
|
||||
for (int d = -1; d <= 1; d++) {
|
||||
gfx->drawLine(cx - 7 + d, cy - 7, cx + 7 + d, cy + 7, C_BG);
|
||||
gfx->drawLine(cx + 7 + d, cy - 7, cx - 7 + d, cy + 7, C_BG);
|
||||
}
|
||||
}
|
||||
|
||||
static void drawMic(int cx, int cy, uint16_t color) {
|
||||
gfx->fillRoundRect(cx - 7, cy - 16, 14, 24, 7, color);
|
||||
gfx->fillArc(cx, cy - 2, 13, 11, 0, 180, color);
|
||||
gfx->fillRect(cx - 1, cy + 11, 3, 6, color);
|
||||
gfx->fillRect(cx - 7, cy + 16, 15, 2, color);
|
||||
}
|
||||
|
||||
// 12 dots around a circle with a fading tail - mirrors the LED ring spinner.
|
||||
static void drawSpinner(int cx, int cy, uint16_t color, uint32_t t) {
|
||||
int head = (t / 80) % 12;
|
||||
for (int i = 0; i < 12; i++) {
|
||||
float a = i * (TWO_PI / 12) - HALF_PI;
|
||||
int age = (head - i + 12) % 12;
|
||||
uint16_t c = age == 0 ? color : age < 4 ? C_DIM : C_FAINT;
|
||||
gfx->fillCircle(cx + cosf(a) * 20, cy + sinf(a) * 20, age == 0 ? 4 : 3, c);
|
||||
}
|
||||
}
|
||||
|
||||
static void drawProgress(int y, uint8_t pct, uint16_t color) {
|
||||
gfx->drawRoundRect(8, y, 112, 12, 4, C_DIM);
|
||||
int w = 108 * pct / 100;
|
||||
if (w > 0) gfx->fillRoundRect(10, y + 2, w, 8, 3, color);
|
||||
}
|
||||
|
||||
static void drawStatusBar() {
|
||||
char buf[16];
|
||||
gfx->setFont(F_SMALL);
|
||||
gfx->setTextColor(C_DIM);
|
||||
gfx->setCursor(3, 10);
|
||||
if (timeIsValid()) {
|
||||
time_t now = time(nullptr);
|
||||
strftime(buf, sizeof(buf), "%H:%M", localtime(&now));
|
||||
gfx->print(buf);
|
||||
} else {
|
||||
gfx->print("--:--");
|
||||
}
|
||||
// Wi-Fi bars
|
||||
bool wifi = WiFi.status() == WL_CONNECTED;
|
||||
for (int i = 0; i < 3; i++) {
|
||||
int h = 3 + i * 2;
|
||||
gfx->fillRect(108 + i * 4, 10 - h, 3, h, wifi ? C_GREEN : C_FAINT);
|
||||
}
|
||||
if (!wifi && (millis() / 500) % 2) gfx->fillRect(108, 8, 3, 2, C_AMBER);
|
||||
// SD card
|
||||
uint16_t sd = app.state == ST_NO_SD ? C_RED : C_DIM;
|
||||
gfx->fillRect(97, 3, 7, 8, sd);
|
||||
gfx->fillTriangle(102, 3, 104, 3, 104, 5, C_BG);
|
||||
gfx->drawFastHLine(0, 14, 128, C_FAINT);
|
||||
}
|
||||
|
||||
// "mm:ss" under an hour, "h:mm:ss" from then on.
|
||||
static void formatDuration(char *buf, size_t len, uint32_t secs) {
|
||||
if (secs >= 3600) snprintf(buf, len, "%lu:%02lu:%02lu", secs / 3600, secs / 60 % 60, secs % 60);
|
||||
else snprintf(buf, len, "%02lu:%02lu", secs / 60, secs % 60);
|
||||
}
|
||||
|
||||
static void footer(const char *s) { textCentered(s, 124, F_SMALL, C_DIM); }
|
||||
|
||||
// Short label for a recording path: "/recordings/REC_2026-10-02_14-30-05.wav" -> "10-02 14:30:05"
|
||||
static String recLabel(const char *path) {
|
||||
const char *p = strstr(path, "REC_");
|
||||
if (!p || strlen(p) < 23) return String(path);
|
||||
String s = String(p + 9, 5) + " " + String(p + 15, 8);
|
||||
s.replace('-', ':');
|
||||
s.setCharAt(2, '-');
|
||||
return s;
|
||||
}
|
||||
|
||||
// ---------- screens ----------
|
||||
static void drawScreen(uint32_t t) {
|
||||
char buf[48];
|
||||
switch (app.state) {
|
||||
case ST_BOOT:
|
||||
drawMic(64, 52, C_BLUE);
|
||||
textCentered("VoiceNotes", 92, F_MED, C_TEXT);
|
||||
textCentered("Connecting Wi-Fi...", 108, F_SMALL, C_DIM);
|
||||
break;
|
||||
|
||||
case ST_NO_SD:
|
||||
gfx->fillRoundRect(50, 30, 28, 36, 4, C_RED);
|
||||
gfx->fillTriangle(70, 30, 78, 30, 78, 38, C_BG);
|
||||
textCentered("No microSD", 88, F_MED, C_TEXT);
|
||||
textCentered("Insert a FAT32 card", 104, F_SMALL, C_DIM);
|
||||
break;
|
||||
|
||||
case ST_IDLE:
|
||||
drawMic(64, 46, C_DIM);
|
||||
textCentered("Ready", 84, F_MED, C_TEXT);
|
||||
if (app.lastRecording[0]) {
|
||||
snprintf(buf, sizeof(buf), "Last %s", recLabel(app.lastRecording).c_str());
|
||||
textCentered(buf, 100, F_SMALL, C_DIM);
|
||||
}
|
||||
footer("PLUS rec | hold: notes");
|
||||
break;
|
||||
|
||||
case ST_RECORDING: {
|
||||
if ((t / 500) % 2) gfx->fillCircle(46, 27, 5, C_RED);
|
||||
else gfx->drawCircle(46, 27, 5, C_RED);
|
||||
gfx->setFont(F_MED);
|
||||
gfx->setTextColor(C_RED);
|
||||
gfx->setCursor(56, 32);
|
||||
gfx->print("REC");
|
||||
|
||||
uint32_t sec = recorderSeconds();
|
||||
formatDuration(buf, sizeof(buf), sec);
|
||||
textCentered(buf, 66, F_BIG, C_TEXT);
|
||||
|
||||
// Rolling level meter, one bar per frame.
|
||||
levelHistory[levelPos] = app.micLevel;
|
||||
levelPos = (levelPos + 1) % 32;
|
||||
for (int i = 0; i < 32; i++) {
|
||||
float v = levelHistory[(levelPos + i) % 32];
|
||||
int h = max(1, (int)(v * 30));
|
||||
gfx->fillRect(i * 4, 100 - h / 2 - 1, 3, h, v > 0.85f ? C_AMBER : C_RED);
|
||||
}
|
||||
uint32_t left = app.maxSeconds - min(sec, (uint32_t)app.maxSeconds);
|
||||
if (left < 60) snprintf(buf, sizeof(buf), "BOOT stop | %lus left", left);
|
||||
else snprintf(buf, sizeof(buf), "BOOT: stop");
|
||||
footer(buf);
|
||||
break;
|
||||
}
|
||||
|
||||
case ST_SAVED:
|
||||
drawCheck(64, 44, C_GREEN);
|
||||
textCentered("Saved", 82, F_MED, C_TEXT);
|
||||
textCentered(recLabel(app.lastRecording).c_str(), 96, F_SMALL, C_DIM);
|
||||
formatDuration(buf, sizeof(buf), app.recordedBytes / (SAMPLE_RATE * 2));
|
||||
snprintf(buf + strlen(buf), sizeof(buf) - strlen(buf), " %.1f MB", app.recordedBytes / 1048576.0f);
|
||||
textCentered(buf, 106, F_SMALL, C_DIM);
|
||||
footer("hold PLUS: notes");
|
||||
break;
|
||||
|
||||
case ST_UPLOADING:
|
||||
// Arrow-up-into-cloud
|
||||
gfx->fillCircle(54, 42, 10, C_BLUE);
|
||||
gfx->fillCircle(70, 38, 13, C_BLUE);
|
||||
gfx->fillRoundRect(42, 40, 44, 14, 6, C_BLUE);
|
||||
gfx->fillTriangle(64, 30, 57, 40, 71, 40, C_BG);
|
||||
gfx->fillRect(62, 40, 5, 10, C_BG);
|
||||
textCentered("Uploading", 76, F_MED, C_TEXT);
|
||||
drawProgress(86, app.progress, C_BLUE);
|
||||
if (app.partCount > 1) snprintf(buf, sizeof(buf), "Part %u of %u %u%%", app.partIndex, app.partCount, app.progress);
|
||||
else snprintf(buf, sizeof(buf), "%u%%", app.progress);
|
||||
textCentered(buf, 110, F_SMALL, C_DIM);
|
||||
break;
|
||||
|
||||
case ST_TRANSCRIBING:
|
||||
drawSpinner(64, 44, C_CYAN, t);
|
||||
textCentered("Transcribing", 84, F_MED, C_TEXT);
|
||||
if (app.partCount > 1) snprintf(buf, sizeof(buf), "Part %u of %u", app.partIndex, app.partCount);
|
||||
else snprintf(buf, sizeof(buf), "speech -> text");
|
||||
textCentered(buf, 100, F_SMALL, C_DIM);
|
||||
break;
|
||||
|
||||
case ST_SUMMARIZING:
|
||||
drawSpinner(64, 44, C_PURPLE, t);
|
||||
textCentered("Summarizing", 84, F_MED, C_TEXT);
|
||||
if (app.received) snprintf(buf, sizeof(buf), "Claude: %lu chars", app.received);
|
||||
else snprintf(buf, sizeof(buf), "Claude is thinking...");
|
||||
textCentered(buf, 100, F_SMALL, C_DIM);
|
||||
break;
|
||||
|
||||
case ST_SYNCING:
|
||||
drawSpinner(64, 44, C_GREEN, t);
|
||||
textCentered("Saving", 84, F_MED, C_TEXT);
|
||||
textCentered("to Nextcloud", 100, F_SMALL, C_DIM);
|
||||
break;
|
||||
|
||||
case ST_DONE:
|
||||
drawCheck(64, 44, C_GREEN);
|
||||
textCentered("Notes saved", 82, F_MED, C_TEXT);
|
||||
textWrapped(app.detail, 96, C_DIM, 2);
|
||||
break;
|
||||
|
||||
case ST_ERROR:
|
||||
drawCross(64, 40, C_RED);
|
||||
textCentered("Error", 74, F_MED, C_TEXT);
|
||||
textWrapped(app.detail, 88, C_DIM, 4);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------- LED ring ----------
|
||||
static void spin(CRGB color, uint32_t t, uint16_t periodMs, int comets) {
|
||||
fadeToBlackBy(leds, NUM_LEDS, 70);
|
||||
int head = (t / (periodMs / NUM_LEDS)) % NUM_LEDS;
|
||||
for (int c = 0; c < comets; c++) leds[(head + c * NUM_LEDS / comets) % NUM_LEDS] = color;
|
||||
}
|
||||
|
||||
static void updateLeds(uint32_t t) {
|
||||
uint32_t inState = millis() - app.stateSince;
|
||||
switch (app.state) {
|
||||
case ST_BOOT:
|
||||
spin(CRGB(255, 120, 0), t, 800, 1);
|
||||
break;
|
||||
case ST_NO_SD:
|
||||
fill_solid(leds, NUM_LEDS, (t / 700) % 2 ? CRGB(120, 0, 0) : CRGB::Black);
|
||||
break;
|
||||
case ST_IDLE: {
|
||||
uint8_t b = beatsin8(12, 4, 28); // slow "breathing"
|
||||
CRGB c = WiFi.status() == WL_CONNECTED ? CRGB(b / 2, b / 2, b) : CRGB(b, b / 2, 0);
|
||||
fill_solid(leds, NUM_LEDS, c);
|
||||
break;
|
||||
}
|
||||
case ST_RECORDING: {
|
||||
// Dim red base, plus a VU meter that grows around the ring with the voice level.
|
||||
fill_solid(leds, NUM_LEDS, CRGB(beatsin8(40, 10, 40), 0, 0));
|
||||
float lit = app.micLevel * NUM_LEDS;
|
||||
for (int i = 0; i < NUM_LEDS; i++) {
|
||||
if (i < (int)lit) leds[i] = CRGB(255, 0, 0);
|
||||
else if (i == (int)lit) leds[i] = CRGB((uint8_t)(255 * (lit - i)), 0, 0);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case ST_SAVED: {
|
||||
uint8_t b = inState < 1500 ? 255 - inState / 6 : 5;
|
||||
fill_solid(leds, NUM_LEDS, CRGB(0, b, 0));
|
||||
break;
|
||||
}
|
||||
case ST_UPLOADING: {
|
||||
// Fill the ring with overall progress across all parts.
|
||||
uint8_t parts = max((uint8_t)1, app.partCount);
|
||||
uint8_t done = app.partIndex ? app.partIndex - 1 : 0;
|
||||
int full = (done * 100 + app.progress) * NUM_LEDS / (parts * 100);
|
||||
for (int i = 0; i < NUM_LEDS; i++)
|
||||
leds[i] = i < full ? CRGB(0, 60, 255) : i == full && (t / 250) % 2 ? CRGB(0, 30, 120) : CRGB::Black;
|
||||
break;
|
||||
}
|
||||
case ST_TRANSCRIBING:
|
||||
spin(CRGB(0, 200, 220), t, 900, 1);
|
||||
break;
|
||||
case ST_SUMMARIZING:
|
||||
spin(CRGB(150, 0, 255), t, 1200, 2);
|
||||
break;
|
||||
case ST_SYNCING:
|
||||
spin(CRGB(0, 220, 80), t, 900, 1);
|
||||
break;
|
||||
case ST_DONE:
|
||||
fill_solid(leds, NUM_LEDS, CRGB(0, beatsin8(30, 20, 200), 40));
|
||||
break;
|
||||
case ST_ERROR:
|
||||
fill_solid(leds, NUM_LEDS, (t / 200) % 2 && inState < 2400 ? CRGB(255, 0, 0) : CRGB(40, 0, 0));
|
||||
break;
|
||||
}
|
||||
FastLED.show();
|
||||
}
|
||||
|
||||
void uiUpdate() {
|
||||
static uint32_t lastFrame = 0;
|
||||
uint32_t t = millis();
|
||||
if (t - lastFrame < 50) return;
|
||||
lastFrame = t;
|
||||
|
||||
updateLeds(t);
|
||||
|
||||
gfx->fillScreen(C_BG);
|
||||
drawStatusBar();
|
||||
drawScreen(t);
|
||||
SpiLock lock;
|
||||
gfx->flush();
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
// 0.85" LCD screens and LED-ring animations for each app state.
|
||||
#pragma once
|
||||
|
||||
bool uiBegin(); // LCD + LED ring
|
||||
void uiUpdate(); // call every loop(); redraws at ~20 fps
|
||||
Reference in New Issue
Block a user