Compare commits

..
50 Commits
Author SHA1 Message Date
Kristian WindandClaude Opus 5 c8d67dd07d feat(notes): notekroppe forsegles, og søgningen åbner dem selv
CI / OpenAPI is valid (push) Successful in 38s
CI / Go (push) Failing after 18m36s
CI / End-to-end (push) Failing after 18m54s
CI / Docker image builds (push) Successful in 7m27s
Documentation / build (push) Failing after 7m7s
Documentation / deploy (push) Skipped
Release / Build and publish image (push) Failing after 9m59s
Grunden er den samme som for postkassers kodeord og OAuth-tokens: en
sikkerhedskopi kan hentes gennem fladen, og indtil nu bar hver eneste af dem hver
eneste note i klartekst. Noter er dér, folk skriver kodeord og API-nøgler ned — det
ved vi, fordi der ligger sådan nogle i den installation, det her blev bygget til.

**Både titel og krop.** Titlen *er* første linje, så en forseglet krop ved siden af
en læselig titel ville efterlade begyndelsen af hver eneste note i klartekst.

**Og de to kopier gik med.** Det var derfor det ikke var gjort for længe siden:
`notes_fts` indekserede kroppen, og den genererede `fold`-søjle bar en
translittereret kopi af den. At forsegle kroppen og lade dem stå ville have været
teater — en angriber med filen kunne læse hver note ud af indekset. Begge er væk,
og tabellen er bygget om, fordi SQLite ikke kan fjerne en genereret søjle på plads.

**Søgningen åbner teksten i Go.** Målt frem for gættet: tolv hundrede noter à 3,3 kB
koster 33–43 ms pr. søgning, og fladen venter 250 ms efter sidste tastetryk, før den
spørger — så det ligger inde i pausen. Tiden går ikke med at åbne konvolutterne;
fire megabyte AES-GCM er få millisekunder. Den går med at folde og småskrive hver
krop, og det er lineært: regn med omtrent 350 ms ved ti tusind noter. Dér er det den
forkerte form, og svaret er da et nøglet indeks — hvis pris er, at præfiks- og
delordssøgningen falder bort.

bm25 forsvandt med indekset. Det, der er beholdt, er den halvdel, der betød mest:
titlen vejer ti gange kroppen. Med et loft pr. ord, så en note, der nævner ordet
fyrre gange, ikke slår en, der nævner alle ordene én gang — det er den fejl, en ren
optælling laver, og den er let at lave og svær at få øje på.

**En bagudfyldning ved opstart, ikke efterhånden.** Instansens indstillinger
forsegles, når de skrives, og det er rigtigt for dem. For et arkiv på tolv hundrede
importerede noter, som ingen redigerer igen, betyder "når de skrives" aldrig — og
det er præcis dét arkiv, der er værd at forsegle. Idempotent, så den koster et
gennemløb af id'er hver anden gang, og `updated_at` røres ikke: at lukke en
konvolut om en note er ikke at skrive den, og et stempel med i aften ville kaste
den rækkefølge væk, importen brugte en dag på at hente hjem.

En fejl under bagudfyldningen stopper opstarten. Halvdelen af noterne lukket er
ikke i stykker — `v1:`-præfikset gør, at begge slags kan læses — men en instans,
der starter og lader som om den beskytter noget, den ikke gør, er værre end en, der
siger fra.

Hvad der stadig ligger åbent, står i SECURITY.md og på dokumentationssiden:
opgavetitler, beskrivelser, kommentarer, og `note_links`, som røber hvilke
projekter og opgaver en note peger på, men ikke hvad der står.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 22:40:55 +02:00
Kristian WindandClaude Opus 5 23bd784b07 docs: SECURITY.md sagde noget, dagen havde gjort forkert
"Ingen ratebegrænsning ud over login og nulstilling" var sandt i morges. Det er
det ikke længere: andenfaktoren, beaconet og indgående post har fået deres eget
loft i løbet af dagen — det sidste tilføjede jeg selv og skrev ikke det her.

En SECURITY.md, der siger noget forkert, er værre end de fleste fejl. Den er
netop den fil, nogen læser for at afgøre, om de tør køre det her, og hvert eneste
punkt i den er en påstand, de ikke selv kan efterprøve uden at læse koden.

Samme sted mangler den største åbne ting: notekroppe, opgavetitler, beskrivelser
og kommentarer er ikke krypteret. Forseglingen dækker legitimation — postkassers
kodeord, OAuth-tokens, andenfaktorens hemmelighed, AI-nøgler — og intet andet. En
note med et kodeord i har det i klartekst i hver sikkerhedskopi, man kan hente.
Det stod i overleveringen og på dokumentationssiden for noter, og det hørte
hjemme her.

README's "ikke færdigt"-liste havde Gmail med. Den kører i produktion, inklusive
flagningen og dubletbehandlingen, der tog en dag at få rigtig.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 22:16:52 +02:00
Kristian WindandClaude Opus 5 264f48ae11 fix(quickadd): "hver anden tirsdag" blev til en dato og to ord i titlen
Release / Build and publish image (push) Failing after 10m3s
Vendingen er den, folk bruger, og recurrence-pakken har kunnet den siden den blev
skrevet — den oversætter "hver anden" til "hver 2." netop for at slippe for at
skrive hvert eneste mønster i to udgaver.

Sætningen nåede bare aldrig frem til den. Mønsteret i hurtig tilføjelse tog et tal
efter "hver", men ikke et ord, så det, der blev givet videre, var "hver anden".
Det er ikke en regel. Så blev de to ord stående i titlen, "tirsdag" blev læst som
en forfaldsdato, og opgaven gentog sig ikke — uden at noget sagde det.

Sløjfen, der prøver sig frem, kunne ikke redde den: den fjerner ord fra enden, én
ad gangen, og kan ikke hente et ord, mønsteret aldrig tog med.

Fundet af en prøve, jeg skrev til noget andet, hvor jeg brugte vendingen som en
selvfølge og opdagede, at den ikke var det.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 22:11:01 +02:00
Kristian WindandClaude Opus 5 2219f6af13 fix(i18n): en gentagelse siges nu på læserens sprog
Release / Build and publish image (push) Failing after 10m2s
`recurrence.Describe` skriver dansk. Fladen kunne navngive de fem almindelige
regler selv, og alt andet — "hver mandag", "hverdage", "hver anden uge tirsdag" —
faldt tilbage til serverens tekst, som er dansk uanset hvad kontoen er sat til.

Serveren kan ikke bare tage en locale med, og det er ikke dovenskab: `toTaskJSON`
har niogtyve kaldesteder, og mange af dem er WebSocket-udsendelser, hvor én
nyttelast når alle i et projekt på én gang. Der findes ikke ét sprog at skrive den
i.

Men reglen er den samme, uanset hvem der læser den. Den ligger allerede i
nyttelasten, så beskrivelsen hører hjemme dér, hvor læseren er — i fladen.

Serverens tekst bliver stående som reserve. Den er stadig den, et API-kald og en
MCP-forbindelse får, og de har ingen flade at oversætte i.

Ugedagen kommer fra `Intl` og ikke fra en tabel — ikke for at spare syv strenge pr.
sprog, men for at få de navne, sproget faktisk bruger. Dagene sorteres, som ugen
går, frem for som reglen tilfældigvis skrev dem: "mandag og torsdag" og "torsdag og
mandag" er den samme regel, og den ene af dem er den, folk siger.

Og to har sit eget ord på begge sprog — "hver anden", "every other". Det er ikke
tallet to sat ind i den samme sætning, så det er sin egen nøgle. Et sprog, der
bøjer enheden efter tallet, kan ikke sættes sammen af stumper; derfor står tallet
inde i teksten frem for at blive limet på.

Prøven bruger `hver mandag` og `hverdage`, fordi de netop *ikke* står på listen
over de fem, fladen altid har kunnet. En tredje vending, jeg først skrev prøven
med — "hver anden tirsdag" — viste sig ikke at være en, hurtig tilføjelse
overhovedet læser: den lagde "hver anden" i titlen og gjorde "tirsdag" til en dato.
Det er en anden sag, og den står nu skrevet ét sted, hvor den kan tages op.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 22:00:05 +02:00
Kristian WindandClaude Opus 5 f64b7c1f01 docs: Noter og Kalender havde ingen sider, mens forsiden lovede begge
Landingssiden reklamerer for noter og for en kalender, og dokumentationen havde
hverken en side om den ene eller den anden. Det hul blev skrevet ned i morges og
er det her.

**Noter** siger, hvad der er værd at vide, og hvad der ikke virker: at første linje
*er* titlen og udledes ved hver gemning, at arkivet og papirkurven er to
forskellige ting, at søgningen finder `grøn` når man skriver `gron` og
`keepInventory` når man skriver `inventory` — og at notekroppe ligger i klartekst i
hver sikkerhedskopi, man kan hente. Det sidste hører hjemme dér, hvor nogen står
med en note og overvejer at skrive et kodeord i den.

**Kalender** samler det, der før var spredt: gitteret, ugen som et døgn, træk der
sætter et klokkeslæt, abonnementer på ICS og webcal, og Google-forbindelsen med
det forbehold, den skal have — den er skrevet efter dokumentationen og har aldrig
kørt mod Google. `caldav.md` er nu den anden retning alene: verdandes egne opgaver
læst af noget andet.

En ting, byggeriet lærte: `</>` i inline-kode vælter Pythons html-parser på 3.9
med "we should not get here!". CI kører 3.12 og er ligeglad, men en side, der ikke
kan bygges lokalt, er en side ingen retter. Knappen er beskrevet med ord i stedet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 21:50:27 +02:00
Kristian WindandClaude Opus 5 868588de10 feat(projects): et projekts kalender kan vises som en uge
Ugen fandtes på Kommende og på Kalender, men ikke dér hvor et projekt bor — og det
er dér, "hvornår sker de her fire ting" oftest bliver spurgt.

Inde i kalendervisningen frem for som en fjerde `view_mode`. `view_mode` er
projektets og delt med alle, der kan se det, mens uge eller måned er et spørgsmål
om hvor bred skærmen er, foran hvilken der sidder én person — samme grund som
sidebjælkens bredde ligger i localStorage af. Og en fjerde værdi ville kræve en
tabelombygning: `CHECK (view_mode IN ('list','board','calendar'))` kan SQLite ikke
ændre på plads.

**Og så fandt den en fejl, der har ligget og ventet.** `cursor` blev sat med
`span === 'week' ? … : …`, hvilket læser `span` i det øjeblik komponenten bliver
til og fanger den værdi for altid. Byggeriet har advaret om det lige så længe. Så
længe hver visning havde sin egen komponent, gjorde det ingen forskel — og det
gjorde det i samme sekund, en vælger begyndte at skifte `span` på den samme.
Markøren stod på den 1. i måneden, og et skift til uge landede på ugen omkring den
1.: "27. jul.–2. aug." for en person, der kiggede på august.

Rettet, med den ene undtagelse skrevet ned: en månedsvisnings markør *er* den
første i måneden, så et skift til uge, mens man står i den indeværende måned,
skal lande på den uge, i dag ligger i — ikke på ugen omkring den 1.

Og en anden fælde, jeg selv gik i undervejs: en underklasses felter sættes *efter*
basens konstruktør er kørt. Nøglen til localStorage som et felt på en underklasse
betød, at valget blev gemt ét sted og læst et andet — en uge, man havde valgt, var
en måned igen efter en genindlæsning. Nøglen er et argument nu; et argument har
ingen rækkefølge at tage fejl af.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 21:45:38 +02:00
Kristian WindandClaude Opus 5 dcb71f4c14 feat(calendar): abonnement på flere kalendere, ICS og webcal
Release / Build and publish image (push) Failing after 10m3s
Bedt om som "abonnering af flere kalendere (ics og webcal?)". Det er også den
eneste vej for en privat konto: instansens OAuth-klient er Internal, og Google
afviser en konto udefra med `org_internal`, uanset hvad der ellers er sat op.

**Samme tabeller.** Et abonnement er én kalender med begivenheder i, og det er
præcis det, `calendars` og `calendar_events` allerede beskriver — med vinduet,
farven, `shown` og hele fejemaskinen omkring sig. En tabel til ville være en anden
vej til det samme sted, og to veje til ét sted er to steder at rette den dag noget
om begivenheder ændrer sig.

Det, der skulle laves om, var identiteten. En OAuth-konto er én pr. person og
udbyder — tokenet er identiteten — mens et abonnement er én pr. adresse, og man har
flere. `calendar_accounts_once` er derfor delvist nu: det holder for rækkerne uden
adresse, og et andet indeks holder adresserne unikke. Delvise indekser er den ene
ting, SQLite kan her, som ellers krævede en tabelombygning.

**`webcal://` skrives om til `https://`.** Det er ikke en protokol, noget taler —
det er en konvention, der får et skrivebord til at give adressen til
kalenderprogrammet i stedet for til browseren, og alle udgivere skriver deres links
sådan. At afvise den ville være at afvise folk på en teknikalitet.

**Adressen gennem `safedial`, begge halvdele.** Afvist hvor den skrives ned, så
den, der står med feltet, får svaret med det samme — og igen på den *opslåede*
adresse hver gang den hentes, fordi et navn, der blev godkendt ved indtastningen,
kan svare 127.0.0.1, når fejemaskinen kører klokken tre om natten. Det er den
sætning, `internal/safedial` selv står og advarer om, og den blev skrevet efter et
SSRF-fund i morges.

**Hentet én gang, før den gemmes.** En adresse, der ikke kan læses, er en tastefejl
langt oftere end en server, der tilfældigvis er nede, og at sige det, mens nogen
står med feltet, er de to sekunder værd. Et abonnement, der lydløst aldrig fyldes,
er den samme tavshed, notifikationerne havde.

Læseren er ny i `internal/ics`, som hidtil kun kunne skrive. To ting, den med vilje
ikke gør, står skrevet ved den: den udfolder ikke gentagelser, og den opløser ikke
tidszoner. Begge dele er større end de ser ud, og begge har ét sted at høre til
den dag de skal.

En ting, formatet lærte: DTEND er eksklusiv for en heldagsbegivenhed. Gemmes den,
som filen skriver den, dækker hver eneste helligdag også dagen efter sig.

Én adresse, der fejler, stopper ikke de andre. Et abonnement er en anden persons
server, og den ordinære tilstand for en anden persons server er, at den er i orden
indtil en morgen hvor den ikke er.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 21:26:13 +02:00
Kristian WindandClaude Opus 5 968a29ca67 feat(notes): listen står i grupper, og projektet har sin egen farve
Release / Build and publish image (push) Failing after 9m56s
Meldt ind som "den er for tæt på en eller anden måde". Grunden var ikke afstanden
mellem rækkerne: hver række sagde tre ting, og to af dem gentog sig selv nedad.
Datoen stod på hver eneste række — også når tyve noter deler dag — og projektet
fik en linje for sig selv. Tre linjer pr. note, gange tolv hundrede.

Gentagelserne er flyttet derhen, hvor de siger noget.

**Datoen er blevet til overskriften.** I dag · I går · Denne uge · August 2026 …
og så er den væk fra rækken. Noten går fra tre linjer til to, og luften kommer af,
at der er mindre at skrive — ikke af at listen bliver længere.

**Favoritterne har fået deres egen gruppe.** De lå allerede først; der stod bare
ikke hvorfor.

**Grupperingen følger sorteringen.** Sorteret på "Oprettet" grupperes der på
oprettelsesdatoen, på "Senest rørt" på den — ellers ville listen stå i én
rækkefølge og være grupperet efter en anden, og så er overskrifterne løgn.
Sorteret på navn er der ingen dato at gruppere på, så det bliver forbogstavet, og
datoen kommer tilbage i rækken, hvor den så er det eneste tilbage, der siger noget.

**Arkivet grupperes med.** Det er dér, de tolv hundrede importerede ligger, og
dér, det betyder mest.

**Projektet har sin egen farve** — den samme prik som i sidebjælken. Mærkaten sagde
det samme som alle de andre i en grå, der var lysere end uddraget, og projektet er
dét, man skanner efter, når man leder i tolv hundrede noter. Prikken bærer farven
og ikke teksten: fem projektfarver som skriftfarve er fem forskellige grader af
læsbarhed, og den lyseste er ikke læsbar.

Fire ting, der blev fundet ved at se på det:

- Uddraget gentog titlen, fordi titlen *er* første linje. Det var skjult, så længe
  en dato stod imellem dem.
- Uddraget klippes ikke af sig selv. Det lå før i en flex-boks, der ejede
  afkortningen; da datoen flyttede op, forsvandt boksen, og sætningen løb videre
  ind under favoritstjernen.
- Overskriftens stjerne hed `.star` ligesom rækkens favoritknap og arvede dens
  gennemsigtighed. Den stod der aldrig. To ting, der ligner hinanden, er ikke det
  samme.
- En `aria-label` på foldeknappen erstatter knappens indhold som dens navn — og
  knappen *er* overskriften, så overskriften kom til at hedde "Fold I dag sammen".
  Sidebjælkens gruppehoveder gør det rigtigt, og gør det nu begge steder.

Sammenfoldning huskes ikke på tværs af besøg: en gruppe, der åbner foldet uden at
nogen bad om det, er en liste, der ser tom ud.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 21:01:40 +02:00
Kristian WindandClaude Opus 5 fcd6dca852 fix(push): "der kommer ikke nogen" kunne ikke besvares
Release / Build and publish image (push) Failing after 10m0s
Alt om det var usynligt. Indstillingssiden sagde "til for denne enhed", så snart
der fandtes et abonnement — også når hver eneste push blev afvist. Og en afvist
push gik i en logline på **Debug**, som ikke skrives på en instans, der kører på
standardniveau. Så en levering, der fejler lydløst, og en levering, der aldrig
sker, så ens ud, og forskellen er hele diagnosen.

Tre forskellige problemer havde ét udseende: browseren tilmeldte sig aldrig,
tjenesten afviste den, eller der var ikke noget forfaldent endnu.

**En prøveknap** under Indstillinger → Notifikationer sender én notifikation
gennem den rigtige rute — ikke en genvej, så det, den beviser, er det, der
faktisk skal virke — og siger hvad hver enhed gjorde ved den. Kom den ikke frem,
og afviste ingen den, står der, at det så er browseren eller styresystemet, der
holder den tilbage. Nu peger svaret på ét af de tre steder.

Loglinjen er hævet fra Debug til Warn. Det var den ene linje, der ville have sagt
hvorfor.

Teksten kommer fra klienten, fordi ordbøgerne gør. En server, der bærer sin egen
kopi af "det her er en prøve", bærer den på ét sprog.

**Og en lækage, prøven fandt.** Go pakker en transportfejl som `*url.Error`, og
dens tekst er `Post "https://fcm.googleapis.com/fcm/send/dQw4…": dial tcp: …` —
hele endepunktet, som er den legitimation, push-tjenesten udstedte til den enhed.
Den var på vej ud i svaret, i loglinjen og i ethvert skærmbillede af nogen af dem.
Prøven var skrevet for at bevise, at kun værten blev nævnt, og det var sådan den
blev fundet. Fejlen pakkes ud, og adressen skrubbes bagefter alligevel: den næste
fejltype, der bærer den med, kommer ikke med en advarsel.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 20:41:33 +02:00
Kristian WindandClaude Opus 5 ae673086bc feat(tasks): tid på en opgave, og et træk i kalenderen sætter den
Release / Build and publish image (push) Failing after 9m58s
Bedt om: "jeg vil have tid på opgaver, og jeg vil kunne justere dem i kalenderen."

Serveren kunne det hele i forvejen — `due_date` og `due_time` læses sammen og
bliver til et `due_datetime` i kontoens tidszone. Det var fladen, der ikke havde
nogen vej til at sige det.

**Et felt i opgaveruden.** Tomt, indtil nogen sætter det: en opgave uden
klokkeslæt er det almindelige, og et felt, der åbner på 00:00, påstår, at noget
skal ske ved midnat.

**Og en fejl, der lå og ventede.** Serveren læser en dato uden en tid som "ryd
tiden". Så et gem af datoen alene tog klokkeslættet med sig — man rettede dagen og
mistede timen — og det samme gjorde hvert eneste træk i kalenderen. Ingen af
delene sagde noget.

`reschedule` siger nu tiden højt, tre måder, fordi de to første ligner hinanden i
JavaScript og derfor skal skilles ad ét sted frem for at blive forvekslet fire:

    undefined   behold den, opgaven har   — månedens ruder, hvor der ingen tid er at læse
    ''          ryd den                   — heldagsbåndet
    'HH:MM'     sæt den                   — et sted på døgnet

**Hvor på søjlen man slipper, er klokkeslættet.** Målt mod søjlens egen kasse frem
for mod siden, så det bliver rigtigt uanset hvor gitteret sidder og hvor langt der
er rullet. Rundet til kvarter: minuttet under markøren er ikke en oplysning, nogen
har — pegefingeren er bredere end et minut — og 14:07 er et tal, der ser ud som om
det blev valgt.

Prøven går hele vejen: sæt tiden, genindlæs og se at serveren tog den, ret datoen
og se at timen bliver, find opgaven på sit klokkeslæt i ugegitteret, og træk den op
i båndet igen. To ting, den lærte undervejs, står skrevet ved den: et træk over
fire hundrede pixels skal gå i skridt, ellers har browseren for få mellemstationer
til at holde gesten i live — og klokken skal være tidlig, for klokken fjorten
ligger under skærmkanten, og at måle noget dernede ruller siden væk under målet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 20:29:03 +02:00
Kristian WindandClaude Opus 5 e29fd04abd docs: noternes opbygning, skrevet til at blive løftet ud
Hvordan noter er bygget her: modellen, syntaksen, søgningens tre lag,
editorkontrakten, vedhæftninger, ind- og udførsel — og begrundelserne, som er den
del, der ikke kan læses ud af skemaet.

Den begynder med fjorten spørgsmål, der skal besvares før der skrives kode, og
siger det til den, der læser den, med rene ord: gæt ikke en standard. Grunden
står i filen — omtrent en tredjedel af designet her er arvet fra de omgivelser,
noterne bor i, og løftet ind i et program med andre omgivelser uden at afgøre
hvad der skal med, giver noget, der virker en uge og derefter slås med én.

Den siger også, hvad der ikke virker: notekroppe i klartekst i hver
sikkerhedskopi, import der aldrig fletter, ingen sync. En specifikation, der kun
beskriver det, der virker, lyver ved fortielse.

I repoet frem for uden for det, som HANDOFF.md og NOTES-PLAN.md er. De to er
arbejdspapirer om *dette* program; den her er beskrivelsen af en del af det, og
den bliver kun forkert, hvis koden flytter sig uden at den følger med — hvilket
er lettere at opdage, når den ligger ved siden af.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 20:02:59 +02:00
Kristian WindandClaude Opus 5 be581eb8ae fix(search): et ord inde i et ord kunne ikke findes
Release / Build and publish image (push) Failing after 9m57s
Meldt ind som "søgning har ikke det store held": en note med
`gamerule keepInventory true` kunne ikke findes ved at skrive "keep inventory".

Et indeks er bygget af tokens, og et token er det, en tokenizer besluttede var et
ord. `keepInventory` er ét. Så "keep inventory" spørger efter to ord, der ikke
findes nogen steder, og søgningen svarer, at der ikke er noget — mens noten står
lige der.

Præfikssøgningen tager halvdelen af det. `keep*` rammer faktisk
`keepinventory`; `inventory*` kan ikke, for det er ikke et præfiks af noget. Og
de to led er bundet sammen med AND, så den halvdel, der virker, bliver kasseret af
den, der ikke kan.

Det er ikke et sært tilfælde i en notesapp. Ti års noter er fulde af kommandoer,
identifikatorer og filnavne, og hver eneste af dem er flere ord, indekset tror er
ét: `keepInventory`, `VACUUM INTO`, `apple-notes-til-markdown`.

Indekset svarer først. Har det ingenting, læses teksten: hvert ord skal stå et
sted i noten, hvor som helst, i hvilket som helst ord. `fold` er
`title || ' ' || body` med de danske bogstaver translittereret, så én søjle dækker
titlen, brødteksten og "gron" for "grøn" på én gang.

Opgavesøgningen har fået den samme arm. At have den i den ene af de to ville være
værre end at have den i ingen af dem: "den finder det i mine noter, men ikke i
mine opgaver" er den næste melding, og svaret på den ville være denne linje.

Hvad det koster, og hvornår formen er forkert, står skrevet ved koden: en LIKE med
et indledende % kan ikke bruge et indeks, så den læser hver note. Ved tolv hundrede
noter er det fire megabyte og et par millisekunder, og den kører kun, når den
hurtige vej allerede kom tomhændet tilbage — altså netop den søgning, der var ved
at skuffe nogen. Ved hundrede tusind er svaret et FTS5-`trigram`-indeks, ikke en
større scanning.

Prøverne er set fejle uden rettelsen, og de dækker begge veje: "inventory" alene,
som ingen præfikssøgning kan nå, og "Bedrock", som stadig skal besvares af
indekset — ellers er rangeringen, der blev bygget til tolv hundrede noter, stille
forsvundet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 18:33:22 +02:00
Kristian WindandClaude Opus 5 9d34b57166 feat(calendar): ugen er et døgn, og dagene er lige brede
Release / Build and publish image (push) Failing after 9m58s
To ting, meldt ind fra en uge, der ikke svarede på det, man åbnede den for.

**Dagene var ikke lige brede.** Gitteret stod på `repeat(7, 1fr)`, og `1fr` er
`minmax(auto, 1fr)`: `auto` som minimum betyder, at en søjle ikke kan blive
smallere end sit længste ord. Én opgave, der hed "Gårdbutikken på Møllerup Gods —
opfølgning på mail", gjorde onsdag dobbelt så bred som mandag og klemte resten af
ugen sammen om den. `minmax(0, 1fr)` lader søjlen blive så smal, den skal, og så
klipper brikken sin tekst af — hvilket den allerede kunne.

**Ugen var syv punktopstillinger.** Den svarede på "hvad skal der ske torsdag" og
ikke på "hvornår", og det andet er det, man åbner en uge for. To møder klokken ti
er en konflikt, man skal kunne se, ikke læse sig til.

Nu et gitter med en tidsrende og syv dage: timerne ned ad siden, begivenheder
placeret efter klokkeslæt, og et bånd foroven til det, der ikke har et — en
heldagsbegivenhed lagt klokken nul ville påstå, den holdt fra midnat til midnat.

Overlappende ting deler bredden i sammenhængende klynger frem for parvist:
overlapper A med B og B med C, skal alle tre dele, også når A og C ikke rører
hinanden. Ellers ville A og C hver tro, de havde halvdelen, og lægge sig oven på
hver sin halvdel af B.

Hvilke timer der tegnes, kommer fra ugens egne begivenheder med arbejdsdagen som
gulv. Et helt døgn er fireogtyve rækker, hvoraf de otte altid er tomme, og så er
gitteret noget man ruller i frem for noget man ser — men en vagt klokken fem skal
med, uden at alle andre uger betaler for den med tomme rækker.

Måneden er ikke rørt ud over søjlebredden. En dag i en månedsrude er halvanden
centimeter høj; et døgn tegnet i den er ikke en visning, det er en streg.

Brikkerne er flyttet ud i to snippets, som begge visninger tegner med. De viser
de samme to ting og skal vise dem ens; det er placeringen, der er forskellig, og
så er det kun placeringen, der står to steder.

`data-date` følger med over på ugens timesøjler — det er sådan noget uden for
komponenten peger på en dag. Båndet får sit eget `data-allday`, så de to ikke
bliver til det samme svar på ét spørgsmål.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 17:01:58 +02:00
Kristian WindandClaude Opus 5 4b3abc44fb feat(site): hvad der er landet, og hvad man tager på sig ved at køre det
To dele.

**Lately** siger, hvad der er kommet til, fem linjer med versionsnummer i egen
søjle. En landingsside kan påstå, at et projekt er i live; en liste over hvad der
faktisk landede siger det bedre. Ikke en changelog — den ville gå i stå — så den
peger på release-noterne, som skrives alligevel.

**At your own risk** siger det, MIT-licensen siger, og som ingen læser: uden
garanti af nogen art. Dertil det, der er sandt om netop dette: én persons projekt,
version 0.x, du hoster det selv, så data og sikkerhedskopier er dine — og en
sikkerhedskopi, ingen har gendannet, er ikke en sikkerhedskopi endnu. At der er
lavet to gennemgange af koden, at hvert fund er skrevet ned og lukket, og at
**ingen udefra har auditeret det**.

Sat i samme skriftstørrelse som alt andet på siden. Et forbehold med mindre skrift
end resten er skrevet for at kunne peges på bagefter, og det er den ene ting,
siden ikke skal gøre.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 16:19:44 +02:00
Kristian WindandClaude Opus 5 b87c9ea6e2 fix(auth): en afvist passkey sagde "forkert e-mail eller adgangskode"
Release / Build and publish image (push) Failing after 10m0s
Der er hverken en e-mail eller en adgangskode i et passkey-login. Loginet er
discoverable — enheden ved selv, hvilken konto nøglen hører til, og der bliver
ikke tastet noget. Alligevel fik en afvisning fladens generiske 401-tekst, og den
nævner begge dele.

Det sendte en person ud at lede efter en tastefejl i sin konto, der aldrig havde
været der. Det, der faktisk var sket, var at instansen var flyttet fra http til
https: en passkey hører til præcis den adresse, den blev lavet på — `RPOrigins`
bygges af `VERDANDE_BASE_URL` med skema og alt — så et login på det ene skema mod
en server, der venter det andet, kan ikke verificeres.

Beskeden siger nu det, og siger også hvor man skal kigge. Serveren har hele tiden
svaret "that key was not accepted"; det var oversættelsen i fladen, der gjorde
den til noget andet.

Aflysning bliver behandlet som før — den er ikke en fejl, og en fejl for den ville
læse som om nøglen blev afvist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 15:59:34 +02:00
Kristian WindandClaude Opus 5 ce1d27f8ca fix(notes): én note, der ikke kunne tegnes, låste hele ruden
Release / Build and publish image (push) Failing after 9m53s
Meldt ind som "jeg kan ikke skifte mellem noter — uanset hvilken jeg trykker på,
åbner de ikke".

Editoren indlæste med `note.body.trim()` uden værn. `body` er en streng fra
serveren hver eneste gang — indtil den ikke er, og så kaster den her inde i et
`$effect`. En effekt, der kaster, tager komponentens opdateringer med sig: den
forrige note bliver stående, og *ingen* note kan åbnes bagefter. Klikket kan ikke
engang lande, hvilket er derfor det ikke ligner en fejl i noten, man klikkede på,
men i hele ruden.

To ting, ikke én. Værnet, så en manglende krop er en tom note. Og en ramme om
tegningen, så det næste, der en dag ikke kan tegnes, heller ikke bliver til det
her: en note, der fejler, viser sin egen tekst uden formatering, og resten af
ruden virker videre.

`loadedId` sættes nu før tegningen frem for efter. Ellers ville effekten prøve
igen ved hver eneste ændring bagefter og kaste hver gang.

`textContent` og ikke `innerHTML` i faldbakken: teksten er ikke tegnet, og at
lægge den ind som markup ville være at køre præcis det, tegningen nægtede.
Gemningen er ikke i fare — den skriver `draft`, som kommer fra svaret og ikke fra
det, der står i editoren, indtil nogen taster i den.

Prøven tvinger formen frem i stedet for at vente på den nyttelast, der gjorde
det. Hvad der end fik `body` til at mangle, er svaret det samme: én note må ikke
kunne låse de tolv hundrede andre ude. Den er set fejle uden rettelsen — med
netop den fejl, konsollen viste.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 15:32:36 +02:00
Kristian WindandClaude Opus 5 cab4e703f5 fix(sikkerhed): CalDAV lod en viewer skrive, og Web Push kunne ringe indad
Release / Build and publish image (push) Failing after 10m0s
To huller fundet ved en gennemgang efter dagens arbejde. Begge har en prøve, der
er set fejle før rettelsen.

CalDAV PUT afgjorde adgang på det projekt, klienten *nævnte i adressen*, og tog
derefter opgavens id fra den samme adresse. Intet bandt de to sammen. En viewer
på et delt projekt kunne pege på et projekt, de selv ejer — indbakken, som alle
har og alle må redigere — og lægge den delte opgaves id i filnavnet. Projekt-
tjekket gik godt på indbakken, og skrivningen landede på en opgave, de kun må
læse. GetTask lukkede det ikke: den svarer "må denne person se det", og det må en
viewer. `CanEdit` er spørgsmålet, og det er det samme, handleUpdateTask altid har
stillet — API-døren har spurgt hele tiden, CalDAV-døren spurgte ikke.

Opgaven skal nu også ligge, hvor adressen siger. Uden det kan en redaktør på to
projekter nå det ene gennem det andets adresse; intet eskaleres af det i sig
selv, men en adresse, der har lov at lyve om hvor noget bor, er sådan sagen
ovenfor kom ind.

Web Push blev leveret med en almindelig http.Client. Adressen kommer fra
browserens abonnement — altså fra den, der er logget ind — og serveren POSTer til
den på et skema, de selv vælger: sæt en påmindelse et minut ude, og jobbet går
igen hvert minut. Tilmeldingen afviser en privat adresse, og den kontrol er ikke
nok alene: den kører på URL'en som skrevet, og et navn, den godkender, kan svare
127.0.0.1 næste gang det slås op. Det er præcis den sætning, `internal/safedial`
selv står og advarer om. Leveringen går gennem safedial nu, som tjekker den
opslåede adresse ved forbindelsen og hvert viderestillingshop på samme måde.
Kontrollen ved tilmelding bliver, fordi den siger nej, mens nogen kigger på
skærmen.

Og en hærdning: /inbound/mail har fået et loft. Det er den svagere af de to
offentlige skriveveje — beaconet skriver en række for en fremmed, denne skriver
ingenting uden en 256-bit adresse, man skal kende i forvejen — men adressen deles
med mailservere og står på en indstillingsside, og når den først er kendt, var der
intet loft på, hvad der kunne laves med den. Tallet er sat generøst: modtageren er
mailserveren, én adresse der leverer det hele.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 15:15:28 +02:00
Kristian WindandClaude Opus 5 43f8c0ea85 feat(site): landingssiden siger hvad der findes nu, og siger det med kanter
Release / Build and publish image (push) Failing after 10m0s
Siden stod fra før kalenderen, noternes billeder og arkiv, sikkerhedsarbejdet og
beaconet. Nu står de der: søgning der læser dansk, kalenderen, Venter på andre,
gentagelser, AI slået fra som udgangspunkt, andenfaktor, at ingen skriver sig
selv op, og beaconet — sagt som det er, tændt fra start og med ét klik til at
slukke.

Google Kalender står med sit forbehold på selve kortet. Den er skrevet efter
dokumentationen og har aldrig kørt mod Google, og en landingsside, der siger
noget andet, er den ene sætning, resten af siden ikke kan bære.

Ikke-målene løj om posten: "No mail integration beyond Gmail and the forwarding
address" har været forkert, siden IMAP kom til. iCloud og Fastmail virker. Det,
der faktisk ikke virker, er Outlook, og det er nu det, der står.

Designet: en bjælke der bliver stående, fordi siden er 5.600 px og vejen ud kun lå
i toppen. Fire fakta under knapperne, hvor en landingsside plejer at sætte
logoer, den ikke har gjort sig fortjent til. Kort med egne kanter frem for celler
i ét gitter — ti celler i tre søjler er et hul i tegningen, atten kort i tre
grupper er ikke. To paneler under "Running it", fordi de er alternativer og ikke
trin. Ikke-målene som en liste, der kan tælles. Og tre skærmbilleder, der lå i
site/screenshots uden at være brugt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 14:57:15 +02:00
Kristian WindandClaude Opus 5 8ecefbbdc2 fix(notes): rullebjælken svævede midt i arket
Det er `.page`, der ruller — og det var også `.page`, der blev smallere for at
holde linjelængden nede. Så bjælken fulgte med ind til tekstsøjlens kant og stod
med en håndflade tom rude til højre for sig, en tredjedel inde i ruden.

Grænsen sættes som polstring i stedet for som bredde. Elementet er lige så bredt
som ruden — bjælken bliver ved kanten, hvor den hører til — og teksten er
nøjagtig lige så bred som før. `max()` mod nul, fordi en smal rude ellers ville få
negativ polstring, og `border-box` er sat globalt, så bredden er hele boksen og
ikke boksen plus polstringen; uden det ville det give en vandret rullebjælke.

Kildevisningen deler `.page` med editoren og havde samme fejl. Den følger med.

Målt: elementet slutter nu på rudens indholdskant i stedet for 74 px inde, og
`scrollWidth - clientWidth` er nul.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 14:57:15 +02:00
Kristian WindandClaude Opus 5 6b4448b245 fix(settings): AI, Gmail og kalenderen delte én række og slettede hinanden
Meldt ind som "AI-informationerne gemmes ikke". De blev gemt. De blev taget
bagefter.

`user_settings` blev oprettet med `user_id` som hele sin primærnøgle — én række
per person, uanset scope — og upserten skrev `scope` om ved konflikt. Der er tre
scopes i live: `ai`, `gmail` og `calendar`. De delte den ene række, så hver
skrivning slettede de to andre, og hver eneste svarede 204. Det virker begge
veje: at forbinde kalenderen slettede Gmail-forbindelsen, og Gmail slettede
kalenderen.

Fælden var set komme to gange. 0011 gav instansens indstillinger deres egen
tabel delvis af den grund og skrev advarslen ned med rene ord; 0014 flyttede
postkasserne ud og skrev, at advarslen "nær var trådt i for anden gang". Tredje
gang var kalenderen, som lagde et `calendar`-scope ved siden af de to andre.

Lektien værd at holde fast i: en advarsel i en migration bliver læst af den, der
redigerer *den* migration, og det var ingen. Formen er rettet her, så der ikke er
noget tilbage at advare om — `PRIMARY KEY (user_id, scope)`, og upserten rammer
den nøgle.

Tabellen bygges om, fordi SQLite ikke kan ændre en primærnøgle på plads. De rækker,
der findes, kopieres som de er: hver person har højst én, og den holder det scope,
der vandt sidst. Det, de tidligere skrivninger tog, er overskrevet og kan ikke
hentes tilbage her — efter opdateringen skal AI-udbyderen sættes op igen, og
Gmail-forbindelsen tjekkes.

To prøver: én mod store-laget, hvor formen var fejlen, og én gennem handlerne,
hvor den blev set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 14:56:48 +02:00
Kristian WindandClaude Opus 5 e46aaa462a feat(projects): sektioner kan trækkes op og ned, og båndet står ikke længere skævt
Rækkefølgen af sektioner *er* planen — "Planlægning" står før "I gang", fordi det
er den vej arbejdet går — og indtil nu kunne den kun laves om ved at slette en
sektion og lave den igen, hvilket smider dens opgaver ud i det usektionerede
område på vejen.

Hele overskriften er håndtaget. En sektionsoverskrift indeholder ikke noget, der
selv bliver trukket, så der er ingen geste at stjæle — hvilket er grunden til, at
sidebjælkens rækker skulle have et greb, og grunden til at disse ikke skal. Ikke
draggable mens den omdøbes: et draggable element lader ikke markøren vælge tekst
i det felt, det indeholder.

Serversiden er hele listen på én gang som projekter og grupper, og af samme
grund: et projekt har en håndfuld sektioner, så 0, 1, 2 … er én lille transaktion
uden præcision at løbe tør for, og en liste kan ikke ende halvt anvendt.
`ReorderSections` er scopet til projektet og ikke til en ejer — en sektion har
ingen ejer, den hører til et projekt, og om du må rykke rundt er et spørgsmål om
det projekt, som handleren allerede har stillet. Id'er fra et andet projekt
springes over af WHERE-klausulen; der er en prøve på præcis det, fordi ingenting
ellers kontrollerer id'erne enkeltvis.

Båndet stod fire pixels skævt. Det var trukket ud med --s2 og polstret med --s3 —
samme idé gjort med to forskellige tal — så det blev otte pixels bredere end
listen i hver side, og teksten landede fire pixels uden for den kolonne, rækkerne
holder. Til venstre var det usynligt. Til højre var det Omdøb og Slet, der stod
fire pixels forbi alt over og under sig, hvilket er præcis langt nok til at ligne
en fejl og ikke langt nok til at se hvorfor. En række er `.row` med
`padding: var(--s2)` inde i en boks uden, så båndet har nu samme polstring og
ingen margen.

Board-visningen er ikke rørt: der er kolonnerne vandrette, og det er en anden
geste.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 14:56:26 +02:00
Kristian WindandClaude Opus 5 98bfa216d6 fix(look): mærket og de faste visninger fulgte med op, når listen rullede
Hele sidebjælken rullede, så runen og I dag forsvandt over kanten, så snart der
var projekter nok — og de er dem, man er på vej hen til oftest. Alt under de
faste visninger ligger i en rulleboks nu, så rullebjælken begynder under dem.

En boks indeni frem for `position: sticky` udenom: et klæbende hoved efterlader
rullebjælken i fuld højde ved siden af sig, og det var netop den, der skulle
starte længere nede. Snippet-erklæringerne bliver liggende udenfor — de tegner
ikke noget selv, men de kaldes oppe fra .views, og en snippet erklæret indeni
hører kun til indeni.

Chevronen for en mappe dækkede gruppens prik med et felt i --surface. På den
sænkede bjælke er det en lys firkant, og en foldet gruppe har sin chevron fremme
permanent, så klodsen stod der hele tiden. Prikken tones ud i stedet.

Det kostede en fælde: en `opacity` under 1 laver en stakkontekst og maler
elementet, som var det placeret med `z-index: 0`. Prikken lagde sig dermed oven
på knappen, den lige var forsvundet for, og tog klikket — usynlig og stadig i
vejen, så gruppen ikke kunne foldes. Ét lag op på knappen, og rækkefølgen er
som før.

Prøven for vandret rul måler nu begge bokse. Bjælken selv er `overflow: hidden`,
så en for bred række ville blive klippet dér og aldrig nå at tælle med — den
ville bestå af den forkerte grund.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 14:56:01 +02:00
Kristian WindandClaude Opus 5 6c3d66108b fix(look): stregen under hurtig tilføjelse havde ingen luft under sig
Release / Build and publish image (push) Failing after 9m56s
Den skiller "skriv en ny" fra "det, der allerede er". Uden afstand sad den første
sektion klods op ad den, så den læste som en kant om sektionen frem for som en
afslutning på feltet over. En streg skal have plads på begge sider for at kunne
ses som en streg.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 11:36:23 +02:00
Kristian WindandClaude Opus 5 1437667483 fix(notes): stjerne og arkiv står oven på hinanden
Release / Build and publish image (push) Failing after 9m56s
To knapper ved siden af hinanden er toogfyrre pixels taget fra en liste, der er
tre hundrede bred — og de er det, man rører en gang imellem, mens titlen og
uddraget er det, man læser hele tiden. I en søjle koster de toogtyve.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 11:23:21 +02:00
Kristian WindandClaude Opus 5 035d079a8e fix(notes): teksten fulgte ikke med, når ruden blev bred
Release / Build and publish image (push) Failing after 9m59s
46rem var sat for at holde linjelængden i skak, og det er stadig det rigtige
hensyn — men det blev sat som ét tal, og på en skærm halvanden gang så bred stod
teksten i en smal søjle med en håndflade tom plads ved siden af. Kodeblokke og
billeder, som er de to ting i en note, der *vil* have bredde, blev klemt sammen
uden grund.

clamp frem for et tal: en smal rude får det hele, en bred får en linjelængde,
øjet kan finde tilbage i. På 1920 er tekstsøjlen nu 1088 px af arkets 1304 i
stedet for 736.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 11:08:26 +02:00
Kristian WindandClaude Opus 5 aadfb197b6 feat(notes): arkiv, flere ad gangen, og en kopiér-knap på koden
Release / Build and publish image (push) Failing after 9m55s
**Arkivet, fordi papirkurven er det forkerte redskab.** Den siger "det her var en
fejl, og om tredive dage er det væk" — så alt, der er værd at beholde, kan ikke
komme i den. Et arkiv på tolv hundrede importerede noter er præcis det tilfælde,
hvor de fleste er værd at beholde og næsten ingen er værd at kigge på; uden et
sted at lægge dem er listen hele arkivet for evigt, og at finde ugens note
betyder at læse forbi et årti.

En kolonne, ikke en mappe. Noterne afviste et andet hierarki i 0018, og
arkivering er ikke arkivering i den forstand: det er én oplysning om, hvorvidt en
note er aktuel, og den skal kunne besvares ved siden af `deleted_at`, som er den
anden af samme slags. `updated_at` røres ikke — at rydde op er ikke at skrive.

**Flere ad gangen.** ⌘-klik lægger til, skift-klik tager intervallet i den
rækkefølge, listen faktisk står i — ikke notens egen, som er en anden, når der er
sorteret eller søgt. Trækker man en note, der er markeret, følger hele
markeringen med; trækker man en, der ikke er, er det den ene, for ellers ville et
træk i en tilfældig række flytte halvtreds noter, man havde glemt var markeret.

Ét kald frem for et pr. note: efter en import af tolv hundrede er det at vælge
halvtreds det almindelige tilfælde. Hver note kontrolleres stadig for sig — et sæt
er en bekvemmelighed for den, der kalder, og aldrig en vej til noget, de ikke
kunne have rørt én ad gangen.

En kant *og* en tone på det markerede: `--accent-sunken` er meget lys i lyst tema,
og tre markerede rækker var svære at skelne fra tre, der ikke var — en dårlig ting
at være i tvivl om lige inden man trykker Slet.

**Kopiér-knappen spiste næsten den kode, den skulle kopiere.** Den ligger inde i
`<pre>` for at kunne stå i hjørnet af blokken, og `colourCode` læste
`pre.textContent` og malede resultatet tilbage — så et klik på "Kopiér" skrev
"Kopieret" ind i koden, og næste gemning lagde det i filen. Nu ét sted, der siger
hvad en kodeblok indeholder, brugt af både farvelægningen og oversættelsen til
Markdown: var de to uenige, ville forskellen ende i noten. Prøven klikker,
læser udklipsholderen og læser filen bagefter.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 11:03:02 +02:00
Kristian WindandClaude Opus 5 8aad7c13b0 perf(notes): listen sendte hele noter for at vise halvfems tegn af dem
Release / Build and publish image (push) Failing after 9m55s
Chrome frøs på notesiden, og der var to grunde.

**Nyttelasten.** Listen viser en titel og omkring halvfems tegn. Den fik hele
noter for at gøre det: med tolv hundrede blev svaret 3,9 MB, hvoraf otteoghalvfems
procent var brødtekst, ingen ville læse på den skærm — hentet, tolket og holdt i
hukommelsen hver gang siden blev åbnet. Nu 763 KB.

Kroppen tømmes frem for at blive forkortet, og det er den vigtige halvdel. Et felt,
der hedder `body` og indeholder et *stykke* af kroppen, er en fælde: editoren ville
åbne det, pausen ville gemme det, og noten ville være skåret ned til sit eget
uddrag af ingenting andet end at være blevet kigget på. Tom er en form, klienten
kan se efter — og den gør det og henter hele noten, når den åbner en.

Det kostede en adskillelse mere: hvilken række der er *valgt*, og hvilken note der
er *indlæst*, er nu to ting. Editoren indlæser sin tekst, når notens id skifter, så
hvis den fik uddraget først og hele noten bagefter, ville id'et være det samme
begge gange og den anden aldrig blive vist. Man ville sidde med en tom note, der
lige havde haft en titel.

**Tegningen.** Alle tolv hundrede rækker fik layout og maling, også de tusind
ingen kan se: 2,1 sekunder til første tegning. `content-visibility: auto` lader
browseren springe det over og tage det, når man ruller derhen — 0,95 sekunder, og
uden en eneste linje JavaScript, ingen rullematematik og ingen liste, der kan komme
ud af trit med sig selv.

**Og søgefeltet.** Det var en pille med et forstørrelsesglas tegnet ind som et
baggrundsbillede med farven skrevet i hånden — #8b918d, uanset tema — hvilket
gjorde det til det ene sted på siden, der ikke fulgte resten. Nu samme form som
søgefeltet i toppen, et rigtigt ikon i `currentColor`, en ryd-knap og et antal
fundne.

Det havde heller ingen pause: hver tast sendte en søgning gennem tolv hundrede
noter, så "kaffe" var fem forespørgsler, hvoraf de fire var forældede inden de kom
tilbage — og svarene kunne lande i den forkerte rækkefølge. Nu én.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 10:36:35 +02:00
Kristian WindandClaude Opus 5 3a02bc8304 fix(notes): søgningen svarede med det forkerte først, og et træk gjorde ingenting
Release / Build and publish image (push) Failing after 9m55s
**Søgningen sorterede efter dato.** Usynligt med ti noter og ubrugeligt med tolv
hundrede: de blev alle importeret samme aften, så `updated_at` siger ingenting, og
loftet skar listen af ved halvtreds vilkårlige rækker. At søge "Tesla" svarede med
en note om nginx først og noten, der hedder "Tesla Model Y", bagefter — hvilket
læses som en søgning, der ikke virker, fordi det fra brugerens plads er præcis
dét.

Nu bm25, med titlen vægtet ti gange kroppen: et ord i titlen er, hvad noten
*handler om*, mens det samme ord én gang i en lang krop er en henvisning. `fold`
vejer som kroppen, ellers ville et dansk match slå et præcist. Sammenføjet frem
for `rowid IN (…)`, fordi rangeringen hører til matchet, og en underforespørgsel
smider den væk. Fladen sorterer ikke om, når der er søgt — det ville kaste netop
det væk, der gør en søgning i tolv hundrede noter brugbar.

**Et træk af en note hen på et projekt gjorde ingenting.** `ondragover` tog imod
både opgaver og noter, så rækken lyste op — og `ondrop` sendte kun opgaver videre,
mens noten faldt ned i grenen, der omarrangerer projekter, hvor den ikke betyder
noget. Den værst mulige form for i stykker: den så ud, som om den ville virke,
helt indtil man slap.

Det står i overleveringen som "virker i koden, men ikke fastholdt i prøver, fordi
prøven blev upålidelig". Den er skrevet igen, og den ser på resultatet — ligger
noten i projektet bagefter — frem for på om rækken blev fremhævet, hvilket var
netop det, den gamle kunne se og som blev ved med at være sandt, mens funktionen
var i stykker.

**Og datoen i listen var altid `updated_at`.** For tolv hundrede importerede noter
er det i aftes, så en note fra 2017 stod som "i går", og den dato, man sorterede
efter, var ingen steder at se. Rækken viser nu den dato, listen er sorteret efter,
og har begge i sit værktøjstip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 09:58:41 +02:00
Kristian WindandClaude Opus 5 4aa8bd6a40 fix(sikkerhed): resten af gennemgangen — SSRF, tre ubegrænsede veje, og fire hemmeligheder
Release / Build and publish image (push) Failing after 9m54s
**H2 — SSRF gennem AI-udbyderens adresse.** `base_url` blev gemt uden kontrol og
hentet med en almindelig klient, og udbyderens svar blev givet tilbage ordret.
Det var en fungerende skanner af det interne net for enhver bruger: peg den på en
vært, læs de første tre hundrede bytes af det, der svarede. På en homelab er det
det meste af, hvad der er værd at nå.

Nu gennem `safedial`, og svarets krop går i loggen frem for i svaret. Operatøren
har stadig brug for beskeden — for en lokal model er den som regel selve
problemet — så den er der, bare kun for den, der kører instansen.

**M2 — Web Push havde tre.** Endepunktet blev gemt ukontrolleret og POSTet til på
en tidsplan, den, der ringer, selv vælger: sæt en påmindelse et minut ude, og
serveren henter adressen, hvert minut. Upserten flyttede desuden ejerskabet med
`user_id = excluded.user_id`, så enhver, der kendte en andens endepunkt, kunne
overtage det — offerets telefon ville få angriberens beskeder og ingen af sine
egne. Og `DeletePushSubscription` havde ingen ejer-kontrol.

**H3 — tre veje uden loft.** FTS-udtrykket byggede omkring halvtreds bytes pr.
ord uden grænse; en søgning på en kvart megabyte blev til to megabyte
MATCH-udtryk og seksogtyve sekunder inde i SQLite, mens den holdt på én af kun
fire til otte forbindelser. Nogle få af dem samtidig, og databasen er utilgængelig
for alle på instansen, logins inklusive. `filterquery.Parse` er kvadratisk —
halvtreds tusind led er syv sekunders CPU og seksogfyrre gigabyte allokering i én
forespørgsel — og havde hverken loft på antal eller på dybde. Og noteimporten
læste hver fil i arkivet uden grænse: 64 MB komprimeret kan blive til
seksogtres gigabyte skrevet ned på den disk, databasen ligger på.

**M1 — `comment_id` blev taget på tro.** Rettighedskontrollen så på opgaven i
adressen, mens filen blev hængt på den kommentar, formularen bad om. En viewer på
et delt projekt kan liste dets kommentarer, så hun kunne læse et id og lægge en
fil under en andens kommentar i et projekt, hun kun må læse.

**M4 — fire hemmeligheder lå i klartekst i hver sikkerhedskopi.** TOTP-koden og
AI-nøglen forsegles nu; begge konverterer sig selv næste gang de skrives.
Kalenderfeedets og mailadressens tokens gør ikke — de slås op på deres præcise
værdi og vises i fladen mere end én gang, hvilket en hash ikke kan. Det står nu
skrevet i `SECURITY.md`, som påstod det modsatte. At rette det betyder, at
adressen kun kan vises én gang, og det er en beslutning om, hvordan funktionen
virker.

Og en prøve på, at politikken overhovedet bliver sendt. Det var den, der gjorde
XSS-en i nat til indsat HTML frem for kørt script, og der var ingen prøve, der
rørte den — en foranstaltning, ingen prøve rører, kan forsvinde i en oprydning,
uden at nogen opdager det. Målt på script-src alene: `style-src 'unsafe-inline'`
står der med vilje, og en indsat stil kan gøre en side grim, ikke køre kode.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 08:39:52 +02:00
Kristian WindandClaude Opus 5 ce3fc466b2 fix(auth): andenfaktoren kunne gættes igennem, og et token kunne tilmelde sin egen
To huller fra sikkerhedsgennemgangen, og de er samme historie fortalt to gange:
den ene kontrol, der findes til når adgangskoden allerede er tabt, kunne omgås af
netop den, der havde adgangskoden.

**H1 — spanden blev fyldt af adgangskoden alene.** `handleLogin` nulstillede
ratebegrænseren ved hver korrekt adgangskode, *før* den så på, om der var en
andenfaktor. Det gav én, der havde kodeordet, ubegrænsede portioner på ti gæt:
log ind, få en frisk spand og en frisk ventende session, brug ti gæt på koden, log
ind igen. Med tre gyldige koder i ethvert øjeblik og tyve forespørgsler i
sekundet er det lige odds inden for en eftermiddag.

Spanden ryddes nu først, når login er *færdigt*, og der er kommet en spand mere:
koder tælles pr. konto og ikke kun pr. adresse. Adressespanden er slået af enhver
med mere end én adresse, hvilket er det almindelige — en telefon og en bærbar er
allerede to. Det, der gør en sekscifret kode sikker, er at gæt er en knap
ressource, og knapheden skal tælles imod det, der bliver angrebet.

Den ventende session får lov at leve. At afslutte den ved første forkerte ciffer
blev overvejet og valgt fra: det ville koste en, der taster ét tal forkert, sin
adgangskode igen, og det køber næsten ingenting oven i loftet pr. konto.

**M3 — et lækket token kunne tilmelde sin egen andenfaktor.** `/auth/totp/setup`
lå kun bag `requireAuth` og spurgte om ingenting, mens `disable` altid har
krævet adgangskoden: døren åbnede med et token og lukkede kun med et kodeord. Et
stjålet token kunne hente hemmeligheden, lægge den i sin egen godkender, bekræfte
den og beholde den eneste kopi af gendannelseskoderne. Ejerens næste login ville
stoppe ved en kode, de ikke havde, og de kunne ikke slå den fra uden at logge ind
først. Et tyveri bliver til et lejemål — den sætning står allerede i filen ved
siden af om passkeys, og det er den samme regel.

Nu bag `requireSession` sammen med passkeys og sessionerne, og `setup` spørger om
adgangskoden ligesom `disable`. Symmetrien er pointen: at slå den til er mindst
lige så indgribende som at slå den fra.

Og en note kan oprettes inde fra et projekt. At ligge i et projekt *er* måden en
note deles på, så det er én handling frem for to — man skulle før oprette noten
under Noter og bagefter finde projektet frem i "Del i". Ruden findes nu, også før
der er en eneste note, for ellers er der ingen knap dér, hvor man opdager at der
skal skrives noget ned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 08:29:43 +02:00
Kristian WindandClaude Opus 5 bdde9cc6a5 merge: Google Kalender som menupunkt, med Verdandes egen kalender lagt over
Release / Build and publish image (push) Failing after 9m58s
Læs bredt, skriv snævert — decision 5 i planen, og kun den første halvdel:
kalenderen læses, og der skrives ingenting tilbage.

**OAuth-blokeringen er reel og blev ikke gået udenom.** En klient registreret som
*Internal* tager kun konti i sit eget workspace — også til kalendere, ikke kun
Gmail. Andreas' private konto bliver afvist med `org_internal`, og det er
klienten, der virker efter hensigten, ikke en fejl. Fladen siger det nu med ord,
man kan handle på, frem for at vise Googles egen skærm, der ikke gør.

Vejen udenom, som ikke er bygget: kalenderens egen hemmelige ICS-adresse. Ingen
OAuth, virker for enhver konto, og `internal/ics` findes allerede. Det er det
næste stykke, og det eneste, der låser den private konto op.

`internal/google` er flyttet ud af `internal/gmail`, fordi de to er det samme
flow, der er uenige om én streng. Egen callback-adresse frem for at dele Gmails:
én callback, der finder ud af hvilken token-butik den skal skrive i, går galt ved
at overskrive en Gmail-forbindelse, der virkede.

Egne tabeller frem for `mailboxes`. `trigger_kind`, `seen` og `last_uid` svarer
alle på "hvad gør en besked til en opgave", og ingenting her bliver til en
opgave; én konto har mange kalendere, og `mailboxes` har ikke plads til en
en-til-mange. Afgørende i praksis: `mailboxes.kind` har en CHECK, og SQLite kan
ikke ændre en — en tredje værdi koster den tabelombygning, som migration 0021
skulle igennem.

Begivenheder erstattes pr. kalender pr. vindue frem for at flettes med et
sync-token: et vindue, der slettes og skrives forfra, kan ikke komme ud af trit,
og et sync-token er endnu en ting, der skal være rigtig for altid, for at gitteret
ikke stille viser noget forkert.

`CalendarView` er udvidet, ikke kopieret. En begivenhed er et fyldt bånd, hvor en
opgave er et kort med en farvet kant, så de to læses forskelligt før et ord er
læst — og den kan ikke trækkes og er aldrig en knap: der er ikke noget at gøre
færdigt ved en andens møde.

To værn på det, der kommer fra Google: farven skal matche et hex-mønster, før den
må stå i en style, og linket bliver kun til et href, hvis det begynder med https.

Konflikten i jobs.go var to nye jobs skrevet på samme linje. Begge skal med.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:39:12 +02:00
Kristian WindandClaude Opus 5 6b2cd0f259 merge: engelske skærmbilleder, og otte steder hvor fladen blev ved med at være dansk
Release / Build and publish image (push) Failing after 9m54s
Skærmbillederne er engelske, fjorten i alt, og der er to mennesker i dem. Det
sidste er ikke pynt: halvdelen af det, programmet er til, findes ikke med én
konto — en ansvarlig, et navn i historikken, en medlemsliste, og "Venter på
andre" som andet end en tom side.

Sproget kommer fra kontoens locale og ikke fra strenge skrevet i hånden, hvilket
er dét, der fandt fejlene: otte steder, hvor grænsefladen blev ved med at være
dansk, uanset hvad kontoen sagde. "Forsinket" og "I dag" skrevet ind i markup,
"Uden sektion" som boardets første kolonne, gentagelsesreglerne halvt oversat,
alle nitten navne på hændelser i historikken, og et `da-DK` hårdkodet i Kommende.
Gentagelsesreglerne er flyttet til `when.js`, så menuen og mærket ikke kan blive
uenige igen — det er præcis, hvad den fil er til.

README's billeder havde været brudte siden `37605ba`, som lavede PNG'erne om til
WebP og slettede dem, uden at teksten fulgte med: forsidebilledet og alle seks i
tabellen var 404 på GitHub. Og noterne — hele funktionen — stod ikke nævnt hverken
i README, på forsiden eller i dokumentationen.

Nu skrives PNG til docs og WebP til site, altså præcis dét, de to steder læser.
Før efterlod hver kørsel tolv usporede PNG'er i site, som intet pegede på.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:34:16 +02:00
Kristian Wind dcd66b415e docs(web): registreringsteksten fortæller nu, at klienten dækker begge dele
Hjælpeteksten sagde "slå Gmail API til", og det er kun det halve nu. Den samme
klient bærer både Gmail og Google Kalender og skal have begge redirect-URI'er —
og kalenderpanelet siger, hvor dets egen kommer fra, så ingen leder efter et
felt til en klienthemmelighed, der allerede står ovenfor.
2026-08-20 07:33:24 +02:00
Kristian WindandClaude Opus 5 3991ec9445 feat(notes): et billede kan indsættes og trækkes ind i en note
Rapporteret fra brug: man kunne ikke indsætte billeder eller trække filer ind i
noter. Det passede — importen kunne tage billeder med, og intet andet kunne.
Indsætningen tog kun `text/plain` og smed resten væk, og der var slet ingen
håndtering af et træk. At indsætte et skærmbillede er det mest almindelige, der
findes i et noteprogram, og det gjorde ingenting overhovedet.

Der manglede også et sted at lægge filen. Vedhæftninger på en note fandtes i
databasen, siden migration 0021, men kun importen kunne skrive dem; der var ingen
rute. Nu er der en, med editor-rettigheder på noten og samme svar for "findes
ikke" og "ikke din", så et id ikke kan afprøves.

Skrevet ind som `![](…)` og ikke som en data-URL: en note er Markdown i en
database, og et fotografi fra en telefon som base64 midt i teksten ville gøre
noten til fire megabyte tekst, der skal læses hver gang listen tegnes.

`execCommand` frem for at bygge noden selv, fordi det er dét, der lægger
handlingen i browserens egen fortryd-historik — ⌘Z tager billedet ud igen,
ligesom det tager et ord ud.

`preventDefault` på både dragover og drop. Uden den første kommer der intet drop;
uden den anden åbner browseren filen i stedet, og man mister det, man var i gang
med at skrive, til en fane med et billede i. Kun filer: et træk inde fra appen
selv bærer vores egen nyttelast og skal falde igennem til den, der lyttede.

Prøven går begge veje ind, ser at billederne bliver hentet og tegnet, og læser
filen bagefter for at sikre, at der står to bilagsadresser og ingen data-URL.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:32:03 +02:00
Kristian Wind a5f2f769c4 fix(web): begivenhederne i en celle sorteres, hvor de tegnes
Serveren sorterer allerede, men en celle, der viser 14:00 over 09:15, ligner
ikke en liste i forkert rækkefølge — den ligner et forkert klokkeslæt på
chippen, og det er langt værre at skulle udelukke. Heldags først: en heldags
begivenhed er et bånd hen over dagen, ikke noget der sker på et punkt i den.
2026-08-20 07:30:55 +02:00
Kristian WindandClaude Opus 5 9d743a5d46 fix(tools): shots skrev PNG'er ned i mappen med WebP
Standardstien var site/screenshots, som siden landingssiden gik over til WebP
kun indeholder WebP. En kørsel lagde tolv PNG'er ved siden af dem, som ingen
tekst peger på. Den skriver nu til docs/, hvor de andre PNG'er ligger.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:27:48 +02:00
Kristian Wind 031bb18067 fix(calendar): en tom kalender skal sige hvorfor, ikke bare være tom
At vælge en kalender er nogen, der siger "vis mig den her". At gemme valget og
lade gitteret stå tomt, til fejeren kommer forbi et kvarter senere, er en
kalender, der påstår, at fjorten dage er fri — og personen står på siden med
knappen under hånden. Gemningen henter nu selv.

Og er der forbundet en konto uden at der er valgt nogen kalendere, siger
visningen dét. "Der er ikke valgt en kalender" og "der er ikke booket noget"
ser ens ud, og kun den ene er noget at gøre noget ved.
2026-08-20 07:27:42 +02:00
Kristian WindandClaude Opus 5 71a9fe63d6 docs: noterne findes også i teksten, og hvert billede har et sted at høre til
README'ets billeder havde været brudte siden landingssiden gik over til WebP:
site/screenshots/*.png blev slettet dengang, og markup'en pegede stadig på
dem. Den peger nu på PNG'erne i docs/, som er dem, der findes.

Noter er en af de største ting i appen og stod ikke ét sted i teksten —
hverken i README'et, på landingssiden eller i dokumentationen. Nu gør de,
med de to nye billeder.

Fem billeder pegede ingen tekst på: upcoming, sections, delegated, settings
og admin. De har fået et afsnit i docs/index.md, hvor hvert enkelt beviser
noget, siden siger. Landingssidens gitter viste den lyse udgave to gange —
den ene af dem er blevet til noterne.

Og to billedtekster passede ikke længere på billedet: temaerne viser nu også
de fire udseender, og opgaven i skuffen viser ikke sin samtale.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:25:43 +02:00
Kristian WindandClaude Opus 5 eeaaf7506b docs: skærmbillederne er engelske, og der er to mennesker i dem
Landingssiden og dokumentationen læses på engelsk, og billederne var danske
hele vejen — både grænsefladen og indholdet. Kontoen bliver nu oprettet med
engelsk sprog frem for at oversætte noget her: det er sådan appen selv gør
det, og det er også dét, der får en glemt oversættelse til at stå i et
billede i stedet for at vente på en bruger.

Demodataene er skrevet om til engelsk arbejde og husholdning. Og der er en
person mere: en invitation, en konto, tre opgaver hun sidder på og en
kommentar. Halvdelen af det, appen er til for, findes først, når der er to i
den — en ansvarlig på en opgave, et navn i historikken, og "Venter på andre"
som andet end en tom side.

Tre billeder viste ikke det, teksten sagde. `detail` var en tom formular, så
opgaven har fået beskrivelse, undertasks, en fil, en påmindelse og en samtale.
`themes` viste profilfeltet over temaerne, så den ruller nu ned til dem.
`delegated` var en tom tilstand.

Noterne er kommet til — listen med en note åben, og den samme note som kilde.
Fjorten billeder i alt.

WebP'en skrives af den browser, der i forvejen står åben, frem for af cwebp:
et program mere at installere hører ikke til i et script, hvis hele pointe er,
at der ikke er noget at installere. PNG'erne ligger nu kun i docs/ og WebP'en
kun i site/, hvilket er præcis dét, de to steder læser — før lagde et kørsel
tolv usporede PNG'er i site/, som ingen brugte.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:24:12 +02:00
Kristian Wind 1b1e57df29 docs: kalenderen den anden vej, og én registrering til begge Google-ting
`caldav.md` beskrev to måder at sende opgaver *ud*. Den tredje er den modsatte
retning og hører til samme sted, fordi det er dér, man ville lede.

`configuration.md` hed "Gmail" og handler nu om Google: samme klient, to
redirect-URI'er, to API'er, to scopes. Bemærkningen om en Internal-app står
begge steder, fordi den gælder begge veje — og for én privat konto er
kalenderens egen hemmelige ICS-adresse den kortere vej. Intet i verdande
abonnerer på sådan en endnu.
2026-08-20 07:24:01 +02:00
Kristian WindandClaude Opus 5 dce8800f93 fix(web): otte steder hvor grænsefladen blev ved med at være dansk
Skærmbillederne skulle tages på engelsk, og det var dét, der fandt dem:
en konto sat til engelsk fik stadig "Forsinket" over de forsinkede opgaver,
"Uden sektion" over den første kolonne på tavlen, "Aldrig" i gentagelses-
menuen, "hver dag" på rækken, "til instansen · inviteret af" på invitationen,
"3 projekter" på kontoen, og hele historikken — nitten hændelser, alle på
dansk. En funktion, der kun virker på det ene sprog, er en funktion, der er i
stykker.

Gentagelsen er den eneste, der ikke bare manglede en nøgle. Serveren skriver
`recurrence_text` på dansk og skal blive ved med det: én besked går over
WebSocket til alle i projektet på én gang, og de læser ikke det samme sprog.
Så de regler, der har et navn, får det i when.js — på den side, der ved, hvis
skærm det er — og alt mere specifikt falder tilbage til serverens ord.

Mellemrummet i historikken er skrevet ud. Svelte fjerner det, der åbner en
blok, så "inviterede" og adressen løb sammen til ét ord.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:23:42 +02:00
Kristian Wind c3f2181efe feat(web): Kalender som menupunkt, med Googles begivenheder lagt over egne opgaver
Samme gitter som Kommende, givet en liste mere at tegne. Ikke et gitter nummer
to, og grunden er den del, der er nem at få næsten rigtig: et mål skal afgøre
ud fra `dataTransfer.types` alene, om det tager imod, og to kopier af det er to
steder at være næsten rigtig. En opgave trukket her flyttes præcis som alle
andre steder, fordi det er den samme kode, der gør det.

En begivenhed er ikke en opgave, og cellen siger det før et eneste ord er
læst: en flade af farve, hvor en opgave er et kort med en farvet kant. Den kan
ikke trækkes — forbindelsen er læsende, og en chip, der letter fra siden og så
ikke vil lande, læses som en fejl frem for som en regel — og den er ikke en
knap, for der er ikke noget at lukke ved en andens møde. Farven er Googles egen
for den kalender, så to kalendere i ét gitter kendes fra hinanden på den farve,
personen allerede kender dem på.

Farven tjekkes frem for at blive troet: den ender i en inline `style`, og en
værdi, der ikke er en farve, kan lukke erklæringen og åbne en anden. Linket
tegnes kun, hvis det begynder med `https://` — et href er dér, hvor en
`javascript:`-adresse bliver til et klik, der kører.

Prøven bruger uge-visningen med vilje: en månedscelle skæres af ved tre chips,
og resten af suiten lægger opgaver på andre dage. Datoen læses ud af siden, ikke
ud af testprocessen, som de øvrige har lært at gøre.
2026-08-20 07:22:25 +02:00
Kristian WindandClaude Opus 5 241c5ed37d fix(mail): den samme mail blev en opgave igen og igen
Release / Build and publish image (push) Failing after 9m58s
Rapporteret fra drift med et skærmbillede af den samme mail seks gange i træk.

**IMAP: flaget kommer af, når mailen er blevet en opgave.** Markøren var et uid —
husk det højeste, begynd over det næste gang — og den holdt kun, hvis uids var,
hvad serveren sagde. Den brød på et fetch-svar uden uid, på en UIDVALIDITY, der
skiftede under en genopbygget postkasse, og på to postkasse-rækker, der pegede
samme sted og hver især førte deres eget regnskab. Alle tre gav det samme
symptom.

Så regnskabet flytter derhen, hvor spørgsmålet bliver stillet. "Flaget" betyder
"lav det her om til en opgave"; når den er en opgave, har flaget gjort sit
arbejde og er ikke sandt længere. Så er der ingenting at huske, og dermed
ingenting, der kan komme ud af trit — en anden synkronisering, en anden
postkasse-række eller en database gendannet fra en kopi finder alle det samme
svar, fordi svaret ligger i postkassen.

Flaget tages af *efter* opgaven er skrevet. Et flag fjernet fra en mail, hvis
opgave aldrig blev gemt, er en mail, ingen ser igen.

**Gmail havde den samme fejl af en anden grund.** Listen over allerede behandlede
beskeder blev bygget om ved hver kørsel af *kun de id'er, Gmail svarede med den
gang* — for at den ikke skulle vokse uendeligt. Men forespørgslen er begrænset,
så på en konto med mere stjernemarkeret post, end der er plads til i ét svar,
glemte hver kørsel det, den forrige havde husket. De kom tilbage som ubehandlede
og blev opgaver igen. Det var ikke en grænse, det var en måde at glemme på.
Listen har nu et loft på antal i stedet, nyeste først.

Gmail kan ikke få taget stjernen af: scopet er `gmail.readonly`, med vilje, fordi
det er dét, der gør samtykkeskærmen til noget, et menneske kan sige ja til. Derfor
skal regnskabet blive på vores side dér — men det skal så være et regnskab, der
kan holde.

Prøven kører hele vejen mod en rigtig IMAP-server: find de flagede, tag flaget af,
og spørg så forfra med markøren sat til nul — altså det værst tænkelige tilfælde,
hvor alt, vi selv havde husket, er væk. Der kommer ingenting tilbage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 07:04:40 +02:00
Kristian Wind 987d4edfce feat(calendar): en læst Google-kalender, med sit eget endpoint og sin egen sweep
Samme registrering som Gmail, et andet scope og en anden redirect-URI. Google
udsteder en refresh-token pr. godkendelse og holder op til hundrede i live pr.
konto, hver med sine egne scopes — så en forbundet kalender rører ikke en
Gmail, der virker, og at afbryde den ene tager ikke den anden med.

Callbacken har sin egen sti. Én callback, der ud fra gemt tilstand afgjorde,
hvilket af de to flows der kom tilbage, ville lægge valget af *hvilket
token-lager der skrives* bag et opslag — og går det galt, går det galt ved at
overskrive en virkende Gmail-forbindelse med kalender-tokens.

Begivenhederne caches i et vindue på 90 dage tilbage og et år frem. Et
månedsgitter bladrer, og et kald til Google pr. piletast er en kalender,
der er langsommere end den kalender, den viser. Vinduet har en kant, og
endpointet siger hvor den er: et tomt svar uden forklaring er en kalender,
der lydløst påstår, at en dag er fri.
2026-08-20 07:01:14 +02:00
Kristian WindandClaude Opus 5 78891bae6f feat(beacon): tæl installationer, sig præcis hvad der sendes, og lad folk sige nej
Spørgsmålet er ét: hvor mange bruger det her. Svaret er to værdier én gang i
døgnet — et tilfældigt id, installationen selv fandt på, og versionen. Ikke andet.
Beskeden er lille nok til at stå i fuld længde på indstillingssiden, og den gør
det, fordi et løfte om telemetri er præcis så meget værd som læserens mulighed
for at efterprøve det.

**Ikke obligatorisk, og grunden er praktisk frem for principiel:** det kan ikke
håndhæves. Domænet kan blokeres, binæren kan patches, og en fork fjerner den i
første commit — så obligatorisk køber intet over slået-til-som-udgangspunkt og
koster den tillid, resten af programmet arbejder for. Hemmeligheder forsegles
her, nøglen holdes uden for databasen, og sikkerhedskopisiden advarer om, hvad en
kopi indeholder. Telemetri, man ikke må sige nej til, er den ene sætning i den
historie, der ikke passer.

Uenigheden med `config.UpdateCheck`, som er slået fra med den modsatte
begrundelse, står skrevet i pakkens egen dokumentation frem for at blive glattet
ud. Den er reel, og læseren fortjener at se begge halvdele.

Id'et findes på og gemmes ved første brug frem for at blive udledt af værtsnavn
eller adresse. En hash af noget om maskinen ville være stabil, bekvem og
samtidig en værdi, modtageren kunne holde op mod et gæt — og det er forskellen
mellem anonym og pseudonym.

Modtagerens side er den samme binary med en kontakt: `POST /api/v1/beacon` uden
login, fordi den, der ringer, er en fremmed installation. Den svarer 404, når
instansen ikke er tælleren, så en almindelig installation ikke er en spamspand,
der svarer 200 på alt. Ratebegrænset som login-ruterne, og tabellen har et loft:
endepunktet indsætter en række på et id, den, der ringer, har fundet på, og uden
et loft kunne enhver fylde disken.

Tallet er "set inden for tredive dage" og ikke et løbende total. Et total stiger
kun og holder op med at betyde "hvor mange bruger det" og begynder at betyde
"hvor mange har prøvet det".

`internal/safedial` kom til undervejs: modtagerens adresse kan skrives i fladen,
og en adresse, en bruger angiver, må ikke kunne pege indad. Kontrollen ligger på
den opslåede adresse og ikke på URL'en, fordi et navn kan svare 127.0.0.1 — og
svare noget andet anden gang, efter en kontrol ved parsning er gået godt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 06:59:50 +02:00
Kristian Wind 7828c1e6ed feat(gcal): Google Calendar kan læses, og kalenderen får sine egne tabeller
Klienten henter kalenderlisten og begivenhederne i et vindue, og laver Googles
to former om til den ene, et gitter kan spørge om: dage i begge ender, begge
med. Googles heldags-slutdato er eksklusiv, og et møde, der slutter ved midnat,
slutter aftenen før — begge dele rettes ét sted frem for hos hver læser.

Egne tabeller frem for `mailboxes`. De to ligner hinanden på afstand og er
uenige om hver eneste kolonne bagefter: `trigger_kind`, `seen`, `last_uid` og
`project_id` svarer alle på "hvad laver en besked om til en opgave", og intet
her bliver til en opgave. Én konto rummer desuden mange kalendere, og en
en-til-mange er der ikke plads til. Det afgørende er mindre og mere praktisk:
`mailboxes.kind` har en CHECK, og SQLite kan ikke ændre en CHECK — en tredje
værdi koster en tabelombygning med fremmednøgler slået fra. Derfor er der ingen
CHECK på `provider` her: CalDAV skal koste en række, ikke en ombygning.
2026-08-20 06:50:57 +02:00
Kristian Wind a46d9158e1 refactor(google): OAuth-flowet flytter ud af gmail, så to ting kan dele det
Gmail og Google Calendar logger ind gennem den samme registrering med den
samme PKCE og den samme fornyelse. Det eneste, der adskiller dem, er ét ord:
scopet. En kopi mere ville være et sted mere, hvor `access_type=offline` kan
blive glemt — og det er dén fejl, der får en forbindelse til at dø lydløst en
time efter, den blev lavet.

internal/gmail beholder sit API og sit scope; navnene på flowet er aliaser, så
hverken kaldere eller prøver ændrer sig. Prøverne for PKCE, AuthURL og
Configured flytter med koden.
2026-08-20 06:46:10 +02:00
Kristian WindandClaude Opus 5 4b5f3f0864 chore: agenternes arbejdstræer hører ikke i repoet
Release / Build and publish image (push) Failing after 9m56s
Et `git add -A` fangede .claude/worktrees/ som et indlejret repo. Det er en
agents arbejdskopi og skal ingen steder.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 06:39:45 +02:00
Kristian WindandClaude Opus 5 052f8a43fb fix(notes): en note kunne smugle et script ind gennem et billedes alt-tekst
Indført i aften, sammen med billedvisningen, og fundet ved at læse den igen.

`escapeHtml` tager &, < og > — ikke anførselstegn, og det er rigtigt for tekst
mellem to tags, hvor et anførselstegn ikke betyder noget. `alt` er ikke tekst
mellem to tags. Det står inde i en attribut, og der lukker det første
anførselstegn den, hvorefter resten står som markup:

    !["  onerror="alert(1)](/api/v1/attachments/aaaa)

blev til et <img> med en onerror på. Adressen var i forvejen bundet til vores
eget bilagsmønster, så den var ikke vejen ind — teksten ved siden af var.

Det er ikke en teoretisk vej. En note deles ved at ligge i et projekt, så den
læses af alle, der kan læse projektet, og scriptet ville køre på vores eget
domæne med sessionscookien, hos den der åbnede noten.

En værdi i en attribut har brug for sin egen undslipning, og har nu en. Prøven
skriver noten, åbner den og ser efter, at hændelsen står som tekst i alt og ikke
som en attribut på elementet — og at billedets 404, som er selve vejen en
onerror ville blive kaldt ad, ikke kalder noget.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 06:39:19 +02:00
Kristian WindandClaude Opus 5 dee2d5e30b fix(faerdige): en opgave lukket efter midnat lå under "I går"
Release / Build and publish image (push) Failing after 9m56s
Fundet af prøverne, som for første gang kørte forbi midnat.

Dagen blev læst som `completed_at.slice(0, 10)` — de ti første tegn af et
tidsstempel, der er i UTC. Overskriften sammenlignede med *lokal* dato. I dansk
sommertid er de to ikke den samme dag i de første to timer efter midnat, så en
opgave lukket 00.30 blev sat under "I går", indtil klokken blev to. Tyve timer i
døgnet er de ens, hvilket er derfor prøven har været grøn hver eneste gang før.
Dagen udregnes nu lokalt, som den overskrift den skal svare til.

Og prøven for invitationslinket kunne kun bestå seks dage om ugen. Den ledte
efter "males inden fredag" som én tekst, men hurtig tilføjelse læser "fredag" som
forfaldsdatoen og tager ordet ud af titlen — tilbage står "males inden" plus et
mærke, hvis ord skifter med ugedagen. Kørt på en torsdag hedder mærket "I
morgen", og så fandt den ingenting. Prøven handler om invitationer, ikke om
datoer, så den ser nu på titlen alene.

Uddraget i notelisten viste `![](/api/v1/attachments/01a0…` for hver note med et
billede i — den eneste tekst i listen, ingen havde skrevet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 00:17:50 +02:00
139 changed files with 14580 additions and 1148 deletions
+3
View File
@@ -66,3 +66,6 @@ node_modules.nosync/
# Kept on disk because it is how a session hands over to the next one.
HANDOFF.md
NOTES-PLAN.md
# Agenternes arbejdstræer. Et git-repo inde i et git-repo, som ikke hører til her.
.claude/worktrees/
+437
View File
@@ -0,0 +1,437 @@
# Notes, as verdande builds them
A description of one working notes system: the data model, the syntax, the search,
the rendering contract, the import and export formats, and — more useful than any
of it — the decisions that look arbitrary until you know what they cost.
It is written to be lifted into another program. Nothing here depends on Go,
SQLite or Svelte except where it says so; where an answer was forced by the tool,
that is called out so you can pick a different one on purpose rather than by
accident.
---
## Read this part first: what to ask before you build
**This document is not a drop-in. It describes a notes system that lives inside a
task app with projects, roles, a trash and a search box, and about a third of its
design is inherited from that.** Lifting it into a program with different
surroundings without deciding which parts to keep produces something that works
for a week and then fights you.
**If you are an assistant working from this file: ask these questions and get
answers before writing code.** Do not guess a default. Each one changes what you
build, not just how you build it.
### About the surroundings
1. **Does the host app already have a place to file things** — projects, folders,
notebooks, tags? verdande deliberately gave notes no hierarchy of their own and
filed them under the projects that already existed. If your app has nothing,
you need an answer for where a note lives, and "a folder tree" is the expensive
one.
2. **Is there more than one person?** Sharing, roles and "who wrote this" run
through the whole model below. Single-user changes the schema and removes about
half the permission surface.
3. **Is there already a trash, and does it expire?** Notes lean on one. Without
it, deleting is either permanent or you are building a trash.
4. **Is there an existing full-text index?** The search here is FTS5-shaped. On
Postgres it is `tsvector`, on a client-side app it is probably a scan, and the
ranking section below changes accordingly.
### About the notes themselves
5. **Markdown, rich text, or blocks?** verdande stores Markdown and renders it.
That decision is defended below; it is also the one that most constrains
everything else.
6. **Is the first line the title, or is there a title field?** Both work. Deriving
it means one less thing to keep in step and one less thing a person can leave
blank, and it forces the rule "the first line is a heading" on the editor.
7. **What links to what?** `#project`, `[[note]]` and task references exist here
because the app has projects and tasks. Yours may have none of them, or others.
8. **Do notes carry files?** Attachments bring content-type handling, a store, an
authorisation path and an XSS surface. Skipping them is a legitimate answer.
### About scale and language
9. **How many notes, realistically?** Several choices below flip at around ten
thousand: the list payload, the search fallback, rendering the whole list.
10. **Which languages?** The folding described under *Search* is Danish. Every
language has its own version of that problem and the generic answer (strip
diacritics) is wrong for several of them.
11. **Where does the text end up besides the screen?** Search indexes, AI
features, exports and backups each read it. Plaintext makes all of them cheap
and makes a stolen backup expensive — see *What is still open*.
### About integration, specifically
12. **Who owns the note's identity — you or the other system?** If notes will sync
with something else, ids and timestamps are the whole problem, and the
`updated_at` rule below becomes the first thing you have to change.
13. **Is this a read of somebody else's notes, or the store of record?** A copy
needs conflict rules; a store of record needs export.
14. **What happens to a note whose project, task or link target is deleted?**
Answered here as "the link goes dead and the note is untouched". If your host
app cascades instead, say so before the first delete.
---
## The shape of a note
```
id an opaque string, unique
project_id nullable — where it is filed, or loose
title derived, never set by a caller
body Markdown, the whole note
created_by nullable — survives the account being deleted
pinned favourite; the list puts these first
archived_at nullable timestamp — put away, not thrown away
deleted_at nullable timestamp — the trash
created_at
updated_at
```
### Markdown in a column
Not rich text, and not a format of our own.
Export was a requirement before a line was written, and **if the stored form is
the exported form there is no conversion left to lose anything in.** A note stays
readable by any editor on the day the program is gone. It is also what makes
search and anything an AI does over notes cheap: both read text.
The cost is real and you should know it before choosing: everything the editor can
do has to survive a round trip through Markdown, and anything Markdown cannot say,
the editor must not offer. There is no place to hide a feature.
### The title is the first line
Always, and re-derived on every save — not "when it is empty".
Derived once and then left alone, a title goes stale the moment somebody rewrites
the opening of a note, and the list ends up calling a note by a name that is
nowhere in it. **The column is a cache of the body, not a field beside it**, and no
caller can set it, so every caller gets the same answer.
This forces one editor rule: a new note opens on its title, and the first line is
styled as a heading. Otherwise a note whose first line is a sentence is a note
named by its first sentence.
### Archive and trash are two different bits
The trash says *this was a mistake, and in thirty days it is gone*. The archive
says *this is finished, and I do not want to read past it.*
A twelve-hundred-note import is exactly the case where most notes are worth
keeping and almost none are worth looking at. Without an archive, the list is the
whole archive forever, and finding this week's note means reading past a decade.
Both are nullable timestamps rather than booleans: *when was this put away* answers
questions a flag cannot, and it costs the same byte.
---
## Filing: no hierarchy of its own
A note belongs to a project, or to nobody.
The reasoning is worth repeating because it is easy to talk yourself out of:
projects already exist, are already shared, already have roles, already have a
trash. **A second hierarchy to file things in is one too many**, and a note filed
under a project inherits every one of those answers instead of needing its own.
- A note with a project: everybody who can read the project can read the note; an
editor can change it.
- A note without one: the author's, and nobody else's.
Sharing a note *is* filing it in a project. There is no access list of its own —
a second way to give somebody access is a second place to get it wrong, and the
two would disagree the day one of them changed.
---
## Links: three kinds, parsed out of the text
The body is the truth. Links are an index over it, rebuilt on every save.
| Syntax | Means | Notes |
|---|---|---|
| `#project` | a project | Same characters the task quick-add parser uses, on purpose |
| `[[a note]]` | another note, by title | The spelling every note app has settled on |
| `/opgave/<uuid>` or `task:<uuid>` | a task | What a link pasted out of the app carries |
Patterns as used here:
```
project (?:^|\s)#([\p{L}\p{N}_-]{1,64})
note \[\[([^\]\n]{1,200})\]\]
task (?:/opgave/|task:)([0-9a-fA-F-]{36})
```
Three decisions inside that table:
- **`#Firma` in a note means the same project as `#Firma` in the task box.** Not a
second kind of tag that happens to look alike. That is the whole of "tag a note
and see it in the project".
- **The link table is torn down and rebuilt on save, not diffed.** A body is small
and the set is small; a diff is a second description of the same fact, and it
can disagree with the text. That is the one thing this must never do.
- **The target is not a foreign key.** A link to something deleted is a fact about
the note. It should read as a dead link rather than vanish, and it must not
block the delete.
The index exists so the question can be asked **backwards**: *what has been written
about this task?* Answering that by searching every note's prose would be too slow
to leave a panel open on.
---
## Search
Three layers, in the order they answer.
### 1. Full-text, ranked
FTS5 over `title`, `body` and a folded copy, with `unicode61 remove_diacritics 2`.
Ranking is bm25 with **the title weighted ten times the body**. A word in the title
is what the note is *about*; the same word once in a long body is a mention.
This matters more than it sounds. Ordering by "last touched" is invisible with ten
notes and useless with twelve hundred imported the same evening: searching "Tesla"
answered with a note about nginx first and the note actually called "Tesla Model Y"
second — which reads as a search that does not work, because from where the person
is sitting it is one.
### 2. Folding, for the language you are in
`remove_diacritics` does not touch `ø`, `æ` or `å` — Unicode considers them
letters in their own right, not accented forms. So a Dane cannot find "grøn" by
typing "gron", which is how people type when the keyboard is not theirs.
The answer here is a generated column carrying a transliterated copy:
```
ø,Ø → o æ,Æ → ae å,Å → aa
```
indexed alongside the body and **weighted the same as the body** — otherwise a
folded match outranks an exact one.
> **Ask, do not copy.** This mapping is Danish. German wants `ü → ue`, Swedish
> disagrees with Danish about `ä`, and Turkish has a dotless `ı` that breaks
> naive lowercasing entirely. Get the mapping from somebody who types the
> language.
### 3. A word inside a word
An index is built of tokens, and a token is whatever the tokenizer decided was a
word. `gamerule keepInventory true` is three of them, and the middle one is
`keepinventory` — so a search for "keep inventory" asks for two words that are
nowhere and gets nothing, while the note sits right there.
A prefix query gets half of it: `keep*` matches, `inventory*` cannot, and the two
are ANDed, so the half that works is thrown away by the half that cannot.
**This is not exotic in a notes app.** A decade of notes is full of commands,
identifiers and file names, and every one of them is several words the index
thinks is one.
So: the index answers first; when it has nothing, the text is read, and every term
must appear somewhere in the note, in any word. A `LIKE '%term%'` over the folded
column covers title, body and language folding at once.
What it costs: a leading `%` cannot use an index, so it scans. At twelve hundred
notes that is a few megabytes and a millisecond or two, and it only runs when the
fast path already came back empty — which is exactly the search that was about to
disappoint somebody. **At a hundred thousand notes it is the wrong shape**, and the
answer then is a trigram index, not a bigger scan.
It does not rank. There is no match to rank — that is the whole situation.
---
## The list, and why the body is empty in it
A list showing a title and ninety characters of preview must not fetch whole
notes. Twelve hundred notes came to 3.9 MB, of which 98 % was body nobody could
see.
The important half is *how* it is trimmed: **the body is emptied, not shortened.**
A field called `body` holding a *piece* of the body is a trap. The editor opens
it, the autosave pause writes it back, and the note is now cut down to its own
excerpt — destroyed by nothing more than having been looked at. Send `preview` as
its own field and leave `body` empty; the client sees the empty body and fetches
the whole note.
That costs one more separation, and it is not optional:
> **Which row is selected and which note is loaded are two different things.**
The editor loads its text when the note's id changes. Hand it the excerpt first
and the whole note second and the id is the same both times — so the second one
never renders, and somebody sits looking at a note that had a title a moment ago.
Highlight the row immediately from the list; give the editor the note only when it
is whole.
Rendering cost, separately: with twelve hundred rows, first paint took 2.1 s. CSS
`content-visibility: auto` with an intrinsic-size guess took it to 0.95 s with no
virtual list, no scroll maths, and no list that can fall out of step with itself.
Try that before building windowing.
---
## The editor contract
Rich text on top, Markdown underneath, and a source view that shows the file.
The supported subset — deliberately small, because everything in it has to survive
the round trip:
| Block | Markdown |
|---|---|
| Title | `# ` |
| Heading | `## ` |
| Subheading | `### ` |
| Quote | `> ` |
| Code block | ```` ``` ```` with an optional language |
| Bullet and numbered lists | `- `, `1. `, nested |
Inline: `**bold**`, `*italic*`, `~~strikethrough~~`, `` `code` ``, `<u>underline</u>`,
and images as `![alt](url)`.
Five rules that were each learned the hard way:
1. **Every line becomes exactly one block.** Markdown normally joins consecutive
lines into a paragraph. Here it must not: the person typed a line and expects
to get it back, and an editor that silently reflows what somebody wrote is an
editor they stop trusting.
2. **Load on id change, never while typing.** Writing to the editor's HTML puts
the caret back at the start, which mid-sentence is unusable.
3. **One definition of what a code block contains.** A copy button inside the
`<pre>` and a syntax highlighter reading `textContent` will disagree, and the
difference ends up in the file — a click on "Copy" wrote the word *Copied* into
the code and the next save put it on disk. Have one function that says what a
block holds, used by the highlighter and the Markdown serialiser both.
4. **A note that cannot be rendered must not take the pane with it.** One note
arriving in an unexpected shape threw inside the editor's reactive effect, and
from then on *no* note would open — the previous one just stayed on screen. Put
a guard around the render, fall back to showing the raw text, and let the rest
of the pane keep working.
5. **Escaping between tags and escaping inside an attribute are different jobs.**
An escaper written for text between two tags handles `&`, `<`, `>` — and not
quotes, because between tags they are harmless. Inside an attribute they close
it. `![" onerror="alert(1)](…)` is the whole exploit.
---
## Attachments
Images pasted or dragged into a note, stored content-addressed by hash, referenced
from the body as ordinary Markdown images.
The security shape, which is the part worth copying:
- **Served as `application/octet-stream` with `nosniff` by default.** Attachments
are the one place a user supplies content another user opens. An uploaded SVG or
HTML file rendered inline runs its own script on your origin with the session
cookie attached.
- **One exception, as a named allowlist:** a short list of raster image types
served inline as their exact type — `png`, `jpeg`, `gif`, `webp`, `avif`, `heic`.
A note carries its pictures in the text, and served as octet-stream every one of
them is a broken image.
- **The list is named one type at a time, never matched on an `image/` prefix.**
The prefix lets `image/svg+xml` through, and an SVG is a document that can run
script.
- **The stored path is checked against the files directory again at read time.** A
traversal that somehow got into the database must not become one that reads
`/etc/passwd`.
- **Reaching an attachment follows the note, which follows its project.** Same
chain as everything else. Do not let a client hand you the parent id and take it
on trust.
---
## Import and export
The same format both ways: **a zip of Markdown files**, one per note, with a YAML
front matter block.
```markdown
---
created: 2019-04-02T10:15:00Z
modified: 2024-11-30T21:04:00Z
---
# Møde med Anders
Han vil gerne have kaffe hver uge.
```
- `created` / `created_at` / `date` and `modified` / `updated` / `updated_at` are
all accepted on the way in. Obsidian and Bear write this block; so should you.
- Images travel in the zip beside the notes, and the links in the text are
relative — **a note and its pictures must end up in the same archive**, which
matters the moment an export is split into parts.
- File names collide: two notes called "Møde" are two notes, and a zip with one
entry twice is a zip that loses one.
Limits that a real import taught, all of them cheap to add and expensive to omit:
- A ceiling on the upload, on the number of entries, and on the **unpacked** size.
64 MB of zip expanded to 66 GB is a normal shape for an attack.
- A per-file limit as well as a whole-archive budget: one enormous photograph and
ten thousand small ones are different attacks with the same ending.
- The entry name is never used as a path. Store by hash; the name is only a key
for rewriting links.
- **Count what you skip, and say so.** A note whose name contained an ellipsis was
dropped silently by a filter looking for `..` — one out of 1141, found only by
counting at both ends.
### Timestamps on the way in
An importer needs to write `created_at` and `updated_at` from the file, and the
normal save path must refuse to let it.
`updated_at` means *when did this program last write this note*. Let any caller set
it and the column means whatever the last writer felt like. Give the importer its
own narrow entry point instead, and say in the comment why it exists.
Without those dates, a move of twelve hundred notes is a pile that was all written
the same evening — and the order they were written in is half of what an archive
is.
---
## What is still open here
Written down because a specification that only lists what works is a specification
that lies by omission.
- **Note bodies are plaintext at rest.** They hold API keys and passwords, and
they are in every backup anybody can download. Encrypting them is bigger than it
sounds: the body is in the full-text index and the folded column is generated
from it, so both are a second plaintext copy in the same database. Encrypt and
both have to go, and search has to be rewritten to decrypt in application code —
fine at a thousand notes, a full scan per keystroke at ten thousand, at which
point the answer is a keyed index (an HMAC per word).
- **Import never merges.** Every run creates new notes. Say so out loud in the UI.
- **There is no sync.** Export and the API are the way out.
---
## A short list of the traps, for the person in a hurry
1. A `body` field holding an excerpt will be saved over the real one.
2. Selected row and loaded note are two different pieces of state.
3. The title must be re-derived on every save or it goes stale.
4. Diacritic stripping is not folding, and folding is language-specific.
5. A tokenizer cannot see a word inside a word; decide what you do about it.
6. An attachment served inline is a script on your origin unless you allowlist by
exact type and send `nosniff`.
7. Escaping for text and escaping for attributes are different functions.
8. One render failure must not be allowed to freeze the whole pane.
9. An unpacked-size limit is not optional on an archive import.
10. Let the importer set timestamps; let nobody else.
+28 -11
View File
@@ -6,7 +6,7 @@
</p>
<p align="center">
<img src="site/screenshots/today.png" alt="The Today view: projects, filters and labels in the sidebar, beside the tasks due today." width="900">
<img src="docs/screenshots/today.png" alt="The Today view: projects, filters and labels in the sidebar, beside the tasks due today." width="900">
</p>
---
@@ -35,6 +35,11 @@ mail-to-task address, and an MCP endpoint so Claude can work your task list dire
in Firma, labelled regnskab. Danish and English, mixed freely in one line.
- **Projects, sections, sub-tasks, labels, saved filters.** List, board and calendar
views, and foldable groups over the projects in the sidebar.
- **Notes, beside the work.** Rich text that is Markdown underneath — headings,
lists, quotes, fenced code. A note is shared by filing it in a project;
`#project` and `[[another note]]` are links, and a task shows the notes that
mention it. A zip of Markdown goes in — Obsidian, Bear, or Apple Notes through
the exporter in [`tools/`](tools/) — and comes back out the same shape.
- **Drag where it means something.** Reorder a list or a board, file a project under
a group, drop a task on another project, or drop it on another day.
- **Sharing.** Owner, editor and viewer roles. There is no open registration:
@@ -90,21 +95,30 @@ Apple Reminders, forward mail into it, and add it to Claude as a connector.
| Project templates | Done, tested |
| Nightly backups with rotation | Done, tested |
| Comments, attachments, notifications, Web Push | Built |
| Notes: rich text, Markdown on disk, links to projects, notes and tasks | Done, tested |
| Notes in and out: a zip of Markdown, both directions | Done, tested |
| Todoist CSV import and export, account export | Done, tested |
| MCP server for Claude | Done, tested |
| CalDAV server (two-way VTODO) | Done, tested |
| Mail-to-task | Done, tested |
| AI layer (Anthropic/OpenAI/Google/local) | Built, degrades when unconfigured |
| Gmail: OAuth2 with PKCE, polling, one-way to tasks | Done, tested |
| Google Calendar: read-only, laid over the task calendar | Built, never run against Google |
| Calendar subscriptions: ICS and webcal, several per person | Done, tested |
| Times on tasks, set by dragging in the week grid | Done, tested |
| Notes: archive, grouped list, images, source view | Done, tested |
| Update notice for administrators | Done, tested |
| Personal API tokens | Done, tested |
| OpenAPI 3.1 spec, checked against the router | Done, tested |
| End-to-end smoke tests (Playwright) | Done |
| Documentation, landing page, licence | Done |
**Not finished:** Web Push, the AI providers and the Gmail API calls are
**Not finished:** Web Push, the AI providers and the Google Calendar API calls are
implemented against their specifications and have only ever been exercised there,
not against live services. Everything else has an interface: opening a task shows
not against live services. Gmail is no longer in that list — it runs in production,
including the flagging and de-duplication that took a day to get right. Calendar
subscriptions over ICS and webcal are proven; the Google half of the calendar is
not. Everything else has an interface: opening a task shows
its description, sub-tasks, comments, files and reminders, and the settings pages
cover the account, notifications, integrations, AI, API tokens, import, export and
templates.
@@ -116,17 +130,20 @@ accented `o` and a Danish-first app that cannot do this is broken for its own us
| | |
|---|---|
| ![Board view](site/screenshots/board.png) | ![Light theme](site/screenshots/today-light.png) |
| ![Board view](docs/screenshots/board.png) | ![Light theme](docs/screenshots/today-light.png) |
| Board view — one column per section. | The light theme. |
| ![A week on Upcoming](site/screenshots/week.png) | ![A task, opened](site/screenshots/detail.png) |
| A week at a time, so a task can be dragged across a month boundary. | A task, opened. |
| ![The five themes](site/screenshots/themes.png) | ![Instance history](site/screenshots/history.png) |
| Five themes, all measured for contrast. | What has been done on the server. |
| ![A week on Upcoming](docs/screenshots/week.png) | ![A task, opened](docs/screenshots/detail.png) |
| A week at a time, so a task can be dragged across a month boundary. | A task, opened: description, priority, labels, sub-tasks and its files. |
| ![Notes](docs/screenshots/notes.png) | ![The same note as Markdown](docs/screenshots/notes-source.png) |
| Notes, filed in a project to share them. | The same note as its source — which is what the export writes to disk. |
| ![Themes and looks](docs/screenshots/themes.png) | ![Instance history](docs/screenshots/history.png) |
| Five themes and four looks: how bright, and how it reads. | What has been done on the server. |
Regenerate these with `node scripts/generate-screenshots.js` from `web/`. It starts
the real binary, creates its own account, fills it with demo data and photographs
eleven surfaces — nothing to set up and nothing to remember. (`go run ./tools/shots`
still works, and is the way to photograph an instance that is already running.)
the real binary, creates its own account and a second one to share with, fills them
with demo data and photographs fourteen surfaces — nothing to set up and nothing to
remember. (`go run ./tools/shots` still works, and is the way to photograph an
instance that is already running.)
## Running it
+44 -9
View File
@@ -10,13 +10,28 @@ reply within a week.
- **Passwords** are Argon2id with the parameters recorded in the hash, so they can
be raised later without locking anyone out.
- **Every token** — session, invite, reset, API, calendar feed, mail address — is
stored only as a SHA-256 hash. A database dump contains nothing that can be
presented.
- **Session, invite, reset and API tokens** are stored only as a SHA-256 hash. A
database dump contains nothing that can be presented.
- **The calendar feed address and the mail-to-task address are not hashed.** They
are looked up by their exact value, and they are shown in the interface more
than once, which a hash cannot do. Anybody holding a backup file therefore holds
those two URLs: the feed reads that person's tasks without a login, and the mail
address files tasks into their inbox. Rotating either from **Settings →
Integrations** invalidates the old one. This is written down rather than fixed
because fixing it means the address can only ever be shown once, which is a
decision about how the feature works and not only about how it is stored.
- **Mailbox passwords, OAuth tokens, TOTP secrets and AI keys** are sealed with
AES-GCM under a key kept outside the database — see `internal/secret`. That is
what makes a stray copy of the database less than a stray copy of everything:
the backup does not carry the key.
- **Sessions** are `httpOnly`, `SameSite=Lax`, and `__Host-` prefixed over HTTPS.
- **Uploads** are stored content-addressed by hash and always served as
`application/octet-stream` with `nosniff`, so an uploaded SVG cannot run script
on this origin.
- **Uploads** are stored content-addressed by hash and served as
`application/octet-stream` with `nosniff` — with one exception, added so that a
note can show the pictures in it: a short allowlist of raster image types is
served inline as its own type. The list is named one format at a time rather
than matched on an `image/` prefix, because the prefix would let `image/svg+xml`
through, and an SVG is a document that can run script. `nosniff` stays, so the
browser must read the file as that image or not at all.
- **The filter language** compiles to parameterised SQL. No typed text is ever
concatenated into a statement.
- **Permission checks** are route middleware, and anything you may not see answers
@@ -24,8 +39,28 @@ reply within a week.
- **The CSP** is computed at startup by hashing the inline scripts in the built
page, rather than allowing `unsafe-inline`.
- **Note titles and bodies are sealed** under the same key, since 0027. Both, not
only the body: a note's title *is* its first line, so a readable title beside a
sealed body would leave the beginning of every note in the clear.
The two plaintext copies went with it. `notes_fts` indexed the body and the
generated `fold` column held a transliterated copy of it — sealing the body while
leaving those would have been theatre, since a stolen file could be read straight
out of the index. Search opens the text in Go instead, which costs 33–43 ms
across twelve hundred notes and is the wrong shape somewhere around ten thousand.
- **Task titles, descriptions and comments are still in the clear**, and so is
`note_links` — ids derived from a note's body, so a stolen file still shows which
projects and tasks a note points at, though not what it says.
## What it does not do
verdande has no rate limiting beyond the login and password-reset endpoints, and
no audit log of reads. It is built to be run behind something — a Cloudflare
Tunnel, a reverse proxy, a VPN — by somebody who knows what they exposed.
Rate limiting covers the endpoints a stranger can reach: signing in, resetting a
password, the second factor, the installation beacon, and inbound mail. **Nothing
else is limited** — an authenticated caller can ask as often as they like, and a
signed-in account that wanted to be a nuisance can be one.
There is no audit log of reads. Who *changed* something is recorded per project;
who looked at it is not.
It is built to be run behind something — a Cloudflare Tunnel, a reverse proxy, a
VPN — by somebody who knows what they exposed.
+18
View File
@@ -64,6 +64,22 @@ func run() error {
}
db.UseSecrets(box)
// Noter, der stadig ligger åbne, lukkes nu.
//
// Ved opstart og ikke efterhånden som de skrives: et arkiv på tolv hundrede
// importerede noter, som ingen redigerer igen, ville ellers aldrig blive
// forseglet — og det er præcis dét arkiv, der er værd at forsegle. Idempotent,
// så den koster et gennemløb af id'er hver anden gang.
//
// En fejl her stopper opstarten. Halvdelen af noterne lukket er ikke i stykker,
// men en instans, der starter og lader som om den beskytter noget, den ikke gør,
// er værre end en, der siger fra.
if sealed, err := db.SealNotes(context.Background()); err != nil {
return fmt.Errorf("seal notes: %w", err)
} else if sealed > 0 {
log.Info("notes sealed", "count", sealed)
}
httpapi.Version = version
api := httpapi.New(cfg, db, log, webAssets(log))
@@ -75,7 +91,9 @@ func run() error {
runner := jobs.New(cfg, db, log, api.Mail(), api.Hub())
runner.SyncGmail = api.SyncGmail
runner.SyncMailbox = api.SyncMailbox
runner.SyncCalendars = api.SyncCalendars
runner.Push = api.PushToUser
runner.SendBeacon = api.SendBeacon
runner.Start(jobCtx)
srv := &http.Server{
+16 -7
View File
@@ -1,13 +1,15 @@
# Calendar and CalDAV
Two ways to get your tasks into a calendar, and they are not the same thing.
Two ways to get your tasks into a calendar. Getting somebody else's calendar
*into* verdande is the other direction and has its own page — see
[Calendar](calendar.md).
| | [Feed](#subscribing-to-a-feed) | [CalDAV](#connecting-a-caldav-client) |
|---|---|---|
| Direction | Read only | Two-way |
| Works with | Anything that subscribes to a URL | Apple Reminders, Thunderbird, DAVx⁵ |
| You can tick things off | No | Yes |
| Setup | Paste a URL | Username and token |
| | [Feed](#subscribing-to-a-feed) | [CalDAV](#connecting-a-caldav-client) | [Calendars in](calendar.md) |
|---|---|---|---|
| Direction | Tasks out, read only | Tasks out, two-way | Events **in**, read only |
| Works with | Anything that subscribes to a URL | Apple Reminders, Thunderbird, DAVx⁵ | A Google account |
| You can tick things off | No | Yes | No — an event is not a task |
| Setup | Paste a URL | Username and token | Connect, then pick calendars |
## Subscribing to a feed
@@ -39,6 +41,13 @@ most people actually keep.
**Settings → Calendar feed → Rotate** issues a new one and breaks the old
immediately.
## Showing a Google calendar in verdande
Moved. Connecting a Google account, subscribing to an address, and what the grid
does with either now live on their own page — see [Calendar](calendar.md).
This page is the other direction: verdande's own tasks, read by something else.
## Connecting a CalDAV client
This is the one where ticking something off in Apple Reminders ticks it off here.
+91
View File
@@ -0,0 +1,91 @@
# Calendar
Your dated tasks as a grid, with other people's calendars laid under them.
**Calendar** in the menu shows a month or a week. The week is a day: hours down the
side, and everything with a time on it in its place. Two things at ten o'clock sit
beside each other, because that is a clash you should be able to see rather than
read.
## What is in it
- **Your tasks**, on the day they are due — and at the time, if they have one.
- **Events from calendars you have connected**, read-only.
Drag a task to another day to move it. Drop it on the hour column and it takes that
time; drop it in the band across the top and it keeps the day and loses the time.
Events cannot be dragged: verdande holds a copy of somebody else's calendar, and a
chip that lets itself be moved is a promise the server cannot keep.
A project's own calendar is under **Calendar** on the project page, with the same
week and month.
## Subscribing to a calendar
**Settings → Integrations → Subscriptions.** Paste an address and it is fetched at
once.
This works with anything that publishes an `.ics` file: public holidays, a club's
fixtures, a shared calendar from work, your own calendar's secret address from
Google, Apple or Fastmail.
- **`webcal://` addresses work.** They are the same URL over https with a scheme
that makes a desktop hand it to the calendar app; it is rewritten for you.
- **Several at a time.** Unlike the Google connection, which is one account, a
subscription is one address and you can have as many as you like.
- **Read-only, refreshed every fifteen minutes.**
- The address is fetched once before it is stored, so a typo tells you now rather
than by silently never filling.
!!! note "This is the way in for a private account"
If the instance's Google registration is *Internal*, Google refuses accounts
from outside the organisation with `org_internal` — no setting on your side
changes that. Your calendar's own secret ICS address goes through this panel
and works regardless.
## Connecting Google Calendar
**Settings → Integrations → Google Calendar.** It shares the OAuth client Gmail
uses, so the instance needs that registration first — see
[Configuration](configuration.md#google).
Two things have to be true in Google Cloud, and only the owner of the registration
can do them:
1. **Google Calendar API** is enabled for the project.
2. The calendar callback is listed as an authorised redirect URI. The exact string
is printed in the panel — copy it from there rather than typing it. Google
compares it exactly: scheme, host, path, no trailing slash. `redirect_uri_mismatch`
almost always means one of those four differs.
Once connected, tick the calendars you want to see. The list is refreshed on every
sync, so a calendar added or renamed since last time appears under its new name.
!!! warning "Written to the specification, not yet proven against Google"
The Google Calendar half is implemented from Google's documentation and tested
against a stand-in server. It has not been exercised against Google itself.
The subscription route above and the outgoing feed below have.
## Your own calendar, elsewhere
Two ways out, for two different jobs:
- **[The ICS feed](caldav.md#subscribing-to-a-feed)** — a read-only address for Apple Calendar,
Google or Thunderbird. Your tasks appear as dated entries.
- **[CalDAV](caldav.md)** — two-way. Tick something off in Apple Reminders and it
is ticked off here.
The feed address is a credential in itself: anybody holding it can read your tasks
without logging in, because a calendar client cannot sign in. Rotate it from
**Settings → Integrations** if it gets out.
## What it does not do
- **It does not write to a connected calendar.** Events are a copy.
- **It does not expand repeating events** from a subscribed file. A weekly meeting
in somebody else's calendar is stored as its first occurrence.
- **It does not resolve named time zones** from a subscribed file. An event carrying
`TZID=Europe/Copenhagen` shows the clock the file wrote, rather than one this
server has moved.
- **Only Google, and only reading**, for account connections. Everything else goes
through a subscription.
+26 -7
View File
@@ -39,12 +39,24 @@ Any Go duration: `720h`, `30m`, `45s`.
yourself, password resets are written to the log, and reminders arrive in the
app and as Web Push rather than by email.
## Gmail
## Google
Only needed if anyone will connect a mailbox. Register an OAuth client in Google
Cloud (**APIs & Services → Credentials → OAuth client ID → Web application**) with
the redirect URI `https://your-address/oauth/gmail/callback`, enable the Gmail API,
and add the `gmail.readonly` scope.
Only needed if anyone will connect a Gmail mailbox or a Google calendar. One
registration covers both: register an OAuth client in Google Cloud
(**APIs & Services → Credentials → OAuth client ID → Web application**) and add
**both** redirect URIs:
```
https://your-address/oauth/gmail/callback
https://your-address/oauth/calendar/callback
```
Then enable the APIs and scopes for whichever you want:
| For | Enable | Scope |
|---|---|---|
| Mailboxes | Gmail API | `gmail.readonly` |
| Calendar | Google Calendar API | `calendar.readonly` |
| Variable | What it does |
|---|---|
@@ -52,8 +64,15 @@ and add the `gmail.readonly` scope.
| `VERDANDE_GMAIL_CLIENT_SECRET` | The client secret. |
One registration serves every user on the instance; each person authorises their
own mailbox from **Settings → Gmail**. The redirect URI is derived from
`VERDANDE_BASE_URL`, so it cannot drift out of step with what you registered.
own mailbox from **Settings → Integrations → Gmail** and their own calendar from
**Settings → Integrations → Google Calendar**. Both redirect URIs are derived from
`VERDANDE_BASE_URL`, so they cannot drift out of step with what you registered.
!!! note "Two connections, two grants"
Connecting a calendar does not disturb a working Gmail connection, and
disconnecting one does not take the other with it. Google issues a refresh
token per authorisation and keeps up to a hundred of them live per account,
each carrying only the scopes it was granted with.
## Secrets
+30
View File
@@ -22,6 +22,9 @@ forward mail to, and an MCP endpoint so Claude can work your task list directly.
Firma, labelled regnskab. Danish and English, mixed freely in one line.
- **Projects, sections, sub-tasks, labels, [saved filters](filters.md).** List,
board and calendar views, and foldable groups over the projects in the sidebar.
- **Notes, beside the work.** Rich text to write in, Markdown on disk. A note is
shared by filing it in a project; `#project` and `[[another note]]` are links,
and a task shows what has been written about it.
- **Drag where it means something** — reorder a list or a board, file a project
under a group, drop a task on another project, or drop it on another day.
- **Sharing** with owner, editor and viewer roles. Anyone outside your instance
@@ -31,6 +34,33 @@ forward mail to, and an MCP endpoint so Claude can work your task list directly.
- **[Calendar](caldav.md), [mail](mail.md) and [Claude](mcp.md)** connected over
open standards rather than private formats.
## What it looks like
![Upcoming as a month](screenshots/upcoming.png)
*Upcoming, as a month. A week and a plain list are the other two, and a task can be
dragged from one day onto another.*
![A project as a list](screenshots/sections.png)
*The same project as a list, where sections read as bands. As a board they are
columns.*
![Waiting on others](screenshots/delegated.png)
*What you have handed to somebody else, grouped by whoever has it.*
![Settings, integrations](screenshots/settings.png)
*Where [Gmail](mailboxes.md), the [calendar feed](caldav.md) and the
[mail-to-task address](mail.md) are connected.*
![Settings, people](screenshots/admin.png)
*Inviting somebody to the instance. There is no open registration — everybody
arrives through a link, and picks their own password. See
[signing in](signing-in.md).*
## Getting it running
verdande is built to be deployed as a **Rune** in
+1 -1
View File
@@ -16,7 +16,7 @@ administrator who can read yours.
**Settings → Integrations → Gmail → Connect.**
Gmail needs an OAuth client, which is the instance's registration with Google
rather than anybody's account — see [Configuration](configuration.md#gmail).
rather than anybody's account — see [Configuration](configuration.md#google).
Once it is set up, each person connects their own mailbox through it.
Choose what makes a task:
+163
View File
@@ -0,0 +1,163 @@
# Notes
Markdown in, Markdown out. Nothing in between.
A note is written as rich text — headings, lists, quotations, code — and stored as
the Markdown it looks like. That is not a detail about the database: it is what
makes a note yours. Export gives you the same text you were looking at, and on the
day this program is gone, every note opens in any editor.
## Writing
Start typing. **The first line is the title** — always, and re-derived every time
you save, so a note you rewrite the top of is called by what it now says rather
than by what it once did.
| Style | Written as | Shortcut |
|---|---|---|
| Title | `# ` at the start of the line | pick it in **Aa** |
| Heading | `## ` | |
| Subheading | `### ` | |
| Quote | `> ` | |
| Code block | three backticks, with an optional language | |
| Bullet and numbered lists | `- ` and `1. `, nested with Tab | |
| Bold | two asterisks around the words | ⌘B |
| Italic | one asterisk around the words | ⌘I |
| Underline | a `u` tag around the words | ⌘U |
| Strikethrough | two tildes around the words | |
| Code | one backtick around the words | |
Every line becomes exactly one block. Markdown would normally join consecutive
lines into a paragraph; here it does not, because you typed a line and expect to
get it back.
Press the source button in the toolbar to see the note as the file it is. Edits there are edits to the
note.
### Pictures
Paste one, or drag it onto the note. It is stored with the note and travels with
it in an export. Every raster format works; SVG is stored but not shown inline —
it is a document that can run script, and a note is not a place to run somebody
else's.
### Code blocks
A fenced block is coloured for the language it names, and guesses when it does not.
Each one carries a **Copy** button in its corner.
## Linking
Three kinds of reference, parsed out of the text as you write it:
| You type | It means |
|---|---|
| `#Firma` | The project — the same `#Firma` quick add understands, not a second kind of tag |
| `[[Møde med Anders]]` | Another note, by its title |
| A pasted task link | That task |
Links are an index over the text, rebuilt every time the note is saved. The text
stays the truth.
**The interesting direction is backwards.** A task shows what has been written
about it, and a project shows the notes filed in it — without anybody having to
remember to attach anything. Write `#Firma` in a note and it appears on that
project's page.
A link to something that has been deleted reads as a dead link. It is a fact about
the note, and it should not vanish because something it mentioned did.
## Filing and sharing
A note belongs to a project, or to nobody.
- **In a project:** everybody who can read the project can read the note, and an
editor can change it. Choose it under **Share in** at the foot of the note.
- **In no project:** yours, and nobody else's.
There is no folder tree. Projects, groups and labels already exist, are already
shared and already have roles — a second hierarchy to file things in is one too
many, and a note in a project inherits every one of those answers.
## Finding
The list groups itself: favourites first, then **Today**, **Yesterday**, **This
week**, and then by month. Sort by created, last touched or name, and the grouping
follows — sorted by name it groups by first letter instead.
Search finds a word wherever it is:
- **`gron` finds `grøn`.** Unicode does not consider `ø` an accented `o`, so
stripping diacritics is not enough for Danish. A folded copy of every note is
indexed beside the original.
- **`inventory` finds `keepInventory`.** An index is built of whole words, and a
decade of notes is full of commands and identifiers that are several words in
one. When the index has nothing, the text itself is read.
- Results are ranked by how well they match, with the title weighted ten times the
body. A word in the title is what the note is about; the same word once in a long
body is a mention.
## Putting notes away
Two different things, and the difference matters:
- **Archive** — this is finished and I do not want to read past it. It stays
findable and comes back whenever you ask.
- **Trash** — this was a mistake. It is gone in thirty days.
Select several with ⌘-click and shift-click to archive or delete them together.
## In and out
**Export** gives you a zip of Markdown files, one per note, with the dates in a
YAML front matter block that Obsidian and Bear also read:
```markdown
---
created: 2019-04-02T10:15:00Z
modified: 2024-11-30T21:04:00Z
---
# Møde med Anders
Han vil gerne have kaffe hver uge.
```
**Import** takes the same shape back. Pictures travel in the zip beside the notes,
and the links in the text are relative — so a note and its pictures must be in the
same archive, which matters when a large export is split into parts.
Two things to know before importing:
- **Nothing is merged.** Every run creates new notes. Importing the same archive
twice gives you two of everything.
- **The dates are read from the front matter.** Without them, a move of a thousand
notes is a pile that was all written the same evening — and the order they were
written in is half of what an archive is.
### From Apple Notes
`tools/apple-notes-til-markdown.py` in the repository exports Apple Notes to
exactly this shape, front matter and pictures included. It is macOS-only, because
Apple Notes is.
For a large library, `tools/pak-noter-i-portioner.py` splits the export into
uploadable parts, keeping each note with its own pictures.
## What is encrypted, and what is not
**A note's title and body are sealed** in the database, under a key kept in a file
beside it rather than inside it. A backup you download does not carry the key, so
it does not carry your notes.
Both halves, not only the body: a note's title is its first line, so a readable
title beside a sealed body would leave the beginning of every note in the open.
**What is still in the clear:** task titles, descriptions and comments — and the
list of what a note links to, so a stolen file shows which projects and tasks a
note points at, though not what it says.
Sealing the notes cost the search its index — a full-text index over a note's body
is another copy of that body — so search reads the notes themselves now. At twelve
hundred notes that is about forty milliseconds, and the box waits a quarter of a
second after you stop typing, so you will not see it.
+547
View File
@@ -1396,6 +1396,204 @@ paths:
description: The environment variables that are not set.
"403": { $ref: "#/components/responses/Forbidden" }
/api/v1/notes/bulk:
post:
tags: [notes]
summary: Archive, unarchive or delete a set of notes in one call.
description: |
One call rather than one per note, because the thing being asked is one
thing: "put these away". Fifty separate requests would be fifty round
trips, fifty chances for half of them to fail, and a list that redraws
fifty times — and after an import of twelve hundred notes, selecting fifty
is the ordinary case rather than the extreme one.
Each note is still checked on its own. A set is a convenience for the
caller and never a way to touch something they could not have touched one
at a time; anything they may not write to is counted in `skipped` rather
than failing the whole call, so one note somebody else just deleted does
not stop the other forty-nine.
Archiving is not deleting. The trash says "this was a mistake, and in
thirty days it is gone"; the archive says "this is finished, and I want to
keep it".
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [ids]
properties:
ids:
type: array
items: { type: string }
description: At most 500.
archived:
type: boolean
description: Put away, or bring back out.
delete:
type: boolean
description: Into the trash instead. Wins over `archived`.
responses:
"200":
description: How many were changed, and how many were passed over.
content:
application/json:
schema:
type: object
properties:
done: { type: integer }
skipped: { type: integer }
"422":
description: No ids, too many, or neither archived nor delete was asked for.
/api/v1/notes/{noteID}/attachments:
post:
tags: [notes]
summary: Put a file into a note.
description: |
A picture pasted or dropped into a note. Until this existed the import
could bring files in and nothing else could, so the most ordinary act in
a notes app — paste a screenshot — did nothing at all.
Editor rights on the note. The same answer for "no such note" and "not
yours", so an id cannot be probed.
The response carries the attachment's URL; the editor writes that into the
note's text as a Markdown image. The file is never inlined as a data URL —
a photograph as base64 would make the note itself megabytes of text.
parameters:
- name: noteID
in: path
required: true
schema: { type: string }
requestBody:
required: true
content:
multipart/form-data:
schema:
type: object
required: [file]
properties:
file:
type: string
format: binary
responses:
"201":
description: Stored, and hung on the note.
"404":
description: No such note, or not yours to write in.
"413":
description: Larger than 25 MB.
/api/v1/beacon:
post:
tags: [auth]
summary: Report that an installation exists. Unauthenticated.
description: |
The receiving end of the beacon. Open on purpose: the caller is a
stranger's installation, which is the whole feature.
Answers 404 unless this instance has been told it is the collector, so an
ordinary installation is not a spam sink that answers 200 to anything.
Rate limited like the login routes, and the table behind it has a ceiling:
the endpoint inserts a row keyed on an id the caller invented, so without
one anybody could grow it until the volume is full.
Nothing about the request is recorded but the two values in the body. No
source address is read or written here.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [instance_id]
properties:
instance_id:
type: string
description: A random id the installation generated for itself.
version:
type: string
responses:
"204":
description: Recorded.
"400":
description: The id or the version is not the expected shape.
"404":
description: This instance is not a collector.
/api/v1/beacon/count:
get:
tags: [auth]
summary: How many installations reported in the last thirty days.
description: |
Unauthenticated and cached for a minute, and only answered when the
collector has been told to publish it.
A window rather than a running total: a total only climbs, so it stops
meaning "how many people use this" and starts meaning "how many ever
tried it".
responses:
"200":
description: The count.
content:
application/json:
schema:
type: object
properties:
installs: { type: integer }
window_days: { type: integer }
"404":
description: Not a collector, or not published.
/api/v1/beacon/settings:
get:
tags: [auth]
summary: What this instance reports, and to whom. Administrators only.
description: |
Includes the exact payload that will be sent, so the settings page can
print it in full rather than describe it.
responses:
"200":
description: The current configuration, and the counts if this is the collector.
content:
application/json:
schema:
type: object
properties:
enabled: { type: boolean }
collector_url: { type: string }
instance_id: { type: string }
version: { type: string }
is_collector: { type: boolean }
publish_count: { type: boolean }
last_ping_at: { type: string, format: date-time }
put:
tags: [auth]
summary: Turn the beacon off, or point it somewhere else. Administrators only.
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
enabled: { type: boolean }
collector_url:
type: string
description: |
Where reports go. Fetched through a dialer that refuses
loopback and private address space, so this cannot be used to
make the server reach inside its own network.
is_collector: { type: boolean }
publish_count: { type: boolean }
responses:
"200":
description: The configuration as it now stands.
"422":
description: The collector address is not an http or https URL.
/api/v1/panel/restart:
post:
tags: [auth]
@@ -1851,6 +2049,125 @@ paths:
"404": { $ref: "#/components/responses/NotFound" }
"422": { $ref: "#/components/responses/Validation" }
/api/v1/projects/{projectID}/sections/reorder:
parameters:
- { name: projectID, in: path, required: true, schema: { type: string } }
post:
tags: [projects]
summary: Set the order the project's sections appear in. Editors and owners.
description: |
The whole list at once, for the same reasons as `/projects/reorder`: a
project has a handful of sections, so sending the order you want is
simpler than midpoints and cannot land half-applied.
Ids belonging to another project are skipped rather than refused.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [ids]
properties:
ids:
type: array
maxItems: 500
items: { type: string }
description: Section ids, top to bottom.
responses:
"204": { description: Reordered. }
"404": { $ref: "#/components/responses/NotFound" }
"422": { $ref: "#/components/responses/Validation" }
/api/v1/calendar/subscriptions:
post:
tags: [calendar]
summary: Subscribe to a calendar you only have an address to.
description: |
An ICS or `webcal://` address. Several per person, unlike the Google
connection, which is one — the address is the identity here, the way the
token is for OAuth.
The address is fetched once before it is stored: an address that cannot be
read is a typo far more often than a server that happens to be down, and
saying so while somebody is still looking at the field is worth two
seconds. `webcal://` is rewritten to `https://`, and the destination is
checked against the private-network wall both when it is written down and
again on the resolved address every time it is fetched.
requestBody:
required: true
content:
application/json:
schema:
type: object
required: [url]
properties:
url: { type: string, description: "https:// or webcal://" }
name:
type: string
description: Optional. The calendar's own name is used when this is empty.
responses:
"201":
description: Subscribed, and fetched once.
"422": { $ref: "#/components/responses/Validation" }
/api/v1/calendar/subscriptions/{id}:
parameters:
- { name: id, in: path, required: true, schema: { type: string } }
delete:
tags: [calendar]
summary: Stop subscribing. The cached events go with it.
responses:
"204": { description: Gone. }
"404": { $ref: "#/components/responses/NotFound" }
/api/v1/push/test:
post:
tags: [notifications]
summary: Send one notification to your own devices, and say what happened.
description: |
The answer to "no notifications arrive". A push that a service refuses is
the ordinary failure — a stale subscription, a browser that revoked
permission without telling the server — and every one of them looks
exactly like silence from the outside.
It goes through the same path a real notification takes, so what it proves
is the thing that has to work. The title and body come from the caller
because the dictionaries live in the interface.
requestBody:
required: false
content:
application/json:
schema:
type: object
properties:
title: { type: string, maxLength: 200 }
body: { type: string, maxLength: 500 }
responses:
"200":
description: What each device did with it.
content:
application/json:
schema:
type: object
properties:
subscriptions:
type: integer
description: Devices registered for this account.
sent:
type: integer
failed:
type: array
items:
type: object
properties:
service:
type: string
description: The push service's host — never the endpoint, which is a credential.
reason:
type: string
description: What the service said.
/api/v1/sections/{sectionID}:
parameters:
- { name: sectionID, in: path, required: true, schema: { type: string } }
@@ -2507,6 +2824,126 @@ paths:
created: { type: integer }
"502": { $ref: "#/components/responses/Upstream" }
/api/v1/calendar:
get:
tags: [integrations]
summary: The Google Calendar connection and the calendars in it.
description: |
Read-only. verdande shows a Google calendar beside its own dated tasks and
never writes one back — writing needs the `calendar.events` scope and an
answer to what happens when both sides moved the same meeting, which is a
synchronisation model rather than a scope.
The list of calendars is fetched from Google when it is empty, so an
account that connected during a hiccup corrects itself on the next look
rather than showing an empty panel for ever.
responses:
"200": { $ref: "#/components/responses/Calendar" }
delete:
tags: [integrations]
summary: Disconnect the calendar and forget its tokens.
description: >
The cached events go too. Unlike the tasks a mailbox made, none of this is
the person's own work — it is a copy of somebody else's calendar, and it
means nothing without the account.
responses:
"204": { description: Disconnected. }
/api/v1/calendar/calendars:
put:
tags: [integrations]
summary: Choose which calendars are drawn.
description: >
The whole set in one write rather than one flip at a time, for the reason
the projects' order is written the same way: the question is "these ones",
and a list built out of several requests can land half applied. A calendar
turned off has its cached events removed; they come back if it is turned
on again.
requestBody:
required: true
content:
application/json:
schema:
type: object
properties:
shown:
type: array
items: { type: string }
responses:
"204": { description: Saved. }
"404": { $ref: "#/components/responses/NotFound" }
/api/v1/calendar/authorize:
post:
tags: [integrations]
summary: Begin the OAuth flow for a calendar.
description: |
The same registration Gmail signs in through, asking for
`calendar.readonly` and coming back to `/oauth/calendar/callback`. Google
issues a refresh token per authorisation, so connecting a calendar does not
disturb a working Gmail connection.
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
url: { type: string }
"409":
$ref: "#/components/responses/Conflict"
/api/v1/calendar/sync:
post:
tags: [integrations]
summary: Refresh the chosen calendars now.
description: >
Refreshes the list of calendars and replaces the cached window of events
for the ones that are shown. `partial` means the budget ran out before the
last calendar; what was read is kept, and the rest arrive on the next run.
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
events: { type: integer }
partial: { type: boolean }
"502": { $ref: "#/components/responses/Upstream" }
/api/v1/calendar/events:
get:
tags: [integrations]
summary: The events of the chosen calendars, between two days.
description: |
Both ends inclusive, and an event that merely overlaps the range comes
back — a grid asks which cells an event covers, not where it begins.
`from` and `to` in the answer are the window verdande holds a copy of,
which is not the window that was asked for. A grid paged past it gets no
events, and an empty answer with no explanation is a calendar quietly
claiming a day is clear.
parameters:
- { name: from, in: query, required: true, schema: { type: string, format: date } }
- { name: to, in: query, required: true, schema: { type: string, format: date } }
responses:
"200":
description: OK
content:
application/json:
schema:
type: object
properties:
events:
type: array
items: { $ref: "#/components/schemas/CalendarEvent" }
from: { type: string, format: date }
to: { type: string, format: date }
"422": { $ref: "#/components/responses/Validation" }
/api/v1/export/notes.zip:
get:
tags: [notes]
@@ -3150,6 +3587,26 @@ paths:
responses:
"302": { description: "Back into the app, with the outcome as a query parameter." }
/oauth/calendar/callback:
get:
tags: [integrations]
summary: Where Google sends the browser after consent to a calendar.
description: |
A URI of its own rather than a reuse of Gmail's. One callback deciding from
stored state which of the two flows had come back would put the choice of
which token store to write behind a lookup — and when that goes wrong it
goes wrong by overwriting a working Gmail connection with calendar tokens.
Answers with a redirect to `/indstillinger/integrationer`, not with JSON.
An `org_internal` error here means the OAuth app is registered as Internal
and the account is outside the organisation.
parameters:
- { name: code, in: query, schema: { type: string } }
- { name: state, in: query, schema: { type: string } }
- { name: error, in: query, schema: { type: string } }
responses:
"302": { description: "Back into the app, with the outcome as a query parameter." }
/mcp:
post:
tags: [integrations]
@@ -3332,6 +3789,40 @@ components:
email: { type: string }
trigger: { type: string, enum: ["", starred, label, both] }
label: { type: string }
Calendar:
description: OK
content:
application/json:
schema:
type: object
properties:
connected: { type: boolean }
account:
type: string
description: >
The address of the connected account, taken from the id of its
primary calendar — which is where Google puts it, so no profile
scope has to be asked for to learn it.
read_only:
type: boolean
description: >
Always true today. Stated rather than assumed: an interface that
quietly looks editable is one somebody will try to edit.
has_client:
type: boolean
description: >
Whether an OAuth client is registered on this instance at all, so
the page can explain rather than offer a button that answers 409.
redirect_uri:
type: string
description: >
What has to be registered with Google, spelled out. Derived from
VERDANDE_BASE_URL and matched exactly — the single most likely
thing to be wrong, and the error Google gives names neither value.
last_sync_at: { type: string, format: date-time }
calendars:
type: array
items: { $ref: "#/components/schemas/Calendar" }
ImportResult:
description: Created
content:
@@ -3349,6 +3840,62 @@ components:
items: { type: string }
schemas:
Calendar:
type: object
description: >
One calendar inside a connected account. Kept whether or not it is shown,
so turning one off and on again gives back the same row.
properties:
id: { type: string }
remote_id: { type: string, description: "The id Google knows it by; an address for the primary calendar." }
name: { type: string }
colour:
type: string
description: >
Google's own hex value, not one of verdande's ten named colours. Those
are names so the interface can restyle them per theme and they are
verdande's palette; a Google calendar is told apart by the colour the
person already knows it by.
time_zone: { type: string }
primary: { type: boolean }
writable:
type: boolean
description: >
Whether the connected account could write to it. Nothing does — the
scope is read-only — and it is stored so the day something can, the
interface knows which calendars could even be offered.
shown: { type: boolean }
CalendarEvent:
type: object
description: >
One occurrence, in the shape a grid asks about. A recurrence is expanded by
Google into its occurrences, so what arrives here is a day, not a rule.
properties:
id: { type: string }
calendar_id: { type: string }
summary: { type: string }
starts_at:
type: string
description: >
RFC 3339 as Google wrote it, offset and all. Empty for an all-day
event, which has a day rather than a moment — inventing midnight for it
is how an event moves a day when a container moves time zone.
ends_at: { type: string }
start_day: { type: string, format: date }
end_day:
type: string
format: date
description: >
The last day the event covers, inclusive. Google's all-day end date is
exclusive and is corrected on the way in, once, rather than by every
reader.
all_day: { type: boolean }
location: { type: string }
url: { type: string, description: "Opens the event in Google's own interface." }
calendar_name: { type: string }
colour: { type: string }
Note:
type: object
description: A page of Markdown. The body is the stored form and the exported form.
Binary file not shown.

Before

Width:  |  Height:  |  Size: 166 KiB

After

Width:  |  Height:  |  Size: 239 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 181 KiB

After

Width:  |  Height:  |  Size: 276 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 91 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 228 KiB

After

Width:  |  Height:  |  Size: 306 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 243 KiB

After

Width:  |  Height:  |  Size: 241 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 328 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 308 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 176 KiB

After

Width:  |  Height:  |  Size: 215 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 230 KiB

After

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 199 KiB

After

Width:  |  Height:  |  Size: 232 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 154 KiB

After

Width:  |  Height:  |  Size: 222 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 221 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 188 KiB

After

Width:  |  Height:  |  Size: 209 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 167 KiB

After

Width:  |  Height:  |  Size: 180 KiB

+37 -2
View File
@@ -19,6 +19,8 @@ import (
"fmt"
"io"
"net/http"
"github.com/kristianwind/verdande/internal/safedial"
"strings"
"time"
)
@@ -76,7 +78,16 @@ func New(cfg Config) *Client {
// Generous: a local model on modest hardware can take a while to produce
// its first token, and the alternative is a timeout that only ever fires
// for the people running their own.
http: &http.Client{Timeout: 120 * time.Second},
// safedial, not a plain client. `base_url` is typed into a field in the
// interface, so it is a user-supplied address — and a server that fetches a
// user-supplied address is a way to ask it to reach things the caller
// cannot: the panel next door, a database on the same bridge, the cloud
// metadata endpoint. On a homelab that is most of what is worth reaching.
//
// The check is on the resolved address rather than on the URL, because a
// name answers whatever its owner says — and can answer differently the
// second time, after a parse-time check has passed.
http: safedial.Client(120 * time.Second),
}
}
@@ -268,11 +279,35 @@ func (c *Client) do(req *http.Request, out any) error {
// The provider's own message is included: for a local model it is usually
// the actual problem ("model not found"), and hiding it would leave the
// operator with nothing to go on.
return fmt.Errorf("ai: %s said %s: %s", req.URL.Host, resp.Status, truncate(string(raw), 300))
// The provider's body is no longer passed back to the caller.
//
// It was, and combined with a free-text base URL that made a working
// internal scanner: point it at a host, read the first three hundred bytes
// of whatever answered. The operator still needs the message — for a local
// model it is usually the actual problem — so it goes to the log, where it
// is available to the person running the instance and to nobody else.
return &upstreamError{Host: req.URL.Host, Status: resp.Status, Body: truncate(string(raw), 300)}
}
return json.Unmarshal(raw, out)
}
// upstreamError keeps the provider's own words for the log and away from the
// response. Error() is what reaches a caller; Detail() is what is written down.
type upstreamError struct {
Host string
Status string
Body string
}
func (e *upstreamError) Error() string {
return fmt.Sprintf("ai: %s said %s", e.Host, e.Status)
}
// Detail is the whole of it, for the instance's own error log.
func (e *upstreamError) Detail() string {
return fmt.Sprintf("ai: %s said %s: %s", e.Host, e.Status, e.Body)
}
func truncate(s string, max int) string {
if len(s) <= max {
return s
+139
View File
@@ -0,0 +1,139 @@
// Package beacon answers one question for the project, and refuses to answer any
// other: how many installations are out there.
//
// It sends two values once a day — an anonymous id the installation made up for
// itself, and the version it is running — and nothing else. No IP is stored by
// the collector, no hostname, no domain, no account, no usage. The payload is
// small enough to be printed in full on the settings page, and it is, because a
// promise about telemetry is worth exactly as much as the reader's ability to
// check it.
//
// # On by default, and off in one click
//
// The alternative that was asked for first was mandatory, and the argument
// against it is practical rather than principled: it cannot be enforced. The
// domain can be blocked, the binary can be patched, and a fork removes it in its
// first commit — so mandatory buys nothing over on-by-default, while costing the
// trust the rest of this program works for. Secrets are sealed here, the key is
// kept out of the database, and the backup page warns you what a copy contains.
// Telemetry you are not allowed to refuse is the one sentence in that story that
// does not fit.
//
// So: on by default, said plainly, one switch to turn it off. That collects
// essentially the same number and leaves nobody feeling tricked.
//
// # Where this disagrees with the rest of the program
//
// `config.UpdateCheck` is off unless asked for, and its comment gives the reason:
// "a self-hosted app that reaches out without being told to has broken the deal
// its operator made by self-hosting." That is the same situation as this one, and
// the two defaults do not agree.
//
// The disagreement is written down here rather than smoothed over, because it is
// a real one and the reader deserves to see both halves. The case for treating
// them differently is that the update check reveals an instance to a third party
// (GitHub) and returns something the operator can act on, while this returns
// nothing and goes to the project itself — and that a count nobody sends is a
// count that does not exist, whereas an update nobody checks for is one you can
// still go and look up. Whether that is a distinction or an excuse is a judgement
// call, and it was made deliberately by the person whose instance this is.
//
// If it is ever decided the other way, the change is one word: the default in
// `beaconSettings`. Nothing else in this package assumes which way it points.
//
// # What the collector can still see
//
// A request has a source address whether or not anybody writes it down. The
// collector does not store or log it, and the code is arranged so that it never
// reaches a function that could — but it passes through a reverse proxy and a
// tunnel on the way, and those keep their own logs. Saying "we send no IP" would
// be a half-truth; what is true is that nothing here records one.
package beacon
import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"regexp"
"time"
)
// DefaultCollector is where a fresh installation reports, unless it is told
// otherwise. Somebody running their own fleet points it at their own instance;
// the field is on the settings page for exactly that.
const DefaultCollector = "https://verdande.yggdrasilpanel.com/api/v1/beacon"
// Payload is the entire message. Two fields, and adding a third is a change to
// what this program promises, not a change to a struct.
type Payload struct {
InstanceID string `json:"instance_id"`
Version string `json:"version"`
}
// InstanceID and Version are checked on the way in as well as written on the way
// out, because the collector's endpoint is unauthenticated: anything that reaches
// it is a stranger's bytes, and it should be able to store only the two shapes it
// expects.
var (
idShape = regexp.MustCompile(`^[0-9a-fA-F-]{16,64}$`)
versionShape = regexp.MustCompile(`^v?[0-9]{1,4}(\.[0-9]{1,5}){0,3}(-[0-9A-Za-z.]{1,20})?$`)
)
// ValidID reports whether an id is the shape this program writes.
func ValidID(s string) bool { return idShape.MatchString(s) }
// ValidVersion allows a version number and the word this binary uses when it was
// built outside a release.
func ValidVersion(s string) bool { return s == "dev" || versionShape.MatchString(s) }
// Send delivers one ping. It is deliberately dull: a short timeout, no retry, no
// queue, and a failure that is returned rather than made anybody's problem.
//
// No retry, because the next pass is tomorrow and a missed day changes a count of
// installations by nothing. A telemetry ping that retries is a telemetry ping
// that can hammer somebody's server on the day it is down.
func Send(ctx context.Context, client *http.Client, collector string, p Payload) error {
if !ValidID(p.InstanceID) {
return fmt.Errorf("beacon: refusing to send a malformed id")
}
// Versionen kontrolleres også her, ikke kun hos modtageren. Kommentaren ved
// mønstrene lovede begge veje, og gjorde det ikke — og en installation, der
// sender noget mærkeligt, skal opdage det hos sig selv frem for at være noget,
// en fremmed collector skal filtrere fra.
if !ValidVersion(p.Version) {
return fmt.Errorf("beacon: refusing to send a malformed version")
}
body, err := json.Marshal(p)
if err != nil {
return err
}
ctx, cancel := context.WithTimeout(ctx, 15*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodPost, collector, bytes.NewReader(body))
if err != nil {
return err
}
req.Header.Set("Content-Type", "application/json")
// A plain user agent with the version already in the payload. Nothing is
// learned from it that the body does not already say, and it makes the request
// legible in a log rather than anonymous-looking.
req.Header.Set("User-Agent", "verdande-beacon/1")
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
// Read and discard, so the connection can be reused rather than dropped.
_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 4<<10))
if resp.StatusCode >= 400 {
return fmt.Errorf("beacon: collector answered %d", resp.StatusCode)
}
return nil
}
+114
View File
@@ -0,0 +1,114 @@
package beacon
import (
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"testing"
)
// Hvad der bliver sendt, tegn for tegn.
//
// Prøven er skrevet imod *hele* kroppen og ikke imod enkelte felter, fordi det er
// den, der er løftet: siden siger "præcis det her og ikke mere", og et felt lagt
// til senere skal vælte noget. En prøve, der kun så efter, at instance_id var med,
// ville lade en ny nøgle glide igennem uden at nogen opdagede det.
func TestSendsExactlyTwoFields(t *testing.T) {
var got map[string]any
var contentType string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
contentType = r.Header.Get("Content-Type")
body, _ := io.ReadAll(r.Body)
if err := json.Unmarshal(body, &got); err != nil {
t.Errorf("kroppen er ikke JSON: %v", err)
}
w.WriteHeader(http.StatusNoContent)
}))
defer srv.Close()
err := Send(context.Background(), srv.Client(), srv.URL, Payload{
InstanceID: "01a01b16-4e54-7000-9eaa-2f8eb0629c3a",
Version: "v0.26.3",
})
if err != nil {
t.Fatalf("Send: %v", err)
}
if contentType != "application/json" {
t.Errorf("Content-Type = %q", contentType)
}
if len(got) != 2 {
t.Fatalf("der blev sendt %d felter, ikke to: %v", len(got), got)
}
if got["instance_id"] != "01a01b16-4e54-7000-9eaa-2f8eb0629c3a" {
t.Errorf("instance_id = %v", got["instance_id"])
}
if got["version"] != "v0.26.3" {
t.Errorf("version = %v", got["version"])
}
}
// En collector, der er nede, er ikke denne installations problem.
func TestSendReportsAFailureRatherThanRetrying(t *testing.T) {
calls := 0
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
w.WriteHeader(http.StatusInternalServerError)
}))
defer srv.Close()
err := Send(context.Background(), srv.Client(), srv.URL, Payload{
InstanceID: "01a01b16-4e54-7000-9eaa-2f8eb0629c3a", Version: "v1.0.0",
})
if err == nil {
t.Fatal("en 500 skal give en fejl")
}
if calls != 1 {
t.Errorf("der blev prøvet %d gange; en telemetri-ping må ikke hamre på en server, der er nede", calls)
}
}
// Et id, der ikke er id-formet, sendes slet ikke — så en fejl her ikke bliver til
// noget, en fremmed collector skal filtrere fra.
func TestSendRefusesAMalformedID(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("der blev sendt noget, som ikke skulle sendes")
}))
defer srv.Close()
if err := Send(context.Background(), srv.Client(), srv.URL, Payload{
InstanceID: "; DROP TABLE beacon_installs", Version: "v1.0.0",
}); err == nil {
t.Fatal("et misformet id skal afvises før det forlader maskinen")
}
}
func TestShapes(t *testing.T) {
ok := []string{"01a01b16-4e54-7000-9eaa-2f8eb0629c3a", "89df44d7-3742-4611-9a43-db0b07733c6d"}
for _, s := range ok {
if !ValidID(s) {
t.Errorf("ValidID(%q) = false", s)
}
}
bad := []string{"", "kort", "../../etc/passwd", "01a01b16-4e54-7000-9eaa-2f8eb0629c3a'; --",
"<script>alert(1)</script>"}
for _, s := range bad {
if ValidID(s) {
t.Errorf("ValidID(%q) = true", s)
}
}
for _, s := range []string{"v0.26.3", "0.26.3", "v1.2.3-rc.1", "dev"} {
if !ValidVersion(s) {
t.Errorf("ValidVersion(%q) = false", s)
}
}
for _, s := range []string{"", "v<script>", "v" + string(make([]byte, 200))} {
if ValidVersion(s) {
t.Errorf("ValidVersion(%q) = true", s)
}
}
}
+31 -12
View File
@@ -59,10 +59,19 @@ type Config struct {
// without spending the real one on every run.
GmailSyncBudget time.Duration
// GmailEndpoints redirects the Gmail calls at a server the tests control. Empty
// in production, where it means Google.
GmailTokenURL string
GmailAPIURL string
// CalendarSyncBudget is the same bound for one calendar refresh, and exists for
// the same reason: a slow Google held the request open past what Cloudflare is
// willing to wait, and the browser got Cloudflare's HTML instead of an error
// this server could explain.
CalendarSyncBudget time.Duration
// Where Google is. Empty in production; a test points them at a server it
// controls, which is the only way to exercise a Google that is slow or refuses.
// The token endpoint is shared — Gmail and Calendar sign in through the same
// registration — and the two APIs are not.
GoogleTokenURL string
GmailAPIURL string
CalendarAPIURL string
PanelURL string
PanelToken string
@@ -98,14 +107,15 @@ func Load() (*Config, error) {
TrashRetention: 30 * 24 * time.Hour,
Dev: envBool("VERDANDE_DEV", false),
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
GmailSyncBudget: 25 * time.Second,
PanelURL: strings.TrimSuffix(env("VERDANDE_PANEL_URL", ""), "/"),
PanelToken: env("VERDANDE_PANEL_TOKEN", ""),
PanelServerID: env("VERDANDE_PANEL_SERVER_ID", ""),
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
SecretKey: env("VERDANDE_SECRET_KEY", ""),
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
GmailSyncBudget: 25 * time.Second,
CalendarSyncBudget: 25 * time.Second,
PanelURL: strings.TrimSuffix(env("VERDANDE_PANEL_URL", ""), "/"),
PanelToken: env("VERDANDE_PANEL_TOKEN", ""),
PanelServerID: env("VERDANDE_PANEL_SERVER_ID", ""),
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
SecretKey: env("VERDANDE_SECRET_KEY", ""),
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
}
var err error
@@ -152,6 +162,15 @@ func (c *Config) BackupsDir() string { return filepath.Join(c.DataDir, "backups"
// configured separately and left to drift out of step with it.
func (c *Config) GmailRedirectURL() string { return c.BaseURL + "/oauth/gmail/callback" }
// CalendarRedirectURL is a second registered URI rather than a reuse of Gmail's.
//
// One callback that decided from server-side state which of the two flows had come
// back would save the operator one line in Google Cloud, and would put the choice
// of *which token store to write* behind a lookup. When that goes wrong it goes
// wrong by overwriting a working Gmail connection with calendar tokens — a failure
// nobody would think to look for. Two paths cannot do that.
func (c *Config) CalendarRedirectURL() string { return c.BaseURL + "/oauth/calendar/callback" }
func env(key, def string) string {
if v, ok := os.LookupEnv(key); ok && v != "" {
return v
+30
View File
@@ -60,6 +60,20 @@ func Parse(query string, ctx Context) (Compiled, error) {
if len(p.tokens) == 0 {
return Compiled{}, fmt.Errorf("filteret er tomt")
}
// Refused before it is parsed, not while.
//
// `combine` rebuilds the whole accumulated SQL string and re-copies the
// argument slice per operator, so the cost is quadratic in the number of
// terms: fifty thousand of them — three hundred kilobytes, which fits in a
// query string — took seven seconds of CPU and forty-six gigabytes of
// allocation in one request. The preview endpoint runs this on whatever is
// typed, and there is no write timeout on the server to cut it off.
//
// Two hundred terms is a filter nobody has ever written and far below where it
// costs anything.
if len(p.tokens) > maxTokens {
return Compiled{}, fmt.Errorf("filteret er for langt")
}
node, err := p.parseExpression()
if err != nil {
@@ -140,10 +154,19 @@ func isOperator(r rune) bool {
// --- parser -------------------------------------------------------------------
// The ceilings on one filter. See Parse for why.
const (
maxTokens = 200
maxDepth = 32
)
type parser struct {
tokens []token
pos int
ctx Context
// depth guards against a filter that is only parentheses. Each level is a
// stack frame, and nothing else stops somebody sending a megabyte of them.
depth int
}
func (p *parser) done() bool { return p.pos >= len(p.tokens) }
@@ -187,6 +210,13 @@ func (p *parser) parseTerm() (Compiled, error) {
if p.done() {
return Compiled{}, fmt.Errorf("filteret slutter midt i et udtryk")
}
// Nesting is stack depth. A filter made of half a million opening parentheses
// is a megabyte of text and a stack frame for each.
p.depth++
defer func() { p.depth-- }()
if p.depth > maxDepth {
return Compiled{}, fmt.Errorf("filteret har for mange niveauer")
}
switch t := p.peek(); t.kind {
case tokNot:
+339
View File
@@ -0,0 +1,339 @@
// Package gcal reads a Google Calendar. It never writes one.
//
// The scope is calendar.readonly, and that is the whole of the decision: verdande
// already is a CalDAV server, so a Google event shown here is a second calendar
// laid over the first, not a copy Google and verdande now both have to agree
// about. Writing needs `calendar.events`, and it needs an answer to what happens
// when both sides moved the same meeting — which is a synchronisation model, not a
// scope. Read first; write later and to one calendar at a time.
//
// What the API gives back and what this returns are deliberately different shapes.
// Google answers with RFC 3339 instants for a timed event and bare dates for an
// all-day one, and with an *exclusive* end date for the second. A calendar grid
// asks one question — "which cells does this event cover" — so that is what an
// Event answers, in days, worked out once here rather than in every caller.
package gcal
import (
"context"
"fmt"
"net/http"
"net/url"
"strings"
"time"
"github.com/kristianwind/verdande/internal/google"
)
// Scope is read-only, and only the calendar.
//
// Google also offers calendar.calendarlist.readonly and calendar.events.readonly,
// which together are strictly less than this — they leave out access rules and
// account settings, which verdande has no use for. They are the tighter pair and
// worth moving to; they are also newer, and this one has been documented and
// accepted unchanged for a decade. A consent screen that fails with `invalid_scope`
// is not a place to be adventurous on somebody else's Google account.
const Scope = "https://www.googleapis.com/auth/calendar.readonly"
const defaultAPIBase = "https://www.googleapis.com/calendar/v3"
// The OAuth flow, which is Google's rather than the calendar's — the same one
// Gmail signs in through, asking for the scope above. Aliased so a caller reads
// one package for one feature.
type (
Config = google.Config
PKCE = google.PKCE
Token = google.Token
)
var NewPKCE = google.NewPKCE
// ErrUnauthorized means the tokens are no longer good — access was revoked, or the
// password changed. Re-exported so a caller need not know the flow lives elsewhere.
var ErrUnauthorized = google.ErrUnauthorized
type Client struct {
accessToken string
// base is where the Calendar API is. A field rather than a constant so a test
// can point it at a server it controls; empty is Google. There is no other way
// to exercise a calendar that is slow, empty, or refuses.
base string
http *http.Client
}
func NewClient(accessToken string) *Client {
return &Client{accessToken: accessToken, http: &http.Client{Timeout: 30 * time.Second}}
}
// At points the client at another server. Tests only; the zero value is Google.
func (c *Client) At(base string) *Client {
c.base = base
return c
}
func (c *Client) api() string {
if c.base != "" {
return c.base
}
return defaultAPIBase
}
// Calendar is one entry in the person's calendar list.
type Calendar struct {
ID string
// Name is the summary Google holds. A calendar somebody renamed for themselves
// carries summaryOverride instead, and that is the name they will look for.
Name string
// Colour is Google's own hex value for the calendar, which is what makes two
// calendars in one grid tellable apart. Their own colour rather than one picked
// here: it is the colour they already know the calendar by.
Colour string
TimeZone string
Primary bool
// Owner means the connected account can write to it. Nothing writes today —
// it is stored so the day something does, the interface knows which calendars
// could even be offered.
Owner bool
}
// Calendars lists what the connected account can see.
func (c *Client) Calendars(ctx context.Context) ([]Calendar, error) {
var out []Calendar
page := ""
// Bounded rather than "until Google stops": a page token that never empties is
// a loop that never ends, and the difference between a bug at Google and a
// hung sync should not be this server hanging.
for range 10 {
v := url.Values{}
v.Set("minAccessRole", "reader")
v.Set("maxResults", "250")
if page != "" {
v.Set("pageToken", page)
}
var parsed struct {
Items []struct {
ID string `json:"id"`
Summary string `json:"summary"`
SummaryOverride string `json:"summaryOverride"`
BackgroundColor string `json:"backgroundColor"`
TimeZone string `json:"timeZone"`
Primary bool `json:"primary"`
AccessRole string `json:"accessRole"`
Deleted bool `json:"deleted"`
} `json:"items"`
NextPageToken string `json:"nextPageToken"`
}
if err := c.get(ctx, c.api()+"/users/me/calendarList?"+v.Encode(), &parsed); err != nil {
return nil, err
}
for _, item := range parsed.Items {
if item.Deleted {
continue
}
name := item.SummaryOverride
if name == "" {
name = item.Summary
}
out = append(out, Calendar{
ID: item.ID, Name: name, Colour: item.BackgroundColor,
TimeZone: item.TimeZone, Primary: item.Primary,
Owner: item.AccessRole == "owner" || item.AccessRole == "writer",
})
}
if parsed.NextPageToken == "" {
break
}
page = parsed.NextPageToken
}
return out, nil
}
// Event is one occurrence, in the shape a calendar grid asks about.
type Event struct {
ID string
Summary string
// StartsAt and EndsAt are RFC 3339 for a timed event and empty for an all-day
// one — an all-day event has no instant, only a day, and inventing midnight for
// it is how an event moves a day every time a server changes time zone.
StartsAt string
EndsAt string
// StartDay and EndDay are the first and last day the event covers, inclusive at
// both ends. Google's all-day end date is exclusive; that is corrected here,
// once, rather than in every reader.
StartDay string
EndDay string
AllDay bool
Location string
// URL opens the event in Google's own interface. Same reasoning as the Gmail
// link on a task made from a mail: what verdande shows is a pointer to the
// event, not a second copy of it that somebody could try to edit.
URL string
}
// Events lists what falls inside a window, one calendar at a time.
//
// `singleEvents` expands a recurrence into its occurrences, which is the only form
// a grid can draw: an RRULE is an instruction, and a Tuesday cell needs the Tuesday
// it produced. It also means verdande does not have to grow a second recurrence
// engine that agrees with Google's about Danish summer time.
//
// from and to are dates, inclusive. The window is the caller's business; this
// turns it into the half-open interval Google wants.
func (c *Client) Events(ctx context.Context, calendarID, from, to string) ([]Event, error) {
start, err := time.Parse(dateLayout, from)
if err != nil {
return nil, fmt.Errorf("gcal: from: %w", err)
}
end, err := time.Parse(dateLayout, to)
if err != nil {
return nil, fmt.Errorf("gcal: to: %w", err)
}
var out []Event
page := ""
for range 20 {
v := url.Values{}
v.Set("singleEvents", "true")
v.Set("orderBy", "startTime")
v.Set("maxResults", "2500")
// UTC, and a day of slack at each end. An event at 23:30 on the last day in
// Copenhagen is 21:30 UTC that day, but one at 00:30 on the first day is
// 22:30 UTC the day *before* — so a window cut exactly at the requested
// dates loses the events at both edges. The extra day is thrown away again
// by the day fields below, which are computed from the event's own offset.
v.Set("timeMin", start.AddDate(0, 0, -1).Format(time.RFC3339))
v.Set("timeMax", end.AddDate(0, 0, 2).Format(time.RFC3339))
if page != "" {
v.Set("pageToken", page)
}
var parsed struct {
Items []rawEvent `json:"items"`
NextPageToken string `json:"nextPageToken"`
}
url := c.api() + "/calendars/" + url.PathEscape(calendarID) + "/events?" + v.Encode()
if err := c.get(ctx, url, &parsed); err != nil {
return nil, err
}
for _, item := range parsed.Items {
// A cancelled occurrence of a recurring event still comes back, so that
// a client keeping its own copy knows to drop it. verdande replaces the
// window wholesale, so it only has to not add it.
if item.Status == "cancelled" {
continue
}
event, ok := item.event()
if !ok {
continue
}
if event.EndDay < from || event.StartDay > to {
continue // the slack above, thrown away
}
out = append(out, event)
}
if parsed.NextPageToken == "" {
break
}
page = parsed.NextPageToken
}
return out, nil
}
const dateLayout = "2006-01-02"
type rawEvent struct {
ID string `json:"id"`
Status string `json:"status"`
Summary string `json:"summary"`
Location string `json:"location"`
HTMLLink string `json:"htmlLink"`
Start when `json:"start"`
End when `json:"end"`
}
type when struct {
Date string `json:"date"`
DateTime string `json:"dateTime"`
}
// event turns Google's two shapes into the one a grid can read.
//
// The second return is false for an event with neither a date nor a dateTime —
// which Google does return, for an event somebody is still being asked about. It
// belongs on no day, so there is no cell to draw it in.
func (r rawEvent) event() (Event, bool) {
e := Event{
ID: r.ID, Summary: strings.TrimSpace(r.Summary),
Location: r.Location, URL: r.HTMLLink,
}
if e.Summary == "" {
// Google's own interface says this for an event with no title, and a blank
// chip in a grid reads as a rendering fault rather than as an empty title.
e.Summary = "(uden titel)"
}
switch {
case r.Start.Date != "":
e.AllDay = true
e.StartDay = r.Start.Date
// Google's all-day end is the day *after* the last one, so a one-day event
// ends tomorrow. Subtracting a day is what makes "covers this cell" a
// comparison rather than a special case at every call site.
e.EndDay = r.Start.Date
if day, err := time.Parse(dateLayout, r.End.Date); err == nil {
if last := day.AddDate(0, 0, -1); !last.Before(mustDay(r.Start.Date)) {
e.EndDay = last.Format(dateLayout)
}
}
case r.Start.DateTime != "":
e.StartsAt = r.Start.DateTime
e.EndsAt = r.End.DateTime
// The day is the date part of the timestamp Google returned, which carries
// the event's own offset. Parsing it into a time.Time and formatting it
// back would ask *this process* what day it is — and a container running in
// UTC would then move a 23:30 meeting in Copenhagen to the day before.
e.StartDay = dayOf(r.Start.DateTime)
e.EndDay = dayOf(r.End.DateTime)
if e.EndDay == "" {
e.EndDay = e.StartDay
}
// An event ending at midnight ends the evening before, as far as a grid is
// concerned. Without this, every meeting that runs to 24:00 puts a chip in
// the following day as well.
if e.EndDay > e.StartDay && strings.Contains(r.End.DateTime, "T00:00:00") {
if day, err := time.Parse(dateLayout, e.EndDay); err == nil {
e.EndDay = day.AddDate(0, 0, -1).Format(dateLayout)
}
}
default:
return Event{}, false
}
if e.EndDay < e.StartDay {
e.EndDay = e.StartDay
}
return e, true
}
// dayOf takes the date out of an RFC 3339 timestamp without interpreting it.
func dayOf(stamp string) string {
if len(stamp) < len(dateLayout) {
return ""
}
day := stamp[:len(dateLayout)]
if _, err := time.Parse(dateLayout, day); err != nil {
return ""
}
return day
}
func mustDay(s string) time.Time {
day, _ := time.Parse(dateLayout, s)
return day
}
func (c *Client) get(ctx context.Context, url string, out any) error {
return google.Get(ctx, c.http, c.accessToken, url, out)
}
+255
View File
@@ -0,0 +1,255 @@
package gcal
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// fakeGoogle answers the two calls this package makes, out of whatever the test
// hands it. There is no other way to exercise this: the real Calendar needs an
// account, a registered client and somebody's consent, and none of the three can be
// had inside a test — so the shapes Google returns are written down here and the
// parsing is checked against them.
func fakeGoogle(t *testing.T, calendarList any, events map[string]any) *Client {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case strings.HasPrefix(r.URL.Path, "/users/me/calendarList"):
_ = json.NewEncoder(w).Encode(calendarList)
case strings.HasPrefix(r.URL.Path, "/calendars/"):
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/calendars/"), "/events")
body, ok := events[id]
if !ok {
http.Error(w, `{"error":"not found"}`, http.StatusNotFound)
return
}
_ = json.NewEncoder(w).Encode(body)
default:
http.Error(w, "unexpected path "+r.URL.Path, http.StatusNotFound)
}
}))
t.Cleanup(srv.Close)
return NewClient("token").At(srv.URL)
}
func TestCalendarsPrefersTheNameThePersonGaveIt(t *testing.T) {
client := fakeGoogle(t, map[string]any{
"items": []map[string]any{
{"id": "primary", "summary": "kw@nolimit.dk", "backgroundColor": "#3f51b5",
"timeZone": "Europe/Copenhagen", "primary": true, "accessRole": "owner"},
{"id": "fam", "summary": "Family", "summaryOverride": "Familien",
"backgroundColor": "#0b8043", "accessRole": "reader"},
// A calendar removed from the list still arrives, flagged.
{"id": "gone", "summary": "Væk", "deleted": true, "accessRole": "reader"},
},
}, nil)
got, err := client.Calendars(t.Context())
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d calendars, want 2 — a deleted one is not one you can show", len(got))
}
if got[1].Name != "Familien" {
t.Errorf("name = %q; summaryOverride is the name the person will look for", got[1].Name)
}
if !got[0].Primary || !got[0].Owner {
t.Errorf("the primary calendar came back as %+v", got[0])
}
if got[1].Owner {
t.Error("a reader was reported as an owner; nothing may offer to write to it")
}
if got[0].Colour != "#3f51b5" {
t.Errorf("colour = %q; two calendars in one grid are told apart by it", got[0].Colour)
}
}
// The three shapes an event arrives in, and the one shape a grid can read.
func TestEventsBecomeDaysAGridCanAskAbout(t *testing.T) {
client := fakeGoogle(t, nil, map[string]any{
"primary": map[string]any{
"items": []map[string]any{
{
"id": "timed", "summary": "Bestyrelsesmøde", "htmlLink": "https://cal/1",
"start": map[string]string{"dateTime": "2026-08-20T14:00:00+02:00"},
"end": map[string]string{"dateTime": "2026-08-20T15:30:00+02:00"},
},
{
// All-day, one day. Google's end is the day after.
"id": "allday", "summary": "Fridag",
"start": map[string]string{"date": "2026-08-21"},
"end": map[string]string{"date": "2026-08-22"},
},
{
// All-day, three days.
"id": "trip", "summary": "Aarhus",
"start": map[string]string{"date": "2026-08-24"},
"end": map[string]string{"date": "2026-08-27"},
},
{
// Runs to midnight, which is the evening before as far as a
// grid is concerned.
"id": "late", "summary": "Nattevagt",
"start": map[string]string{"dateTime": "2026-08-22T20:00:00+02:00"},
"end": map[string]string{"dateTime": "2026-08-23T00:00:00+02:00"},
},
{
// A cancelled occurrence of a recurring event.
"id": "off", "status": "cancelled", "summary": "Aflyst",
"start": map[string]string{"date": "2026-08-25"},
"end": map[string]string{"date": "2026-08-26"},
},
{
// Neither a date nor a dateTime: belongs on no day at all.
"id": "nowhere", "summary": "Uden tid",
},
},
},
})
got, err := client.Events(t.Context(), "primary", "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
byID := map[string]Event{}
for _, e := range got {
byID[e.ID] = e
}
if len(got) != 4 {
t.Fatalf("got %d events, want 4: %+v", len(got), got)
}
if _, ok := byID["off"]; ok {
t.Error("a cancelled occurrence was kept; it is an event that is not happening")
}
if _, ok := byID["nowhere"]; ok {
t.Error("an event with no time was kept; there is no cell to draw it in")
}
timed := byID["timed"]
if timed.AllDay || timed.StartDay != "2026-08-20" || timed.EndDay != "2026-08-20" {
t.Errorf("timed = %+v", timed)
}
if timed.StartsAt != "2026-08-20T14:00:00+02:00" {
t.Errorf("the instant was rewritten: %q", timed.StartsAt)
}
allday := byID["allday"]
if !allday.AllDay || allday.StartDay != "2026-08-21" || allday.EndDay != "2026-08-21" {
t.Errorf("allday = %+v; Google's end date is exclusive and has to be corrected here",
allday)
}
if allday.StartsAt != "" {
t.Errorf("an all-day event was given an instant: %q — it has a day, not a moment",
allday.StartsAt)
}
if trip := byID["trip"]; trip.StartDay != "2026-08-24" || trip.EndDay != "2026-08-26" {
t.Errorf("trip = %+v, want 24th to 26th inclusive", trip)
}
if late := byID["late"]; late.EndDay != "2026-08-22" {
t.Errorf("late ends on %q; an event running to midnight ends the evening before, "+
"or every one of them puts a chip in the following day", late.EndDay)
}
}
// An event with no title has to say so. A blank chip in a grid reads as a
// rendering fault rather than as an empty title.
func TestAnEventWithNoTitleStillSaysSomething(t *testing.T) {
client := fakeGoogle(t, nil, map[string]any{
"c": map[string]any{"items": []map[string]any{{
"id": "x",
"start": map[string]string{"date": "2026-08-20"},
"end": map[string]string{"date": "2026-08-21"},
}}},
})
got, err := client.Events(t.Context(), "c", "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].Summary == "" {
t.Fatalf("got %+v", got)
}
}
// The window is asked for with a day of slack at each end, because an event late
// in the evening in Copenhagen is the next day in UTC. The slack must not leak
// out: an event on the day before the window is not in the window.
func TestTheSlackAtTheEdgesIsThrownAwayAgain(t *testing.T) {
client := fakeGoogle(t, nil, map[string]any{
"c": map[string]any{"items": []map[string]any{
{"id": "before", "summary": "For tidligt",
"start": map[string]string{"date": "2026-08-09"},
"end": map[string]string{"date": "2026-08-10"}},
{"id": "inside", "summary": "Med",
"start": map[string]string{"date": "2026-08-10"},
"end": map[string]string{"date": "2026-08-11"}},
{"id": "after", "summary": "For sent",
"start": map[string]string{"date": "2026-08-21"},
"end": map[string]string{"date": "2026-08-22"}},
}},
})
got, err := client.Events(t.Context(), "c", "2026-08-10", "2026-08-20")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].ID != "inside" {
t.Fatalf("got %+v, want only the one inside the window", got)
}
}
// A page token has to be followed, or a busy calendar silently shows its first
// page and nothing else — which is a calendar that lies about a day being clear.
func TestEveryPageIsRead(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
body := map[string]any{
"items": []map[string]any{{"id": "a", "summary": "Første",
"start": map[string]string{"date": "2026-08-11"},
"end": map[string]string{"date": "2026-08-12"}}},
"nextPageToken": "more",
}
if r.URL.Query().Get("pageToken") == "more" {
body = map[string]any{"items": []map[string]any{{"id": "b", "summary": "Anden",
"start": map[string]string{"date": "2026-08-12"},
"end": map[string]string{"date": "2026-08-13"}}}}
}
_ = json.NewEncoder(w).Encode(body)
}))
defer srv.Close()
got, err := NewClient("t").At(srv.URL).Events(t.Context(), "c", "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d events, want both pages: %+v", len(got), got)
}
}
// A 401 is not a failure to retry; it is a connection that has ended. Telling the
// two apart is what stops a revoked account being polled every ten minutes for ever.
func TestARefusedTokenIsItsOwnError(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, `{"error":"invalid_credentials"}`, http.StatusUnauthorized)
}))
defer srv.Close()
_, err := NewClient("spent").At(srv.URL).Calendars(t.Context())
if err != ErrUnauthorized {
t.Fatalf("err = %v, want ErrUnauthorized", err)
}
}
// Read-only, and only the calendar. The consent screen is what somebody agrees to,
// and a scope that quietly grew is a promise quietly broken.
func TestTheScopeIsReadOnly(t *testing.T) {
if !strings.HasSuffix(Scope, "calendar.readonly") {
t.Errorf("scope = %q; nothing here writes a calendar", Scope)
}
}
+41 -213
View File
@@ -1,5 +1,5 @@
// Package gmail implements the OAuth2 authorisation-code flow with PKCE against
// Google, and the small slice of the Gmail API verdande needs.
// Package gmail is the small slice of the Gmail API verdande needs, and the scope
// it asks for.
//
// The connection is one-way on purpose: a starred or labelled message becomes a
// task, and nothing verdande does afterwards touches the mailbox. Unstarring the
@@ -9,200 +9,51 @@
//
// Scope is gmail.readonly. verdande never needs to send, modify or delete, and
// asking for less is what makes the consent screen something a person can agree to.
//
// The OAuth flow itself moved to internal/google when Calendar needed the same one.
// The names below are aliases so that every call site — and this package's own
// tests — still read as Gmail's, which is what they are: one registration, one
// flow, one scope each.
package gmail
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"time"
"github.com/kristianwind/verdande/internal/google"
)
const (
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
defaultTokenEndpoint = "https://oauth2.googleapis.com/token"
defaultAPIBase = "https://gmail.googleapis.com/gmail/v1"
// Read-only, and only mail. Not profile, not contacts, not send.
const Scope = "https://www.googleapis.com/auth/gmail.readonly"
// Read-only, and only mail. Not profile, not contacts, not send.
Scope = "https://www.googleapis.com/auth/gmail.readonly"
const defaultAPIBase = "https://gmail.googleapis.com/gmail/v1"
// The OAuth flow, which is Google's rather than Gmail's.
type (
Config = google.Config
PKCE = google.PKCE
Token = google.Token
)
// Endpoints is where Google is.
//
// A field rather than a constant so a test can point it at a server it controls.
// There is no other way to exercise what happens when Gmail is slow or refuses —
// and "the sync comes back before whatever sits in front of this server gives up
// on it" is exactly the kind of promise worth a test and impossible to make
// against the real thing.
//
// The zero value is Google, so nothing outside a test has to know it exists.
type Endpoints struct {
Token string
API string
}
func (e Endpoints) token() string {
if e.Token != "" {
return e.Token
}
return defaultTokenEndpoint
}
func (e Endpoints) api() string {
if e.API != "" {
return e.API
}
return defaultAPIBase
}
// Config is the OAuth client the operator registered in Google Cloud. It belongs to
// the instance rather than to a person: one registration, any number of users.
type Config struct {
ClientID string
ClientSecret string
RedirectURL string
// Endpoints is Google unless a test says otherwise.
Endpoints Endpoints
}
func (c Config) Configured() bool {
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
}
// PKCE is the proof-key pair for one authorisation attempt.
//
// PKCE on a server-side client is not strictly required — the client secret already
// authenticates the token exchange — but it costs nothing and closes the window in
// which an authorisation code intercepted from the redirect could be replayed
// before verdande gets to it.
type PKCE struct {
Verifier string
Challenge string
State string
}
func NewPKCE() (PKCE, error) {
verifier, err := randomString(64)
if err != nil {
return PKCE{}, err
}
state, err := randomString(32)
if err != nil {
return PKCE{}, err
}
sum := sha256.Sum256([]byte(verifier))
return PKCE{
Verifier: verifier,
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
State: state,
}, nil
}
// AuthURL is where the browser is sent to ask the person for consent.
func (c Config) AuthURL(p PKCE) string {
v := url.Values{}
v.Set("client_id", c.ClientID)
v.Set("redirect_uri", c.RedirectURL)
v.Set("response_type", "code")
v.Set("scope", Scope)
v.Set("state", p.State)
v.Set("code_challenge", p.Challenge)
v.Set("code_challenge_method", "S256")
// Without both of these Google returns no refresh token on a re-authorisation,
// and the connection silently stops working an hour later.
v.Set("access_type", "offline")
v.Set("prompt", "consent")
return authEndpoint + "?" + v.Encode()
}
type Token struct {
AccessToken string
RefreshToken string
ExpiresAt time.Time
}
// Exchange trades the authorisation code for tokens.
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
form := url.Values{}
form.Set("code", code)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("redirect_uri", c.RedirectURL)
form.Set("grant_type", "authorization_code")
form.Set("code_verifier", p.Verifier)
return c.token(ctx, form)
}
// Refresh gets a fresh access token. Google does not return a new refresh token
// here, so the caller keeps the one it has.
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
form := url.Values{}
form.Set("refresh_token", refreshToken)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("grant_type", "refresh_token")
token, err := c.token(ctx, form)
if err != nil {
return Token{}, err
}
token.RefreshToken = refreshToken
return token, nil
}
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.Endpoints.token(),
strings.NewReader(form.Encode()))
if err != nil {
return Token{}, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
if err != nil {
return Token{}, err
}
defer resp.Body.Close()
var parsed struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresIn int `json:"expires_in"`
Error string `json:"error"`
ErrorDescription string `json:"error_description"`
}
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return Token{}, fmt.Errorf("gmail: reading token response: %w", err)
}
if parsed.Error != "" {
return Token{}, fmt.Errorf("gmail: %s: %s", parsed.Error, parsed.ErrorDescription)
}
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
return Token{}, fmt.Errorf("gmail: token exchange failed with %s", resp.Status)
}
return Token{
AccessToken: parsed.AccessToken,
RefreshToken: parsed.RefreshToken,
// A minute of slack, so a token that is about to expire is refreshed
// rather than used and rejected.
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
}, nil
}
var (
NewPKCE = google.NewPKCE
ErrUnauthorized = google.ErrUnauthorized
)
// --- the API ---------------------------------------------------------------------
type Client struct {
accessToken string
endpoints Endpoints
http *http.Client
// base is where the Gmail API is. A field rather than a constant so a test can
// point it at a server it controls — and "the sync comes back before whatever
// sits in front of this server gives up on it" is exactly the kind of promise
// worth a test and impossible to make against the real thing. Empty is Google.
base string
http *http.Client
}
func NewClient(accessToken string) *Client {
@@ -210,11 +61,18 @@ func NewClient(accessToken string) *Client {
}
// At points the client at another server. Tests only; the zero value is Google.
func (c *Client) At(e Endpoints) *Client {
c.endpoints = e
func (c *Client) At(base string) *Client {
c.base = base
return c
}
func (c *Client) api() string {
if c.base != "" {
return c.base
}
return defaultAPIBase
}
// Message is the part of a Gmail message a task is made from.
type Message struct {
ID string
@@ -233,7 +91,7 @@ func (c *Client) Profile(ctx context.Context) (string, error) {
var parsed struct {
EmailAddress string `json:"emailAddress"`
}
if err := c.get(ctx, c.endpoints.api()+"/users/me/profile", &parsed); err != nil {
if err := c.get(ctx, c.api()+"/users/me/profile", &parsed); err != nil {
return "", err
}
return parsed.EmailAddress, nil
@@ -257,7 +115,7 @@ func (c *Client) List(ctx context.Context, query string, max int) ([]string, err
ID string `json:"id"`
} `json:"messages"`
}
if err := c.get(ctx, c.endpoints.api()+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
if err := c.get(ctx, c.api()+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
return nil, err
}
@@ -288,7 +146,7 @@ func (c *Client) Get(ctx context.Context, id string) (Message, error) {
} `json:"headers"`
} `json:"payload"`
}
if err := c.get(ctx, c.endpoints.api()+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
if err := c.get(ctx, c.api()+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
return Message{}, err
}
@@ -308,31 +166,9 @@ func (c *Client) Get(ctx context.Context, id string) (Message, error) {
}
func (c *Client) get(ctx context.Context, url string, out any) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+c.accessToken)
resp, err := c.http.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusUnauthorized {
return ErrUnauthorized
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("gmail: %s", resp.Status)
}
return json.NewDecoder(resp.Body).Decode(out)
return google.Get(ctx, c.http, c.accessToken, url, out)
}
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
// or changed their password. The caller disconnects rather than retrying.
var ErrUnauthorized = fmt.Errorf("gmail: authorisation was refused")
// SenderName pulls "Anders Jensen" out of "Anders Jensen <anders@example.dk>",
// falling back to the address when there is no display name.
func SenderName(from string) string {
@@ -375,11 +211,3 @@ func Query(trigger, label string) string {
terms = append(terms, "newer_than:30d")
return strings.Join(terms, " ")
}
func randomString(n int) (string, error) {
b := make([]byte, n)
if _, err := rand.Read(b); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
}
+4 -98
View File
@@ -1,113 +1,19 @@
package gmail
import (
"crypto/sha256"
"encoding/base64"
"net/url"
"strings"
"testing"
)
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
// Getting this wrong means Google refuses every exchange with a message that does
// not say which end is at fault.
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256([]byte(p.Verifier))
want := base64.RawURLEncoding.EncodeToString(sum[:])
if p.Challenge != want {
t.Errorf("challenge = %q, want %q", p.Challenge, want)
}
// RFC 7636 requires 43–128 characters, and no padding.
if n := len(p.Verifier); n < 43 || n > 128 {
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
}
if strings.ContainsAny(p.Verifier, "=+/") {
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
}
if p.State == "" {
t.Error("no state was generated")
}
// Two attempts must not share anything.
other, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
if other.Verifier == p.Verifier || other.State == p.State {
t.Error("two authorisation attempts produced the same secrets")
}
}
func TestAuthURL(t *testing.T) {
cfg := Config{
ClientID: "client-123",
ClientSecret: "secret",
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
}
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
raw := cfg.AuthURL(p)
parsed, err := url.Parse(raw)
if err != nil {
t.Fatalf("the URL does not parse: %v", err)
}
q := parsed.Query()
checks := map[string]string{
"client_id": cfg.ClientID,
"redirect_uri": cfg.RedirectURL,
"response_type": "code",
"scope": Scope,
"code_challenge": p.Challenge,
"code_challenge_method": "S256",
"state": p.State,
// Without both of these Google returns no refresh token on a
// re-authorisation, and the connection dies silently an hour later.
"access_type": "offline",
"prompt": "consent",
}
for key, want := range checks {
if got := q.Get(key); got != want {
t.Errorf("%s = %q, want %q", key, got, want)
}
}
// The verifier itself must never leave the server.
if strings.Contains(raw, p.Verifier) {
t.Error("the code verifier was put in the authorisation URL")
}
// Read-only, and only mail.
// The scope is the one thing verdande chooses about the connection, and it is the
// one the person reads on the consent screen. The flow itself is exercised in
// internal/google; what belongs here is that Gmail still asks for read-only mail.
func TestTheScopeIsReadOnlyMail(t *testing.T) {
if !strings.HasSuffix(Scope, "gmail.readonly") {
t.Errorf("scope = %q; verdande never needs to send or modify", Scope)
}
}
func TestConfigured(t *testing.T) {
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
if !full.Configured() {
t.Error("a complete config is not reported as configured")
}
for _, partial := range []Config{
{},
{ClientID: "a"},
{ClientID: "a", ClientSecret: "b"},
{ClientSecret: "b", RedirectURL: "c"},
} {
if partial.Configured() {
t.Errorf("%+v was reported as configured", partial)
}
}
}
func TestQuery(t *testing.T) {
cases := []struct {
trigger, label string
+224
View File
@@ -0,0 +1,224 @@
// Package google implements the OAuth2 authorisation-code flow with PKCE against
// Google.
//
// It was written inside internal/gmail, because Gmail was the only thing that
// needed it. Calendar needs the same flow — the same endpoints, the same PKCE, the
// same refresh — and the only thing that differs between them is one string: the
// scope. A second copy would be a second place for "access_type=offline" to be
// forgotten, which is the mistake that makes a connection die silently an hour
// after it was made.
//
// What is *not* here is the API calls. Those differ completely, so gmail and gcal
// keep their own clients and share only the way in.
package google
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"net/url"
"strings"
"time"
)
const (
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
defaultTokenEndpoint = "https://oauth2.googleapis.com/token"
)
// Config is the OAuth client the operator registered in Google Cloud, plus the
// scope one feature is asking for.
//
// The client belongs to the instance rather than to a person: one registration,
// any number of users. The scope belongs to the feature: Gmail and Calendar sign
// in through the same registration and ask for different things, and Google issues
// a refresh token per authorisation — so connecting one does not disturb the other.
type Config struct {
ClientID string
ClientSecret string
RedirectURL string
Scope string
// TokenURL is where Google is. A field rather than a constant so a test can
// point it at a server it controls: there is no other way to exercise what
// happens when the token endpoint is slow or refuses. Empty is Google, so
// nothing outside a test has to know it exists.
TokenURL string
}
func (c Config) Configured() bool {
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
}
func (c Config) tokenURL() string {
if c.TokenURL != "" {
return c.TokenURL
}
return defaultTokenEndpoint
}
// PKCE is the proof-key pair for one authorisation attempt.
//
// PKCE on a server-side client is not strictly required — the client secret already
// authenticates the token exchange — but it costs nothing and closes the window in
// which an authorisation code intercepted from the redirect could be replayed
// before verdande gets to it.
type PKCE struct {
Verifier string
Challenge string
State string
}
func NewPKCE() (PKCE, error) {
verifier, err := randomString(64)
if err != nil {
return PKCE{}, err
}
state, err := randomString(32)
if err != nil {
return PKCE{}, err
}
sum := sha256.Sum256([]byte(verifier))
return PKCE{
Verifier: verifier,
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
State: state,
}, nil
}
// AuthURL is where the browser is sent to ask the person for consent.
func (c Config) AuthURL(p PKCE) string {
v := url.Values{}
v.Set("client_id", c.ClientID)
v.Set("redirect_uri", c.RedirectURL)
v.Set("response_type", "code")
v.Set("scope", c.Scope)
v.Set("state", p.State)
v.Set("code_challenge", p.Challenge)
v.Set("code_challenge_method", "S256")
// Without both of these Google returns no refresh token on a re-authorisation,
// and the connection silently stops working an hour later.
v.Set("access_type", "offline")
v.Set("prompt", "consent")
return authEndpoint + "?" + v.Encode()
}
type Token struct {
AccessToken string
RefreshToken string
ExpiresAt time.Time
}
// Exchange trades the authorisation code for tokens.
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
form := url.Values{}
form.Set("code", code)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("redirect_uri", c.RedirectURL)
form.Set("grant_type", "authorization_code")
form.Set("code_verifier", p.Verifier)
return c.token(ctx, form)
}
// Refresh gets a fresh access token. Google does not return a new refresh token
// here, so the caller keeps the one it has.
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
form := url.Values{}
form.Set("refresh_token", refreshToken)
form.Set("client_id", c.ClientID)
form.Set("client_secret", c.ClientSecret)
form.Set("grant_type", "refresh_token")
token, err := c.token(ctx, form)
if err != nil {
return Token{}, err
}
token.RefreshToken = refreshToken
return token, nil
}
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.tokenURL(),
strings.NewReader(form.Encode()))
if err != nil {
return Token{}, err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
if err != nil {
return Token{}, err
}
defer resp.Body.Close()
var parsed struct {
AccessToken string `json:"access_token"`
RefreshToken string `json:"refresh_token"`
ExpiresIn int `json:"expires_in"`
Error string `json:"error"`
ErrorDescription string `json:"error_description"`
}
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
return Token{}, fmt.Errorf("google: reading token response: %w", err)
}
if parsed.Error != "" {
return Token{}, fmt.Errorf("google: %s: %s", parsed.Error, parsed.ErrorDescription)
}
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
return Token{}, fmt.Errorf("google: token exchange failed with %s", resp.Status)
}
return Token{
AccessToken: parsed.AccessToken,
RefreshToken: parsed.RefreshToken,
// A minute of slack, so a token that is about to expire is refreshed
// rather than used and rejected.
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
}, nil
}
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
// or changed their password. The caller disconnects rather than retrying.
var ErrUnauthorized = fmt.Errorf("google: authorisation was refused")
// Get performs an authorised GET and decodes the JSON body into out.
//
// Shared by the two API clients because the interesting part is the same in both:
// a 401 is not a failure to retry, it is a connection that has ended, and telling
// those apart is what stops a revoked account being polled every ten minutes for
// ever.
func Get(ctx context.Context, client *http.Client, accessToken, url string, out any) error {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+accessToken)
resp, err := client.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusUnauthorized {
return ErrUnauthorized
}
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("google: %s", resp.Status)
}
return json.NewDecoder(resp.Body).Decode(out)
}
func randomString(n int) (string, error) {
b := make([]byte, n)
if _, err := rand.Read(b); err != nil {
return "", err
}
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
}
+127
View File
@@ -0,0 +1,127 @@
package google
import (
"crypto/sha256"
"encoding/base64"
"net/url"
"strings"
"testing"
)
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
// Getting this wrong means Google refuses every exchange with a message that does
// not say which end is at fault.
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256([]byte(p.Verifier))
want := base64.RawURLEncoding.EncodeToString(sum[:])
if p.Challenge != want {
t.Errorf("challenge = %q, want %q", p.Challenge, want)
}
// RFC 7636 requires 43–128 characters, and no padding.
if n := len(p.Verifier); n < 43 || n > 128 {
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
}
if strings.ContainsAny(p.Verifier, "=+/") {
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
}
if p.State == "" {
t.Error("no state was generated")
}
// Two attempts must not share anything.
other, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
if other.Verifier == p.Verifier || other.State == p.State {
t.Error("two authorisation attempts produced the same secrets")
}
}
func TestAuthURL(t *testing.T) {
cfg := Config{
ClientID: "client-123",
ClientSecret: "secret",
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
Scope: "https://www.googleapis.com/auth/gmail.readonly",
}
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
raw := cfg.AuthURL(p)
parsed, err := url.Parse(raw)
if err != nil {
t.Fatalf("the URL does not parse: %v", err)
}
q := parsed.Query()
checks := map[string]string{
"client_id": cfg.ClientID,
"redirect_uri": cfg.RedirectURL,
"response_type": "code",
"scope": cfg.Scope,
"code_challenge": p.Challenge,
"code_challenge_method": "S256",
"state": p.State,
// Without both of these Google returns no refresh token on a
// re-authorisation, and the connection dies silently an hour later.
"access_type": "offline",
"prompt": "consent",
}
for key, want := range checks {
if got := q.Get(key); got != want {
t.Errorf("%s = %q, want %q", key, got, want)
}
}
// The verifier itself must never leave the server.
if strings.Contains(raw, p.Verifier) {
t.Error("the code verifier was put in the authorisation URL")
}
}
// The scope is the one thing that differs between the two features signing in
// through this, so it has to come out of the config rather than out of a constant
// somebody could forget to change.
func TestTheScopeComesFromTheConfig(t *testing.T) {
p, err := NewPKCE()
if err != nil {
t.Fatal(err)
}
calendar := "https://www.googleapis.com/auth/calendar.readonly"
raw := Config{ClientID: "a", Scope: calendar}.AuthURL(p)
parsed, err := url.Parse(raw)
if err != nil {
t.Fatal(err)
}
if got := parsed.Query().Get("scope"); got != calendar {
t.Errorf("scope = %q, want %q — asking for the wrong one is a consent screen "+
"that grants the wrong thing", got, calendar)
}
}
func TestConfigured(t *testing.T) {
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
if !full.Configured() {
t.Error("a complete config is not reported as configured")
}
for _, partial := range []Config{
{},
{ClientID: "a"},
{ClientID: "a", ClientSecret: "b"},
{ClientSecret: "b", RedirectURL: "c"},
} {
if partial.Configured() {
t.Errorf("%+v was reported as configured", partial)
}
}
}
+42 -1
View File
@@ -154,7 +154,21 @@ func (s *Server) handleLogin(w http.ResponseWriter, r *http.Request) {
}
}
s.loginLimiter.reset(clientIP(r))
// The bucket is cleared only when the login is *finished*.
//
// It used to be cleared here, on the strength of the password alone — before
// the second factor had been asked for at all. That handed somebody who
// already had the password an unlimited supply of ten-guess batches: log in,
// get a fresh bucket and a fresh pending session, spend ten guesses on the
// code, log in again. At three valid codes in any moment and twenty requests a
// second, that is even odds inside an afternoon — against the one control that
// exists precisely for when the password is already gone.
//
// So an account with a second factor clears nothing yet. handleLoginTOTP does
// it, once the code is right.
if !user.TOTPEnabled {
s.loginLimiter.reset(clientIP(r))
}
// A session that still needs a code gets a short life of its own: long enough
// to fetch a phone, not long enough to be worth stealing.
@@ -191,6 +205,19 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
session := sessionFrom(r.Context())
// Counted per account as well as per address.
//
// The address bucket alone is defeated by anybody with more than one, which is
// the ordinary case — a phone and a laptop is already two. A code is six
// digits: the only thing that makes it safe is that guesses are scarce, and
// scarcity has to be counted against the thing being attacked.
if ok, retry := s.totpLimiter.allow(user.ID); !ok {
w.Header().Set("Retry-After", retryAfterSeconds(retry))
writeError(w, http.StatusTooManyRequests, CodeRateLimited,
"too many verification codes; wait a moment and try again")
return
}
err := auth.VerifyTOTP(user.TOTPSecret, req.Code, time.Now())
if err != nil {
// A recovery code is accepted in the same field. Making somebody find a
@@ -198,6 +225,15 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
// introduce a second concept.
used, rerr := s.db.UseRecoveryCode(r.Context(), user.ID, req.Code)
if rerr != nil || !used {
// The pending session is left alive on purpose.
//
// Ending it on the first wrong digit was considered and rejected: it
// would make somebody who mistyped one number type their password again,
// and it buys almost nothing. What makes a six-digit code safe is that
// guesses are scarce, and the per-account bucket above is what makes
// them scarce — ten an hour, however many times the password is
// re-entered. The attack this closes was never "one session, many
// guesses"; it was "a fresh bucket for every password check".
writeError(w, http.StatusUnauthorized, CodeUnauthorized,
"that verification code is not valid")
return
@@ -205,6 +241,11 @@ func (s *Server) handleLoginTOTP(w http.ResponseWriter, r *http.Request) {
s.log.Info("recovery code used", "user", user.ID)
}
// Finished. Now the buckets may be cleared — both of them, so somebody who
// fumbled the code twice is not still waiting once they get it right.
s.loginLimiter.reset(clientIP(r))
s.totpLimiter.reset(user.ID)
if err := s.db.PromoteSession(r.Context(), session.ID, s.cfg.SessionTTL); err != nil {
s.internal(w, r, "promote session", err)
return
+76 -3
View File
@@ -27,6 +27,9 @@ type testServer struct {
*httptest.Server
db *store.DB
client *http.Client
// api er den samme server, routeren hænger på. Et par prøver har brug for at
// nå ind i den — se `icsFetch` — og uden den her er der kun håndtaget udefra.
api *Server
}
func newTestServer(t *testing.T) *testServer {
@@ -62,7 +65,8 @@ func newTestServerWith(t *testing.T, configure func(*config.Config)) *testServer
// Discard: a passing test should print nothing, and these handlers log freely.
log := slog.New(slog.NewTextHandler(io.Discard, nil))
srv := httptest.NewServer(New(cfg, db, log, nil))
api := New(cfg, db, log, nil)
srv := httptest.NewServer(api)
t.Cleanup(srv.Close)
jar, err := cookiejar.New(nil)
@@ -75,6 +79,7 @@ func newTestServerWith(t *testing.T, configure func(*config.Config)) *testServer
// The cookie jar is what makes this a session test rather than a
// sequence of unrelated requests.
client: &http.Client{Jar: jar, Timeout: 10 * time.Second},
api: api,
}
}
@@ -301,7 +306,11 @@ func TestTOTPEnrolmentAndTwoStepLogin(t *testing.T) {
// Begin: a secret exists, but 2FA is not on yet — closing the tab here must
// not lock the account.
resp, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", nil)
// Med adgangskoden: at slå andenfaktoren til er mindst lige så indgribende som
// at slå den fra, og den har altid spurgt.
resp, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "et langt kodeord",
})
if resp.StatusCode != http.StatusOK {
t.Fatalf("totp setup: %d %v", resp.StatusCode, body)
}
@@ -368,7 +377,9 @@ func TestRecoveryCodeCompletesLoginOnce(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", nil)
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "et langt kodeord",
})
secret := body["secret"].(string)
_, body = ts.do(t, "POST", "/api/v1/auth/totp/confirm", map[string]string{
"code": codeFor(t, secret),
@@ -897,3 +908,65 @@ func TestUserAgentsAreSummarisedForAPerson(t *testing.T) {
}
}
}
// Andenfaktoren kan ikke slås til uden adgangskoden.
//
// Døren åbnede før med et token og lukkede kun med et kodeord: `disable` har
// altid spurgt, `setup` spurgte om ingenting. Et stjålet token kunne derfor hente
// hemmeligheden, lægge den i sin egen godkender, bekræfte den og beholde den
// eneste kopi af gendannelseskoderne — hvorefter ejerens næste login stoppede ved
// en kode, de ikke havde, og de ikke kunne slå den fra uden at logge ind først.
func TestTOTPCannotBeSwitchedOnWithoutThePassword(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, _ := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "det forkerte kodeord",
})
if resp.StatusCode != http.StatusUnprocessableEntity {
t.Fatalf("status %d, ventede at en forkert adgangskode blev afvist", resp.StatusCode)
}
resp, _ = ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{})
if resp.StatusCode == http.StatusOK {
t.Fatal("en tom adgangskode blev accepteret")
}
}
// Ratebegrænseren må ikke nulstilles af adgangskoden alene.
//
// Den blev det, og det gav én, der havde kodeordet, ubegrænsede portioner på ti
// gæt: log ind, få en frisk spand og en frisk ventende session, brug ti gæt på
// koden, log ind igen. Andenfaktoren findes præcis til det tilfælde, hvor
// kodeordet allerede er tabt, så den portion skal betales for hver gang.
func TestAPasswordAloneDoesNotRefillTheCodeAttempts(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, body := ts.do(t, "POST", "/api/v1/auth/totp/setup", map[string]string{
"password": "et langt kodeord",
})
secret := body["secret"].(string)
ts.do(t, "POST", "/api/v1/auth/totp/confirm", map[string]string{"code": codeFor(t, secret)})
ts.do(t, "POST", "/api/v1/auth/logout", nil)
// Kodeordet er rigtigt hver gang; kun koden er forkert. Uden loftet ville det
// her kunne blive ved i det uendelige.
refused := 0
for i := 0; i < 14; i++ {
resp, _ := ts.do(t, "POST", "/api/v1/auth/login", map[string]string{
"email": "kristian@example.dk", "password": "et langt kodeord",
})
if resp.StatusCode == http.StatusTooManyRequests {
refused++
continue
}
resp, _ = ts.do(t, "POST", "/api/v1/auth/login/totp", map[string]string{"code": "000000"})
if resp.StatusCode == http.StatusTooManyRequests {
refused++
}
}
if refused == 0 {
t.Fatal("fjorten runder med forkerte koder blev aldrig afvist; spanden fyldes stadig af adgangskoden")
}
}
+297
View File
@@ -0,0 +1,297 @@
package httpapi
import (
"context"
"net/http"
"strings"
"time"
"github.com/kristianwind/verdande/internal/beacon"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
// The beacon, from the instance's side.
//
// Two halves that happen to live in the same binary: the part that reports in,
// which every installation runs, and the part that receives, which only the one
// acting as collector turns on. See internal/beacon for what is sent and why it
// is on by default rather than compulsory.
const beaconScope = "beacon"
// beaconConfig is what this instance has been told to do. The zero value is what
// a fresh installation gets, which is why Enabled defaults to true elsewhere and
// not here — a zero struct must never mean "silently on".
type beaconConfig struct {
Enabled bool `json:"enabled"`
Collector string `json:"collector_url"`
InstanceID string `json:"instance_id"`
IsCollector bool `json:"is_collector"`
PublishCount bool `json:"publish_count"`
LastPingAt int64 `json:"last_ping_at,omitempty"`
}
// beaconSettings reads the configuration, filling in the defaults and minting the
// anonymous id on first use.
//
// The id is generated here rather than derived from anything about the machine.
// A hash of the hostname or the base URL would be stable and convenient and would
// also be a value the collector could compare against a guess — which is the
// difference between anonymous and pseudonymous, and it is the whole point.
func (s *Server) beaconSettings(ctx context.Context) (beaconConfig, error) {
cfg := beaconConfig{Enabled: true, Collector: beacon.DefaultCollector}
stored, err := s.db.InstanceSettings(ctx, beaconScope)
if err != nil {
return cfg, err
}
if v, ok := stored["enabled"].(bool); ok {
cfg.Enabled = v
}
if v, ok := stored["collector_url"].(string); ok && strings.TrimSpace(v) != "" {
cfg.Collector = strings.TrimSpace(v)
}
if v, ok := stored["instance_id"].(string); ok && beacon.ValidID(v) {
cfg.InstanceID = v
}
if v, ok := stored["is_collector"].(bool); ok {
cfg.IsCollector = v
}
if v, ok := stored["publish_count"].(bool); ok {
cfg.PublishCount = v
}
if v, ok := stored["last_ping_at"].(float64); ok {
cfg.LastPingAt = int64(v)
}
if cfg.InstanceID == "" {
cfg.InstanceID = store.NewID()
if err := s.saveBeaconSettings(ctx, cfg); err != nil {
return cfg, err
}
}
return cfg, nil
}
func (s *Server) saveBeaconSettings(ctx context.Context, cfg beaconConfig) error {
return s.db.SetInstanceSettings(ctx, beaconScope, map[string]any{
"enabled": cfg.Enabled,
"collector_url": cfg.Collector,
"instance_id": cfg.InstanceID,
"is_collector": cfg.IsCollector,
"publish_count": cfg.PublishCount,
"last_ping_at": cfg.LastPingAt,
})
}
// --- what the settings page shows ------------------------------------------------
type beaconStatusJSON struct {
Enabled bool `json:"enabled"`
Collector string `json:"collector_url"`
InstanceID string `json:"instance_id"`
Version string `json:"version"`
IsCollector bool `json:"is_collector"`
PublishCount bool `json:"publish_count"`
LastPingAt string `json:"last_ping_at,omitempty"`
Stats *store.BeaconStats `json:"stats,omitempty"`
}
func (s *Server) handleBeaconStatus(w http.ResponseWriter, r *http.Request) {
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
out := beaconStatusJSON{
Enabled: cfg.Enabled, Collector: cfg.Collector, InstanceID: cfg.InstanceID,
Version: Version, IsCollector: cfg.IsCollector, PublishCount: cfg.PublishCount,
}
if cfg.LastPingAt > 0 {
out.LastPingAt = time.Unix(cfg.LastPingAt, 0).Format(time.RFC3339)
}
if cfg.IsCollector {
stats, err := s.db.BeaconSummary(r.Context())
if err != nil {
s.internal(w, r, "beacon summary", err)
return
}
out.Stats = &stats
}
writeJSON(w, http.StatusOK, out)
}
type beaconRequest struct {
Enabled *bool `json:"enabled"`
Collector *string `json:"collector_url"`
IsCollector *bool `json:"is_collector"`
PublishCount *bool `json:"publish_count"`
}
func (s *Server) handleSetBeacon(w http.ResponseWriter, r *http.Request) {
var req beaconRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
if req.Enabled != nil {
cfg.Enabled = *req.Enabled
}
if req.IsCollector != nil {
cfg.IsCollector = *req.IsCollector
}
if req.PublishCount != nil {
cfg.PublishCount = *req.PublishCount
}
if req.Collector != nil {
url := strings.TrimSpace(*req.Collector)
if url == "" {
url = beacon.DefaultCollector
}
// http and https only, and said as a field error rather than a 500: an
// operator pointing this at their own collector will mistype it, and the
// page should say which field is wrong.
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
writeFieldErrors(w, map[string]string{"collector_url": "must start with http:// or https://"})
return
}
cfg.Collector = url
}
if err := s.saveBeaconSettings(r.Context(), cfg); err != nil {
s.internal(w, r, "save beacon settings", err)
return
}
s.handleBeaconStatus(w, r)
}
// --- the collector's side ---------------------------------------------------------
// handleBeaconPing receives one report. Unauthenticated, because the whole point
// is that a stranger's installation can reach it.
//
// Nothing about the request is recorded except the two values in the body. The
// source address is not read here, and there is no code path from this function
// to anything that could write one down — which is the only version of that
// promise worth making.
func (s *Server) handleBeaconPing(w http.ResponseWriter, r *http.Request) {
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
// An instance that is not a collector does not quietly accept and drop pings —
// it says there is nothing here. Otherwise every installation in the world is a
// spam sink that answers 200 to anything.
if !cfg.IsCollector {
writeError(w, http.StatusNotFound, CodeNotFound, "this instance is not a beacon collector")
return
}
r.Body = http.MaxBytesReader(w, r.Body, 4<<10)
var p beacon.Payload
if err := decodeJSON(w, r, &p); err != nil {
return
}
// Checked, not trusted. This is the one endpoint anybody can post to, and the
// two columns behind it should only ever hold the two shapes this program
// writes — an id that is not id-shaped is somebody probing, not an install.
if !beacon.ValidID(p.InstanceID) {
writeFieldErrors(w, map[string]string{"instance_id": "malformed"})
return
}
if p.Version != "" && !beacon.ValidVersion(p.Version) {
writeFieldErrors(w, map[string]string{"version": "malformed"})
return
}
if err := s.db.RecordBeacon(r.Context(), p.InstanceID, p.Version); err != nil {
s.internal(w, r, "record beacon", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleBeaconCount publishes the number, when the collector has been told to.
//
// Thirty days rather than a running total: a total only climbs, so it stops
// meaning "how many people use this" and starts meaning "how many ever tried it".
func (s *Server) handleBeaconCount(w http.ResponseWriter, r *http.Request) {
cfg, err := s.beaconSettings(r.Context())
if err != nil {
s.internal(w, r, "beacon settings", err)
return
}
if !cfg.IsCollector || !cfg.PublishCount {
writeError(w, http.StatusNotFound, CodeNotFound, "not published")
return
}
n, err := s.db.BeaconCount(r.Context(), 30*24*time.Hour)
if err != nil {
s.internal(w, r, "beacon count", err)
return
}
w.Header().Set("Cache-Control", "public, max-age=60")
writeJSON(w, http.StatusOK, map[string]any{"installs": n, "window_days": 30})
}
// SendBeacon reports this installation, at most once a day.
//
// Exported and handed to the jobs runner the same way the mailbox syncs are: the
// settings, the anonymous id and the version all live on this side, and the
// runner should not have to know about any of them.
//
// The day is counted from the last successful send, not from a wall-clock hour.
// A fleet of installations that all pinged at 03:00 would arrive in one spike;
// counted this way they spread themselves out by when each was started.
func (s *Server) SendBeacon(ctx context.Context) error {
cfg, err := s.beaconSettings(ctx)
if err != nil {
return err
}
if !cfg.Enabled {
return nil
}
// The collector does not report to itself over the network. It would work, and
// it would also mean the one instance whose count matters most depends on its
// own tunnel being up to be counted.
if cfg.IsCollector {
if err := s.db.RecordBeacon(ctx, cfg.InstanceID, Version); err != nil {
return err
}
cfg.LastPingAt = time.Now().Unix()
return s.saveBeaconSettings(ctx, cfg)
}
if time.Since(time.Unix(cfg.LastPingAt, 0)) < 24*time.Hour {
return nil
}
// safedial, ikke http.DefaultClient. Adressen kan skrives i fladen, og en
// administrator, der peger den på 127.0.0.1 eller på 169.254.169.254, ville
// ellers have en maskine, der henter indefra på kommando — og svaret på om den
// svarede, er i sig selv det, sådan et forsøg leder efter.
if err := beacon.Send(ctx, safedial.Client(15*time.Second), cfg.Collector, beacon.Payload{
InstanceID: cfg.InstanceID,
Version: Version,
}); err != nil {
// Logged at info, not error. The collector being unreachable is not this
// installation's problem and must never look like one — it is somebody
// else's server, and the only consequence is a number being one lower.
s.log.Info("beacon not delivered", "err", err)
return nil
}
cfg.LastPingAt = time.Now().Unix()
return s.saveBeaconSettings(ctx, cfg)
}
+27
View File
@@ -231,6 +231,33 @@ func (s *Server) handleCalDAVPut(w http.ResponseWriter, r *http.Request) {
existing, getErr := s.db.GetTask(r.Context(), taskID, user.ID)
if getErr == nil {
// The task decides, not the URL.
//
// The check above is about the project the *client named*, and a client
// names both halves of this address. Asking the project for permission and
// then taking the task id from the same string ties nothing together: a
// viewer on a shared project could name a project of their own — the Inbox,
// which everybody owns and may edit — and put the shared task's id in the
// filename. The project check passed on the Inbox and the write landed on a
// task they may only read.
//
// GetTask does not close it either: it answers "may this person see it",
// and a viewer may. `CanEdit` is the question, and it is the same one
// handleUpdateTask asks — the API door has always asked it.
role, err := store.TaskRole(r.Context(), s.db, taskID, user.ID)
if err != nil || !role.CanEdit() {
http.Error(w, "forbidden", http.StatusForbidden)
return
}
// And it has to live where the address says it does. Without this an editor
// on two projects can reach either through the other's URL — nothing is
// escalated by that on its own, but a URL allowed to lie about where a
// resource lives is how the case above got in.
if existing.ProjectID != projectID {
http.NotFound(w, r)
return
}
// An update. Only the fields a task client can express are touched;
// everything else the task carries stays as it is.
content := parsed.Summary
+808
View File
@@ -0,0 +1,808 @@
package httpapi
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/kristianwind/verdande/internal/gcal"
"github.com/kristianwind/verdande/internal/ics"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
// defaultCalendarSyncBudget bounds one "fetch now" when nothing else says
// otherwise. Well under the hundred seconds a proxy is willing to wait, so this
// server is the one that answers a slow Google rather than Cloudflare's own HTML
// page — which is how a failure arrives with no code, no message and nothing in
// the error log.
const defaultCalendarSyncBudget = 25 * time.Second
// The window verdande keeps a copy of.
//
// A month grid pages, so asking Google on every arrow press would mean a round trip
// for a question answered ninety seconds ago — a calendar slower than the calendar
// it is showing. A cached window is the alternative, and a window has to end
// somewhere. Ninety days back is roughly "when was that", a year forward is past
// anything anybody has booked; between them they cover every screen the grid can
// reach without a person noticing an edge.
//
// The edge is still real, and the events endpoint says where it is rather than
// answering an empty list for a month outside it. A calendar that quietly shows no
// events is a calendar that lies about a day being clear.
const (
calendarWindowBack = 90
calendarWindowAhead = 365
calendarDateLayout = "2006-01-02"
calendarProviderName = "google"
)
func (s *Server) calendarClient(accessToken string) *gcal.Client {
return gcal.NewClient(accessToken).At(s.cfg.CalendarAPIURL)
}
// calendarConfig is the same registration Gmail signs in through, asking for a
// different scope and coming back to a different URI.
//
// One OAuth client, two features. Google issues a refresh token per authorisation
// and keeps up to a hundred of them live per account, each carrying the scopes it
// was granted with — so connecting the calendar does not disturb a working Gmail,
// and disconnecting one does not take the other with it.
func (s *Server) calendarConfig(ctx context.Context) gcal.Config {
cfg := s.gmailConfig(ctx)
cfg.Scope = gcal.Scope
cfg.RedirectURL = s.cfg.CalendarRedirectURL()
return cfg
}
// --- connecting ------------------------------------------------------------------
// handleCalendarAuthorize starts the flow. Same shape as Gmail's, and the same
// reason for keeping the verifier server-side: the callback arrives as a top-level
// navigation from Google, and a SameSite=Lax cookie is the kind of thing that works
// until a browser tightens its defaults.
func (s *Server) handleCalendarAuthorize(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
cfg := s.calendarConfig(r.Context())
if !cfg.Configured() {
writeError(w, http.StatusConflict, CodeGmailNotConfigured,
"no Google OAuth client is registered on this server")
return
}
pkce, err := gcal.NewPKCE()
if err != nil {
s.internal(w, r, "calendar pkce", err)
return
}
// Its own settings scope, not Gmail's. Two handshakes in one row would mean a
// person who started both flows had one verifier, and whichever came back
// second would be refused for a reason nothing on the screen could explain.
settings, err := s.db.UserSettings(r.Context(), user.ID, "calendar")
if err != nil {
s.internal(w, r, "calendar settings", err)
return
}
settings["pkce_verifier"] = pkce.Verifier
settings["pkce_state"] = pkce.State
// The attempt expires: a stored verifier that lives forever is a replay window
// that lives forever.
settings["pkce_expires"] = time.Now().Add(10 * time.Minute).Unix()
if err := s.db.SetUserSettings(r.Context(), user.ID, "calendar", settings); err != nil {
s.internal(w, r, "calendar settings", err)
return
}
writeJSON(w, http.StatusOK, map[string]string{"url": cfg.AuthURL(pkce)})
}
// handleCalendarCallback is where Google sends the browser back.
//
// It answers with a redirect into the app rather than with JSON: the person is
// looking at a browser tab, not at a fetch response.
func (s *Server) handleCalendarCallback(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
settingsURL := s.cfg.BaseURL + "/indstillinger/integrationer"
if errParam := r.URL.Query().Get("error"); errParam != "" {
// Somebody clicked "cancel" on the consent screen — or, on an Internal
// OAuth app, Google refused an account outside the organisation with
// `org_internal`. Both arrive here, and the page names what it was given.
http.Redirect(w, r, settingsURL+"?calendar="+errParam, http.StatusFound)
return
}
code := r.URL.Query().Get("code")
state := r.URL.Query().Get("state")
if code == "" || state == "" {
http.Redirect(w, r, settingsURL+"?calendar=invalid", http.StatusFound)
return
}
settings, err := s.db.UserSettings(r.Context(), user.ID, "calendar")
if err != nil {
s.internal(w, r, "calendar settings", err)
return
}
verifier, _ := settings["pkce_verifier"].(string)
expected, _ := settings["pkce_state"].(string)
expires, _ := settings["pkce_expires"].(float64)
// The state has to match, or this endpoint accepts a code from anywhere.
if verifier == "" || expected == "" || state != expected {
http.Redirect(w, r, settingsURL+"?calendar=state", http.StatusFound)
return
}
if int64(expires) < time.Now().Unix() {
http.Redirect(w, r, settingsURL+"?calendar=expired", http.StatusFound)
return
}
token, err := s.calendarConfig(r.Context()).Exchange(r.Context(), code,
gcal.PKCE{Verifier: verifier})
if err != nil {
s.log.Error("calendar exchange", "err", err, "user", user.ID)
http.Redirect(w, r, settingsURL+"?calendar=failed", http.StatusFound)
return
}
if token.RefreshToken == "" {
// Without one the connection dies in an hour. Google only withholds it when
// the consent prompt was skipped, which access_type and prompt are meant to
// prevent — so this means something is wrong with the client registration
// rather than with this request.
s.log.Error("calendar returned no refresh token", "user", user.ID)
http.Redirect(w, r, settingsURL+"?calendar=norefresh", http.StatusFound)
return
}
// The one-time values are cleared: keeping a spent verifier serves nothing and
// is one more secret at rest.
delete(settings, "pkce_verifier")
delete(settings, "pkce_state")
delete(settings, "pkce_expires")
if err := s.db.SetUserSettings(r.Context(), user.ID, "calendar", settings); err != nil {
s.internal(w, r, "clear calendar handshake", err)
return
}
account := &store.CalendarAccount{
UserID: user.ID, Provider: calendarProviderName,
RefreshToken: token.RefreshToken, AccessToken: token.AccessToken,
ExpiresAt: token.ExpiresAt,
}
if err := s.db.SaveCalendarAccount(r.Context(), account); err != nil {
s.internal(w, r, "save calendar tokens", err)
return
}
// The list of calendars, fetched now so the page that opens next has something
// to tick rather than an empty panel and a button. Best effort: an account that
// connects but cannot list itself is still connected, and the status endpoint
// asks again when it finds nothing.
if _, err := s.refreshCalendarList(r.Context(), account); err != nil {
s.log.Warn("calendar list", "err", err, "user", user.ID)
}
s.log.Info("calendar connected", "user", user.ID, "account", account.Account)
http.Redirect(w, r, settingsURL+"?calendar=connected", http.StatusFound)
}
// --- what the settings page reads --------------------------------------------------
type calendarStatusJSON struct {
Connected bool `json:"connected"`
Account string `json:"account,omitempty"`
Calendars []store.Calendar `json:"calendars"`
// RedirectURI is what has to be registered with Google, spelled out. Derived
// from VERDANDE_BASE_URL and matched exactly — the single most likely thing to
// be wrong, and the error Google gives for it names neither value.
RedirectURI string `json:"redirect_uri"`
// HasClient says whether there is an OAuth registration to connect through at
// all, so the page can explain rather than offer a button that answers 409.
HasClient bool `json:"has_client"`
LastSyncAt string `json:"last_sync_at,omitempty"`
// ReadOnly is stated rather than assumed. Nothing writes a Google calendar, and
// an interface that quietly looks editable is one somebody will try to edit.
ReadOnly bool `json:"read_only"`
// Abonnementer står ved siden af Google-kontoen og ikke inde i den: man kan
// have det ene uden det andet, og det er hele pointen med dem — en privat
// konto, der ikke kan bruge en Internal OAuth-klient, kan abonnere.
Subscriptions []store.CalendarAccount `json:"subscriptions"`
}
func (s *Server) handleGetCalendar(w http.ResponseWriter, r *http.Request) {
// Læst først, så de også kommer med, når der ingen Google-konto er.
subscriptions, err := s.db.Subscriptions(r.Context(), userFrom(r.Context()).ID)
if err != nil {
s.internal(w, r, "subscriptions", err)
return
}
user := userFrom(r.Context())
out := calendarStatusJSON{
Calendars: []store.Calendar{},
RedirectURI: s.cfg.CalendarRedirectURL(),
HasClient: s.calendarConfig(r.Context()).Configured(),
ReadOnly: true,
}
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
if err != nil {
s.internal(w, r, "calendar account", err)
return
}
if account == nil || account.RefreshToken == "" {
out.Subscriptions = subscriptions
writeJSON(w, http.StatusOK, out)
return
}
out.Connected = true
out.Account = account.Account
if !account.LastSyncAt.IsZero() {
out.LastSyncAt = account.LastSyncAt.Format(time.RFC3339)
}
calendars, err := s.db.Calendars(r.Context(), account.ID)
if err != nil {
s.internal(w, r, "calendars", err)
return
}
// An account connected during a Google hiccup has no calendars and no way back:
// the person sees an empty list and nothing to press. Asking again here costs
// one request on a page nobody opens often, and means the wrong answer lasts
// until the next look rather than for ever. Same reasoning as the Gmail address.
if len(calendars) == 0 {
if fetched, err := s.refreshCalendarList(r.Context(), account); err == nil {
calendars = fetched
out.Account = account.Account
} else {
s.log.Warn("calendar list", "err", err, "user", user.ID)
}
}
if calendars != nil {
out.Calendars = calendars
}
out.Subscriptions = subscriptions
writeJSON(w, http.StatusOK, out)
}
type showCalendarsRequest struct {
// Shown is the whole set, not one flip. Same reasoning as the projects' order:
// a person has a handful of calendars, the question is "these ones", and a list
// that can only be built by several requests can land half applied.
Shown []string `json:"shown"`
}
func (s *Server) handleSetCalendars(w http.ResponseWriter, r *http.Request) {
var req showCalendarsRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
user := userFrom(r.Context())
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
if err != nil {
s.internal(w, r, "calendar account", err)
return
}
if account == nil {
// A 404 rather than a 409: everything the caller may not see answers 404,
// and "there is no such connection" is the true answer either way.
writeError(w, http.StatusNotFound, CodeNotFound, "no calendar is connected")
return
}
if err := s.db.ShowCalendars(r.Context(), account.ID, req.Shown); err != nil {
s.internal(w, r, "show calendars", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
// handleDisconnectCalendar forgets the connection, and the copy of the calendars
// with it. Unlike the tasks a mailbox made, none of this is the person's own work.
func (s *Server) handleDisconnectCalendar(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
account, err := s.db.CalendarAccountFor(r.Context(), user.ID, calendarProviderName)
if err != nil {
s.internal(w, r, "calendar account", err)
return
}
if account != nil {
if err := s.db.DeleteCalendarAccount(r.Context(), user.ID, account.ID); err != nil {
s.internal(w, r, "disconnect calendar", err)
return
}
}
w.WriteHeader(http.StatusNoContent)
}
// --- what the grid reads -----------------------------------------------------------
type calendarEventsJSON struct {
Events []store.CalendarEvent `json:"events"`
// From and To are the window verdande holds a copy of, not the window that was
// asked for. A grid that has paged past it gets no events, and an empty answer
// with no explanation is a calendar quietly claiming a day is clear.
From string `json:"from"`
To string `json:"to"`
}
func (s *Server) handleCalendarEvents(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
from := r.URL.Query().Get("from")
to := r.URL.Query().Get("to")
fields := map[string]string{}
if !validDay(from) {
fields["from"] = "must be a date, as 2026-08-20"
}
if !validDay(to) {
fields["to"] = "must be a date, as 2026-08-20"
}
if len(fields) > 0 {
writeFieldErrors(w, fields)
return
}
events, err := s.db.CalendarEvents(r.Context(), user.ID, from, to)
if err != nil {
s.internal(w, r, "calendar events", err)
return
}
if events == nil {
events = []store.CalendarEvent{}
}
windowFrom, windowTo := calendarWindow(time.Now())
writeJSON(w, http.StatusOK, calendarEventsJSON{
Events: events, From: windowFrom, To: windowTo,
})
}
// handleCalendarSyncNow refreshes immediately, so somebody who has just connected
// can see it work rather than wondering whether it did.
func (s *Server) handleCalendarSyncNow(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
// A budget of its own, well under what sits in front of this server. An unset
// budget is not a budget of nothing: a zero here would expire the context
// before the first call and make every sync a silent no-op.
budget := s.cfg.CalendarSyncBudget
if budget <= 0 {
budget = defaultCalendarSyncBudget
}
ctx, cancel := context.WithTimeout(r.Context(), budget)
defer cancel()
count, err := s.SyncCalendars(ctx, user)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) {
// Not a failure: Google is slow or busy, some calendars were read, and
// the rest arrive on the next run. Every calendar is committed as it is
// read, so a short budget is a shorter run rather than a lost one.
writeJSON(w, http.StatusOK, map[string]any{"events": count, "partial": true})
return
}
s.upstream(w, r, CodeCalendarFailed, "sync calendar", err)
return
}
writeJSON(w, http.StatusOK, map[string]int{"events": count})
}
// SyncCalendars refreshes every calendar this person has chosen to see.
//
// Returns how many events were written. Also used by the background job, which is
// why it takes a context and a user rather than a request.
func (s *Server) SyncCalendars(ctx context.Context, user *store.User) (int, error) {
// Abonnementer først, og deres fejl tæller ikke mod Google.
//
// De to har intet med hinanden at gøre ud over at ende samme sted: et
// abonnement er en adresse, der hentes, hvor Google er et token, der udløber.
// En kalender, hvis vært er nede, må ikke få det til at se ud som om
// Google-forbindelsen er væk — og omvendt.
written, err := s.syncSubscriptions(ctx, user)
if err != nil {
s.log.Warn("calendar subscriptions", "err", err, "user", user.ID)
}
account, err := s.db.CalendarAccountFor(ctx, user.ID, calendarProviderName)
if err != nil {
return written, err
}
if account == nil || account.RefreshToken == "" {
return written, nil // not connected; nothing to do and nothing to complain about
}
// The same guard the mailboxes have, for the same reason: a press of "fetch
// now" during the sweep would otherwise have two runs replacing the same window
// at once, and the loser's delete would land after the winner's insert.
unlock := s.lockSync("calendar:" + account.ID)
defer unlock()
if fresh, err := s.db.CalendarAccountFor(ctx, user.ID, calendarProviderName); err == nil && fresh != nil {
account = fresh
}
access, err := s.calendarAccessToken(ctx, account)
if err != nil {
return 0, err
}
client := s.calendarClient(access)
// The list first, so a calendar added or renamed since the last run is drawn
// under its new name and one that has gone stops being drawn at all.
calendars, err := s.calendarList(ctx, client, account)
if err != nil {
return 0, s.forgetIfRevoked(ctx, user, account, err)
}
from, to := calendarWindow(time.Now())
for _, calendar := range calendars {
if !calendar.Shown {
continue
}
// A spent budget ends the run rather than turning into a failed fetch per
// calendar, logged one at a time.
if ctx.Err() != nil {
return written, ctx.Err()
}
events, err := client.Events(ctx, calendar.RemoteID, from, to)
if err != nil {
if errors.Is(err, gcal.ErrUnauthorized) {
return written, s.forgetIfRevoked(ctx, user, account, err)
}
// One calendar somebody was unshared from must not stop the others.
s.log.Warn("calendar events", "err", err, "calendar", calendar.Name)
continue
}
rows := make([]store.CalendarEvent, 0, len(events))
for _, e := range events {
rows = append(rows, store.CalendarEvent{
RemoteID: e.ID, Summary: e.Summary,
StartsAt: e.StartsAt, EndsAt: e.EndsAt,
StartDay: e.StartDay, EndDay: e.EndDay,
AllDay: e.AllDay, Location: e.Location, URL: e.URL,
})
}
if err := s.db.ReplaceCalendarEvents(ctx, calendar.ID, from, to, rows); err != nil {
return written, err
}
written += len(rows)
}
if err := s.db.MarkCalendarSynced(ctx, account.ID, time.Now()); err != nil {
return written, err
}
return written, nil
}
// calendarAccessToken returns a usable access token, refreshing only when the one
// held has actually expired.
func (s *Server) calendarAccessToken(ctx context.Context, account *store.CalendarAccount) (string, error) {
if account.AccessToken != "" && account.ExpiresAt.After(time.Now()) {
return account.AccessToken, nil
}
token, err := s.calendarConfig(ctx).Refresh(ctx, account.RefreshToken)
if err != nil {
return "", err
}
account.AccessToken = token.AccessToken
account.ExpiresAt = token.ExpiresAt
if err := s.db.SaveCalendarAccount(ctx, account); err != nil {
return "", err
}
return account.AccessToken, nil
}
// refreshCalendarList fetches the calendar list with a token it obtains itself.
// Used by the paths that have an account but no client yet — connecting, and the
// status endpoint correcting an empty list.
func (s *Server) refreshCalendarList(ctx context.Context, account *store.CalendarAccount) ([]store.Calendar, error) {
access, err := s.calendarAccessToken(ctx, account)
if err != nil {
return nil, err
}
return s.calendarList(ctx, s.calendarClient(access), account)
}
// calendarList writes what the provider says and returns what is stored, which is
// not the same list: the provider says what exists, and only verdande knows which
// of them somebody chose to look at.
func (s *Server) calendarList(ctx context.Context, client *gcal.Client, account *store.CalendarAccount) ([]store.Calendar, error) {
remote, err := client.Calendars(ctx)
if err != nil {
return nil, err
}
list := make([]store.Calendar, 0, len(remote))
for _, c := range remote {
list = append(list, store.Calendar{
RemoteID: c.ID, Name: c.Name, Colour: c.Colour,
TimeZone: c.TimeZone, Primary: c.Primary, Writable: c.Owner,
})
// The primary calendar is the account, and its id is the address. It is the
// only place the connected account names itself without a second scope —
// asking for a profile scope to learn an address already in the answer would
// be one more thing on the consent screen for nothing.
if c.Primary && account.Account == "" {
account.Account = c.ID
if err := s.db.SaveCalendarAccount(ctx, account); err != nil {
return nil, err
}
}
}
if err := s.db.ReplaceCalendars(ctx, account.ID, list); err != nil {
return nil, err
}
return s.db.Calendars(ctx, account.ID)
}
// forgetIfRevoked disconnects when Google says the grant is gone, and passes every
// other failure through untouched.
//
// The distinction is the point. A revoked account polled every ten minutes for ever
// is a log line a week long and a connection the person believes they removed; a
// Google that is merely down is one that will answer next time.
func (s *Server) forgetIfRevoked(ctx context.Context, user *store.User, account *store.CalendarAccount, err error) error {
if errors.Is(err, gcal.ErrUnauthorized) {
s.log.Info("calendar access was revoked; disconnecting", "user", user.ID)
_ = s.db.DeleteCalendarAccount(ctx, user.ID, account.ID)
}
return err
}
// calendarWindow is the span verdande keeps a copy of, measured from today.
//
// Built from the calendar date rather than by truncating the instant: Truncate
// rounds against the Unix epoch, which is midnight UTC, and in Copenhagen that is
// two in the morning — so for two hours of every summer day the window would start
// and end a day out.
func calendarWindow(now time.Time) (string, string) {
day := time.Date(now.Year(), now.Month(), now.Day(), 0, 0, 0, 0, now.Location())
return day.AddDate(0, 0, -calendarWindowBack).Format(calendarDateLayout),
day.AddDate(0, 0, calendarWindowAhead).Format(calendarDateLayout)
}
func validDay(s string) bool {
_, err := time.Parse(calendarDateLayout, s)
return err == nil
}
// --- abonnementer -----------------------------------------------------------------
// maxICSBytes bounds one fetch. A published calendar of a decade of holidays is a
// few hundred kilobytes; ten megabytes is not a calendar, and reading it into
// memory on a schedule with nobody watching is how a homelab instance falls over
// at three in the morning.
const maxICSBytes = 10 << 20
// normaliseCalendarURL turns what somebody pastes into something that can be
// fetched.
//
// `webcal://` is not a scheme anything speaks. It is a convention — the same URL
// over https, with a scheme that makes a desktop hand it to the calendar app
// instead of the browser — and every publisher writes their links that way. A
// person pasting one has pasted the right address; refusing it would be refusing
// them on a technicality.
func normaliseCalendarURL(raw string) (string, error) {
raw = strings.TrimSpace(raw)
if after, ok := strings.CutPrefix(raw, "webcal://"); ok {
raw = "https://" + after
}
if !strings.HasPrefix(raw, "https://") && !strings.HasPrefix(raw, "http://") {
return "", errors.New("must be an https:// or webcal:// address")
}
// The same wall the AI provider's base_url goes through, and for the same
// reason: this is an address a user typed, and the server is about to fetch it
// on a schedule of their choosing.
if reason := safedial.CheckURL(raw); reason != "" {
return "", errors.New(reason)
}
return raw, nil
}
// icsClient is the client a subscribed calendar is fetched with.
//
// safedial, checked on the *resolved* address at connect time and on every
// redirect hop — because a name that passed the check when it was pasted can
// answer 127.0.0.1 when the sweep runs at three in the morning.
//
// The field exists so a test can put an ordinary client here: httptest listens on
// loopback, which is exactly what this refuses, so a test of the *format* cannot
// otherwise reach its own server. It is never set outside a test, and the wall it
// bypasses is tested where it belongs — `normaliseCalendarURL` refuses a private
// address where it is written down, and `internal/push` proves the connect-time
// half.
func (s *Server) icsClient() *http.Client {
if s.icsFetch != nil {
return s.icsFetch
}
return safedial.Client(30 * time.Second)
}
// fetchICS reads a subscribed calendar.
//
// safedial, not a plain client — checked on the resolved address at connect time
// and on every redirect hop, because a name that passed the check when it was
// pasted can answer 127.0.0.1 when the sweep runs at three in the morning.
func (s *Server) fetchICS(ctx context.Context, url string) (ics.Subscribed, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return ics.Subscribed{}, err
}
req.Header.Set("Accept", "text/calendar, text/plain;q=0.9, */*;q=0.5")
req.Header.Set("User-Agent", "verdande/"+Version)
resp, err := s.icsClient().Do(req)
if err != nil {
return ics.Subscribed{}, err
}
defer resp.Body.Close()
if resp.StatusCode >= 300 {
return ics.Subscribed{}, fmt.Errorf("%s said %s", resp.Request.URL.Host, resp.Status)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, maxICSBytes+1))
if err != nil {
return ics.Subscribed{}, err
}
if len(body) > maxICSBytes {
return ics.Subscribed{}, errors.New("that calendar is too large")
}
return ics.ParseCalendar(string(body)), nil
}
// syncSubscriptions refreshes every calendar this person subscribes to by address.
//
// One failing address does not stop the others. A subscription is somebody else's
// server, and the ordinary state of somebody else's server is that it is fine
// until one morning it is not — a holiday feed that 404s must not take the
// football club's fixtures down with it.
func (s *Server) syncSubscriptions(ctx context.Context, user *store.User) (int, error) {
subs, err := s.db.Subscriptions(ctx, user.ID)
if err != nil || len(subs) == 0 {
return 0, err
}
from, to := calendarWindow(time.Now())
written := 0
var failures []string
for _, sub := range subs {
if ctx.Err() != nil {
return written, ctx.Err()
}
unlock := s.lockSync("calendar:" + sub.ID)
calendarID, err := s.db.SubscribedCalendarID(ctx, sub.ID)
if err != nil {
unlock()
failures = append(failures, sub.URL)
continue
}
cal, err := s.fetchICS(ctx, sub.URL)
if err != nil {
unlock()
s.log.Warn("subscribed calendar", "err", err, "url", sub.URL, "user", user.ID)
failures = append(failures, sub.URL)
continue
}
// The name the file calls itself, once. A subscription is added under its
// address because that is all there is to go on before it has been fetched.
if err := s.db.RenameSubscribedCalendar(ctx, sub.ID, cal.Name); err != nil {
s.log.Warn("name subscribed calendar", "err", err)
}
rows := make([]store.CalendarEvent, 0, len(cal.Events))
for _, e := range cal.Events {
// Outside the window is not stored. The grid asks about a window, the
// sweep replaces a window, and a decade of holidays in one file would
// otherwise be a decade of rows rewritten every fifteen minutes.
if e.EndDay < from || e.StartDay > to {
continue
}
rows = append(rows, store.CalendarEvent{
RemoteID: e.UID, Summary: e.Summary, Location: e.Location, URL: e.URL,
StartsAt: e.StartsAt, EndsAt: e.EndsAt,
StartDay: e.StartDay, EndDay: e.EndDay, AllDay: e.AllDay,
})
}
if err := s.db.ReplaceCalendarEvents(ctx, calendarID, from, to, rows); err != nil {
unlock()
return written, err
}
if err := s.db.MarkCalendarSynced(ctx, sub.ID, time.Now()); err != nil {
s.log.Warn("mark subscription synced", "err", err)
}
written += len(rows)
unlock()
}
if len(failures) == len(subs) {
return written, fmt.Errorf("none of the %d subscribed calendars could be read", len(subs))
}
return written, nil
}
// handleSubscribeCalendar adds a calendar somebody only has an address to.
//
// Fetched once, here, before it is stored. An address that cannot be read is a
// typo far more often than it is a server that happens to be down, and telling
// somebody now — while they are looking at the field they just pasted into — is
// worth the two seconds. A subscription that silently never fills is the same
// silence the notifications had.
func (s *Server) handleSubscribeCalendar(w http.ResponseWriter, r *http.Request) {
var req struct {
URL string `json:"url"`
Name string `json:"name"`
}
if err := decodeJSON(w, r, &req); err != nil {
return
}
url, err := normaliseCalendarURL(req.URL)
if err != nil {
writeFieldErrors(w, map[string]string{"url": err.Error()})
return
}
user := userFrom(r.Context())
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
defer cancel()
cal, err := s.fetchICS(ctx, url)
if err != nil {
s.log.Warn("subscribe calendar", "err", err, "user", user.ID)
writeFieldErrors(w, map[string]string{"url": "that address could not be read"})
return
}
name := strings.TrimSpace(req.Name)
if name == "" {
name = strings.TrimSpace(cal.Name)
}
if name == "" {
name = url
}
account, err := s.db.SubscribeCalendar(r.Context(), user.ID, url, name)
if err != nil {
// The unique index is the one refusal worth naming: subscribing twice to
// the same address is the same calendar fetched twice, with no way to say
// which one counts.
if errors.Is(err, store.ErrDuplicate) {
writeFieldErrors(w, map[string]string{"url": "that calendar is already subscribed"})
return
}
s.storeError(w, r, "subscribe calendar", err)
return
}
// The events it already fetched, rather than making the person wait fifteen
// minutes for the sweep to prove it worked.
if _, err := s.syncSubscriptions(r.Context(), user); err != nil {
s.log.Warn("first subscription sync", "err", err)
}
writeJSON(w, http.StatusCreated, account)
}
func (s *Server) handleUnsubscribeCalendar(w http.ResponseWriter, r *http.Request) {
user := userFrom(r.Context())
if err := s.db.DeleteCalendarAccount(r.Context(), user.ID, chi.URLParam(r, "id")); err != nil {
s.storeError(w, r, "unsubscribe calendar", err)
return
}
w.WriteHeader(http.StatusNoContent)
}
+471
View File
@@ -0,0 +1,471 @@
package httpapi
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/kristianwind/verdande/internal/config"
"github.com/kristianwind/verdande/internal/store"
)
// fakeCalendarGoogle stands in for the parts of Google this feature touches: the
// token endpoint, the calendar list and one calendar's events.
//
// There is no other way to exercise any of it. The real thing needs a registered
// OAuth client, an account and somebody's consent, and none of the three can be had
// inside a test — so the shapes Google returns are written down and the behaviour
// is checked against them, the way internal/imap is checked against go-imap's own
// in-process server.
type fakeCalendarGoogle struct {
*httptest.Server
// refreshes counts token refreshes, so a test can prove one is not made for a
// token that has not expired.
refreshes int
// refuse makes every API call answer 401, which is what a revoked grant does.
refuse bool
events map[string][]map[string]any
}
func newFakeCalendarGoogle(t *testing.T) *fakeCalendarGoogle {
t.Helper()
g := &fakeCalendarGoogle{events: map[string][]map[string]any{}}
g.Server = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
switch {
case r.URL.Path == "/token":
g.refreshes++
_ = json.NewEncoder(w).Encode(map[string]any{
"access_token": "ya29.fresh", "refresh_token": "1//0-refresh",
"expires_in": 3600,
})
case g.refuse:
http.Error(w, `{"error":"invalid_credentials"}`, http.StatusUnauthorized)
case strings.HasPrefix(r.URL.Path, "/users/me/calendarList"):
_ = json.NewEncoder(w).Encode(map[string]any{"items": []map[string]any{
{"id": "kw@nolimit.dk", "summary": "kw@nolimit.dk", "primary": true,
"accessRole": "owner", "backgroundColor": "#3f51b5",
"timeZone": "Europe/Copenhagen"},
{"id": "fam", "summary": "Familien", "accessRole": "reader",
"backgroundColor": "#0b8043"},
}})
case strings.HasPrefix(r.URL.Path, "/calendars/"):
id := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/calendars/"), "/events")
items := g.events[id]
if items == nil {
items = []map[string]any{}
}
_ = json.NewEncoder(w).Encode(map[string]any{"items": items})
default:
http.Error(w, "unexpected "+r.URL.Path, http.StatusNotFound)
}
}))
t.Cleanup(g.Close)
return g
}
// connectedCalendar returns a server with a calendar account already connected —
// the state the OAuth callback leaves behind, without the consent screen nothing
// can automate.
func connectedCalendar(t *testing.T, g *fakeCalendarGoogle, tune func(*config.Config)) (*testServer, *store.User) {
t.Helper()
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GoogleTokenURL = g.URL + "/token"
cfg.CalendarAPIURL = g.URL
if tune != nil {
tune(cfg)
}
})
ts.bootstrap(t)
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
if err != nil {
t.Fatal(err)
}
account := &store.CalendarAccount{
UserID: user.ID, Provider: "google", RefreshToken: "1//0-refresh",
}
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
return ts, user
}
// The scope on the consent screen is what somebody agrees to. It has to be the
// read-only one, and the redirect has to be the calendar's own — sharing Gmail's
// would put "which token store do I write" behind a lookup.
func TestAuthorizeAsksForReadOnlyCalendar(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, _ := connectedCalendar(t, g, nil)
resp, body := ts.do(t, "POST", "/api/v1/calendar/authorize", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
url, _ := body["url"].(string)
for _, want := range []string{
"scope=" + "https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar.readonly",
"%2Foauth%2Fcalendar%2Fcallback",
"access_type=offline",
"prompt=consent",
} {
if !strings.Contains(url, want) {
t.Errorf("the authorisation URL is missing %q:\n%s", want, url)
}
}
if strings.Contains(url, "calendar.events&") || strings.Contains(url, "auth%2Fcalendar&") {
t.Errorf("a writing scope was asked for:\n%s", url)
}
}
// Without a registered client there is nothing to connect through, and the answer
// has to say that rather than "that already exists".
func TestAuthorizeWithoutAClientSaysSo(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, body := ts.do(t, "POST", "/api/v1/calendar/authorize", nil)
if resp.StatusCode != http.StatusConflict {
t.Fatalf("status %d, want 409", resp.StatusCode)
}
if body["code"] != CodeGmailNotConfigured {
t.Errorf("code = %v, want %q", body["code"], CodeGmailNotConfigured)
}
}
// The whole path, minus the consent screen: sync fetches the list, refreshes the
// events of what was chosen, and the grid endpoint answers with them.
func TestSyncFetchesTheChosenCalendarsAndTheGridReadsThem(t *testing.T) {
g := newFakeCalendarGoogle(t)
g.events["fam"] = []map[string]any{
{"id": "e1", "summary": "Bestyrelsesmøde", "htmlLink": "https://calendar.google.com/e1",
"start": map[string]string{"dateTime": "2026-08-20T14:00:00+02:00"},
"end": map[string]string{"dateTime": "2026-08-20T15:30:00+02:00"}},
}
g.events["kw@nolimit.dk"] = []map[string]any{
{"id": "e2", "summary": "Skal ikke med",
"start": map[string]string{"date": "2026-08-21"},
"end": map[string]string{"date": "2026-08-22"}},
}
ts, user := connectedCalendar(t, g, nil)
// The list arrives on the first look at the settings page, which is what
// somebody would be doing at this point.
resp, body := ts.do(t, "GET", "/api/v1/calendar", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if body["connected"] != true {
t.Fatalf("not reported as connected: %v", body)
}
if body["read_only"] != true {
t.Error("the connection did not say it is read-only; an interface that looks " +
"editable is one somebody will try to edit")
}
// The primary calendar's id is the address, which is how the account names
// itself without asking for a profile scope as well.
if body["account"] != "kw@nolimit.dk" {
t.Errorf("account = %v", body["account"])
}
list, _ := body["calendars"].([]any)
if len(list) != 2 {
t.Fatalf("got %d calendars, want 2: %v", len(list), body["calendars"])
}
var famID string
for _, raw := range list {
c := raw.(map[string]any)
if c["shown"] == true {
t.Errorf("%v was shown without anybody choosing it", c["name"])
}
if c["remote_id"] == "fam" {
famID = c["id"].(string)
}
}
// Nothing is chosen, so a sync writes nothing: an account with a dozen shared
// calendars must not fill somebody's grid the moment it is connected.
resp, body = ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp.StatusCode != http.StatusOK || body["events"] != float64(0) {
t.Fatalf("an unchosen calendar was fetched: %d %v", resp.StatusCode, body)
}
resp, _ = ts.do(t, "PUT", "/api/v1/calendar/calendars",
map[string]any{"shown": []string{famID}})
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("choosing a calendar: status %d", resp.StatusCode)
}
resp, body = ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
}
if body["events"] != float64(1) {
t.Fatalf("events = %v, want 1 — only the chosen calendar", body["events"])
}
resp, body = ts.do(t, "GET",
"/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("events: status %d, body %v", resp.StatusCode, body)
}
events, _ := body["events"].([]any)
if len(events) != 1 {
t.Fatalf("got %d events, want 1: %v", len(events), body["events"])
}
event := events[0].(map[string]any)
if event["summary"] != "Bestyrelsesmøde" || event["start_day"] != "2026-08-20" {
t.Errorf("the event came back as %v", event)
}
// The calendar's colour rides along, or the chip has to ask for it per row.
if event["colour"] != "#0b8043" {
t.Errorf("colour = %v; two calendars in one grid are told apart by it", event["colour"])
}
// The window verdande holds is stated, so a grid paged past it can say so
// rather than showing an empty month it cannot explain.
if body["from"] == "" || body["to"] == "" {
t.Error("the answer does not say which window it covers")
}
// And it is one person's calendar. Somebody else on the same instance sees
// nothing of it.
other := ts.newUser(t, "andreas@example.dk", "Andreas")
_, body = other.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if events, _ := body["events"].([]any); len(events) != 0 {
t.Errorf("somebody else's calendar was visible: %v", body["events"])
}
_ = user
}
// A token that has not expired must not be traded in for another. Refreshing on
// every run burns the quota and makes the failure mode of a bad refresh token
// arrive on every sweep rather than once an hour.
func TestAFreshTokenIsNotRefreshed(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, user := connectedCalendar(t, g, nil)
account, err := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
if err != nil {
t.Fatal(err)
}
account.AccessToken = "ya29.still-good"
account.ExpiresAt = time.Now().Add(time.Hour)
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
if resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil); resp.StatusCode != http.StatusOK {
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
}
if g.refreshes != 0 {
t.Errorf("the token was refreshed %d times while it was still good", g.refreshes)
}
// And an expired one is.
account.ExpiresAt = time.Now().Add(-time.Minute)
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
if resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil); resp.StatusCode != http.StatusOK {
t.Fatalf("sync: status %d, body %v", resp.StatusCode, body)
}
if g.refreshes != 1 {
t.Errorf("an expired token was refreshed %d times, want 1", g.refreshes)
}
}
// Access revoked at Google's end is a connection that has ended, not a failure to
// retry. Polling it every fifteen minutes for ever is a week of log lines and a
// connection the person believes they removed.
func TestRevokedAccessDisconnects(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, user := connectedCalendar(t, g, nil)
account, _ := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
account.AccessToken = "ya29.revoked"
account.ExpiresAt = time.Now().Add(time.Hour)
if err := ts.db.SaveCalendarAccount(t.Context(), account); err != nil {
t.Fatal(err)
}
g.refuse = true
resp, body := ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp.StatusCode != StatusUpstreamRefused {
t.Fatalf("status %d, want %d — Google saying no is not this server breaking",
resp.StatusCode, StatusUpstreamRefused)
}
if body["code"] != CodeCalendarFailed {
t.Errorf("code = %v, want %q — `internal_error` is shown as a generic sentence, "+
"which throws the diagnosis away", body["code"], CodeCalendarFailed)
}
gone, err := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
if err != nil {
t.Fatal(err)
}
if gone != nil {
t.Error("the connection is still there after Google refused the token")
}
// And the failure is in the error log, which is the point of having one: a
// sync that has been failing for a week is invisible otherwise.
_, logged := ts.do(t, "GET", "/api/v1/errors", nil)
rows, _ := logged["errors"].([]any)
if len(rows) == 0 {
t.Fatal("nothing was recorded")
}
if row := rows[0].(map[string]any); row["what"] != "sync calendar" {
t.Errorf("what = %v, want \"sync calendar\"", row["what"])
}
}
// Disconnecting takes the copy with it. None of it is the person's own work, and
// a calendar left behind after the account is gone is a frozen lie about a day.
func TestDisconnectingRemovesTheEvents(t *testing.T) {
g := newFakeCalendarGoogle(t)
g.events["fam"] = []map[string]any{
{"id": "e1", "summary": "Møde",
"start": map[string]string{"date": "2026-08-20"},
"end": map[string]string{"date": "2026-08-21"}},
}
ts, user := connectedCalendar(t, g, nil)
ts.do(t, "GET", "/api/v1/calendar", nil)
account, _ := ts.db.CalendarAccountFor(t.Context(), user.ID, "google")
calendars, _ := ts.db.Calendars(t.Context(), account.ID)
var famID string
for _, c := range calendars {
if c.RemoteID == "fam" {
famID = c.ID
}
}
ts.do(t, "PUT", "/api/v1/calendar/calendars", map[string]any{"shown": []string{famID}})
ts.do(t, "POST", "/api/v1/calendar/sync", nil)
if resp, _ := ts.do(t, "DELETE", "/api/v1/calendar", nil); resp.StatusCode != http.StatusNoContent {
t.Fatalf("disconnect: status %d", resp.StatusCode)
}
_, body := ts.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if events, _ := body["events"].([]any); len(events) != 0 {
t.Errorf("events survived the disconnect: %v", body["events"])
}
_, body = ts.do(t, "GET", "/api/v1/calendar", nil)
if body["connected"] != false {
t.Error("still reported as connected")
}
}
// A range that is not a date has to be refused, not guessed at. An empty `from`
// compared as a string matches everything before it, which is a grid quietly
// answering a question nobody asked.
func TestEventsRefuseARangeThatIsNotADate(t *testing.T) {
g := newFakeCalendarGoogle(t)
ts, _ := connectedCalendar(t, g, nil)
for _, query := range []string{
"", "?from=2026-08-01", "?from=&to=2026-08-31",
"?from=i+morgen&to=2026-08-31", "?from=2026-08-01&to=2026-13-40",
} {
resp, body := ts.do(t, "GET", "/api/v1/calendar/events"+query, nil)
if resp.StatusCode != http.StatusUnprocessableEntity {
t.Errorf("%q: status %d, want 422 (%v)", query, resp.StatusCode, body)
}
}
}
// A person with no calendar connected still opens the view. It has to answer, and
// answer emptily, rather than fail — the grid is also verdande's own tasks, and
// they must not disappear because Google is not set up.
func TestTheGridAnswersWithNoCalendarConnected(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, body := ts.do(t, "GET", "/api/v1/calendar/events?from=2026-08-01&to=2026-08-31", nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if events, ok := body["events"].([]any); !ok || len(events) != 0 {
t.Errorf("events = %v, want an empty list rather than null", body["events"])
}
resp, body = ts.do(t, "GET", "/api/v1/calendar", nil)
if resp.StatusCode != http.StatusOK || body["connected"] != false {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if body["has_client"] != false {
t.Error("an instance with no OAuth registration claimed to have one; the page " +
"would offer a button that answers 409")
}
// Spelled out even when nothing is connected: it is what has to be registered
// with Google, and it is the single most likely thing to be wrong.
if uri, _ := body["redirect_uri"].(string); !strings.HasSuffix(uri, "/oauth/calendar/callback") {
t.Errorf("redirect_uri = %q", uri)
}
}
// A slow Google must not hold the request open until something in front of this
// server gives up on it and answers with its own HTML.
func TestASlowCalendarDoesNotHangTheRequest(t *testing.T) {
blocked := make(chan struct{})
defer close(blocked)
slow := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
select {
case <-blocked:
case <-r.Context().Done():
}
}))
defer slow.Close()
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GoogleTokenURL = slow.URL + "/token"
cfg.CalendarAPIURL = slow.URL
// A second rather than the real twenty-five: what is being tested is that a
// budget is applied at all, and spending the production one on every run
// would put half a minute into the suite to learn the same thing.
cfg.CalendarSyncBudget = time.Second
})
ts.bootstrap(t)
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
if err != nil {
t.Fatal(err)
}
if err := ts.db.SaveCalendarAccount(t.Context(), &store.CalendarAccount{
UserID: user.ID, Provider: "google", RefreshToken: "r",
AccessToken: "a", ExpiresAt: time.Now().Add(time.Hour),
}); err != nil {
t.Fatal(err)
}
// Its own client, because the shared one gives up after ten seconds — which is
// shorter than the budget being tested and would prove nothing either way.
req, _ := http.NewRequest("POST", ts.URL+"/api/v1/calendar/sync", nil)
req.Header.Set("Sec-Fetch-Site", "same-origin")
for _, c := range ts.client.Jar.Cookies(mustParse(t, ts.URL)) {
req.AddCookie(c)
}
started := time.Now()
resp, err := (&http.Client{Timeout: 90 * time.Second}).Do(req)
if err != nil {
t.Fatalf("the request never came back (%v after %s); the sync has no budget of its "+
"own, and whatever sits in front of this server answers for it",
err, time.Since(started).Round(time.Second))
}
defer resp.Body.Close()
if took := time.Since(started); took > 10*time.Second {
t.Errorf("the sync took %s; the budget is not being applied", took.Round(time.Second))
}
if resp.StatusCode >= 500 {
t.Errorf("status %d — a slow Google is not this server breaking", resp.StatusCode)
}
}
+18 -3
View File
@@ -225,17 +225,32 @@ func (s *Server) handleUploadAttachment(w http.ResponseWriter, r *http.Request)
a := &store.Attachment{
TaskID: taskID,
CommentID: r.FormValue("comment_id"),
Filename: safeFilename(header.Filename),
MimeType: detectMime(header.Filename, header.Header.Get("Content-Type")),
Size: size,
Path: stored,
UploadedBy: user.ID,
}
// An attachment hangs on a task or on a comment, never both — the schema says
// so, and the form may have supplied a comment id.
if a.CommentID != "" {
a.TaskID = ""
//
// The comment is resolved and checked against the task in the URL. It was
// taken on trust, which meant the permission check above guarded the *task
// somebody named* while the file landed on *whatever comment they asked for*.
// A viewer on a shared project can list its comments, so she could read an id
// there and then post to a task in her own project with that id attached: the
// file appears under somebody else's comment, in a project she may only read,
// and is served to everyone in it from this origin under a name she chose.
// Somebody removed from a project could do the same with ids they kept.
if id := r.FormValue("comment_id"); id != "" {
owner, err := s.db.CommentTask(r.Context(), id)
if err != nil || owner != taskID {
os.Remove(filepath.Join(s.cfg.FilesDir(), stored))
writeError(w, http.StatusNotFound, CodeNotFound, "not found")
return
}
a.CommentID, a.TaskID = id, ""
}
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
+66
View File
@@ -0,0 +1,66 @@
package httpapi
import (
"strings"
"testing"
)
// Politikken skal stå i svaret, og den skal stadig være stram.
//
// Den er dét, der gjorde en XSS i noteeditoren til indsat HTML frem for kørt
// script, dengang en `alt` slap uden om sin undslupning — og der var ingen prøve
// på, at headeren overhovedet blev sendt. En sikkerhedsforanstaltning, ingen
// prøve rører, er en, der kan forsvinde i en oprydning uden at nogen opdager det.
func TestEveryResponseCarriesAStrictCSP(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
resp, _ := ts.do(t, "GET", "/api/v1/auth/me", nil)
policy := resp.Header.Get("Content-Security-Policy")
if policy == "" {
t.Fatal("der er ingen Content-Security-Policy på svaret")
}
// Målt på script-src alene, og ikke på hele politikken.
//
// `style-src 'unsafe-inline'` står der med vilje og er forklaret i csp.go:
// Svelte skriver stilarter ind i dokumentet, og en indsat stil kan gøre en
// side grim, ikke køre kode. Det er script-src, der afgør, om indsat HTML
// bliver til et kørende script, og det er derfor den, prøven ser på.
scriptSrc := ""
for _, part := range strings.Split(policy, ";") {
if strings.HasPrefix(strings.TrimSpace(part), "script-src") {
scriptSrc = part
}
}
if scriptSrc == "" {
t.Fatal("der er ingen script-src i politikken")
}
for _, forbidden := range []string{"'unsafe-inline'", "'unsafe-eval'", "'unsafe-hashes'", "'strict-dynamic'"} {
if strings.Contains(scriptSrc, forbidden) {
t.Errorf("script-src indeholder %s: %q", forbidden, strings.TrimSpace(scriptSrc))
}
}
// Og de fire, der skal være der.
for _, want := range []string{"script-src", "object-src 'none'", "base-uri 'none'", "frame-ancestors 'none'"} {
if !strings.Contains(policy, want) {
t.Errorf("politikken mangler %q", want)
}
}
}
// Et bilag må ikke kunne blive læst som et dokument.
func TestAttachmentsAreNotServedAsDocuments(t *testing.T) {
for _, mime := range []string{"image/svg+xml", "text/html", "application/xhtml+xml", "text/xml"} {
if inlineImage[mime] {
t.Errorf("%s står på listen over det, der vises inline", mime)
}
}
// Og listen skal faktisk kunne noget, ellers virker billeder i noter ikke.
for _, mime := range []string{"image/png", "image/jpeg"} {
if !inlineImage[mime] {
t.Errorf("%s mangler; billeder i noter kan ikke vises", mime)
}
}
}
+5
View File
@@ -63,6 +63,11 @@ const (
// diagnosis — a wrong app password, a host that is not listening, a folder that
// is not there — and it says it in its own words.
CodeMailboxFailed = "mailbox_failed"
// And for a calendar read from Google. Its own code rather than the Gmail one,
// because the sentence the interface wraps it in names where it came from — and
// "Gmail said no" about a calendar sends somebody to the wrong panel.
CodeCalendarFailed = "calendar_failed"
)
// errLastAdmin is returned to the caller that already wrote the response, so it
+40 -10
View File
@@ -25,18 +25,16 @@ const defaultGmailSyncBudget = 25 * time.Second
// to redeploy a container to paste a client id.
// gmailClient is the API client, pointed wherever the config says Google is.
func (s *Server) gmailClient(accessToken string) *gmail.Client {
return gmail.NewClient(accessToken).At(gmail.Endpoints{
Token: s.cfg.GmailTokenURL, API: s.cfg.GmailAPIURL,
})
return gmail.NewClient(accessToken).At(s.cfg.GmailAPIURL)
}
func (s *Server) gmailConfig(ctx context.Context) gmail.Config {
endpoints := gmail.Endpoints{Token: s.cfg.GmailTokenURL, API: s.cfg.GmailAPIURL}
cfg := gmail.Config{
ClientID: s.cfg.GmailClientID,
ClientSecret: s.cfg.GmailClientSecret,
RedirectURL: s.cfg.GmailRedirectURL(),
Endpoints: endpoints,
Scope: gmail.Scope,
TokenURL: s.cfg.GoogleTokenURL,
}
if cfg.ClientID != "" && cfg.ClientSecret != "" {
return cfg
@@ -251,6 +249,17 @@ func (s *Server) handleGmailSyncNow(w http.ResponseWriter, r *http.Request) {
//
// Returns how many were made. Also used by the background job, which is why it
// takes a context and a user rather than a request.
// maxGmailSeen bounds the list of message ids already turned into tasks.
//
// Gmail is read-only here — the scope is `gmail.readonly`, deliberately, because
// asking for less is what makes the consent screen something a person can agree
// to — so verdande cannot unstar a message the way the IMAP side unflags one. The
// record has to live on this side, and a record on this side has to be bounded.
//
// A count rather than the previous rule of "whatever Gmail returned this run",
// which was not a bound but a way of forgetting.
const maxGmailSeen = 2000
func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
box, err := s.db.MailboxOfKind(ctx, user.ID, "gmail")
if err != nil {
@@ -260,7 +269,7 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
// The same guard as the mailboxes read over IMAP, for the same reason: a
// press of "Fetch now" during the sweep would otherwise make two runs from
// one marker.
unlock := s.lockMailbox(box.ID)
unlock := s.lockSync(box.ID)
defer unlock()
if fresh, err := s.db.MailboxOfKind(ctx, user.ID, "gmail"); err == nil && fresh != nil {
box = fresh
@@ -362,14 +371,35 @@ func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
}
if created > 0 {
// Only the ids Gmail still returns are kept, so the list cannot grow
// without bound as messages fall out of the 30-day window.
keep := make([]string, 0, len(ids))
// The ids this run saw, on top of the ones already known.
//
// It used to keep only the ids Gmail returned *this run*, to stop the list
// growing without bound. That threw away every id that was not on the page
// it happened to fetch — and the query is capped, so on an account with more
// starred mail than one page, each run forgot the messages the last one had
// remembered. They came back unseen, and became tasks again, and again. That
// is the duplicate.
//
// Bounded by a count instead. Newest first, and a ceiling far above any
// plausible page, so an id leaves the list because it is old rather than
// because it was not in the last answer.
keep := make([]string, 0, len(ids)+len(box.Seen))
added := map[string]bool{}
for _, id := range ids {
if seen[id] {
if seen[id] && !added[id] {
keep = append(keep, id)
added[id] = true
}
}
for _, id := range box.Seen {
if !added[id] {
keep = append(keep, id)
added[id] = true
}
}
if len(keep) > maxGmailSeen {
keep = keep[:maxGmailSeen]
}
box.Seen = keep
box.LastSyncAt = time.Now()
if err := s.db.SaveMailbox(ctx, box); err != nil {
+2 -2
View File
@@ -29,7 +29,7 @@ func TestAGmailFailureSaysWhatGmailSaidAndIsRecorded(t *testing.T) {
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GmailTokenURL = google.URL + "/token"
cfg.GoogleTokenURL = google.URL + "/token"
cfg.GmailAPIURL = google.URL
})
ts.bootstrap(t)
@@ -97,7 +97,7 @@ func TestASlowMailboxDoesNotHangTheRequest(t *testing.T) {
ts := newTestServerWith(t, func(cfg *config.Config) {
cfg.GmailClientID = "test-client"
cfg.GmailClientSecret = "test-secret"
cfg.GmailTokenURL = google.URL + "/token"
cfg.GoogleTokenURL = google.URL + "/token"
cfg.GmailAPIURL = google.URL
// A second rather than the real twenty-five: what is being tested is that a
// budget is applied at all, and spending the production one on every run
+19
View File
@@ -11,6 +11,7 @@ import (
"github.com/kristianwind/verdande/internal/ai"
"github.com/kristianwind/verdande/internal/auth"
"github.com/kristianwind/verdande/internal/quickadd"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
@@ -214,6 +215,24 @@ func (s *Server) handleSetAISettings(w http.ResponseWriter, r *http.Request) {
return
}
// The address is checked where it is typed, as well as when it is dialled.
//
// safedial refuses a private destination at connect time and is the actual
// wall; this is so the person entering it hears about it now rather than
// discovering it later as a failed summary. A scheme other than http or https
// is refused outright — file:// and gopher:// are not providers.
if url := strings.TrimSpace(req.BaseURL); url != "" {
if !strings.HasPrefix(url, "http://") && !strings.HasPrefix(url, "https://") {
writeFieldErrors(w, map[string]string{"base_url": "must start with http:// or https://"})
return
}
if reason := safedial.CheckURL(url); reason != "" {
writeFieldErrors(w, map[string]string{"base_url": reason})
return
}
req.BaseURL = url
}
existing, err := s.aiConfig(r, user.ID)
if err != nil {
s.internal(w, r, "ai settings", err)
+326
View File
@@ -2,13 +2,18 @@ package httpapi
import (
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
"github.com/kristianwind/verdande/internal/config"
"github.com/kristianwind/verdande/internal/store"
)
// --- MCP -----------------------------------------------------------------------------
@@ -250,6 +255,116 @@ func (ts *testServer) apiToken(t *testing.T, userID string) string {
return token
}
// The promise of sharing has to hold at every door, not just the API's.
//
// TestViewerCannotMutateThroughTheAPI asserts a viewer cannot write, and it
// asserts it against /api/v1. CalDAV is a second door into the same tasks, and it
// authorised differently: it checked the caller's standing in the project *named
// in the URL* and then took the task id from the URL as well. Nothing tied the
// two together.
//
// So a viewer on somebody else's project could name a project of their own —
// their Inbox, which everybody owns and everybody may edit — and put the other
// project's task id in the filename. The project check passed on the Inbox, and
// the write landed on a task in a project they may only read.
func TestAViewerCannotWriteThroughCalDAV(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
ts.newUser(t, "viewer@example.dk", "Viewer")
_, project := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Delt"})
projectID := project["id"].(string)
_, task := ts.do(t, "POST", "/api/v1/tasks", map[string]any{
"content": "en opgave", "project_id": projectID,
})
taskID := task["id"].(string)
if resp, body := ts.do(t, "POST", "/api/v1/projects/"+projectID+"/invites", map[string]any{
"email": "viewer@example.dk", "role": "viewer",
}); resp.StatusCode != http.StatusOK {
t.Fatalf("invite: %d %v", resp.StatusCode, body)
}
them, err := ts.db.UserByEmail(t.Context(), "viewer@example.dk")
if err != nil {
t.Fatal(err)
}
token := ts.apiToken(t, them.ID)
// Their own Inbox: a project they own and may edit, and the one every account
// has. It is what makes the URL check pass.
projects, err := ts.db.ListProjects(t.Context(), them.ID, false)
if err != nil {
t.Fatal(err)
}
var inbox string
for _, p := range projects {
if p.IsInbox {
inbox = p.ID
}
}
if inbox == "" {
t.Fatal("the viewer has no inbox")
}
vtodo := "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nBEGIN:VTODO\r\nUID:" + taskID +
"\r\nSUMMARY:kapret\r\nSTATUS:COMPLETED\r\nEND:VTODO\r\nEND:VCALENDAR\r\n"
resp := ts.dav(t, "PUT", "/caldav/"+them.ID+"/"+inbox+"/"+taskID+".ics",
token, them.Email, vtodo)
if resp.StatusCode < 400 {
t.Errorf("a viewer wrote through CalDAV: status %d", resp.StatusCode)
}
// And the task is as its owner left it. The status alone is not the assertion:
// the question is whether anything moved.
_, after := ts.do(t, "GET", "/api/v1/tasks/"+taskID, nil)
if after["content"] != "en opgave" {
t.Errorf("the task was renamed by a viewer: %v", after["content"])
}
if after["completed"] != false {
t.Error("the task was completed by a viewer")
}
}
// A task may only be addressed under the project it is actually in.
//
// The same hole seen from the other side: an editor on two projects could reach a
// task in one of them through the other's URL. Nothing is escalated by it — they
// may edit both — but it is the check that was missing, and a URL that lies about
// where a resource lives is how the viewer case got in.
func TestCalDAVRefusesATaskFromAnotherProject(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
if err != nil {
t.Fatal(err)
}
token := ts.apiToken(t, user.ID)
_, one := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Et"})
_, two := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "To"})
_, task := ts.do(t, "POST", "/api/v1/tasks", map[string]any{
"content": "hører til Et", "project_id": one["id"].(string),
})
taskID := task["id"].(string)
vtodo := "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nBEGIN:VTODO\r\nUID:" + taskID +
"\r\nSUMMARY:flyttet i smug\r\nEND:VTODO\r\nEND:VCALENDAR\r\n"
resp := ts.dav(t, "PUT", "/caldav/"+user.ID+"/"+two["id"].(string)+"/"+taskID+".ics",
token, user.Email, vtodo)
if resp.StatusCode < 400 {
t.Errorf("a task was written under the wrong project: status %d", resp.StatusCode)
}
_, after := ts.do(t, "GET", "/api/v1/tasks/"+taskID, nil)
if after["content"] != "hører til Et" {
t.Errorf("the task changed: %v", after["content"])
}
}
func TestCalDAVRequiresCredentials(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
@@ -562,6 +677,48 @@ func TestAIKeyIsNeverReturned(t *testing.T) {
}
}
// Connecting something else must not delete the AI provider.
//
// Reported from use as "the AI settings are not saved": they were saved, and then
// taken. `user_settings` had `user_id` as its whole primary key, and the upsert
// renamed `scope` on conflict — so the `ai`, `gmail` and `calendar` scopes shared
// one row and each write deleted the other two. Every write returned 204.
//
// Through the handlers, because that is where it was seen. The shape is asserted
// against the store as well: see TestOnePersonCanHaveSettingsForMoreThanOneThing.
func TestTheAIProviderSurvivesConnectingSomethingElse(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
ts.do(t, "PUT", "/api/v1/ai/settings", map[string]any{
"provider": "anthropic", "model": "claude-sonnet-5", "api_key": "sk-hemmelig-nøgle",
})
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
if err != nil {
t.Fatal(err)
}
// What the calendar and the mailbox handlers do when a handshake starts: write
// their own scope for the same person.
for _, scope := range []string{"calendar", "gmail"} {
if err := ts.db.SetUserSettings(t.Context(), user.ID, scope,
map[string]any{"pkce_state": "s-" + scope}); err != nil {
t.Fatalf("save %s: %v", scope, err)
}
}
_, body := ts.do(t, "GET", "/api/v1/ai/settings", nil)
if body["provider"] != "anthropic" {
t.Errorf("the provider is %v after connecting two other things, want anthropic", body["provider"])
}
if body["model"] != "claude-sonnet-5" {
t.Errorf("the model is %v", body["model"])
}
if body["has_key"] != true {
t.Error("the key was taken with it")
}
}
func TestGmailSettings(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
@@ -1316,3 +1473,172 @@ func TestTheMailAddressCanBeRotated(t *testing.T) {
t.Error("the second rotation returned the same address")
}
}
// "Der kommer ingen notifikationer" må kunne besvares.
//
// Alt om det var usynligt: indstillingssiden sagde "til for denne enhed", så snart
// der fandtes et abonnement, og en push, tjenesten afviste, gik i en Debug-linje,
// som ikke skrives på en instans, der kører på standardniveau. Man kunne ikke
// skelne "browseren tilmeldte sig aldrig" fra "tjenesten afviste den" fra "der er
// ikke noget forfaldent endnu" — tre forskellige problemer med ét udseende.
func TestATestNotificationSaysWhatHappenedToIt(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
// Ingen enheder: svaret skal sige nul frem for at lade som om det gik godt.
resp, body := ts.do(t, "POST", "/api/v1/push/test", map[string]any{
"title": "Prøve", "body": "kom den frem?",
})
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d, body %v", resp.StatusCode, body)
}
if body["subscriptions"] != float64(0) {
t.Errorf("subscriptions = %v uden tilmeldte enheder", body["subscriptions"])
}
// En enhed, der peger på en tjeneste, som ikke findes. Den skal komme tilbage
// som et navngivent afslag og ikke som tavshed.
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
if err != nil {
t.Fatal(err)
}
if err := ts.db.SavePushSubscription(t.Context(), &store.PushSubscription{
UserID: user.ID,
Endpoint: "https://push.example.invalid/abc",
P256dh: "BEl62iUYgUivxIkv69yViEuiBIa-Ib9-SkvMeAtA3LFgDzkrxZJjSgSnfckjBJuBkr3qBUYIHBQFLXYp5Nksh8U",
Auth: "aUZmY2xDS2FQUXJJVmJIeA",
}); err != nil {
t.Fatal(err)
}
_, body = ts.do(t, "POST", "/api/v1/push/test", map[string]any{"title": "Prøve"})
if body["subscriptions"] != float64(1) {
t.Fatalf("subscriptions = %v", body["subscriptions"])
}
failed, _ := body["failed"].([]any)
if len(failed) != 1 {
t.Fatalf("failed har %d poster, want 1 — afslaget blev slugt", len(failed))
}
first, _ := failed[0].(map[string]any)
if first["service"] != "push.example.invalid" {
t.Errorf("service = %v", first["service"])
}
if first["reason"] == "" || first["reason"] == nil {
t.Error("afslaget kom uden en grund")
}
// Og aldrig hele adressen: resten af den er den legitimation, tjenesten gav
// enheden, og den hører hverken i et svar, en log eller et skærmbillede.
if strings.Contains(fmt.Sprint(first), "/abc") {
t.Errorf("endepunktet lækkede med ud: %v", first)
}
}
// Et abonnement på en kalender, man kun har en adresse til.
//
// Bedt om som "abonnering af flere kalendere (ics og webcal?)". Google-vejen er
// OAuth og én konto pr. person; det her er en adresse, og man har flere. Det er
// også den eneste vej for en privat konto, som en Internal-registrering afviser
// med `org_internal`.
//
// Prøven går uden om handleren og ind i det, den kalder. Grunden er væggen:
// `httptest` lytter på loopback, og safedial nægter — både når adressen skrives
// ned og igen når den hentes. Det er rigtigt, og det skal ikke kunne slås fra fra
// en prøve. Så adressereglerne prøves for sig i TestWebcalIsRewrittenToHTTPS, og
// her prøves resten af vejen: hentning, format, navn, gitteret og afmeldingen.
func TestSubscribingToACalendarByAddress(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
day := time.Now().AddDate(0, 0, 2).Format("20060102")
feedBody := "BEGIN:VCALENDAR\r\nVERSION:2.0\r\nX-WR-CALNAME:Helligdage\r\n" +
"BEGIN:VEVENT\r\nUID:en@example.dk\r\n" +
"DTSTART;VALUE=DATE:" + day + "\r\nDTEND;VALUE=DATE:" + day + "\r\n" +
"SUMMARY:Grundlovsdag\r\nEND:VEVENT\r\nEND:VCALENDAR\r\n"
var hits int
feed := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
hits++
w.Header().Set("Content-Type", "text/calendar")
w.Write([]byte(feedBody))
}))
defer feed.Close()
ts.api.icsFetch = &http.Client{Timeout: 10 * time.Second}
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
if err != nil {
t.Fatal(err)
}
// Tilføjet under sin adresse, som er alt der er at gå efter, før den er hentet.
if _, err := ts.db.SubscribeCalendar(t.Context(), user.ID, feed.URL, feed.URL); err != nil {
t.Fatal(err)
}
if _, err := ts.api.syncSubscriptions(t.Context(), user); err != nil {
t.Fatal(err)
}
if hits == 0 {
t.Fatal("adressen blev aldrig hentet")
}
// Navnet kommer fra filen. X-WR-CALNAME er det, alle udgivere bruger; der
// findes ingen standardegenskab for det.
_, status := ts.do(t, "GET", "/api/v1/calendar", nil)
subs, _ := status["subscriptions"].([]any)
if len(subs) != 1 {
t.Fatalf("%d abonnementer i svaret", len(subs))
}
first, _ := subs[0].(map[string]any)
if first["account"] != "Helligdage" {
t.Errorf("navnet er %v, want Helligdage", first["account"])
}
// Og begivenheden står i gitteret.
from := time.Now().AddDate(0, 0, -1).Format("2006-01-02")
to := time.Now().AddDate(0, 0, 7).Format("2006-01-02")
_, events := ts.do(t, "GET", "/api/v1/calendar/events?from="+from+"&to="+to, nil)
list, _ := events["events"].([]any)
if len(list) != 1 {
t.Fatalf("%d begivenheder efter abonnementet", len(list))
}
if e, _ := list[0].(map[string]any); e["summary"] != "Grundlovsdag" {
t.Errorf("begivenheden er %v", e["summary"])
}
// Den samme adresse to gange er den samme kalender hentet to gange, og der er
// ingen måde at sige hvilken der gælder.
if _, err := ts.db.SubscribeCalendar(t.Context(), user.ID, feed.URL, ""); !errors.Is(err, store.ErrDuplicate) {
t.Errorf("den samme adresse blev accepteret to gange: %v", err)
}
// Fjernet igen, og begivenhederne går med.
resp, _ := ts.do(t, "DELETE", "/api/v1/calendar/subscriptions/"+first["id"].(string), nil)
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("unsubscribe: status %d", resp.StatusCode)
}
_, events = ts.do(t, "GET", "/api/v1/calendar/events?from="+from+"&to="+to, nil)
if list, _ := events["events"].([]any); len(list) != 0 {
t.Errorf("%d begivenheder blev tilbage efter afmeldingen", len(list))
}
}
// webcal:// er https med et andet skema foran. Det er ikke en protokol, noget
// taler — det er en konvention, der får et skrivebord til at give adressen til
// kalenderprogrammet i stedet for til browseren, og alle udgivere skriver deres
// links sådan. At afvise den ville være at afvise folk på en teknikalitet.
func TestWebcalIsRewrittenToHTTPS(t *testing.T) {
got, err := normaliseCalendarURL("webcal://example.dk/kalender.ics")
if err != nil {
t.Fatal(err)
}
if got != "https://example.dk/kalender.ics" {
t.Errorf("blev til %q", got)
}
// Og en adresse indad afvises, hvor den skrives — ikke først når den hentes.
if _, err := normaliseCalendarURL("http://127.0.0.1:8080/x.ics"); err == nil {
t.Error("en loopback-adresse blev godtaget")
}
if _, err := normaliseCalendarURL("file:///etc/passwd"); err == nil {
t.Error("et skema, der ikke er http, blev godtaget")
}
}
+28 -3
View File
@@ -143,7 +143,7 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
// One reader of this mailbox at a time. Held for the whole run, marker and all:
// releasing before the marker is written would leave exactly the gap this is
// here to close.
unlock := s.lockMailbox(m.ID)
unlock := s.lockSync(m.ID)
defer unlock()
// Re-read under the lock. The row that came in may have been fetched before the
@@ -175,6 +175,10 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
created := 0
highest := m.LastUID
// The uids of the mails that actually became tasks. Only those get unflagged,
// and only after the task is committed: a flag removed from a mail whose task
// was never written is a mail nobody will ever see again.
done := make([]uint32, 0, len(messages))
for _, msg := range messages {
if ctx.Err() != nil {
// What was made so far is already committed and counted, and the marker
@@ -204,12 +208,27 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
continue
}
created++
done = append(done, msg.UID)
if msg.UID > highest {
highest = msg.UID
}
s.hub.Publish(projectID, "task.created", toTaskJSON(*task))
}
// The flag comes off what became a task. This is what actually stops a mail
// arriving twice; the marker below is now a second line rather than the only
// one. See imap.Unflag for why the record belongs in the mailbox and not here.
//
// Outside the request's context, like the marker: a cancelled request that
// already created tasks must still take the flags off them, or the next run
// makes them all again.
if err := client.Unflag(done); err != nil {
// Not fatal, and not the caller's problem. A folder somebody opened
// read-only, or a server that refuses the STORE, still gets the uid marker
// underneath — worse, but not broken.
s.log.Warn("could not unflag imported mail", "err", err, "mailbox", m.ID, "count", len(done))
}
// A run that got all the way through moves the marker to the whole of what the
// search considered, not merely to the last message it happened to read. That
// is what stops the same mail becoming a task again on the next run when the
@@ -231,8 +250,14 @@ func (s *Server) SyncMailbox(ctx context.Context, user *store.User, m *store.Mai
return created, nil
}
// lockMailbox serialises one mailbox against itself and returns the release.
func (s *Server) lockMailbox(id string) func() {
// lockSync serialises one thing that polls against itself and returns the release.
//
// A mailbox to begin with; a calendar connection now takes the same lock under a
// key of its own. Two runs at once — somebody pressing "fetch now" while the sweep
// is in the middle of the same thing — both start from the state as it was, and
// the loser writes last. For a mailbox that made forty copies of one mail in a
// second; for a calendar it would be a window deleted after it was refilled.
func (s *Server) lockSync(id string) func() {
value, _ := s.syncing.LoadOrStore(id, &sync.Mutex{})
mu := value.(*sync.Mutex)
mu.Lock()
+231 -5
View File
@@ -6,7 +6,9 @@ import (
"io"
"mime"
"net/http"
"os"
"path"
"path/filepath"
"sort"
"strings"
"time"
@@ -46,7 +48,7 @@ func (s *Server) handleListNotes(w http.ResponseWriter, r *http.Request) {
s.internal(w, r, "search notes", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": s.readable(r, found)})
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(s.readable(r, found))})
return
}
@@ -60,7 +62,20 @@ func (s *Server) handleListNotes(w http.ResponseWriter, r *http.Request) {
s.internal(w, r, "notes in project", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": found})
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
return
}
// The archive is the same list asked the other way round, so it is the same
// endpoint with a flag rather than a second one that would have to be kept in
// step with this.
if r.URL.Query().Get("archived") != "" {
found, err := s.db.ArchivedNotes(r.Context(), user.ID)
if err != nil {
s.internal(w, r, "list archived notes", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
return
}
@@ -72,7 +87,41 @@ func (s *Server) handleListNotes(w http.ResponseWriter, r *http.Request) {
s.internal(w, r, "list notes", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"notes": found})
writeJSON(w, http.StatusOK, map[string]any{"notes": briefly(found)})
}
// briefly replaces each note's body with the first of it.
//
// The list shows a title and about ninety characters. It was being sent whole
// notes to do that: with twelve hundred of them the response was 845 KB, of which
// ninety-eight per cent was body text nobody would read on that screen — fetched,
// parsed and held in memory on every visit.
//
// Body is emptied rather than shortened, and that is the important half. A field
// called `body` that holds a *piece* of the body is a trap: the editor would open
// it, the debounce would save it, and the note would be cut down to its own
// preview by nothing more than being looked at. Empty is a shape the client can
// check, and it does — see the notes page, which fetches the whole note when it
// opens one.
//
// Four hundred characters rather than ninety: the client strips Markdown before
// it shows anything, and a heading marker and a bold word are characters that
// vanish. Cheap insurance against a preview that ends up shorter than the line it
// is meant to fill.
const previewChars = 400
func briefly(notes []store.Note) []store.Note {
out := make([]store.Note, 0, len(notes))
for _, n := range notes {
body := []rune(n.Body)
if len(body) > previewChars {
body = body[:previewChars]
}
n.Preview = string(body)
n.Body = ""
out = append(out, n)
}
return out
}
// readable drops what this person may not see.
@@ -292,6 +341,14 @@ func noteFilename(title string) string {
// a few megabytes — and small enough that a wrong file is refused rather than read.
const maxNoteImportBytes = 64 << 20
// What the archive may become once it is unpacked, and how many entries it may
// hold. See handleImportNotes: the upload cap bounds what arrives, and says
// nothing at all about what it expands to.
const (
maxNoteImportUnpacked = 2 << 30
maxNoteImportFiles = 20_000
)
// handleImportNotes reads a zip of Markdown files and makes a note of each.
//
// The mirror of the export, and deliberately the same shape: whatever comes out of
@@ -330,24 +387,57 @@ func (s *Server) handleImportNotes(w http.ResponseWriter, r *http.Request) {
// note is read its pictures already have addresses to be rewritten to — a note
// and its images arrive in whatever order the archive happens to hold them, and
// one pass would leave half the links pointing at nothing.
// The archive as a whole, not just the upload.
//
// The 64 MB cap is on what arrives; deflate reaches about 1032:1, so those 64
// MB can become sixty-six gigabytes on the way out — written into the same
// volume the database lives on. The upload limit says nothing about that, and
// nothing else did either.
//
// Two ceilings: one per file, and one across the archive. A single enormous
// photograph and ten thousand small ones are different attacks with the same
// ending.
if len(zr.File) > maxNoteImportFiles {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
"that archive holds too many files")
return
}
budget := int64(maxNoteImportUnpacked)
stored := map[string]string{}
for _, f := range zr.File {
name := f.Name
if f.FileInfo().IsDir() || skipFile(name) || strings.EqualFold(path.Ext(name), ".md") {
continue
}
// The header's own claim first, which costs nothing to check and rejects
// the obvious case before a byte is written. It is only a claim, so the
// reader below is limited as well.
if f.UncompressedSize64 > maxUploadBytes {
s.log.Warn("import: a file in the archive is too large", "file", name)
skipped++
continue
}
rc, err := f.Open()
if err != nil {
continue
}
hashed, size, err := s.storeUpload(rc)
hashed, size, err := s.storeUpload(io.LimitReader(rc, min(budget, maxUploadBytes)+1))
rc.Close()
if err != nil {
s.log.Warn("import note file", "err", err, "file", name)
continue
}
if size > maxUploadBytes || size > budget {
// Over the line once it was actually read: remove what was written
// rather than keeping a truncated file that would open as nothing.
os.Remove(filepath.Join(s.cfg.FilesDir(), hashed))
s.log.Warn("import: archive is larger unpacked than allowed", "file", name)
skipped++
continue
}
budget -= size
stored[name] = hashed
_ = size
files++
}
@@ -579,3 +669,139 @@ func mimeOf(name string) string {
}
return "application/octet-stream"
}
// handleUploadNoteAttachment takes a file into a note.
//
// The import could already do this, and nothing else could: a note brought in
// from Apple Notes arrived with its pictures, and a note written here could not
// be given one. Pasting a screenshot into a note is the most ordinary thing there
// is, and it did nothing at all.
//
// Editor rights on the note, checked the same way every other write to it is. The
// answer for "no such note" and "not yours" is the same, so an id cannot be
// probed by watching which one comes back.
func (s *Server) handleUploadNoteAttachment(w http.ResponseWriter, r *http.Request) {
n, err := s.db.Note(r.Context(), chi.URLParam(r, "noteID"))
if err != nil {
s.internal(w, r, "get note", err)
return
}
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
writeError(w, http.StatusNotFound, CodeNotFound, "no such note")
return
}
r.Body = http.MaxBytesReader(w, r.Body, maxUploadBytes+(1<<20))
if err := r.ParseMultipartForm(8 << 20); err != nil {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
"that file is too large (25 MB maximum)")
return
}
defer r.MultipartForm.RemoveAll()
file, header, err := r.FormFile("file")
if err != nil {
writeFieldErrors(w, map[string]string{"file": "required"})
return
}
defer file.Close()
if header.Size > maxUploadBytes {
writeError(w, http.StatusRequestEntityTooLarge, CodePayloadTooLarge,
"that file is too large (25 MB maximum)")
return
}
stored, size, err := s.storeUpload(file)
if err != nil {
s.internal(w, r, "store upload", err)
return
}
user := userFrom(r.Context())
a := &store.Attachment{
NoteID: n.ID,
Filename: safeFilename(header.Filename),
MimeType: detectMime(header.Filename, header.Header.Get("Content-Type")),
Size: size,
Path: stored,
UploadedBy: user.ID,
}
if err := s.db.CreateAttachment(r.Context(), a); err != nil {
os.Remove(filepath.Join(s.cfg.FilesDir(), stored))
s.internal(w, r, "record attachment", err)
return
}
writeJSON(w, http.StatusCreated, toAttachmentJSON(*a))
}
// --- putting notes away, one or many ---------------------------------------------
type noteBulkRequest struct {
IDs []string `json:"ids"`
Archived *bool `json:"archived"`
Delete bool `json:"delete"`
}
// maxNoteBulk bounds one call. Enough to select a screenful several times over,
// and few enough that the loop below cannot become somebody's afternoon.
const maxNoteBulk = 500
// handleNoteBulk archives, unarchives or deletes a set of notes in one request.
//
// One call rather than one per note, because the thing being asked is one thing:
// "put these away". Fifty separate requests would be fifty round trips, fifty
// chances for half of them to fail, and a list that redraws fifty times — and
// after an import of twelve hundred notes, selecting fifty is the ordinary case,
// not the extreme one.
//
// Each note is still checked on its own. A set is a convenience for the caller
// and must never become a way to touch something they could not have touched one
// at a time.
func (s *Server) handleNoteBulk(w http.ResponseWriter, r *http.Request) {
var req noteBulkRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
if len(req.IDs) == 0 {
writeFieldErrors(w, map[string]string{"ids": "required"})
return
}
if len(req.IDs) > maxNoteBulk {
writeFieldErrors(w, map[string]string{"ids": "too many at once"})
return
}
done, skipped := 0, 0
for _, id := range req.IDs {
n, err := s.db.Note(r.Context(), id)
if err != nil {
s.internal(w, r, "get note", err)
return
}
// Not an error, and not a 404 for the whole call: a set can contain a note
// somebody else deleted a second ago, and the other forty-nine should still
// be put away. Counted so the answer is honest about it.
if n == nil || !s.mayTouchNote(r, n, store.RoleEditor) {
skipped++
continue
}
switch {
case req.Delete:
err = s.db.DeleteNote(r.Context(), id)
case req.Archived != nil:
err = s.db.SetNoteArchived(r.Context(), id, *req.Archived)
default:
writeFieldErrors(w, map[string]string{"archived": "say archived or delete"})
return
}
if err != nil {
s.internal(w, r, "note bulk", err)
return
}
done++
}
writeJSON(w, http.StatusOK, map[string]any{"done": done, "skipped": skipped})
}
@@ -0,0 +1,77 @@
package httpapi
import (
"archive/zip"
"bytes"
"io"
"testing"
)
// En zip-bombe skal ikke kunne skrive gigabyte ned på den disk, databasen ligger
// på.
//
// Uploaden er begrænset til 64 MB, og det siger ingenting om, hvad de 64 MB
// bliver til: deflate når omkring 1032:1, så de kan pakkes ud til seksogtres
// gigabyte. Prøven bygger en rigtig lille bombe — fire megabyte nuller, som
// pakker til nogle få kilobyte — og ser efter, at loftet på det udpakkede holder.
//
// Skrevet imod tallene frem for imod endepunktet, fordi det, der skal holde, er
// forholdet: en fil, hvis udpakkede størrelse overstiger loftet, må ikke læses
// færdig.
func TestAZipBombIsRefusedByItsUnpackedSize(t *testing.T) {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, err := zw.Create("bombe.bin")
if err != nil {
t.Fatal(err)
}
// Nuller pakker næsten helt væk. Fire megabyte er nok til at vise forholdet
// uden at prøven selv bruger en gigabyte.
if _, err := w.Write(make([]byte, 4<<20)); err != nil {
t.Fatal(err)
}
if err := zw.Close(); err != nil {
t.Fatal(err)
}
packed := int64(buf.Len())
if packed > 64<<10 {
t.Fatalf("prøvens egen bombe pakkede kun til %d bytes; den viser ikke noget", packed)
}
zr, err := zip.NewReader(bytes.NewReader(buf.Bytes()), packed)
if err != nil {
t.Fatal(err)
}
f := zr.File[0]
if f.UncompressedSize64 != 4<<20 {
t.Fatalf("udpakket størrelse = %d", f.UncompressedSize64)
}
// Det, importen gør: læs højst det, budgettet tillader, plus én byte — så en
// fil, der er præcis på grænsen, kan skelnes fra en, der er over.
rc, err := f.Open()
if err != nil {
t.Fatal(err)
}
defer rc.Close()
budget := int64(1 << 20)
n, err := io.Copy(io.Discard, io.LimitReader(rc, budget+1))
if err != nil {
t.Fatal(err)
}
if n <= budget {
t.Fatalf("læste %d bytes af en fil på 4 MB med et budget på %d — loftet fanger den ikke", n, budget)
}
}
// Tallene skal stå i et forhold, der giver mening: en enkelt fil må aldrig kunne
// bruge hele arkivets budget, og arkivet må ikke være mindre end én fil.
func TestTheImportCeilingsAgreeWithEachOther(t *testing.T) {
if maxNoteImportUnpacked <= maxUploadBytes {
t.Error("arkivets budget er ikke større end én enkelt fil")
}
if maxNoteImportFiles < 2000 {
t.Error("filloftet er under det, en rigtig Apple Noter-eksport indeholder")
}
}
+8 -1
View File
@@ -328,12 +328,19 @@ func TestAnImportedNoteKeepsItsPictures(t *testing.T) {
t.Fatalf("imported %d notes and %d files, want 2 and 1", out.Created, out.Files)
}
// The list carries a preview, not the whole note — see briefly(). The body is
// fetched by id, which is what the interface does when it opens one.
_, listed := ts.do(t, "GET", "/api/v1/notes?q=kvittering", nil)
notes := listed["notes"].([]any)
if len(notes) != 1 {
t.Fatalf("found %d notes", len(notes))
}
body := notes[0].(map[string]any)["body"].(string)
noteID := notes[0].(map[string]any)["id"].(string)
if got := notes[0].(map[string]any)["body"]; got != nil && got != "" {
t.Errorf("the list carried a body: %v — an editor handed that would save the truncation", got)
}
_, whole := ts.do(t, "GET", "/api/v1/notes/"+noteID, nil)
body := whole["body"].(string)
// The link points at the file where it now lives, not at a path inside an
// archive nobody kept.
+143 -5
View File
@@ -1,12 +1,16 @@
package httpapi
import (
"errors"
"net/http"
"net/url"
"strings"
"time"
"github.com/go-chi/chi/v5"
"github.com/kristianwind/verdande/internal/push"
"github.com/kristianwind/verdande/internal/safedial"
"github.com/kristianwind/verdande/internal/store"
)
@@ -104,18 +108,44 @@ func (s *Server) PushToUser(userID, title, body, projectID string) {
s.pushToUser(userID, title, body, projectID)
}
func (s *Server) pushToUser(userID, title, body, projectID string) {
// pushOutcome is what happened, per device.
//
// It exists because "nothing arrives" was unanswerable. A push that a service
// refuses is the ordinary failure here — a stale subscription, a VAPID subject the
// service will not take, a browser that revoked permission without telling the
// server — and every one of them looked exactly like silence.
type pushOutcome struct {
Subscriptions int `json:"subscriptions"`
Sent int `json:"sent"`
Failed []pushFailure `json:"failed"`
}
// pushFailure names the service and quotes what it said.
//
// The host and not the endpoint: the rest of that URL is the credential the push
// service issued for this device, and it does not belong in an answer, a log line
// or a screenshot.
type pushFailure struct {
Service string `json:"service"`
Reason string `json:"reason"`
}
func (s *Server) pushToUser(userID, title, body, projectID string) pushOutcome {
ctx, cancel := contextWithBackgroundTimeout(30 * time.Second)
defer cancel()
var out pushOutcome
subs, err := s.db.ListPushSubscriptions(ctx, userID)
if err != nil || len(subs) == 0 {
return
return out
}
out.Subscriptions = len(subs)
public, private, err := s.vapidKeys(ctx)
if err != nil {
s.log.Warn("vapid keys", "err", err)
return
out.Failed = append(out.Failed, pushFailure{Service: "verdande", Reason: err.Error()})
return out
}
payload := push.Payload{Title: title, Body: body, URL: "/projekt/" + projectID}
@@ -126,13 +156,101 @@ func (s *Server) pushToUser(userID, title, body, projectID string) {
Public: public, Private: private, Subject: "mailto:" + s.cfg.SMTP.From,
})
if err == nil {
out.Sent++
continue
}
s.log.Debug("web push failed", "err", err, "endpoint", sub.Endpoint)
// Warn, not Debug.
//
// This was the one line that would have said why no notification ever
// arrived, and at Debug it is not printed on an instance running at the
// default level. A delivery that fails silently and a delivery that does
// not happen look the same from the outside, and the difference is the
// whole diagnosis.
reason := pushReason(sub.Endpoint, err)
s.log.Warn("web push refused", "reason", reason, "service", pushHost(sub.Endpoint), "user", userID)
out.Failed = append(out.Failed, pushFailure{
Service: pushHost(sub.Endpoint),
Reason: reason,
})
if recErr := s.db.RecordPushFailure(ctx, sub.Endpoint, push.IsGone(err)); recErr != nil {
s.log.Warn("record push failure", "err", recErr)
}
}
return out
}
// pushReason is what went wrong, with the endpoint taken back out of it.
//
// Go wraps a transport failure as `*url.Error`, and its message is `Post
// "https://fcm.googleapis.com/fcm/send/dQw4w9…": dial tcp: …` — the whole
// endpoint, which is the credential the push service issued for that device. It
// would have gone into the answer, the log line and any screenshot of either. A
// test written to prove the host was reported instead is what found it.
//
// Unwrapped first, so the cause is what is quoted. Then the endpoint is scrubbed
// anyway: the next error type to carry it will not come with a warning.
func pushReason(endpoint string, err error) string {
var urlErr *url.Error
if errors.As(err, &urlErr) && urlErr.Err != nil {
err = urlErr.Err
}
reason := err.Error()
if endpoint != "" {
reason = strings.ReplaceAll(reason, endpoint, pushHost(endpoint))
}
return reason
}
// pushHost is the service an endpoint belongs to, and nothing else from it.
func pushHost(endpoint string) string {
u, err := url.Parse(endpoint)
if err != nil || u.Host == "" {
return "ukendt"
}
return u.Host
}
// handlePushTest sends one notification to this account's devices and says what
// happened to it.
//
// Reported as "der kommer ikke nogen". Everything about that was invisible: the
// settings page said "on for this device" as soon as a subscription existed, and a
// refusal from the push service went into a Debug line nobody sees. There was no
// way to tell "the browser never subscribed" from "the service refused it" from
// "nothing has come due yet" — three different problems with one appearance.
//
// It goes through the same path as a real notification rather than a shortcut, so
// what it proves is the thing that actually has to work.
//
// The words come from the client, which is where the dictionaries are. A server
// that carries its own copy of "This is a test" carries it in one language, and
// this one deliberately keeps its prose in the interface — see
// `internal/i18n`. The text is only ever shown back to the person who asked for
// it, on their own devices.
func (s *Server) handlePushTest(w http.ResponseWriter, r *http.Request) {
var req struct {
Title string `json:"title"`
Body string `json:"body"`
}
if err := decodeJSON(w, r, &req); err != nil {
return
}
title := strings.TrimSpace(req.Title)
if title == "" {
title = "verdande"
}
// Bounded: this is written into a notification on somebody's lock screen, and
// a megabyte of title is not a notification.
if len(title) > 200 {
title = title[:200]
}
body := strings.TrimSpace(req.Body)
if len(body) > 500 {
body = body[:500]
}
user := userFrom(r.Context())
writeJSON(w, http.StatusOK, s.pushToUser(user.ID, title, body, ""))
}
func (s *Server) vapidKeys(ctx contextLike) (string, string, error) {
@@ -170,6 +288,23 @@ func (s *Server) handleSubscribePush(w http.ResponseWriter, r *http.Request) {
return
}
// The endpoint is an address this server will POST to, on a schedule the
// caller chooses — set a reminder a minute out and the push job fetches it,
// again every minute. Unchecked, that is a way to make the instance reach
// anything it can route to, on demand.
//
// https only: a push service is on the public internet and speaks TLS. Nothing
// legitimate needs http here, and allowing it is the difference between a
// blind request and a readable one.
if !strings.HasPrefix(req.Endpoint, "https://") {
writeFieldErrors(w, map[string]string{"endpoint": "must be an https address"})
return
}
if reason := safedial.CheckURL(req.Endpoint); reason != "" {
writeFieldErrors(w, map[string]string{"endpoint": reason})
return
}
err := s.db.SavePushSubscription(r.Context(), &store.PushSubscription{
UserID: userFrom(r.Context()).ID,
Endpoint: req.Endpoint,
@@ -189,7 +324,10 @@ func (s *Server) handleUnsubscribePush(w http.ResponseWriter, r *http.Request) {
if err := decodeJSON(w, r, &req); err != nil {
return
}
if err := s.db.DeletePushSubscription(r.Context(), req.Endpoint); err != nil {
// Scoped to the caller. It was not, so anybody who learned somebody else's
// endpoint could unsubscribe their device — a small thing to do to somebody
// and no reason at all to be able to.
if err := s.db.DeletePushSubscription(r.Context(), userFrom(r.Context()).ID, req.Endpoint); err != nil {
s.internal(w, r, "delete push subscription", err)
return
}
+3 -3
View File
@@ -102,7 +102,7 @@ func TestTwoRunsOfOneMailboxDoNotOverlap(t *testing.T) {
// Stands in for a run: takes the lock, notes how many are inside, and holds it
// until told to let go.
run := func(done chan<- struct{}) {
unlock := srv.lockMailbox("box-1")
unlock := srv.lockSync("box-1")
mu.Lock()
inside++
if inside > most {
@@ -136,10 +136,10 @@ func TestTwoRunsOfOneMailboxDoNotOverlap(t *testing.T) {
// And two different mailboxes must not wait for each other: one slow host is
// not everybody's problem.
a := srv.lockMailbox("box-a")
a := srv.lockSync("box-a")
done := make(chan struct{})
go func() {
srv.lockMailbox("box-b")()
srv.lockSync("box-b")()
close(done)
}()
select {
+28
View File
@@ -356,6 +356,34 @@ func (s *Server) handleCreateSection(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusCreated, sectionJSON{sec.ID, sec.ProjectID, sec.Name, sec.SortOrder})
}
// handleReorderSections sets the order of the sections inside one project.
//
// Under /projects/{projectID}/ rather than beside the other section routes, which
// hang off a section id: the thing being changed here is the project's list, not
// any one section in it. That also puts it behind the same project permission
// check as everything else in this subtree, instead of resolving a section to its
// project one id at a time and hoping they all named the same one.
func (s *Server) handleReorderSections(w http.ResponseWriter, r *http.Request) {
var req reorderProjectsRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
// The same ceiling as the projects list, and a bound on the transaction rather
// than an opinion about how many sections a project may have.
if len(req.IDs) > 500 {
writeFieldErrors(w, map[string]string{"ids": "at most 500 at a time"})
return
}
projectID := chi.URLParam(r, "projectID")
if err := s.db.ReorderSections(r.Context(), projectID, req.IDs); err != nil {
s.internal(w, r, "reorder sections", err)
return
}
s.activity(r, projectID, "", "section.updated", nil)
w.WriteHeader(http.StatusNoContent)
}
// sectionAccess resolves a section to its project and checks the caller may edit
// it. Sections have no permissions of their own — they belong to a project, and
// that is where access is decided.
+65
View File
@@ -327,6 +327,71 @@ func TestAProjectCanHaveSeveralSections(t *testing.T) {
}
}
// Sections can be dragged into another order, and the order is the project's.
//
// The list is what a project's plan *is* — "Planlægning" before "I gang" is the
// order the work happens in — and until this endpoint existed the only way to
// change it was to delete a section and make it again, which moves its tasks to
// the unsectioned area on the way past.
func TestSectionsCanBeReordered(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
_, project := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Ombygning"})
id := project["id"].(string)
ids := map[string]string{}
names := []string{"Planlægning", "I gang", "Til gennemsyn"}
for _, name := range names {
_, body := ts.do(t, "POST", "/api/v1/projects/"+id+"/sections", map[string]any{"name": name})
ids[name] = body["id"].(string)
}
want := []string{"Til gennemsyn", "Planlægning", "I gang"}
order := make([]string, len(want))
for i, name := range want {
order[i] = ids[name]
}
resp, body := ts.do(t, "POST", "/api/v1/projects/"+id+"/sections/reorder",
map[string]any{"ids": order})
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("reorder: status %d, body %v", resp.StatusCode, body)
}
_, listed := ts.do(t, "GET", "/api/v1/projects/"+id+"/sections", nil)
got, _ := listed["sections"].([]any)
if len(got) != len(want) {
t.Fatalf("the project has %d sections, want %d", len(got), len(want))
}
for i, raw := range got {
if name := raw.(map[string]any)["name"]; name != want[i] {
t.Errorf("section %d is %v, want %q", i, name, want[i])
}
}
// A section belonging to somebody else's project is not moved by naming it
// here. The ids in the body are not checked one by one — the WHERE clause is
// what refuses them — so this is the assertion that it actually does.
_, other := ts.do(t, "POST", "/api/v1/projects", map[string]any{"name": "Sommerhus"})
otherID := other["id"].(string)
_, foreign := ts.do(t, "POST", "/api/v1/projects/"+otherID+"/sections",
map[string]any{"name": "Havearbejde"})
foreignID := foreign["id"].(string)
resp, _ = ts.do(t, "POST", "/api/v1/projects/"+id+"/sections/reorder",
map[string]any{"ids": []string{foreignID}})
if resp.StatusCode != http.StatusNoContent {
t.Fatalf("reorder with a foreign id: status %d", resp.StatusCode)
}
_, stillThere := ts.do(t, "GET", "/api/v1/projects/"+otherID+"/sections", nil)
elsewhere, _ := stillThere["sections"].([]any)
if len(elsewhere) != 1 || elsewhere[0].(map[string]any)["name"] != "Havearbejde" {
t.Errorf("the other project's section did not survive: %v", elsewhere)
}
}
// A project can be created straight into a group.
//
// `group_id` was declared on the request type, documented on it, and applied by
+97 -5
View File
@@ -63,7 +63,15 @@ type Server struct {
// the endpoints that check a secret are limited separately and more tightly
// than the rest of the API.
loginLimiter *limiter
// Sat kun af prøver; se icsClient().
icsFetch *http.Client
mailLimiter *limiter
resetLimiter *limiter
// Second factors are counted against the *account*, not the address. An
// address bucket is defeated by anybody with more than one address, which is
// the ordinary case — and a six-digit code is only safe while guesses are
// scarce, so the scarcity has to be counted against the thing being attacked.
totpLimiter *limiter
// Computed once from the built index.html; see csp.go.
csp string
@@ -80,7 +88,16 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
mail: mail.New(cfg.SMTP, cfg.BaseURL, log),
hub: realtime.NewHub(log),
loginLimiter: newLimiter(10, 15*time.Minute),
// Generous on purpose: the caller is the mail server, one address delivering
// everything, and a personal instance gets a handful of forwarded mails a
// day. Sixty a minute never touches real delivery and still puts a ceiling
// on the one public write path that has none.
mailLimiter: newLimiter(60, time.Minute),
resetLimiter: newLimiter(5, time.Hour),
// Tighter than the password bucket, and for a whole hour. Somebody typing
// their own code gets it right on the first or second try; ten in an hour is
// generous for a person and hopeless for a search through a million.
totpLimiter: newLimiter(10, time.Hour),
}
s.csp = contentSecurityPolicy(scriptHashes(web))
if w, err := auth.NewWebAuthn(cfg.BaseURL, "verdande"); err == nil {
@@ -113,7 +130,15 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
// Inbound mail. Delivered by the mail server rather than by a browser, and
// authenticated by the token in the recipient address.
r.Post("/inbound/mail", s.handleInboundMail)
//
// Rate limited for the same reason the beacon is: it is reachable by anybody
// who can route to this instance. It is the weaker case of the two — the
// beacon writes a row for a stranger, this one writes nothing without a
// 256-bit address it has to already know — but an address is shared with mail
// servers and printed on a settings page, and once it is known there was no
// ceiling at all on what could be made with it.
r.Method(http.MethodPost, "/inbound/mail",
s.rateLimit(s.mailLimiter, http.HandlerFunc(s.handleInboundMail)))
// MCP with the token in the query string, for clients that cannot send a
// header. Claude's custom-connector dialog takes a URL and nothing else, so
@@ -133,6 +158,12 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Group(func(r chi.Router) {
r.Use(s.requireAuth)
r.Get("/oauth/gmail/callback", s.handleGmailCallback)
// The calendar comes back on a URI of its own rather than sharing Gmail's.
// One callback deciding from stored state which of the two flows had
// returned would put the choice of *which token store to write* behind a
// lookup — and when that goes wrong it goes wrong by overwriting a working
// Gmail connection with calendar tokens.
r.Get("/oauth/calendar/callback", s.handleCalendarCallback)
})
r.Route("/caldav", s.caldavRoutes)
@@ -143,6 +174,19 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
})
// The beacon's receiving end, and the number it produces. Both without a
// session, because the caller is a stranger's installation — that is the
// whole feature. Rate limited like the login routes: it is the one endpoint
// anybody on the internet can write to, and a row per request is a row per
// request whether or not the request is honest.
//
// Refused with a 404 unless this instance has been told it is the collector,
// so an ordinary installation is not a spam sink that answers 200 to
// anything.
r.Method(http.MethodPost, "/beacon",
s.rateLimit(s.loginLimiter, http.HandlerFunc(s.handleBeaconPing)))
r.Get("/beacon/count", s.handleBeaconCount)
// Reachable without being signed in — these are how you become signed in.
r.Route("/auth", func(r chi.Router) {
r.Get("/setup", s.handleSetupState)
@@ -181,11 +225,7 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Put("/nav-order", s.handleSetNavOrder)
r.Post("/logout", s.handleLogout)
r.Post("/password/change", s.handleChangePassword)
r.Post("/totp/setup", s.handleTOTPSetup)
r.Post("/totp/confirm", s.handleTOTPConfirm)
r.Post("/totp/disable", s.handleTOTPDisable)
r.Get("/recovery-codes", s.handleRecoveryCodesCount)
r.Post("/recovery-codes", s.handleRecoveryCodesRegenerate)
// Behind requireSession, like the API tokens and for the same
// reason: a leaked token must not be able to end the sessions of
@@ -195,6 +235,23 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Get("/sessions", s.handleListSessions)
r.Delete("/sessions/{sessionID}", s.handleDeleteSession)
// Turning the second factor on, and the codes that get you back
// in. Sessions only, and setup asks for the password.
//
// These sat outside this group, which meant the door opened with
// a token and closed only with a password: `disable` has always
// asked for the password, `setup` asked for nothing. A stolen
// token could therefore fetch the secret, load it into its own
// authenticator, confirm it, and keep the only copy of the
// recovery codes. The owner's next login would stop at a code
// they do not have, and they could not turn it off without
// logging in first. A theft becomes a tenancy — the same
// sentence written next door about passkeys, and the same rule.
r.Post("/totp/setup", s.handleTOTPSetup)
r.Post("/totp/confirm", s.handleTOTPConfirm)
r.Post("/totp/disable", s.handleTOTPDisable)
r.Post("/recovery-codes", s.handleRecoveryCodesRegenerate)
// Registering a key, and the list of them. Sessions only, for
// the same reason: a leaked token must not be able to add its
// own way in — that would turn a theft into a tenancy.
@@ -240,6 +297,9 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Get("/key", s.handlePushKey)
r.Post("/subscribe", s.handleSubscribePush)
r.Post("/unsubscribe", s.handleUnsubscribePush)
// One notification, to this account's own devices, reporting what
// each of them did with it. The answer to "der kommer ikke nogen".
r.Post("/test", s.handlePushTest)
})
r.Route("/attachments/{attachmentID}", func(r chi.Router) {
@@ -288,6 +348,13 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Get("/{noteID}", s.handleGetNote)
r.Patch("/{noteID}", s.handleUpdateNote)
r.Delete("/{noteID}", s.handleDeleteNote)
// A picture pasted or dropped into a note. The import could bring
// files in and nothing else could, which made the most ordinary act
// in a notes app — paste a screenshot — do nothing at all.
r.Post("/{noteID}/attachments", s.handleUploadNoteAttachment)
// Mange på én gang. Efter en import af tolv hundrede noter er det at
// vælge halvtreds det almindelige tilfælde, ikke det yderste.
r.Post("/bulk", s.handleNoteBulk)
})
r.Route("/mailboxes", func(r chi.Router) {
@@ -316,6 +383,25 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
})
})
// A calendar read from somewhere else, laid under verdande's own dated
// tasks. Read-only, so there is no write here to gate: the events are a
// copy of somebody else's calendar, and the only thing a person decides
// is which of them to look at.
r.Route("/calendar", func(r chi.Router) {
r.Get("/", s.handleGetCalendar)
r.Put("/calendars", s.handleSetCalendars)
r.Delete("/", s.handleDisconnectCalendar)
r.Post("/authorize", s.handleCalendarAuthorize)
r.Post("/sync", s.handleCalendarSyncNow)
r.Get("/events", s.handleCalendarEvents)
// Abonnementer: en kalender, man kun har en adresse til. Flere pr.
// person, hvor Google er én — det er adressen, der er identiteten,
// hvor tokenet er det for OAuth.
r.Post("/subscriptions", s.handleSubscribeCalendar)
r.Delete("/subscriptions/{id}", s.handleUnsubscribeCalendar)
})
r.Route("/import", func(r chi.Router) {
r.Post("/todoist", s.handleImportTodoist)
r.Post("/csv", s.handleImportCSV)
@@ -365,6 +451,7 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
r.Group(func(r chi.Router) {
r.Use(s.requireProject(store.RoleEditor))
r.Post("/sections", s.handleCreateSection)
r.Post("/sections/reorder", s.handleReorderSections)
})
r.Group(func(r chi.Router) {
@@ -390,6 +477,11 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
// `:latest` on the way, which is what makes a restart an update.
r.Get("/panel", s.handlePanelStatus)
r.Post("/panel/restart", s.handleRestartFromPanel)
// Who this instance reports to, and whether it does. An
// administrator's decision, not a person's: there is one
// installation and it either phones home or it does not.
r.Get("/beacon/settings", s.handleBeaconStatus)
r.Put("/beacon/settings", s.handleSetBeacon)
// A backup file is a complete copy of the database, so this whole
// group is sessions-only as well as administrators-only: a leaked
// token must not be able to download everybody's data.
+40
View File
@@ -6,6 +6,7 @@ import (
"io"
"mime/multipart"
"net/http"
"net/url"
"strings"
"testing"
"time"
@@ -1777,3 +1778,42 @@ func TestQuickAddSectionIDIsCheckedAgainstTheProject(t *testing.T) {
t.Errorf("content = %v", moved["content"])
}
}
// A word inside a word, on the task side.
//
// The notes search grew this first — a note holding `keepInventory` could not be
// found by typing "keep inventory" — and a task is the same sentence with a
// different table behind it. `SpeedAdmin` is one token, so "speed admin" asks for
// two words that are nowhere.
//
// Having it in one of the two searches would be worse than having it in neither:
// "it finds it in my notes but not in my tasks" is the next report.
func TestATaskIsFoundByAWordInsideAWord(t *testing.T) {
ts := newTestServer(t)
ts.bootstrap(t)
ts.do(t, "POST", "/api/v1/tasks", map[string]any{
"content": "Opret kodeord til Kulturskolen (SpeedAdmin)",
})
ts.do(t, "POST", "/api/v1/tasks", map[string]any{"content": "Køb kaffe"})
for _, tc := range []struct {
query string
want int
}{
{"speed admin", 1},
{"admin", 1},
{"speedadmin", 1},
{"kulturskolen", 1},
{"vandhane", 0},
} {
resp, body := ts.do(t, "GET", "/api/v1/search?q="+url.QueryEscape(tc.query), nil)
if resp.StatusCode != http.StatusOK {
t.Fatalf("%q: status %d", tc.query, resp.StatusCode)
}
tasks, _ := body["tasks"].([]any)
if len(tasks) != tc.want {
t.Errorf("%q found %d tasks, want %d", tc.query, len(tasks), tc.want)
}
}
}
+17
View File
@@ -20,7 +20,24 @@ type totpSetupResponse struct {
}
func (s *Server) handleTOTPSetup(w http.ResponseWriter, r *http.Request) {
// The password, the same as disable has always asked for.
//
// Symmetry is the point. Turning the second factor on is at least as
// consequential as turning it off — it decides which device gets to say yes
// from now on — and a door that opens more easily than it closes is not a
// door. Together with the move behind requireSession, this is what stops a
// stolen credential from enrolling its own authenticator and keeping the only
// copy of the recovery codes.
var req totpDisableRequest
if err := decodeJSON(w, r, &req); err != nil {
return
}
user := userFrom(r.Context())
if err := auth.VerifyPassword(user.PasswordHash, req.Password); err != nil {
writeFieldErrors(w, map[string]string{"password": "not correct"})
return
}
if user.TOTPEnabled {
writeError(w, http.StatusConflict, CodeConflict,
"two-factor authentication is already switched on")
+177
View File
@@ -367,3 +367,180 @@ func unescape(s string) string {
r := strings.NewReplacer(`\n`, "\n", `\N`, "\n", `\,`, ",", `\;`, ";", `\\`, `\`)
return r.Replace(s)
}
// --- reading a whole calendar ------------------------------------------------------
// Event is one VEVENT, reduced to what a grid needs — the same shape the Google
// side already stores, so the two arrive in the database identically.
//
// Days at both ends, inclusive, because that is the question a grid asks: which
// cells does this cover. StartsAt keeps the stamp as the file wrote it, offset and
// all: it is what the chip shows, and reparsing it here would ask *this process*
// what time zone it is in.
type Event struct {
UID string
Summary string
Location string
URL string
AllDay bool
StartsAt string // RFC3339, empty for an all-day event
EndsAt string
StartDay string // YYYY-MM-DD
EndDay string
}
// Subscribed is what an .ics file somebody subscribed to says.
//
// Its own name rather than `Calendar`: that one is the *writing* side — what
// verdande publishes as a feed — and the two are different documents that happen
// to share a format. One name for both would be a promise that a round trip
// through this package is lossless, which it is not and does not need to be.
type Subscribed struct {
Name string
TimeZone string
Events []Event
}
// ParseCalendar reads an .ics file that somebody subscribed to.
//
// Forgiving in the same way ParseVTODO is, and for a sharper reason: this file was
// written by somebody else's software and is fetched on a schedule with nobody
// watching. One property this does not understand must not cost the other four
// hundred events in the file — so anything unrecognised is skipped, and an event
// without the two things a grid cannot do without (a uid and a start) is dropped
// rather than stored half-made.
//
// # What it deliberately does not do
//
// **Recurrence is not expanded.** RRULE in a subscribed calendar means "and then
// every Tuesday until 2031", and expanding it here would mean writing a year of
// rows per rule and re-deciding what happens when the file changes. A recurring
// event is stored as its first occurrence, which is honest — it is what the file
// says on its own terms — and the day it matters, the answer is an expander shared
// with `internal/recurrence`, not a second one written here.
//
// **Time zones are not resolved.** A DTSTART with TZID=Europe/Copenhagen is stored
// with the offset the file carries or, when it carries none, as a local stamp. The
// alternative is shipping a tz database and deciding what to do when the file
// names a zone this build has never heard of. The day that matters, it is one
// place.
func ParseCalendar(body string) Subscribed {
var cal Subscribed
var current *Event
depth := 0
for _, line := range unfold(body) {
name, params, value := splitLine(line)
upper := strings.ToUpper(name)
switch upper {
case "BEGIN":
if strings.EqualFold(value, "VEVENT") {
current = &Event{}
}
depth++
continue
case "END":
depth--
if strings.EqualFold(value, "VEVENT") && current != nil {
if current.UID != "" && current.StartDay != "" {
cal.Events = append(cal.Events, *current)
}
current = nil
}
continue
}
if current == nil {
// Calendar-level properties. X-WR-CALNAME is what every publisher uses
// to say what the calendar is called; there is no standard property for
// it, which is why a non-standard one is read here.
switch upper {
case "X-WR-CALNAME":
cal.Name = unescape(value)
case "X-WR-TIMEZONE":
cal.TimeZone = value
}
continue
}
switch upper {
case "UID":
current.UID = value
case "SUMMARY":
current.Summary = unescape(value)
case "LOCATION":
current.Location = unescape(value)
case "URL":
current.URL = value
case "DTSTART":
current.AllDay = isDateOnly(params, value)
current.StartDay, current.StartsAt = readStamp(params, value)
case "DTEND":
day, at := readStamp(params, value)
current.EndsAt = at
// DTEND is exclusive for an all-day event: a one-day event ends on the
// next morning. Stored inclusive, or every all-day event would cover one
// cell too many — and a public holiday would eat the day after it.
if current.AllDay {
day = shiftDay(day, -1)
}
current.EndDay = day
}
}
// A missing DTEND means the event ends when it starts. Filled after the loop
// rather than inside it, because DTEND may arrive before DTSTART in a file
// nobody promised would be in order.
for i := range cal.Events {
if cal.Events[i].EndDay == "" || cal.Events[i].EndDay < cal.Events[i].StartDay {
cal.Events[i].EndDay = cal.Events[i].StartDay
}
}
return cal
}
// isDateOnly reports whether a stamp is a day rather than a moment. VALUE=DATE
// says so outright; an eight-character value says it by being one.
func isDateOnly(params, value string) bool {
if strings.Contains(strings.ToUpper(params), "VALUE=DATE") {
return true
}
return len(value) == 8 && !strings.ContainsAny(value, "T")
}
// readStamp returns the day and, for a timed event, the moment as RFC3339.
//
// Three shapes exist in the wild: 20260821 (a day), 20260821T140000Z (UTC), and
// 20260821T140000 with a TZID parameter (a local time in a named zone). The third
// is kept as written, without a zone: resolving it needs a tz database, and a
// stamp that says 14:00 is closer to the truth than one this process has moved.
func readStamp(params, value string) (day, at string) {
value = strings.TrimSpace(value)
if len(value) < 8 {
return "", ""
}
day = value[0:4] + "-" + value[4:6] + "-" + value[6:8]
if len(value) < 15 || value[8] != 'T' {
return day, ""
}
clock := value[9:11] + ":" + value[11:13] + ":" + value[13:15]
switch {
case strings.HasSuffix(value, "Z"):
return day, day + "T" + clock + "Z"
default:
// A local stamp, with or without a TZID. Written without an offset, which
// is what the interface already treats as "the clock the file said".
return day, day + "T" + clock
}
}
// shiftDay moves a YYYY-MM-DD by whole days.
func shiftDay(day string, by int) string {
t, err := time.Parse("2006-01-02", day)
if err != nil {
return day
}
return t.AddDate(0, 0, by).Format("2006-01-02")
}
+98
View File
@@ -220,3 +220,101 @@ func TestFilename(t *testing.T) {
}
}
}
// En abonnementskalender, som den ser ud fra nogen andens software.
//
// Alle fire former, der findes i naturen, i én fil: en heldagsbegivenhed, en med
// et UTC-stempel, en med TZID og lokal tid, og en over flere dage. Plus en, der
// mangler det, et gitter ikke kan undvære, og en foldet linje — begge dele
// kommer, og de kommer uden varsel.
func TestParseCalendarReadsTheFourShapesThatExist(t *testing.T) {
body := "BEGIN:VCALENDAR\r\n" +
"VERSION:2.0\r\n" +
"X-WR-CALNAME:Helligdage\r\n" +
"BEGIN:VEVENT\r\n" +
"UID:heldag@example.dk\r\n" +
"DTSTART;VALUE=DATE:20260824\r\n" +
"DTEND;VALUE=DATE:20260825\r\n" +
"SUMMARY:Grundlovsdag\r\n" +
"END:VEVENT\r\n" +
"BEGIN:VEVENT\r\n" +
"UID:utc@example.dk\r\n" +
"DTSTART:20260825T120000Z\r\n" +
"DTEND:20260825T133000Z\r\n" +
`SUMMARY:Møde med en meget lang titel\, der er foldet over to lin` + "\r\n" + " jer\r\n" +
"END:VEVENT\r\n" +
"BEGIN:VEVENT\r\n" +
"UID:lokal@example.dk\r\n" +
"DTSTART;TZID=Europe/Copenhagen:20260826T090000\r\n" +
"DTEND;TZID=Europe/Copenhagen:20260826T100000\r\n" +
"SUMMARY:Tandlæge\r\n" +
"END:VEVENT\r\n" +
"BEGIN:VEVENT\r\n" +
"UID:flerdage@example.dk\r\n" +
"DTSTART;VALUE=DATE:20260901\r\n" +
"DTEND;VALUE=DATE:20260904\r\n" +
"SUMMARY:Ferie\r\n" +
"END:VEVENT\r\n" +
"BEGIN:VEVENT\r\n" +
"SUMMARY:Uden uid og uden start\r\n" +
"END:VEVENT\r\n" +
"END:VCALENDAR\r\n"
cal := ParseCalendar(body)
if cal.Name != "Helligdage" {
t.Errorf("kalenderens navn = %q", cal.Name)
}
if len(cal.Events) != 4 {
t.Fatalf("%d begivenheder, want 4 — den femte mangler både uid og start", len(cal.Events))
}
// Heldag. DTEND er eksklusiv i filen: en endagsbegivenhed slutter næste morgen.
// Gemmes inklusivt, ellers dækker hver eneste heldagsbegivenhed én celle for
// meget — og en helligdag æder dagen efter sig.
one := cal.Events[0]
if !one.AllDay || one.StartDay != "2026-08-24" || one.EndDay != "2026-08-24" {
t.Errorf("heldag: %+v", one)
}
if one.StartsAt != "" {
t.Errorf("en heldagsbegivenhed fik et klokkeslæt: %q", one.StartsAt)
}
// UTC, og den foldede linje samlet igen.
two := cal.Events[1]
if two.StartsAt != "2026-08-25T12:00:00Z" || two.EndsAt != "2026-08-25T13:30:00Z" {
t.Errorf("utc: %+v", two)
}
if two.Summary != "Møde med en meget lang titel, der er foldet over to linjer" {
t.Errorf("den foldede linje blev ikke samlet: %q", two.Summary)
}
// Lokal tid med TZID: klokkeslættet står, som filen skrev det. At flytte det
// her ville spørge *denne* proces, hvilken tidszone den er i.
three := cal.Events[2]
if three.StartsAt != "2026-08-26T09:00:00" {
t.Errorf("lokal: %q", three.StartsAt)
}
// Flere dage, begge ender inklusive.
four := cal.Events[3]
if four.StartDay != "2026-09-01" || four.EndDay != "2026-09-03" {
t.Errorf("ferie: %s til %s", four.StartDay, four.EndDay)
}
}
// En fil, der ikke er en kalender, må ikke koste noget. Den hentes på et skema
// uden nogen, der kigger med, og en fejl her ville være en sweep, der stopper.
func TestParseCalendarSurvivesRubbish(t *testing.T) {
for _, body := range []string{
"",
"ikke en kalender overhovedet",
"BEGIN:VCALENDAR\r\nEND:VCALENDAR\r\n",
"BEGIN:VEVENT\r\nDTSTART:x\r\nEND:VEVENT\r\n",
"BEGIN:VEVENT\r\nUID:a\r\nDTSTART:2026\r\nEND:VEVENT\r\n",
} {
if got := ParseCalendar(body); len(got.Events) != 0 {
t.Errorf("%q gav %d begivenheder", body, len(got.Events))
}
}
}
+58
View File
@@ -310,3 +310,61 @@ func cut(text string) string {
}
return text
}
// Unflag takes the flag off the messages that have become tasks.
//
// This is what stops the same mail arriving twice, and it replaces a marker that
// could not be made to hold. The marker was a uid: remember the highest one seen,
// start above it next time. That is correct only if uids are what the server says
// they are, and it broke in every way such a scheme can — a fetch reply without a
// uid, a UIDVALIDITY that changed under a rebuilt mailbox, two mailbox rows
// pointing at the same folder and each keeping its own marker. Every one of those
// produced the same symptom: the same mail became a task again, and again.
//
// Unflagging moves the record out of our database and into the mailbox, where the
// question is asked. "Flagged" means "turn this into a task"; once it is a task,
// the flag has done its job and is not true any more. Nothing has to be
// remembered, so there is nothing to get out of step — a second sync, a second
// mailbox row, or a database restored from a backup all find the same answer,
// because the answer is in the mailbox.
//
// It is destructive in the small way that matters: it changes something in
// somebody's mail. That is the trade, and it is the right way round — a flag is a
// note to self that this needs doing, and it has been done.
//
// The folder is re-selected writable. Since() opens it read-only on purpose, and
// a search should not need write access just because a later step might.
func (c *Client) Unflag(uids []uint32) error {
if len(uids) == 0 {
return nil
}
if _, err := c.c.Select(c.folder, nil).Wait(); err != nil {
return fmt.Errorf("open %s for writing: %w", c.folder, err)
}
set := imap.UIDSetNum()
for _, u := range uids {
if u == 0 {
// No uid means nothing to address. The message keeps its flag and will
// be seen again — which is the failure this whole function exists to
// prevent, so it is worth saying rather than skipping quietly.
continue
}
set.AddNum(imap.UID(u))
}
if len(set) == 0 {
return nil
}
// Silent: the server has nothing to tell us that we would act on, and the
// untagged replies would only have to be collected and thrown away.
cmd := c.c.Store(set, &imap.StoreFlags{
Op: imap.StoreFlagsDel,
Silent: true,
Flags: []imap.Flag{imap.FlagFlagged},
}, nil)
if err := cmd.Close(); err != nil {
return fmt.Errorf("unflag in %s: %w", c.folder, err)
}
return nil
}
+78
View File
@@ -316,3 +316,81 @@ func TestNothingAtOrBelowTheMarkerComesBack(t *testing.T) {
}
}
}
// Den samme mail må ikke kunne blive en opgave to gange.
//
// Prøven kører hele vejen imod en rigtig server: den finder de flagede mails,
// tager flaget af dem, og spørger så forfra som en ny synkronisering ville — med
// markøren sat til nul, altså det værst tænkelige tilfælde, hvor alt, hvad vi
// selv havde husket, er væk. Der skal ikke komme noget tilbage.
//
// Det er det, markøren ikke kunne love. Den holdt kun, hvis uids var, hvad
// serveren sagde, og fejlede på et fetch-svar uden uid, på en UIDVALIDITY, der
// skiftede, og på to postkasse-rækker, der pegede samme sted og hver især førte
// deres eget regnskab. Alle tre gav det samme: mailen blev en opgave igen.
func TestAnUnflaggedMailDoesNotComeBack(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "Faktura 4711", from: "anders@example.dk", body: "Vedhæftet.", flagged: true},
{subject: "Nyhedsbrev", from: "noreply@example.com", body: "Læs mere", flagged: false},
{subject: "Kontrakt", from: "jura@example.dk", body: "Til underskrift", flagged: true},
})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(msgs) != 2 {
t.Fatalf("fandt %d flagede mails, ventede 2", len(msgs))
}
uids := make([]uint32, 0, len(msgs))
for _, m := range msgs {
uids = append(uids, m.UID)
}
if err := client.Unflag(uids); err != nil {
t.Fatalf("Unflag: %v", err)
}
// Forfra, og fra nul: selv en instans, der intet husker, må ikke finde dem igen.
again, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(again) != 0 {
t.Fatalf("%d mails kom igen efter at flaget var taget af — den første af dem er %q",
len(again), again[0].Subject)
}
}
// Et tomt kald må ikke åbne mappen skrivbar for ingenting.
func TestUnflagWithNothingToDoIsQuiet(t *testing.T) {
addr, pool := serveMailbox(t, []mail{
{subject: "Faktura", from: "a@example.dk", body: "x", flagged: true},
})
client, err := Dial(Account{Host: addr, Username: "kw", Password: "hemmelig", RootCAs: pool})
if err != nil {
t.Fatal(err)
}
defer client.Close()
if err := client.Unflag(nil); err != nil {
t.Errorf("Unflag(nil) = %v", err)
}
// Kun nuller: der er intet at adressere, og mailen skal beholde sit flag.
if err := client.Unflag([]uint32{0, 0}); err != nil {
t.Errorf("Unflag(nuller) = %v", err)
}
msgs, _, err := client.Since(0, 25)
if err != nil {
t.Fatal(err)
}
if len(msgs) != 1 {
t.Errorf("mailen mistede sit flag, uden at nogen havde adresseret den")
}
}
+69
View File
@@ -41,10 +41,19 @@ type Runner struct {
// reason as SyncGmail: it owns turning a message into a task.
SyncMailbox func(ctx context.Context, user *store.User, m *store.Mailbox) (int, error)
// SyncCalendars refreshes one person's connected calendars. Supplied the same
// way and for the same reason: the HTTP layer owns the token refresh.
SyncCalendars func(ctx context.Context, user *store.User) (int, error)
// Push delivers a notification to somebody's devices. Supplied by the HTTP
// layer, which owns the VAPID keys and the subscription list.
Push func(userID, title, body, projectID string)
// SendBeacon reports this installation to the collector, if it is switched on
// and a day has passed. Supplied by the HTTP layer for the same reason as the
// syncs: that is where the instance settings and the version live.
SendBeacon func(ctx context.Context) error
wg sync.WaitGroup
}
@@ -69,6 +78,17 @@ func (r *Runner) Start(ctx context.Context) {
// account for a to-do app. Ten minutes is fast enough for something somebody
// starred and slow enough not to look like abuse.
r.every(ctx, "mailboxes", 10*time.Minute, r.syncMailboxes)
// Fifteen minutes rather than ten. A calendar is not a mailbox: nothing here
// becomes a task, so being a quarter of an hour behind on a meeting next
// Tuesday costs nothing — and the "fetch now" button covers the one case where
// it matters, which is somebody who has just moved something and is looking at
// the grid to check. A person who has just connected does not wait for this
// either; connecting fetches the list, and the first sync is one click away.
r.every(ctx, "calendars", 15*time.Minute, r.syncCalendars)
// Checked hourly, sent at most daily — the same shape as the backup, and for
// the same reason: a container asleep at the scheduled minute would otherwise
// skip the day. SendBeacon decides whether a day has passed.
r.every(ctx, "beacon", time.Hour, r.sendBeacon)
}
func (r *Runner) Wait() { r.wg.Wait() }
@@ -103,6 +123,17 @@ func (r *Runner) every(ctx context.Context, name string, interval time.Duration,
}()
}
// sendBeacon reports that this installation exists, at most once a day.
//
// Every decision about what is sent, and why this is on by default rather than
// compulsory, lives in internal/beacon. This is only the clock.
func (r *Runner) sendBeacon(ctx context.Context) error {
if r.SendBeacon == nil {
return nil
}
return r.SendBeacon(ctx)
}
// --- reminders ------------------------------------------------------------------
// sendDueReminders delivers everything that has come due since the last pass.
@@ -322,6 +353,44 @@ func (r *Runner) syncMailboxes(ctx context.Context) error {
return nil
}
// --- calendars ------------------------------------------------------------------------
// syncCalendars refreshes everybody's connected calendars.
//
// Its own sweep rather than a branch inside the mailbox one. They share nothing
// past the word "poll": that loop asks each mailbox how far it has read and makes
// tasks, this one replaces a window of somebody else's events. Folding them
// together would mean one interval for two questions that do not want the same
// answer, and a Google that is down would count against a mailbox that is not.
//
// Nobody is waiting, so no budget of its own — the request handler has one because
// a person is watching a spinner. One account that has gone quiet costs this run
// its own timeout and nothing else: the loop moves on per person.
func (r *Runner) syncCalendars(ctx context.Context) error {
if r.SyncCalendars == nil {
return nil
}
userIDs, err := r.db.UsersWithCalendars(ctx)
if err != nil {
return err
}
for _, id := range userIDs {
user, err := r.db.UserByID(ctx, id)
if err != nil || user == nil {
continue
}
written, err := r.SyncCalendars(ctx, user)
if err != nil {
r.log.Warn("calendar sync", "err", err, "user", id)
continue
}
if written > 0 {
r.log.Info("calendar refreshed", "user", id, "events", written)
}
}
return nil
}
// --- housekeeping -----------------------------------------------------------------
func (r *Runner) emptyTrash(ctx context.Context) error {
+15 -1
View File
@@ -32,6 +32,8 @@ import (
"net/url"
"strings"
"time"
"github.com/kristianwind/verdande/internal/safedial"
)
type Subscription struct {
@@ -101,7 +103,19 @@ func Send(ctx context.Context, sub Subscription, payload Payload, vapid VAPID) e
req.Header.Set("Urgency", "normal")
req.Header.Set("Authorization", "vapid t="+token+", k="+vapid.Public)
resp, err := (&http.Client{Timeout: 15 * time.Second}).Do(req)
// safedial, not a plain client. The endpoint comes from the browser's
// subscription, which is to say from whoever is signed in, and this server
// POSTs to it on a schedule they choose — set a reminder a minute out and the
// job goes again every minute.
//
// The subscribe handler already refuses a private address, and that check is
// not enough on its own: it runs on the URL as written, and a name it approves
// can answer 127.0.0.1 the next time it is looked up. safedial checks the
// *resolved* address at connect time, which is the only moment the address
// that will actually be used is known — and it checks every redirect hop the
// same way. The parse-time check stays, because it says no while somebody is
// looking at the screen.
resp, err := safedial.Client(15 * time.Second).Do(req)
if err != nil {
return err
}
+66
View File
@@ -0,0 +1,66 @@
package push
import (
"context"
"crypto/ecdh"
"crypto/rand"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// A subscription must not be able to point the server at its own network.
//
// The endpoint is whatever the browser handed over, which is to say whatever the
// person signed in handed over, and this server POSTs to it on a schedule they
// choose. The subscribe handler refuses a private address when it is written
// down — and that check runs on the URL as written, so a name it approves can
// answer 127.0.0.1 the next time it is looked up. This is the other half: the
// address that will actually be dialled, checked at the moment of dialling.
func TestSendRefusesToReachInside(t *testing.T) {
reached := false
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
reached = true
w.WriteHeader(http.StatusCreated)
}))
defer srv.Close()
// httptest listens on 127.0.0.1, which is exactly the destination in question.
if !strings.Contains(srv.URL, "127.0.0.1") {
t.Skipf("the test server is not on loopback: %s", srv.URL)
}
err := Send(context.Background(), testSubscription(t, srv.URL), Payload{Title: "hej"}, testVAPID(t))
if err == nil {
t.Fatal("Send reached a loopback address without complaining")
}
if reached {
t.Error("the request arrived: the wall let it through and only the answer failed")
}
}
func testSubscription(t *testing.T, endpoint string) Subscription {
t.Helper()
key, err := ecdh.P256().GenerateKey(rand.Reader)
if err != nil {
t.Fatal(err)
}
auth := make([]byte, 16)
if _, err := rand.Read(auth); err != nil {
t.Fatal(err)
}
return Subscription{
Endpoint: endpoint,
P256dh: b64.EncodeToString(key.PublicKey().Bytes()),
Auth: b64.EncodeToString(auth),
}
}
func testVAPID(t *testing.T) VAPID {
t.Helper()
pub, priv, err := GenerateVAPIDKeys()
if err != nil {
t.Fatal(err)
}
return VAPID{Public: pub, Private: priv, Subject: "mailto:kristian@example.dk"}
}
+11 -1
View File
@@ -650,8 +650,18 @@ func pad2(n int) string {
// The trailing group is bounded rather than greedy: without a limit, "hver mandag
// kl 10 p1" would hand the whole tail to the recurrence parser, which would refuse
// all of it and leave the task not repeating at all.
//
// Mellemleddet må være et ord og ikke kun et tal. "hver anden tirsdag" er den måde,
// folk siger det på — recurrence-pakken har oversat "hver anden" til "hver 2."
// siden den blev skrevet — men her stoppede mønsteret efter "anden" og gav den
// sætning videre. Den er ikke en regel, så "hver anden" blev stående i titlen og
// "tirsdag" blev en forfaldsdato. Sløjfen nedenfor kan kun *fjerne* ord fra
// enden; den kan ikke hente et, mønsteret aldrig tog med.
//
// Fundet af en prøve, der brugte vendingen som selvfølgelig og opdagede, at den
// ikke var det.
var reRepeat = regexp.MustCompile(
`(?i)(?:^|\s)((?:hver|hvert|every|each)\s+(?:\d+\.?\s+)?[\p{L}]+(?:\s*(?:,|\bog\b|\band\b)\s*[\p{L}]+)*` +
`(?i)(?:^|\s)((?:hver|hvert|every|each)\s+(?:\d+\.?\s+|(?:anden|andet|tredje|fjerde|femte|sjette|other)\s+)?[\p{L}]+(?:\s*(?:,|\bog\b|\band\b)\s*[\p{L}]+)*` +
`|hverdage|ugedage|weekdays?|dagligt|daily|ugentligt|weekly|månedligt|maanedligt|monthly|årligt|aarligt|yearly|annually` +
`|den\s+\d{1,2}\.?\s+i\s+måneden|den\s+\d{1,2}\.?\s+i\s+maaneden)(?:\s|$)`)
+33
View File
@@ -603,3 +603,36 @@ func TestASectionIsWrittenWithASlash(t *testing.T) {
}
}
}
// "hver anden tirsdag" er den måde, folk siger det på.
//
// recurrence-pakken har oversat "hver anden" til "hver 2." siden den blev skrevet,
// men sætningen nåede aldrig frem til den: mønsteret her tog kun ét ord efter
// "hver", så det, der blev givet videre, var "hver anden" — ikke en regel. Så blev
// de to ord stående i titlen, og "tirsdag" blev til en forfaldsdato. Opgaven
// gentog sig ikke, og der stod ikke noget om det.
//
// Fundet af en prøve, der brugte vendingen som selvfølgelig og opdagede, at den
// ikke var det.
func TestEveryOtherWeekdayIsARepetition(t *testing.T) {
for _, tc := range []struct {
text string
rule string
left string
}{
{"vande planterne hver anden tirsdag", "FREQ=WEEKLY;BYDAY=TU;INTERVAL=2", "vande planterne"},
{"vande planterne hver 2. tirsdag", "FREQ=WEEKLY;BYDAY=TU;INTERVAL=2", "vande planterne"},
{"water the plants every other tuesday", "FREQ=WEEKLY;BYDAY=TU;INTERVAL=2", "water the plants"},
{"gå tur hver tredje søndag", "FREQ=WEEKLY;BYDAY=SU;INTERVAL=3", "gå tur"},
// Den, der virkede i forvejen, skal blive ved med at gøre det.
{"tømme opvasker hver mandag", "FREQ=WEEKLY;BYDAY=MO", "tømme opvasker"},
} {
got := Parse(tc.text, time.Date(2026, 8, 20, 9, 0, 0, 0, time.UTC), "da")
if got.Recurrence != tc.rule {
t.Errorf("%q gav reglen %q, want %q", tc.text, got.Recurrence, tc.rule)
}
if got.Content != tc.left {
t.Errorf("%q efterlod titlen %q, want %q", tc.text, got.Content, tc.left)
}
}
}
+136
View File
@@ -0,0 +1,136 @@
// Package safedial refuses to open a connection to this machine or to the
// network it sits on.
//
// The problem it solves is the same one three times: somewhere a user supplies a
// URL or a host, and the server fetches it. An AI provider's base address, a push
// endpoint, an IMAP host, a beacon collector. Each of them is a feature, and each
// of them is also a way to ask the server to reach something the caller cannot —
// the panel next door, a metadata endpoint, a database on the same bridge — and
// to report back whether it answered. On a homelab that is the whole point of the
// box being where it is.
//
// # Why the check is on the address and not on the URL
//
// Parsing the URL and refusing "localhost" or "127.0.0.1" is the obvious version
// and it does not work: a name resolves to whatever its owner says, so
// `evil.example.com` can answer 127.0.0.1 — and can answer differently the second
// time, after a parse-time check has passed. That is DNS rebinding, and the only
// place it can be caught is after resolution and before the connection, which is
// exactly where net.Dialer's Control hook runs.
//
// # What is refused
//
// Loopback, link-local (including 169.254.169.254, which is where cloud metadata
// lives), the private ranges, unique-local IPv6, and the unspecified address.
// Everything else is allowed: this is not an allowlist of the internet, it is a
// wall around the inside.
package safedial
import (
"fmt"
"net"
"net/http"
"net/url"
"syscall"
"time"
)
// ErrPrivate is returned when a connection was refused for where it pointed. It
// deliberately does not say which address it resolved to: the answer is itself
// the thing an attacker is fishing for.
type ErrPrivate struct{ Host string }
func (e *ErrPrivate) Error() string {
return fmt.Sprintf("refusing to connect to %s: it resolves to an address on this machine or its private network", e.Host)
}
// Blocked reports whether an address must not be dialled.
func Blocked(ip net.IP) bool {
return ip == nil ||
ip.IsLoopback() ||
ip.IsUnspecified() ||
ip.IsLinkLocalUnicast() ||
ip.IsLinkLocalMulticast() ||
ip.IsInterfaceLocalMulticast() ||
ip.IsMulticast() ||
ip.IsPrivate()
}
// Control is a net.Dialer Control function that refuses a private destination.
//
// It runs after the name has been resolved and before the socket is connected,
// which is the only moment where the address that will actually be used is known.
func Control(network, address string, _ syscall.RawConn) error {
host, _, err := net.SplitHostPort(address)
if err != nil {
return err
}
ip := net.ParseIP(host)
if Blocked(ip) {
return &ErrPrivate{Host: host}
}
return nil
}
// Dialer is a net.Dialer that will not reach inside.
func Dialer(timeout time.Duration) *net.Dialer {
return &net.Dialer{Timeout: timeout, Control: Control}
}
// Client is an http.Client for fetching something a user named.
//
// Redirects are followed but capped, and every hop goes through the same dialer —
// a redirect to 127.0.0.1 is the oldest way around a check that only looked at the
// URL somebody typed.
func Client(timeout time.Duration) *http.Client {
return &http.Client{
Timeout: timeout,
Transport: &http.Transport{
DialContext: Dialer(timeout).DialContext,
TLSHandshakeTimeout: timeout,
ResponseHeaderTimeout: timeout,
},
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if len(via) >= 5 {
return fmt.Errorf("too many redirects")
}
return nil
},
}
}
// AllowPrivate is the same client without the wall, for a destination the
// operator configured rather than a user — the panel, and nothing else. Named so
// that using it is a decision somebody wrote down.
func AllowPrivate(timeout time.Duration) *http.Client {
return &http.Client{Timeout: timeout}
}
// CheckURL reports why an address must not be fetched, or "" if it may be.
//
// A convenience for saying no at the moment somebody types an address, rather
// than later when a request quietly fails. It is NOT the protection — a name can
// resolve one way now and another way when it is dialled, which is why Control
// exists and runs every time. Anything this rejects is certainly wrong; anything
// it accepts is merely not obviously wrong.
func CheckURL(raw string) string {
u, err := url.Parse(raw)
if err != nil || u.Host == "" {
return "that is not a URL"
}
host := u.Hostname()
if ip := net.ParseIP(host); ip != nil && Blocked(ip) {
return "that address is on this machine or its private network"
}
// A bare name is looked up once so an obvious "localhost" is caught here. A
// failure to resolve is not an error: the host may simply not exist yet, and
// refusing to save an address because DNS was slow would be its own bug.
if ips, err := net.LookupIP(host); err == nil {
for _, ip := range ips {
if Blocked(ip) {
return "that address is on this machine or its private network"
}
}
}
return ""
}
+55
View File
@@ -0,0 +1,55 @@
package safedial
import (
"net"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestBlockedCoversTheInside(t *testing.T) {
blocked := []string{
"127.0.0.1", "::1", "0.0.0.0",
// Skyens metadata-endepunkt. Den ene adresse, enhver SSRF sigter efter.
"169.254.169.254",
"10.0.0.5", "172.16.0.1", "192.168.1.10", "fd00::1", "fe80::1",
}
for _, s := range blocked {
if !Blocked(net.ParseIP(s)) {
t.Errorf("Blocked(%s) = false", s)
}
}
for _, s := range []string{"93.184.216.34", "8.8.8.8", "2606:2800:220:1:248:1893:25c8:1946"} {
if Blocked(net.ParseIP(s)) {
t.Errorf("Blocked(%s) = true, men det er en almindelig adresse", s)
}
}
}
// En httptest-server lytter på loopback, hvilket gør den til den perfekte prøve:
// den findes, den svarer, og den må ikke kunne nås.
func TestClientRefusesLoopback(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
t.Error("forbindelsen blev oprettet; den skulle være afvist")
}))
defer srv.Close()
_, err := Client(3 * time.Second).Get(srv.URL)
if err == nil {
t.Fatal("en forespørgsel til loopback skal afvises")
}
}
func TestAllowPrivateStillReaches(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNoContent)
}))
defer srv.Close()
resp, err := AllowPrivate(3 * time.Second).Get(srv.URL)
if err != nil {
t.Fatalf("den skal kunne nå det, operatøren selv har peget på: %v", err)
}
resp.Body.Close()
}
+138
View File
@@ -0,0 +1,138 @@
package store
import (
"context"
"time"
)
// The collector's side of the beacon: what has reported in, and how many.
//
// Every method here is about counting installations, never about identifying one.
// There is no lookup by anything but the id an installation made up for itself,
// and there is nothing to join it against.
// BeaconStats is the whole of what the collector can say.
type BeaconStats struct {
Total int `json:"total"`
Active7d int `json:"active_7d"`
Active30d int `json:"active_30d"`
ByVersion map[string]int `json:"by_version"`
LastPingAt *time.Time `json:"last_ping_at,omitempty"`
}
// maxBeaconInstalls bounds the table.
//
// The receiving endpoint is unauthenticated — it has to be, since the caller is a
// stranger's installation — and it inserts a row keyed on an id the caller made
// up. Anybody can therefore mint ids and grow this table until the volume the
// database lives on is full. A ceiling turns that from a disk-fill into a number
// that is merely wrong, which is the difference between an outage and an
// annoyance. A hundred thousand is far above any plausible real count and far
// below anything that matters on disk.
const maxBeaconInstalls = 100_000
// RecordBeacon writes down that an installation is alive, and on which version.
//
// `first_seen` is kept from the original row on purpose: it is the only thing
// that distinguishes an installation that has been running for a year from one
// that appeared this morning, and it costs nothing to keep.
func (db *DB) RecordBeacon(ctx context.Context, instanceID, version string) error {
now := time.Now().Unix()
// An installation already known keeps reporting whatever the ceiling says: the
// cap must never stop counting the real ones, only stop new rows from being
// invented without limit.
var known int
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs WHERE instance_id = ?`, instanceID).Scan(&known); err != nil {
return err
}
if known == 0 {
var total int
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs`).Scan(&total); err != nil {
return err
}
if total >= maxBeaconInstalls {
// Not an error to the caller. Their installation did nothing wrong, and
// there is nothing for them to do about it.
return nil
}
}
_, err := db.ExecContext(ctx, `
INSERT INTO beacon_installs (instance_id, version, first_seen, last_seen)
VALUES (?, ?, ?, ?)
ON CONFLICT (instance_id) DO UPDATE SET
version = excluded.version,
last_seen = excluded.last_seen`,
instanceID, version, now, now)
return err
}
// BeaconCount is how many installations have reported within the window.
//
// A window rather than a running total, and that is not a detail. A total only
// ever climbs: every instance anybody ever started for ten minutes stays in it
// forever, so the number stops meaning "how many people use this" and starts
// meaning "how many people have ever tried it". The question being asked is the
// first one.
func (db *DB) BeaconCount(ctx context.Context, within time.Duration) (int, error) {
var n int
err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs WHERE last_seen >= ?`,
time.Now().Add(-within).Unix()).Scan(&n)
return n, err
}
// BeaconSummary is everything the settings page shows.
func (db *DB) BeaconSummary(ctx context.Context) (BeaconStats, error) {
out := BeaconStats{ByVersion: map[string]int{}}
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM beacon_installs`).Scan(&out.Total); err != nil {
return out, err
}
var err error
if out.Active7d, err = db.BeaconCount(ctx, 7*24*time.Hour); err != nil {
return out, err
}
if out.Active30d, err = db.BeaconCount(ctx, 30*24*time.Hour); err != nil {
return out, err
}
// Versions of the ones still alive, not of everything ever seen: an instance
// that stopped reporting two years ago should not still be counted as running
// the version it was on when it stopped.
rows, err := db.QueryContext(ctx, `
SELECT version, count(*) FROM beacon_installs
WHERE last_seen >= ? AND version <> ''
GROUP BY version`, time.Now().Add(-30*24*time.Hour).Unix())
if err != nil {
return out, err
}
defer rows.Close()
for rows.Next() {
var v string
var n int
if err := rows.Scan(&v, &n); err != nil {
return out, err
}
out.ByVersion[v] = n
}
if err := rows.Err(); err != nil {
return out, err
}
var last *int64
if err := db.QueryRowContext(ctx,
`SELECT max(last_seen) FROM beacon_installs`).Scan(&last); err != nil {
return out, err
}
if last != nil {
t := time.Unix(*last, 0)
out.LastPingAt = &t
}
return out, nil
}
+531
View File
@@ -0,0 +1,531 @@
package store
import (
"context"
"database/sql"
"errors"
"fmt"
"strings"
"time"
)
// CalendarAccount is one connected account belonging to one person.
type CalendarAccount struct {
ID string `json:"id"`
UserID string `json:"-"`
Provider string `json:"provider"`
Account string `json:"account,omitempty"`
// URL er tom for en OAuth-konto og adressen for et abonnement. Den er
// identiteten på et abonnement, som tokenet er det for Google: to rækker med
// den samme adresse for den samme person ville være den samme kalender hentet
// to gange, uden nogen måde at sige hvilken der gælder.
URL string `json:"url,omitempty"`
RefreshToken string `json:"-"` // never leaves the server
AccessToken string `json:"-"`
ExpiresAt time.Time `json:"-"`
LastSyncAt time.Time `json:"last_sync_at,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
// Calendar is one calendar inside an account, shown or not.
type Calendar struct {
ID string `json:"id"`
AccountID string `json:"-"`
RemoteID string `json:"remote_id"`
Name string `json:"name"`
Colour string `json:"colour,omitempty"`
TimeZone string `json:"time_zone,omitempty"`
Primary bool `json:"primary"`
// Writable says the connected account could write to it. Nothing does — the
// scope is read-only — and it is stored so the day something can, the interface
// knows which calendars could even be offered.
Writable bool `json:"writable"`
Shown bool `json:"shown"`
SortOrder int `json:"-"`
}
// CalendarEvent is one occurrence, in the shape a grid asks about: days at both
// ends, inclusive.
type CalendarEvent struct {
ID string `json:"id"`
CalendarID string `json:"calendar_id"`
RemoteID string `json:"-"`
Summary string `json:"summary"`
StartsAt string `json:"starts_at,omitempty"`
EndsAt string `json:"ends_at,omitempty"`
StartDay string `json:"start_day"`
EndDay string `json:"end_day"`
AllDay bool `json:"all_day"`
Location string `json:"location,omitempty"`
URL string `json:"url,omitempty"`
// CalendarName and Colour ride along with the event rather than being looked up
// per row by the caller. Same reasoning as the assignee names on a task list:
// the chip has to be drawn in the calendar's own colour, and a second request
// per event to learn it is a request per event.
CalendarName string `json:"calendar_name,omitempty"`
Colour string `json:"colour,omitempty"`
}
const calendarAccountColumns = `id, user_id, provider, account, refresh_token,
access_token, expires_at, last_sync_at, created_at`
// Med adressen. Skrivningen tilføjer den selv, fordi indsættelsen navngiver
// søjlerne; læsningen skal bede om den.
const calendarAccountRead = calendarAccountColumns + `, url`
// CalendarAccountFor returns a person's connection to one provider, or nil.
func (db *DB) CalendarAccountFor(ctx context.Context, userID, provider string) (*CalendarAccount, error) {
// `url = ''`: en OAuth-konto. Uden det ville et abonnement hos den samme
// udbyder kunne svare på spørgsmålet "hvad er personen forbundet til med
// Google", og et abonnement har hverken token eller konto at svare med.
row := db.QueryRowContext(ctx, `SELECT `+calendarAccountRead+`
FROM calendar_accounts WHERE user_id = ? AND provider = ? AND url = ''`, userID, provider)
a, err := db.scanCalendarAccount(row)
if err == sql.ErrNoRows {
return nil, nil
}
if err != nil {
return nil, err
}
return &a, nil
}
// SaveCalendarAccount writes one, sealing the two columns that must not travel
// with a backup. Called by hand rather than by the settings tables' own seal,
// which does not reach this table.
func (db *DB) SaveCalendarAccount(ctx context.Context, a *CalendarAccount) error {
if a.ID == "" {
a.ID = NewID()
}
if a.Provider == "" {
a.Provider = "google"
}
if a.CreatedAt.IsZero() {
a.CreatedAt = time.Now()
}
refresh, err := db.sealValue(a.RefreshToken)
if err != nil {
return fmt.Errorf("seal refresh_token: %w", err)
}
access, err := db.sealValue(a.AccessToken)
if err != nil {
return fmt.Errorf("seal access_token: %w", err)
}
// The conflict is on the pair, not on the id: a reconnect builds a fresh
// struct with no id in it, and matching on the id would insert a second row
// that the unique index then refuses — which reads to the person as "connecting
// failed" when what happened is that it already had.
// `WHERE url = ''` fordi indekset er delvist siden 0026.
//
// En OAuth-konto er én pr. person og udbyder — det er tokenet, der er
// identiteten — mens et abonnement er én pr. adresse, og man har flere. De to
// bor i samme tabel, fordi et abonnement *er* en kalender med begivenheder i,
// og den halvdel er den samme. Målet for konflikten skal sige, hvilken af de to
// slags rækker der tales om, ellers ved SQLite det ikke.
_, err = db.ExecContext(ctx, `
INSERT INTO calendar_accounts (`+calendarAccountColumns+`, url)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (user_id, provider) WHERE url = '' DO UPDATE SET
account = excluded.account,
refresh_token = excluded.refresh_token,
access_token = excluded.access_token,
expires_at = excluded.expires_at,
last_sync_at = excluded.last_sync_at`,
a.ID, a.UserID, a.Provider, a.Account, refresh, access,
unixOrZero(a.ExpiresAt), unixOrZero(a.LastSyncAt), a.CreatedAt.Unix(), a.URL)
if err != nil {
return err
}
if a.URL != "" {
return nil
}
// The row that is actually there wins. On a reconnect the insert above updated
// an existing row, so the id in hand is one nothing points at — and the
// calendars are hung off the id, so writing them against it would attach them
// to nothing.
return db.QueryRowContext(ctx,
`SELECT id FROM calendar_accounts WHERE user_id = ? AND provider = ? AND url = ''`,
a.UserID, a.Provider).Scan(&a.ID)
}
// MarkCalendarSynced records how far a run got. Written separately from the rest
// so a sync never rewrites the credentials it is holding in memory.
func (db *DB) MarkCalendarSynced(ctx context.Context, accountID string, at time.Time) error {
_, err := db.ExecContext(ctx,
`UPDATE calendar_accounts SET last_sync_at = ? WHERE id = ?`, at.Unix(), accountID)
return err
}
// DeleteCalendarAccount disconnects one. The calendars and their events go with it
// — unlike the tasks a mailbox made, none of this is the person's own work; it is
// a copy of somebody else's calendar and it has no meaning without the account.
func (db *DB) DeleteCalendarAccount(ctx context.Context, userID, id string) error {
_, err := db.ExecContext(ctx,
`DELETE FROM calendar_accounts WHERE id = ? AND user_id = ?`, id, userID)
return err
}
// UsersWithCalendars returns everybody the sweep has to visit.
func (db *DB) UsersWithCalendars(ctx context.Context) ([]string, error) {
rows, err := db.QueryContext(ctx, `SELECT DISTINCT user_id FROM calendar_accounts`)
if err != nil {
return nil, err
}
defer rows.Close()
var ids []string
for rows.Next() {
var id string
if err := rows.Scan(&id); err != nil {
return nil, err
}
ids = append(ids, id)
}
return ids, rows.Err()
}
// Calendars returns what one account holds, in the order the provider listed them.
func (db *DB) Calendars(ctx context.Context, accountID string) ([]Calendar, error) {
rows, err := db.QueryContext(ctx, `
SELECT id, account_id, remote_id, name, colour, time_zone, primary_one,
writable, shown, sort_order
FROM calendars WHERE account_id = ? ORDER BY sort_order, name`, accountID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Calendar
for rows.Next() {
var c Calendar
if err := rows.Scan(&c.ID, &c.AccountID, &c.RemoteID, &c.Name, &c.Colour,
&c.TimeZone, &c.Primary, &c.Writable, &c.Shown, &c.SortOrder); err != nil {
return nil, err
}
out = append(out, c)
}
return out, rows.Err()
}
// ReplaceCalendars writes the list the provider just gave, keeping what the person
// chose.
//
// The choice survives because it is not in the list: Google says which calendars
// exist and what they are called, and only verdande knows which of them somebody
// wants to look at. Merged rather than deleted and rewritten, so ticking a calendar
// on does not come undone the next time the sweep runs.
//
// A calendar that has disappeared from the account is removed, and its events go
// with it — a calendar somebody was unshared from should stop being drawn, not
// stay behind as a frozen copy of what it held on the last good day.
func (db *DB) ReplaceCalendars(ctx context.Context, accountID string, list []Calendar) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
keep := map[string]bool{}
for i, c := range list {
keep[c.RemoteID] = true
// The insert cannot carry `shown`: on conflict it would overwrite the
// choice with the default. A new calendar starts hidden — an account
// with a dozen shared calendars in it should not fill somebody's grid
// the moment they connect it.
if _, err := tx.ExecContext(ctx, `
INSERT INTO calendars
(id, account_id, remote_id, name, colour, time_zone, primary_one,
writable, shown, sort_order)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (account_id, remote_id) DO UPDATE SET
name = excluded.name,
colour = excluded.colour,
time_zone = excluded.time_zone,
primary_one = excluded.primary_one,
writable = excluded.writable,
sort_order = excluded.sort_order`,
NewID(), accountID, c.RemoteID, c.Name, c.Colour, c.TimeZone,
c.Primary, c.Writable, c.Shown, i); err != nil {
return err
}
}
rows, err := tx.QueryContext(ctx,
`SELECT id, remote_id FROM calendars WHERE account_id = ?`, accountID)
if err != nil {
return err
}
var gone []string
for rows.Next() {
var id, remote string
if err := rows.Scan(&id, &remote); err != nil {
rows.Close()
return err
}
if !keep[remote] {
gone = append(gone, id)
}
}
// Closed before the next statement runs on this transaction. An open result
// set with a query behind it is the deadlock the pool comment in store.go
// exists to make slow rather than fatal.
rows.Close()
if err := rows.Err(); err != nil {
return err
}
for _, id := range gone {
if _, err := tx.ExecContext(ctx, `DELETE FROM calendars WHERE id = ?`, id); err != nil {
return err
}
}
return nil
})
}
// ShowCalendars sets which of an account's calendars are drawn.
//
// The whole set in one write rather than one flip at a time, for the reason the
// projects' order is written the same way: a person has a handful of calendars, the
// question is "these ones", and a list that can only be built by several requests
// can land half applied.
func (db *DB) ShowCalendars(ctx context.Context, accountID string, shown []string) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx,
`UPDATE calendars SET shown = 0 WHERE account_id = ?`, accountID); err != nil {
return err
}
for _, id := range shown {
// Scoped in the SQL rather than checked first: an id read in one step
// and written in the next is a window, and the account is the whole of
// the permission here.
if _, err := tx.ExecContext(ctx,
`UPDATE calendars SET shown = 1 WHERE id = ? AND account_id = ?`,
id, accountID); err != nil {
return err
}
}
// Events of a calendar nobody is looking at are dead weight in a table the
// grid queries. They come back with the next sync if it is turned on again.
_, err := tx.ExecContext(ctx, `
DELETE FROM calendar_events WHERE calendar_id IN
(SELECT id FROM calendars WHERE account_id = ? AND shown = 0)`, accountID)
return err
})
}
// ReplaceCalendarEvents swaps one calendar's events inside a window.
//
// Replaced rather than merged, and no sync token. Google offers incremental sync,
// and it would save bytes verdande is not short of while costing the one thing it
// cannot afford: a token that has to be right, forever, for the grid not to be
// quietly wrong. A window that is deleted and rewritten cannot drift — an event
// somebody moved out of the window disappears because it is not in the answer, not
// because verdande remembered to delete it.
//
// Bounded by the window at both ends, so a sync of September does not throw away
// August.
func (db *DB) ReplaceCalendarEvents(ctx context.Context, calendarID, from, to string, events []CalendarEvent) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, `
DELETE FROM calendar_events
WHERE calendar_id = ? AND end_day >= ? AND start_day <= ?`,
calendarID, from, to); err != nil {
return err
}
for _, e := range events {
if _, err := tx.ExecContext(ctx, `
INSERT INTO calendar_events
(id, calendar_id, remote_id, summary, starts_at, ends_at,
start_day, end_day, all_day, location, url)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
NewID(), calendarID, e.RemoteID, e.Summary, e.StartsAt, e.EndsAt,
e.StartDay, e.EndDay, e.AllDay, e.Location, e.URL); err != nil {
return err
}
}
return nil
})
}
// CalendarEvents returns everything one person can see between two days,
// inclusive at both ends.
//
// Scoped to the caller in the join rather than checked afterwards: the events of
// somebody else's calendar are somebody else's, and a filter applied after the read
// is a filter that can be forgotten.
func (db *DB) CalendarEvents(ctx context.Context, userID, from, to string) ([]CalendarEvent, error) {
rows, err := db.QueryContext(ctx, `
SELECT e.id, e.calendar_id, e.remote_id, e.summary, e.starts_at, e.ends_at,
e.start_day, e.end_day, e.all_day, e.location, e.url, c.name, c.colour
FROM calendar_events e
JOIN calendars c ON c.id = e.calendar_id
JOIN calendar_accounts a ON a.id = c.account_id
WHERE a.user_id = ? AND c.shown = 1 AND e.end_day >= ? AND e.start_day <= ?
ORDER BY e.start_day, e.all_day DESC, e.starts_at`, userID, from, to)
if err != nil {
return nil, err
}
defer rows.Close()
var out []CalendarEvent
for rows.Next() {
var e CalendarEvent
if err := rows.Scan(&e.ID, &e.CalendarID, &e.RemoteID, &e.Summary, &e.StartsAt,
&e.EndsAt, &e.StartDay, &e.EndDay, &e.AllDay, &e.Location, &e.URL,
&e.CalendarName, &e.Colour); err != nil {
return nil, err
}
out = append(out, e)
}
return out, rows.Err()
}
func (db *DB) scanCalendarAccount(row scanner) (CalendarAccount, error) {
var a CalendarAccount
var expires, lastSync, created int64
err := row.Scan(&a.ID, &a.UserID, &a.Provider, &a.Account, &a.RefreshToken,
&a.AccessToken, &expires, &lastSync, &created, &a.URL)
if err != nil {
return a, err
}
for _, field := range []*string{&a.RefreshToken, &a.AccessToken} {
plain, err := db.unsealValue(*field)
if err != nil {
return a, err
}
*field = plain
}
a.ExpiresAt = timeOrZero(expires)
a.LastSyncAt = timeOrZero(lastSync)
a.CreatedAt = time.Unix(created, 0)
return a, nil
}
// --- abonnementer ---------------------------------------------------------------
// SubscribeCalendar adds a calendar somebody only has an address to.
//
// One row in `calendar_accounts` with a url and no tokens, and one row in
// `calendars` hung off it — because a subscription *is* one calendar, where an
// OAuth account is a list of them somebody picks from. Written together so a
// subscription is never a row with nothing under it: the sweep looks for accounts,
// and one without a calendar would be fetched for ever and stored nowhere.
//
// Shown from the start. Ticking a box after adding an address is a second step for
// a decision the person already made by adding it.
func (db *DB) SubscribeCalendar(ctx context.Context, userID, url, name string) (*CalendarAccount, error) {
account := &CalendarAccount{
ID: NewID(),
UserID: userID,
Provider: "ics",
Account: name,
URL: url,
CreatedAt: time.Now(),
}
err := db.Tx(ctx, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, `
INSERT INTO calendar_accounts (id, user_id, provider, account, url, created_at)
VALUES (?, ?, 'ics', ?, ?, ?)`,
account.ID, userID, name, url, account.CreatedAt.Unix()); err != nil {
return err
}
_, err := tx.ExecContext(ctx, `
INSERT INTO calendars (id, account_id, remote_id, name, shown)
VALUES (?, ?, ?, ?, 1)`,
NewID(), account.ID, url, name)
return err
})
if isUniqueViolation(err) {
return nil, ErrDuplicate
}
if err != nil {
return nil, err
}
return account, nil
}
// Subscriptions lists the calendars a person subscribes to by address.
func (db *DB) Subscriptions(ctx context.Context, userID string) ([]CalendarAccount, error) {
rows, err := db.QueryContext(ctx, `SELECT `+calendarAccountRead+`
FROM calendar_accounts WHERE user_id = ? AND url <> ''
ORDER BY created_at`, userID)
if err != nil {
return nil, err
}
defer rows.Close()
out := []CalendarAccount{}
for rows.Next() {
a, err := db.scanCalendarAccount(rows)
if err != nil {
return nil, err
}
out = append(out, a)
}
return out, rows.Err()
}
// AllSubscriptions is every subscription on the instance, for the sweep.
//
// The sweep runs for the server rather than for a session, so it cannot ask "whose
// is this" the way a handler can — it is handed the user id along with the row and
// writes against that.
func (db *DB) AllSubscriptions(ctx context.Context) ([]CalendarAccount, error) {
rows, err := db.QueryContext(ctx, `SELECT `+calendarAccountRead+`
FROM calendar_accounts WHERE url <> '' ORDER BY last_sync_at`)
if err != nil {
return nil, err
}
defer rows.Close()
out := []CalendarAccount{}
for rows.Next() {
a, err := db.scanCalendarAccount(rows)
if err != nil {
return nil, err
}
out = append(out, a)
}
return out, rows.Err()
}
// RenameSubscribedCalendar writes the name the file calls itself.
//
// A subscription is added with the address as its name, because that is all there
// is to go on before it has been fetched once. The file usually says better —
// X-WR-CALNAME is what every publisher uses — and taking it means the list says
// "Danske helligdage" instead of a URL nobody reads.
//
// Only when the person has not renamed it themselves: a name they typed is an
// instruction, and a name from the file is a guess.
func (db *DB) RenameSubscribedCalendar(ctx context.Context, accountID, name string) error {
if strings.TrimSpace(name) == "" {
return nil
}
return db.Tx(ctx, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, `
UPDATE calendar_accounts SET account = ?
WHERE id = ? AND (account = '' OR account = url)`, name, accountID); err != nil {
return err
}
_, err := tx.ExecContext(ctx, `
UPDATE calendars SET name = ?
WHERE account_id = ? AND (name = '' OR name = remote_id)`, name, accountID)
return err
})
}
// SubscribedCalendarID is the single calendar row under a subscription.
func (db *DB) SubscribedCalendarID(ctx context.Context, accountID string) (string, error) {
var id string
err := db.QueryRowContext(ctx,
`SELECT id FROM calendars WHERE account_id = ? LIMIT 1`, accountID).Scan(&id)
if errors.Is(err, sql.ErrNoRows) {
return "", ErrNotFound
}
return id, err
}
+358
View File
@@ -0,0 +1,358 @@
package store
import (
"context"
"strings"
"testing"
"time"
)
func connectedAccount(t *testing.T, db *DB, userID string) *CalendarAccount {
t.Helper()
a := &CalendarAccount{
UserID: userID, Provider: "google", Account: "kw@nolimit.dk",
RefreshToken: "1//0-a-real-looking-refresh-token", AccessToken: "ya29.access",
ExpiresAt: time.Now().Add(time.Hour),
}
if err := db.SaveCalendarAccount(context.Background(), a); err != nil {
t.Fatal(err)
}
return a
}
// The same claim the mailbox tokens have to meet: what is written must not be
// readable in the file a backup copies.
func TestCalendarTokensAreNotStoredInTheClear(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
var refresh, access string
if err := db.QueryRowContext(ctx,
`SELECT refresh_token, access_token FROM calendar_accounts WHERE id = ?`,
a.ID).Scan(&refresh, &access); err != nil {
t.Fatal(err)
}
if strings.Contains(refresh, "real-looking") || strings.Contains(access, "ya29") {
t.Fatalf("a token is in the row as written: %q %q", refresh, access)
}
// The address stays readable, which is the reason for sealing named columns
// rather than the whole row.
var account string
if err := db.QueryRowContext(ctx,
`SELECT account FROM calendar_accounts WHERE id = ?`, a.ID).Scan(&account); err != nil {
t.Fatal(err)
}
if account != "kw@nolimit.dk" {
t.Errorf("the address was sealed too: %q", account)
}
back, err := db.CalendarAccountFor(ctx, userID, "google")
if err != nil {
t.Fatal(err)
}
if back.RefreshToken != a.RefreshToken || back.AccessToken != a.AccessToken {
t.Errorf("the tokens did not survive the round trip: %+v", back)
}
}
// Connecting again must replace what was there, not fail. It is the same OAuth
// flow run a second time, and the person's word for it is "connect", not "add".
func TestConnectingAgainReplacesTheAccount(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
first := connectedAccount(t, db, userID)
second := &CalendarAccount{
UserID: userID, Provider: "google", Account: "kw@nolimit.dk",
RefreshToken: "1//0-fresh", AccessToken: "ya29.fresh",
}
if err := db.SaveCalendarAccount(ctx, second); err != nil {
t.Fatalf("connecting a second time failed: %v", err)
}
// And the id in hand is the row that is actually there. The calendars hang off
// it, so an id nothing points at would attach them to nothing.
if second.ID != first.ID {
t.Errorf("the second connect came back with id %q, want the existing %q",
second.ID, first.ID)
}
back, err := db.CalendarAccountFor(ctx, userID, "google")
if err != nil {
t.Fatal(err)
}
if back.RefreshToken != "1//0-fresh" {
t.Errorf("the token came back as %q", back.RefreshToken)
}
}
// Google says which calendars exist; only verdande knows which ones somebody wants
// to look at. A refresh that came back with the same list must not undo the choice.
func TestRefreshingTheListKeepsWhatWasChosen(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
list := []Calendar{
{RemoteID: "primary", Name: "kw@nolimit.dk", Colour: "#3f51b5", Primary: true},
{RemoteID: "fam", Name: "Familien", Colour: "#0b8043"},
{RemoteID: "old", Name: "Et delt"},
}
if err := db.ReplaceCalendars(ctx, a.ID, list); err != nil {
t.Fatal(err)
}
// Nothing is shown to begin with: an account with a dozen shared calendars in
// it should not fill somebody's grid the moment they connect it.
stored, err := db.Calendars(ctx, a.ID)
if err != nil {
t.Fatal(err)
}
if len(stored) != 3 {
t.Fatalf("got %d calendars, want 3", len(stored))
}
for _, c := range stored {
if c.Shown {
t.Errorf("%q was shown without anybody asking for it", c.Name)
}
}
var famID string
for _, c := range stored {
if c.RemoteID == "fam" {
famID = c.ID
}
}
if err := db.ShowCalendars(ctx, a.ID, []string{famID}); err != nil {
t.Fatal(err)
}
// The sweep runs again, and one calendar has been renamed and one has gone.
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{
{RemoteID: "primary", Name: "kw@nolimit.dk", Colour: "#3f51b5", Primary: true},
{RemoteID: "fam", Name: "Familien og hunden", Colour: "#0b8043"},
}); err != nil {
t.Fatal(err)
}
stored, err = db.Calendars(ctx, a.ID)
if err != nil {
t.Fatal(err)
}
if len(stored) != 2 {
t.Fatalf("got %d calendars; one that has left the account should stop being drawn",
len(stored))
}
for _, c := range stored {
switch c.RemoteID {
case "fam":
if !c.Shown {
t.Error("the choice was undone by a refresh of the list")
}
if c.Name != "Familien og hunden" {
t.Errorf("the new name did not arrive: %q", c.Name)
}
if c.ID != famID {
t.Error("the row was rebuilt rather than updated; its id is what was chosen")
}
case "primary":
if c.Shown {
t.Error("a calendar nobody chose became shown")
}
}
}
}
// A window is replaced, not merged. An event moved out of it has to disappear
// because it is absent from the answer — and the months on either side must not go
// with it.
func TestReplacingAWindowLeavesTheMonthsAroundItAlone(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "primary", Name: "Min"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
cal := stored[0]
if err := db.ShowCalendars(ctx, a.ID, []string{cal.ID}); err != nil {
t.Fatal(err)
}
july := CalendarEvent{RemoteID: "j", Summary: "Juli", StartDay: "2026-07-15", EndDay: "2026-07-15"}
august := CalendarEvent{RemoteID: "a", Summary: "August", StartDay: "2026-08-15", EndDay: "2026-08-15"}
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-07-01", "2026-07-31",
[]CalendarEvent{july}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{august}); err != nil {
t.Fatal(err)
}
got, err := db.CalendarEvents(ctx, userID, "2026-07-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 2 {
t.Fatalf("got %d events, want both months: %+v", len(got), got)
}
// August is synced again and the meeting has been cancelled. July stays.
if err := db.ReplaceCalendarEvents(ctx, cal.ID, "2026-08-01", "2026-08-31", nil); err != nil {
t.Fatal(err)
}
got, err = db.CalendarEvents(ctx, userID, "2026-07-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].Summary != "Juli" {
t.Fatalf("got %+v; syncing August threw away July", got)
}
}
// The grid asks "which cells does this cover", so an event that straddles the edge
// of the question has to come back.
func TestAnEventThatStraddlesTheWindowIsFound(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min", Colour: "#0b8043"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-09-30",
[]CalendarEvent{
{RemoteID: "trip", Summary: "Aarhus", StartDay: "2026-08-28", EndDay: "2026-09-02", AllDay: true},
}); err != nil {
t.Fatal(err)
}
// A week that contains neither end of it.
got, err := db.CalendarEvents(ctx, userID, "2026-08-31", "2026-09-01")
if err != nil {
t.Fatal(err)
}
if len(got) != 1 {
t.Fatalf("got %+v; an event covering the whole week was not found", got)
}
// The calendar's own colour rides along, or the chip has to ask for it per row.
if got[0].Colour != "#0b8043" || got[0].CalendarName != "Min" {
t.Errorf("the calendar did not come with the event: %+v", got[0])
}
}
// Somebody else's calendar is somebody else's.
func TestOnlyTheOwnersEventsComeBack(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
other := &User{Email: "andreas@example.dk", Name: "Andreas"}
if err := db.CreateUser(ctx, other, "Indbakke"); err != nil {
t.Fatal(err)
}
theirs := &CalendarAccount{UserID: other.ID, Provider: "google", RefreshToken: "r"}
if err := db.SaveCalendarAccount(ctx, theirs); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendars(ctx, theirs.ID, []Calendar{{RemoteID: "c", Name: "Deres"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, theirs.ID)
if err := db.ShowCalendars(ctx, theirs.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{{RemoteID: "x", Summary: "Privat", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
t.Fatal(err)
}
got, err := db.CalendarEvents(ctx, userID, "2026-08-01", "2026-08-31")
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Fatalf("somebody else's calendar came back: %+v", got)
}
}
// A calendar turned off stops being drawn, and stops costing the grid anything.
func TestTurningACalendarOffRemovesItsEvents(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{{RemoteID: "x", Summary: "Møde", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
t.Fatal(err)
}
if err := db.ShowCalendars(ctx, a.ID, nil); err != nil {
t.Fatal(err)
}
var count int
if err := db.QueryRowContext(ctx, `SELECT count(*) FROM calendar_events`).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 0 {
t.Errorf("%d events of a calendar nobody is looking at are still in the table", count)
}
}
// Disconnecting takes the calendars and the events with it. Unlike the tasks a
// mailbox made, none of this is the person's own work — it is a copy of somebody
// else's calendar and it means nothing without the account.
func TestDisconnectingTakesEverythingWithIt(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
a := connectedAccount(t, db, userID)
if err := db.ReplaceCalendars(ctx, a.ID, []Calendar{{RemoteID: "c", Name: "Min"}}); err != nil {
t.Fatal(err)
}
stored, _ := db.Calendars(ctx, a.ID)
if err := db.ShowCalendars(ctx, a.ID, []string{stored[0].ID}); err != nil {
t.Fatal(err)
}
if err := db.ReplaceCalendarEvents(ctx, stored[0].ID, "2026-08-01", "2026-08-31",
[]CalendarEvent{{RemoteID: "x", Summary: "Møde", StartDay: "2026-08-10", EndDay: "2026-08-10"}}); err != nil {
t.Fatal(err)
}
if err := db.DeleteCalendarAccount(ctx, userID, a.ID); err != nil {
t.Fatal(err)
}
for _, table := range []string{"calendar_accounts", "calendars", "calendar_events"} {
var count int
if err := db.QueryRowContext(ctx, `SELECT count(*) FROM `+table).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 0 {
t.Errorf("%s still holds %d rows after disconnecting", table, count)
}
}
}
// The sweep finds people by the presence of an account, so it has to keep working
// now that the credential in it is ciphertext.
func TestTheSweepFindsAConnectedCalendar(t *testing.T) {
db, userID := sealedStore(t)
connectedAccount(t, db, userID)
users, err := db.UsersWithCalendars(context.Background())
if err != nil {
t.Fatal(err)
}
if len(users) != 1 || users[0] != userID {
t.Fatalf("the sweep found %v", users)
}
}
+5 -1
View File
@@ -195,10 +195,14 @@ func TestAGmailConnectedBeforeTheMoveSurvivesIt(t *testing.T) {
}
values := `{"refresh_token":"` + sealed + `","trigger":"label","label":"Verdande",` +
`"email":"kw@nolimit.dk","seen":["abc","def"],"project_id":"p-1"}`
// The conflict target is (user_id, scope) since 0025 — before that the table
// held one row per person and the upsert renamed its scope, which is the bug
// 0025 exists to undo. Nothing conflicts here; the clause only has to be legal
// against the schema this test is running on.
if _, err := db.ExecContext(context.Background(), `
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
VALUES (?, 'gmail', ?, 0)
ON CONFLICT (user_id) DO UPDATE SET scope = 'gmail', values_json = excluded.values_json`,
ON CONFLICT (user_id, scope) DO UPDATE SET values_json = excluded.values_json`,
userID, values); err != nil {
t.Fatal(err)
}
+26
View File
@@ -0,0 +1,26 @@
-- One row per installation that has ever reported in.
--
-- This table only exists on the instance that is acting as the collector. Every
-- other instance has it too, empty, because the schema is the same everywhere —
-- and because "am I the collector" is a setting, not a build.
--
-- What is deliberately NOT here is the point of the whole feature: there is no
-- column for an IP address, a hostname, a domain, an account, a count of tasks,
-- or anything else. The two things stored are the two things sent, and the table
-- is the promise written down where it cannot drift from the prose — a column
-- added here later is a promise broken, and it should be as awkward to add as
-- possible.
--
-- `instance_id` is the primary key rather than a rowid alias, because the same
-- installation reporting daily must update its row rather than grow a new one.
-- What is wanted is "how many installations", not "how many pings".
CREATE TABLE beacon_installs (
instance_id TEXT PRIMARY KEY,
version TEXT NOT NULL DEFAULT '',
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL
);
-- The count is always "seen in the last N days", so last_seen is what every query
-- sorts and filters on.
CREATE INDEX beacon_by_last_seen ON beacon_installs (last_seen DESC);
@@ -0,0 +1,111 @@
-- Calendars read from somewhere else: the account, the calendars in it, and the
-- events they hold.
--
-- Not in `mailboxes`, and it was the first thing considered. The two look alike
-- from a distance — a person connects a Google account over OAuth, tokens are
-- sealed, a sweep polls it — and every column past that disagrees. `trigger_kind`,
-- `seen`, `last_uid` and `project_id` all answer "what turns a message into a
-- task", and nothing here turns into a task. What is here instead is a *set*:
-- one account holds many calendars, and a person picks which of them to look at.
-- `mailboxes` has no room for a one-to-many, and giving it one would push the
-- other kind's columns onto rows that can never use them.
--
-- The decisive part is smaller and more practical. `mailboxes.kind` carries
-- `CHECK (kind IN ('gmail', 'imap'))`, and SQLite cannot alter a CHECK — adding a
-- third value means rebuilding the table with foreign keys off, which is the
-- expensive, careful thing migration 0021 had to do for a reason. Paying that to
-- store something that shares no column with the table is the wrong trade.
--
-- Hence no CHECK on `provider` here. CalDAV is the next source to read, and it
-- should cost a row, not a rebuild.
-- One connected account per person per provider. The tokens live here rather than
-- on each calendar: a refresh token copied onto ten rows is ten copies of one
-- secret, ten refreshes racing each other, and ten places to miss when access is
-- revoked.
--
-- `refresh_token` and `access_token` are written sealed; see internal/secret. The
-- store's own seal/unseal only covers the settings tables, so this table's methods
-- call them by hand — the same fact worth knowing that migration 0014 wrote down.
CREATE TABLE calendar_accounts (
id TEXT PRIMARY KEY,
user_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
provider TEXT NOT NULL DEFAULT 'google',
-- The address the account reports, so the settings page can say which one is
-- connected rather than only that one is.
account TEXT NOT NULL DEFAULT '',
refresh_token TEXT NOT NULL DEFAULT '',
access_token TEXT NOT NULL DEFAULT '',
expires_at INTEGER NOT NULL DEFAULT 0,
last_sync_at INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL
);
-- Connecting again replaces what was there, the way Gmail's does: the OAuth flow
-- is one account at a time, so a second row for the same provider would be a
-- second set of tokens for the same grant and no way to say which is live.
CREATE UNIQUE INDEX calendar_accounts_once ON calendar_accounts (user_id, provider);
-- One row per calendar the account can see, kept whether or not it is shown.
--
-- `shown` rather than deleting the ones that are not: a person who turns a calendar
-- off and on again should get the same row back, and the list is refreshed from
-- Google on every sync — so a calendar that only existed while it was ticked would
-- lose its choice every time somebody added a new one.
--
-- `colour` is Google's own hex for the calendar. Not one of verdande's ten named
-- colours, which is the rule everywhere else in this database: those are names so
-- the interface can restyle them per theme, and they are *verdande's* palette. A
-- Google calendar is told apart by the colour the person already knows it by, and
-- that colour belongs to Google.
CREATE TABLE calendars (
id TEXT PRIMARY KEY,
account_id TEXT NOT NULL REFERENCES calendar_accounts (id) ON DELETE CASCADE,
-- The id Google knows it by. Long, and an address for the primary calendar.
remote_id TEXT NOT NULL,
name TEXT NOT NULL DEFAULT '',
colour TEXT NOT NULL DEFAULT '',
time_zone TEXT NOT NULL DEFAULT '',
primary_one INTEGER NOT NULL DEFAULT 0,
writable INTEGER NOT NULL DEFAULT 0,
shown INTEGER NOT NULL DEFAULT 0,
sort_order INTEGER NOT NULL DEFAULT 0
);
CREATE UNIQUE INDEX calendars_once ON calendars (account_id, remote_id);
-- The events themselves, cached.
--
-- Cached rather than fetched per view, and the reason is what a month grid does:
-- it pages. Asking Google on every arrow press means a token refresh, a round trip
-- and a spinner for a question that was answered ninety seconds ago — and a
-- calendar that is slower than the calendar it is showing is not worth having.
-- The window is refreshed by the sweep and by the "fetch now" button.
--
-- Days rather than instants, and both ends inclusive. A grid asks one question —
-- "which cells does this cover" — and `start_day <= ? AND end_day >= ?` is that
-- question. `starts_at` is kept as Google wrote it, offset and all, because it is
-- what the chip shows and reparsing it here would ask *this process* what time
-- zone it is in. An all-day event has neither: it has a day, not a moment, and
-- inventing midnight for it is how an event moves a day when a container moves
-- time zone.
CREATE TABLE calendar_events (
id TEXT PRIMARY KEY,
calendar_id TEXT NOT NULL REFERENCES calendars (id) ON DELETE CASCADE,
remote_id TEXT NOT NULL,
summary TEXT NOT NULL DEFAULT '',
starts_at TEXT NOT NULL DEFAULT '',
ends_at TEXT NOT NULL DEFAULT '',
start_day TEXT NOT NULL,
end_day TEXT NOT NULL,
all_day INTEGER NOT NULL DEFAULT 0,
location TEXT NOT NULL DEFAULT '',
url TEXT NOT NULL DEFAULT ''
);
-- The one query there is: everything a person can see, between two days. Ordered
-- by the account so the join has somewhere to start, then by the days the grid
-- compares against.
CREATE INDEX calendar_events_by_day ON calendar_events (calendar_id, start_day, end_day);
@@ -0,0 +1,23 @@
-- A note can be put away without being thrown away.
--
-- The trash already exists and is the wrong tool. It says "this was a mistake,
-- and in thirty days it is gone" — so anything worth keeping cannot go in it, and
-- an archive of twelve hundred imported notes is exactly the case where most of
-- them are worth keeping and almost none are worth looking at. Without somewhere
-- to put them, the list is the whole archive forever, and finding this week's note
-- means reading past a decade.
--
-- A column rather than a folder. Verdande already refused a second hierarchy for
-- notes — see 0018 — and archiving is not filing: it is one bit about whether a
-- note is current, and it has to be answerable next to `deleted_at`, which is the
-- other bit of the same kind.
--
-- Nullable timestamp rather than a boolean, for the same reason `deleted_at` is
-- one: "when was this put away" answers questions a flag cannot, and it costs the
-- same byte.
ALTER TABLE notes ADD COLUMN archived_at INTEGER;
-- The list asks "not deleted, not archived, newest first" on every load, and with
-- twelve hundred notes that is the query that has to be quick.
CREATE INDEX notes_current ON notes (archived_at, deleted_at, updated_at DESC)
WHERE deleted_at IS NULL;
@@ -0,0 +1,37 @@
-- One row per person *and scope*, which is what the table always meant.
--
-- `user_settings` was created with `user_id` as its whole primary key, and its
-- upsert rewrote `scope` on conflict. So a person had exactly one settings row,
-- and whichever scope was written last was the only one that survived: saving an
-- AI provider deleted the Gmail connection, and connecting a calendar deleted the
-- AI provider. Nothing reported it — the write succeeded, and the loss was in the
-- row it landed on.
--
-- This was seen coming twice. Migration 0011 gave the instance's settings their
-- own table partly for this reason and wrote the warning down in as many words;
-- 0014 moved mailboxes out and said the warning "was nearly walked into a second
-- time". The third time was the calendar, which added a `calendar` scope beside
-- `ai` and `gmail` — and walked in.
--
-- The lesson worth keeping: a warning in a migration is read by whoever is editing
-- *that* migration, and nobody was. The shape is fixed here so there is nothing
-- left to warn about.
--
-- SQLite cannot change a primary key in place, so the table is rebuilt. Existing
-- rows are copied as they are: each person has at most one, and it holds whatever
-- scope won last. What the earlier writes lost is already gone and cannot be
-- recovered here — the values were overwritten, not orphaned.
CREATE TABLE user_settings_new (
user_id TEXT NOT NULL REFERENCES users (id) ON DELETE CASCADE,
scope TEXT NOT NULL DEFAULT 'general',
values_json TEXT NOT NULL DEFAULT '{}',
updated_at INTEGER NOT NULL,
PRIMARY KEY (user_id, scope)
);
INSERT INTO user_settings_new (user_id, scope, values_json, updated_at)
SELECT user_id, scope, values_json, updated_at FROM user_settings;
DROP TABLE user_settings;
ALTER TABLE user_settings_new RENAME TO user_settings;
@@ -0,0 +1,30 @@
-- Abonnement på en kalender, man kun har en adresse til.
--
-- Google-vejen er OAuth: én konto pr. person, hvor tokenet er identiteten, og
-- `calendar_accounts_once` sørgede for netop det — én række pr. (person,
-- udbyder). Et abonnement er en anden slags ting. Der er intet token og ingen
-- konto, kun en adresse, og man har flere: helligdage, en fodboldklub, en
-- fælleskalender fra arbejdet. Det er derfor det blev bedt om som "flere
-- kalendere".
--
-- Samme tabel frem for en ny. Et abonnement er én kalender med begivenheder i, og
-- det er præcis det, `calendars` og `calendar_events` allerede beskriver — med
-- vinduet, `shown`, farven og hele fejemaskinen omkring sig. En tabel til ville
-- være en anden vej til det samme sted, og to veje til ét sted er to steder at
-- rette den dag noget om begivenheder ændrer sig.
--
-- Adressen bliver identiteten, som tokenet er det for Google. To rækker med den
-- samme adresse for den samme person ville være den samme kalender hentet to
-- gange, og der er ingen måde at sige hvilken der gælder.
ALTER TABLE calendar_accounts ADD COLUMN url TEXT NOT NULL DEFAULT '';
-- Det gamle indeks holder stadig for OAuth-rækkerne — dem uden adresse — og
-- slipper abonnementerne fri. Delvise indekser er den ene ting SQLite kan her,
-- som en tabelombygning ellers skulle til for.
DROP INDEX calendar_accounts_once;
CREATE UNIQUE INDEX calendar_accounts_once
ON calendar_accounts (user_id, provider) WHERE url = '';
CREATE UNIQUE INDEX calendar_subscription_once
ON calendar_accounts (user_id, url) WHERE url <> '';
@@ -0,0 +1,75 @@
-- verdande:rebuild-tables
--
-- Notekroppe forsegles.
--
-- Grunden er den samme som for postkassers kodeord og OAuth-tokens: en
-- sikkerhedskopi kan hentes gennem fladen, og indtil nu bar hver eneste af dem
-- hver eneste note i klartekst. Noter er dér, folk skriver kodeord og API-nøgler
-- ned — det ved vi, fordi der ligger sådan nogle i denne installation.
--
-- # Hvorfor det ikke var gjort for længe siden
--
-- Kroppen var kopieret to steder mere, og begge kopier var klartekst:
--
-- * `fold` — en genereret søjle med title || ' ' || body, translittereret, så
-- "grøn" kan findes ved at skrive "gron".
-- * `notes_fts` — et FTS5-indeks over titel, krop og fold.
--
-- At forsegle kroppen og lade dem stå ville have været teater: en angriber med
-- filen kunne læse hver note ud af indekset. Så de skal begge væk, og søgningen
-- skal skrives om til at åbne teksten i Go.
--
-- # Hvad det koster, og hvornår formen er forkert
--
-- Søgningen læser nu hver note. Målt: tolv hundrede noter à 3,3 kB koster 33–43 ms
-- pr. søgning, og fladen venter 250 ms efter sidste tastetryk, før den spørger — så
-- det ligger inde i pausen. Tiden går ikke med at åbne konvolutterne, men med at
-- folde og småskrive hver krop; det er lineært, så regn med omtrent 350 ms ved ti
-- tusind noter. Dér er det den forkerte form, og svaret er da et *nøglet* indeks:
-- en HMAC pr. ord under den samme nøgle. Bemærk hvad det koster, når det bliver
-- aktuelt — et nøglet indeks kan slå et helt ord op og ikke andet, så præfiks- og
-- delordssøgningen, der findes i dag, ville falde bort.
--
-- # Hvad der stadig ligger i klartekst
--
-- Alt andet. Opgavetitler, beskrivelser, kommentarer — og `note_links`, som er
-- id'er udledt af kroppen, så en fil røber stadig *hvilke* projekter og opgaver en
-- note peger på, men ikke hvad der står. Det er skrevet i SECURITY.md, og det er
-- ikke løst her.
--
-- # Selve ombygningen
--
-- SQLite kan ikke fjerne en genereret søjle på plads, så tabellen bygges om.
-- Rækkerne kopieres med kroppen, som den er: forseglingen sker i Go, hvor nøglen
-- er, og en opstart efter denne migration går rækkerne igennem og lukker dem. Til
-- den tid kan en søjle indeholde begge dele, og `v1:`-præfikset findes netop for
-- at kunne læses uden at gætte.
CREATE TABLE notes_sealed (
id TEXT PRIMARY KEY,
project_id TEXT REFERENCES projects (id) ON DELETE CASCADE,
title TEXT NOT NULL DEFAULT '',
body TEXT NOT NULL DEFAULT '',
created_by TEXT REFERENCES users (id) ON DELETE SET NULL,
pinned INTEGER NOT NULL DEFAULT 0,
created_at INTEGER NOT NULL,
updated_at INTEGER NOT NULL,
deleted_at INTEGER,
archived_at INTEGER
);
INSERT INTO notes_sealed
(id, project_id, title, body, created_by, pinned, created_at, updated_at, deleted_at, archived_at)
SELECT id, project_id, title, body, created_by, pinned, created_at, updated_at, deleted_at, archived_at
FROM notes;
DROP TRIGGER notes_fts_insert;
DROP TRIGGER notes_fts_delete;
DROP TRIGGER notes_fts_update;
DROP TABLE notes_fts;
DROP TABLE notes;
ALTER TABLE notes_sealed RENAME TO notes;
CREATE INDEX notes_by_project ON notes (project_id, deleted_at, updated_at DESC);
CREATE INDEX notes_current ON notes (archived_at, deleted_at, updated_at DESC)
WHERE deleted_at IS NULL;
+214 -30
View File
@@ -3,22 +3,35 @@ package store
import (
"context"
"database/sql"
"fmt"
"regexp"
"sort"
"strings"
"time"
)
// Note is a page of Markdown belonging to a project, or to nobody.
type Note struct {
ID string `json:"id"`
ProjectID string `json:"project_id,omitempty"`
Title string `json:"title"`
Body string `json:"body"`
CreatedBy string `json:"created_by,omitempty"`
Pinned bool `json:"pinned"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `json:"deleted_at,omitempty"`
ID string `json:"id"`
ProjectID string `json:"project_id,omitempty"`
Title string `json:"title"`
Body string `json:"body"`
CreatedBy string `json:"created_by,omitempty"`
Pinned bool `json:"pinned"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
DeletedAt *time.Time `json:"deleted_at,omitempty"`
ArchivedAt *time.Time `json:"archived_at,omitempty"`
// Preview is the first of the body, for a list that shows one line of it.
//
// It exists so a list of twelve hundred notes does not have to carry twelve
// hundred whole notes: the list rendered 845 KB of JSON to show ninety
// characters of each. When it is set, Body is deliberately empty — see
// handleListNotes. That is not tidiness but a guard: an editor handed a
// truncated body would save the truncation, and the note would be cut down to
// its own preview.
Preview string `json:"preview,omitempty"`
// Links is what the body points at, read out of the text on the way in. Never
// set by a caller: it is derived, and a caller that set it would be describing
@@ -33,7 +46,16 @@ type NoteLink struct {
}
const noteColumns = `id, project_id, title, body, created_by, pinned,
created_at, updated_at, deleted_at`
created_at, updated_at, deleted_at, archived_at`
// The same columns, said with the table in front of them.
//
// Needed only by the search, which joins `notes_fts` — and that table carries its
// own `id` and `project_id` (UNINDEXED, so the row can be found from a match).
// Unqualified, SQLite cannot tell which one is meant and refuses the query.
const noteColumnsQualified = `notes.id, notes.project_id, notes.title, notes.body,
notes.created_by, notes.pinned, notes.created_at, notes.updated_at, notes.deleted_at,
notes.archived_at`
// Reference syntax, and deliberately the same characters the quick-add parser
// already uses. A `#Firma` in a note means the same project as a `#Firma` typed
@@ -69,6 +91,24 @@ func (db *DB) SaveNote(ctx context.Context, n *Note) error {
// it, and every caller gets the same answer because none of them can set it.
n.Title = firstLine(n.Body)
// Forseglet på vej ned, åbnet på vej op.
//
// Både titel og krop: titlen *er* første linje, så en forseglet krop ved siden
// af en læselig titel ville efterlade begyndelsen af hver eneste note i
// klartekst — og det er dér, folk skriver hvad noten handler om.
//
// `n` selv røres ikke. Kalderen har den note i hånden og læser videre i den
// bagefter; at forsegle den på plads ville give en note, hvis krop er
// ciffertekst, til den, der lige har gemt den.
title, err := db.sealValue(n.Title)
if err != nil {
return fmt.Errorf("seal note title: %w", err)
}
body, err := db.sealValue(n.Body)
if err != nil {
return fmt.Errorf("seal note body: %w", err)
}
return db.Tx(ctx, func(tx *sql.Tx) error {
var projectID any
if n.ProjectID != "" {
@@ -88,7 +128,7 @@ func (db *DB) SaveNote(ctx context.Context, n *Note) error {
body = excluded.body,
pinned = excluded.pinned,
updated_at = excluded.updated_at`,
n.ID, projectID, n.Title, n.Body, createdBy, n.Pinned,
n.ID, projectID, title, body, createdBy, n.Pinned,
n.CreatedAt.Unix(), n.UpdatedAt.Unix()); err != nil {
return err
}
@@ -162,7 +202,7 @@ func firstLine(body string) string {
func (db *DB) Note(ctx context.Context, id string) (*Note, error) {
row := db.QueryRowContext(ctx,
`SELECT `+noteColumns+` FROM notes WHERE id = ? AND deleted_at IS NULL`, id)
n, err := scanNote(row)
n, err := db.scanNote(row)
if err == sql.ErrNoRows {
return nil, nil
}
@@ -199,8 +239,23 @@ func (db *DB) LooseNotes(ctx context.Context, userID string) ([]Note, error) {
// shared, and a list that dropped it at that moment would make sharing look like
// deleting — which is exactly how it read before this existed.
func (db *DB) AllNotes(ctx context.Context, userID string) ([]Note, error) {
return db.notesWhere(ctx, `
WHERE deleted_at IS NULL
return db.allNotes(ctx, userID, false)
}
// ArchivedNotes is the same list, the other way round: what has been put away.
func (db *DB) ArchivedNotes(ctx context.Context, userID string) ([]Note, error) {
return db.allNotes(ctx, userID, true)
}
func (db *DB) allNotes(ctx context.Context, userID string, archived bool) ([]Note, error) {
// Archived notes are left out of the ordinary list rather than mixed in and
// greyed: the point of putting something away is not seeing it.
state := "notes.archived_at IS NULL"
if archived {
state = "notes.archived_at IS NOT NULL"
}
return db.notesJoin(ctx, noteColumnsQualified, `
WHERE `+state+` AND deleted_at IS NULL
AND (
(project_id IS NULL AND created_by = ?)
OR project_id IN (
@@ -228,23 +283,104 @@ func (db *DB) NotesLinking(ctx context.Context, kind, targetID string) ([]Note,
// SearchNotes matches title and body, in either spelling of a Danish word.
func (db *DB) SearchNotes(ctx context.Context, query string, limit int) ([]Note, error) {
if strings.TrimSpace(query) == "" {
terms := SearchTerms(query)
if len(terms) == 0 {
return nil, nil
}
if limit <= 0 {
limit = 50
}
expr := MatchExprOver(query, "title", "body")
if expr == "" {
return nil, nil
// Hver note læses og åbnes.
//
// Der er ikke noget indeks at spørge længere, og det er med vilje: `notes_fts`
// og den genererede `fold`-søjle var begge klartekstkopier af kroppen, så en
// forseglet krop ved siden af dem ville have været teater — en fil ville stadig
// røbe hver note. Se 0027.
//
// Målt frem for gættet: tolv hundrede noter à 3,3 kB koster **33–43 ms** pr.
// søgning. Fladen venter 250 ms efter sidste tastetryk, før den spørger, så det
// ligger inde i pausen og ses ikke.
//
// Tiden går ikke med at åbne konvolutterne — fire megabyte AES-GCM er få
// millisekunder — men med at folde og småskrive hver krop for hver søgning.
// Det er lineært, så regn med omtrent 350 ms ved ti tusind noter, og dér er det
// den forkerte form. Svaret er da et *nøglet* indeks: en HMAC pr. ord under
// samme nøgle. Bemærk hvad det koster, når det bliver aktuelt — et nøglet
// indeks kan slå et helt ord op og ikke andet, så præfiks- og delordssøgningen,
// der findes i dag, ville falde bort.
//
// Papirkurven springes over i SQL frem for i løkken: det er den ene begrænsning,
// der kan bruge et indeks, og den fjerner alt hvad der er slettet uden at åbne
// det.
notes, err := db.notesWhere(ctx, `WHERE deleted_at IS NULL`)
if err != nil {
return nil, err
}
// By rowid, the way tasks do it: `id` is UNINDEXED in the table, so selecting
// on it would work and read the whole index to do it.
return db.notesWhere(ctx, `
WHERE deleted_at IS NULL
AND rowid IN (SELECT rowid FROM notes_fts WHERE notes_fts MATCH ?)
ORDER BY updated_at DESC
LIMIT ?`, expr, limit)
type scored struct {
note Note
score int
}
found := make([]scored, 0, limit)
for _, note := range notes {
score := scoreNote(note, terms)
if score == 0 {
continue
}
found = append(found, scored{note: note, score: score})
}
// Bedste først, og ved lige stand den senest rørte. Uden det andet led er
// rækkefølgen mellem to lige gode svar den, SQLite tilfældigvis gav dem, og den
// skifter mellem to søgninger på det samme.
sort.SliceStable(found, func(i, j int) bool {
if found[i].score != found[j].score {
return found[i].score > found[j].score
}
return found[i].note.UpdatedAt.After(found[j].note.UpdatedAt)
})
out := make([]Note, 0, min(limit, len(found)))
for i, f := range found {
if i >= limit {
break
}
out = append(out, f.note)
}
return out, nil
}
// scoreNote er hvor godt en note svarer på ordene, eller nul for slet ikke.
//
// Erstatter bm25, som forsvandt med indekset. Den er grovere, og det er værd at
// vide hvor: bm25 vægter et ord efter hvor sjældent det er i hele samlingen, og
// det kan ikke gøres uden en samling at tælle i. Det, der er beholdt, er den
// halvdel, der betød mest — **titlen vejer ti gange kroppen**. Et ord i titlen er
// hvad noten *handler om*; det samme ord én gang i en lang krop er en omtale.
//
// Alle ord skal findes. Det er den samme regel som før, hvor leddene var bundet
// med AND.
func scoreNote(note Note, terms []string) int {
title := FoldDanish(strings.ToLower(note.Title))
body := FoldDanish(strings.ToLower(note.Body))
score := 0
for _, term := range terms {
needle := FoldDanish(strings.ToLower(term))
inTitle := strings.Count(title, needle)
inBody := strings.Count(body, needle)
if inTitle == 0 && inBody == 0 {
// Ét ord, der ikke findes, er et nej for hele noten.
return 0
}
// Loft pr. ord, så en note, der nævner ordet fyrre gange, ikke slår en, der
// nævner alle ordene én gang. Det er den fejl, en ren optælling laver, og
// den er let at lave og svær at få øje på.
score += 10*min(inTitle, 3) + min(inBody, 5)
}
return score
}
// SetNoteTimes writes the dates a note had somewhere else.
@@ -278,6 +414,27 @@ func (db *DB) SetNoteTimes(ctx context.Context, id string, created, updated time
return err
}
// SetNoteArchived puts a note away, or takes it back out.
//
// Separate from DeleteNote because they mean different things and the difference
// is the whole feature: the trash says "this was a mistake, and in thirty days it
// is gone", and the archive says "this is finished, and I want to keep it". A
// twelve-hundred-note import is mostly the second, and without somewhere to put
// them the list is the whole archive forever.
//
// `updated_at` is deliberately not touched. Archiving is not editing, and a list
// sorted by when things were last written should not be reordered by somebody
// tidying up.
func (db *DB) SetNoteArchived(ctx context.Context, id string, archived bool) error {
var at any
if archived {
at = time.Now().Unix()
}
_, err := db.ExecContext(ctx,
`UPDATE notes SET archived_at = ? WHERE id = ? AND deleted_at IS NULL`, at, id)
return err
}
// DeleteNote puts it in the trash. Nothing here removes a row.
func (db *DB) DeleteNote(ctx context.Context, id string) error {
_, err := db.ExecContext(ctx,
@@ -295,7 +452,13 @@ func (db *DB) RestoreNote(ctx context.Context, id string) error {
}
func (db *DB) notesWhere(ctx context.Context, where string, args ...any) ([]Note, error) {
rows, err := db.QueryContext(ctx, `SELECT `+noteColumns+` FROM notes `+where, args...)
return db.notesJoin(ctx, noteColumns, where, args...)
}
// notesJoin is notesWhere with the column list spelled out, for the one caller
// that joins another table and must therefore say which `id` it means.
func (db *DB) notesJoin(ctx context.Context, columns, where string, args ...any) ([]Note, error) {
rows, err := db.QueryContext(ctx, `SELECT `+columns+` FROM notes `+where, args...)
if err != nil {
return nil, err
}
@@ -303,7 +466,7 @@ func (db *DB) notesWhere(ctx context.Context, where string, args ...any) ([]Note
var out []Note
for rows.Next() {
n, err := scanNote(rows)
n, err := db.scanNote(rows)
if err != nil {
return nil, err
}
@@ -331,14 +494,18 @@ func (db *DB) linksOf(ctx context.Context, noteID string) ([]NoteLink, error) {
return out, rows.Err()
}
func scanNote(row scanner) (Note, error) {
// scanNote læser en række og åbner det, der er forseglet.
//
// En metode og ikke en funktion, fordi nøglen sidder på `db`. Det er den eneste
// grund; alt andet i den er den samme aflæsning som før.
func (db *DB) scanNote(row scanner) (Note, error) {
var n Note
var projectID, createdBy sql.NullString
var created, updated int64
var deleted sql.NullInt64
var deleted, archived sql.NullInt64
if err := row.Scan(&n.ID, &projectID, &n.Title, &n.Body, &createdBy,
&n.Pinned, &created, &updated, &deleted); err != nil {
&n.Pinned, &created, &updated, &deleted, &archived); err != nil {
return n, err
}
n.ProjectID = projectID.String
@@ -349,5 +516,22 @@ func scanNote(row scanner) (Note, error) {
t := time.Unix(deleted.Int64, 0)
n.DeletedAt = &t
}
if archived.Valid {
t := time.Unix(archived.Int64, 0)
n.ArchivedAt = &t
}
// Åbnet her, ét sted, så ingen kalder kan komme til at læse ciffertekst.
//
// En værdi skrevet før der var en nøgle kommer tilbage som sig selv — `v1:`
// findes netop for at kunne læse en blandet søjle — så en installation, der
// endnu ikke har fået sin bagudfyldning, virker imens.
for _, field := range []*string{&n.Title, &n.Body} {
plain, err := db.unsealValue(*field)
if err != nil {
return n, fmt.Errorf("open note %s: %w", n.ID, err)
}
*field = plain
}
return n, nil
}
+100
View File
@@ -333,3 +333,103 @@ func TestATagFindsItsProjectWhateverTheCase(t *testing.T) {
t.Error("a note title did not match itself")
}
}
// En søgning skal svare med det, der passer bedst, øverst.
//
// Den sorterede efter `updated_at`, hvilket var usynligt med ti noter og
// ubrugeligt med tolv hundrede: de blev alle importeret samme aften, så datoen
// siger ingenting, og loftet skar så listen af ved halvtreds tilfældige rækker.
// At søge "Tesla" svarede med en note om nginx først og noten, der hedder "Tesla
// Model Y", bagefter — hvilket læses som en søgning, der ikke virker, fordi det
// fra brugerens plads er præcis dét.
func TestSearchAnswersWithTheBestMatchFirst(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
// Skrevet i den rækkefølge, der ville give det forkerte svar: den, der kun
// nævner ordet i kroppen, gemmes sidst og har derfor den nyeste dato.
for _, body := range []string{
"# Tesla Model Y\n\nBilen selv.\n",
"# Ladestandere\n\nNoter om Tesla og andre mærker.\n",
"# nginx proxy manager backup\n\nEn konfiguration. Tesla nævnes i forbifarten.\n",
} {
if err := db.SaveNote(ctx, &Note{CreatedBy: userID, Body: body}); err != nil {
t.Fatal(err)
}
}
found, err := db.SearchNotes(ctx, "Tesla", 10)
if err != nil {
t.Fatal(err)
}
if len(found) != 3 {
t.Fatalf("fandt %d noter, ventede 3", len(found))
}
if found[0].Title != "Tesla Model Y" {
t.Errorf("øverst stod %q; den note, der hedder det, man søgte efter, skal stå først", found[0].Title)
}
}
// A word inside a word.
//
// Reported as "søgning har ikke det store held": a note holding
// `gamerule keepInventory true` could not be found by typing "keep inventory".
// The index is built of tokens, and `keepInventory` is one token — so the two
// words are nowhere, and the search says there is nothing while the note is right
// there.
//
// A decade of notes is full of these: commands, identifiers, file names. Every one
// of them is several words the index thinks is one.
func TestAWordInsideAWordIsStillFound(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
if err := db.SaveNote(ctx, &Note{
CreatedBy: userID,
Body: "Minecraft Bedrock\n\ngamerule keepInventory true",
}); err != nil {
t.Fatal(err)
}
// A second note, so a fallback that simply returns everything would be caught.
if err := db.SaveNote(ctx, &Note{
CreatedBy: userID,
Body: "Indkøb\n\nKaffe og filtre",
}); err != nil {
t.Fatal(err)
}
for _, q := range []string{
"keep inventory", // the two words, which is how somebody looks for it
"inventory", // the tail on its own — no prefix query can reach this
"keepinventory", // written as one, which the index does hold
"KEEP INVENTORY", // and case is not a question anybody is asking
"vacuum", // not in either note
} {
found, err := db.SearchNotes(ctx, q, 10)
if err != nil {
t.Fatalf("%q: %v", q, err)
}
want := 1
if q == "vacuum" {
want = 0
}
if len(found) != want {
t.Errorf("%q found %d notes, want %d", q, len(found), want)
continue
}
if want == 1 && found[0].Title != "Minecraft Bedrock" {
t.Errorf("%q found %q", q, found[0].Title)
}
}
// The fast path is still the fast path: a word the index does hold must not be
// answered by the scan, or the ranking that was built for twelve hundred notes
// is quietly gone.
found, err := db.SearchNotes(ctx, "Bedrock", 10)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 {
t.Fatalf("an ordinary word found %d notes", len(found))
}
}
+37 -5
View File
@@ -140,11 +140,17 @@ func (db *DB) SavePushSubscription(ctx context.Context, sub *PushSubscription) e
if sub.ID == "" {
sub.ID = NewID()
}
// The upsert no longer moves the row between accounts.
//
// `user_id = excluded.user_id` was in the DO UPDATE, so anybody who learned
// another person's endpoint could claim it by subscribing with it: the
// victim's device would then receive the claimant's notifications and none of
// its own. A re-subscribe from the same person still refreshes the keys, which
// is what the upsert is actually for.
_, err := db.ExecContext(ctx,
`INSERT INTO push_subscriptions (id, user_id, endpoint, p256dh, auth, user_agent, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (endpoint) DO UPDATE SET
user_id = excluded.user_id,
p256dh = excluded.p256dh,
auth = excluded.auth,
user_agent = excluded.user_agent,
@@ -175,7 +181,21 @@ func (db *DB) ListPushSubscriptions(ctx context.Context, userID string) ([]PushS
return out, rows.Err()
}
func (db *DB) DeletePushSubscription(ctx context.Context, endpoint string) error {
// DeletePushSubscription removes one of this person's devices.
//
// Scoped by user as well as by endpoint. It was endpoint alone, which meant
// anybody who learned somebody else's endpoint could silently unsubscribe their
// phone — and the endpoint is the only thing about a subscription that ever
// leaves the browser.
func (db *DB) DeletePushSubscription(ctx context.Context, userID, endpoint string) error {
_, err := db.ExecContext(ctx,
`DELETE FROM push_subscriptions WHERE endpoint = ? AND user_id = ?`, endpoint, userID)
return err
}
// dropPushEndpoint removes a subscription the push service has declared dead.
// Unexported so the only *request* path to a delete stays the scoped one.
func (db *DB) dropPushEndpoint(ctx context.Context, endpoint string) error {
_, err := db.ExecContext(ctx, `DELETE FROM push_subscriptions WHERE endpoint = ?`, endpoint)
return err
}
@@ -186,7 +206,10 @@ func (db *DB) DeletePushSubscription(ctx context.Context, endpoint string) error
// silently unsubscribing somebody's phone.
func (db *DB) RecordPushFailure(ctx context.Context, endpoint string, permanent bool) error {
if permanent {
return db.DeletePushSubscription(ctx, endpoint)
// Not scoped by user, and that is right here: this is the push service
// saying the endpoint is gone, not a person asking for something. There is
// no caller to check against — the caller is the delivery job.
return db.dropPushEndpoint(ctx, endpoint)
}
_, err := db.ExecContext(ctx,
`UPDATE push_subscriptions SET failures = failures + 1 WHERE endpoint = ?`, endpoint)
@@ -289,6 +312,11 @@ var secretFields = map[string]bool{
"access_token": true,
"password": true,
"client_secret": true,
// The AI provider's key. Missed when this list was written, and it is exactly
// the same kind of thing as the rest: a live credential, in a settings blob,
// in a database anybody with a backup file can read. Existing rows convert
// themselves the next time the settings are saved.
"api_key": true,
}
// seal returns a copy of values with the secret fields sealed. Without a key it
@@ -363,11 +391,15 @@ func (db *DB) SetUserSettings(ctx context.Context, userID, scope string, values
if err != nil {
return err
}
// On (user_id, scope), which is the key since 0025. It used to be on `user_id`
// alone and set `scope = excluded.scope` — so writing one scope did not add a
// row beside the others, it took theirs and renamed it. The `ai`, `gmail` and
// `calendar` settings each deleted the other two, and the write reported
// success every time.
_, err = db.ExecContext(ctx, `
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
VALUES (?, ?, ?, ?)
ON CONFLICT (user_id) DO UPDATE SET
scope = excluded.scope,
ON CONFLICT (user_id, scope) DO UPDATE SET
values_json = excluded.values_json,
updated_at = excluded.updated_at`,
userID, scope, string(raw), time.Now().Unix())
+29
View File
@@ -646,6 +646,35 @@ func (db *DB) ReorderProjects(ctx context.Context, userID string, ids []string)
})
}
// ReorderSections writes the order of one project's sections, top to bottom.
//
// The whole list at once, the same as ReorderProjects and for the same reasons: a
// project has a handful of sections, so writing 0, 1, 2 … is one small
// transaction that cannot run out of precision, and a list is impossible to get
// into the half-ordered state a sequence of individual moves leaves behind when
// one of them fails.
//
// Scoped to the project rather than to a user. A section has no owner of its own —
// it belongs to a project, and whether the caller may rearrange it is a question
// about that project, which the handler has already asked. Ids belonging to some
// other project are skipped by the WHERE clause rather than refused, so a stale
// list from a client that has just moved a section elsewhere cannot rearrange a
// project the caller was never looking at.
func (db *DB) ReorderSections(ctx context.Context, projectID string, ids []string) error {
return db.Tx(ctx, func(tx *sql.Tx) error {
now := time.Now().Unix()
for i, id := range ids {
if _, err := tx.ExecContext(ctx,
`UPDATE sections SET sort_order = ?, updated_at = ?
WHERE id = ? AND project_id = ? AND deleted_at IS NULL`,
float64(i), now, id, projectID); err != nil {
return err
}
}
return nil
})
}
// SectionByName resolves a "/name" from quick add within one project.
//
// Case-insensitive, and scoped to the project the task is landing in — a section
+43
View File
@@ -20,6 +20,13 @@ func FoldDanish(s string) string {
return r.Replace(s)
}
// The ceilings on a search. See MatchExprOver for why they exist; both are far
// above anything a person types and far below anything that costs.
const (
maxSearchInput = 512
maxSearchTerms = 16
)
// MatchExpr turns what someone typed into the search box into an FTS5 MATCH
// expression.
//
@@ -51,6 +58,23 @@ func MatchExprOver(query string, columns ...string) string {
return ""
}
// Capped, because the expression is built from the input and then run inside
// SQLite on one of a handful of pooled connections.
//
// Every term becomes roughly fifty bytes of FTS5, and every one of them is a
// prefix query — which is what makes ⌘K feel like it searches as you type, and
// also what makes each one expensive. A quarter of a megabyte pasted into the
// box became a two-megabyte MATCH expression and half a minute inside SQLite,
// holding a connection the whole time. A few of those at once and the database
// is unavailable to everybody on the instance, logins included.
//
// Sixteen terms is far past anything a person types into a search box and
// nowhere near enough to hurt. Silently, because a message about a limit
// nobody has reached on purpose is noise.
if len(terms) > maxSearchTerms {
terms = terms[:maxSearchTerms]
}
var b strings.Builder
for i, term := range terms {
if i > 0 {
@@ -66,10 +90,29 @@ func MatchExprOver(query string, columns ...string) string {
return b.String()
}
// SearchTerms is the words a query asks about, capped the same way the MATCH
// expression is.
//
// Exported because two searches need the same list: the FTS expression is built
// from it, and so is the substring arm that finds a word living inside another
// word. Two tokenizers would be two answers to "what did they type".
func SearchTerms(query string) []string {
terms := tokenize(query)
if len(terms) > maxSearchTerms {
terms = terms[:maxSearchTerms]
}
return terms
}
// tokenize splits on everything that is not a letter or a digit. Splitting on
// punctuation rather than whitespace alone means "q3/2026" searches for "q3" and
// "2026" instead of for a term FTS5's own tokenizer would never have indexed.
func tokenize(query string) []string {
// Trimmed before it is split, so a megabyte of text is not walked rune by rune
// only to have all but sixteen of the results thrown away.
if len(query) > maxSearchInput {
query = query[:maxSearchInput]
}
fields := strings.FieldsFunc(query, func(r rune) bool {
return !unicode.IsLetter(r) && !unicode.IsDigit(r)
})
+269
View File
@@ -124,3 +124,272 @@ func sealedStore(t *testing.T) (*DB, string) {
}
return db, user.ID
}
// Three integrations, one person, and none of them may delete the others.
//
// `user_settings` had `user_id` as its whole primary key and an upsert that
// rewrote `scope` on conflict, so every person had exactly one settings row and
// the last write won the lot: saving an AI provider deleted the Gmail connection,
// and connecting a calendar deleted the AI provider. The write reported success
// each time, which is why it was reported as "the AI settings are not saved"
// rather than as "connecting the calendar deleted them".
//
// Written against the store rather than the handlers, because the shape was the
// bug: every caller was correct on its own.
func TestOnePersonCanHaveSettingsForMoreThanOneThing(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
want := map[string]map[string]any{
"ai": {"provider": "anthropic", "model": "claude-sonnet-5", "api_key": "sk-ant-xxx"},
"gmail": {"refresh_token": "1//0-gmail", "label": "Verdande"},
"calendar": {"refresh_token": "1//0-calendar", "calendar_id": "primary"},
}
// In this order on purpose: the AI settings are written first and read last,
// so a scope that steals the row is caught by the one written earliest.
for _, scope := range []string{"ai", "gmail", "calendar"} {
if err := db.SetUserSettings(ctx, userID, scope, want[scope]); err != nil {
t.Fatalf("save %s: %v", scope, err)
}
}
for _, scope := range []string{"ai", "gmail", "calendar"} {
got, err := db.UserSettings(ctx, userID, scope)
if err != nil {
t.Fatalf("read %s: %v", scope, err)
}
for key, value := range want[scope] {
if got[key] != value {
t.Errorf("%s.%s is %v, want %v", scope, key, got[key], value)
}
}
}
// And a second write to one scope updates it rather than adding another row —
// the upsert still has to upsert.
if err := db.SetUserSettings(ctx, userID, "ai", map[string]any{"provider": "openai"}); err != nil {
t.Fatal(err)
}
var rows int
if err := db.QueryRowContext(ctx,
`SELECT count(*) FROM user_settings WHERE user_id = ?`, userID).Scan(&rows); err != nil {
t.Fatal(err)
}
if rows != 3 {
t.Errorf("the person has %d settings rows, want 3", rows)
}
back, err := db.UserSettings(ctx, userID, "ai")
if err != nil {
t.Fatal(err)
}
if back["provider"] != "openai" {
t.Errorf("the second write did not land: %v", back["provider"])
}
}
// The upgrade path, which is the one nobody runs: an existing database with a row
// in the old shape, opened by a binary that carries 0025.
//
// Every other test here starts empty and runs all the migrations in order, which
// exercises the migration's SQL but not its *situation*. The situation is a table
// that already holds somebody's settings, and the question is whether they come
// through it and whether a second scope can be written afterwards.
func TestSettingsSurviveTheUpgradeToPerScopeRows(t *testing.T) {
dir := t.TempDir()
path := filepath.Join(dir, "test.db")
db, err := Open(path)
if err != nil {
t.Fatal(err)
}
ctx := context.Background()
user := &User{Email: "kristian@example.dk", Name: "Kristian"}
if err := db.CreateUser(ctx, user, "Indbakke"); err != nil {
t.Fatal(err)
}
// Wind the database back to how v0.33.3 left it: user_id as the whole primary
// key, and 0025 not yet applied.
for _, stmt := range []string{
`PRAGMA foreign_keys = OFF`,
`CREATE TABLE us_old (
user_id TEXT PRIMARY KEY REFERENCES users (id) ON DELETE CASCADE,
scope TEXT NOT NULL DEFAULT 'general',
values_json TEXT NOT NULL DEFAULT '{}',
updated_at INTEGER NOT NULL
)`,
`INSERT INTO us_old (user_id, scope, values_json, updated_at)
SELECT user_id, scope, values_json, updated_at FROM user_settings`,
`DROP TABLE user_settings`,
`ALTER TABLE us_old RENAME TO user_settings`,
`DELETE FROM schema_migrations WHERE name = '0025_user_settings_per_scope.sql'`,
`PRAGMA foreign_keys = ON`,
} {
if _, err := db.ExecContext(ctx, stmt); err != nil {
t.Fatalf("rewind (%s): %v", stmt, err)
}
}
// A Gmail connection, written the way the old upsert wrote one.
if _, err := db.ExecContext(ctx, `
INSERT INTO user_settings (user_id, scope, values_json, updated_at)
VALUES (?, 'gmail', '{"label":"Verdande"}', 0)`, user.ID); err != nil {
t.Fatal(err)
}
if err := db.Close(); err != nil {
t.Fatal(err)
}
// And now the new binary opens it.
db, err = Open(path)
if err != nil {
t.Fatalf("the upgrade refused to open the database: %v", err)
}
defer db.Close()
gmail, err := db.UserSettings(ctx, user.ID, "gmail")
if err != nil {
t.Fatal(err)
}
if gmail["label"] != "Verdande" {
t.Errorf("the Gmail settings did not survive the migration: %v", gmail)
}
// The thing that was reported: set up the AI provider, and have it still be
// there afterwards.
if err := db.SetUserSettings(ctx, user.ID, "ai", map[string]any{
"provider": "anthropic", "model": "claude-sonnet-5",
}); err != nil {
t.Fatalf("saving the AI provider after the upgrade: %v", err)
}
ai, err := db.UserSettings(ctx, user.ID, "ai")
if err != nil {
t.Fatal(err)
}
if ai["provider"] != "anthropic" {
t.Errorf("the AI provider is %v after the upgrade, want anthropic", ai["provider"])
}
again, err := db.UserSettings(ctx, user.ID, "gmail")
if err != nil {
t.Fatal(err)
}
if again["label"] != "Verdande" {
t.Errorf("saving the AI provider took the Gmail settings: %v", again)
}
}
// Det, en note siger, må ikke kunne læses i filen, som en sikkerhedskopi kopierer.
//
// Grunden er den samme som for postkassers kodeord: en sikkerhedskopi kan hentes
// gennem fladen. Noter er dér, folk skriver kodeord og API-nøgler ned — det ved vi,
// fordi der ligger sådan nogle i den installation, det her blev bygget til.
func TestANoteIsNotStoredInTheClear(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
const secretLine = "gamerule keepInventory true"
n := &Note{CreatedBy: userID, Body: "Minecraft Bedrock\n\n" + secretLine}
if err := db.SaveNote(ctx, n); err != nil {
t.Fatal(err)
}
var title, body string
if err := db.QueryRowContext(ctx,
`SELECT title, body FROM notes WHERE id = ?`, n.ID).Scan(&title, &body); err != nil {
t.Fatal(err)
}
if strings.Contains(body, secretLine) {
t.Errorf("kroppen står i klartekst i rækken: %s", body)
}
// Og titlen med. Titlen *er* første linje, så en forseglet krop ved siden af en
// læselig titel ville efterlade begyndelsen af hver eneste note i klartekst.
if strings.Contains(title, "Minecraft") {
t.Errorf("titlen står i klartekst i rækken: %s", title)
}
// Kalderens egen note er urørt: den har den i hånden og læser videre i den.
if n.Body != "Minecraft Bedrock\n\n"+secretLine {
t.Errorf("gemningen ændrede kalderens note: %q", n.Body)
}
// Læst tilbage er den sig selv igen.
back, err := db.Note(ctx, n.ID)
if err != nil {
t.Fatal(err)
}
if back.Body != n.Body || back.Title != "Minecraft Bedrock" {
t.Errorf("noten kom anderledes tilbage: %+v", back)
}
// Og søgningen virker stadig — den åbner teksten nu, hvor den før spurgte et
// indeks, der var en klartekstkopi af den.
found, err := db.SearchNotes(ctx, "keep inventory", 10)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 {
t.Fatalf("søgningen fandt %d noter i en forseglet samling", len(found))
}
}
// Noter skrevet før nøglen fandtes skal lukkes, ikke efterlades.
//
// Uden en bagudfyldning ville kroppene blive forseglet efterhånden som de bliver
// *skrevet* — og et arkiv på tolv hundrede importerede noter, som ingen redigerer
// igen, ville aldrig blive det. Det er præcis dét arkiv, der er værd at forsegle.
func TestOldNotesAreSealedOnStartup(t *testing.T) {
db, userID := sealedStore(t)
ctx := context.Background()
// Skrevet uden om Go-laget, som en note fra før 0027 ligger.
if _, err := db.ExecContext(ctx, `
INSERT INTO notes (id, title, body, created_by, created_at, updated_at)
VALUES ('n-gammel', 'Immich DB', 'Immich DB\nkodeord: hemmeligt', ?, 100, 200)`,
userID); err != nil {
t.Fatal(err)
}
sealed, err := db.SealNotes(ctx)
if err != nil {
t.Fatal(err)
}
if sealed != 1 {
t.Fatalf("%d noter blev lukket, want 1", sealed)
}
var body string
var updated int64
if err := db.QueryRowContext(ctx,
`SELECT body, updated_at FROM notes WHERE id = 'n-gammel'`).Scan(&body, &updated); err != nil {
t.Fatal(err)
}
if strings.Contains(body, "hemmeligt") {
t.Errorf("den gamle note ligger stadig åben: %s", body)
}
// `updated_at` er ikke rørt. Det betyder "hvornår skrev dette program noten
// sidst", og at lukke en konvolut om den er ikke at skrive den — en
// bagudfyldning, der stemplede tolv hundrede noter med i aften, ville kaste den
// rækkefølge væk, importen brugte en dag på at hente hjem.
if updated != 200 {
t.Errorf("bagudfyldningen flyttede updated_at til %d", updated)
}
// Og den kan læses.
back, err := db.Note(ctx, "n-gammel")
if err != nil {
t.Fatal(err)
}
if back == nil || !strings.Contains(back.Body, "hemmeligt") {
t.Fatalf("den gamle note kunne ikke læses tilbage: %+v", back)
}
// Kørt igen gør den ingenting: en note, der allerede er lukket, kendes på sit
// præfiks.
again, err := db.SealNotes(ctx)
if err != nil {
t.Fatal(err)
}
if again != 0 {
t.Errorf("anden kørsel lukkede %d noter om igen", again)
}
}
+75
View File
@@ -38,6 +38,81 @@ type DB struct {
// misplaced key look like a corrupt disk.
func (db *DB) UseSecrets(box *secret.Box) { db.box = box }
// SealNotes lukker de noter, der stadig ligger åbne.
//
// Kaldes én gang ved opstart, efter nøglen er sat. Uden den ville kroppene blive
// forseglet efterhånden som de bliver *skrevet* — sådan som instansens
// indstillinger gør det — og for et arkiv på tolv hundrede importerede noter, som
// ingen redigerer igen, betyder det aldrig. En bagudfyldning er den eneste måde,
// den her ændring rent faktisk beskytter noget på.
//
// Idempotent: en note, der allerede er forseglet, kendes på sit præfiks og røres
// ikke. Så den kan køre ved hver opstart uden at koste andet end et gennemløb af
// id'er og en længde.
//
// Skriver med SQL frem for gennem SaveNote. SaveNote udleder titlen af kroppen og
// bygger links om, og begge dele er allerede rigtige her — det, der skal ske, er
// alene, at de to søjler skal skiftes ud med sig selv, lukket.
func (db *DB) SealNotes(ctx context.Context) (int, error) {
if db.box == nil {
return 0, nil
}
rows, err := db.QueryContext(ctx, `SELECT id, title, body FROM notes`)
if err != nil {
return 0, err
}
type open struct{ id, title, body string }
var pending []open
for rows.Next() {
var o open
if err := rows.Scan(&o.id, &o.title, &o.body); err != nil {
rows.Close()
return 0, err
}
if secret.IsSealed(o.body) && (o.title == "" || secret.IsSealed(o.title)) {
continue
}
pending = append(pending, o)
}
rows.Close()
if err := rows.Err(); err != nil {
return 0, err
}
if len(pending) == 0 {
return 0, nil
}
// Hele bunken i én transaktion. En halvt forseglet tabel er ikke i stykker —
// præfikset gør, at begge slags kan læses — men en afbrudt kørsel, der skal
// tages om, er lettere at ræsonnere om, når den enten skete eller ikke gjorde.
err = db.Tx(ctx, func(tx *sql.Tx) error {
for _, o := range pending {
title, err := db.sealValue(o.title)
if err != nil {
return err
}
body, err := db.sealValue(o.body)
if err != nil {
return err
}
// `updated_at` røres ikke. Det betyder "hvornår skrev dette program noten
// sidst", og at lukke en konvolut om den er ikke at skrive den — en
// bagudfyldning, der stemplede tolv hundrede noter med i aften, ville
// kaste den rækkefølge væk, importen brugte en dag på at hente hjem.
if _, err := tx.ExecContext(ctx,
`UPDATE notes SET title = ?, body = ? WHERE id = ?`, title, body, o.id); err != nil {
return err
}
}
return nil
})
if err != nil {
return 0, err
}
return len(pending), nil
}
// dsn is the connection string.
//
// SQLite is a single file, but it is being driven by an HTTP server with concurrent
+28 -1
View File
@@ -166,15 +166,42 @@ func (db *DB) ListTasks(ctx context.Context, userID string, f TaskFilter) ([]Tas
// labels are short, and their names are typed by the person searching for
// them, so there is no spelling to be generous about. The diacritic
// folding that matters for task text does not earn a second index here.
//
// The third arm is a word inside a word. An index is built of tokens, and
// `SpeedAdmin` is one of them — so "speed admin" asks for two words that are
// nowhere. A prefix query gets half of it (`speed*` matches, `admin*` cannot)
// and the two are joined with AND, so the half that works is thrown away.
//
// Notes got the same arm, and for the same reason. Having it in one of the
// two would be worse than having it in neither: "it finds it in my notes but
// not in my tasks" is a bug report waiting, and the answer to it would be
// this line.
//
// It scans, and it only ever runs when somebody typed something in the box —
// Today and Kommende do not set `Search`. Every term has to appear, so the
// arms are ANDed with each other while the whole thing is ORed with the two
// above.
inside := make([]string, 0, maxSearchTerms)
insideArgs := make([]any, 0, maxSearchTerms)
for _, term := range SearchTerms(f.Search) {
inside = append(inside, "t.fold LIKE ?")
insideArgs = append(insideArgs, "%"+FoldDanish(term)+"%")
}
text := ""
if len(inside) > 0 {
text = " OR (" + strings.Join(inside, " AND ") + ")"
}
where = append(where, `(
t.rowid IN (SELECT rowid FROM tasks_fts WHERE tasks_fts MATCH ?)
OR t.id IN (
SELECT tl.task_id FROM task_labels tl
JOIN labels l ON l.id = tl.label_id
WHERE lower(l.name) LIKE ?
)
)`+text+`
)`)
args = append(args, expr, "%"+strings.ToLower(strings.TrimSpace(f.Search))+"%")
args = append(args, insideArgs...)
}
if f.FilterSQL != "" {
where = append(where, "("+f.FilterSQL+")")
+25 -2
View File
@@ -16,6 +16,11 @@ var (
ErrNotFound = errors.New("store: not found")
ErrEmailInUse = errors.New("store: that email address already has an account")
ErrInviteInvalid = errors.New("store: invite is not valid")
// ErrDuplicate er en unik nøgle, der blev brudt, oversat til noget en handler
// kan spørge om. Drivereren udstiller ikke sin egen fejltype brugbart, og en
// handler, der matcher på tekst, er en handler, der går i stykker den dag
// driveren omformulerer sig.
ErrDuplicate = errors.New("store: already exists")
)
type User struct {
@@ -129,7 +134,14 @@ func (db *DB) scanUser(ctx context.Context, query string, args ...any) (*User, e
return nil, err
}
u.TOTPSecret = secret.String
// Opened on the way out. A row written before the key existed is returned as
// itself — see secret.Unseal — so this does not need a migration; a secret
// converts the next time it is written.
if plain, err := db.unsealValue(secret.String); err == nil {
u.TOTPSecret = plain
} else {
u.TOTPSecret = secret.String
}
u.TOTPEnabled = totpEnabled == 1
u.IsAdmin = isAdmin == 1
// A malformed value is read as "nothing folded" rather than as an error: the
@@ -433,9 +445,20 @@ func (db *DB) UpdateProfile(ctx context.Context, userID, name, timezone, locale
}
func (db *DB) SetTOTPSecret(ctx context.Context, userID, secret string, enabled bool) error {
// Sealed, like a mailbox password and for the same reason: the copies, not the
// server. A TOTP secret in a downloadable backup lets whoever holds the copy
// generate valid codes for every account on the instance — which turns the
// second factor from a defence into a file somebody can take home.
//
// It was missed when internal/secret was written, along with three other
// values; see SECURITY.md.
var value any
if secret != "" {
value = secret
sealed, err := db.sealValue(secret)
if err != nil {
return err
}
value = sealed
}
_, err := db.ExecContext(ctx,
`UPDATE users SET totp_secret = ?, totp_enabled = ?, updated_at = ? WHERE id = ?`,
+3 -1
View File
@@ -50,10 +50,12 @@ nav:
- Using it:
- Signing in: signing-in.md
- Quick add: quick-add.md
- Notes: notes.md
- Filters: filters.md
- Importing from Todoist: import.md
- Connecting things:
- Calendar and CalDAV: caldav.md
- Calendar: calendar.md
- Feeds and CalDAV: caldav.md
- Mail to task: mail.md
- Mailboxes: mailboxes.md
- Claude (MCP): mcp.md
+605 -80
View File
@@ -6,12 +6,12 @@
<title>verdande — self-hosted tasks, shared</title>
<meta
name="description"
content="Self-hosted tasks and projects, shared. One Go binary with SQLite inside it. Your mail as an inbox, CalDAV, passkeys, and an MCP connector for Claude."
content="Self-hosted tasks, notes and projects, shared. One Go binary with SQLite inside it. Your mail as an inbox, a CalDAV server, notes in Markdown, passkeys, and an MCP connector for Claude."
/>
<meta name="theme-color" content="#111312" />
<meta property="og:title" content="verdande" />
<meta property="og:description" content="Self-hosted tasks and projects, shared. One binary, SQLite inside it." />
<meta property="og:description" content="Self-hosted tasks, notes and projects, shared. One binary, SQLite inside it." />
<meta property="og:type" content="website" />
<meta property="og:image" content="screenshots/today.webp" />
<meta property="og:image:alt" content="The Today view: projects and filters beside a list of what is due." />
@@ -76,10 +76,10 @@
}
.theme-toggle {
position: fixed;
top: 1rem;
right: 1rem;
z-index: 10;
/* In the bar now rather than floating over the corner. Fixed, it sat
on top of whatever happened to scroll under it — on a narrow window
that was the first screenshot's own corner. */
flex: none;
width: 2.25rem;
height: 2.25rem;
display: grid;
@@ -141,10 +141,86 @@
padding: 0 1.5rem;
}
/* --- the bar that stays -------------------------------------------
A page this long had its only way out at the very top: somebody who
read as far as "What it will not do" had to scroll back past nine
screenshots to reach the documentation. The bar is the way out from
wherever you are.
`sticky` and not `fixed`: sticky keeps its space in the flow, so
nothing below has to be pushed down by a number that then has to be
kept in step with the bar's own height.
Full width, with the wrap *inside* it. Put the other way round, the
background would end where the column ends and the page would scroll
past it on both sides. */
.topbar {
position: sticky;
top: 0;
z-index: 20;
background: var(--ground);
border-bottom: 1px solid var(--line);
}
.bar {
display: flex;
align-items: center;
gap: 1.25rem;
padding: 0.625rem 1.5rem;
}
.bar-brand {
display: flex;
align-items: center;
gap: 0.5rem;
color: var(--ink);
text-decoration: none;
font-weight: 560;
letter-spacing: -0.01em;
}
.bar-brand .rune {
color: var(--accent);
font-size: 1.125rem;
line-height: 1;
}
.bar-links {
display: flex;
align-items: center;
gap: 1.25rem;
margin-left: auto;
font-size: 0.875rem;
}
.bar-links a {
color: var(--ink-muted);
text-decoration: none;
}
.bar-links a:hover {
color: var(--ink);
}
/* The three in-page links go first on a narrow screen: they are the ones
a phone can reach by scrolling anyway. Documentation and Source cannot
be reached any other way, so they stay. */
@media (max-width: 640px) {
.bar-links .inpage {
display: none;
}
}
/* Or the bar covers the heading you just jumped to. */
:target {
scroll-margin-top: 4.5rem;
}
/* --- hero --- */
header {
padding: 6rem 0 4rem;
.hero {
padding: 5rem 0 3.5rem;
text-align: center;
}
@@ -172,7 +248,7 @@
}
.norn {
margin: 2rem auto 0;
margin: 0 auto 1.75rem;
max-width: 34rem;
font-size: 0.9375rem;
color: var(--ink-faint);
@@ -289,44 +365,104 @@
/* --- features --- */
/* A hairline between sections rather than under every heading: the page
is one column, and the line says where one subject stops. */
section {
padding: 3rem 0;
padding: 3.5rem 0;
border-top: 1px solid var(--line);
}
h2 {
display: flex;
align-items: baseline;
gap: 0.75rem;
font-size: 1.5rem;
font-weight: 560;
letter-spacing: -0.02em;
margin: 0 0 2rem;
}
/* A short accent rule in front of every h2. The sections had nothing to
mark them but their own words, and at a glance the page read as one
long run of paragraphs. */
h2::before {
content: '';
flex: none;
width: 1.125rem;
height: 2px;
border-radius: 1px;
background: var(--accent);
transform: translateY(-0.35em);
}
/* Three groups rather than eighteen cards in a row. Grouped, the list is
read as three answers — what it does, what it talks to, what it
promises — and any of the three can be skipped. */
.group-label {
margin: 2.5rem 0 1rem;
font-size: 0.75rem;
font-weight: 560;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--ink-faint);
}
.group-label:first-of-type {
margin-top: 0;
}
/* Cards with their own edges, not cells in one table.
The table shared a hairline grid, which meant the last row had to be
full or it was a hole in the drawing — and it was a hole, for as long
as there were ten of them. A card that stands alone on its row is a
card; an empty cell is a mistake. */
.grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(260px, 1fr));
gap: 1px;
background: var(--line);
border: 1px solid var(--line);
border-radius: var(--radius-lg);
overflow: hidden;
grid-template-columns: repeat(auto-fit, minmax(15rem, 1fr));
gap: 0.875rem;
}
.cell {
background: var(--ground);
padding: 1.5rem;
background: var(--surface);
border: 1px solid var(--line);
border-radius: var(--radius-lg);
padding: 1.125rem 1.25rem 1.25rem;
transition: border-color 150ms ease;
}
.cell:hover {
border-color: var(--line-strong);
}
.cell h3 {
margin: 0 0 0.5rem;
font-size: 1rem;
margin: 0 0 0.4375rem;
font-size: 0.9375rem;
font-weight: 560;
}
.cell p {
margin: 0;
font-size: 0.9375rem;
font-size: 0.875rem;
color: var(--ink-muted);
}
.cell code {
font-family: var(--font-mono);
font-size: 0.8125em;
color: var(--ink);
}
/* A card that has something to admit rather than something to offer.
It is not a footnote and it is not fine print: it is the same size as
everything else, and it says the same thing the README says. */
.cell .caveat {
margin: 0.5rem 0 0;
padding-top: 0.5rem;
border-top: 1px solid var(--line);
font-size: 0.8125rem;
color: var(--ink-faint);
}
/* --- install --- */
pre {
@@ -374,6 +510,168 @@
color: var(--ink-muted);
}
/* Four facts under the buttons, in the place where a landing page usually
puts logos it has not earned. They are the four questions somebody
self-hosting asks before anything else. */
ul.facts {
list-style: none;
display: flex;
flex-wrap: wrap;
justify-content: center;
gap: 0.5rem 1.25rem;
margin: 2rem 0 0;
padding: 0;
font-size: 0.8125rem;
color: var(--ink-faint);
}
ul.facts li {
display: flex;
align-items: center;
gap: 1.25rem;
}
/* The separator belongs between two facts, not after the last one — so it
is drawn by the item that follows it. */
ul.facts li + li::before {
content: '';
width: 3px;
height: 3px;
border-radius: 50%;
background: var(--line-strong);
}
/* On a phone the row wraps, and a separator drawn by the item that follows
it lands at the *start* of the new line — a dot with nothing before it.
No separators there; the wrap is the separation. */
@media (max-width: 640px) {
ul.facts {
gap: 0.375rem 1.125rem;
}
ul.facts li + li::before {
display: none;
}
}
/* Two ways to run it, side by side, because they are alternatives rather
than steps. Stacked, the second read as what you do after the first. */
.panels {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(19rem, 1fr));
gap: 0.875rem;
}
.panel {
background: var(--surface);
border: 1px solid var(--line);
border-radius: var(--radius-lg);
padding: 1.25rem;
}
.panel h3 {
margin: 0 0 0.5rem;
font-size: 0.9375rem;
font-weight: 560;
}
/* Sunk rather than raised: inside a card, a block in --surface would be
the same colour as the card it sits in. */
.panel pre {
background: var(--ground);
margin: 1rem 0 0;
}
.panel p {
margin: 0;
font-size: 0.875rem;
color: var(--ink-muted);
}
/* Versionsnummeret i sin egen søjle, så øjet kan løbe ned ad dem uden at
læse sætningerne. `ch` frem for pixels: søjlen er lige så bred som de
tal, der står i den, og den følger med, hvis skriften skifter. */
ul.lately {
list-style: none;
margin: 1.5rem 0 0;
padding: 0;
}
ul.lately li {
display: grid;
grid-template-columns: 5ch 1fr;
gap: 0 1rem;
padding: 0.875rem 0;
border-top: 1px solid var(--line);
font-size: 0.9375rem;
color: var(--ink-muted);
}
ul.lately .ver {
font-family: var(--font-mono);
font-size: 0.8125rem;
color: var(--ink-faint);
/* Tallet står på linje med den første tekstlinje og ikke over den. */
line-height: 1.7;
}
@media (max-width: 520px) {
ul.lately li {
grid-template-columns: 1fr;
gap: 0.25rem;
}
}
/* Risikoen står i kort, ikke i småt.
Et forbehold sat med mindre skrift end resten er et forbehold, der er
skrevet for at kunne peges på bagefter — og det er den ene ting, denne
side ikke skal gøre. Samme størrelse som alt andet. */
.risk {
border: 1px solid var(--line-strong);
border-radius: var(--radius-lg);
padding: 1.5rem;
background: var(--surface);
}
.risk p {
margin: 0 0 1rem;
font-size: 0.9375rem;
color: var(--ink-muted);
}
.risk p:last-child {
margin-bottom: 0;
}
.risk strong {
color: var(--ink);
}
/* The non-goals were one paragraph of seven sentences, which is the shape
nobody reads. Each is its own line now, and they can be counted. */
ul.nogo {
list-style: none;
display: grid;
grid-template-columns: repeat(auto-fit, minmax(16rem, 1fr));
gap: 0.5rem 1.75rem;
margin: 0;
padding: 0;
font-size: 0.9375rem;
color: var(--ink-muted);
}
ul.nogo li {
position: relative;
padding-left: 1.25rem;
}
ul.nogo li::before {
content: '—';
position: absolute;
left: 0;
color: var(--ink-faint);
}
.demo-note {
margin: 1rem 0 0;
font-family: var(--font);
@@ -524,27 +822,38 @@
</head>
<body>
<button class="theme-toggle" type="button" aria-label="Switch between light and dark" title="Light or dark">
<span aria-hidden="true">◐</span>
</button>
<div class="wrap">
<header>
<div class="topbar">
<div class="wrap bar">
<a class="bar-brand" href="#top">
<span class="rune" aria-hidden="true">ᚹ</span>
verdande
</a>
<nav class="bar-links" aria-label="Sections">
<a class="inpage" href="#screens">Screens</a>
<a class="inpage" href="#features">Features</a>
<a class="inpage" href="#running">Running it</a>
<a href="https://kristianwind.github.io/verdande/">Documentation</a>
<a href="https://github.com/kristianwind/verdande">Source</a>
<button class="theme-toggle" type="button" aria-label="Switch between light and dark" title="Light or dark">
<span aria-hidden="true">◐</span>
</button>
</nav>
</div>
</div>
<div class="wrap" id="top">
<header class="hero">
<span class="mark" aria-hidden="true">ᚹ</span>
<h1>verdande</h1>
<p class="lede">
Self-hosted tasks and projects, shared with the people you share the work with.
One binary, SQLite inside it.
Self-hosted tasks, notes and projects, shared with the people you share the work
with. One binary, SQLite inside it.
</p>
<p class="rune-note">
Built to be deployed as a <strong>Rune</strong> in
<a href="https://github.com/kristianwind/yggdrasil">Yggdrasil Panel</a> — upload
the manifest, fill in two fields, press start. It also runs anywhere Docker does.
</p>
<p class="norn">
Verdande is one of the three Norns — the one who spins <em>that which is
becoming</em>. Not what has been, and not what is fated to be: the thing you are
in the middle of. Which is what a to-do list is.
</p>
<div class="actions">
<a class="button primary" href="https://kristianwind.github.io/verdande/self-hosting/">
@@ -553,6 +862,14 @@
<a class="button" href="https://github.com/kristianwind/verdande">Source</a>
<a class="button" href="https://kristianwind.github.io/verdande/">Documentation</a>
</div>
<ul class="facts">
<li>One Go binary</li>
<li>SQLite inside it</li>
<li>amd64 and arm64</li>
<li>No account anywhere but yours</li>
<li>MIT</li>
</ul>
</header>
<!-- No screenshot: the quick-add parser is the thing worth showing, and it
@@ -584,7 +901,7 @@
</div>
</div>
<section>
<section id="screens">
<h2>What it looks like</h2>
<figure class="shot only-dark">
<img
@@ -621,13 +938,13 @@
</figure>
<figure class="shot">
<img
src="screenshots/today-light.webp"
alt="The same Today view in the light theme."
src="screenshots/notes.webp"
alt="Notes: a list of notes on the left, one open on the right with headings, bullet lists, a quotation and a shell command in a code block."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>The light theme.</figcaption>
<figcaption>Notes. Rich text on top, Markdown underneath.</figcaption>
</figure>
<figure class="shot">
<img
@@ -642,7 +959,7 @@
<figure class="shot">
<img
src="screenshots/detail.webp"
alt="A task open in the detail drawer, with its description, due date, labels, sub-tasks and comments."
alt="A task open in the detail drawer, with its description, priority, due date, labels, sub-tasks and an attached file."
width="1400"
height="900"
loading="lazy"
@@ -652,12 +969,12 @@
<figure class="shot">
<img
src="screenshots/themes.webp"
alt="The settings page showing the five themes as swatches."
alt="The settings page showing five themes as swatches and, below them, four looks written in the typeface each one selects."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>Five themes, all of them measured for contrast.</figcaption>
<figcaption>Five themes and four looks: how bright, and how it reads.</figcaption>
</figure>
<figure class="shot">
<img
@@ -669,11 +986,43 @@
/>
<figcaption>What has been done on the server, across every project.</figcaption>
</figure>
<figure class="shot">
<img
src="screenshots/delegated.webp"
alt="Waiting on others: tasks that have been handed to somebody else, grouped under the person who has them."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>Waiting on others — grouped by who has it.</figcaption>
</figure>
<figure class="shot">
<img
src="screenshots/notes-source.webp"
alt="A note with the source view open: the same note as Markdown, with headings, a list, a quotation, a wiki link and a fenced code block."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>The same note as Markdown. What is on disk is what you see.</figcaption>
</figure>
<figure class="shot">
<img
src="screenshots/settings.webp"
alt="The integrations settings page: connecting Gmail with an OAuth client, and below it the calendar feed address."
width="1400"
height="900"
loading="lazy"
/>
<figcaption>Integrations: your mailbox, your calendar feed, your address.</figcaption>
</figure>
</div>
</section>
<section>
<section id="features">
<h2>What it does</h2>
<p class="group-label">Doing the work</p>
<div class="grid">
<div class="cell">
<h3>Reads what you type</h3>
@@ -682,6 +1031,50 @@
one line — in Danish and English, mixed freely.
</p>
</div>
<div class="cell">
<h3>Nothing to remember</h3>
<p>
Start typing anywhere and you are writing a task, with the letter you
pressed already in the field. <code>⌘K</code> goes anywhere else.
</p>
</div>
<div class="cell">
<h3>Search that reads Danish</h3>
<p>
Type <code>gron</code> and find <code>grøn</code>. Unicode does not
consider <code>ø</code> an accented <code>o</code>, so a search that
leans on that alone is broken for the people this was built for.
</p>
</div>
<div class="cell">
<h3>Repeats, and says so in words</h3>
<p>
Recurring tasks are RRULE underneath and "every other Tuesday" on the
screen. Reminders are absolute or relative, and go out to the devices
you have allowed them on.
</p>
</div>
<div class="cell">
<h3>Notes, beside the work</h3>
<p>
Rich text to write in, Markdown on disk — and a source view to see it.
Images are pasted or dragged in. <code>#project</code> and
<code>[[another note]]</code> are links, and a task shows what has been
written about it. Archive what is done with it; the bin is for mistakes.
</p>
</div>
<div class="cell">
<h3>Waiting on others</h3>
<p>
What you have handed to somebody else, on its own page, grouped under
the person who has it. The thing a shared list stops being able to tell
you the moment it is shared.
</p>
</div>
</div>
<p class="group-label">Connected to what you already use</p>
<div class="grid">
<div class="cell">
<h3>Shared, properly</h3>
<p>
@@ -696,6 +1089,27 @@
here. Not a read-only feed pretending.
</p>
</div>
<div class="cell">
<h3>A calendar of its own</h3>
<p>
A month or a week of what is due, and a secret ICS address to subscribe
to from Apple Calendar, Google or Thunderbird. A Google calendar can be
laid over the top, read-only.
</p>
<p class="caveat">
The Google half is written to Google's documentation and has not yet
been run against Google. The feed and CalDAV have.
</p>
</div>
<div class="cell">
<h3>Your mail, as an inbox</h3>
<p>
Star a mail in Gmail, or flag one over IMAP in iCloud or Fastmail, and
it becomes a task. Forward one to your own address instead and the
subject is parsed like anything else you type. One way: unstarring it
later does nothing to the task you made.
</p>
</div>
<div class="cell">
<h3>Claude, connected</h3>
<p>
@@ -704,35 +1118,39 @@
</p>
</div>
<div class="cell">
<h3>Your mail, as an inbox</h3>
<h3>AI only if you ask for it</h3>
<p>
Star a mail in Gmail, or flag one in iCloud or Fastmail, and it becomes
a task. Forward one to your own address instead and the subject is
parsed like anything else you type. One way: unstarring it later does
nothing to the task you made.
Off by default, your own key, and your choice of Anthropic, OpenAI,
Google or something local. Nothing leaves the server until you have
filled that in, and everything works without it.
</p>
</div>
</div>
<p class="group-label">Yours, and checkable</p>
<div class="grid">
<div class="cell">
<h3>Nothing to remember</h3>
<h3>More than one way in</h3>
<p>
Start typing anywhere and you are writing a task, with the letter you
pressed already in the field. Sign in with a passkey and there is no
password to forget either.
</p>
</div>
<div class="cell">
<h3>Yours to leave</h3>
<p>
Import from Todoist, export back to it, or take everything as JSON. The
round trip is tested, not promised.
Passkeys, or a password hashed with Argon2id and a second factor with
recovery codes. Every device you are signed in on is listed, and any of
them can be signed out from here.
</p>
</div>
<div class="cell">
<h3>Keys stay behind</h3>
<p>
Mail tokens and mailbox passwords are encrypted in the database, and the
key lives in a file beside it — not inside it. So a backup you download,
or one that ends up somewhere it should not, does not open your mail.
Mail tokens, mailbox passwords, second-factor secrets and AI keys are
encrypted in the database, and the key lives in a file beside it — not
inside it. So a backup you download, or one that ends up somewhere it
should not, does not open your mail.
</p>
</div>
<div class="cell">
<h3>Nobody signs themselves up</h3>
<p>
The first account is the administrator, and after that people arrive by
invitation. An address on the internet is not an open door.
</p>
</div>
<div class="cell">
@@ -743,50 +1161,157 @@
instance's record, not the person's.
</p>
</div>
<div class="cell">
<h3>Yours to leave</h3>
<p>
Import from Todoist, export back to it, take everything as JSON, or
hand the notes a zip of Markdown and get the same zip back out. The
round trip is tested, not promised.
</p>
</div>
<div class="cell">
<h3>It counts itself, out loud</h3>
<p>
Once a day an instance says it exists — a random id, the version, and
nothing else. No tasks, no names, no addresses. It is on to begin with,
the settings page prints the exact payload, and one switch stops it.
</p>
</div>
</div>
</section>
<section>
<section id="recently">
<h2>Lately</h2>
<p class="muted">
A landing page can say a project is alive; a list of what landed says it
better. This is not a changelog — every release has notes, and they are on
<a href="https://github.com/kristianwind/verdande/releases">the releases page</a>.
</p>
<ul class="lately">
<li>
<span class="ver">0.34</span>
Sections can be dragged into the order the work actually happens in. The
sidebar keeps its head still while the projects scroll.
</li>
<li>
<span class="ver">0.33</span>
Notes grew an archive, selection of many at once, pictures you can paste
or drag in, a source view, and a copy button on code blocks. Search
stopped answering by date and started answering by relevance.
</li>
<li>
<span class="ver">0.32</span>
A calendar of its own as a menu item, with a read-only Google calendar
that can be laid over it.
</li>
<li>
<span class="ver">0.31</span>
Mailboxes over IMAP, so flagging a mail in iCloud or Fastmail makes a
task — not only Gmail.
</li>
<li>
<span class="ver">0.30</span>
An installation count that says exactly what it sends, and can be turned
off in one click.
</li>
</ul>
</section>
<section id="running">
<h2>Running it</h2>
<h3 class="sub">As a Rune in Yggdrasil Panel</h3>
<p class="muted">
This is what it was built for. <strong>Runes → Carve a rune</strong>, upload
<code>rune/verdande.yaml</code>, create a server from it, set the public address
and your mail server, and start it. No port forwarding — give it a subdomain and
the panel routes it through your Cloudflare Tunnel.
</p>
<h3 class="sub">Or with Docker, anywhere</h3>
<pre><code>docker run -d --name verdande \
<div class="panels">
<div class="panel">
<h3>As a Rune in Yggdrasil Panel</h3>
<p>
This is what it was built for. <strong>Runes → Carve a rune</strong>,
upload <code>rune/verdande.yaml</code>, create a server from it, set the
public address and your mail server, and start it. No port forwarding —
give it a subdomain and the panel routes it through your Cloudflare
Tunnel.
</p>
</div>
<div class="panel">
<h3>Or with Docker, anywhere</h3>
<p>Open the address afterwards and create the first account. That one is the administrator.</p>
<pre><code>docker run -d --name verdande \
-p 8080:8080 \
-v verdande-data:/data \
-e VERDANDE_BASE_URL=https://todo.example.dk \
ghcr.io/kristianwind/verdande:latest</code></pre>
<p class="muted" style="margin-top: 1rem">
Open the address and create the first account. That one is the administrator.
</p>
</div>
</div>
<ul class="plain">
<li>No database to provision — SQLite is inside the binary.</li>
<li>Distroless image: no shell, no package manager, one static file.</li>
<li>amd64 and arm64, so a Raspberry Pi is fine.</li>
<li>Nightly backups, fourteen kept, written with <code>VACUUM INTO</code>.</li>
<li>Twenty-four migrations, run on start. Upgrading is pulling the new image.</li>
</ul>
</section>
<section>
<section id="limits">
<h2>What it will not do</h2>
<p class="muted">
No karma or points. No team workspaces. No native apps. No mail integration
beyond Gmail and the forwarding address. No languages beyond Danish and English.
And no sync protocol of its own — CalDAV and the API <em>are</em> the sync.
</p>
<ul class="nogo">
<li>No karma, streaks or points.</li>
<li>No team workspaces — projects and the people in them are the whole model.</li>
<li>No native apps. It is a PWA, and it installs on a phone.</li>
<li>No Outlook or Exchange: Microsoft has closed IMAP to passwords and wants its own flow.</li>
<li>No languages beyond Danish and English.</li>
<li>No sync protocol of its own — CalDAV and the API <em>are</em> the sync.</li>
<li>No hosted version. There is nowhere to sign up, and that is the point.</li>
</ul>
</section>
<section id="risk">
<h2>At your own risk</h2>
<div class="risk">
<p>
verdande is <strong>MIT licensed and provided as is, without warranty of
any kind</strong>. That sentence is in every MIT licence and it is usually
skipped, so here it is in the open: if you run this, you run it at your
own risk, and nobody is on the hook if it loses your work.
</p>
<p>
It is one person's project, at version 0.x, and 0.x means the shape can
still change. <strong>You host it, so the data and the backups are
yours.</strong> It writes a backup every night and keeps fourteen — and a
backup nobody has ever restored is not yet a backup. Restore one before
you need to.
</p>
<p>
Security is worked on rather than claimed. There have been two full
reviews of the code, every finding is written down and closed, and what is
protected and what is not is spelled out in
<a href="https://github.com/kristianwind/verdande/blob/main/SECURITY.md">SECURITY.md</a>
— including the parts that are a deliberate trade-off. <strong>Nobody
outside the project has audited it.</strong> The last review found two ways
to reach something you should not and closed both; a third review will find
more, which is the point of having one.
</p>
<p>
It is built to sit behind something — a tunnel, a reverse proxy, a VPN —
put there by somebody who knows what they exposed. Found a hole? Mail it
rather than opening an issue; the address is in SECURITY.md and you get an
answer within a week.
</p>
</div>
</section>
<footer>
<!-- The Norn belongs at the end. It was the third paragraph of the hero,
where it stood between somebody arriving and the buttons they came
for — and it is a closing thought, not an opening one. -->
<p class="norn">
Verdande is one of the three Norns — the one who spins <em>that which is
becoming</em>. Not what has been, and not what is fated to be: the thing you are
in the middle of. Which is what a to-do list is.
</p>
<p>
MIT licensed ·
<a href="https://github.com/kristianwind/verdande">github.com/kristianwind/verdande</a>
· <a href="https://kristianwind.github.io/verdande/">Documentation</a>
</p>
<p>
Built to run as a Rune in
Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 34 KiB

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 56 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 52 KiB

Some files were not shown because too many files have changed in this diff Show More