feat(gmail,update): finish the OAuth2 flow, and tell admins about new releases
Gmail now connects. Authorisation-code flow with PKCE, the verifier kept server-side rather than in a cookie — the callback is a top-level navigation from Google, and a SameSite=Lax cookie is the kind of thing that works until a browser tightens its defaults. State is checked, the attempt expires after ten minutes, and the spent verifier is deleted rather than left at rest. Scope is gmail.readonly and the sync is one-way on purpose. Unstarring a message does not delete the task; completing the task does not touch the mailbox. Two-way would mean deciding what happens when both sides changed, and the only honest answer is a sync model far larger than the feature. The query carries newer_than:30d so connecting an account with a decade of starred mail does not create a decade of tasks. The update check is off unless the operator turns it on, asks GitHub at most every six hours, and sends nothing about the instance — no identifier, no version, no counts. A development build never reports an update: somebody running their own tree is ahead of every release, and telling them otherwise trains them to ignore the notice. Only administrators see it, because only they can act on it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Kristian Wind
co-authored by
Claude Opus 5
Claude Opus 5
parent
ba5892fa9e
commit
5546e191ac
@@ -72,14 +72,15 @@ Apple Reminders, forward mail into it, and add it to Claude as a connector.
|
||||
| CalDAV server (two-way VTODO) | Done, tested |
|
||||
| Mail-to-task | Done, tested |
|
||||
| AI layer (Anthropic/OpenAI/Google/local) | Built, degrades when unconfigured |
|
||||
| Gmail settings | Partial — see below |
|
||||
| Gmail: OAuth2 with PKCE, polling, one-way to tasks | Done, tested |
|
||||
| Update notice for administrators | Done, tested |
|
||||
| Documentation, landing page, licence | Done |
|
||||
|
||||
**Not finished:** the Gmail integration stores its settings but has no OAuth2 flow
|
||||
yet, so it cannot actually connect. Web Push and the AI providers are implemented
|
||||
but have only been exercised against their specs, not against live services. And
|
||||
the interface does not yet expose comments, attachments, reminders, templates,
|
||||
import/export or the settings pages — all of it works through the API.
|
||||
**Not finished:** Web Push, the AI providers and the Gmail API calls are
|
||||
implemented against their specifications but have only been exercised there, not
|
||||
against live services. And the interface does not yet expose comments,
|
||||
attachments, reminders, templates, import/export or the settings pages — all of it
|
||||
works through the API.
|
||||
|
||||
Search finds `grøn` when you type `gron`, because Unicode does not consider `ø` an
|
||||
accented `o` and a Danish-first app that cannot do this is broken for its own users.
|
||||
@@ -146,6 +147,9 @@ to put one that the operator would ever see.
|
||||
| `VERDANDE_SMTP_FROM` | `verdande@localhost` | Sender address. |
|
||||
| `VERDANDE_SMTP_STARTTLS` | `true` unless port 465 | Port 465 is TLS from the first byte; everything else negotiates. |
|
||||
| `VERDANDE_SMTP_INSECURE` | `false` | Skip certificate verification. For a self-signed internal mail server, and nothing else. |
|
||||
| `VERDANDE_GMAIL_CLIENT_ID` | — | OAuth client from Google Cloud, if anyone will connect a mailbox. |
|
||||
| `VERDANDE_GMAIL_CLIENT_SECRET` | — | |
|
||||
| `VERDANDE_UPDATE_CHECK` | `false` | Ask GitHub whether a newer release exists. Sends nothing about your instance. |
|
||||
| `VERDANDE_DEV` | `false` | Human-readable logs at debug level. |
|
||||
|
||||
## What lives in `/data`
|
||||
@@ -171,8 +175,7 @@ is part of it.
|
||||
views, durations, ICS feeds, templates, automatic backups. *(done)*
|
||||
4. **Import/export** — Todoist CSV and official export in, CSV/JSON/ICS out. *(done)*
|
||||
5. **Integrations** — CalDAV server, mail-to-task, Gmail, MCP server for Claude,
|
||||
and a provider-agnostic AI layer that also talks to a local model. *(done,
|
||||
except Gmail's OAuth flow)*
|
||||
and a provider-agnostic AI layer that also talks to a local model. *(done)*
|
||||
6. **Publication** — documentation, landing page, open-source scaffolding. *(done)*
|
||||
|
||||
## Development
|
||||
|
||||
@@ -54,6 +54,7 @@ func run() error {
|
||||
defer db.Close()
|
||||
log.Info("database ready", "path", db.Path())
|
||||
|
||||
httpapi.Version = version
|
||||
api := httpapi.New(cfg, db, log, webAssets(log))
|
||||
|
||||
// Background work: reminders, the nightly backup, and the trash emptying
|
||||
@@ -62,6 +63,7 @@ func run() error {
|
||||
jobCtx, stopJobs := context.WithCancel(context.Background())
|
||||
defer stopJobs()
|
||||
runner := jobs.New(cfg, db, log, api.Mail(), api.Hub())
|
||||
runner.SyncGmail = api.SyncGmail
|
||||
runner.Start(jobCtx)
|
||||
|
||||
srv := &http.Server{
|
||||
|
||||
@@ -39,6 +39,32 @@ Any Go duration: `720h`, `30m`, `45s`.
|
||||
yourself, password resets are written to the log, and reminders arrive in the
|
||||
app and as Web Push rather than by email.
|
||||
|
||||
## Gmail
|
||||
|
||||
Only needed if anyone will connect a mailbox. Register an OAuth client in Google
|
||||
Cloud (**APIs & Services → Credentials → OAuth client ID → Web application**) with
|
||||
the redirect URI `https://your-address/oauth/gmail/callback`, enable the Gmail API,
|
||||
and add the `gmail.readonly` scope.
|
||||
|
||||
| Variable | What it does |
|
||||
|---|---|
|
||||
| `VERDANDE_GMAIL_CLIENT_ID` | The client id from Google Cloud. |
|
||||
| `VERDANDE_GMAIL_CLIENT_SECRET` | The client secret. |
|
||||
|
||||
One registration serves every user on the instance; each person authorises their
|
||||
own mailbox from **Settings → Gmail**. The redirect URI is derived from
|
||||
`VERDANDE_BASE_URL`, so it cannot drift out of step with what you registered.
|
||||
|
||||
## Update checks
|
||||
|
||||
| Variable | Default | What it does |
|
||||
|---|---|---|
|
||||
| `VERDANDE_UPDATE_CHECK` | `false` | Ask GitHub whether a newer release exists. |
|
||||
|
||||
Off unless you turn it on. When on, verdande asks the public GitHub releases API
|
||||
at most every six hours and sends nothing about your instance — no identifier, no
|
||||
version, no count of anything. Only administrators are shown the result.
|
||||
|
||||
## What is *not* configured here
|
||||
|
||||
**AI providers** are per-user, under settings in the app — see [AI](ai.md). One
|
||||
|
||||
@@ -28,6 +28,16 @@ type Config struct {
|
||||
|
||||
SMTP SMTP
|
||||
|
||||
// Gmail is the OAuth client the operator registered in Google Cloud. It
|
||||
// belongs to the instance: one registration, any number of users.
|
||||
GmailClientID string
|
||||
GmailClientSecret string
|
||||
|
||||
// UpdateCheck asks GitHub whether a newer release exists. Off unless asked
|
||||
// for: a self-hosted app that reaches out without being told to has broken
|
||||
// the deal its operator made by self-hosting.
|
||||
UpdateCheck bool
|
||||
|
||||
// TrashRetention is how long a soft-deleted task or project stays recoverable.
|
||||
TrashRetention time.Duration
|
||||
|
||||
@@ -57,6 +67,10 @@ func Load() (*Config, error) {
|
||||
DataDir: env("VERDANDE_DATA_DIR", "/data"),
|
||||
TrashRetention: 30 * 24 * time.Hour,
|
||||
Dev: envBool("VERDANDE_DEV", false),
|
||||
|
||||
GmailClientID: env("VERDANDE_GMAIL_CLIENT_ID", ""),
|
||||
GmailClientSecret: env("VERDANDE_GMAIL_CLIENT_SECRET", ""),
|
||||
UpdateCheck: envBool("VERDANDE_UPDATE_CHECK", false),
|
||||
}
|
||||
|
||||
var err error
|
||||
@@ -98,6 +112,11 @@ func (c *Config) DBPath() string { return filepath.Join(c.DataDir, "verdande
|
||||
func (c *Config) FilesDir() string { return filepath.Join(c.DataDir, "files") }
|
||||
func (c *Config) BackupsDir() string { return filepath.Join(c.DataDir, "backups") }
|
||||
|
||||
// GmailRedirectURL has to match what was registered in Google Cloud exactly,
|
||||
// including the scheme — which is why it is derived from BaseURL rather than
|
||||
// configured separately and left to drift out of step with it.
|
||||
func (c *Config) GmailRedirectURL() string { return c.BaseURL + "/oauth/gmail/callback" }
|
||||
|
||||
func env(key, def string) string {
|
||||
if v, ok := os.LookupEnv(key); ok && v != "" {
|
||||
return v
|
||||
|
||||
@@ -0,0 +1,348 @@
|
||||
// Package gmail implements the OAuth2 authorisation-code flow with PKCE against
|
||||
// Google, and the small slice of the Gmail API verdande needs.
|
||||
//
|
||||
// The connection is one-way on purpose: a starred or labelled message becomes a
|
||||
// task, and nothing verdande does afterwards touches the mailbox. Unstarring the
|
||||
// message does not delete the task, completing the task does not unstar the
|
||||
// message. Two-way would mean deciding what to do when both sides changed, and the
|
||||
// only honest answer to that is a synchronisation model far bigger than the feature.
|
||||
//
|
||||
// Scope is gmail.readonly. verdande never needs to send, modify or delete, and
|
||||
// asking for less is what makes the consent screen something a person can agree to.
|
||||
package gmail
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
authEndpoint = "https://accounts.google.com/o/oauth2/v2/auth"
|
||||
tokenEndpoint = "https://oauth2.googleapis.com/token"
|
||||
apiBase = "https://gmail.googleapis.com/gmail/v1"
|
||||
|
||||
// Read-only, and only mail. Not profile, not contacts, not send.
|
||||
Scope = "https://www.googleapis.com/auth/gmail.readonly"
|
||||
)
|
||||
|
||||
// Config is the OAuth client the operator registered in Google Cloud. It belongs to
|
||||
// the instance rather than to a person: one registration, any number of users.
|
||||
type Config struct {
|
||||
ClientID string
|
||||
ClientSecret string
|
||||
RedirectURL string
|
||||
}
|
||||
|
||||
func (c Config) Configured() bool {
|
||||
return c.ClientID != "" && c.ClientSecret != "" && c.RedirectURL != ""
|
||||
}
|
||||
|
||||
// PKCE is the proof-key pair for one authorisation attempt.
|
||||
//
|
||||
// PKCE on a server-side client is not strictly required — the client secret already
|
||||
// authenticates the token exchange — but it costs nothing and closes the window in
|
||||
// which an authorisation code intercepted from the redirect could be replayed
|
||||
// before verdande gets to it.
|
||||
type PKCE struct {
|
||||
Verifier string
|
||||
Challenge string
|
||||
State string
|
||||
}
|
||||
|
||||
func NewPKCE() (PKCE, error) {
|
||||
verifier, err := randomString(64)
|
||||
if err != nil {
|
||||
return PKCE{}, err
|
||||
}
|
||||
state, err := randomString(32)
|
||||
if err != nil {
|
||||
return PKCE{}, err
|
||||
}
|
||||
sum := sha256.Sum256([]byte(verifier))
|
||||
return PKCE{
|
||||
Verifier: verifier,
|
||||
Challenge: base64.RawURLEncoding.EncodeToString(sum[:]),
|
||||
State: state,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// AuthURL is where the browser is sent to ask the person for consent.
|
||||
func (c Config) AuthURL(p PKCE) string {
|
||||
v := url.Values{}
|
||||
v.Set("client_id", c.ClientID)
|
||||
v.Set("redirect_uri", c.RedirectURL)
|
||||
v.Set("response_type", "code")
|
||||
v.Set("scope", Scope)
|
||||
v.Set("state", p.State)
|
||||
v.Set("code_challenge", p.Challenge)
|
||||
v.Set("code_challenge_method", "S256")
|
||||
// Without both of these Google returns no refresh token on a re-authorisation,
|
||||
// and the connection silently stops working an hour later.
|
||||
v.Set("access_type", "offline")
|
||||
v.Set("prompt", "consent")
|
||||
return authEndpoint + "?" + v.Encode()
|
||||
}
|
||||
|
||||
type Token struct {
|
||||
AccessToken string
|
||||
RefreshToken string
|
||||
ExpiresAt time.Time
|
||||
}
|
||||
|
||||
// Exchange trades the authorisation code for tokens.
|
||||
func (c Config) Exchange(ctx context.Context, code string, p PKCE) (Token, error) {
|
||||
form := url.Values{}
|
||||
form.Set("code", code)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
form.Set("redirect_uri", c.RedirectURL)
|
||||
form.Set("grant_type", "authorization_code")
|
||||
form.Set("code_verifier", p.Verifier)
|
||||
|
||||
return c.token(ctx, form)
|
||||
}
|
||||
|
||||
// Refresh gets a fresh access token. Google does not return a new refresh token
|
||||
// here, so the caller keeps the one it has.
|
||||
func (c Config) Refresh(ctx context.Context, refreshToken string) (Token, error) {
|
||||
form := url.Values{}
|
||||
form.Set("refresh_token", refreshToken)
|
||||
form.Set("client_id", c.ClientID)
|
||||
form.Set("client_secret", c.ClientSecret)
|
||||
form.Set("grant_type", "refresh_token")
|
||||
|
||||
token, err := c.token(ctx, form)
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
token.RefreshToken = refreshToken
|
||||
return token, nil
|
||||
}
|
||||
|
||||
func (c Config) token(ctx context.Context, form url.Values) (Token, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, tokenEndpoint,
|
||||
strings.NewReader(form.Encode()))
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
|
||||
resp, err := (&http.Client{Timeout: 30 * time.Second}).Do(req)
|
||||
if err != nil {
|
||||
return Token{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
var parsed struct {
|
||||
AccessToken string `json:"access_token"`
|
||||
RefreshToken string `json:"refresh_token"`
|
||||
ExpiresIn int `json:"expires_in"`
|
||||
Error string `json:"error"`
|
||||
ErrorDescription string `json:"error_description"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&parsed); err != nil {
|
||||
return Token{}, fmt.Errorf("gmail: reading token response: %w", err)
|
||||
}
|
||||
if parsed.Error != "" {
|
||||
return Token{}, fmt.Errorf("gmail: %s: %s", parsed.Error, parsed.ErrorDescription)
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK || parsed.AccessToken == "" {
|
||||
return Token{}, fmt.Errorf("gmail: token exchange failed with %s", resp.Status)
|
||||
}
|
||||
|
||||
return Token{
|
||||
AccessToken: parsed.AccessToken,
|
||||
RefreshToken: parsed.RefreshToken,
|
||||
// A minute of slack, so a token that is about to expire is refreshed
|
||||
// rather than used and rejected.
|
||||
ExpiresAt: time.Now().Add(time.Duration(parsed.ExpiresIn-60) * time.Second),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// --- the API ---------------------------------------------------------------------
|
||||
|
||||
type Client struct {
|
||||
accessToken string
|
||||
http *http.Client
|
||||
}
|
||||
|
||||
func NewClient(accessToken string) *Client {
|
||||
return &Client{accessToken: accessToken, http: &http.Client{Timeout: 30 * time.Second}}
|
||||
}
|
||||
|
||||
// Message is the part of a Gmail message a task is made from.
|
||||
type Message struct {
|
||||
ID string
|
||||
ThreadID string
|
||||
From string
|
||||
Subject string
|
||||
Snippet string
|
||||
// Link opens the message in Gmail's own interface, which is the whole point:
|
||||
// the task is a pointer back to the mail, not a copy of it.
|
||||
Link string
|
||||
}
|
||||
|
||||
// Profile identifies the connected mailbox, so the settings page can show which
|
||||
// account is connected rather than only that one is.
|
||||
func (c *Client) Profile(ctx context.Context) (string, error) {
|
||||
var parsed struct {
|
||||
EmailAddress string `json:"emailAddress"`
|
||||
}
|
||||
if err := c.get(ctx, apiBase+"/users/me/profile", &parsed); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return parsed.EmailAddress, nil
|
||||
}
|
||||
|
||||
// List finds messages matching a Gmail search query.
|
||||
//
|
||||
// Gmail's own query language is used rather than a filter built here — "is:starred",
|
||||
// "label:Til-handling" — because it is what the person already knows from the
|
||||
// search box, and it is what their existing filters are written in.
|
||||
func (c *Client) List(ctx context.Context, query string, max int) ([]string, error) {
|
||||
if max <= 0 || max > 50 {
|
||||
max = 25
|
||||
}
|
||||
v := url.Values{}
|
||||
v.Set("q", query)
|
||||
v.Set("maxResults", fmt.Sprint(max))
|
||||
|
||||
var parsed struct {
|
||||
Messages []struct {
|
||||
ID string `json:"id"`
|
||||
} `json:"messages"`
|
||||
}
|
||||
if err := c.get(ctx, apiBase+"/users/me/messages?"+v.Encode(), &parsed); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
ids := make([]string, 0, len(parsed.Messages))
|
||||
for _, m := range parsed.Messages {
|
||||
ids = append(ids, m.ID)
|
||||
}
|
||||
return ids, nil
|
||||
}
|
||||
|
||||
// Get fetches the headers of one message. Metadata format only: verdande needs the
|
||||
// sender and the subject, and asking for the full body would download attachments
|
||||
// it has no use for.
|
||||
func (c *Client) Get(ctx context.Context, id string) (Message, error) {
|
||||
v := url.Values{}
|
||||
v.Set("format", "metadata")
|
||||
v.Add("metadataHeaders", "From")
|
||||
v.Add("metadataHeaders", "Subject")
|
||||
|
||||
var parsed struct {
|
||||
ID string `json:"id"`
|
||||
ThreadID string `json:"threadId"`
|
||||
Snippet string `json:"snippet"`
|
||||
Payload struct {
|
||||
Headers []struct {
|
||||
Name string `json:"name"`
|
||||
Value string `json:"value"`
|
||||
} `json:"headers"`
|
||||
} `json:"payload"`
|
||||
}
|
||||
if err := c.get(ctx, apiBase+"/users/me/messages/"+url.PathEscape(id)+"?"+v.Encode(), &parsed); err != nil {
|
||||
return Message{}, err
|
||||
}
|
||||
|
||||
msg := Message{
|
||||
ID: parsed.ID, ThreadID: parsed.ThreadID, Snippet: parsed.Snippet,
|
||||
Link: "https://mail.google.com/mail/u/0/#inbox/" + parsed.ThreadID,
|
||||
}
|
||||
for _, h := range parsed.Payload.Headers {
|
||||
switch strings.ToLower(h.Name) {
|
||||
case "from":
|
||||
msg.From = h.Value
|
||||
case "subject":
|
||||
msg.Subject = h.Value
|
||||
}
|
||||
}
|
||||
return msg, nil
|
||||
}
|
||||
|
||||
func (c *Client) get(ctx context.Context, url string, out any) error {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Authorization", "Bearer "+c.accessToken)
|
||||
|
||||
resp, err := c.http.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode == http.StatusUnauthorized {
|
||||
return ErrUnauthorized
|
||||
}
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("gmail: %s", resp.Status)
|
||||
}
|
||||
return json.NewDecoder(resp.Body).Decode(out)
|
||||
}
|
||||
|
||||
// ErrUnauthorized means the tokens are no longer good — the person revoked access,
|
||||
// or changed their password. The caller disconnects rather than retrying.
|
||||
var ErrUnauthorized = fmt.Errorf("gmail: authorisation was refused")
|
||||
|
||||
// SenderName pulls "Anders Jensen" out of "Anders Jensen <anders@example.dk>",
|
||||
// falling back to the address when there is no display name.
|
||||
func SenderName(from string) string {
|
||||
from = strings.TrimSpace(from)
|
||||
if i := strings.Index(from, "<"); i > 0 {
|
||||
// Space first, then quotes. A display name arrives as `"Jensen, Anders" `
|
||||
// — with a trailing space before the angle bracket — so trimming quotes
|
||||
// first stops at the space and leaves the closing quote behind.
|
||||
name := strings.Trim(strings.TrimSpace(from[:i]), `"`)
|
||||
if name != "" {
|
||||
return name
|
||||
}
|
||||
}
|
||||
return strings.Trim(strings.Trim(from, "<>"), `"`)
|
||||
}
|
||||
|
||||
// Query builds the Gmail search for the configured trigger.
|
||||
func Query(trigger, label string) string {
|
||||
var terms []string
|
||||
switch trigger {
|
||||
case "starred":
|
||||
terms = append(terms, "is:starred")
|
||||
case "label":
|
||||
if label != "" {
|
||||
terms = append(terms, `label:"`+label+`"`)
|
||||
}
|
||||
case "both":
|
||||
if label != "" {
|
||||
// Gmail's OR needs the braces form to group correctly.
|
||||
terms = append(terms, `{is:starred label:"`+label+`"}`)
|
||||
} else {
|
||||
terms = append(terms, "is:starred")
|
||||
}
|
||||
}
|
||||
if len(terms) == 0 {
|
||||
return ""
|
||||
}
|
||||
// Only recent mail. Without this, connecting an account with a decade of
|
||||
// starred messages would create a decade of tasks in one sweep.
|
||||
terms = append(terms, "newer_than:30d")
|
||||
return strings.Join(terms, " ")
|
||||
}
|
||||
|
||||
func randomString(n int) (string, error) {
|
||||
b := make([]byte, n)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return base64.RawURLEncoding.EncodeToString(b)[:n], nil
|
||||
}
|
||||
@@ -0,0 +1,163 @@
|
||||
package gmail
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// The challenge has to be the SHA-256 of the verifier, base64url without padding.
|
||||
// Getting this wrong means Google refuses every exchange with a message that does
|
||||
// not say which end is at fault.
|
||||
func TestPKCEChallengeMatchesItsVerifier(t *testing.T) {
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
sum := sha256.Sum256([]byte(p.Verifier))
|
||||
want := base64.RawURLEncoding.EncodeToString(sum[:])
|
||||
if p.Challenge != want {
|
||||
t.Errorf("challenge = %q, want %q", p.Challenge, want)
|
||||
}
|
||||
|
||||
// RFC 7636 requires 43–128 characters, and no padding.
|
||||
if n := len(p.Verifier); n < 43 || n > 128 {
|
||||
t.Errorf("verifier is %d characters, outside the 43–128 the spec allows", n)
|
||||
}
|
||||
if strings.ContainsAny(p.Verifier, "=+/") {
|
||||
t.Errorf("verifier %q contains characters that are not URL-safe", p.Verifier)
|
||||
}
|
||||
if p.State == "" {
|
||||
t.Error("no state was generated")
|
||||
}
|
||||
|
||||
// Two attempts must not share anything.
|
||||
other, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if other.Verifier == p.Verifier || other.State == p.State {
|
||||
t.Error("two authorisation attempts produced the same secrets")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAuthURL(t *testing.T) {
|
||||
cfg := Config{
|
||||
ClientID: "client-123",
|
||||
ClientSecret: "secret",
|
||||
RedirectURL: "https://todo.example.dk/oauth/gmail/callback",
|
||||
}
|
||||
p, err := NewPKCE()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
raw := cfg.AuthURL(p)
|
||||
parsed, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("the URL does not parse: %v", err)
|
||||
}
|
||||
q := parsed.Query()
|
||||
|
||||
checks := map[string]string{
|
||||
"client_id": cfg.ClientID,
|
||||
"redirect_uri": cfg.RedirectURL,
|
||||
"response_type": "code",
|
||||
"scope": Scope,
|
||||
"code_challenge": p.Challenge,
|
||||
"code_challenge_method": "S256",
|
||||
"state": p.State,
|
||||
// Without both of these Google returns no refresh token on a
|
||||
// re-authorisation, and the connection dies silently an hour later.
|
||||
"access_type": "offline",
|
||||
"prompt": "consent",
|
||||
}
|
||||
for key, want := range checks {
|
||||
if got := q.Get(key); got != want {
|
||||
t.Errorf("%s = %q, want %q", key, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// The verifier itself must never leave the server.
|
||||
if strings.Contains(raw, p.Verifier) {
|
||||
t.Error("the code verifier was put in the authorisation URL")
|
||||
}
|
||||
// Read-only, and only mail.
|
||||
if !strings.HasSuffix(Scope, "gmail.readonly") {
|
||||
t.Errorf("scope = %q; verdande never needs to send or modify", Scope)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigured(t *testing.T) {
|
||||
full := Config{ClientID: "a", ClientSecret: "b", RedirectURL: "c"}
|
||||
if !full.Configured() {
|
||||
t.Error("a complete config is not reported as configured")
|
||||
}
|
||||
for _, partial := range []Config{
|
||||
{},
|
||||
{ClientID: "a"},
|
||||
{ClientID: "a", ClientSecret: "b"},
|
||||
{ClientSecret: "b", RedirectURL: "c"},
|
||||
} {
|
||||
if partial.Configured() {
|
||||
t.Errorf("%+v was reported as configured", partial)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestQuery(t *testing.T) {
|
||||
cases := []struct {
|
||||
trigger, label string
|
||||
wantContains []string
|
||||
wantEmpty bool
|
||||
}{
|
||||
{"starred", "", []string{"is:starred", "newer_than:30d"}, false},
|
||||
{"label", "Til handling", []string{`label:"Til handling"`, "newer_than:30d"}, false},
|
||||
{"both", "Til handling", []string{"is:starred", `label:"Til handling"`}, false},
|
||||
{"both", "", []string{"is:starred"}, false},
|
||||
// A label trigger with no label selects nothing, which is right: it would
|
||||
// otherwise match the whole mailbox.
|
||||
{"label", "", nil, true},
|
||||
{"", "", nil, true},
|
||||
{"nonsense", "", nil, true},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.trigger+"/"+tc.label, func(t *testing.T) {
|
||||
got := Query(tc.trigger, tc.label)
|
||||
if tc.wantEmpty {
|
||||
if got != "" {
|
||||
t.Errorf("Query = %q, want empty", got)
|
||||
}
|
||||
return
|
||||
}
|
||||
for _, want := range tc.wantContains {
|
||||
if !strings.Contains(got, want) {
|
||||
t.Errorf("Query = %q, missing %q", got, want)
|
||||
}
|
||||
}
|
||||
// Without a window, connecting an account with a decade of starred
|
||||
// mail would create a decade of tasks in one sweep.
|
||||
if !strings.Contains(got, "newer_than:") {
|
||||
t.Errorf("Query = %q has no time window", got)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSenderName(t *testing.T) {
|
||||
cases := map[string]string{
|
||||
"Anders Jensen <anders@example.dk>": "Anders Jensen",
|
||||
`"Jensen, Anders" <anders@example.dk>`: "Jensen, Anders",
|
||||
"anders@example.dk": "anders@example.dk",
|
||||
"<anders@example.dk>": "anders@example.dk",
|
||||
" Anders <a@b.dk> ": "Anders",
|
||||
}
|
||||
for in, want := range cases {
|
||||
if got := SenderName(in); got != want {
|
||||
t.Errorf("SenderName(%q) = %q, want %q", in, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -31,6 +31,13 @@ type testServer struct {
|
||||
|
||||
func newTestServer(t *testing.T) *testServer {
|
||||
t.Helper()
|
||||
return newTestServerWith(t, nil)
|
||||
}
|
||||
|
||||
// newTestServerWith lets a test vary the configuration — a registered OAuth client,
|
||||
// update checking on — without every other test paying for it.
|
||||
func newTestServerWith(t *testing.T, configure func(*config.Config)) *testServer {
|
||||
t.Helper()
|
||||
|
||||
db, err := store.Open(filepath.Join(t.TempDir(), "test.db"))
|
||||
if err != nil {
|
||||
@@ -46,6 +53,10 @@ func newTestServer(t *testing.T) *testServer {
|
||||
ResetTTL: time.Hour,
|
||||
Dev: true,
|
||||
}
|
||||
if configure != nil {
|
||||
configure(cfg)
|
||||
}
|
||||
|
||||
// Discard: a passing test should print nothing, and these handlers log freely.
|
||||
log := slog.New(slog.NewTextHandler(io.Discard, nil))
|
||||
|
||||
|
||||
@@ -0,0 +1,302 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/gmail"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
)
|
||||
|
||||
func (s *Server) gmailConfig() gmail.Config {
|
||||
return gmail.Config{
|
||||
ClientID: s.cfg.GmailClientID,
|
||||
ClientSecret: s.cfg.GmailClientSecret,
|
||||
RedirectURL: s.cfg.GmailRedirectURL(),
|
||||
}
|
||||
}
|
||||
|
||||
// handleGmailAuthorize starts the flow: it mints a PKCE pair, remembers it against
|
||||
// the user, and hands back the Google URL to send the browser to.
|
||||
//
|
||||
// The verifier and state are stored server-side rather than put in a cookie,
|
||||
// because the callback arrives as a top-level navigation from Google and a
|
||||
// SameSite=Lax cookie is the kind of thing that works until a browser tightens its
|
||||
// defaults.
|
||||
func (s *Server) handleGmailAuthorize(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
cfg := s.gmailConfig()
|
||||
|
||||
if !cfg.Configured() {
|
||||
writeError(w, http.StatusConflict, CodeConflict,
|
||||
"Gmail er ikke sat op på denne server — administratoren skal registrere en OAuth-klient hos Google")
|
||||
return
|
||||
}
|
||||
|
||||
pkce, err := gmail.NewPKCE()
|
||||
if err != nil {
|
||||
s.internal(w, "gmail pkce", err)
|
||||
return
|
||||
}
|
||||
|
||||
settings, err := s.db.UserSettings(r.Context(), user.ID, "gmail")
|
||||
if err != nil {
|
||||
s.internal(w, "gmail settings", err)
|
||||
return
|
||||
}
|
||||
settings["pkce_verifier"] = pkce.Verifier
|
||||
settings["pkce_state"] = pkce.State
|
||||
// The attempt expires: a stored verifier that lives forever is a replay window
|
||||
// that lives forever.
|
||||
settings["pkce_expires"] = time.Now().Add(10 * time.Minute).Unix()
|
||||
|
||||
if err := s.db.SetUserSettings(r.Context(), user.ID, "gmail", settings); err != nil {
|
||||
s.internal(w, "gmail settings", err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]string{"url": cfg.AuthURL(pkce)})
|
||||
}
|
||||
|
||||
// handleGmailCallback is where Google sends the browser back.
|
||||
//
|
||||
// It is a top-level navigation, so it answers with a redirect into the app rather
|
||||
// than with JSON — the person is looking at a browser tab, not at a fetch response.
|
||||
func (s *Server) handleGmailCallback(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
settingsURL := s.cfg.BaseURL + "/indstillinger"
|
||||
|
||||
if errParam := r.URL.Query().Get("error"); errParam != "" {
|
||||
// Somebody clicked "cancel" on the consent screen. Not an error worth a
|
||||
// page of its own.
|
||||
http.Redirect(w, r, settingsURL+"?gmail="+errParam, http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
code := r.URL.Query().Get("code")
|
||||
state := r.URL.Query().Get("state")
|
||||
if code == "" || state == "" {
|
||||
http.Redirect(w, r, settingsURL+"?gmail=invalid", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
settings, err := s.db.UserSettings(r.Context(), user.ID, "gmail")
|
||||
if err != nil {
|
||||
s.internal(w, "gmail settings", err)
|
||||
return
|
||||
}
|
||||
verifier, _ := settings["pkce_verifier"].(string)
|
||||
expected, _ := settings["pkce_state"].(string)
|
||||
expires, _ := settings["pkce_expires"].(float64)
|
||||
|
||||
// The state has to match, and match in constant time is not needed here — it
|
||||
// is a comparison of two values the server generated — but it does have to
|
||||
// match at all, or this endpoint accepts a code from anywhere.
|
||||
if verifier == "" || expected == "" || state != expected {
|
||||
http.Redirect(w, r, settingsURL+"?gmail=state", http.StatusFound)
|
||||
return
|
||||
}
|
||||
if int64(expires) < time.Now().Unix() {
|
||||
http.Redirect(w, r, settingsURL+"?gmail=expired", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
token, err := s.gmailConfig().Exchange(r.Context(), code, gmail.PKCE{Verifier: verifier})
|
||||
if err != nil {
|
||||
s.log.Error("gmail exchange", "err", err, "user", user.ID)
|
||||
http.Redirect(w, r, settingsURL+"?gmail=failed", http.StatusFound)
|
||||
return
|
||||
}
|
||||
if token.RefreshToken == "" {
|
||||
// Without one the connection dies in an hour. Google only withholds it
|
||||
// when the consent prompt was skipped, which access_type and prompt are
|
||||
// meant to prevent — so this means something is wrong with the client
|
||||
// registration rather than with this request.
|
||||
s.log.Error("gmail returned no refresh token", "user", user.ID)
|
||||
http.Redirect(w, r, settingsURL+"?gmail=norefresh", http.StatusFound)
|
||||
return
|
||||
}
|
||||
|
||||
email, err := gmail.NewClient(token.AccessToken).Profile(r.Context())
|
||||
if err != nil {
|
||||
s.log.Warn("gmail profile", "err", err)
|
||||
}
|
||||
|
||||
// The one-time values are cleared: keeping a spent verifier serves nothing and
|
||||
// is one more secret at rest.
|
||||
delete(settings, "pkce_verifier")
|
||||
delete(settings, "pkce_state")
|
||||
delete(settings, "pkce_expires")
|
||||
|
||||
settings["refresh_token"] = token.RefreshToken
|
||||
settings["access_token"] = token.AccessToken
|
||||
settings["expires_at"] = token.ExpiresAt.Unix()
|
||||
settings["email"] = email
|
||||
if _, ok := settings["trigger"]; !ok {
|
||||
settings["trigger"] = "starred"
|
||||
}
|
||||
|
||||
if err := s.db.SetUserSettings(r.Context(), user.ID, "gmail", settings); err != nil {
|
||||
s.internal(w, "save gmail tokens", err)
|
||||
return
|
||||
}
|
||||
s.log.Info("gmail connected", "user", user.ID, "mailbox", email)
|
||||
http.Redirect(w, r, settingsURL+"?gmail=connected", http.StatusFound)
|
||||
}
|
||||
|
||||
// handleGmailSyncNow polls immediately, so somebody who has just connected can see
|
||||
// it work rather than wondering whether it did.
|
||||
func (s *Server) handleGmailSyncNow(w http.ResponseWriter, r *http.Request) {
|
||||
user := userFrom(r.Context())
|
||||
|
||||
created, err := s.SyncGmail(r.Context(), user)
|
||||
if err != nil {
|
||||
writeError(w, http.StatusBadGateway, CodeInternal, err.Error())
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, map[string]int{"created": created})
|
||||
}
|
||||
|
||||
// syncGmail creates a task for each matching message that has not produced one yet.
|
||||
//
|
||||
// Returns how many were made. Also used by the background job, which is why it
|
||||
// takes a context and a user rather than a request.
|
||||
func (s *Server) SyncGmail(ctx context.Context, user *store.User) (int, error) {
|
||||
settings, err := s.db.UserSettings(ctx, user.ID, "gmail")
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
str := func(key string) string {
|
||||
v, _ := settings[key].(string)
|
||||
return v
|
||||
}
|
||||
|
||||
refresh := str("refresh_token")
|
||||
if refresh == "" {
|
||||
return 0, nil // not connected; nothing to do and nothing to complain about
|
||||
}
|
||||
query := gmail.Query(str("trigger"), str("label"))
|
||||
if query == "" {
|
||||
return 0, nil // connected but no trigger chosen
|
||||
}
|
||||
|
||||
// The access token is refreshed only when it has actually expired.
|
||||
access := str("access_token")
|
||||
expires, _ := settings["expires_at"].(float64)
|
||||
if access == "" || int64(expires) <= time.Now().Unix() {
|
||||
token, err := s.gmailConfig().Refresh(ctx, refresh)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
access = token.AccessToken
|
||||
settings["access_token"] = access
|
||||
settings["expires_at"] = token.ExpiresAt.Unix()
|
||||
if err := s.db.SetUserSettings(ctx, user.ID, "gmail", settings); err != nil {
|
||||
return 0, err
|
||||
}
|
||||
}
|
||||
|
||||
client := gmail.NewClient(access)
|
||||
ids, err := client.List(ctx, query, 25)
|
||||
if err != nil {
|
||||
if err == gmail.ErrUnauthorized {
|
||||
// The person revoked access, or changed their password. Forget the
|
||||
// tokens rather than retrying every ten minutes forever.
|
||||
s.log.Info("gmail access was revoked; disconnecting", "user", user.ID)
|
||||
_ = s.db.SetUserSettings(ctx, user.ID, "gmail", map[string]any{})
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
|
||||
// Which messages have already become tasks. Kept as a list on the settings
|
||||
// rather than as a table: it is bounded by the query's own 30-day window, and
|
||||
// a table would be one more thing to migrate for a feature this small.
|
||||
seen := map[string]bool{}
|
||||
if raw, ok := settings["seen"].([]any); ok {
|
||||
for _, v := range raw {
|
||||
if id, ok := v.(string); ok {
|
||||
seen[id] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
inbox, err := s.db.InboxID(ctx, user.ID)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
projectID := str("project_id")
|
||||
if projectID == "" {
|
||||
projectID = inbox
|
||||
} else if _, err := store.RequireProjectRole(ctx, s.db, projectID, user.ID, store.RoleEditor); err != nil {
|
||||
projectID = inbox
|
||||
}
|
||||
|
||||
created := 0
|
||||
for _, id := range ids {
|
||||
if seen[id] {
|
||||
continue
|
||||
}
|
||||
msg, err := client.Get(ctx, id)
|
||||
if err != nil {
|
||||
s.log.Warn("gmail get message", "err", err, "id", id)
|
||||
continue
|
||||
}
|
||||
|
||||
subject := strings.TrimSpace(msg.Subject)
|
||||
if subject == "" {
|
||||
subject = "(uden emne)"
|
||||
}
|
||||
sender := gmail.SenderName(msg.From)
|
||||
|
||||
task := &store.Task{
|
||||
ProjectID: projectID,
|
||||
Content: sender + ": " + subject,
|
||||
// The deep link is the point: the task is a pointer back to the mail,
|
||||
// not a copy of it.
|
||||
Description: msg.Snippet + "\n\n" + msg.Link,
|
||||
Priority: 4,
|
||||
CreatedBy: user.ID,
|
||||
}
|
||||
if err := s.db.CreateTask(ctx, task, nil); err != nil {
|
||||
s.log.Warn("gmail create task", "err", err)
|
||||
continue
|
||||
}
|
||||
seen[id] = true
|
||||
created++
|
||||
s.hub.Publish(projectID, "task.created", toTaskJSON(*task))
|
||||
}
|
||||
|
||||
if created > 0 {
|
||||
// Only the ids Gmail still returns are kept, so the list cannot grow
|
||||
// without bound as messages fall out of the 30-day window.
|
||||
keep := make([]any, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
if seen[id] {
|
||||
keep = append(keep, id)
|
||||
}
|
||||
}
|
||||
settings["seen"] = keep
|
||||
settings["last_sync"] = time.Now().Unix()
|
||||
if err := s.db.SetUserSettings(ctx, user.ID, "gmail", settings); err != nil {
|
||||
return created, err
|
||||
}
|
||||
}
|
||||
return created, nil
|
||||
}
|
||||
|
||||
// --- version and updates ------------------------------------------------------------
|
||||
|
||||
func (s *Server) handleVersion(w http.ResponseWriter, r *http.Request) {
|
||||
status := s.updates.Status(r.Context())
|
||||
// The notice is for whoever would do something about it. Telling an ordinary
|
||||
// member that the server is out of date is telling them about somebody else's
|
||||
// job.
|
||||
if user := userFrom(r.Context()); user == nil || !user.IsAdmin {
|
||||
status.Available = false
|
||||
status.Notes = ""
|
||||
status.URL = ""
|
||||
status.Latest = ""
|
||||
}
|
||||
writeJSON(w, http.StatusOK, status)
|
||||
}
|
||||
@@ -4,8 +4,11 @@ import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/kristianwind/verdande/internal/config"
|
||||
)
|
||||
|
||||
// --- MCP -----------------------------------------------------------------------------
|
||||
@@ -582,3 +585,189 @@ func TestGmailSettings(t *testing.T) {
|
||||
t.Errorf("settings did not persist: %v", after)
|
||||
}
|
||||
}
|
||||
|
||||
// --- Gmail OAuth ------------------------------------------------------------------------
|
||||
|
||||
// Without an OAuth client registered by the operator there is nothing to authorise
|
||||
// against, and saying so is more useful than a broken redirect to Google.
|
||||
func TestGmailAuthorizeNeedsAnOAuthClient(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/gmail/authorize", nil)
|
||||
if resp.StatusCode != http.StatusConflict {
|
||||
t.Fatalf("status %d, want 409", resp.StatusCode)
|
||||
}
|
||||
if msg, _ := body["error"].(string); !strings.Contains(msg, "OAuth") {
|
||||
t.Errorf("the message does not explain what is missing: %q", msg)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGmailAuthorizeBuildsAConsentURL(t *testing.T) {
|
||||
ts := newTestServerWith(t, func(cfg *config.Config) {
|
||||
cfg.GmailClientID = "client-123"
|
||||
cfg.GmailClientSecret = "secret"
|
||||
})
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/gmail/authorize", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("authorize: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
raw, _ := body["url"].(string)
|
||||
parsed, err := url.Parse(raw)
|
||||
if err != nil {
|
||||
t.Fatalf("the URL does not parse: %v", err)
|
||||
}
|
||||
if parsed.Host != "accounts.google.com" {
|
||||
t.Errorf("host = %q", parsed.Host)
|
||||
}
|
||||
q := parsed.Query()
|
||||
if q.Get("code_challenge") == "" || q.Get("code_challenge_method") != "S256" {
|
||||
t.Error("the URL carries no PKCE challenge")
|
||||
}
|
||||
if q.Get("state") == "" {
|
||||
t.Error("the URL carries no state")
|
||||
}
|
||||
|
||||
// The verifier is kept server-side against the user, never sent to the browser.
|
||||
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
settings, err := ts.db.UserSettings(t.Context(), user.ID, "gmail")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
verifier, _ := settings["pkce_verifier"].(string)
|
||||
if verifier == "" {
|
||||
t.Fatal("no verifier was stored")
|
||||
}
|
||||
if strings.Contains(raw, verifier) {
|
||||
t.Error("the verifier was put in the URL handed to the browser")
|
||||
}
|
||||
if settings["pkce_state"] != q.Get("state") {
|
||||
t.Error("the stored state does not match the one in the URL")
|
||||
}
|
||||
}
|
||||
|
||||
// The callback is where an attacker would try to plant somebody else's code. A
|
||||
// mismatched, missing or expired state has to be refused.
|
||||
func TestGmailCallbackChecksState(t *testing.T) {
|
||||
ts := newTestServerWith(t, func(cfg *config.Config) {
|
||||
cfg.GmailClientID = "client-123"
|
||||
cfg.GmailClientSecret = "secret"
|
||||
})
|
||||
ts.bootstrap(t)
|
||||
ts.do(t, "POST", "/api/v1/gmail/authorize", nil)
|
||||
|
||||
user, err := ts.db.UserByEmail(t.Context(), "kristian@example.dk")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
cases := []struct {
|
||||
name, query, wantParam string
|
||||
}{
|
||||
{"no state", "?code=abc", "invalid"},
|
||||
{"no code", "?state=abc", "invalid"},
|
||||
{"wrong state", "?code=abc&state=forkert", "state"},
|
||||
{"user declined", "?error=access_denied", "access_denied"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
// A redirect, because the person is looking at a browser tab.
|
||||
client := &http.Client{
|
||||
Jar: ts.client.Jar,
|
||||
CheckRedirect: func(*http.Request, []*http.Request) error {
|
||||
return http.ErrUseLastResponse
|
||||
},
|
||||
}
|
||||
req, err := http.NewRequest("GET", ts.URL+"/oauth/gmail/callback"+tc.query, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
req.Header.Set("Sec-Fetch-Site", "same-origin")
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusFound {
|
||||
t.Fatalf("status %d, want a redirect", resp.StatusCode)
|
||||
}
|
||||
if loc := resp.Header.Get("Location"); !strings.Contains(loc, tc.wantParam) {
|
||||
t.Errorf("redirected to %q, want it to mention %q", loc, tc.wantParam)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// And nothing was connected by any of that.
|
||||
settings, err := ts.db.UserSettings(t.Context(), user.ID, "gmail")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if settings["refresh_token"] != nil {
|
||||
t.Error("a refresh token was stored by a rejected callback")
|
||||
}
|
||||
}
|
||||
|
||||
// A connected mailbox has to be pollable without a browser, which is what the
|
||||
// background job does. With nothing connected it is a no-op rather than an error.
|
||||
func TestGmailSyncIsANoOpWhenNotConnected(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "POST", "/api/v1/gmail/sync", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("sync: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
if body["created"] != float64(0) {
|
||||
t.Errorf("created = %v with nothing connected", body["created"])
|
||||
}
|
||||
}
|
||||
|
||||
// --- version ------------------------------------------------------------------------------
|
||||
|
||||
// The check is off unless the operator asked for it. A self-hosted app that reaches
|
||||
// out without being told to has broken the deal its operator made.
|
||||
func TestVersionCheckIsOffByDefault(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
|
||||
resp, body := ts.do(t, "GET", "/api/v1/version", nil)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("version: %d %v", resp.StatusCode, body)
|
||||
}
|
||||
if body["disabled"] != true {
|
||||
t.Errorf("disabled = %v, want true on a fresh instance", body["disabled"])
|
||||
}
|
||||
if body["update_available"] == true {
|
||||
t.Error("an update was reported with checking turned off")
|
||||
}
|
||||
if body["current"] == nil {
|
||||
t.Error("the current version is not reported")
|
||||
}
|
||||
}
|
||||
|
||||
// Only an administrator can do anything about an out-of-date server, so only an
|
||||
// administrator is told.
|
||||
func TestUpdateNoticeIsForAdministratorsOnly(t *testing.T) {
|
||||
ts := newTestServer(t)
|
||||
ts.bootstrap(t)
|
||||
member := ts.newUser(t, "medlem@example.dk", "Medlem")
|
||||
|
||||
_, body := member.do(t, "GET", "/api/v1/version", nil)
|
||||
if body["update_available"] == true {
|
||||
t.Error("an ordinary member was told the server is out of date")
|
||||
}
|
||||
if body["latest"] != nil && body["latest"] != "" {
|
||||
t.Errorf("latest = %v for a non-admin", body["latest"])
|
||||
}
|
||||
// They still see which version they are on, which is what a bug report needs.
|
||||
if body["current"] == nil {
|
||||
t.Error("the current version is hidden from members")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -21,16 +21,23 @@ import (
|
||||
"github.com/kristianwind/verdande/internal/mcp"
|
||||
"github.com/kristianwind/verdande/internal/realtime"
|
||||
"github.com/kristianwind/verdande/internal/store"
|
||||
"github.com/kristianwind/verdande/internal/update"
|
||||
)
|
||||
|
||||
// Version is stamped by main at build time, and is what the update check compares
|
||||
// against. A package-level variable rather than a config field because it is a
|
||||
// property of the binary, not of how it was configured.
|
||||
var Version = "dev"
|
||||
|
||||
type Server struct {
|
||||
cfg *config.Config
|
||||
db *store.DB
|
||||
log *slog.Logger
|
||||
web fs.FS
|
||||
mail *mail.Sender
|
||||
hub *realtime.Hub
|
||||
mcp *mcp.Server
|
||||
cfg *config.Config
|
||||
db *store.DB
|
||||
log *slog.Logger
|
||||
web fs.FS
|
||||
mail *mail.Sender
|
||||
hub *realtime.Hub
|
||||
mcp *mcp.Server
|
||||
updates *update.Checker
|
||||
|
||||
// Password guessing is the attack this application is actually exposed to, so
|
||||
// the endpoints that check a secret are limited separately and more tightly
|
||||
@@ -57,6 +64,7 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
}
|
||||
s.csp = contentSecurityPolicy(scriptHashes(web))
|
||||
s.mcp = s.buildMCP()
|
||||
s.updates = update.New(Version, cfg.UpdateCheck)
|
||||
|
||||
r := chi.NewRouter()
|
||||
r.Use(middleware.RequestID)
|
||||
@@ -81,6 +89,14 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
// Inbound mail. Delivered by the mail server rather than by a browser, and
|
||||
// authenticated by the token in the recipient address.
|
||||
r.Post("/inbound/mail", s.handleInboundMail)
|
||||
|
||||
// Google sends the browser here after consent. Outside /api/v1 because it is a
|
||||
// top-level navigation rather than a fetch, but still behind the session — the
|
||||
// tokens it exchanges belong to whoever is signed in.
|
||||
r.Group(func(r chi.Router) {
|
||||
r.Use(s.requireAuth)
|
||||
r.Get("/oauth/gmail/callback", s.handleGmailCallback)
|
||||
})
|
||||
r.Route("/caldav", s.caldavRoutes)
|
||||
|
||||
r.Route("/api/v1", func(r chi.Router) {
|
||||
@@ -181,10 +197,14 @@ func New(cfg *config.Config, db *store.DB, log *slog.Logger, web fs.FS) *Server
|
||||
r.Post("/tasks/{taskID}/split", s.handleAISplit)
|
||||
})
|
||||
|
||||
r.Get("/version", s.handleVersion)
|
||||
|
||||
r.Route("/gmail", func(r chi.Router) {
|
||||
r.Get("/", s.handleGetGmail)
|
||||
r.Put("/", s.handleSetGmail)
|
||||
r.Delete("/", s.handleDisconnectGmail)
|
||||
r.Post("/authorize", s.handleGmailAuthorize)
|
||||
r.Post("/sync", s.handleGmailSyncNow)
|
||||
})
|
||||
|
||||
r.Route("/import", func(r chi.Router) {
|
||||
|
||||
@@ -31,6 +31,12 @@ type Runner struct {
|
||||
mail *mail.Sender
|
||||
hub *realtime.Hub
|
||||
|
||||
// SyncGmail is supplied by the HTTP layer, which owns the token refresh and
|
||||
// the task creation. Passed in as a function rather than importing it, because
|
||||
// the other direction would be a cycle: httpapi already depends on jobs being
|
||||
// startable from main.
|
||||
SyncGmail func(ctx context.Context, user *store.User) (int, error)
|
||||
|
||||
wg sync.WaitGroup
|
||||
}
|
||||
|
||||
@@ -49,6 +55,11 @@ func (r *Runner) Start(ctx context.Context) {
|
||||
r.every(ctx, "backup", time.Hour, r.nightlyBackup)
|
||||
r.every(ctx, "trash", time.Hour, r.emptyTrash)
|
||||
r.every(ctx, "sessions", 6*time.Hour, r.purgeSessions)
|
||||
// Ten minutes is a compromise: Gmail's push notifications would be instant but
|
||||
// need a public webhook and a Cloud Pub/Sub topic, which is a lot of Google
|
||||
// account for a to-do app. Ten minutes is fast enough for something somebody
|
||||
// starred and slow enough not to look like abuse.
|
||||
r.every(ctx, "gmail", 10*time.Minute, r.syncGmailAccounts)
|
||||
}
|
||||
|
||||
func (r *Runner) Wait() { r.wg.Wait() }
|
||||
@@ -214,6 +225,33 @@ func (r *Runner) rotateBackups() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- gmail ---------------------------------------------------------------------------
|
||||
|
||||
// syncGmailAccounts polls every connected mailbox.
|
||||
//
|
||||
// One at a time, and a failure on one account does not stop the others: a revoked
|
||||
// token or an expired refresh is a problem for that person alone.
|
||||
func (r *Runner) syncGmailAccounts(ctx context.Context) error {
|
||||
if r.SyncGmail == nil {
|
||||
return nil
|
||||
}
|
||||
users, err := r.db.UsersWithGmail(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, user := range users {
|
||||
created, err := r.SyncGmail(ctx, &user)
|
||||
if err != nil {
|
||||
r.log.Warn("gmail sync", "err", err, "user", user.ID)
|
||||
continue
|
||||
}
|
||||
if created > 0 {
|
||||
r.log.Info("tasks created from gmail", "user", user.ID, "count", created)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- housekeeping -----------------------------------------------------------------
|
||||
|
||||
func (r *Runner) emptyTrash(ctx context.Context) error {
|
||||
|
||||
@@ -270,6 +270,44 @@ func (db *DB) UserByMailToken(ctx context.Context, token string) (*User, error)
|
||||
return db.scanUser(ctx, `SELECT `+userColumns+` FROM users WHERE mail_token = ?`, token)
|
||||
}
|
||||
|
||||
// UsersWithGmail returns everybody who has connected a mailbox, which is who the
|
||||
// poller has to visit. Selected by the presence of a refresh token rather than by a
|
||||
// flag, so a disconnected account drops out of the sweep the moment its tokens are
|
||||
// cleared.
|
||||
func (db *DB) UsersWithGmail(ctx context.Context) ([]User, error) {
|
||||
rows, err := db.QueryContext(ctx, `
|
||||
SELECT u.id FROM users u
|
||||
JOIN user_settings s ON s.user_id = u.id
|
||||
WHERE s.scope = 'gmail' AND s.values_json LIKE '%"refresh_token":"%'`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var ids []string
|
||||
for rows.Next() {
|
||||
var id string
|
||||
if err := rows.Scan(&id); err != nil {
|
||||
rows.Close()
|
||||
return nil, err
|
||||
}
|
||||
ids = append(ids, id)
|
||||
}
|
||||
err = rows.Err()
|
||||
rows.Close()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
out := make([]User, 0, len(ids))
|
||||
for _, id := range ids {
|
||||
user, err := db.UserByID(ctx, id)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, *user)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// --- per-user settings ------------------------------------------------------------------
|
||||
|
||||
// UserSettings reads one scope of a person's integration settings. A missing row is
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
// Package update checks whether a newer verdande has been released.
|
||||
//
|
||||
// It asks GitHub's public releases API and nothing else. No identifier is sent, no
|
||||
// telemetry is collected, and the request carries only a user agent naming the
|
||||
// project — a self-hosted app that phones home with anything about its instance has
|
||||
// broken the deal its operator made by self-hosting.
|
||||
//
|
||||
// The check is off unless the operator turns it on, and the result is cached for
|
||||
// six hours so a busy instance does not become a burst of requests to GitHub.
|
||||
package update
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const releasesURL = "https://api.github.com/repos/kristianwind/verdande/releases/latest"
|
||||
|
||||
// Status is what the interface shows.
|
||||
type Status struct {
|
||||
Current string `json:"current"`
|
||||
Latest string `json:"latest,omitempty"`
|
||||
Available bool `json:"update_available"`
|
||||
URL string `json:"url,omitempty"`
|
||||
// Notes is the release's own description, so somebody can decide whether the
|
||||
// update is worth a restart tonight or next week.
|
||||
Notes string `json:"notes,omitempty"`
|
||||
CheckedAt string `json:"checked_at,omitempty"`
|
||||
// Disabled distinguishes "no update" from "we never looked", which the
|
||||
// interface has to show differently or it implies a guarantee it cannot make.
|
||||
Disabled bool `json:"disabled"`
|
||||
}
|
||||
|
||||
type Checker struct {
|
||||
current string
|
||||
enabled bool
|
||||
|
||||
mu sync.RWMutex
|
||||
cached Status
|
||||
fetched time.Time
|
||||
}
|
||||
|
||||
func New(currentVersion string, enabled bool) *Checker {
|
||||
return &Checker{
|
||||
current: currentVersion,
|
||||
enabled: enabled,
|
||||
cached: Status{Current: currentVersion, Disabled: !enabled},
|
||||
}
|
||||
}
|
||||
|
||||
// Status returns what is known, refreshing at most every six hours.
|
||||
func (c *Checker) Status(ctx context.Context) Status {
|
||||
if !c.enabled {
|
||||
return Status{Current: c.current, Disabled: true}
|
||||
}
|
||||
|
||||
c.mu.RLock()
|
||||
cached, fetched := c.cached, c.fetched
|
||||
c.mu.RUnlock()
|
||||
|
||||
if time.Since(fetched) < 6*time.Hour && fetched != (time.Time{}) {
|
||||
return cached
|
||||
}
|
||||
|
||||
status, err := c.fetch(ctx)
|
||||
if err != nil {
|
||||
// A failed check is not an error anybody needs to see. GitHub being
|
||||
// briefly unreachable is not a problem with this instance, and the last
|
||||
// known answer is better than a red banner.
|
||||
c.mu.Lock()
|
||||
// Backed off by marking it fetched, so a network outage does not mean a
|
||||
// request to GitHub on every page load.
|
||||
c.fetched = time.Now()
|
||||
result := c.cached
|
||||
c.mu.Unlock()
|
||||
return result
|
||||
}
|
||||
|
||||
c.mu.Lock()
|
||||
c.cached, c.fetched = status, time.Now()
|
||||
c.mu.Unlock()
|
||||
return status
|
||||
}
|
||||
|
||||
func (c *Checker) fetch(ctx context.Context) (Status, error) {
|
||||
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||
defer cancel()
|
||||
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, releasesURL, nil)
|
||||
if err != nil {
|
||||
return Status{}, err
|
||||
}
|
||||
req.Header.Set("Accept", "application/vnd.github+json")
|
||||
// The only thing sent about this instance is the project name. Not the base
|
||||
// URL, not the version, not a count of anything.
|
||||
req.Header.Set("User-Agent", "verdande")
|
||||
|
||||
resp, err := (&http.Client{Timeout: 10 * time.Second}).Do(req)
|
||||
if err != nil {
|
||||
return Status{}, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return Status{}, fmt.Errorf("update: github said %s", resp.Status)
|
||||
}
|
||||
|
||||
var release struct {
|
||||
TagName string `json:"tag_name"`
|
||||
HTMLURL string `json:"html_url"`
|
||||
Body string `json:"body"`
|
||||
Draft bool `json:"draft"`
|
||||
Prerelease bool `json:"prerelease"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&release); err != nil {
|
||||
return Status{}, err
|
||||
}
|
||||
if release.Draft || release.Prerelease {
|
||||
return Status{Current: c.current, CheckedAt: nowRFC3339()}, nil
|
||||
}
|
||||
|
||||
return Status{
|
||||
Current: c.current,
|
||||
Latest: release.TagName,
|
||||
Available: IsNewer(c.current, release.TagName),
|
||||
URL: release.HTMLURL,
|
||||
Notes: truncate(release.Body, 2000),
|
||||
CheckedAt: nowRFC3339(),
|
||||
}, nil
|
||||
}
|
||||
|
||||
// IsNewer compares two semver tags.
|
||||
//
|
||||
// A development build never reports an update. Somebody running `go run` against
|
||||
// their own working tree is ahead of every release, not behind it, and telling them
|
||||
// otherwise every time they open the app would train them to ignore the notice.
|
||||
func IsNewer(current, latest string) bool {
|
||||
if current == "" || latest == "" {
|
||||
return false
|
||||
}
|
||||
if current == "dev" || strings.Contains(current, "-dirty") {
|
||||
return false
|
||||
}
|
||||
|
||||
c := parseVersion(current)
|
||||
l := parseVersion(latest)
|
||||
if c == nil || l == nil {
|
||||
return false
|
||||
}
|
||||
|
||||
for i := 0; i < 3; i++ {
|
||||
if l[i] != c[i] {
|
||||
return l[i] > c[i]
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// parseVersion reads "v1.2.3" into its three numbers, ignoring any pre-release or
|
||||
// build suffix. Returns nil for anything that is not a version.
|
||||
func parseVersion(v string) []int {
|
||||
v = strings.TrimPrefix(strings.TrimSpace(v), "v")
|
||||
// Drop a pre-release or build suffix: 1.2.3-rc1 compares as 1.2.3, which is
|
||||
// close enough for "is there something newer" and avoids implementing the
|
||||
// whole precedence section of the semver spec for a notice in a corner.
|
||||
if i := strings.IndexAny(v, "-+"); i >= 0 {
|
||||
v = v[:i]
|
||||
}
|
||||
|
||||
parts := strings.Split(v, ".")
|
||||
if len(parts) != 3 {
|
||||
return nil
|
||||
}
|
||||
out := make([]int, 3)
|
||||
for i, p := range parts {
|
||||
n, err := strconv.Atoi(p)
|
||||
if err != nil || n < 0 {
|
||||
return nil
|
||||
}
|
||||
out[i] = n
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func nowRFC3339() string { return time.Now().UTC().Format(time.RFC3339) }
|
||||
|
||||
func truncate(s string, max int) string {
|
||||
s = strings.TrimSpace(s)
|
||||
if len(s) <= max {
|
||||
return s
|
||||
}
|
||||
return s[:max] + "…"
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package update
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestIsNewer(t *testing.T) {
|
||||
cases := []struct {
|
||||
current, latest string
|
||||
want bool
|
||||
}{
|
||||
{"v1.0.0", "v1.0.1", true},
|
||||
{"v1.0.0", "v1.1.0", true},
|
||||
{"v1.0.0", "v2.0.0", true},
|
||||
{"v1.2.3", "v1.2.3", false},
|
||||
{"v1.2.3", "v1.2.2", false},
|
||||
{"v2.0.0", "v1.9.9", false},
|
||||
// The comparison is numeric, not lexical: 10 is after 9.
|
||||
{"v1.9.0", "v1.10.0", true},
|
||||
{"v1.10.0", "v1.9.0", false},
|
||||
// Without the v.
|
||||
{"1.0.0", "1.0.1", true},
|
||||
// Pre-release suffixes compare as their base version.
|
||||
{"v1.0.0-rc1", "v1.0.0", false},
|
||||
{"v1.0.0", "v1.0.1-rc1", true},
|
||||
|
||||
// A development build is ahead of every release, not behind it. Telling
|
||||
// somebody running their own tree that they are out of date every time
|
||||
// they open the app teaches them to ignore the notice.
|
||||
{"dev", "v1.0.0", false},
|
||||
{"v1.0.0-dirty", "v2.0.0", false},
|
||||
|
||||
// Anything unparseable reports nothing rather than guessing.
|
||||
{"", "v1.0.0", false},
|
||||
{"v1.0.0", "", false},
|
||||
{"v1.0.0", "latest", false},
|
||||
{"v1.0", "v1.0.1", false},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.current+"→"+tc.latest, func(t *testing.T) {
|
||||
if got := IsNewer(tc.current, tc.latest); got != tc.want {
|
||||
t.Errorf("IsNewer(%q, %q) = %v, want %v", tc.current, tc.latest, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Off unless asked for, and when off it must not reach out at all.
|
||||
func TestDisabledCheckerNeverCallsOut(t *testing.T) {
|
||||
c := New("v1.0.0", false)
|
||||
|
||||
status := c.Status(context.Background())
|
||||
if !status.Disabled {
|
||||
t.Error("a disabled checker does not report itself as disabled")
|
||||
}
|
||||
if status.Available {
|
||||
t.Error("a disabled checker reported an update")
|
||||
}
|
||||
if status.Current != "v1.0.0" {
|
||||
t.Errorf("Current = %q", status.Current)
|
||||
}
|
||||
// CheckedAt stays empty, which is how the interface tells "no update" apart
|
||||
// from "we never looked".
|
||||
if status.CheckedAt != "" {
|
||||
t.Errorf("CheckedAt = %q on a disabled checker", status.CheckedAt)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user