BlueGit

timfa bluegit

Merged

Release: stage naar master

!44 · merged 3 weeks ago

timfa wants to merge 9 commits from stage into master

Pipeline

#179 Passed
Stage: build

Reviews

  • No reviews yet.

Merged by timfa on 24 October 2026 e594c80f

Overview Commits 9 Pipelines 1 Changes 42

Changes

42 files changed +2153 −65
cmd/bluegit/main.go +1 −0
@@ -107,6 +107,7 @@ func serve(args []string) error {
107 107 if m, ok := provider.(auth.Mailer); ok && cfg.Auth.ClientSecret != "" {
108 108 srv.Mail = m
109 109 }
110 srv.InitReadCache()
110 111 abandoned, err := st.FailAbandonedJobs(ctx)
111 112 if err != nil {
112 113 return err
@@ -107,6 +107,7 @@ func serve(args []string) error {
107 if m, ok := provider.(auth.Mailer); ok && cfg.Auth.ClientSecret != "" {
107 if m, ok := provider.(auth.Mailer); ok && cfg.Auth.ClientSecret != "" {
108 srv.Mail = m
108 srv.Mail = m
109 }
109 }
110 srv.InitReadCache()
110 abandoned, err := st.FailAbandonedJobs(ctx)
111 abandoned, err := st.FailAbandonedJobs(ctx)
111 if err != nil {
112 if err != nil {
112 return err
113 return err
docs/agents.md added +90 −0
@@ -0,0 +1,90 @@
1 # BlueGit voor agents en scripts
2
3 `scripts/bg` is een kleine CLI op de BlueGit-API. Hij heeft alleen `bash`, `curl` en `jq` nodig.
4
5 ```sh
6 export BLUEGIT_URL=https://git.timfalken.com # standaard
7 export BLUEGIT_TOKEN=... # optioneel, nooit printen
8 scripts/bg help
9 ```
10
11 ## Rechten
12
13 - **Publieke projecten** zijn zonder token leesbaar, net als in de UI. Dat geldt voor status, pipelines, jobs, logs, MR's, issues, bestanden en releases.
14 - **Privé projecten**, en projecten in een privé groep, geven `404` zonder token of als je geen lid bent. Je ziet dus ook niet of zo'n project bestaat.
15 - **Schrijven** (MR, notitie, issue, branch, release, retry) vraagt altijd een token, met de rol die de UI ook vraagt. Zonder token krijg je `401`, met te weinig rechten `403`.
16 - **Een token** werkt met de rechten van de eigenaar, beperkt door de scopes van het token (`read_api`, `api`, …).
17
18 `bg` zet fouten op stderr met de HTTP-code en eindigt met exit 1. Een 404 kan dus ook "geen toegang" betekenen.
19
20 ## Pipelines
21
22 ```sh
23 bg pipeline status timfa/bluegit --branch stage # nieuwste pipeline op een branch
24 bg pipeline status timfa/bluegit --sha 3e6b2f1 # op commit (prefix van 7+ tekens)
25 bg pipeline status timfa/bluegit --mr 40 # van MR 40 (incl. pushes naar de bronbranch)
26 bg pipeline status timfa/bluegit --id 158
27
28 bg pipeline wait timfa/bluegit --sha "$(git rev-parse HEAD)" --timeout 1200 --interval 10
29 bg pipeline logs timfa/bluegit 158 611
30 bg pipeline retry timfa/bluegit 158 # alleen mislukte jobs (+ wat daarop wachtte)
31 bg pipeline retry timfa/bluegit 158 --all # nieuwe pipeline
32 ```
33
34 `wait` wacht eerst tot er voor de selectie een pipeline is (handig direct na een push) en daarna tot die klaar is. De exitcodes:
35
36 | exit | betekenis |
37 |---|---|
38 | 0 | success |
39 | 1 | failed of canceled; de laatste 80 regels van elke mislukte job staan op stderr (`BG_LOG_LINES` om aan te passen) |
40 | 2 | timeout |
41
42 Jobs met `allow_failure` die faalden geven een waarschuwing, maar het blijft exit 0.
43
44 Typisch voor een cloud-agent na een push:
45
46 ```sh
47 git push bluegit HEAD:refs/heads/mijn-branch
48 sha=$(git rev-parse HEAD)
49 if scripts/bg pipeline wait timfa/bluegit --sha "$sha" --timeout 1800; then
50 echo groen
51 else
52 echo "rood of timeout ($?)"; exit 1
53 fi
54 ```
55
56 ## Merge requests
57
58 ```sh
59 bg mr list timfa/bluegit # open
60 bg mr list timfa/bluegit --state all --source mijn-branch
61 bg mr create timfa/bluegit --source mijn-branch --target stage --title "fix: ..." --description "..."
62 bg mr show timfa/bluegit 40
63 bg mr comment timfa/bluegit 40 "Ziet er goed uit"
64 bg mr merge timfa/bluegit 40 # verwijdert de bronbranch; --keep-source om te houden
65 ```
66
67 ## Issues, branches, releases, bestanden
68
69 ```sh
70 bg issue list timfa/bluegit [open|closed|all]
71 bg issue create timfa/bluegit "Titel" "Beschrijving"
72 bg issue comment timfa/bluegit 7 "Opgelost in !40"
73 bg issue close timfa/bluegit 7
74
75 bg branch create timfa/bluegit mijn-branch --from stage
76 bg release create timfa/bluegit v1.2.0 --commit <sha> --asset build:bluegit:bluegit-linux-amd64
77 bg file get timfa/bluegit README.md --ref stage
78 ```
79
80 Een release-asset komt uit een artifact van een job in dezelfde repo (`JOB:PAD[:NAAM]`). Een los bestand uploaden kan de API nog niet.
81
82 ## Ruwe API
83
84 De CLI is dun. Alles staat ook in `/api/v1/openapi.json`. Handige filters:
85
86 - `GET /api/v1/projects/{owner}/{repo}/pipelines?sha=…|ref=…|mr=…`
87 - `GET /api/v1/projects/{owner}/{repo}/merge_requests?state=open|merged|closed|all&source_branch=…&target_branch=…`
88 - `GET /api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/log`
89
90 Owner en repo zijn twee padsegmenten (`timfa/bluegit`), niet urlencoded.
@@ -0,0 +1,90 @@
1# BlueGit voor agents en scripts
2
3`scripts/bg` is een kleine CLI op de BlueGit-API. Hij heeft alleen `bash`, `curl` en `jq` nodig.
4
5```sh
6export BLUEGIT_URL=https://git.timfalken.com # standaard
7export BLUEGIT_TOKEN=... # optioneel, nooit printen
8scripts/bg help
9```
10
11## Rechten
12
13- **Publieke projecten** zijn zonder token leesbaar, net als in de UI. Dat geldt voor status, pipelines, jobs, logs, MR's, issues, bestanden en releases.
14- **Privé projecten**, en projecten in een privé groep, geven `404` zonder token of als je geen lid bent. Je ziet dus ook niet of zo'n project bestaat.
15- **Schrijven** (MR, notitie, issue, branch, release, retry) vraagt altijd een token, met de rol die de UI ook vraagt. Zonder token krijg je `401`, met te weinig rechten `403`.
16- **Een token** werkt met de rechten van de eigenaar, beperkt door de scopes van het token (`read_api`, `api`, …).
17
18`bg` zet fouten op stderr met de HTTP-code en eindigt met exit 1. Een 404 kan dus ook "geen toegang" betekenen.
19
20## Pipelines
21
22```sh
23bg pipeline status timfa/bluegit --branch stage # nieuwste pipeline op een branch
24bg pipeline status timfa/bluegit --sha 3e6b2f1 # op commit (prefix van 7+ tekens)
25bg pipeline status timfa/bluegit --mr 40 # van MR 40 (incl. pushes naar de bronbranch)
26bg pipeline status timfa/bluegit --id 158
27
28bg pipeline wait timfa/bluegit --sha "$(git rev-parse HEAD)" --timeout 1200 --interval 10
29bg pipeline logs timfa/bluegit 158 611
30bg pipeline retry timfa/bluegit 158 # alleen mislukte jobs (+ wat daarop wachtte)
31bg pipeline retry timfa/bluegit 158 --all # nieuwe pipeline
32```
33
34`wait` wacht eerst tot er voor de selectie een pipeline is (handig direct na een push) en daarna tot die klaar is. De exitcodes:
35
36| exit | betekenis |
37|---|---|
38| 0 | success |
39| 1 | failed of canceled; de laatste 80 regels van elke mislukte job staan op stderr (`BG_LOG_LINES` om aan te passen) |
40| 2 | timeout |
41
42Jobs met `allow_failure` die faalden geven een waarschuwing, maar het blijft exit 0.
43
44Typisch voor een cloud-agent na een push:
45
46```sh
47git push bluegit HEAD:refs/heads/mijn-branch
48sha=$(git rev-parse HEAD)
49if scripts/bg pipeline wait timfa/bluegit --sha "$sha" --timeout 1800; then
50 echo groen
51else
52 echo "rood of timeout ($?)"; exit 1
53fi
54```
55
56## Merge requests
57
58```sh
59bg mr list timfa/bluegit # open
60bg mr list timfa/bluegit --state all --source mijn-branch
61bg mr create timfa/bluegit --source mijn-branch --target stage --title "fix: ..." --description "..."
62bg mr show timfa/bluegit 40
63bg mr comment timfa/bluegit 40 "Ziet er goed uit"
64bg mr merge timfa/bluegit 40 # verwijdert de bronbranch; --keep-source om te houden
65```
66
67## Issues, branches, releases, bestanden
68
69```sh
70bg issue list timfa/bluegit [open|closed|all]
71bg issue create timfa/bluegit "Titel" "Beschrijving"
72bg issue comment timfa/bluegit 7 "Opgelost in !40"
73bg issue close timfa/bluegit 7
74
75bg branch create timfa/bluegit mijn-branch --from stage
76bg release create timfa/bluegit v1.2.0 --commit <sha> --asset build:bluegit:bluegit-linux-amd64
77bg file get timfa/bluegit README.md --ref stage
78```
79
80Een release-asset komt uit een artifact van een job in dezelfde repo (`JOB:PAD[:NAAM]`). Een los bestand uploaden kan de API nog niet.
81
82## Ruwe API
83
84De CLI is dun. Alles staat ook in `/api/v1/openapi.json`. Handige filters:
85
86- `GET /api/v1/projects/{owner}/{repo}/pipelines?sha=…|ref=…|mr=…`
87- `GET /api/v1/projects/{owner}/{repo}/merge_requests?state=open|merged|closed|all&source_branch=…&target_branch=…`
88- `GET /api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/log`
89
90Owner en repo zijn twee padsegmenten (`timfa/bluegit`), niet urlencoded.
internal/ci/eval.go +7 −2
@@ -116,6 +116,9 @@ func (p Pipeline) Select(ev Eval) ([]Job, error) {
116 116 if ev.Env == nil {
117 117 ev.Env = map[string]string{}
118 118 }
119 if len(p.Workflow) > 0 && ev.Env["CI_PIPELINE_SOURCE"] == "merge_request_event" {
120 ev.workflowMR, _ = p.Allow(ev)
121 }
119 122 var out []Job
120 123 active := map[string]bool{}
121 124 var pending []Job
@@ -260,8 +263,10 @@ func onlyExcept(j Job, ev Eval) (bool, error) {
260 263 mr := ev.Env["CI_PIPELINE_SOURCE"] == "merge_request_event"
261 264 if len(j.Only) == 0 && len(j.Except) == 0 {
262 265 // Zonder rules of only draait een job op een branch of tag,
263 // niet in een merge-request-pipeline.
264 if mr {
266 // niet in een merge-request-pipeline. Behalve als workflow:rules
267 // merge-request-pipelines expliciet toestaat: dan is een MR-pipeline
268 // zonder de gewone jobs nooit de bedoeling (anders ontstaat er geen).
269 if mr && !ev.workflowMR {
265 270 return false, nil
266 271 }
267 272 return true, nil
@@ -116,6 +116,9 @@ func (p Pipeline) Select(ev Eval) ([]Job, error) {
116 if ev.Env == nil {
116 if ev.Env == nil {
117 ev.Env = map[string]string{}
117 ev.Env = map[string]string{}
118 }
118 }
119 if len(p.Workflow) > 0 && ev.Env["CI_PIPELINE_SOURCE"] == "merge_request_event" {
120 ev.workflowMR, _ = p.Allow(ev)
121 }
119 var out []Job
122 var out []Job
120 active := map[string]bool{}
123 active := map[string]bool{}
121 var pending []Job
124 var pending []Job
@@ -260,8 +263,10 @@ func onlyExcept(j Job, ev Eval) (bool, error) {
260 mr := ev.Env["CI_PIPELINE_SOURCE"] == "merge_request_event"
263 mr := ev.Env["CI_PIPELINE_SOURCE"] == "merge_request_event"
261 if len(j.Only) == 0 && len(j.Except) == 0 {
264 if len(j.Only) == 0 && len(j.Except) == 0 {
262 // Zonder rules of only draait een job op een branch of tag,
265 // Zonder rules of only draait een job op een branch of tag,
263 // niet in een merge-request-pipeline.
266 // niet in een merge-request-pipeline. Behalve als workflow:rules
264 if mr {
267 // merge-request-pipelines expliciet toestaat: dan is een MR-pipeline
268 // zonder de gewone jobs nooit de bedoeling (anders ontstaat er geen).
269 if mr && !ev.workflowMR {
265 return false, nil
270 return false, nil
266 }
271 }
267 return true, nil
272 return true, nil
internal/ci/legacy_cycles_test.go added +92 −0
@@ -0,0 +1,92 @@
1 package ci
2
3 import (
4 "os"
5 "reflect"
6 "testing"
7 )
8
9 const lcYML = `stages: [test, build]
10 workflow:
11 rules:
12 - if: $CI_PIPELINE_SOURCE == "merge_request_event"
13 - if: $CI_COMMIT_BRANCH == "master"
14 - if: $CI_PIPELINE_SOURCE == "web"
15 - if: $CI_PIPELINE_SOURCE == "api"
16 .master:
17 rules:
18 - if: $CI_PIPELINE_SOURCE == "merge_request_event"
19 when: never
20 - if: $CI_COMMIT_BRANCH == "master"
21 .rust:
22 image: rust
23 test:
24 extends: .rust
25 stage: test
26 script: [cargo test]
27 build:linux:
28 extends: .rust
29 stage: build
30 script: [build]
31 build:windows:
32 extends: [.rust, .master]
33 stage: build
34 script: [build]
35 `
36
37 func lcJobs(t *testing.T, yml, src, branch string) []string {
38 t.Helper()
39 spec, err := ParseIn([]byte(yml), func(string) ([]byte, error) { return nil, os.ErrNotExist })
40 if err != nil {
41 t.Fatal(err)
42 }
43 var mr *MRInfo
44 if src == "merge_request_event" {
45 mr = &MRInfo{ID: 1, IID: 1, SourceBranch: branch, TargetBranch: "master"}
46 }
47 ev := Eval{Env: Variables(Ref{SHA: "abc", FullRef: "refs/heads/" + branch, Name: branch, Kind: "branch", DefaultBranch: "master", Source: src, MR: mr})}
48 ok, err := spec.Allow(ev)
49 if err != nil {
50 t.Fatal(err)
51 }
52 if !ok {
53 return nil
54 }
55 js, err := spec.Select(ev)
56 if err != nil {
57 t.Fatal(err)
58 }
59 names := []string{}
60 for _, j := range js {
61 names = append(names, j.Name)
62 }
63 return names
64 }
65
66 func TestLegacyCyclesPipelines(t *testing.T) {
67 cases := []struct {
68 src, branch string
69 want []string
70 }{
71 // workflow staat MR-pipelines toe: jobs zonder rules draaien mee.
72 {"merge_request_event", "ada/x", []string{"test", "build:linux"}},
73 // geërfde rules (extends .master) gelden ook bij api en web.
74 {"api", "ada/x", []string{"test", "build:linux"}},
75 {"web", "ada/x", []string{"test", "build:linux"}},
76 {"push", "ada/x", nil},
77 {"push", "master", []string{"test", "build:linux", "build:windows"}},
78 {"api", "master", []string{"test", "build:linux", "build:windows"}},
79 }
80 for _, c := range cases {
81 if got := lcJobs(t, lcYML, c.src, c.branch); !reflect.DeepEqual(got, c.want) && !(len(got) == 0 && len(c.want) == 0) {
82 t.Errorf("%s op %s: %v, wil %v", c.src, c.branch, got, c.want)
83 }
84 }
85 }
86
87 func TestPlainJobsSkipMRWithoutWorkflow(t *testing.T) {
88 yml := "test:\n image: alpine\n script: [x]\n"
89 if got := lcJobs(t, yml, "merge_request_event", "ada/x"); len(got) != 0 {
90 t.Fatalf("zonder workflow hoort een gewone job niet in een MR-pipeline (GitLab): %v", got)
91 }
92 }
@@ -0,0 +1,92 @@
1package ci
2
3import (
4 "os"
5 "reflect"
6 "testing"
7)
8
9const lcYML = `stages: [test, build]
10workflow:
11 rules:
12 - if: $CI_PIPELINE_SOURCE == "merge_request_event"
13 - if: $CI_COMMIT_BRANCH == "master"
14 - if: $CI_PIPELINE_SOURCE == "web"
15 - if: $CI_PIPELINE_SOURCE == "api"
16.master:
17 rules:
18 - if: $CI_PIPELINE_SOURCE == "merge_request_event"
19 when: never
20 - if: $CI_COMMIT_BRANCH == "master"
21.rust:
22 image: rust
23test:
24 extends: .rust
25 stage: test
26 script: [cargo test]
27build:linux:
28 extends: .rust
29 stage: build
30 script: [build]
31build:windows:
32 extends: [.rust, .master]
33 stage: build
34 script: [build]
35`
36
37func lcJobs(t *testing.T, yml, src, branch string) []string {
38 t.Helper()
39 spec, err := ParseIn([]byte(yml), func(string) ([]byte, error) { return nil, os.ErrNotExist })
40 if err != nil {
41 t.Fatal(err)
42 }
43 var mr *MRInfo
44 if src == "merge_request_event" {
45 mr = &MRInfo{ID: 1, IID: 1, SourceBranch: branch, TargetBranch: "master"}
46 }
47 ev := Eval{Env: Variables(Ref{SHA: "abc", FullRef: "refs/heads/" + branch, Name: branch, Kind: "branch", DefaultBranch: "master", Source: src, MR: mr})}
48 ok, err := spec.Allow(ev)
49 if err != nil {
50 t.Fatal(err)
51 }
52 if !ok {
53 return nil
54 }
55 js, err := spec.Select(ev)
56 if err != nil {
57 t.Fatal(err)
58 }
59 names := []string{}
60 for _, j := range js {
61 names = append(names, j.Name)
62 }
63 return names
64}
65
66func TestLegacyCyclesPipelines(t *testing.T) {
67 cases := []struct {
68 src, branch string
69 want []string
70 }{
71 // workflow staat MR-pipelines toe: jobs zonder rules draaien mee.
72 {"merge_request_event", "ada/x", []string{"test", "build:linux"}},
73 // geërfde rules (extends .master) gelden ook bij api en web.
74 {"api", "ada/x", []string{"test", "build:linux"}},
75 {"web", "ada/x", []string{"test", "build:linux"}},
76 {"push", "ada/x", nil},
77 {"push", "master", []string{"test", "build:linux", "build:windows"}},
78 {"api", "master", []string{"test", "build:linux", "build:windows"}},
79 }
80 for _, c := range cases {
81 if got := lcJobs(t, lcYML, c.src, c.branch); !reflect.DeepEqual(got, c.want) && !(len(got) == 0 && len(c.want) == 0) {
82 t.Errorf("%s op %s: %v, wil %v", c.src, c.branch, got, c.want)
83 }
84 }
85}
86
87func TestPlainJobsSkipMRWithoutWorkflow(t *testing.T) {
88 yml := "test:\n image: alpine\n script: [x]\n"
89 if got := lcJobs(t, yml, "merge_request_event", "ada/x"); len(got) != 0 {
90 t.Fatalf("zonder workflow hoort een gewone job niet in een MR-pipeline (GitLab): %v", got)
91 }
92}
internal/ci/model.go +3 −0
@@ -111,6 +111,9 @@ type Eval struct {
111 111 Env map[string]string
112 112 Changed []string
113 113 Files []string
114 // workflowMR: workflow:rules staat merge-request-pipelines expliciet
115 // toe. Jobs zonder rules of only draaien dan ook daarin.
116 workflowMR bool
114 117 }
115 118
116 119 // MRInfo is de merge request waar een pipeline bij hoort.
@@ -111,6 +111,9 @@ type Eval struct {
111 Env map[string]string
111 Env map[string]string
112 Changed []string
112 Changed []string
113 Files []string
113 Files []string
114 // workflowMR: workflow:rules staat merge-request-pipelines expliciet
115 // toe. Jobs zonder rules of only draaien dan ook daarin.
116 workflowMR bool
114}
117}
115
118
116// MRInfo is de merge request waar een pipeline bij hoort.
119// MRInfo is de merge request waar een pipeline bij hoort.
internal/config/config.go +8 −1
@@ -24,7 +24,11 @@ type Config struct {
24 24 StatePath string `yaml:"state_path"`
25 25 // StoragePath is optioneel. Leeg betekent dat de beheerder het pad
26 26 // in de webinterface kiest. Er is geen standaardpad.
27 StoragePath string `yaml:"storage_path"`
27 StoragePath string `yaml:"storage_path"`
28 // CachePath is de optionele leescache (bij voorkeur op een SSD).
29 // Leeg betekent geen cache. Beheer kan het overschrijven.
30 CachePath string `yaml:"cache_path"`
31 CacheMaxMB int64 `yaml:"cache_max_mb"`
28 32 TrustedProxies []string `yaml:"trusted_proxies"`
29 33 // BehindProxy is het oude veld. Het wordt geweigerd: één boolean
30 34 // vertrouwt ofwel iedereen, ofwel niemand. De lijst hierboven beslist.
@@ -120,6 +124,9 @@ func Load(path string) (Config, error) {
120 124 if v := os.Getenv("BLUEGIT_STORAGE_PATH"); v != "" {
121 125 c.StoragePath = v
122 126 }
127 if v := os.Getenv("BLUEGIT_CACHE_PATH"); v != "" {
128 c.CachePath = v
129 }
123 130 if v := os.Getenv("BLUEGIT_LISTEN"); v != "" {
124 131 c.Listen = v
125 132 }
@@ -24,7 +24,11 @@ type Config struct {
24 StatePath string `yaml:"state_path"`
24 StatePath string `yaml:"state_path"`
25 // StoragePath is optioneel. Leeg betekent dat de beheerder het pad
25 // StoragePath is optioneel. Leeg betekent dat de beheerder het pad
26 // in de webinterface kiest. Er is geen standaardpad.
26 // in de webinterface kiest. Er is geen standaardpad.
27 StoragePath string `yaml:"storage_path"`
27 StoragePath string `yaml:"storage_path"`
28 // CachePath is de optionele leescache (bij voorkeur op een SSD).
29 // Leeg betekent geen cache. Beheer kan het overschrijven.
30 CachePath string `yaml:"cache_path"`
31 CacheMaxMB int64 `yaml:"cache_max_mb"`
28 TrustedProxies []string `yaml:"trusted_proxies"`
32 TrustedProxies []string `yaml:"trusted_proxies"`
29 // BehindProxy is het oude veld. Het wordt geweigerd: één boolean
33 // BehindProxy is het oude veld. Het wordt geweigerd: één boolean
30 // vertrouwt ofwel iedereen, ofwel niemand. De lijst hierboven beslist.
34 // vertrouwt ofwel iedereen, ofwel niemand. De lijst hierboven beslist.
@@ -120,6 +124,9 @@ func Load(path string) (Config, error) {
120 if v := os.Getenv("BLUEGIT_STORAGE_PATH"); v != "" {
124 if v := os.Getenv("BLUEGIT_STORAGE_PATH"); v != "" {
121 c.StoragePath = v
125 c.StoragePath = v
122 }
126 }
127 if v := os.Getenv("BLUEGIT_CACHE_PATH"); v != "" {
128 c.CachePath = v
129 }
123 if v := os.Getenv("BLUEGIT_LISTEN"); v != "" {
130 if v := os.Getenv("BLUEGIT_LISTEN"); v != "" {
124 c.Listen = v
131 c.Listen = v
125 }
132 }
internal/i18n/catalog_cache.go added +72 −0
@@ -0,0 +1,72 @@
1 package i18n
2
3 func init() {
4 addCatalog(en, cacheEN)
5 addCatalog(nl, cacheNL)
6 addCatalog(de, cacheDE)
7 addCatalog(fr, cacheFR)
8 }
9
10 var cacheEN = map[string]string{
11 "setup.cacheTitle": "Read cache",
12 "setup.cacheLead": "Optional. Overview pages (README, file lists, branches, tags, counts) are kept here so they don't have to be read from the repository disk. Pick a path on a fast disk; empty turns the cache off. Only the cache/ subfolder is ever cleared.",
13 "setup.cacheUse": "In use: %s of %s, %d entries (hits %d, misses %d since start).",
14 "setup.cacheOff": "The cache is off.",
15 "setup.cachePath": "Cache path",
16 "setup.cacheMax": "Maximum size (MB, 0 = 512)",
17 "setup.cacheClear": "Clear cache",
18 "setup.cacheSaved": "Cache settings saved.",
19 "setup.cacheCleared": "Cache cleared.",
20 "setup.locTitle": "What is stored where",
21 "err.cacheAbs": "The cache path must be absolute.",
22 "err.cacheInside": "The cache path may not be inside the storage or state folder.",
23 "err.cacheDir": "The cache folder is not writable: %s",
24 }
25
26 var cacheNL = map[string]string{
27 "setup.cacheTitle": "Leescache",
28 "setup.cacheLead": "Optioneel. Overzichtspagina's (README, bestandslijsten, branches, tags, aantallen) worden hier bewaard, zodat ze niet van de repository-schijf hoeven te komen. Kies een pad op een snelle schijf; leeg zet de cache uit. Alleen de submap cache/ wordt ooit geleegd.",
29 "setup.cacheUse": "In gebruik: %s van %s, %d items (%d treffers, %d missers sinds de start).",
30 "setup.cacheOff": "De cache staat uit.",
31 "setup.cachePath": "Cachepad",
32 "setup.cacheMax": "Maximale grootte (MB, 0 = 512)",
33 "setup.cacheClear": "Cache legen",
34 "setup.cacheSaved": "Cache-instellingen opgeslagen.",
35 "setup.cacheCleared": "Cache geleegd.",
36 "setup.locTitle": "Wat staat waar",
37 "err.cacheAbs": "Het cachepad moet absoluut zijn.",
38 "err.cacheInside": "Het cachepad mag niet in de opslag- of state-map liggen.",
39 "err.cacheDir": "De cachemap is niet beschrijfbaar: %s",
40 }
41
42 var cacheDE = map[string]string{
43 "setup.cacheTitle": "Lese-Cache",
44 "setup.cacheLead": "Optional. Übersichtsseiten (README, Dateilisten, Branches, Tags, Zähler) werden hier abgelegt, damit sie nicht von der Repository-Platte gelesen werden müssen. Wähle einen Pfad auf einer schnellen Platte; leer schaltet den Cache aus. Geleert wird nur der Unterordner cache/.",
45 "setup.cacheUse": "Belegt: %s von %s, %d Einträge (%d Treffer, %d Fehlgriffe seit dem Start).",
46 "setup.cacheOff": "Der Cache ist aus.",
47 "setup.cachePath": "Cache-Pfad",
48 "setup.cacheMax": "Maximale Größe (MB, 0 = 512)",
49 "setup.cacheClear": "Cache leeren",
50 "setup.cacheSaved": "Cache-Einstellungen gespeichert.",
51 "setup.cacheCleared": "Cache geleert.",
52 "setup.locTitle": "Was liegt wo",
53 "err.cacheAbs": "Der Cache-Pfad muss absolut sein.",
54 "err.cacheInside": "Der Cache-Pfad darf nicht im Speicher- oder State-Ordner liegen.",
55 "err.cacheDir": "Der Cache-Ordner ist nicht beschreibbar: %s",
56 }
57
58 var cacheFR = map[string]string{
59 "setup.cacheTitle": "Cache de lecture",
60 "setup.cacheLead": "Facultatif. Les pages de synthèse (README, listes de fichiers, branches, tags, compteurs) sont conservées ici pour éviter de lire le disque des dépôts. Choisissez un chemin sur un disque rapide ; vide désactive le cache. Seul le sous-dossier cache/ est vidé.",
61 "setup.cacheUse": "Utilisé : %s sur %s, %d entrées (%d succès, %d échecs depuis le démarrage).",
62 "setup.cacheOff": "Le cache est désactivé.",
63 "setup.cachePath": "Chemin du cache",
64 "setup.cacheMax": "Taille maximale (Mo, 0 = 512)",
65 "setup.cacheClear": "Vider le cache",
66 "setup.cacheSaved": "Réglages du cache enregistrés.",
67 "setup.cacheCleared": "Cache vidé.",
68 "setup.locTitle": "Où se trouve quoi",
69 "err.cacheAbs": "Le chemin du cache doit être absolu.",
70 "err.cacheInside": "Le chemin du cache ne peut pas se trouver dans le dossier de stockage ou d'état.",
71 "err.cacheDir": "Le dossier du cache n'est pas accessible en écriture : %s",
72 }
@@ -0,0 +1,72 @@
1package i18n
2
3func init() {
4 addCatalog(en, cacheEN)
5 addCatalog(nl, cacheNL)
6 addCatalog(de, cacheDE)
7 addCatalog(fr, cacheFR)
8}
9
10var cacheEN = map[string]string{
11 "setup.cacheTitle": "Read cache",
12 "setup.cacheLead": "Optional. Overview pages (README, file lists, branches, tags, counts) are kept here so they don't have to be read from the repository disk. Pick a path on a fast disk; empty turns the cache off. Only the cache/ subfolder is ever cleared.",
13 "setup.cacheUse": "In use: %s of %s, %d entries (hits %d, misses %d since start).",
14 "setup.cacheOff": "The cache is off.",
15 "setup.cachePath": "Cache path",
16 "setup.cacheMax": "Maximum size (MB, 0 = 512)",
17 "setup.cacheClear": "Clear cache",
18 "setup.cacheSaved": "Cache settings saved.",
19 "setup.cacheCleared": "Cache cleared.",
20 "setup.locTitle": "What is stored where",
21 "err.cacheAbs": "The cache path must be absolute.",
22 "err.cacheInside": "The cache path may not be inside the storage or state folder.",
23 "err.cacheDir": "The cache folder is not writable: %s",
24}
25
26var cacheNL = map[string]string{
27 "setup.cacheTitle": "Leescache",
28 "setup.cacheLead": "Optioneel. Overzichtspagina's (README, bestandslijsten, branches, tags, aantallen) worden hier bewaard, zodat ze niet van de repository-schijf hoeven te komen. Kies een pad op een snelle schijf; leeg zet de cache uit. Alleen de submap cache/ wordt ooit geleegd.",
29 "setup.cacheUse": "In gebruik: %s van %s, %d items (%d treffers, %d missers sinds de start).",
30 "setup.cacheOff": "De cache staat uit.",
31 "setup.cachePath": "Cachepad",
32 "setup.cacheMax": "Maximale grootte (MB, 0 = 512)",
33 "setup.cacheClear": "Cache legen",
34 "setup.cacheSaved": "Cache-instellingen opgeslagen.",
35 "setup.cacheCleared": "Cache geleegd.",
36 "setup.locTitle": "Wat staat waar",
37 "err.cacheAbs": "Het cachepad moet absoluut zijn.",
38 "err.cacheInside": "Het cachepad mag niet in de opslag- of state-map liggen.",
39 "err.cacheDir": "De cachemap is niet beschrijfbaar: %s",
40}
41
42var cacheDE = map[string]string{
43 "setup.cacheTitle": "Lese-Cache",
44 "setup.cacheLead": "Optional. Übersichtsseiten (README, Dateilisten, Branches, Tags, Zähler) werden hier abgelegt, damit sie nicht von der Repository-Platte gelesen werden müssen. Wähle einen Pfad auf einer schnellen Platte; leer schaltet den Cache aus. Geleert wird nur der Unterordner cache/.",
45 "setup.cacheUse": "Belegt: %s von %s, %d Einträge (%d Treffer, %d Fehlgriffe seit dem Start).",
46 "setup.cacheOff": "Der Cache ist aus.",
47 "setup.cachePath": "Cache-Pfad",
48 "setup.cacheMax": "Maximale Größe (MB, 0 = 512)",
49 "setup.cacheClear": "Cache leeren",
50 "setup.cacheSaved": "Cache-Einstellungen gespeichert.",
51 "setup.cacheCleared": "Cache geleert.",
52 "setup.locTitle": "Was liegt wo",
53 "err.cacheAbs": "Der Cache-Pfad muss absolut sein.",
54 "err.cacheInside": "Der Cache-Pfad darf nicht im Speicher- oder State-Ordner liegen.",
55 "err.cacheDir": "Der Cache-Ordner ist nicht beschreibbar: %s",
56}
57
58var cacheFR = map[string]string{
59 "setup.cacheTitle": "Cache de lecture",
60 "setup.cacheLead": "Facultatif. Les pages de synthèse (README, listes de fichiers, branches, tags, compteurs) sont conservées ici pour éviter de lire le disque des dépôts. Choisissez un chemin sur un disque rapide ; vide désactive le cache. Seul le sous-dossier cache/ est vidé.",
61 "setup.cacheUse": "Utilisé : %s sur %s, %d entrées (%d succès, %d échecs depuis le démarrage).",
62 "setup.cacheOff": "Le cache est désactivé.",
63 "setup.cachePath": "Chemin du cache",
64 "setup.cacheMax": "Taille maximale (Mo, 0 = 512)",
65 "setup.cacheClear": "Vider le cache",
66 "setup.cacheSaved": "Réglages du cache enregistrés.",
67 "setup.cacheCleared": "Cache vidé.",
68 "setup.locTitle": "Où se trouve quoi",
69 "err.cacheAbs": "Le chemin du cache doit être absolu.",
70 "err.cacheInside": "Le chemin du cache ne peut pas se trouver dans le dossier de stockage ou d'état.",
71 "err.cacheDir": "Le dossier du cache n'est pas accessible en écriture : %s",
72}
internal/i18n/catalog_hookev.go +8 −0
@@ -8,6 +8,8 @@ func init() {
8 8 }
9 9
10 10 var hookEvEN = map[string]string{
11 "err.badSHA": "Give sha as 7 to 40 hex characters.",
12 "err.pipeFilter": "Use either ref or mr, not both.",
11 13 "set.ev.issue_opened": "issue opened",
12 14 "set.ev.issue_closed": "issue closed",
13 15 "set.ev.issue_reopened": "issue reopened",
@@ -17,6 +19,8 @@ var hookEvEN = map[string]string{
17 19 }
18 20
19 21 var hookEvNL = map[string]string{
22 "err.badSHA": "Geef sha als 7 tot 40 hextekens.",
23 "err.pipeFilter": "Gebruik ref of mr, niet allebei.",
20 24 "set.ev.issue_opened": "issue geopend",
21 25 "set.ev.issue_closed": "issue gesloten",
22 26 "set.ev.issue_reopened": "issue heropend",
@@ -26,6 +30,8 @@ var hookEvNL = map[string]string{
26 30 }
27 31
28 32 var hookEvDE = map[string]string{
33 "err.badSHA": "Gib sha als 7 bis 40 Hexzeichen an.",
34 "err.pipeFilter": "Nutze ref oder mr, nicht beides.",
29 35 "set.ev.issue_opened": "Issue eröffnet",
30 36 "set.ev.issue_closed": "Issue geschlossen",
31 37 "set.ev.issue_reopened": "Issue wieder geöffnet",
@@ -35,6 +41,8 @@ var hookEvDE = map[string]string{
35 41 }
36 42
37 43 var hookEvFR = map[string]string{
44 "err.badSHA": "Donnez sha en 7 à 40 caractères hexadécimaux.",
45 "err.pipeFilter": "Utilisez ref ou mr, pas les deux.",
38 46 "set.ev.issue_opened": "ticket ouvert",
39 47 "set.ev.issue_closed": "ticket fermé",
40 48 "set.ev.issue_reopened": "ticket rouvert",
@@ -8,6 +8,8 @@ func init() {
8}
8}
9
9
10var hookEvEN = map[string]string{
10var hookEvEN = map[string]string{
11 "err.badSHA": "Give sha as 7 to 40 hex characters.",
12 "err.pipeFilter": "Use either ref or mr, not both.",
11 "set.ev.issue_opened": "issue opened",
13 "set.ev.issue_opened": "issue opened",
12 "set.ev.issue_closed": "issue closed",
14 "set.ev.issue_closed": "issue closed",
13 "set.ev.issue_reopened": "issue reopened",
15 "set.ev.issue_reopened": "issue reopened",
@@ -17,6 +19,8 @@ var hookEvEN = map[string]string{
17}
19}
18
20
19var hookEvNL = map[string]string{
21var hookEvNL = map[string]string{
22 "err.badSHA": "Geef sha als 7 tot 40 hextekens.",
23 "err.pipeFilter": "Gebruik ref of mr, niet allebei.",
20 "set.ev.issue_opened": "issue geopend",
24 "set.ev.issue_opened": "issue geopend",
21 "set.ev.issue_closed": "issue gesloten",
25 "set.ev.issue_closed": "issue gesloten",
22 "set.ev.issue_reopened": "issue heropend",
26 "set.ev.issue_reopened": "issue heropend",
@@ -26,6 +30,8 @@ var hookEvNL = map[string]string{
26}
30}
27
31
28var hookEvDE = map[string]string{
32var hookEvDE = map[string]string{
33 "err.badSHA": "Gib sha als 7 bis 40 Hexzeichen an.",
34 "err.pipeFilter": "Nutze ref oder mr, nicht beides.",
29 "set.ev.issue_opened": "Issue eröffnet",
35 "set.ev.issue_opened": "Issue eröffnet",
30 "set.ev.issue_closed": "Issue geschlossen",
36 "set.ev.issue_closed": "Issue geschlossen",
31 "set.ev.issue_reopened": "Issue wieder geöffnet",
37 "set.ev.issue_reopened": "Issue wieder geöffnet",
@@ -35,6 +41,8 @@ var hookEvDE = map[string]string{
35}
41}
36
42
37var hookEvFR = map[string]string{
43var hookEvFR = map[string]string{
44 "err.badSHA": "Donnez sha en 7 à 40 caractères hexadécimaux.",
45 "err.pipeFilter": "Utilisez ref ou mr, pas les deux.",
38 "set.ev.issue_opened": "ticket ouvert",
46 "set.ev.issue_opened": "ticket ouvert",
39 "set.ev.issue_closed": "ticket fermé",
47 "set.ev.issue_closed": "ticket fermé",
40 "set.ev.issue_reopened": "ticket rouvert",
48 "set.ev.issue_reopened": "ticket rouvert",
internal/readcache/cache.go added +295 −0
@@ -0,0 +1,295 @@
1 // Package readcache is een optionele, wegwerpbare cache voor afgeleide
2 // paginadata (README, trees, laatste commits, ref-lijsten, tellingen).
3 // De bron blijft altijd git: een fout in de cache betekent een miss.
4 //
5 // Indeling: <root>/cache/<repoID>/{sha,refs}/<hash>. Alleen cache/ is van
6 // deze package; Clear en LRU raken niets anders onder root aan.
7 package readcache
8
9 import (
10 "container/list"
11 "crypto/sha256"
12 "encoding/hex"
13 "encoding/json"
14 "errors"
15 "io/fs"
16 "os"
17 "path/filepath"
18 "strconv"
19 "strings"
20 "sync"
21 "sync/atomic"
22 )
23
24 // DefaultMax is de standaardgrootte: 512 MiB.
25 const DefaultMax = 512 << 20
26
27 // Cache is veilig voor gelijktijdig gebruik. Een nil *Cache is uit.
28 type Cache struct {
29 dir string
30 max int64
31
32 mu sync.Mutex
33 lru *list.List // voorkant = recent
34 items map[string]*list.Element
35 size int64
36
37 hits, misses atomic.Int64
38 gens map[int64]uint64
39 allGen uint64
40 }
41
42 type entry struct {
43 rel string
44 size int64
45 }
46
47 // Open maakt <root>/cache aan en leest de bestaande inhoud in.
48 func Open(root string, max int64) (*Cache, error) {
49 root = strings.TrimSpace(root)
50 if root == "" {
51 return nil, errors.New("readcache: geen pad")
52 }
53 if max <= 0 {
54 max = DefaultMax
55 }
56 dir := filepath.Join(root, "cache")
57 if err := os.MkdirAll(dir, 0o750); err != nil {
58 return nil, err
59 }
60 c := &Cache{dir: dir, max: max, lru: list.New(), items: map[string]*list.Element{}, gens: map[int64]uint64{}}
61 type found struct {
62 rel string
63 size int64
64 mod int64
65 }
66 var all []found
67 _ = filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
68 if err != nil || d.IsDir() {
69 return nil
70 }
71 if strings.HasPrefix(d.Name(), ".tmp-") {
72 _ = os.Remove(p)
73 return nil
74 }
75 info, err := d.Info()
76 if err != nil {
77 return nil
78 }
79 rel, _ := filepath.Rel(dir, p)
80 all = append(all, found{rel, info.Size(), info.ModTime().UnixNano()})
81 return nil
82 })
83 // oudste achteraan
84 for i := 1; i < len(all); i++ {
85 for j := i; j > 0 && all[j].mod > all[j-1].mod; j-- {
86 all[j], all[j-1] = all[j-1], all[j]
87 }
88 }
89 for _, f := range all {
90 c.items[f.rel] = c.lru.PushBack(&entry{f.rel, f.size})
91 c.size += f.size
92 }
93 c.mu.Lock()
94 c.evictLocked()
95 c.mu.Unlock()
96 return c, nil
97 }
98
99 // Dir is de cachemap zelf (<root>/cache).
100 func (c *Cache) Dir() string {
101 if c == nil {
102 return ""
103 }
104 return c.dir
105 }
106
107 func repoDir(repoID int64) string { return strconv.FormatInt(repoID, 10) }
108
109 // rel geeft het pad voor een sleutel. refs=true hoort bij een ref-naam
110 // en verdwijnt bij elke ref-wijziging; anders is de sleutel onveranderlijk.
111 func rel(repoID int64, refs bool, key string) string {
112 sum := sha256.Sum256([]byte(key))
113 ns := "sha"
114 if refs {
115 ns = "refs"
116 }
117 return filepath.Join(repoDir(repoID), ns, hex.EncodeToString(sum[:16]))
118 }
119
120 // GetJSON vult v uit de cache. false bij een miss of corrupte entry.
121 func (c *Cache) GetJSON(repoID int64, refs bool, key string, v any) bool {
122 if c == nil {
123 return false
124 }
125 r := rel(repoID, refs, key)
126 b, err := os.ReadFile(filepath.Join(c.dir, r))
127 if err != nil {
128 c.misses.Add(1)
129 return false
130 }
131 // eerste regel is de volledige sleutel, tegen hash-botsingen
132 nl := strings.IndexByte(string(b), '\n')
133 if nl < 0 || string(b[:nl]) != key || json.Unmarshal(b[nl+1:], v) != nil {
134 c.misses.Add(1)
135 c.remove(r)
136 return false
137 }
138 c.hits.Add(1)
139 c.mu.Lock()
140 if el, ok := c.items[r]; ok {
141 c.lru.MoveToFront(el)
142 }
143 c.mu.Unlock()
144 return true
145 }
146
147 // Gen is de generatie van een repo. Lees die vóór git en geef hem aan
148 // PutJSONAt: een ref-wijziging tussendoor maakt de schrijfactie ongeldig.
149 func (c *Cache) Gen(repoID int64) uint64 {
150 if c == nil {
151 return 0
152 }
153 c.mu.Lock()
154 defer c.mu.Unlock()
155 return c.gens[repoID] + c.allGen<<32
156 }
157
158 // PutJSON schrijft atomisch. Fouten worden genegeerd.
159 func (c *Cache) PutJSON(repoID int64, refs bool, key string, v any) {
160 c.PutJSONAt(repoID, refs, key, v, c.Gen(repoID))
161 }
162
163 // PutJSONAt schrijft alleen als de generatie nog gen is.
164 func (c *Cache) PutJSONAt(repoID int64, refs bool, key string, v any, gen uint64) {
165 if c == nil || strings.ContainsRune(key, '\n') {
166 return
167 }
168 body, err := json.Marshal(v)
169 if err != nil {
170 return
171 }
172 data := append([]byte(key+"\n"), body...)
173 if int64(len(data)) > c.max/4 {
174 return
175 }
176 r := rel(repoID, refs, key)
177 full := filepath.Join(c.dir, r)
178 if err := os.MkdirAll(filepath.Dir(full), 0o750); err != nil {
179 return
180 }
181 tmp, err := os.CreateTemp(filepath.Dir(full), ".tmp-*")
182 if err != nil {
183 return
184 }
185 _, werr := tmp.Write(data)
186 cerr := tmp.Close()
187 if werr != nil || cerr != nil || os.Rename(tmp.Name(), full) != nil {
188 _ = os.Remove(tmp.Name())
189 return
190 }
191 c.mu.Lock()
192 if c.gens[repoID]+c.allGen<<32 != gen {
193 c.mu.Unlock()
194 _ = os.Remove(full)
195 return
196 }
197 if el, ok := c.items[r]; ok {
198 c.size -= el.Value.(*entry).size
199 c.lru.Remove(el)
200 }
201 c.items[r] = c.lru.PushFront(&entry{r, int64(len(data))})
202 c.size += int64(len(data))
203 c.evictLocked()
204 c.mu.Unlock()
205 }
206
207 func (c *Cache) evictLocked() {
208 for c.size > c.max {
209 el := c.lru.Back()
210 if el == nil {
211 return
212 }
213 e := el.Value.(*entry)
214 _ = os.Remove(filepath.Join(c.dir, e.rel))
215 c.lru.Remove(el)
216 delete(c.items, e.rel)
217 c.size -= e.size
218 }
219 }
220
221 func (c *Cache) remove(r string) {
222 _ = os.Remove(filepath.Join(c.dir, r))
223 c.mu.Lock()
224 if el, ok := c.items[r]; ok {
225 c.size -= el.Value.(*entry).size
226 c.lru.Remove(el)
227 delete(c.items, r)
228 }
229 c.mu.Unlock()
230 }
231
232 func (c *Cache) dropPrefix(repoID int64, prefix string) {
233 c.mu.Lock()
234 defer c.mu.Unlock()
235 c.gens[repoID]++
236 _ = os.RemoveAll(filepath.Join(c.dir, prefix))
237 for r, el := range c.items {
238 if r == prefix || strings.HasPrefix(r, prefix+string(filepath.Separator)) {
239 c.size -= el.Value.(*entry).size
240 c.lru.Remove(el)
241 delete(c.items, r)
242 }
243 }
244 }
245
246 // DropRefs vergeet alles wat van een ref-naam afhangt.
247 func (c *Cache) DropRefs(repoID int64) {
248 if c != nil {
249 c.dropPrefix(repoID, filepath.Join(repoDir(repoID), "refs"))
250 }
251 }
252
253 // DropRepo vergeet alles van één repo (verwijderd, hernoemd, zichtbaarheid).
254 func (c *Cache) DropRepo(repoID int64) {
255 if c != nil {
256 c.dropPrefix(repoID, repoDir(repoID))
257 }
258 }
259
260 // Clear leegt de hele cache/-map, en niets daarbuiten.
261 func (c *Cache) Clear() {
262 if c == nil {
263 return
264 }
265 c.mu.Lock()
266 defer c.mu.Unlock()
267 ents, _ := os.ReadDir(c.dir)
268 for _, e := range ents {
269 _ = os.RemoveAll(filepath.Join(c.dir, e.Name()))
270 }
271 c.allGen++
272 c.lru.Init()
273 c.items = map[string]*list.Element{}
274 c.size = 0
275 }
276
277 // Usage is de stand voor Beheer.
278 type Usage struct {
279 Dir string
280 Bytes int64
281 Max int64
282 Entries int
283 Hits int64
284 Misses int64
285 }
286
287 // Usage geeft de huidige stand.
288 func (c *Cache) Usage() Usage {
289 if c == nil {
290 return Usage{}
291 }
292 c.mu.Lock()
293 defer c.mu.Unlock()
294 return Usage{Dir: c.dir, Bytes: c.size, Max: c.max, Entries: len(c.items), Hits: c.hits.Load(), Misses: c.misses.Load()}
295 }
@@ -0,0 +1,295 @@
1// Package readcache is een optionele, wegwerpbare cache voor afgeleide
2// paginadata (README, trees, laatste commits, ref-lijsten, tellingen).
3// De bron blijft altijd git: een fout in de cache betekent een miss.
4//
5// Indeling: <root>/cache/<repoID>/{sha,refs}/<hash>. Alleen cache/ is van
6// deze package; Clear en LRU raken niets anders onder root aan.
7package readcache
8
9import (
10 "container/list"
11 "crypto/sha256"
12 "encoding/hex"
13 "encoding/json"
14 "errors"
15 "io/fs"
16 "os"
17 "path/filepath"
18 "strconv"
19 "strings"
20 "sync"
21 "sync/atomic"
22)
23
24// DefaultMax is de standaardgrootte: 512 MiB.
25const DefaultMax = 512 << 20
26
27// Cache is veilig voor gelijktijdig gebruik. Een nil *Cache is uit.
28type Cache struct {
29 dir string
30 max int64
31
32 mu sync.Mutex
33 lru *list.List // voorkant = recent
34 items map[string]*list.Element
35 size int64
36
37 hits, misses atomic.Int64
38 gens map[int64]uint64
39 allGen uint64
40}
41
42type entry struct {
43 rel string
44 size int64
45}
46
47// Open maakt <root>/cache aan en leest de bestaande inhoud in.
48func Open(root string, max int64) (*Cache, error) {
49 root = strings.TrimSpace(root)
50 if root == "" {
51 return nil, errors.New("readcache: geen pad")
52 }
53 if max <= 0 {
54 max = DefaultMax
55 }
56 dir := filepath.Join(root, "cache")
57 if err := os.MkdirAll(dir, 0o750); err != nil {
58 return nil, err
59 }
60 c := &Cache{dir: dir, max: max, lru: list.New(), items: map[string]*list.Element{}, gens: map[int64]uint64{}}
61 type found struct {
62 rel string
63 size int64
64 mod int64
65 }
66 var all []found
67 _ = filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
68 if err != nil || d.IsDir() {
69 return nil
70 }
71 if strings.HasPrefix(d.Name(), ".tmp-") {
72 _ = os.Remove(p)
73 return nil
74 }
75 info, err := d.Info()
76 if err != nil {
77 return nil
78 }
79 rel, _ := filepath.Rel(dir, p)
80 all = append(all, found{rel, info.Size(), info.ModTime().UnixNano()})
81 return nil
82 })
83 // oudste achteraan
84 for i := 1; i < len(all); i++ {
85 for j := i; j > 0 && all[j].mod > all[j-1].mod; j-- {
86 all[j], all[j-1] = all[j-1], all[j]
87 }
88 }
89 for _, f := range all {
90 c.items[f.rel] = c.lru.PushBack(&entry{f.rel, f.size})
91 c.size += f.size
92 }
93 c.mu.Lock()
94 c.evictLocked()
95 c.mu.Unlock()
96 return c, nil
97}
98
99// Dir is de cachemap zelf (<root>/cache).
100func (c *Cache) Dir() string {
101 if c == nil {
102 return ""
103 }
104 return c.dir
105}
106
107func repoDir(repoID int64) string { return strconv.FormatInt(repoID, 10) }
108
109// rel geeft het pad voor een sleutel. refs=true hoort bij een ref-naam
110// en verdwijnt bij elke ref-wijziging; anders is de sleutel onveranderlijk.
111func rel(repoID int64, refs bool, key string) string {
112 sum := sha256.Sum256([]byte(key))
113 ns := "sha"
114 if refs {
115 ns = "refs"
116 }
117 return filepath.Join(repoDir(repoID), ns, hex.EncodeToString(sum[:16]))
118}
119
120// GetJSON vult v uit de cache. false bij een miss of corrupte entry.
121func (c *Cache) GetJSON(repoID int64, refs bool, key string, v any) bool {
122 if c == nil {
123 return false
124 }
125 r := rel(repoID, refs, key)
126 b, err := os.ReadFile(filepath.Join(c.dir, r))
127 if err != nil {
128 c.misses.Add(1)
129 return false
130 }
131 // eerste regel is de volledige sleutel, tegen hash-botsingen
132 nl := strings.IndexByte(string(b), '\n')
133 if nl < 0 || string(b[:nl]) != key || json.Unmarshal(b[nl+1:], v) != nil {
134 c.misses.Add(1)
135 c.remove(r)
136 return false
137 }
138 c.hits.Add(1)
139 c.mu.Lock()
140 if el, ok := c.items[r]; ok {
141 c.lru.MoveToFront(el)
142 }
143 c.mu.Unlock()
144 return true
145}
146
147// Gen is de generatie van een repo. Lees die vóór git en geef hem aan
148// PutJSONAt: een ref-wijziging tussendoor maakt de schrijfactie ongeldig.
149func (c *Cache) Gen(repoID int64) uint64 {
150 if c == nil {
151 return 0
152 }
153 c.mu.Lock()
154 defer c.mu.Unlock()
155 return c.gens[repoID] + c.allGen<<32
156}
157
158// PutJSON schrijft atomisch. Fouten worden genegeerd.
159func (c *Cache) PutJSON(repoID int64, refs bool, key string, v any) {
160 c.PutJSONAt(repoID, refs, key, v, c.Gen(repoID))
161}
162
163// PutJSONAt schrijft alleen als de generatie nog gen is.
164func (c *Cache) PutJSONAt(repoID int64, refs bool, key string, v any, gen uint64) {
165 if c == nil || strings.ContainsRune(key, '\n') {
166 return
167 }
168 body, err := json.Marshal(v)
169 if err != nil {
170 return
171 }
172 data := append([]byte(key+"\n"), body...)
173 if int64(len(data)) > c.max/4 {
174 return
175 }
176 r := rel(repoID, refs, key)
177 full := filepath.Join(c.dir, r)
178 if err := os.MkdirAll(filepath.Dir(full), 0o750); err != nil {
179 return
180 }
181 tmp, err := os.CreateTemp(filepath.Dir(full), ".tmp-*")
182 if err != nil {
183 return
184 }
185 _, werr := tmp.Write(data)
186 cerr := tmp.Close()
187 if werr != nil || cerr != nil || os.Rename(tmp.Name(), full) != nil {
188 _ = os.Remove(tmp.Name())
189 return
190 }
191 c.mu.Lock()
192 if c.gens[repoID]+c.allGen<<32 != gen {
193 c.mu.Unlock()
194 _ = os.Remove(full)
195 return
196 }
197 if el, ok := c.items[r]; ok {
198 c.size -= el.Value.(*entry).size
199 c.lru.Remove(el)
200 }
201 c.items[r] = c.lru.PushFront(&entry{r, int64(len(data))})
202 c.size += int64(len(data))
203 c.evictLocked()
204 c.mu.Unlock()
205}
206
207func (c *Cache) evictLocked() {
208 for c.size > c.max {
209 el := c.lru.Back()
210 if el == nil {
211 return
212 }
213 e := el.Value.(*entry)
214 _ = os.Remove(filepath.Join(c.dir, e.rel))
215 c.lru.Remove(el)
216 delete(c.items, e.rel)
217 c.size -= e.size
218 }
219}
220
221func (c *Cache) remove(r string) {
222 _ = os.Remove(filepath.Join(c.dir, r))
223 c.mu.Lock()
224 if el, ok := c.items[r]; ok {
225 c.size -= el.Value.(*entry).size
226 c.lru.Remove(el)
227 delete(c.items, r)
228 }
229 c.mu.Unlock()
230}
231
232func (c *Cache) dropPrefix(repoID int64, prefix string) {
233 c.mu.Lock()
234 defer c.mu.Unlock()
235 c.gens[repoID]++
236 _ = os.RemoveAll(filepath.Join(c.dir, prefix))
237 for r, el := range c.items {
238 if r == prefix || strings.HasPrefix(r, prefix+string(filepath.Separator)) {
239 c.size -= el.Value.(*entry).size
240 c.lru.Remove(el)
241 delete(c.items, r)
242 }
243 }
244}
245
246// DropRefs vergeet alles wat van een ref-naam afhangt.
247func (c *Cache) DropRefs(repoID int64) {
248 if c != nil {
249 c.dropPrefix(repoID, filepath.Join(repoDir(repoID), "refs"))
250 }
251}
252
253// DropRepo vergeet alles van één repo (verwijderd, hernoemd, zichtbaarheid).
254func (c *Cache) DropRepo(repoID int64) {
255 if c != nil {
256 c.dropPrefix(repoID, repoDir(repoID))
257 }
258}
259
260// Clear leegt de hele cache/-map, en niets daarbuiten.
261func (c *Cache) Clear() {
262 if c == nil {
263 return
264 }
265 c.mu.Lock()
266 defer c.mu.Unlock()
267 ents, _ := os.ReadDir(c.dir)
268 for _, e := range ents {
269 _ = os.RemoveAll(filepath.Join(c.dir, e.Name()))
270 }
271 c.allGen++
272 c.lru.Init()
273 c.items = map[string]*list.Element{}
274 c.size = 0
275}
276
277// Usage is de stand voor Beheer.
278type Usage struct {
279 Dir string
280 Bytes int64
281 Max int64
282 Entries int
283 Hits int64
284 Misses int64
285}
286
287// Usage geeft de huidige stand.
288func (c *Cache) Usage() Usage {
289 if c == nil {
290 return Usage{}
291 }
292 c.mu.Lock()
293 defer c.mu.Unlock()
294 return Usage{Dir: c.dir, Bytes: c.size, Max: c.max, Entries: len(c.items), Hits: c.hits.Load(), Misses: c.misses.Load()}
295}
internal/readcache/cache_test.go added +90 −0
@@ -0,0 +1,90 @@
1 package readcache
2
3 import (
4 "os"
5 "path/filepath"
6 "strings"
7 "testing"
8 )
9
10 func TestRoundTripAndInvalidation(t *testing.T) {
11 root := t.TempDir()
12 keep := filepath.Join(root, "data", "bluegit.db")
13 _ = os.MkdirAll(filepath.Dir(keep), 0o750)
14 _ = os.WriteFile(keep, []byte("db"), 0o600)
15 c, err := Open(root, 1<<20)
16 if err != nil {
17 t.Fatal(err)
18 }
19 c.PutJSON(1, false, "tree:abc", []string{"a", "b"})
20 c.PutJSON(1, true, "branches", []string{"main"})
21 c.PutJSON(2, true, "branches", []string{"dev"})
22 var got []string
23 if !c.GetJSON(1, false, "tree:abc", &got) || len(got) != 2 {
24 t.Fatalf("miss: %v", got)
25 }
26 c.DropRefs(1)
27 if c.GetJSON(1, true, "branches", &got) {
28 t.Fatal("refs niet vergeten")
29 }
30 if !c.GetJSON(1, false, "tree:abc", &got) || !c.GetJSON(2, true, "branches", &got) {
31 t.Fatal("te veel vergeten")
32 }
33 c.DropRepo(2)
34 if c.GetJSON(2, true, "branches", &got) {
35 t.Fatal("repo niet vergeten")
36 }
37 c.Clear()
38 if c.Usage().Entries != 0 || c.GetJSON(1, false, "tree:abc", &got) {
39 t.Fatal("niet leeg")
40 }
41 if b, err := os.ReadFile(keep); err != nil || string(b) != "db" {
42 t.Fatal("Clear raakte data/ aan")
43 }
44 }
45
46 func TestCorruptEntryIsMiss(t *testing.T) {
47 c, _ := Open(t.TempDir(), 1<<20)
48 c.PutJSON(1, false, "k", 5)
49 _ = os.WriteFile(filepath.Join(c.Dir(), rel(1, false, "k")), []byte("k\n{kapot"), 0o600)
50 var n int
51 if c.GetJSON(1, false, "k", &n) {
52 t.Fatal("corrupt gelezen")
53 }
54 if c.Usage().Entries != 0 {
55 t.Fatal("corrupte entry bleef staan")
56 }
57 }
58
59 func TestLRUAndReopen(t *testing.T) {
60 root := t.TempDir()
61 c, _ := Open(root, 4000)
62 big := strings.Repeat("x", 900)
63 for i := 0; i < 10; i++ {
64 c.PutJSON(1, false, "k"+string(rune('a'+i)), big)
65 }
66 var s string
67 if c.Usage().Bytes > 4000 || c.GetJSON(1, false, "ka", &s) || !c.GetJSON(1, false, "kj", &s) {
68 t.Fatalf("LRU: %+v", c.Usage())
69 }
70 c2, _ := Open(root, 4000)
71 if c2.Usage().Entries != c.Usage().Entries || !c2.GetJSON(1, false, "kj", &s) {
72 t.Fatal("heropenen")
73 }
74 var nilc *Cache
75 nilc.PutJSON(1, false, "x", 1)
76 if nilc.GetJSON(1, false, "x", &s) {
77 t.Fatal("nil-cache")
78 }
79 }
80
81 func TestStalePutAfterDropIsIgnored(t *testing.T) {
82 c, _ := Open(t.TempDir(), 1<<20)
83 gen := c.Gen(1)
84 c.DropRefs(1) // push tijdens het lezen uit git
85 c.PutJSONAt(1, true, "branches", []string{"oud"}, gen)
86 var got []string
87 if c.GetJSON(1, true, "branches", &got) {
88 t.Fatal("verouderde data geschreven")
89 }
90 }
@@ -0,0 +1,90 @@
1package readcache
2
3import (
4 "os"
5 "path/filepath"
6 "strings"
7 "testing"
8)
9
10func TestRoundTripAndInvalidation(t *testing.T) {
11 root := t.TempDir()
12 keep := filepath.Join(root, "data", "bluegit.db")
13 _ = os.MkdirAll(filepath.Dir(keep), 0o750)
14 _ = os.WriteFile(keep, []byte("db"), 0o600)
15 c, err := Open(root, 1<<20)
16 if err != nil {
17 t.Fatal(err)
18 }
19 c.PutJSON(1, false, "tree:abc", []string{"a", "b"})
20 c.PutJSON(1, true, "branches", []string{"main"})
21 c.PutJSON(2, true, "branches", []string{"dev"})
22 var got []string
23 if !c.GetJSON(1, false, "tree:abc", &got) || len(got) != 2 {
24 t.Fatalf("miss: %v", got)
25 }
26 c.DropRefs(1)
27 if c.GetJSON(1, true, "branches", &got) {
28 t.Fatal("refs niet vergeten")
29 }
30 if !c.GetJSON(1, false, "tree:abc", &got) || !c.GetJSON(2, true, "branches", &got) {
31 t.Fatal("te veel vergeten")
32 }
33 c.DropRepo(2)
34 if c.GetJSON(2, true, "branches", &got) {
35 t.Fatal("repo niet vergeten")
36 }
37 c.Clear()
38 if c.Usage().Entries != 0 || c.GetJSON(1, false, "tree:abc", &got) {
39 t.Fatal("niet leeg")
40 }
41 if b, err := os.ReadFile(keep); err != nil || string(b) != "db" {
42 t.Fatal("Clear raakte data/ aan")
43 }
44}
45
46func TestCorruptEntryIsMiss(t *testing.T) {
47 c, _ := Open(t.TempDir(), 1<<20)
48 c.PutJSON(1, false, "k", 5)
49 _ = os.WriteFile(filepath.Join(c.Dir(), rel(1, false, "k")), []byte("k\n{kapot"), 0o600)
50 var n int
51 if c.GetJSON(1, false, "k", &n) {
52 t.Fatal("corrupt gelezen")
53 }
54 if c.Usage().Entries != 0 {
55 t.Fatal("corrupte entry bleef staan")
56 }
57}
58
59func TestLRUAndReopen(t *testing.T) {
60 root := t.TempDir()
61 c, _ := Open(root, 4000)
62 big := strings.Repeat("x", 900)
63 for i := 0; i < 10; i++ {
64 c.PutJSON(1, false, "k"+string(rune('a'+i)), big)
65 }
66 var s string
67 if c.Usage().Bytes > 4000 || c.GetJSON(1, false, "ka", &s) || !c.GetJSON(1, false, "kj", &s) {
68 t.Fatalf("LRU: %+v", c.Usage())
69 }
70 c2, _ := Open(root, 4000)
71 if c2.Usage().Entries != c.Usage().Entries || !c2.GetJSON(1, false, "kj", &s) {
72 t.Fatal("heropenen")
73 }
74 var nilc *Cache
75 nilc.PutJSON(1, false, "x", 1)
76 if nilc.GetJSON(1, false, "x", &s) {
77 t.Fatal("nil-cache")
78 }
79}
80
81func TestStalePutAfterDropIsIgnored(t *testing.T) {
82 c, _ := Open(t.TempDir(), 1<<20)
83 gen := c.Gen(1)
84 c.DropRefs(1) // push tijdens het lezen uit git
85 c.PutJSONAt(1, true, "branches", []string{"oud"}, gen)
86 var got []string
87 if c.GetJSON(1, true, "branches", &got) {
88 t.Fatal("verouderde data geschreven")
89 }
90}
internal/runner/clone_test.go added +43 −0
@@ -0,0 +1,43 @@
1 package runner
2
3 import (
4 "os"
5 "os/exec"
6 "path/filepath"
7 "testing"
8 )
9
10 func TestCloneCopiesObjects(t *testing.T) {
11 if _, err := exec.LookPath("git"); err != nil {
12 t.Skip("git ontbreekt")
13 }
14 src := t.TempDir()
15 run := func(dir string, args ...string) {
16 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
17 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@t", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@t")
18 if out, err := cmd.CombinedOutput(); err != nil {
19 t.Fatalf("%v: %s", args, out)
20 }
21 }
22 run(src, "init", "-q", "-b", "master")
23 _ = os.WriteFile(filepath.Join(src, "a"), []byte("a"), 0o644)
24 run(src, "add", "a")
25 run(src, "commit", "-qm", "a")
26 bare := filepath.Join(t.TempDir(), "r.git")
27 if out, err := exec.Command("git", "clone", "-q", "--bare", src, bare).CombinedOutput(); err != nil {
28 t.Fatal(string(out))
29 }
30 work := filepath.Join(t.TempDir(), "w")
31 if out, err := exec.Command("git", cloneArgs(bare, work)...).CombinedOutput(); err != nil {
32 t.Fatal(string(out))
33 }
34 // geen gedeelde inodes met de bare repo
35 _ = filepath.Walk(filepath.Join(work, ".git", "objects"), func(p string, info os.FileInfo, err error) error {
36 if err == nil && !info.IsDir() {
37 if st, ok := sysNlink(info); ok && st > 1 {
38 t.Fatalf("hardlink: %s", p)
39 }
40 }
41 return nil
42 })
43 }
@@ -0,0 +1,43 @@
1package runner
2
3import (
4 "os"
5 "os/exec"
6 "path/filepath"
7 "testing"
8)
9
10func TestCloneCopiesObjects(t *testing.T) {
11 if _, err := exec.LookPath("git"); err != nil {
12 t.Skip("git ontbreekt")
13 }
14 src := t.TempDir()
15 run := func(dir string, args ...string) {
16 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
17 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@t", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@t")
18 if out, err := cmd.CombinedOutput(); err != nil {
19 t.Fatalf("%v: %s", args, out)
20 }
21 }
22 run(src, "init", "-q", "-b", "master")
23 _ = os.WriteFile(filepath.Join(src, "a"), []byte("a"), 0o644)
24 run(src, "add", "a")
25 run(src, "commit", "-qm", "a")
26 bare := filepath.Join(t.TempDir(), "r.git")
27 if out, err := exec.Command("git", "clone", "-q", "--bare", src, bare).CombinedOutput(); err != nil {
28 t.Fatal(string(out))
29 }
30 work := filepath.Join(t.TempDir(), "w")
31 if out, err := exec.Command("git", cloneArgs(bare, work)...).CombinedOutput(); err != nil {
32 t.Fatal(string(out))
33 }
34 // geen gedeelde inodes met de bare repo
35 _ = filepath.Walk(filepath.Join(work, ".git", "objects"), func(p string, info os.FileInfo, err error) error {
36 if err == nil && !info.IsDir() {
37 if st, ok := sysNlink(info); ok && st > 1 {
38 t.Fatalf("hardlink: %s", p)
39 }
40 }
41 return nil
42 })
43}
internal/runner/runner.go +8 −1
@@ -342,7 +342,7 @@ func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([
342 342 return nil, err
343 343 }
344 344 bare := gitrepo.Dir(r.storage(), repo.OwnerPath, repo.Name)
345 if _, err := execOut(ctx, r.GitBin, "clone", "--local", bare, work); err != nil {
345 if _, err := execOut(ctx, r.GitBin, cloneArgs(bare, work)...); err != nil {
346 346 sink.closeSection()
347 347 return nil, fmt.Errorf("clone: %w", err)
348 348 }
@@ -1076,3 +1076,10 @@ func execOut(ctx context.Context, bin string, args ...string) (string, error) {
1076 1076 }
1077 1077 return stdout.String(), nil
1078 1078 }
1079
1080 // cloneArgs kopieert de objecten in plaats van hardlinks te maken: de
1081 // werkmap en de bare repo kunnen andere eigenaars hebben, en met
1082 // fs.protected_hardlinks geeft link() dan "Operation not permitted".
1083 func cloneArgs(bare, work string) []string {
1084 return []string{"clone", "--no-hardlinks", bare, work}
1085 }
@@ -342,7 +342,7 @@ func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([
342 return nil, err
342 return nil, err
343 }
343 }
344 bare := gitrepo.Dir(r.storage(), repo.OwnerPath, repo.Name)
344 bare := gitrepo.Dir(r.storage(), repo.OwnerPath, repo.Name)
345 if _, err := execOut(ctx, r.GitBin, "clone", "--local", bare, work); err != nil {
345 if _, err := execOut(ctx, r.GitBin, cloneArgs(bare, work)...); err != nil {
346 sink.closeSection()
346 sink.closeSection()
347 return nil, fmt.Errorf("clone: %w", err)
347 return nil, fmt.Errorf("clone: %w", err)
348 }
348 }
@@ -1076,3 +1076,10 @@ func execOut(ctx context.Context, bin string, args ...string) (string, error) {
1076 }
1076 }
1077 return stdout.String(), nil
1077 return stdout.String(), nil
1078}
1078}
1079
1080// cloneArgs kopieert de objecten in plaats van hardlinks te maken: de
1081// werkmap en de bare repo kunnen andere eigenaars hebben, en met
1082// fs.protected_hardlinks geeft link() dan "Operation not permitted".
1083func cloneArgs(bare, work string) []string {
1084 return []string{"clone", "--no-hardlinks", bare, work}
1085}
internal/server/admin_cache.go added +195 −0
@@ -0,0 +1,195 @@
1 package server
2
3 import (
4 "encoding/json"
5 "errors"
6 "net/http"
7 "os"
8 "path/filepath"
9 "strconv"
10 "strings"
11
12 "gitlab.com/Timfa/bluegit/internal/i18n"
13 "gitlab.com/Timfa/bluegit/internal/readcache"
14 "gitlab.com/Timfa/bluegit/internal/state"
15 )
16
17 // cacheFile staat in de state-map en wint van config.yml, net als storage_path.
18 const cacheFile = "cache.json"
19
20 // CacheSetting is het gekozen cachepad. Leeg = geen cache.
21 type CacheSetting struct {
22 Path string `json:"path"`
23 MaxMB int64 `json:"max_mb"`
24 }
25
26 func (s *Server) savedCache() (CacheSetting, bool) {
27 if s.StateDir == "" {
28 return CacheSetting{}, false
29 }
30 b, err := os.ReadFile(filepath.Join(s.StateDir, cacheFile))
31 if err != nil {
32 return CacheSetting{}, false
33 }
34 var c CacheSetting
35 if json.Unmarshal(b, &c) != nil {
36 return CacheSetting{}, false
37 }
38 return c, true
39 }
40
41 // CacheConfig is wat nu geldt: opgeslagen keuze, anders config/env.
42 func (s *Server) CacheConfig() CacheSetting {
43 if c, ok := s.savedCache(); ok {
44 return c
45 }
46 return CacheSetting{Path: strings.TrimSpace(s.Cfg.CachePath), MaxMB: s.Cfg.CacheMaxMB}
47 }
48
49 // InitReadCache opent de cache bij het starten. Een fout zet de cache uit.
50 func (s *Server) InitReadCache() {
51 c := s.CacheConfig()
52 if err := s.openCache(c); err != nil && s.Log != nil {
53 s.Log.Warn("leescache uit", "path", c.Path, "err", err)
54 }
55 }
56
57 func (s *Server) openCache(c CacheSetting) error {
58 if strings.TrimSpace(c.Path) == "" {
59 s.SetReadCache(nil)
60 return nil
61 }
62 if !filepath.IsAbs(c.Path) {
63 return i18n.E("err.cacheAbs")
64 }
65 rc, err := readcache.Open(c.Path, c.MaxMB<<20)
66 if err != nil {
67 s.SetReadCache(nil)
68 return err
69 }
70 s.SetReadCache(rc)
71 return nil
72 }
73
74 func (s *Server) opSetCache(path string, maxMB int64) error {
75 path = strings.TrimSpace(path)
76 if path != "" {
77 path = filepath.Clean(path)
78 if !filepath.IsAbs(path) {
79 return fail(400, i18n.E("err.cacheAbs"))
80 }
81 for _, other := range []string{s.StoragePath(), s.StateDir} {
82 if other != "" && (path == filepath.Clean(other) || strings.HasPrefix(path+"/", filepath.Clean(other)+"/")) {
83 return fail(400, i18n.E("err.cacheInside"))
84 }
85 }
86 if err := os.MkdirAll(filepath.Join(path, "cache"), 0o750); err != nil {
87 return fail(400, i18n.E("err.cacheDir", err.Error()))
88 }
89 probe, err := os.CreateTemp(filepath.Join(path, "cache"), ".tmp-probe-*")
90 if err != nil {
91 return fail(400, i18n.E("err.cacheDir", err.Error()))
92 }
93 probe.Close()
94 _ = os.Remove(probe.Name())
95 }
96 if maxMB < 0 {
97 maxMB = 0
98 }
99 c := CacheSetting{Path: path, MaxMB: maxMB}
100 if s.StateDir == "" {
101 return fail(400, errors.New("geen state-map"))
102 }
103 b, _ := json.Marshal(c)
104 tmp := filepath.Join(s.StateDir, cacheFile+".tmp")
105 if err := os.WriteFile(tmp, b, 0o640); err != nil {
106 return fail(500, err)
107 }
108 if err := os.Rename(tmp, filepath.Join(s.StateDir, cacheFile)); err != nil {
109 return fail(500, err)
110 }
111 if err := s.openCache(c); err != nil {
112 return fail(400, err)
113 }
114 return nil
115 }
116
117 // cacheInfo is de stand voor Beheer en de API.
118 func (s *Server) cacheInfo() map[string]any {
119 c := s.CacheConfig()
120 u := s.ReadCache().Usage()
121 out := map[string]any{
122 "path": c.Path, "max_mb": c.MaxMB, "enabled": s.ReadCache() != nil,
123 "bytes": u.Bytes, "max_bytes": u.Max, "entries": u.Entries, "hits": u.Hits, "misses": u.Misses,
124 "locations": s.locations(),
125 "bytes_h": state.Format(uint64(u.Bytes)), "max_h": state.Format(uint64(u.Max)),
126 }
127 return out
128 }
129
130 // locations zegt waar wat staat, met het soort schijf.
131 func (s *Server) locations() []map[string]string {
132 var out []map[string]string
133 add := func(what, p string) {
134 if p != "" {
135 out = append(out, map[string]string{"what": what, "path": p, "disk": state.DiskKind(p)})
136 }
137 }
138 add("database", filepath.Join(s.StateDir, "bluegit.db"))
139 add("logos", filepath.Join(s.StateDir, "logos"))
140 if st := s.StoragePath(); st != "" {
141 add("repos", filepath.Join(st, "repos"))
142 add("artifacts", filepath.Join(st, "artifacts"))
143 }
144 if c := s.ReadCache(); c != nil {
145 add("cache", c.Dir())
146 }
147 return out
148 }
149
150 func (s *Server) apiAdminCache(w http.ResponseWriter, r *http.Request) {
151 if !apiUser(r).InstanceAdmin {
152 s.writeAPIErr(w, r, fail(403, i18n.E("err.forbidden")))
153 return
154 }
155 if r.Method == http.MethodPut {
156 var in CacheSetting
157 if err := readJSON(r, &in); err != nil {
158 s.writeAPIErr(w, r, fail(400, i18n.E("err.json")))
159 return
160 }
161 if !s.apiOK(w, r, s.opSetCache(in.Path, in.MaxMB)) {
162 return
163 }
164 }
165 writeJSON(w, http.StatusOK, s.cacheInfo())
166 }
167
168 func (s *Server) apiAdminCacheClear(w http.ResponseWriter, r *http.Request) {
169 if !apiUser(r).InstanceAdmin {
170 s.writeAPIErr(w, r, fail(403, i18n.E("err.forbidden")))
171 return
172 }
173 s.ReadCache().Clear()
174 writeJSON(w, http.StatusOK, s.cacheInfo())
175 }
176
177 func (s *Server) setupCacheSave(w http.ResponseWriter, r *http.Request) {
178 if _, ok := s.adminOnly(w, r); !ok || !s.csrfOK(w, r) {
179 return
180 }
181 maxMB, _ := strconv.ParseInt(strings.TrimSpace(r.FormValue("max_mb")), 10, 64)
182 if err := s.opSetCache(r.FormValue("cache_path"), maxMB); err != nil {
183 s.renderSetup(w, r, s.trErr(r, err), "")
184 return
185 }
186 s.redirect(w, r, "/setup", s.tr(r, "setup.cacheSaved"), "")
187 }
188
189 func (s *Server) setupCacheClear(w http.ResponseWriter, r *http.Request) {
190 if _, ok := s.adminOnly(w, r); !ok || !s.csrfOK(w, r) {
191 return
192 }
193 s.ReadCache().Clear()
194 s.redirect(w, r, "/setup", s.tr(r, "setup.cacheCleared"), "")
195 }
@@ -0,0 +1,195 @@
1package server
2
3import (
4 "encoding/json"
5 "errors"
6 "net/http"
7 "os"
8 "path/filepath"
9 "strconv"
10 "strings"
11
12 "gitlab.com/Timfa/bluegit/internal/i18n"
13 "gitlab.com/Timfa/bluegit/internal/readcache"
14 "gitlab.com/Timfa/bluegit/internal/state"
15)
16
17// cacheFile staat in de state-map en wint van config.yml, net als storage_path.
18const cacheFile = "cache.json"
19
20// CacheSetting is het gekozen cachepad. Leeg = geen cache.
21type CacheSetting struct {
22 Path string `json:"path"`
23 MaxMB int64 `json:"max_mb"`
24}
25
26func (s *Server) savedCache() (CacheSetting, bool) {
27 if s.StateDir == "" {
28 return CacheSetting{}, false
29 }
30 b, err := os.ReadFile(filepath.Join(s.StateDir, cacheFile))
31 if err != nil {
32 return CacheSetting{}, false
33 }
34 var c CacheSetting
35 if json.Unmarshal(b, &c) != nil {
36 return CacheSetting{}, false
37 }
38 return c, true
39}
40
41// CacheConfig is wat nu geldt: opgeslagen keuze, anders config/env.
42func (s *Server) CacheConfig() CacheSetting {
43 if c, ok := s.savedCache(); ok {
44 return c
45 }
46 return CacheSetting{Path: strings.TrimSpace(s.Cfg.CachePath), MaxMB: s.Cfg.CacheMaxMB}
47}
48
49// InitReadCache opent de cache bij het starten. Een fout zet de cache uit.
50func (s *Server) InitReadCache() {
51 c := s.CacheConfig()
52 if err := s.openCache(c); err != nil && s.Log != nil {
53 s.Log.Warn("leescache uit", "path", c.Path, "err", err)
54 }
55}
56
57func (s *Server) openCache(c CacheSetting) error {
58 if strings.TrimSpace(c.Path) == "" {
59 s.SetReadCache(nil)
60 return nil
61 }
62 if !filepath.IsAbs(c.Path) {
63 return i18n.E("err.cacheAbs")
64 }
65 rc, err := readcache.Open(c.Path, c.MaxMB<<20)
66 if err != nil {
67 s.SetReadCache(nil)
68 return err
69 }
70 s.SetReadCache(rc)
71 return nil
72}
73
74func (s *Server) opSetCache(path string, maxMB int64) error {
75 path = strings.TrimSpace(path)
76 if path != "" {
77 path = filepath.Clean(path)
78 if !filepath.IsAbs(path) {
79 return fail(400, i18n.E("err.cacheAbs"))
80 }
81 for _, other := range []string{s.StoragePath(), s.StateDir} {
82 if other != "" && (path == filepath.Clean(other) || strings.HasPrefix(path+"/", filepath.Clean(other)+"/")) {
83 return fail(400, i18n.E("err.cacheInside"))
84 }
85 }
86 if err := os.MkdirAll(filepath.Join(path, "cache"), 0o750); err != nil {
87 return fail(400, i18n.E("err.cacheDir", err.Error()))
88 }
89 probe, err := os.CreateTemp(filepath.Join(path, "cache"), ".tmp-probe-*")
90 if err != nil {
91 return fail(400, i18n.E("err.cacheDir", err.Error()))
92 }
93 probe.Close()
94 _ = os.Remove(probe.Name())
95 }
96 if maxMB < 0 {
97 maxMB = 0
98 }
99 c := CacheSetting{Path: path, MaxMB: maxMB}
100 if s.StateDir == "" {
101 return fail(400, errors.New("geen state-map"))
102 }
103 b, _ := json.Marshal(c)
104 tmp := filepath.Join(s.StateDir, cacheFile+".tmp")
105 if err := os.WriteFile(tmp, b, 0o640); err != nil {
106 return fail(500, err)
107 }
108 if err := os.Rename(tmp, filepath.Join(s.StateDir, cacheFile)); err != nil {
109 return fail(500, err)
110 }
111 if err := s.openCache(c); err != nil {
112 return fail(400, err)
113 }
114 return nil
115}
116
117// cacheInfo is de stand voor Beheer en de API.
118func (s *Server) cacheInfo() map[string]any {
119 c := s.CacheConfig()
120 u := s.ReadCache().Usage()
121 out := map[string]any{
122 "path": c.Path, "max_mb": c.MaxMB, "enabled": s.ReadCache() != nil,
123 "bytes": u.Bytes, "max_bytes": u.Max, "entries": u.Entries, "hits": u.Hits, "misses": u.Misses,
124 "locations": s.locations(),
125 "bytes_h": state.Format(uint64(u.Bytes)), "max_h": state.Format(uint64(u.Max)),
126 }
127 return out
128}
129
130// locations zegt waar wat staat, met het soort schijf.
131func (s *Server) locations() []map[string]string {
132 var out []map[string]string
133 add := func(what, p string) {
134 if p != "" {
135 out = append(out, map[string]string{"what": what, "path": p, "disk": state.DiskKind(p)})
136 }
137 }
138 add("database", filepath.Join(s.StateDir, "bluegit.db"))
139 add("logos", filepath.Join(s.StateDir, "logos"))
140 if st := s.StoragePath(); st != "" {
141 add("repos", filepath.Join(st, "repos"))
142 add("artifacts", filepath.Join(st, "artifacts"))
143 }
144 if c := s.ReadCache(); c != nil {
145 add("cache", c.Dir())
146 }
147 return out
148}
149
150func (s *Server) apiAdminCache(w http.ResponseWriter, r *http.Request) {
151 if !apiUser(r).InstanceAdmin {
152 s.writeAPIErr(w, r, fail(403, i18n.E("err.forbidden")))
153 return
154 }
155 if r.Method == http.MethodPut {
156 var in CacheSetting
157 if err := readJSON(r, &in); err != nil {
158 s.writeAPIErr(w, r, fail(400, i18n.E("err.json")))
159 return
160 }
161 if !s.apiOK(w, r, s.opSetCache(in.Path, in.MaxMB)) {
162 return
163 }
164 }
165 writeJSON(w, http.StatusOK, s.cacheInfo())
166}
167
168func (s *Server) apiAdminCacheClear(w http.ResponseWriter, r *http.Request) {
169 if !apiUser(r).InstanceAdmin {
170 s.writeAPIErr(w, r, fail(403, i18n.E("err.forbidden")))
171 return
172 }
173 s.ReadCache().Clear()
174 writeJSON(w, http.StatusOK, s.cacheInfo())
175}
176
177func (s *Server) setupCacheSave(w http.ResponseWriter, r *http.Request) {
178 if _, ok := s.adminOnly(w, r); !ok || !s.csrfOK(w, r) {
179 return
180 }
181 maxMB, _ := strconv.ParseInt(strings.TrimSpace(r.FormValue("max_mb")), 10, 64)
182 if err := s.opSetCache(r.FormValue("cache_path"), maxMB); err != nil {
183 s.renderSetup(w, r, s.trErr(r, err), "")
184 return
185 }
186 s.redirect(w, r, "/setup", s.tr(r, "setup.cacheSaved"), "")
187}
188
189func (s *Server) setupCacheClear(w http.ResponseWriter, r *http.Request) {
190 if _, ok := s.adminOnly(w, r); !ok || !s.csrfOK(w, r) {
191 return
192 }
193 s.ReadCache().Clear()
194 s.redirect(w, r, "/setup", s.tr(r, "setup.cacheCleared"), "")
195}
internal/server/api.go +4 −0
@@ -49,6 +49,10 @@ func (s *Server) createRelease(w http.ResponseWriter, r *http.Request) {
49 49 return
50 50 }
51 51 role, err := s.Store.EffectiveRole(r.Context(), author.ID, repo.ID)
52 if err == nil && !role.CanRead() && repo.Visibility != "public" {
53 s.writeAPIErr(w, r, fail(http.StatusNotFound, i18n.E("err.notFound")))
54 return
55 }
52 56 if err != nil || !role.AtLeast(access.RoleMaintainer) {
53 57 s.writeAPIErr(w, r, fail(http.StatusForbidden, i18n.E("err.onlyAdmin")))
54 58 return
@@ -49,6 +49,10 @@ func (s *Server) createRelease(w http.ResponseWriter, r *http.Request) {
49 return
49 return
50 }
50 }
51 role, err := s.Store.EffectiveRole(r.Context(), author.ID, repo.ID)
51 role, err := s.Store.EffectiveRole(r.Context(), author.ID, repo.ID)
52 if err == nil && !role.CanRead() && repo.Visibility != "public" {
53 s.writeAPIErr(w, r, fail(http.StatusNotFound, i18n.E("err.notFound")))
54 return
55 }
52 if err != nil || !role.AtLeast(access.RoleMaintainer) {
56 if err != nil || !role.AtLeast(access.RoleMaintainer) {
53 s.writeAPIErr(w, r, fail(http.StatusForbidden, i18n.E("err.onlyAdmin")))
57 s.writeAPIErr(w, r, fail(http.StatusForbidden, i18n.E("err.onlyAdmin")))
54 return
58 return
internal/server/api_flow.go +57 −1
@@ -78,9 +78,23 @@ func (s *Server) apiMRs(w http.ResponseWriter, r *http.Request) {
78 78 if !s.apiOK(w, r, err) {
79 79 return
80 80 }
81 // Optionele filters: state (open, merged, closed of all), source_branch, target_branch.
82 q := r.URL.Query()
83 wantState := strings.TrimSpace(q.Get("state"))
84 wantSource := strings.TrimSpace(q.Get("source_branch"))
85 wantTarget := strings.TrimSpace(q.Get("target_branch"))
81 86 page, per := pageQuery(r)
82 87 rows := make([]map[string]any, 0, len(list))
83 88 for _, mr := range list {
89 if wantState != "" && wantState != "all" && mr.Status != wantState {
90 continue
91 }
92 if wantSource != "" && mr.SourceBranch != wantSource {
93 continue
94 }
95 if wantTarget != "" && mr.TargetBranch != wantTarget {
96 continue
97 }
84 98 rows = append(rows, s.mrJSON(repo, mr))
85 99 }
86 100 writePage(w, r, slicePage(rows, page, per), len(rows), page, per)
@@ -364,7 +378,11 @@ func (s *Server) apiPipelines(w http.ResponseWriter, r *http.Request) {
364 378 })
365 379 return
366 380 }
367 list, err := s.Store.ListPipelines(r.Context(), repo.ID)
381 f, err := s.pipelineFilter(r, repo)
382 if !s.apiOK(w, r, err) {
383 return
384 }
385 list, err := s.Store.FindPipelines(r.Context(), repo.ID, f)
368 386 if !s.apiOK(w, r, err) {
369 387 return
370 388 }
@@ -697,3 +715,41 @@ func (s *Server) apiReleaseAsset(w http.ResponseWriter, r *http.Request) {
697 715 }
698 716 s.writeAPIErr(w, r, fail(404, i18n.E("err.notFound")))
699 717 }
718
719 // pipelineFilter leest sha, ref en mr uit de query.
720 // ref mag een branchnaam zijn (refs/heads/ en refs/tags/ worden geprobeerd)
721 // of een volledige ref. mr is het MR-nummer: pipelines van die MR plus
722 // pushes naar de bronbranch.
723 func (s *Server) pipelineFilter(r *http.Request, repo store.Repo) (store.PipelineFilter, error) {
724 q := r.URL.Query()
725 var f store.PipelineFilter
726 if sha := strings.TrimSpace(q.Get("sha")); sha != "" {
727 if len(sha) < 7 || len(sha) > 40 || strings.Trim(strings.ToLower(sha), "0123456789abcdef") != "" {
728 return f, fail(400, i18n.E("err.badSHA"))
729 }
730 f.SHA = sha
731 }
732 if ref := strings.TrimSpace(q.Get("ref")); ref != "" {
733 if strings.HasPrefix(ref, "refs/") {
734 f.Refs = []string{ref}
735 } else {
736 f.Refs = []string{"refs/heads/" + ref, "refs/tags/" + ref}
737 }
738 }
739 if raw := strings.TrimSpace(q.Get("mr")); raw != "" {
740 n, err := strconv.ParseInt(raw, 10, 64)
741 if err != nil || n < 1 {
742 return f, fail(400, i18n.E("err.notFound"))
743 }
744 mr, err := s.Store.MergeRequestByNumber(r.Context(), repo.ID, n)
745 if err != nil {
746 return f, notFound(err)
747 }
748 if len(f.Refs) > 0 {
749 return f, fail(400, i18n.E("err.pipeFilter"))
750 }
751 f.MRID = mr.ID
752 f.Refs = []string{"refs/heads/" + mr.SourceBranch}
753 }
754 return f, nil
755 }
@@ -78,9 +78,23 @@ func (s *Server) apiMRs(w http.ResponseWriter, r *http.Request) {
78 if !s.apiOK(w, r, err) {
78 if !s.apiOK(w, r, err) {
79 return
79 return
80 }
80 }
81 // Optionele filters: state (open, merged, closed of all), source_branch, target_branch.
82 q := r.URL.Query()
83 wantState := strings.TrimSpace(q.Get("state"))
84 wantSource := strings.TrimSpace(q.Get("source_branch"))
85 wantTarget := strings.TrimSpace(q.Get("target_branch"))
81 page, per := pageQuery(r)
86 page, per := pageQuery(r)
82 rows := make([]map[string]any, 0, len(list))
87 rows := make([]map[string]any, 0, len(list))
83 for _, mr := range list {
88 for _, mr := range list {
89 if wantState != "" && wantState != "all" && mr.Status != wantState {
90 continue
91 }
92 if wantSource != "" && mr.SourceBranch != wantSource {
93 continue
94 }
95 if wantTarget != "" && mr.TargetBranch != wantTarget {
96 continue
97 }
84 rows = append(rows, s.mrJSON(repo, mr))
98 rows = append(rows, s.mrJSON(repo, mr))
85 }
99 }
86 writePage(w, r, slicePage(rows, page, per), len(rows), page, per)
100 writePage(w, r, slicePage(rows, page, per), len(rows), page, per)
@@ -364,7 +378,11 @@ func (s *Server) apiPipelines(w http.ResponseWriter, r *http.Request) {
364 })
378 })
365 return
379 return
366 }
380 }
367 list, err := s.Store.ListPipelines(r.Context(), repo.ID)
381 f, err := s.pipelineFilter(r, repo)
382 if !s.apiOK(w, r, err) {
383 return
384 }
385 list, err := s.Store.FindPipelines(r.Context(), repo.ID, f)
368 if !s.apiOK(w, r, err) {
386 if !s.apiOK(w, r, err) {
369 return
387 return
370 }
388 }
@@ -697,3 +715,41 @@ func (s *Server) apiReleaseAsset(w http.ResponseWriter, r *http.Request) {
697 }
715 }
698 s.writeAPIErr(w, r, fail(404, i18n.E("err.notFound")))
716 s.writeAPIErr(w, r, fail(404, i18n.E("err.notFound")))
699}
717}
718
719// pipelineFilter leest sha, ref en mr uit de query.
720// ref mag een branchnaam zijn (refs/heads/ en refs/tags/ worden geprobeerd)
721// of een volledige ref. mr is het MR-nummer: pipelines van die MR plus
722// pushes naar de bronbranch.
723func (s *Server) pipelineFilter(r *http.Request, repo store.Repo) (store.PipelineFilter, error) {
724 q := r.URL.Query()
725 var f store.PipelineFilter
726 if sha := strings.TrimSpace(q.Get("sha")); sha != "" {
727 if len(sha) < 7 || len(sha) > 40 || strings.Trim(strings.ToLower(sha), "0123456789abcdef") != "" {
728 return f, fail(400, i18n.E("err.badSHA"))
729 }
730 f.SHA = sha
731 }
732 if ref := strings.TrimSpace(q.Get("ref")); ref != "" {
733 if strings.HasPrefix(ref, "refs/") {
734 f.Refs = []string{ref}
735 } else {
736 f.Refs = []string{"refs/heads/" + ref, "refs/tags/" + ref}
737 }
738 }
739 if raw := strings.TrimSpace(q.Get("mr")); raw != "" {
740 n, err := strconv.ParseInt(raw, 10, 64)
741 if err != nil || n < 1 {
742 return f, fail(400, i18n.E("err.notFound"))
743 }
744 mr, err := s.Store.MergeRequestByNumber(r.Context(), repo.ID, n)
745 if err != nil {
746 return f, notFound(err)
747 }
748 if len(f.Refs) > 0 {
749 return f, fail(400, i18n.E("err.pipeFilter"))
750 }
751 f.MRID = mr.ID
752 f.Refs = []string{"refs/heads/" + mr.SourceBranch}
753 }
754 return f, nil
755}
internal/server/api_routes.go +28 −25
@@ -43,11 +43,11 @@ var apiCatalog = []apiRoute{
43 43
44 44 {http.MethodGet, "/api/v1/projects", access.ScopeRead, authUser, "List projects", (*Server).apiListProjects},
45 45 {http.MethodPost, "/api/v1/projects", access.ScopeWrite, authUser, "Create project", (*Server).apiCreateProject},
46 {http.MethodGet, "/api/v1/projects/{owner}/{repo}", access.ScopeRepoRead, authUser, "Get project", (*Server).apiGetProject},
46 {http.MethodGet, "/api/v1/projects/{owner}/{repo}", access.ScopeRepoRead, authOptional, "Get project", (*Server).apiGetProject},
47 47 {http.MethodPut, "/api/v1/projects/{owner}/{repo}", access.ScopeWrite, authUser, "Update project", (*Server).apiPutProject},
48 48 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}", access.ScopeWrite, authUser, "Delete project", (*Server).apiDeleteProject},
49 49 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/star", access.ScopeWrite, authUser, "Toggle star", (*Server).apiStar},
50 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeRepoRead, authUser, "Project logo", (*Server).apiProjectLogo},
50 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeRepoRead, authOptional, "Project logo", (*Server).apiProjectLogo},
51 51 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeWrite, authUser, "Upload project logo", (*Server).apiProjectLogo},
52 52 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/members", access.ScopeRead, authUser, "List project members", (*Server).apiProjectMembers},
53 53 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/members", access.ScopeWrite, authUser, "Add project member", (*Server).apiProjectMembers},
@@ -67,33 +67,33 @@ var apiCatalog = []apiRoute{
67 67 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/protected_tags", access.ScopeWrite, authUser, "Protect tag pattern", (*Server).apiProtectTags},
68 68 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/protected_tags/{pattern...}", access.ScopeWrite, authUser, "Unprotect tag pattern", (*Server).apiUnprotectTag},
69 69
70 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoRead, authUser, "List branches", (*Server).apiBranches},
70 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoRead, authOptional, "List branches", (*Server).apiBranches},
71 71 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoWrite, authUser, "Create branch", (*Server).apiBranches},
72 72 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/repository/branches/{branch...}", access.ScopeRepoWrite, authUser, "Rename branch", (*Server).apiBranch},
73 73 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/repository/branches/{branch...}", access.ScopeRepoWrite, authUser, "Delete branch", (*Server).apiBranch},
74 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoRead, authUser, "List tags", (*Server).apiTags},
74 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoRead, authOptional, "List tags", (*Server).apiTags},
75 75 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoWrite, authUser, "Create tag", (*Server).apiTags},
76 76 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}", access.ScopeRepoWrite, authUser, "Delete tag", (*Server).apiDeleteTag},
77 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}/archive", access.ScopeRepoRead, authUser, "Download tag archive", (*Server).apiTagArchive},
78 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tree", access.ScopeRepoRead, authUser, "List tree", (*Server).apiTree},
79 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/files/{path...}", access.ScopeRepoRead, authUser, "Read file", (*Server).apiFile},
80 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoRead, authUser, "List commits", (*Server).apiCommits},
77 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}/archive", access.ScopeRepoRead, authOptional, "Download tag archive", (*Server).apiTagArchive},
78 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tree", access.ScopeRepoRead, authOptional, "List tree", (*Server).apiTree},
79 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/files/{path...}", access.ScopeRepoRead, authOptional, "Read file", (*Server).apiFile},
80 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoRead, authOptional, "List commits", (*Server).apiCommits},
81 81 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoWrite, authUser, "Create commit", (*Server).apiCommits},
82 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}", access.ScopeRepoRead, authUser, "Get commit", (*Server).apiCommit},
83 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}/diff", access.ScopeRepoRead, authUser, "Commit diff", (*Server).apiCommitDiff},
82 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}", access.ScopeRepoRead, authOptional, "Get commit", (*Server).apiCommit},
83 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}/diff", access.ScopeRepoRead, authOptional, "Commit diff", (*Server).apiCommitDiff},
84 84
85 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeRead, authUser, "List issues", (*Server).issuesAPI},
85 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeRead, authOptional, "List issues", (*Server).issuesAPI},
86 86 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeWrite, authUser, "Create issue", (*Server).issuesAPI},
87 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeRead, authUser, "Get issue", (*Server).issueAPI},
87 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeRead, authOptional, "Get issue", (*Server).issueAPI},
88 88 {http.MethodPut, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeWrite, authUser, "Update issue", (*Server).issueAPI},
89 89 {http.MethodDelete, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeWrite, authUser, "Delete issue", (*Server).issueAPI},
90 90 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/issues/{number}/notes", access.ScopeWrite, authUser, "Create issue note", (*Server).issueNoteAPI},
91 91
92 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeRead, authUser, "List merge requests", (*Server).apiMRs},
92 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeRead, authOptional, "List merge requests. Optional query: state (open, merged, closed, all), source_branch, target_branch.", (*Server).apiMRs},
93 93 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeWrite, authUser, "Create merge request", (*Server).apiMRs},
94 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeRead, authUser, "Get merge request", (*Server).apiMR},
94 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeRead, authOptional, "Get merge request", (*Server).apiMR},
95 95 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeWrite, authUser, "Update merge request", (*Server).apiMR},
96 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeRead, authUser, "List notes", (*Server).apiMRNotes},
96 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeRead, authOptional, "List notes", (*Server).apiMRNotes},
97 97 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeWrite, authUser, "Create note", (*Server).apiMRNotes},
98 98 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reviews", access.ScopeWrite, authUser, "Submit review", (*Server).apiReview},
99 99 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reviews/dismiss", access.ScopeWrite, authUser, "Dismiss review", (*Server).apiDismissReview},
@@ -102,28 +102,31 @@ var apiCatalog = []apiRoute{
102 102 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/close", access.ScopeWrite, authUser, "Close merge request", (*Server).apiCloseMR},
103 103 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reopen", access.ScopeWrite, authUser, "Reopen merge request", (*Server).apiReopenMR},
104 104
105 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeRead, authUser, "List pipelines", (*Server).apiPipelines},
105 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeRead, authOptional, "List pipelines, newest first. Optional query: sha (7-40 hex, prefix), ref (branch, tag or full ref), mr (merge request number; its pipelines plus pushes to its source branch).", (*Server).apiPipelines},
106 106 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeWrite, authUser, "Start pipeline", (*Server).apiPipelines},
107 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}", access.ScopeRead, authUser, "Get pipeline", (*Server).apiPipeline},
107 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}", access.ScopeRead, authOptional, "Get pipeline", (*Server).apiPipeline},
108 108 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/cancel", access.ScopeWrite, authUser, "Cancel pipeline", (*Server).apiCancelPipeline},
109 109 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/retry", access.ScopeWrite, authUser, "Retry pipeline", (*Server).apiRetryPipeline},
110 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs", access.ScopeRead, authUser, "List jobs", (*Server).apiJobs},
111 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}", access.ScopeRead, authUser, "Get job", (*Server).apiJob},
112 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/log", access.ScopeRead, authUser, "Job log", (*Server).apiJobLog},
110 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs", access.ScopeRead, authOptional, "List jobs", (*Server).apiJobs},
111 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}", access.ScopeRead, authOptional, "Get job", (*Server).apiJob},
112 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/log", access.ScopeRead, authOptional, "Job log", (*Server).apiJobLog},
113 113 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/play", access.ScopeWrite, authUser, "Play manual job", (*Server).apiPlayJob},
114 114 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/retry", access.ScopeWrite, authUser, "Retry job", (*Server).apiRetryJob},
115 115 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/cancel", access.ScopeWrite, authUser, "Cancel job", (*Server).apiCancelJob},
116 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts", access.ScopeRead, authUser, "List artifacts", (*Server).apiArtifacts},
117 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts/{path...}", access.ScopeRead, authUser, "Download artifact", (*Server).apiArtifact},
116 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts", access.ScopeRead, authOptional, "List artifacts", (*Server).apiArtifacts},
117 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts/{path...}", access.ScopeRead, authOptional, "Download artifact", (*Server).apiArtifact},
118 118
119 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoRead, authUser, "Read CI file", (*Server).apiCIFile},
119 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoRead, authOptional, "Read CI file", (*Server).apiCIFile},
120 120 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoWrite, authUser, "Save CI file", (*Server).apiCIFile},
121 121 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/ci/lint", access.ScopeRead, authUser, "Lint CI file", (*Server).apiLintCI},
122 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeRead, authUser, "List releases", (*Server).apiReleases},
122 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeRead, authOptional, "List releases", (*Server).apiReleases},
123 123 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeWrite, authUser, "Publish release", (*Server).createRelease},
124 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeRead, authUser, "Download release asset", (*Server).apiReleaseAsset},
124 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeRead, authOptional, "Download release asset", (*Server).apiReleaseAsset},
125 125 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases", access.ScopeWrite, authRelease, "Publish release from a job or user token", (*Server).createRelease},
126 126
127 127 {http.MethodGet, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Read instance storage", (*Server).apiAdminStorage},
128 128 {http.MethodPut, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Set instance storage", (*Server).apiAdminStorage},
129 {http.MethodGet, "/api/v1/admin/cache", access.ScopeAdmin, authUser, "Read cache settings and usage", (*Server).apiAdminCache},
130 {http.MethodPut, "/api/v1/admin/cache", access.ScopeAdmin, authUser, "Set cache path", (*Server).apiAdminCache},
131 {http.MethodPost, "/api/v1/admin/cache/clear", access.ScopeAdmin, authUser, "Clear the read cache", (*Server).apiAdminCacheClear},
129 132 }
@@ -43,11 +43,11 @@ var apiCatalog = []apiRoute{
43
43
44 {http.MethodGet, "/api/v1/projects", access.ScopeRead, authUser, "List projects", (*Server).apiListProjects},
44 {http.MethodGet, "/api/v1/projects", access.ScopeRead, authUser, "List projects", (*Server).apiListProjects},
45 {http.MethodPost, "/api/v1/projects", access.ScopeWrite, authUser, "Create project", (*Server).apiCreateProject},
45 {http.MethodPost, "/api/v1/projects", access.ScopeWrite, authUser, "Create project", (*Server).apiCreateProject},
46 {http.MethodGet, "/api/v1/projects/{owner}/{repo}", access.ScopeRepoRead, authUser, "Get project", (*Server).apiGetProject},
46 {http.MethodGet, "/api/v1/projects/{owner}/{repo}", access.ScopeRepoRead, authOptional, "Get project", (*Server).apiGetProject},
47 {http.MethodPut, "/api/v1/projects/{owner}/{repo}", access.ScopeWrite, authUser, "Update project", (*Server).apiPutProject},
47 {http.MethodPut, "/api/v1/projects/{owner}/{repo}", access.ScopeWrite, authUser, "Update project", (*Server).apiPutProject},
48 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}", access.ScopeWrite, authUser, "Delete project", (*Server).apiDeleteProject},
48 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}", access.ScopeWrite, authUser, "Delete project", (*Server).apiDeleteProject},
49 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/star", access.ScopeWrite, authUser, "Toggle star", (*Server).apiStar},
49 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/star", access.ScopeWrite, authUser, "Toggle star", (*Server).apiStar},
50 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeRepoRead, authUser, "Project logo", (*Server).apiProjectLogo},
50 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeRepoRead, authOptional, "Project logo", (*Server).apiProjectLogo},
51 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeWrite, authUser, "Upload project logo", (*Server).apiProjectLogo},
51 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/logo", access.ScopeWrite, authUser, "Upload project logo", (*Server).apiProjectLogo},
52 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/members", access.ScopeRead, authUser, "List project members", (*Server).apiProjectMembers},
52 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/members", access.ScopeRead, authUser, "List project members", (*Server).apiProjectMembers},
53 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/members", access.ScopeWrite, authUser, "Add project member", (*Server).apiProjectMembers},
53 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/members", access.ScopeWrite, authUser, "Add project member", (*Server).apiProjectMembers},
@@ -67,33 +67,33 @@ var apiCatalog = []apiRoute{
67 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/protected_tags", access.ScopeWrite, authUser, "Protect tag pattern", (*Server).apiProtectTags},
67 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/protected_tags", access.ScopeWrite, authUser, "Protect tag pattern", (*Server).apiProtectTags},
68 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/protected_tags/{pattern...}", access.ScopeWrite, authUser, "Unprotect tag pattern", (*Server).apiUnprotectTag},
68 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/protected_tags/{pattern...}", access.ScopeWrite, authUser, "Unprotect tag pattern", (*Server).apiUnprotectTag},
69
69
70 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoRead, authUser, "List branches", (*Server).apiBranches},
70 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoRead, authOptional, "List branches", (*Server).apiBranches},
71 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoWrite, authUser, "Create branch", (*Server).apiBranches},
71 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/branches", access.ScopeRepoWrite, authUser, "Create branch", (*Server).apiBranches},
72 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/repository/branches/{branch...}", access.ScopeRepoWrite, authUser, "Rename branch", (*Server).apiBranch},
72 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/repository/branches/{branch...}", access.ScopeRepoWrite, authUser, "Rename branch", (*Server).apiBranch},
73 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/repository/branches/{branch...}", access.ScopeRepoWrite, authUser, "Delete branch", (*Server).apiBranch},
73 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/repository/branches/{branch...}", access.ScopeRepoWrite, authUser, "Delete branch", (*Server).apiBranch},
74 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoRead, authUser, "List tags", (*Server).apiTags},
74 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoRead, authOptional, "List tags", (*Server).apiTags},
75 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoWrite, authUser, "Create tag", (*Server).apiTags},
75 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/tags", access.ScopeRepoWrite, authUser, "Create tag", (*Server).apiTags},
76 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}", access.ScopeRepoWrite, authUser, "Delete tag", (*Server).apiDeleteTag},
76 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}", access.ScopeRepoWrite, authUser, "Delete tag", (*Server).apiDeleteTag},
77 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}/archive", access.ScopeRepoRead, authUser, "Download tag archive", (*Server).apiTagArchive},
77 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tags/{tag}/archive", access.ScopeRepoRead, authOptional, "Download tag archive", (*Server).apiTagArchive},
78 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tree", access.ScopeRepoRead, authUser, "List tree", (*Server).apiTree},
78 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/tree", access.ScopeRepoRead, authOptional, "List tree", (*Server).apiTree},
79 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/files/{path...}", access.ScopeRepoRead, authUser, "Read file", (*Server).apiFile},
79 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/files/{path...}", access.ScopeRepoRead, authOptional, "Read file", (*Server).apiFile},
80 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoRead, authUser, "List commits", (*Server).apiCommits},
80 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoRead, authOptional, "List commits", (*Server).apiCommits},
81 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoWrite, authUser, "Create commit", (*Server).apiCommits},
81 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/repository/commits", access.ScopeRepoWrite, authUser, "Create commit", (*Server).apiCommits},
82 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}", access.ScopeRepoRead, authUser, "Get commit", (*Server).apiCommit},
82 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}", access.ScopeRepoRead, authOptional, "Get commit", (*Server).apiCommit},
83 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}/diff", access.ScopeRepoRead, authUser, "Commit diff", (*Server).apiCommitDiff},
83 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/repository/commits/{sha}/diff", access.ScopeRepoRead, authOptional, "Commit diff", (*Server).apiCommitDiff},
84
84
85 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeRead, authUser, "List issues", (*Server).issuesAPI},
85 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeRead, authOptional, "List issues", (*Server).issuesAPI},
86 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeWrite, authUser, "Create issue", (*Server).issuesAPI},
86 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/issues", access.ScopeWrite, authUser, "Create issue", (*Server).issuesAPI},
87 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeRead, authUser, "Get issue", (*Server).issueAPI},
87 {http.MethodGet, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeRead, authOptional, "Get issue", (*Server).issueAPI},
88 {http.MethodPut, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeWrite, authUser, "Update issue", (*Server).issueAPI},
88 {http.MethodPut, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeWrite, authUser, "Update issue", (*Server).issueAPI},
89 {http.MethodDelete, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeWrite, authUser, "Delete issue", (*Server).issueAPI},
89 {http.MethodDelete, "/api/v1/repos/{owner}/{repo}/issues/{number}", access.ScopeWrite, authUser, "Delete issue", (*Server).issueAPI},
90 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/issues/{number}/notes", access.ScopeWrite, authUser, "Create issue note", (*Server).issueNoteAPI},
90 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/issues/{number}/notes", access.ScopeWrite, authUser, "Create issue note", (*Server).issueNoteAPI},
91
91
92 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeRead, authUser, "List merge requests", (*Server).apiMRs},
92 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeRead, authOptional, "List merge requests. Optional query: state (open, merged, closed, all), source_branch, target_branch.", (*Server).apiMRs},
93 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeWrite, authUser, "Create merge request", (*Server).apiMRs},
93 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests", access.ScopeWrite, authUser, "Create merge request", (*Server).apiMRs},
94 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeRead, authUser, "Get merge request", (*Server).apiMR},
94 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeRead, authOptional, "Get merge request", (*Server).apiMR},
95 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeWrite, authUser, "Update merge request", (*Server).apiMR},
95 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}", access.ScopeWrite, authUser, "Update merge request", (*Server).apiMR},
96 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeRead, authUser, "List notes", (*Server).apiMRNotes},
96 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeRead, authOptional, "List notes", (*Server).apiMRNotes},
97 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeWrite, authUser, "Create note", (*Server).apiMRNotes},
97 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/notes", access.ScopeWrite, authUser, "Create note", (*Server).apiMRNotes},
98 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reviews", access.ScopeWrite, authUser, "Submit review", (*Server).apiReview},
98 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reviews", access.ScopeWrite, authUser, "Submit review", (*Server).apiReview},
99 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reviews/dismiss", access.ScopeWrite, authUser, "Dismiss review", (*Server).apiDismissReview},
99 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reviews/dismiss", access.ScopeWrite, authUser, "Dismiss review", (*Server).apiDismissReview},
@@ -102,28 +102,31 @@ var apiCatalog = []apiRoute{
102 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/close", access.ScopeWrite, authUser, "Close merge request", (*Server).apiCloseMR},
102 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/close", access.ScopeWrite, authUser, "Close merge request", (*Server).apiCloseMR},
103 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reopen", access.ScopeWrite, authUser, "Reopen merge request", (*Server).apiReopenMR},
103 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/merge_requests/{number}/reopen", access.ScopeWrite, authUser, "Reopen merge request", (*Server).apiReopenMR},
104
104
105 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeRead, authUser, "List pipelines", (*Server).apiPipelines},
105 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeRead, authOptional, "List pipelines, newest first. Optional query: sha (7-40 hex, prefix), ref (branch, tag or full ref), mr (merge request number; its pipelines plus pushes to its source branch).", (*Server).apiPipelines},
106 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeWrite, authUser, "Start pipeline", (*Server).apiPipelines},
106 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines", access.ScopeWrite, authUser, "Start pipeline", (*Server).apiPipelines},
107 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}", access.ScopeRead, authUser, "Get pipeline", (*Server).apiPipeline},
107 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}", access.ScopeRead, authOptional, "Get pipeline", (*Server).apiPipeline},
108 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/cancel", access.ScopeWrite, authUser, "Cancel pipeline", (*Server).apiCancelPipeline},
108 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/cancel", access.ScopeWrite, authUser, "Cancel pipeline", (*Server).apiCancelPipeline},
109 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/retry", access.ScopeWrite, authUser, "Retry pipeline", (*Server).apiRetryPipeline},
109 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/retry", access.ScopeWrite, authUser, "Retry pipeline", (*Server).apiRetryPipeline},
110 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs", access.ScopeRead, authUser, "List jobs", (*Server).apiJobs},
110 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs", access.ScopeRead, authOptional, "List jobs", (*Server).apiJobs},
111 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}", access.ScopeRead, authUser, "Get job", (*Server).apiJob},
111 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}", access.ScopeRead, authOptional, "Get job", (*Server).apiJob},
112 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/log", access.ScopeRead, authUser, "Job log", (*Server).apiJobLog},
112 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/log", access.ScopeRead, authOptional, "Job log", (*Server).apiJobLog},
113 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/play", access.ScopeWrite, authUser, "Play manual job", (*Server).apiPlayJob},
113 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/play", access.ScopeWrite, authUser, "Play manual job", (*Server).apiPlayJob},
114 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/retry", access.ScopeWrite, authUser, "Retry job", (*Server).apiRetryJob},
114 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/retry", access.ScopeWrite, authUser, "Retry job", (*Server).apiRetryJob},
115 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/cancel", access.ScopeWrite, authUser, "Cancel job", (*Server).apiCancelJob},
115 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/cancel", access.ScopeWrite, authUser, "Cancel job", (*Server).apiCancelJob},
116 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts", access.ScopeRead, authUser, "List artifacts", (*Server).apiArtifacts},
116 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts", access.ScopeRead, authOptional, "List artifacts", (*Server).apiArtifacts},
117 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts/{path...}", access.ScopeRead, authUser, "Download artifact", (*Server).apiArtifact},
117 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/pipelines/{id}/jobs/{job}/artifacts/{path...}", access.ScopeRead, authOptional, "Download artifact", (*Server).apiArtifact},
118
118
119 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoRead, authUser, "Read CI file", (*Server).apiCIFile},
119 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoRead, authOptional, "Read CI file", (*Server).apiCIFile},
120 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoWrite, authUser, "Save CI file", (*Server).apiCIFile},
120 {http.MethodPut, "/api/v1/projects/{owner}/{repo}/ci/file", access.ScopeRepoWrite, authUser, "Save CI file", (*Server).apiCIFile},
121 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/ci/lint", access.ScopeRead, authUser, "Lint CI file", (*Server).apiLintCI},
121 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/ci/lint", access.ScopeRead, authUser, "Lint CI file", (*Server).apiLintCI},
122 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeRead, authUser, "List releases", (*Server).apiReleases},
122 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeRead, authOptional, "List releases", (*Server).apiReleases},
123 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeWrite, authUser, "Publish release", (*Server).createRelease},
123 {http.MethodPost, "/api/v1/projects/{owner}/{repo}/releases", access.ScopeWrite, authUser, "Publish release", (*Server).createRelease},
124 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeRead, authUser, "Download release asset", (*Server).apiReleaseAsset},
124 {http.MethodGet, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeRead, authOptional, "Download release asset", (*Server).apiReleaseAsset},
125 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases", access.ScopeWrite, authRelease, "Publish release from a job or user token", (*Server).createRelease},
125 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases", access.ScopeWrite, authRelease, "Publish release from a job or user token", (*Server).createRelease},
126
126
127 {http.MethodGet, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Read instance storage", (*Server).apiAdminStorage},
127 {http.MethodGet, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Read instance storage", (*Server).apiAdminStorage},
128 {http.MethodPut, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Set instance storage", (*Server).apiAdminStorage},
128 {http.MethodPut, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Set instance storage", (*Server).apiAdminStorage},
129 {http.MethodGet, "/api/v1/admin/cache", access.ScopeAdmin, authUser, "Read cache settings and usage", (*Server).apiAdminCache},
130 {http.MethodPut, "/api/v1/admin/cache", access.ScopeAdmin, authUser, "Set cache path", (*Server).apiAdminCache},
131 {http.MethodPost, "/api/v1/admin/cache/clear", access.ScopeAdmin, authUser, "Clear the read cache", (*Server).apiAdminCacheClear},
129}
132}
internal/server/api_test.go +10 −0
@@ -276,6 +276,16 @@ func TestAPIPermissionsAndResources(t *testing.T) {
276 276 want(code, body, "heropenen", 200, "open")
277 277 code, body = call(http.MethodPost, "/api/v1/projects/ada/demo/pipelines", devTok, `{"ref":"feature"}`, "")
278 278 want(code, body, "pipeline", 201, `"id"`)
279 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?ref=feature", roTok, "", "")
280 want(code, body, "pipelines op ref", 200, `"ref":"refs/heads/feature"`)
281 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?mr=1", roTok, "", "")
282 want(code, body, "pipelines op mr", 200, `"ref":"refs/heads/feature"`)
283 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?ref=bestaatniet", roTok, "", "")
284 want(code, body, "pipelines leeg", 200, `[]`)
285 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?sha=xyz", roTok, "", "")
286 want(code, body, "slechte sha", 400, "message")
287 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?mr=1&ref=feature", roTok, "", "")
288 want(code, body, "ref en mr", 400, "message")
279 289
280 290 code, body = call(http.MethodPost, "/api/v1/groups", adaTok, `{"name":"Team","description":"samen"}`, "")
281 291 want(code, body, "groep", 201, `"path":"team"`)
@@ -276,6 +276,16 @@ func TestAPIPermissionsAndResources(t *testing.T) {
276 want(code, body, "heropenen", 200, "open")
276 want(code, body, "heropenen", 200, "open")
277 code, body = call(http.MethodPost, "/api/v1/projects/ada/demo/pipelines", devTok, `{"ref":"feature"}`, "")
277 code, body = call(http.MethodPost, "/api/v1/projects/ada/demo/pipelines", devTok, `{"ref":"feature"}`, "")
278 want(code, body, "pipeline", 201, `"id"`)
278 want(code, body, "pipeline", 201, `"id"`)
279 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?ref=feature", roTok, "", "")
280 want(code, body, "pipelines op ref", 200, `"ref":"refs/heads/feature"`)
281 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?mr=1", roTok, "", "")
282 want(code, body, "pipelines op mr", 200, `"ref":"refs/heads/feature"`)
283 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?ref=bestaatniet", roTok, "", "")
284 want(code, body, "pipelines leeg", 200, `[]`)
285 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?sha=xyz", roTok, "", "")
286 want(code, body, "slechte sha", 400, "message")
287 code, body = call(http.MethodGet, "/api/v1/projects/ada/demo/pipelines?mr=1&ref=feature", roTok, "", "")
288 want(code, body, "ref en mr", 400, "message")
279
289
280 code, body = call(http.MethodPost, "/api/v1/groups", adaTok, `{"name":"Team","description":"samen"}`, "")
290 code, body = call(http.MethodPost, "/api/v1/groups", adaTok, `{"name":"Team","description":"samen"}`, "")
281 want(code, body, "groep", 201, `"path":"team"`)
291 want(code, body, "groep", 201, `"path":"team"`)
internal/server/api_visibility_test.go added +171 −0
@@ -0,0 +1,171 @@
1 package server
2
3 import (
4 "context"
5 "fmt"
6 "net/http"
7 "net/http/httptest"
8 "path/filepath"
9 "strings"
10 "testing"
11
12 "gitlab.com/Timfa/bluegit/internal/access"
13 "gitlab.com/Timfa/bluegit/internal/auth"
14 "gitlab.com/Timfa/bluegit/internal/config"
15 "gitlab.com/Timfa/bluegit/internal/store"
16 )
17
18 // Anoniem en als niet-lid: publiek lezen mag, privé en groepsprivé geeft 404,
19 // schrijven zonder token geeft 401. Geldt voor de leesroutes die agents gebruiken.
20 func TestAPIAnonymousReadsRespectVisibility(t *testing.T) {
21 storage := t.TempDir()
22 st, err := store.Open(filepath.Join(storage, "bluegit.db"), bytes32())
23 if err != nil {
24 t.Fatal(err)
25 }
26 t.Cleanup(func() { _ = st.Close() })
27 ctx := context.Background()
28 ada, _ := st.UpsertUser(ctx, 1, "ada", "", false)
29 eve, _ := st.UpsertUser(ctx, 2, "eve", "", false)
30 tok := func(u store.User) string {
31 plain, _, err := st.IssueToken(ctx, u.ID, store.TokenRequest{Name: "t", Scopes: "api"})
32 if err != nil {
33 t.Fatal(err)
34 }
35 return plain
36 }
37 adaTok, eveTok := tok(ada), tok(eve)
38 team, err := st.CreateGroup(ctx, ada.ID, "Team", "team", "", access.RoleDeveloper)
39 if err != nil {
40 t.Fatal(err)
41 }
42 type fixture struct {
43 path string
44 pipe, job int64
45 mr, issues int64
46 }
47 mk := func(kind string, owner int64, ownerPath, name, vis string) fixture {
48 repo, err := st.CreateRepo(ctx, kind, owner, name, "", "master", vis)
49 if err != nil {
50 t.Fatal(err)
51 }
52 sha := strings.Repeat("a", 40)
53 pid, err := st.CreatePipeline(ctx, repo.ID, sha, "refs/heads/master", "push", "ada", "c", []store.NewJob{{Name: "t", Stage: "test", Script: "true"}})
54 if err != nil {
55 t.Fatal(err)
56 }
57 jobs, _ := st.JobsByPipeline(ctx, pid)
58 _ = st.AppendJobLog(ctx, jobs[0].ID, "geheim-log-"+name+"\n")
59 mr, err := st.CreateMergeRequest(ctx, repo.ID, ada.ID, "mr "+name, "", "topic", "master")
60 if err != nil {
61 t.Fatal(err)
62 }
63 if _, err := st.CreateIssue(ctx, repo.ID, ada.ID, store.NewIssue{Title: "issue " + name}); err != nil {
64 t.Fatal(err)
65 }
66 return fixture{path: ownerPath + "/" + name, pipe: pid, job: jobs[0].ID, mr: mr.Number}
67 }
68 pub := mk("user", ada.ID, "ada", "pub", "public")
69 priv := mk("user", ada.ID, "ada", "priv", "private")
70 grp := mk("group", team.ID, "team", "intern", "private")
71
72 cfg := config.Config{
73 BaseURL: "http://127.0.0.1:9", StoragePath: storage,
74 Auth: config.Auth{Mode: "dev", DevUser: "ada", DevAuthID: 1},
75 Git: config.Git{Binary: "git", DefaultBranch: "master"},
76 CI: config.CI{Enabled: true},
77 }
78 srv := &Server{Cfg: cfg, Store: st, Auth: auth.Dev{Username: "ada", AuthID: 1}, Git: "git", HookBin: "/bin/true", StateDir: t.TempDir()}
79 ts := httptest.NewServer(srv.Handler())
80 t.Cleanup(ts.Close)
81 call := func(method, path, token string) (int, string) {
82 t.Helper()
83 req, _ := http.NewRequest(method, ts.URL+path, strings.NewReader("{}"))
84 req.Header.Set("Content-Type", "application/json")
85 if token != "" {
86 req.Header.Set("Authorization", "Bearer "+token)
87 }
88 res, err := http.DefaultClient.Do(req)
89 if err != nil {
90 t.Fatal(err)
91 }
92 return res.StatusCode, read(t, res)
93 }
94 reads := func(f fixture) []string {
95 p := "/api/v1/projects/" + f.path
96 return []string{
97 p,
98 p + "/pipelines",
99 p + "/pipelines?ref=master",
100 p + "/pipelines?sha=aaaaaaa",
101 fmt.Sprintf("%s/pipelines/%d", p, f.pipe),
102 fmt.Sprintf("%s/pipelines/%d/jobs", p, f.pipe),
103 fmt.Sprintf("%s/pipelines/%d/jobs/%d", p, f.pipe, f.job),
104 fmt.Sprintf("%s/pipelines/%d/jobs/%d/log", p, f.pipe, f.job),
105 p + "/merge_requests",
106 p + "/merge_requests?source_branch=topic",
107 fmt.Sprintf("%s/merge_requests/%d", p, f.mr),
108 fmt.Sprintf("%s/merge_requests/%d/notes", p, f.mr),
109 p + "/releases",
110 "/api/v1/repos/" + f.path + "/issues",
111 "/api/v1/repos/" + f.path + "/issues/1",
112 }
113 }
114 pubRepo, _ := st.RepoByPath(ctx, "ada", "pub")
115 if _, err := st.CreateIssue(ctx, pubRepo.ID, ada.ID, store.NewIssue{Title: "vertrouwelijk-x", Confidential: true}); err != nil {
116 t.Fatal(err)
117 }
118 for _, token := range []string{"", eveTok} {
119 if code, body := call(http.MethodGet, "/api/v1/repos/ada/pub/issues/2", token); code == 200 || strings.Contains(body, "vertrouwelijk-x") {
120 t.Fatalf("vertrouwelijk issue zichtbaar: %d %s", code, body)
121 }
122 if _, body := call(http.MethodGet, "/api/v1/repos/ada/pub/issues", token); strings.Contains(body, "vertrouwelijk-x") {
123 t.Fatalf("vertrouwelijk issue in lijst: %s", body)
124 }
125 }
126 for _, path := range reads(pub) {
127 if code, body := call(http.MethodGet, path, ""); code != 200 {
128 t.Fatalf("anoniem publiek %s: %d %s", path, code, body)
129 }
130 }
131 for _, f := range []fixture{priv, grp} {
132 for _, path := range reads(f) {
133 for who, token := range map[string]string{"anoniem": "", "niet-lid": eveTok} {
134 code, body := call(http.MethodGet, path, token)
135 if code != 404 || strings.Contains(body, "geheim-log") || strings.Contains(body, "mr ") || strings.Contains(body, "issue ") {
136 t.Fatalf("%s ziet privé %s: %d %s", who, path, code, body)
137 }
138 }
139 if code, body := call(http.MethodGet, path, adaTok); code != 200 {
140 t.Fatalf("eigenaar %s: %d %s", path, code, body)
141 }
142 }
143 }
144 writes := func(f fixture) []string {
145 p := "/api/v1/projects/" + f.path
146 return []string{
147 p + "/merge_requests",
148 p + "/releases",
149 fmt.Sprintf("%s/merge_requests/%d/notes", p, f.mr),
150 fmt.Sprintf("%s/pipelines/%d/retry", p, f.pipe),
151 "/api/v1/repos/" + f.path + "/issues",
152 }
153 }
154 for _, f := range []fixture{pub, priv, grp} {
155 for _, path := range writes(f) {
156 if code, body := call(http.MethodPost, path, ""); code != 401 {
157 t.Fatalf("anoniem schrijven %s: %d %s", path, code, body)
158 }
159 }
160 }
161 for _, f := range []fixture{priv, grp} {
162 for _, path := range writes(f) {
163 if code, body := call(http.MethodPost, path, eveTok); code != 404 {
164 t.Fatalf("niet-lid schrijft privé %s: %d %s", path, code, body)
165 }
166 }
167 }
168 if code, body := call(http.MethodPost, fmt.Sprintf("/api/v1/projects/%s/pipelines/%d/retry", pub.path, pub.pipe), eveTok); code != 403 {
169 t.Fatalf("niet-lid retry publiek: %d %s", code, body)
170 }
171 }
@@ -0,0 +1,171 @@
1package server
2
3import (
4 "context"
5 "fmt"
6 "net/http"
7 "net/http/httptest"
8 "path/filepath"
9 "strings"
10 "testing"
11
12 "gitlab.com/Timfa/bluegit/internal/access"
13 "gitlab.com/Timfa/bluegit/internal/auth"
14 "gitlab.com/Timfa/bluegit/internal/config"
15 "gitlab.com/Timfa/bluegit/internal/store"
16)
17
18// Anoniem en als niet-lid: publiek lezen mag, privé en groepsprivé geeft 404,
19// schrijven zonder token geeft 401. Geldt voor de leesroutes die agents gebruiken.
20func TestAPIAnonymousReadsRespectVisibility(t *testing.T) {
21 storage := t.TempDir()
22 st, err := store.Open(filepath.Join(storage, "bluegit.db"), bytes32())
23 if err != nil {
24 t.Fatal(err)
25 }
26 t.Cleanup(func() { _ = st.Close() })
27 ctx := context.Background()
28 ada, _ := st.UpsertUser(ctx, 1, "ada", "", false)
29 eve, _ := st.UpsertUser(ctx, 2, "eve", "", false)
30 tok := func(u store.User) string {
31 plain, _, err := st.IssueToken(ctx, u.ID, store.TokenRequest{Name: "t", Scopes: "api"})
32 if err != nil {
33 t.Fatal(err)
34 }
35 return plain
36 }
37 adaTok, eveTok := tok(ada), tok(eve)
38 team, err := st.CreateGroup(ctx, ada.ID, "Team", "team", "", access.RoleDeveloper)
39 if err != nil {
40 t.Fatal(err)
41 }
42 type fixture struct {
43 path string
44 pipe, job int64
45 mr, issues int64
46 }
47 mk := func(kind string, owner int64, ownerPath, name, vis string) fixture {
48 repo, err := st.CreateRepo(ctx, kind, owner, name, "", "master", vis)
49 if err != nil {
50 t.Fatal(err)
51 }
52 sha := strings.Repeat("a", 40)
53 pid, err := st.CreatePipeline(ctx, repo.ID, sha, "refs/heads/master", "push", "ada", "c", []store.NewJob{{Name: "t", Stage: "test", Script: "true"}})
54 if err != nil {
55 t.Fatal(err)
56 }
57 jobs, _ := st.JobsByPipeline(ctx, pid)
58 _ = st.AppendJobLog(ctx, jobs[0].ID, "geheim-log-"+name+"\n")
59 mr, err := st.CreateMergeRequest(ctx, repo.ID, ada.ID, "mr "+name, "", "topic", "master")
60 if err != nil {
61 t.Fatal(err)
62 }
63 if _, err := st.CreateIssue(ctx, repo.ID, ada.ID, store.NewIssue{Title: "issue " + name}); err != nil {
64 t.Fatal(err)
65 }
66 return fixture{path: ownerPath + "/" + name, pipe: pid, job: jobs[0].ID, mr: mr.Number}
67 }
68 pub := mk("user", ada.ID, "ada", "pub", "public")
69 priv := mk("user", ada.ID, "ada", "priv", "private")
70 grp := mk("group", team.ID, "team", "intern", "private")
71
72 cfg := config.Config{
73 BaseURL: "http://127.0.0.1:9", StoragePath: storage,
74 Auth: config.Auth{Mode: "dev", DevUser: "ada", DevAuthID: 1},
75 Git: config.Git{Binary: "git", DefaultBranch: "master"},
76 CI: config.CI{Enabled: true},
77 }
78 srv := &Server{Cfg: cfg, Store: st, Auth: auth.Dev{Username: "ada", AuthID: 1}, Git: "git", HookBin: "/bin/true", StateDir: t.TempDir()}
79 ts := httptest.NewServer(srv.Handler())
80 t.Cleanup(ts.Close)
81 call := func(method, path, token string) (int, string) {
82 t.Helper()
83 req, _ := http.NewRequest(method, ts.URL+path, strings.NewReader("{}"))
84 req.Header.Set("Content-Type", "application/json")
85 if token != "" {
86 req.Header.Set("Authorization", "Bearer "+token)
87 }
88 res, err := http.DefaultClient.Do(req)
89 if err != nil {
90 t.Fatal(err)
91 }
92 return res.StatusCode, read(t, res)
93 }
94 reads := func(f fixture) []string {
95 p := "/api/v1/projects/" + f.path
96 return []string{
97 p,
98 p + "/pipelines",
99 p + "/pipelines?ref=master",
100 p + "/pipelines?sha=aaaaaaa",
101 fmt.Sprintf("%s/pipelines/%d", p, f.pipe),
102 fmt.Sprintf("%s/pipelines/%d/jobs", p, f.pipe),
103 fmt.Sprintf("%s/pipelines/%d/jobs/%d", p, f.pipe, f.job),
104 fmt.Sprintf("%s/pipelines/%d/jobs/%d/log", p, f.pipe, f.job),
105 p + "/merge_requests",
106 p + "/merge_requests?source_branch=topic",
107 fmt.Sprintf("%s/merge_requests/%d", p, f.mr),
108 fmt.Sprintf("%s/merge_requests/%d/notes", p, f.mr),
109 p + "/releases",
110 "/api/v1/repos/" + f.path + "/issues",
111 "/api/v1/repos/" + f.path + "/issues/1",
112 }
113 }
114 pubRepo, _ := st.RepoByPath(ctx, "ada", "pub")
115 if _, err := st.CreateIssue(ctx, pubRepo.ID, ada.ID, store.NewIssue{Title: "vertrouwelijk-x", Confidential: true}); err != nil {
116 t.Fatal(err)
117 }
118 for _, token := range []string{"", eveTok} {
119 if code, body := call(http.MethodGet, "/api/v1/repos/ada/pub/issues/2", token); code == 200 || strings.Contains(body, "vertrouwelijk-x") {
120 t.Fatalf("vertrouwelijk issue zichtbaar: %d %s", code, body)
121 }
122 if _, body := call(http.MethodGet, "/api/v1/repos/ada/pub/issues", token); strings.Contains(body, "vertrouwelijk-x") {
123 t.Fatalf("vertrouwelijk issue in lijst: %s", body)
124 }
125 }
126 for _, path := range reads(pub) {
127 if code, body := call(http.MethodGet, path, ""); code != 200 {
128 t.Fatalf("anoniem publiek %s: %d %s", path, code, body)
129 }
130 }
131 for _, f := range []fixture{priv, grp} {
132 for _, path := range reads(f) {
133 for who, token := range map[string]string{"anoniem": "", "niet-lid": eveTok} {
134 code, body := call(http.MethodGet, path, token)
135 if code != 404 || strings.Contains(body, "geheim-log") || strings.Contains(body, "mr ") || strings.Contains(body, "issue ") {
136 t.Fatalf("%s ziet privé %s: %d %s", who, path, code, body)
137 }
138 }
139 if code, body := call(http.MethodGet, path, adaTok); code != 200 {
140 t.Fatalf("eigenaar %s: %d %s", path, code, body)
141 }
142 }
143 }
144 writes := func(f fixture) []string {
145 p := "/api/v1/projects/" + f.path
146 return []string{
147 p + "/merge_requests",
148 p + "/releases",
149 fmt.Sprintf("%s/merge_requests/%d/notes", p, f.mr),
150 fmt.Sprintf("%s/pipelines/%d/retry", p, f.pipe),
151 "/api/v1/repos/" + f.path + "/issues",
152 }
153 }
154 for _, f := range []fixture{pub, priv, grp} {
155 for _, path := range writes(f) {
156 if code, body := call(http.MethodPost, path, ""); code != 401 {
157 t.Fatalf("anoniem schrijven %s: %d %s", path, code, body)
158 }
159 }
160 }
161 for _, f := range []fixture{priv, grp} {
162 for _, path := range writes(f) {
163 if code, body := call(http.MethodPost, path, eveTok); code != 404 {
164 t.Fatalf("niet-lid schrijft privé %s: %d %s", path, code, body)
165 }
166 }
167 }
168 if code, body := call(http.MethodPost, fmt.Sprintf("/api/v1/projects/%s/pipelines/%d/retry", pub.path, pub.pipe), eveTok); code != 403 {
169 t.Fatalf("niet-lid retry publiek: %d %s", code, body)
170 }
171}
internal/server/branches.go +3 −4
@@ -27,8 +27,7 @@ func (s *Server) branches(w http.ResponseWriter, r *http.Request) {
27 27 v := s.repoView(w, r, repo, role, "title.branches")
28 28 v.Page = "branches"
29 29 v.Wide = true
30 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
31 tips, _ := gitrepo.BranchTips(r.Context(), s.Git, dir)
30 tips, _ := s.cBranchTips(r.Context(), repo)
32 31 patterns, _ := s.Store.ProtectedPatterns(r.Context(), repo.ID)
33 32 lang := s.langOf(r)
34 33 now := time.Now()
@@ -47,7 +46,7 @@ func (s *Server) branches(w http.ResponseWriter, r *http.Request) {
47 46 }
48 47 row.CanEdit = role.CanWriteCode() && !row.Default && !row.Protected
49 48 if !row.Default && repo.DefaultBranch != "" {
50 ahead, behind, err := gitrepo.AheadBehind(r.Context(), s.Git, dir, repo.DefaultBranch, tip.Name)
49 ahead, behind, err := s.cAheadBehind(r.Context(), repo, repo.DefaultBranch, tip.Name)
51 50 if err == nil {
52 51 row.Ahead, row.Behind = ahead, behind
53 52 }
@@ -95,7 +94,7 @@ func (s *Server) branches(w http.ResponseWriter, r *http.Request) {
95 94 board.ShowStale = len(board.Stale) > 0
96 95 }
97 96 v.Branch = board
98 tags, _ := gitrepo.Tags(r.Context(), s.Git, dir)
97 tags, _ := s.cTags(r.Context(), repo)
99 98 for _, name := range tags {
100 99 v.Tags = append(v.Tags, TagRow{Name: name})
101 100 }
@@ -27,8 +27,7 @@ func (s *Server) branches(w http.ResponseWriter, r *http.Request) {
27 v := s.repoView(w, r, repo, role, "title.branches")
27 v := s.repoView(w, r, repo, role, "title.branches")
28 v.Page = "branches"
28 v.Page = "branches"
29 v.Wide = true
29 v.Wide = true
30 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
30 tips, _ := s.cBranchTips(r.Context(), repo)
31 tips, _ := gitrepo.BranchTips(r.Context(), s.Git, dir)
32 patterns, _ := s.Store.ProtectedPatterns(r.Context(), repo.ID)
31 patterns, _ := s.Store.ProtectedPatterns(r.Context(), repo.ID)
33 lang := s.langOf(r)
32 lang := s.langOf(r)
34 now := time.Now()
33 now := time.Now()
@@ -47,7 +46,7 @@ func (s *Server) branches(w http.ResponseWriter, r *http.Request) {
47 }
46 }
48 row.CanEdit = role.CanWriteCode() && !row.Default && !row.Protected
47 row.CanEdit = role.CanWriteCode() && !row.Default && !row.Protected
49 if !row.Default && repo.DefaultBranch != "" {
48 if !row.Default && repo.DefaultBranch != "" {
50 ahead, behind, err := gitrepo.AheadBehind(r.Context(), s.Git, dir, repo.DefaultBranch, tip.Name)
49 ahead, behind, err := s.cAheadBehind(r.Context(), repo, repo.DefaultBranch, tip.Name)
51 if err == nil {
50 if err == nil {
52 row.Ahead, row.Behind = ahead, behind
51 row.Ahead, row.Behind = ahead, behind
53 }
52 }
@@ -95,7 +94,7 @@ func (s *Server) branches(w http.ResponseWriter, r *http.Request) {
95 board.ShowStale = len(board.Stale) > 0
94 board.ShowStale = len(board.Stale) > 0
96 }
95 }
97 v.Branch = board
96 v.Branch = board
98 tags, _ := gitrepo.Tags(r.Context(), s.Git, dir)
97 tags, _ := s.cTags(r.Context(), repo)
99 for _, name := range tags {
98 for _, name := range tags {
100 v.Tags = append(v.Tags, TagRow{Name: name})
99 v.Tags = append(v.Tags, TagRow{Name: name})
101 }
100 }
internal/server/editor.go +1 −1
@@ -83,7 +83,7 @@ func (s *Server) ciEditor(w http.ResponseWriter, r *http.Request) {
83 83 v.Page = "editor"
84 84 v.Rev = ref
85 85 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
86 s.branchList(r, &v, dir)
86 s.branchList(r, &v, repo)
87 87 body, missing := s.ciFileAt(r, dir, ref)
88 88 if missing {
89 89 body = ciTemplate
@@ -83,7 +83,7 @@ func (s *Server) ciEditor(w http.ResponseWriter, r *http.Request) {
83 v.Page = "editor"
83 v.Page = "editor"
84 v.Rev = ref
84 v.Rev = ref
85 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
85 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
86 s.branchList(r, &v, dir)
86 s.branchList(r, &v, repo)
87 body, missing := s.ciFileAt(r, dir, ref)
87 body, missing := s.ciFileAt(r, dir, ref)
88 if missing {
88 if missing {
89 body = ciTemplate
89 body = ciTemplate
internal/server/events.go +1 −0
@@ -44,6 +44,7 @@ func (s *Server) postReceive(ctx context.Context, repo store.Repo, user store.Us
44 44 // AfterPush behandelt elke nieuwe ref zoals een git push.
45 45 // git http, SSH en postReceive zijn de enige aanroepers.
46 46 func (s *Server) AfterPush(repo store.Repo, user store.User, before, after map[string]string) {
47 s.refsChanged(repo)
47 48 if after == nil {
48 49 return
49 50 }
@@ -44,6 +44,7 @@ func (s *Server) postReceive(ctx context.Context, repo store.Repo, user store.Us
44// AfterPush behandelt elke nieuwe ref zoals een git push.
44// AfterPush behandelt elke nieuwe ref zoals een git push.
45// git http, SSH en postReceive zijn de enige aanroepers.
45// git http, SSH en postReceive zijn de enige aanroepers.
46func (s *Server) AfterPush(repo store.Repo, user store.User, before, after map[string]string) {
46func (s *Server) AfterPush(repo store.Repo, user store.User, before, after map[string]string) {
47 s.refsChanged(repo)
47 if after == nil {
48 if after == nil {
48 return
49 return
49 }
50 }
internal/server/gitcache.go added +204 −0
@@ -0,0 +1,204 @@
1 package server
2
3 import (
4 "context"
5 "regexp"
6 "strconv"
7
8 "gitlab.com/Timfa/bluegit/internal/gitrepo"
9 "gitlab.com/Timfa/bluegit/internal/readcache"
10 "gitlab.com/Timfa/bluegit/internal/store"
11 )
12
13 // De leescache zit tussen de pagina's en git. Toegang is al gecontroleerd
14 // voordat een handler hier komt (guard/repoView); de cache weet niets van
15 // rechten en geeft alleen terug wat git voor dezelfde repo zou geven.
16 // Sleutels op een commit-SHA zijn onveranderlijk. Alles op een ref-naam
17 // staat in de refs-ruimte en verdwijnt bij elke ref-wijziging (AfterPush).
18
19 // ReadCache is de actieve cache, of nil.
20 func (s *Server) ReadCache() *readcache.Cache {
21 s.cacheMu.RLock()
22 defer s.cacheMu.RUnlock()
23 return s.rcache
24 }
25
26 // SetReadCache zet of wist de cache.
27 func (s *Server) SetReadCache(c *readcache.Cache) {
28 s.cacheMu.Lock()
29 s.rcache = c
30 s.cacheMu.Unlock()
31 }
32
33 var fullSHA = regexp.MustCompile(`^[0-9a-f]{40}([0-9a-f]{24})?$`)
34
35 func cachedRead[T any](s *Server, repoID int64, refs bool, key string, fn func() (T, error)) (T, error) {
36 c := s.ReadCache()
37 var v T
38 if c != nil && c.GetJSON(repoID, refs, key, &v) {
39 return v, nil
40 }
41 gen := c.Gen(repoID)
42 v, err := fn()
43 if err == nil && c != nil {
44 c.PutJSONAt(repoID, refs, key, v, gen)
45 }
46 return v, err
47 }
48
49 func (s *Server) gdir(repo store.Repo) string {
50 return gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
51 }
52
53 func (s *Server) cRefs(ctx context.Context, repo store.Repo) (map[string]string, error) {
54 return cachedRead(s, repo.ID, true, "refs", func() (map[string]string, error) {
55 return gitrepo.Refs(ctx, s.Git, s.gdir(repo))
56 })
57 }
58
59 // revKey kiest de cachesleutel voor rev: de commit-SHA als die bekend is,
60 // anders de naam in de refs-ruimte.
61 func (s *Server) revKey(ctx context.Context, repo store.Repo, rev string) (refs bool, key string) {
62 if fullSHA.MatchString(rev) {
63 return false, rev
64 }
65 if s.ReadCache() == nil {
66 return true, rev
67 }
68 if m, err := s.cRefs(ctx, repo); err == nil {
69 for _, full := range []string{"refs/heads/" + rev, "refs/tags/" + rev, rev} {
70 if sha, ok := m[full]; ok && fullSHA.MatchString(sha) {
71 return false, sha
72 }
73 }
74 }
75 return true, rev
76 }
77
78 func (s *Server) cBranches(ctx context.Context, repo store.Repo) ([]string, error) {
79 return cachedRead(s, repo.ID, true, "branches", func() ([]string, error) {
80 return gitrepo.Branches(ctx, s.Git, s.gdir(repo))
81 })
82 }
83
84 func (s *Server) cBranchTips(ctx context.Context, repo store.Repo) ([]gitrepo.BranchTip, error) {
85 return cachedRead(s, repo.ID, true, "tips", func() ([]gitrepo.BranchTip, error) {
86 return gitrepo.BranchTips(ctx, s.Git, s.gdir(repo))
87 })
88 }
89
90 func (s *Server) cTags(ctx context.Context, repo store.Repo) ([]string, error) {
91 return cachedRead(s, repo.ID, true, "tags", func() ([]string, error) {
92 return gitrepo.Tags(ctx, s.Git, s.gdir(repo))
93 })
94 }
95
96 func (s *Server) cListTags(ctx context.Context, repo store.Repo) ([]gitrepo.TagInfo, error) {
97 return cachedRead(s, repo.ID, true, "listtags", func() ([]gitrepo.TagInfo, error) {
98 return gitrepo.ListTags(ctx, s.Git, s.gdir(repo))
99 })
100 }
101
102 func (s *Server) cAheadBehind(ctx context.Context, repo store.Repo, base, branch string) (int, int, error) {
103 _, kb := s.revKey(ctx, repo, base)
104 _, kr := s.revKey(ctx, repo, branch)
105 refs := !fullSHA.MatchString(kb) || !fullSHA.MatchString(kr)
106 v, err := cachedRead(s, repo.ID, refs, "ab:"+kb+":"+kr, func() ([2]int, error) {
107 a, b, err := gitrepo.AheadBehind(ctx, s.Git, s.gdir(repo), base, branch)
108 return [2]int{a, b}, err
109 })
110 return v[0], v[1], err
111 }
112
113 type statsVal struct{ Commits, Branches, Tags int }
114
115 func (s *Server) cStats(ctx context.Context, repo store.Repo, rev string) (int, int, int, error) {
116 // branches en tags hangen van refs af; dus altijd in de refs-ruimte
117 v, err := cachedRead(s, repo.ID, true, "stats:"+rev, func() (statsVal, error) {
118 c, b, t, err := gitrepo.Stats(ctx, s.Git, s.gdir(repo), rev)
119 return statsVal{c, b, t}, err
120 })
121 return v.Commits, v.Branches, v.Tags, err
122 }
123
124 func (s *Server) cLastCommit(ctx context.Context, repo store.Repo, rev, path string) (gitrepo.Commit, error) {
125 refs, k := s.revKey(ctx, repo, rev)
126 return cachedRead(s, repo.ID, refs, "last:"+k+":"+path, func() (gitrepo.Commit, error) {
127 return gitrepo.LastCommit(ctx, s.Git, s.gdir(repo), rev, path)
128 })
129 }
130
131 func (s *Server) cTree(ctx context.Context, repo store.Repo, rev, rel string) ([]gitrepo.Entry, error) {
132 refs, k := s.revKey(ctx, repo, rev)
133 return cachedRead(s, repo.ID, refs, "tree:"+k+":"+rel, func() ([]gitrepo.Entry, error) {
134 return gitrepo.Tree(ctx, s.Git, s.gdir(repo), rev, rel)
135 })
136 }
137
138 func (s *Server) cTreeAll(ctx context.Context, repo store.Repo, rev string) ([]gitrepo.Entry, error) {
139 refs, k := s.revKey(ctx, repo, rev)
140 return cachedRead(s, repo.ID, refs, "treeall:"+k, func() ([]gitrepo.Entry, error) {
141 return gitrepo.TreeAll(ctx, s.Git, s.gdir(repo), rev)
142 })
143 }
144
145 func (s *Server) cTouch(ctx context.Context, repo store.Repo, rev string) (map[string]gitrepo.Commit, error) {
146 refs, k := s.revKey(ctx, repo, rev)
147 return cachedRead(s, repo.ID, refs, "touch:"+k, func() (map[string]gitrepo.Commit, error) {
148 return gitrepo.Touch(ctx, s.Git, s.gdir(repo), rev)
149 })
150 }
151
152 type blobVal struct {
153 Text string
154 Binary bool
155 }
156
157 // cDoc is voor kleine documenten zoals de README; grote bestanden lezen
158 // we gewoon uit git.
159 func (s *Server) cDoc(ctx context.Context, repo store.Repo, rev, path string, max int) (string, bool, error) {
160 refs, k := s.revKey(ctx, repo, rev)
161 v, err := cachedRead(s, repo.ID, refs, "doc:"+strconv.Itoa(max)+":"+k+":"+path, func() (blobVal, error) {
162 t, b, err := gitrepo.Blob(ctx, s.Git, s.gdir(repo), rev, path, max)
163 return blobVal{t, b}, err
164 })
165 return v.Text, v.Binary, err
166 }
167
168 // refreshCache is de write-through na elke ref-wijziging: oude ref-data
169 // weg, daarna de overzichtspagina van de standaardbranch opnieuw vullen.
170 // refsChanged vergeet de ref-data meteen (dus nooit verouderd) en vult
171 // op de achtergrond opnieuw.
172 func (s *Server) refsChanged(repo store.Repo) {
173 c := s.ReadCache()
174 if c == nil {
175 return
176 }
177 c.DropRefs(repo.ID)
178 go s.refreshCache(repo)
179 }
180
181 func (s *Server) refreshCache(repo store.Repo) {
182 if s.ReadCache() == nil {
183 return
184 }
185 ctx := context.Background()
186 _, _ = s.cRefs(ctx, repo)
187 _, _ = s.cBranches(ctx, repo)
188 _, _ = s.cBranchTips(ctx, repo)
189 _, _ = s.cTags(ctx, repo)
190 _, _ = s.cListTags(ctx, repo)
191 rev := repo.DefaultBranch
192 if rev == "" {
193 return
194 }
195 _, _, _, _ = s.cStats(ctx, repo, rev)
196 _, _ = s.cLastCommit(ctx, repo, rev, "")
197 _, _ = s.cTouch(ctx, repo, rev)
198 if entries, err := s.cTree(ctx, repo, rev, ""); err == nil {
199 if readme, _ := findDocs(entries); readme != "" {
200 _, _, _ = s.cDoc(ctx, repo, rev, readme, 200_000)
201 }
202 }
203 _ = s.repoLanguage(ctx, repo, rev)
204 }
@@ -0,0 +1,204 @@
1package server
2
3import (
4 "context"
5 "regexp"
6 "strconv"
7
8 "gitlab.com/Timfa/bluegit/internal/gitrepo"
9 "gitlab.com/Timfa/bluegit/internal/readcache"
10 "gitlab.com/Timfa/bluegit/internal/store"
11)
12
13// De leescache zit tussen de pagina's en git. Toegang is al gecontroleerd
14// voordat een handler hier komt (guard/repoView); de cache weet niets van
15// rechten en geeft alleen terug wat git voor dezelfde repo zou geven.
16// Sleutels op een commit-SHA zijn onveranderlijk. Alles op een ref-naam
17// staat in de refs-ruimte en verdwijnt bij elke ref-wijziging (AfterPush).
18
19// ReadCache is de actieve cache, of nil.
20func (s *Server) ReadCache() *readcache.Cache {
21 s.cacheMu.RLock()
22 defer s.cacheMu.RUnlock()
23 return s.rcache
24}
25
26// SetReadCache zet of wist de cache.
27func (s *Server) SetReadCache(c *readcache.Cache) {
28 s.cacheMu.Lock()
29 s.rcache = c
30 s.cacheMu.Unlock()
31}
32
33var fullSHA = regexp.MustCompile(`^[0-9a-f]{40}([0-9a-f]{24})?$`)
34
35func cachedRead[T any](s *Server, repoID int64, refs bool, key string, fn func() (T, error)) (T, error) {
36 c := s.ReadCache()
37 var v T
38 if c != nil && c.GetJSON(repoID, refs, key, &v) {
39 return v, nil
40 }
41 gen := c.Gen(repoID)
42 v, err := fn()
43 if err == nil && c != nil {
44 c.PutJSONAt(repoID, refs, key, v, gen)
45 }
46 return v, err
47}
48
49func (s *Server) gdir(repo store.Repo) string {
50 return gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
51}
52
53func (s *Server) cRefs(ctx context.Context, repo store.Repo) (map[string]string, error) {
54 return cachedRead(s, repo.ID, true, "refs", func() (map[string]string, error) {
55 return gitrepo.Refs(ctx, s.Git, s.gdir(repo))
56 })
57}
58
59// revKey kiest de cachesleutel voor rev: de commit-SHA als die bekend is,
60// anders de naam in de refs-ruimte.
61func (s *Server) revKey(ctx context.Context, repo store.Repo, rev string) (refs bool, key string) {
62 if fullSHA.MatchString(rev) {
63 return false, rev
64 }
65 if s.ReadCache() == nil {
66 return true, rev
67 }
68 if m, err := s.cRefs(ctx, repo); err == nil {
69 for _, full := range []string{"refs/heads/" + rev, "refs/tags/" + rev, rev} {
70 if sha, ok := m[full]; ok && fullSHA.MatchString(sha) {
71 return false, sha
72 }
73 }
74 }
75 return true, rev
76}
77
78func (s *Server) cBranches(ctx context.Context, repo store.Repo) ([]string, error) {
79 return cachedRead(s, repo.ID, true, "branches", func() ([]string, error) {
80 return gitrepo.Branches(ctx, s.Git, s.gdir(repo))
81 })
82}
83
84func (s *Server) cBranchTips(ctx context.Context, repo store.Repo) ([]gitrepo.BranchTip, error) {
85 return cachedRead(s, repo.ID, true, "tips", func() ([]gitrepo.BranchTip, error) {
86 return gitrepo.BranchTips(ctx, s.Git, s.gdir(repo))
87 })
88}
89
90func (s *Server) cTags(ctx context.Context, repo store.Repo) ([]string, error) {
91 return cachedRead(s, repo.ID, true, "tags", func() ([]string, error) {
92 return gitrepo.Tags(ctx, s.Git, s.gdir(repo))
93 })
94}
95
96func (s *Server) cListTags(ctx context.Context, repo store.Repo) ([]gitrepo.TagInfo, error) {
97 return cachedRead(s, repo.ID, true, "listtags", func() ([]gitrepo.TagInfo, error) {
98 return gitrepo.ListTags(ctx, s.Git, s.gdir(repo))
99 })
100}
101
102func (s *Server) cAheadBehind(ctx context.Context, repo store.Repo, base, branch string) (int, int, error) {
103 _, kb := s.revKey(ctx, repo, base)
104 _, kr := s.revKey(ctx, repo, branch)
105 refs := !fullSHA.MatchString(kb) || !fullSHA.MatchString(kr)
106 v, err := cachedRead(s, repo.ID, refs, "ab:"+kb+":"+kr, func() ([2]int, error) {
107 a, b, err := gitrepo.AheadBehind(ctx, s.Git, s.gdir(repo), base, branch)
108 return [2]int{a, b}, err
109 })
110 return v[0], v[1], err
111}
112
113type statsVal struct{ Commits, Branches, Tags int }
114
115func (s *Server) cStats(ctx context.Context, repo store.Repo, rev string) (int, int, int, error) {
116 // branches en tags hangen van refs af; dus altijd in de refs-ruimte
117 v, err := cachedRead(s, repo.ID, true, "stats:"+rev, func() (statsVal, error) {
118 c, b, t, err := gitrepo.Stats(ctx, s.Git, s.gdir(repo), rev)
119 return statsVal{c, b, t}, err
120 })
121 return v.Commits, v.Branches, v.Tags, err
122}
123
124func (s *Server) cLastCommit(ctx context.Context, repo store.Repo, rev, path string) (gitrepo.Commit, error) {
125 refs, k := s.revKey(ctx, repo, rev)
126 return cachedRead(s, repo.ID, refs, "last:"+k+":"+path, func() (gitrepo.Commit, error) {
127 return gitrepo.LastCommit(ctx, s.Git, s.gdir(repo), rev, path)
128 })
129}
130
131func (s *Server) cTree(ctx context.Context, repo store.Repo, rev, rel string) ([]gitrepo.Entry, error) {
132 refs, k := s.revKey(ctx, repo, rev)
133 return cachedRead(s, repo.ID, refs, "tree:"+k+":"+rel, func() ([]gitrepo.Entry, error) {
134 return gitrepo.Tree(ctx, s.Git, s.gdir(repo), rev, rel)
135 })
136}
137
138func (s *Server) cTreeAll(ctx context.Context, repo store.Repo, rev string) ([]gitrepo.Entry, error) {
139 refs, k := s.revKey(ctx, repo, rev)
140 return cachedRead(s, repo.ID, refs, "treeall:"+k, func() ([]gitrepo.Entry, error) {
141 return gitrepo.TreeAll(ctx, s.Git, s.gdir(repo), rev)
142 })
143}
144
145func (s *Server) cTouch(ctx context.Context, repo store.Repo, rev string) (map[string]gitrepo.Commit, error) {
146 refs, k := s.revKey(ctx, repo, rev)
147 return cachedRead(s, repo.ID, refs, "touch:"+k, func() (map[string]gitrepo.Commit, error) {
148 return gitrepo.Touch(ctx, s.Git, s.gdir(repo), rev)
149 })
150}
151
152type blobVal struct {
153 Text string
154 Binary bool
155}
156
157// cDoc is voor kleine documenten zoals de README; grote bestanden lezen
158// we gewoon uit git.
159func (s *Server) cDoc(ctx context.Context, repo store.Repo, rev, path string, max int) (string, bool, error) {
160 refs, k := s.revKey(ctx, repo, rev)
161 v, err := cachedRead(s, repo.ID, refs, "doc:"+strconv.Itoa(max)+":"+k+":"+path, func() (blobVal, error) {
162 t, b, err := gitrepo.Blob(ctx, s.Git, s.gdir(repo), rev, path, max)
163 return blobVal{t, b}, err
164 })
165 return v.Text, v.Binary, err
166}
167
168// refreshCache is de write-through na elke ref-wijziging: oude ref-data
169// weg, daarna de overzichtspagina van de standaardbranch opnieuw vullen.
170// refsChanged vergeet de ref-data meteen (dus nooit verouderd) en vult
171// op de achtergrond opnieuw.
172func (s *Server) refsChanged(repo store.Repo) {
173 c := s.ReadCache()
174 if c == nil {
175 return
176 }
177 c.DropRefs(repo.ID)
178 go s.refreshCache(repo)
179}
180
181func (s *Server) refreshCache(repo store.Repo) {
182 if s.ReadCache() == nil {
183 return
184 }
185 ctx := context.Background()
186 _, _ = s.cRefs(ctx, repo)
187 _, _ = s.cBranches(ctx, repo)
188 _, _ = s.cBranchTips(ctx, repo)
189 _, _ = s.cTags(ctx, repo)
190 _, _ = s.cListTags(ctx, repo)
191 rev := repo.DefaultBranch
192 if rev == "" {
193 return
194 }
195 _, _, _, _ = s.cStats(ctx, repo, rev)
196 _, _ = s.cLastCommit(ctx, repo, rev, "")
197 _, _ = s.cTouch(ctx, repo, rev)
198 if entries, err := s.cTree(ctx, repo, rev, ""); err == nil {
199 if readme, _ := findDocs(entries); readme != "" {
200 _, _, _ = s.cDoc(ctx, repo, rev, readme, 200_000)
201 }
202 }
203 _ = s.repoLanguage(ctx, repo, rev)
204}
internal/server/issues_api.go +10 −2
@@ -459,6 +459,14 @@ func (s *Server) issueAuth(w http.ResponseWriter, r *http.Request, write bool) (
459 459 return store.User{}, store.Repo{}, "", false
460 460 }
461 461 token := bearer(r)
462 if token == "" && !write {
463 // Anoniem lezen mag alleen in een publieke repository, net als in de UI.
464 if repo.Visibility != "public" {
465 http.NotFound(w, r)
466 return store.User{}, store.Repo{}, "", false
467 }
468 return store.User{}, repo, "", true
469 }
462 470 if token == "" {
463 471 http.Error(w, s.tr(r, "err.tokenAbsent"), http.StatusUnauthorized)
464 472 return store.User{}, store.Repo{}, "", false
@@ -477,8 +485,8 @@ func (s *Server) issueAuth(w http.ResponseWriter, r *http.Request, write bool) (
477 485 http.Error(w, s.tr(r, "err.rights"), http.StatusInternalServerError)
478 486 return store.User{}, store.Repo{}, "", false
479 487 }
480 if !write && !role.CanRead() && repo.Visibility != "public" {
481 http.Error(w, s.tr(r, "err.forbidden"), http.StatusForbidden)
488 if !role.CanRead() && repo.Visibility != "public" {
489 http.NotFound(w, r)
482 490 return store.User{}, store.Repo{}, "", false
483 491 }
484 492 return u, repo, role, true
@@ -459,6 +459,14 @@ func (s *Server) issueAuth(w http.ResponseWriter, r *http.Request, write bool) (
459 return store.User{}, store.Repo{}, "", false
459 return store.User{}, store.Repo{}, "", false
460 }
460 }
461 token := bearer(r)
461 token := bearer(r)
462 if token == "" && !write {
463 // Anoniem lezen mag alleen in een publieke repository, net als in de UI.
464 if repo.Visibility != "public" {
465 http.NotFound(w, r)
466 return store.User{}, store.Repo{}, "", false
467 }
468 return store.User{}, repo, "", true
469 }
462 if token == "" {
470 if token == "" {
463 http.Error(w, s.tr(r, "err.tokenAbsent"), http.StatusUnauthorized)
471 http.Error(w, s.tr(r, "err.tokenAbsent"), http.StatusUnauthorized)
464 return store.User{}, store.Repo{}, "", false
472 return store.User{}, store.Repo{}, "", false
@@ -477,8 +485,8 @@ func (s *Server) issueAuth(w http.ResponseWriter, r *http.Request, write bool) (
477 http.Error(w, s.tr(r, "err.rights"), http.StatusInternalServerError)
485 http.Error(w, s.tr(r, "err.rights"), http.StatusInternalServerError)
478 return store.User{}, store.Repo{}, "", false
486 return store.User{}, store.Repo{}, "", false
479 }
487 }
480 if !write && !role.CanRead() && repo.Visibility != "public" {
488 if !role.CanRead() && repo.Visibility != "public" {
481 http.Error(w, s.tr(r, "err.forbidden"), http.StatusForbidden)
489 http.NotFound(w, r)
482 return store.User{}, store.Repo{}, "", false
490 return store.User{}, store.Repo{}, "", false
483 }
491 }
484 return u, repo, role, true
492 return u, repo, role, true
internal/server/mr_page.go +2 −3
@@ -120,12 +120,11 @@ func (s *Server) finishMR(r *http.Request, v *View, repo store.Repo, mr store.Me
120 120 }
121 121
122 122 func (s *Server) fillBranchChoices(r *http.Request, v *View, repo store.Repo) {
123 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
124 names, _ := gitrepo.Branches(r.Context(), s.Git, dir)
123 names, _ := s.cBranches(r.Context(), repo)
125 124 for _, name := range names {
126 125 v.Branches = append(v.Branches, BranchRow{Name: name, Default: name == repo.DefaultBranch})
127 126 }
128 tags, _ := gitrepo.Tags(r.Context(), s.Git, dir)
127 tags, _ := s.cTags(r.Context(), repo)
129 128 for _, name := range tags {
130 129 v.Tags = append(v.Tags, TagRow{Name: name})
131 130 }
@@ -120,12 +120,11 @@ func (s *Server) finishMR(r *http.Request, v *View, repo store.Repo, mr store.Me
120}
120}
121
121
122func (s *Server) fillBranchChoices(r *http.Request, v *View, repo store.Repo) {
122func (s *Server) fillBranchChoices(r *http.Request, v *View, repo store.Repo) {
123 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
123 names, _ := s.cBranches(r.Context(), repo)
124 names, _ := gitrepo.Branches(r.Context(), s.Git, dir)
125 for _, name := range names {
124 for _, name := range names {
126 v.Branches = append(v.Branches, BranchRow{Name: name, Default: name == repo.DefaultBranch})
125 v.Branches = append(v.Branches, BranchRow{Name: name, Default: name == repo.DefaultBranch})
127 }
126 }
128 tags, _ := gitrepo.Tags(r.Context(), s.Git, dir)
127 tags, _ := s.cTags(r.Context(), repo)
129 for _, name := range tags {
128 for _, name := range tags {
130 v.Tags = append(v.Tags, TagRow{Name: name})
129 v.Tags = append(v.Tags, TagRow{Name: name})
131 }
130 }
internal/server/ops_collab.go +4 −0
@@ -138,6 +138,9 @@ func (s *Server) opUpdateRepo(ctx context.Context, user store.User, repo store.R
138 138 if err := s.Store.UpdateRepo(ctx, repo.ID, description, visibility); err != nil {
139 139 return fail(400, err)
140 140 }
141 if visibility != repo.Visibility {
142 s.ReadCache().DropRepo(repo.ID)
143 }
141 144 return nil
142 145 }
143 146
@@ -154,6 +157,7 @@ func (s *Server) opDeleteRepo(ctx context.Context, user store.User, repo store.R
154 157 return fail(400, err)
155 158 }
156 159 _ = os.RemoveAll(dir)
160 s.ReadCache().DropRepo(repo.ID)
157 161 return nil
158 162 }
159 163
@@ -138,6 +138,9 @@ func (s *Server) opUpdateRepo(ctx context.Context, user store.User, repo store.R
138 if err := s.Store.UpdateRepo(ctx, repo.ID, description, visibility); err != nil {
138 if err := s.Store.UpdateRepo(ctx, repo.ID, description, visibility); err != nil {
139 return fail(400, err)
139 return fail(400, err)
140 }
140 }
141 if visibility != repo.Visibility {
142 s.ReadCache().DropRepo(repo.ID)
143 }
141 return nil
144 return nil
142}
145}
143
146
@@ -154,6 +157,7 @@ func (s *Server) opDeleteRepo(ctx context.Context, user store.User, repo store.R
154 return fail(400, err)
157 return fail(400, err)
155 }
158 }
156 _ = os.RemoveAll(dir)
159 _ = os.RemoveAll(dir)
160 s.ReadCache().DropRepo(repo.ID)
157 return nil
161 return nil
158}
162}
159
163
internal/server/ops_git.go +1 −0
@@ -180,6 +180,7 @@ func (s *Server) opDeleteTag(ctx context.Context, user store.User, repo store.Re
180 180 if err := gitrepo.DeleteTag(ctx, s.Git, dir, name); err != nil {
181 181 return fail(400, err)
182 182 }
183 s.refsChanged(repo)
183 184 return nil
184 185 }
185 186
@@ -180,6 +180,7 @@ func (s *Server) opDeleteTag(ctx context.Context, user store.User, repo store.Re
180 if err := gitrepo.DeleteTag(ctx, s.Git, dir, name); err != nil {
180 if err := gitrepo.DeleteTag(ctx, s.Git, dir, name); err != nil {
181 return fail(400, err)
181 return fail(400, err)
182 }
182 }
183 s.refsChanged(repo)
183 return nil
184 return nil
184}
185}
185
186
internal/server/project.go +13 −14
@@ -19,31 +19,30 @@ import (
19 19
20 20 func (s *Server) fillOverview(r *http.Request, v *View, repo store.Repo) {
21 21 v.Page = "overview"
22 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
23 s.branchList(r, v, dir)
22 s.branchList(r, v, repo)
24 23 lang := i18n.Parse(v.Lang)
25 if c, err := gitrepo.LastCommit(r.Context(), s.Git, dir, v.Rev, ""); err == nil && c.SHA != "" {
24 if c, err := s.cLastCommit(r.Context(), repo, v.Rev, ""); err == nil && c.SHA != "" {
26 25 if rows := s.presentCommits(r.Context(), repo.ID, []gitrepo.Commit{c}, lang); len(rows) > 0 {
27 26 s.attachHeadPipeline(r.Context(), repo, &rows[0], lang)
28 27 v.Head = &rows[0]
29 28 }
30 29 }
31 entries, err := gitrepo.Tree(r.Context(), s.Git, dir, v.Rev, "")
32 touch, _ := gitrepo.Touch(r.Context(), s.Git, dir, v.Rev)
30 entries, err := s.cTree(r.Context(), repo, v.Rev, "")
31 touch, _ := s.cTouch(r.Context(), repo, v.Rev)
33 32 if err == nil {
34 33 v.Files = fileRows(entries, "", touch, lang)
35 34 }
36 35 readme, license := findDocs(entries)
37 36 v.LicensePath = license
38 37 if readme != "" {
39 text, binary, err := gitrepo.Blob(r.Context(), s.Git, dir, v.Rev, readme, 200_000)
38 text, binary, err := s.cDoc(r.Context(), repo, v.Rev, readme, 200_000)
40 39 if err == nil && !binary {
41 40 v.ReadmeName = readme
42 41 base := "/" + repo.FullPath() + "/-/blob/"
43 42 v.Readme = template.HTML(markdown.RenderTasks(text, base, v.Rev, "/"+repo.FullPath(), markdown.Tasks{Count: s.tr(r, "tpl.tasks")}))
44 43 }
45 44 }
46 commits, branches, tags, _ := gitrepo.Stats(r.Context(), s.Git, dir, v.Rev)
45 commits, branches, tags, _ := s.cStats(r.Context(), repo, v.Rev)
47 46 v.NCommits, v.NBranches, v.NTags = commits, branches, tags
48 47 if rels, err := s.Store.ListReleases(r.Context(), repo.ID); err == nil {
49 48 v.NReleases = len(rels)
@@ -55,13 +54,13 @@ func (s *Server) fillFiles(r *http.Request, v *View, repo store.Repo, rel string
55 54 v.Page = "files"
56 55 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
57 56 lang := i18n.Parse(v.Lang)
58 s.branchList(r, v, dir)
59 all, _ := gitrepo.TreeAll(r.Context(), s.Git, dir, v.Rev)
57 s.branchList(r, v, repo)
58 all, _ := s.cTreeAll(r.Context(), repo, v.Rev)
60 59 v.Tree = treeRows(all, rel)
61 60 if !isFile {
62 61 v.TreePath = rel
63 entries, err := gitrepo.Tree(r.Context(), s.Git, dir, v.Rev, rel)
64 touch, _ := gitrepo.Touch(r.Context(), s.Git, dir, v.Rev)
62 entries, err := s.cTree(r.Context(), repo, v.Rev, rel)
63 touch, _ := s.cTouch(r.Context(), repo, v.Rev)
65 64 if err == nil {
66 65 v.Files = fileRows(entries, rel, touch, lang)
67 66 }
@@ -82,7 +81,7 @@ func (s *Server) fillFiles(r *http.Request, v *View, repo store.Repo, rel string
82 81 if n, err := gitrepo.BlobSize(r.Context(), s.Git, dir, v.Rev, rel); err == nil {
83 82 v.BlobSize = formatSize(n)
84 83 }
85 if c, err := gitrepo.LastCommit(r.Context(), s.Git, dir, v.Rev, rel); err == nil && c.SHA != "" {
84 if c, err := s.cLastCommit(r.Context(), repo, v.Rev, rel); err == nil && c.SHA != "" {
86 85 if rows := s.presentCommits(r.Context(), repo.ID, []gitrepo.Commit{c}, lang); len(rows) > 0 {
87 86 s.attachHeadPipeline(r.Context(), repo, &rows[0], lang)
88 87 v.Head = &rows[0]
@@ -90,10 +89,10 @@ func (s *Server) fillFiles(r *http.Request, v *View, repo store.Repo, rel string
90 89 }
91 90 }
92 91
93 func (s *Server) branchList(r *http.Request, v *View, dir string) {
92 func (s *Server) branchList(r *http.Request, v *View, repo store.Repo) {
94 93 patterns, _ := s.Store.ProtectedPatterns(r.Context(), v.Repo.ID)
95 94 v.RevProtected = access.BranchProtected(patterns, v.Rev)
96 names, _ := gitrepo.Branches(r.Context(), s.Git, dir)
95 names, _ := s.cBranches(r.Context(), repo)
97 96 seen := false
98 97 for _, name := range names {
99 98 if name == v.Rev {
@@ -19,31 +19,30 @@ import (
19
19
20func (s *Server) fillOverview(r *http.Request, v *View, repo store.Repo) {
20func (s *Server) fillOverview(r *http.Request, v *View, repo store.Repo) {
21 v.Page = "overview"
21 v.Page = "overview"
22 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
22 s.branchList(r, v, repo)
23 s.branchList(r, v, dir)
24 lang := i18n.Parse(v.Lang)
23 lang := i18n.Parse(v.Lang)
25 if c, err := gitrepo.LastCommit(r.Context(), s.Git, dir, v.Rev, ""); err == nil && c.SHA != "" {
24 if c, err := s.cLastCommit(r.Context(), repo, v.Rev, ""); err == nil && c.SHA != "" {
26 if rows := s.presentCommits(r.Context(), repo.ID, []gitrepo.Commit{c}, lang); len(rows) > 0 {
25 if rows := s.presentCommits(r.Context(), repo.ID, []gitrepo.Commit{c}, lang); len(rows) > 0 {
27 s.attachHeadPipeline(r.Context(), repo, &rows[0], lang)
26 s.attachHeadPipeline(r.Context(), repo, &rows[0], lang)
28 v.Head = &rows[0]
27 v.Head = &rows[0]
29 }
28 }
30 }
29 }
31 entries, err := gitrepo.Tree(r.Context(), s.Git, dir, v.Rev, "")
30 entries, err := s.cTree(r.Context(), repo, v.Rev, "")
32 touch, _ := gitrepo.Touch(r.Context(), s.Git, dir, v.Rev)
31 touch, _ := s.cTouch(r.Context(), repo, v.Rev)
33 if err == nil {
32 if err == nil {
34 v.Files = fileRows(entries, "", touch, lang)
33 v.Files = fileRows(entries, "", touch, lang)
35 }
34 }
36 readme, license := findDocs(entries)
35 readme, license := findDocs(entries)
37 v.LicensePath = license
36 v.LicensePath = license
38 if readme != "" {
37 if readme != "" {
39 text, binary, err := gitrepo.Blob(r.Context(), s.Git, dir, v.Rev, readme, 200_000)
38 text, binary, err := s.cDoc(r.Context(), repo, v.Rev, readme, 200_000)
40 if err == nil && !binary {
39 if err == nil && !binary {
41 v.ReadmeName = readme
40 v.ReadmeName = readme
42 base := "/" + repo.FullPath() + "/-/blob/"
41 base := "/" + repo.FullPath() + "/-/blob/"
43 v.Readme = template.HTML(markdown.RenderTasks(text, base, v.Rev, "/"+repo.FullPath(), markdown.Tasks{Count: s.tr(r, "tpl.tasks")}))
42 v.Readme = template.HTML(markdown.RenderTasks(text, base, v.Rev, "/"+repo.FullPath(), markdown.Tasks{Count: s.tr(r, "tpl.tasks")}))
44 }
43 }
45 }
44 }
46 commits, branches, tags, _ := gitrepo.Stats(r.Context(), s.Git, dir, v.Rev)
45 commits, branches, tags, _ := s.cStats(r.Context(), repo, v.Rev)
47 v.NCommits, v.NBranches, v.NTags = commits, branches, tags
46 v.NCommits, v.NBranches, v.NTags = commits, branches, tags
48 if rels, err := s.Store.ListReleases(r.Context(), repo.ID); err == nil {
47 if rels, err := s.Store.ListReleases(r.Context(), repo.ID); err == nil {
49 v.NReleases = len(rels)
48 v.NReleases = len(rels)
@@ -55,13 +54,13 @@ func (s *Server) fillFiles(r *http.Request, v *View, repo store.Repo, rel string
55 v.Page = "files"
54 v.Page = "files"
56 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
55 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
57 lang := i18n.Parse(v.Lang)
56 lang := i18n.Parse(v.Lang)
58 s.branchList(r, v, dir)
57 s.branchList(r, v, repo)
59 all, _ := gitrepo.TreeAll(r.Context(), s.Git, dir, v.Rev)
58 all, _ := s.cTreeAll(r.Context(), repo, v.Rev)
60 v.Tree = treeRows(all, rel)
59 v.Tree = treeRows(all, rel)
61 if !isFile {
60 if !isFile {
62 v.TreePath = rel
61 v.TreePath = rel
63 entries, err := gitrepo.Tree(r.Context(), s.Git, dir, v.Rev, rel)
62 entries, err := s.cTree(r.Context(), repo, v.Rev, rel)
64 touch, _ := gitrepo.Touch(r.Context(), s.Git, dir, v.Rev)
63 touch, _ := s.cTouch(r.Context(), repo, v.Rev)
65 if err == nil {
64 if err == nil {
66 v.Files = fileRows(entries, rel, touch, lang)
65 v.Files = fileRows(entries, rel, touch, lang)
67 }
66 }
@@ -82,7 +81,7 @@ func (s *Server) fillFiles(r *http.Request, v *View, repo store.Repo, rel string
82 if n, err := gitrepo.BlobSize(r.Context(), s.Git, dir, v.Rev, rel); err == nil {
81 if n, err := gitrepo.BlobSize(r.Context(), s.Git, dir, v.Rev, rel); err == nil {
83 v.BlobSize = formatSize(n)
82 v.BlobSize = formatSize(n)
84 }
83 }
85 if c, err := gitrepo.LastCommit(r.Context(), s.Git, dir, v.Rev, rel); err == nil && c.SHA != "" {
84 if c, err := s.cLastCommit(r.Context(), repo, v.Rev, rel); err == nil && c.SHA != "" {
86 if rows := s.presentCommits(r.Context(), repo.ID, []gitrepo.Commit{c}, lang); len(rows) > 0 {
85 if rows := s.presentCommits(r.Context(), repo.ID, []gitrepo.Commit{c}, lang); len(rows) > 0 {
87 s.attachHeadPipeline(r.Context(), repo, &rows[0], lang)
86 s.attachHeadPipeline(r.Context(), repo, &rows[0], lang)
88 v.Head = &rows[0]
87 v.Head = &rows[0]
@@ -90,10 +89,10 @@ func (s *Server) fillFiles(r *http.Request, v *View, repo store.Repo, rel string
90 }
89 }
91}
90}
92
91
93func (s *Server) branchList(r *http.Request, v *View, dir string) {
92func (s *Server) branchList(r *http.Request, v *View, repo store.Repo) {
94 patterns, _ := s.Store.ProtectedPatterns(r.Context(), v.Repo.ID)
93 patterns, _ := s.Store.ProtectedPatterns(r.Context(), v.Repo.ID)
95 v.RevProtected = access.BranchProtected(patterns, v.Rev)
94 v.RevProtected = access.BranchProtected(patterns, v.Rev)
96 names, _ := gitrepo.Branches(r.Context(), s.Git, dir)
95 names, _ := s.cBranches(r.Context(), repo)
97 seen := false
96 seen := false
98 for _, name := range names {
97 for _, name := range names {
99 if name == v.Rev {
98 if name == v.Rev {
internal/server/readcache_test.go added +218 −0
@@ -0,0 +1,218 @@
1 package server
2
3 import (
4 "context"
5 "net/http"
6 "os"
7 "path/filepath"
8 "strings"
9 "testing"
10 "time"
11
12 "gitlab.com/Timfa/bluegit/internal/readcache"
13 "gitlab.com/Timfa/bluegit/internal/store"
14 )
15
16 func cacheServer(t *testing.T) (*Server, string, func(path string) (int, string), *http.Client, string, string) {
17 t.Helper()
18 srv, ts, st := mergeFixServer(t)
19 srv.StateDir = t.TempDir()
20 root := t.TempDir()
21 if err := srv.opSetCache(root, 64); err != nil {
22 t.Fatal(err)
23 }
24 tim := loginClient(t, ts)
25 token := mintToken(t, tim, ts)
26 work := pushPair(t, ts, token, "cached", "hallo\n", "verder\n")
27 git(t, work, "checkout", "master")
28 writeFile(t, filepath.Join(work, "README.md"), "# Eerste versie\n")
29 git(t, work, "add", "README.md")
30 git(t, work, "commit", "-m", "readme")
31 git(t, work, "push", "origin", "master")
32 _ = st
33 get := func(path string) (int, string) {
34 res, err := tim.Get(ts.URL + path)
35 if err != nil {
36 t.Fatal(err)
37 }
38 return res.StatusCode, read(t, res)
39 }
40 return srv, root, get, tim, ts.URL, work
41 }
42
43 func repoOf(t *testing.T, srv *Server, name string) store.Repo {
44 t.Helper()
45 repos, err := srv.Store.AllRepos(context.Background())
46 if err != nil {
47 t.Fatal(err)
48 }
49 for _, r := range repos {
50 if r.Name == name {
51 return r
52 }
53 }
54 t.Fatalf("repo %s ontbreekt", name)
55 return store.Repo{}
56 }
57
58 // waitWarm wacht tot de achtergrond-vulling na een push klaar is.
59 func waitWarm(srv *Server) {
60 time.Sleep(300 * time.Millisecond)
61 }
62
63 func TestReadCacheServesOverviewWithoutRepoDisk(t *testing.T) {
64 srv, root, get, _, _, work := cacheServer(t)
65 waitWarm(srv)
66 pages := []string{"/tim/cached", "/tim/cached/-/branches", "/tim/cached/-/tags"}
67 for _, p := range pages {
68 if code, _ := get(p); code != 200 {
69 t.Fatalf("%s: %d", p, code)
70 }
71 }
72 if srv.ReadCache().Usage().Entries == 0 {
73 t.Fatal("niets gecachet")
74 }
75 // HDD 'weg': de repository-map tijdelijk verplaatsen.
76 repo := repoOf(t, srv, "cached")
77 dir := srv.gdir(repo)
78 if err := os.Rename(dir, dir+".weg"); err != nil {
79 t.Fatal(err)
80 }
81 _, body := get("/tim/cached")
82 if !strings.Contains(body, "Eerste versie") {
83 t.Fatal("README niet uit de cache")
84 }
85 if _, body := get("/tim/cached/-/branches"); !strings.Contains(body, "topic") {
86 t.Fatal("branches niet uit de cache")
87 }
88 if err := os.Rename(dir+".weg", dir); err != nil {
89 t.Fatal(err)
90 }
91
92 // write-through: na een push nooit de oude README
93 writeFile(t, filepath.Join(work, "README.md"), "# Tweede versie\n")
94 git(t, work, "add", "README.md")
95 git(t, work, "commit", "-m", "readme 2")
96 git(t, work, "push", "origin", "master")
97 git(t, work, "push", "origin", "master:nieuwe-branch")
98 _, body = get("/tim/cached")
99 if !strings.Contains(body, "Tweede versie") || strings.Contains(body, "Eerste versie") {
100 t.Fatal("verouderde README na push")
101 }
102 if _, body := get("/tim/cached/-/branches"); !strings.Contains(body, "nieuwe-branch") {
103 t.Fatal("nieuwe branch ontbreekt")
104 }
105
106 // corrupte entries zijn een miss, geen fout
107 _ = filepath.Walk(filepath.Join(root, "cache"), func(p string, info os.FileInfo, err error) error {
108 if err == nil && !info.IsDir() {
109 _ = os.WriteFile(p, []byte("kapot"), 0o600)
110 }
111 return nil
112 })
113 if code, body := get("/tim/cached"); code != 200 || !strings.Contains(body, "Tweede versie") {
114 t.Fatalf("corrupte cache brak de pagina: %d", code)
115 }
116 }
117
118 func TestReadCacheRespectsVisibility(t *testing.T) {
119 srv, _, get, _, base, _ := cacheServer(t)
120 waitWarm(srv)
121 get("/tim/cached")
122 repo := repoOf(t, srv, "cached")
123 anon := &http.Client{}
124 res, _ := anon.Get(base + "/tim/cached")
125 res.Body.Close()
126 if res.StatusCode != 200 {
127 t.Fatalf("publiek anoniem: %d", res.StatusCode)
128 }
129 before := srv.ReadCache().Usage().Entries
130 tim, _ := srv.Store.UserByUsername(context.Background(), "tim")
131 if err := srv.opUpdateRepo(context.Background(), tim, repo, repo.Description, "private"); err != nil {
132 t.Fatal(err)
133 }
134 if after := srv.ReadCache().Usage().Entries; after >= before {
135 t.Fatalf("cache niet gewist bij zichtbaarheid: %d -> %d", before, after)
136 }
137 get("/tim/cached") // eigenaar warmt opnieuw
138 for _, p := range []string{"/tim/cached", "/tim/cached/-/branches", "/tim/cached/-/tags"} {
139 res, _ := anon.Get(base + p)
140 body := read(t, res)
141 if res.StatusCode == 200 || strings.Contains(body, "Eerste versie") {
142 t.Fatalf("%s lekt naar anoniem: %d", p, res.StatusCode)
143 }
144 }
145 }
146
147 func TestReadCacheClearKeepsData(t *testing.T) {
148 srv, root, get, _, _, _ := cacheServer(t)
149 get("/tim/cached")
150 keep := filepath.Join(root, "data", "bewaar.txt")
151 _ = os.MkdirAll(filepath.Dir(keep), 0o750)
152 _ = os.WriteFile(keep, []byte("x"), 0o600)
153 srv.ReadCache().Clear()
154 if srv.ReadCache().Usage().Entries != 0 {
155 t.Fatal("niet leeg")
156 }
157 if _, err := os.Stat(keep); err != nil {
158 t.Fatal("data/ geraakt")
159 }
160 repo := repoOf(t, srv, "cached")
161 srv.ReadCache().PutJSON(repo.ID, false, "x", 1)
162 tim, _ := srv.Store.UserByUsername(context.Background(), "tim")
163 if err := srv.opDeleteRepo(context.Background(), tim, repo, "cached"); err != nil {
164 t.Fatal(err)
165 }
166 if srv.ReadCache().Usage().Entries != 0 {
167 t.Fatal("cache van verwijderde repo bleef staan")
168 }
169 }
170
171 func TestSetCacheValidates(t *testing.T) {
172 srv, _, _ := mergeFixServer(t)
173 srv.StateDir = t.TempDir()
174 if srv.opSetCache("relatief", 0) == nil {
175 t.Fatal("relatief pad geaccepteerd")
176 }
177 if srv.opSetCache(filepath.Join(srv.StoragePath(), "cache"), 0) == nil {
178 t.Fatal("pad in opslag geaccepteerd")
179 }
180 root := t.TempDir()
181 if err := srv.opSetCache(root, 0); err != nil || srv.ReadCache() == nil {
182 t.Fatal(err)
183 }
184 // blijft bewaard over een herstart
185 srv2 := &Server{StateDir: srv.StateDir}
186 srv2.InitReadCache()
187 if srv2.ReadCache() == nil || srv2.CacheConfig().Path != root {
188 t.Fatal("instelling niet bewaard")
189 }
190 if err := srv.opSetCache("", 0); err != nil || srv.ReadCache() != nil {
191 t.Fatal("uitzetten mislukt")
192 }
193 }
194
195 // TestReadCacheColdWarm meet een overzichtspagina koud en warm.
196 func TestReadCacheColdWarm(t *testing.T) {
197 srv, _, get, _, _, _ := cacheServer(t)
198 waitWarm(srv)
199 const n = 20
200 measure := func() time.Duration {
201 start := time.Now()
202 for i := 0; i < n; i++ {
203 get("/tim/cached")
204 }
205 return time.Since(start) / n
206 }
207 keep := srv.ReadCache()
208 srv.SetReadCache(nil)
209 cold := measure()
210 srv.SetReadCache(keep)
211 get("/tim/cached")
212 warm := measure()
213 t.Logf("overzicht: zonder cache %v, warm %v per pagina", cold, warm)
214 if warm > cold*2 {
215 t.Fatalf("warm (%v) veel trager dan koud (%v)", warm, cold)
216 }
217 _ = readcache.DefaultMax
218 }
@@ -0,0 +1,218 @@
1package server
2
3import (
4 "context"
5 "net/http"
6 "os"
7 "path/filepath"
8 "strings"
9 "testing"
10 "time"
11
12 "gitlab.com/Timfa/bluegit/internal/readcache"
13 "gitlab.com/Timfa/bluegit/internal/store"
14)
15
16func cacheServer(t *testing.T) (*Server, string, func(path string) (int, string), *http.Client, string, string) {
17 t.Helper()
18 srv, ts, st := mergeFixServer(t)
19 srv.StateDir = t.TempDir()
20 root := t.TempDir()
21 if err := srv.opSetCache(root, 64); err != nil {
22 t.Fatal(err)
23 }
24 tim := loginClient(t, ts)
25 token := mintToken(t, tim, ts)
26 work := pushPair(t, ts, token, "cached", "hallo\n", "verder\n")
27 git(t, work, "checkout", "master")
28 writeFile(t, filepath.Join(work, "README.md"), "# Eerste versie\n")
29 git(t, work, "add", "README.md")
30 git(t, work, "commit", "-m", "readme")
31 git(t, work, "push", "origin", "master")
32 _ = st
33 get := func(path string) (int, string) {
34 res, err := tim.Get(ts.URL + path)
35 if err != nil {
36 t.Fatal(err)
37 }
38 return res.StatusCode, read(t, res)
39 }
40 return srv, root, get, tim, ts.URL, work
41}
42
43func repoOf(t *testing.T, srv *Server, name string) store.Repo {
44 t.Helper()
45 repos, err := srv.Store.AllRepos(context.Background())
46 if err != nil {
47 t.Fatal(err)
48 }
49 for _, r := range repos {
50 if r.Name == name {
51 return r
52 }
53 }
54 t.Fatalf("repo %s ontbreekt", name)
55 return store.Repo{}
56}
57
58// waitWarm wacht tot de achtergrond-vulling na een push klaar is.
59func waitWarm(srv *Server) {
60 time.Sleep(300 * time.Millisecond)
61}
62
63func TestReadCacheServesOverviewWithoutRepoDisk(t *testing.T) {
64 srv, root, get, _, _, work := cacheServer(t)
65 waitWarm(srv)
66 pages := []string{"/tim/cached", "/tim/cached/-/branches", "/tim/cached/-/tags"}
67 for _, p := range pages {
68 if code, _ := get(p); code != 200 {
69 t.Fatalf("%s: %d", p, code)
70 }
71 }
72 if srv.ReadCache().Usage().Entries == 0 {
73 t.Fatal("niets gecachet")
74 }
75 // HDD 'weg': de repository-map tijdelijk verplaatsen.
76 repo := repoOf(t, srv, "cached")
77 dir := srv.gdir(repo)
78 if err := os.Rename(dir, dir+".weg"); err != nil {
79 t.Fatal(err)
80 }
81 _, body := get("/tim/cached")
82 if !strings.Contains(body, "Eerste versie") {
83 t.Fatal("README niet uit de cache")
84 }
85 if _, body := get("/tim/cached/-/branches"); !strings.Contains(body, "topic") {
86 t.Fatal("branches niet uit de cache")
87 }
88 if err := os.Rename(dir+".weg", dir); err != nil {
89 t.Fatal(err)
90 }
91
92 // write-through: na een push nooit de oude README
93 writeFile(t, filepath.Join(work, "README.md"), "# Tweede versie\n")
94 git(t, work, "add", "README.md")
95 git(t, work, "commit", "-m", "readme 2")
96 git(t, work, "push", "origin", "master")
97 git(t, work, "push", "origin", "master:nieuwe-branch")
98 _, body = get("/tim/cached")
99 if !strings.Contains(body, "Tweede versie") || strings.Contains(body, "Eerste versie") {
100 t.Fatal("verouderde README na push")
101 }
102 if _, body := get("/tim/cached/-/branches"); !strings.Contains(body, "nieuwe-branch") {
103 t.Fatal("nieuwe branch ontbreekt")
104 }
105
106 // corrupte entries zijn een miss, geen fout
107 _ = filepath.Walk(filepath.Join(root, "cache"), func(p string, info os.FileInfo, err error) error {
108 if err == nil && !info.IsDir() {
109 _ = os.WriteFile(p, []byte("kapot"), 0o600)
110 }
111 return nil
112 })
113 if code, body := get("/tim/cached"); code != 200 || !strings.Contains(body, "Tweede versie") {
114 t.Fatalf("corrupte cache brak de pagina: %d", code)
115 }
116}
117
118func TestReadCacheRespectsVisibility(t *testing.T) {
119 srv, _, get, _, base, _ := cacheServer(t)
120 waitWarm(srv)
121 get("/tim/cached")
122 repo := repoOf(t, srv, "cached")
123 anon := &http.Client{}
124 res, _ := anon.Get(base + "/tim/cached")
125 res.Body.Close()
126 if res.StatusCode != 200 {
127 t.Fatalf("publiek anoniem: %d", res.StatusCode)
128 }
129 before := srv.ReadCache().Usage().Entries
130 tim, _ := srv.Store.UserByUsername(context.Background(), "tim")
131 if err := srv.opUpdateRepo(context.Background(), tim, repo, repo.Description, "private"); err != nil {
132 t.Fatal(err)
133 }
134 if after := srv.ReadCache().Usage().Entries; after >= before {
135 t.Fatalf("cache niet gewist bij zichtbaarheid: %d -> %d", before, after)
136 }
137 get("/tim/cached") // eigenaar warmt opnieuw
138 for _, p := range []string{"/tim/cached", "/tim/cached/-/branches", "/tim/cached/-/tags"} {
139 res, _ := anon.Get(base + p)
140 body := read(t, res)
141 if res.StatusCode == 200 || strings.Contains(body, "Eerste versie") {
142 t.Fatalf("%s lekt naar anoniem: %d", p, res.StatusCode)
143 }
144 }
145}
146
147func TestReadCacheClearKeepsData(t *testing.T) {
148 srv, root, get, _, _, _ := cacheServer(t)
149 get("/tim/cached")
150 keep := filepath.Join(root, "data", "bewaar.txt")
151 _ = os.MkdirAll(filepath.Dir(keep), 0o750)
152 _ = os.WriteFile(keep, []byte("x"), 0o600)
153 srv.ReadCache().Clear()
154 if srv.ReadCache().Usage().Entries != 0 {
155 t.Fatal("niet leeg")
156 }
157 if _, err := os.Stat(keep); err != nil {
158 t.Fatal("data/ geraakt")
159 }
160 repo := repoOf(t, srv, "cached")
161 srv.ReadCache().PutJSON(repo.ID, false, "x", 1)
162 tim, _ := srv.Store.UserByUsername(context.Background(), "tim")
163 if err := srv.opDeleteRepo(context.Background(), tim, repo, "cached"); err != nil {
164 t.Fatal(err)
165 }
166 if srv.ReadCache().Usage().Entries != 0 {
167 t.Fatal("cache van verwijderde repo bleef staan")
168 }
169}
170
171func TestSetCacheValidates(t *testing.T) {
172 srv, _, _ := mergeFixServer(t)
173 srv.StateDir = t.TempDir()
174 if srv.opSetCache("relatief", 0) == nil {
175 t.Fatal("relatief pad geaccepteerd")
176 }
177 if srv.opSetCache(filepath.Join(srv.StoragePath(), "cache"), 0) == nil {
178 t.Fatal("pad in opslag geaccepteerd")
179 }
180 root := t.TempDir()
181 if err := srv.opSetCache(root, 0); err != nil || srv.ReadCache() == nil {
182 t.Fatal(err)
183 }
184 // blijft bewaard over een herstart
185 srv2 := &Server{StateDir: srv.StateDir}
186 srv2.InitReadCache()
187 if srv2.ReadCache() == nil || srv2.CacheConfig().Path != root {
188 t.Fatal("instelling niet bewaard")
189 }
190 if err := srv.opSetCache("", 0); err != nil || srv.ReadCache() != nil {
191 t.Fatal("uitzetten mislukt")
192 }
193}
194
195// TestReadCacheColdWarm meet een overzichtspagina koud en warm.
196func TestReadCacheColdWarm(t *testing.T) {
197 srv, _, get, _, _, _ := cacheServer(t)
198 waitWarm(srv)
199 const n = 20
200 measure := func() time.Duration {
201 start := time.Now()
202 for i := 0; i < n; i++ {
203 get("/tim/cached")
204 }
205 return time.Since(start) / n
206 }
207 keep := srv.ReadCache()
208 srv.SetReadCache(nil)
209 cold := measure()
210 srv.SetReadCache(keep)
211 get("/tim/cached")
212 warm := measure()
213 t.Logf("overzicht: zonder cache %v, warm %v per pagina", cold, warm)
214 if warm > cold*2 {
215 t.Fatalf("warm (%v) veel trager dan koud (%v)", warm, cold)
216 }
217 _ = readcache.DefaultMax
218}
internal/server/repo_handlers.go +1 −1
@@ -132,7 +132,7 @@ func (s *Server) commits(w http.ResponseWriter, r *http.Request) {
132 132 v.Wide = true
133 133 v.Query = clipQuery(r.URL.Query().Get("q"))
134 134 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
135 s.branchList(r, &v, dir)
135 s.branchList(r, &v, repo)
136 136 page := commitPageNum(r.URL.Query().Get("page"))
137 137 list, more := s.commitWindow(r.Context(), dir, v.Rev, v.Query, (page-1)*commitPageSize, commitPageSize)
138 138 lang := i18n.Parse(v.Lang)
@@ -132,7 +132,7 @@ func (s *Server) commits(w http.ResponseWriter, r *http.Request) {
132 v.Wide = true
132 v.Wide = true
133 v.Query = clipQuery(r.URL.Query().Get("q"))
133 v.Query = clipQuery(r.URL.Query().Get("q"))
134 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
134 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
135 s.branchList(r, &v, dir)
135 s.branchList(r, &v, repo)
136 page := commitPageNum(r.URL.Query().Get("page"))
136 page := commitPageNum(r.URL.Query().Get("page"))
137 list, more := s.commitWindow(r.Context(), dir, v.Rev, v.Query, (page-1)*commitPageSize, commitPageSize)
137 list, more := s.commitWindow(r.Context(), dir, v.Rev, v.Query, (page-1)*commitPageSize, commitPageSize)
138 lang := i18n.Parse(v.Lang)
138 lang := i18n.Parse(v.Lang)
internal/server/server.go +7 −0
@@ -7,6 +7,7 @@ import (
7 7 "embed"
8 8 "encoding/hex"
9 9 "fmt"
10 "gitlab.com/Timfa/bluegit/internal/readcache"
10 11 "html/template"
11 12 "io/fs"
12 13 "log/slog"
@@ -65,6 +66,9 @@ type Server struct {
65 66 // mergingMR zijn merge requests die opMergeMR nu zelf afrondt.
66 67 // AfterPush mag hun bronbranch niet alvast verwijderen.
67 68 mergingMR sync.Map
69
70 cacheMu sync.RWMutex
71 rcache *readcache.Cache
68 72 }
69 73
70 74 type principal struct {
@@ -101,6 +105,8 @@ func (s *Server) Handler() http.Handler {
101 105 mux.HandleFunc("POST /new/group", s.createGroup)
102 106 mux.HandleFunc("GET /setup", s.setupPage)
103 107 mux.HandleFunc("POST /setup", s.setupSave)
108 mux.HandleFunc("POST /setup/cache", s.setupCacheSave)
109 mux.HandleFunc("POST /setup/cache/clear", s.setupCacheClear)
104 110 mux.HandleFunc("GET /settings", s.account)
105 111 mux.HandleFunc("POST /settings/language", s.saveLanguage)
106 112 mux.HandleFunc("POST /settings/profile", s.saveProfile)
@@ -608,6 +614,7 @@ type View struct {
608 614 StoragePercent int
609 615 StorageBlueGit string
610 616 StoragePartial bool
617 Cache map[string]any
611 618 StorageReserved string
612 619 HasReserved bool
613 620 MinFree string
@@ -7,6 +7,7 @@ import (
7 "embed"
7 "embed"
8 "encoding/hex"
8 "encoding/hex"
9 "fmt"
9 "fmt"
10 "gitlab.com/Timfa/bluegit/internal/readcache"
10 "html/template"
11 "html/template"
11 "io/fs"
12 "io/fs"
12 "log/slog"
13 "log/slog"
@@ -65,6 +66,9 @@ type Server struct {
65 // mergingMR zijn merge requests die opMergeMR nu zelf afrondt.
66 // mergingMR zijn merge requests die opMergeMR nu zelf afrondt.
66 // AfterPush mag hun bronbranch niet alvast verwijderen.
67 // AfterPush mag hun bronbranch niet alvast verwijderen.
67 mergingMR sync.Map
68 mergingMR sync.Map
69
70 cacheMu sync.RWMutex
71 rcache *readcache.Cache
68}
72}
69
73
70type principal struct {
74type principal struct {
@@ -101,6 +105,8 @@ func (s *Server) Handler() http.Handler {
101 mux.HandleFunc("POST /new/group", s.createGroup)
105 mux.HandleFunc("POST /new/group", s.createGroup)
102 mux.HandleFunc("GET /setup", s.setupPage)
106 mux.HandleFunc("GET /setup", s.setupPage)
103 mux.HandleFunc("POST /setup", s.setupSave)
107 mux.HandleFunc("POST /setup", s.setupSave)
108 mux.HandleFunc("POST /setup/cache", s.setupCacheSave)
109 mux.HandleFunc("POST /setup/cache/clear", s.setupCacheClear)
104 mux.HandleFunc("GET /settings", s.account)
110 mux.HandleFunc("GET /settings", s.account)
105 mux.HandleFunc("POST /settings/language", s.saveLanguage)
111 mux.HandleFunc("POST /settings/language", s.saveLanguage)
106 mux.HandleFunc("POST /settings/profile", s.saveProfile)
112 mux.HandleFunc("POST /settings/profile", s.saveProfile)
@@ -608,6 +614,7 @@ type View struct {
608 StoragePercent int
614 StoragePercent int
609 StorageBlueGit string
615 StorageBlueGit string
610 StoragePartial bool
616 StoragePartial bool
617 Cache map[string]any
611 StorageReserved string
618 StorageReserved string
612 HasReserved bool
619 HasReserved bool
613 MinFree string
620 MinFree string
internal/server/setup.go +1 −0
@@ -227,6 +227,7 @@ func (s *Server) renderSetup(w http.ResponseWriter, r *http.Request, problem, pr
227 227 v.Err = s.trErr(r, err)
228 228 }
229 229 }
230 v.Cache = s.cacheInfo()
230 231 s.attachUpdate(&v, s.langOf(r))
231 232 s.render(w, "setup", v)
232 233 }
@@ -227,6 +227,7 @@ func (s *Server) renderSetup(w http.ResponseWriter, r *http.Request, problem, pr
227 v.Err = s.trErr(r, err)
227 v.Err = s.trErr(r, err)
228 }
228 }
229 }
229 }
230 v.Cache = s.cacheInfo()
230 s.attachUpdate(&v, s.langOf(r))
231 s.attachUpdate(&v, s.langOf(r))
231 s.render(w, "setup", v)
232 s.render(w, "setup", v)
232}
233}
internal/server/social.go +6 −7
@@ -150,27 +150,26 @@ func (s *Server) publicRepoCard(ctx context.Context, repo store.Repo, rest []str
150 150
151 151 func (s *Server) repoCard(ctx context.Context, repo store.Repo, page, img, oem string) card.Render {
152 152 prose := card.Excerpt(repo.Description, 180)
153 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
154 153 rev := repo.DefaultBranch
155 154 if prose == "" && s.StoragePath() != "" {
156 if entries, err := gitrepo.Tree(ctx, s.Git, dir, rev, ""); err == nil {
155 if entries, err := s.cTree(ctx, repo, rev, ""); err == nil {
157 156 readme, _ := findDocs(entries)
158 157 if readme != "" {
159 if text, binary, err := gitrepo.Blob(ctx, s.Git, dir, rev, readme, 80_000); err == nil && !binary {
158 if text, binary, err := s.cDoc(ctx, repo, rev, readme, 80_000); err == nil && !binary {
160 159 prose = card.Excerpt(text, 180)
161 160 }
162 161 }
163 162 }
164 163 }
165 164 var facts []string
166 if lang := s.repoLanguage(ctx, dir, rev); lang != "" {
165 if lang := s.repoLanguage(ctx, repo, rev); lang != "" {
167 166 facts = append(facts, lang)
168 167 }
169 168 if counts, _, err := s.Store.StarInfo(ctx, 0, []int64{repo.ID}); err == nil {
170 169 facts = append(facts, starLabel(counts[repo.ID]))
171 170 }
172 171 if s.StoragePath() != "" {
173 if c, err := gitrepo.LastCommit(ctx, s.Git, dir, rev, ""); err == nil && c.SHA != "" {
172 if c, err := s.cLastCommit(ctx, repo, rev, ""); err == nil && c.SHA != "" {
174 173 short := c.SHA
175 174 if len(short) > 8 {
176 175 short = short[:8]
@@ -441,11 +440,11 @@ func (s *Server) readLogo(kind string, id int64, mime string) []byte {
441 440 return buf.Bytes()
442 441 }
443 442
444 func (s *Server) repoLanguage(ctx context.Context, dir, rev string) string {
443 func (s *Server) repoLanguage(ctx context.Context, repo store.Repo, rev string) string {
445 444 if s.StoragePath() == "" {
446 445 return ""
447 446 }
448 entries, err := gitrepo.TreeAll(ctx, s.Git, dir, rev)
447 entries, err := s.cTreeAll(ctx, repo, rev)
449 448 if err != nil {
450 449 return ""
451 450 }
@@ -150,27 +150,26 @@ func (s *Server) publicRepoCard(ctx context.Context, repo store.Repo, rest []str
150
150
151func (s *Server) repoCard(ctx context.Context, repo store.Repo, page, img, oem string) card.Render {
151func (s *Server) repoCard(ctx context.Context, repo store.Repo, page, img, oem string) card.Render {
152 prose := card.Excerpt(repo.Description, 180)
152 prose := card.Excerpt(repo.Description, 180)
153 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
154 rev := repo.DefaultBranch
153 rev := repo.DefaultBranch
155 if prose == "" && s.StoragePath() != "" {
154 if prose == "" && s.StoragePath() != "" {
156 if entries, err := gitrepo.Tree(ctx, s.Git, dir, rev, ""); err == nil {
155 if entries, err := s.cTree(ctx, repo, rev, ""); err == nil {
157 readme, _ := findDocs(entries)
156 readme, _ := findDocs(entries)
158 if readme != "" {
157 if readme != "" {
159 if text, binary, err := gitrepo.Blob(ctx, s.Git, dir, rev, readme, 80_000); err == nil && !binary {
158 if text, binary, err := s.cDoc(ctx, repo, rev, readme, 80_000); err == nil && !binary {
160 prose = card.Excerpt(text, 180)
159 prose = card.Excerpt(text, 180)
161 }
160 }
162 }
161 }
163 }
162 }
164 }
163 }
165 var facts []string
164 var facts []string
166 if lang := s.repoLanguage(ctx, dir, rev); lang != "" {
165 if lang := s.repoLanguage(ctx, repo, rev); lang != "" {
167 facts = append(facts, lang)
166 facts = append(facts, lang)
168 }
167 }
169 if counts, _, err := s.Store.StarInfo(ctx, 0, []int64{repo.ID}); err == nil {
168 if counts, _, err := s.Store.StarInfo(ctx, 0, []int64{repo.ID}); err == nil {
170 facts = append(facts, starLabel(counts[repo.ID]))
169 facts = append(facts, starLabel(counts[repo.ID]))
171 }
170 }
172 if s.StoragePath() != "" {
171 if s.StoragePath() != "" {
173 if c, err := gitrepo.LastCommit(ctx, s.Git, dir, rev, ""); err == nil && c.SHA != "" {
172 if c, err := s.cLastCommit(ctx, repo, rev, ""); err == nil && c.SHA != "" {
174 short := c.SHA
173 short := c.SHA
175 if len(short) > 8 {
174 if len(short) > 8 {
176 short = short[:8]
175 short = short[:8]
@@ -441,11 +440,11 @@ func (s *Server) readLogo(kind string, id int64, mime string) []byte {
441 return buf.Bytes()
440 return buf.Bytes()
442}
441}
443
442
444func (s *Server) repoLanguage(ctx context.Context, dir, rev string) string {
443func (s *Server) repoLanguage(ctx context.Context, repo store.Repo, rev string) string {
445 if s.StoragePath() == "" {
444 if s.StoragePath() == "" {
446 return ""
445 return ""
447 }
446 }
448 entries, err := gitrepo.TreeAll(ctx, s.Git, dir, rev)
447 entries, err := s.cTreeAll(ctx, repo, rev)
449 if err != nil {
448 if err != nil {
450 return ""
449 return ""
451 }
450 }
internal/server/tags.go +2 −3
@@ -21,8 +21,7 @@ func (s *Server) tags(w http.ResponseWriter, r *http.Request) {
21 21 v.Page = "tags"
22 22 v.Wide = true
23 23 v.Query = strings.TrimSpace(r.URL.Query().Get("q"))
24 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
25 s.branchList(r, &v, dir)
24 s.branchList(r, &v, repo)
26 25 lang := i18n.Parse(v.Lang)
27 26 patterns, _ := s.Store.ProtectedTagPatterns(r.Context(), repo.ID)
28 27 released := map[string]bool{}
@@ -31,7 +30,7 @@ func (s *Server) tags(w http.ResponseWriter, r *http.Request) {
31 30 released[rel.Tag] = true
32 31 }
33 32 }
34 infos, _ := gitrepo.ListTags(r.Context(), s.Git, dir)
33 infos, _ := s.cListTags(r.Context(), repo)
35 34 needle := strings.ToLower(v.Query)
36 35 for _, info := range infos {
37 36 if needle != "" && !strings.Contains(strings.ToLower(info.Name), needle) {
@@ -21,8 +21,7 @@ func (s *Server) tags(w http.ResponseWriter, r *http.Request) {
21 v.Page = "tags"
21 v.Page = "tags"
22 v.Wide = true
22 v.Wide = true
23 v.Query = strings.TrimSpace(r.URL.Query().Get("q"))
23 v.Query = strings.TrimSpace(r.URL.Query().Get("q"))
24 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
24 s.branchList(r, &v, repo)
25 s.branchList(r, &v, dir)
26 lang := i18n.Parse(v.Lang)
25 lang := i18n.Parse(v.Lang)
27 patterns, _ := s.Store.ProtectedTagPatterns(r.Context(), repo.ID)
26 patterns, _ := s.Store.ProtectedTagPatterns(r.Context(), repo.ID)
28 released := map[string]bool{}
27 released := map[string]bool{}
@@ -31,7 +30,7 @@ func (s *Server) tags(w http.ResponseWriter, r *http.Request) {
31 released[rel.Tag] = true
30 released[rel.Tag] = true
32 }
31 }
33 }
32 }
34 infos, _ := gitrepo.ListTags(r.Context(), s.Git, dir)
33 infos, _ := s.cListTags(r.Context(), repo)
35 needle := strings.ToLower(v.Query)
34 needle := strings.ToLower(v.Query)
36 for _, info := range infos {
35 for _, info := range infos {
37 if needle != "" && !strings.Contains(strings.ToLower(info.Name), needle) {
36 if needle != "" && !strings.Contains(strings.ToLower(info.Name), needle) {
internal/server/templates/pages.html +16 −0
@@ -27,6 +27,22 @@
27 27 </form>
28 28 <p class="muted">{{t "setup.rules" .MinFree}}</p>
29 29 </section>
30 {{with .Cache}}
31 <section class="panel" id="cache">
32 <h2>{{t "setup.cacheTitle"}}</h2>
33 <p class="muted">{{t "setup.cacheLead"}}</p>
34 {{if .enabled}}<p>{{t "setup.cacheUse" .bytes_h .max_h .entries .hits .misses}}</p>{{else}}<p class="muted">{{t "setup.cacheOff"}}</p>{{end}}
35 <form method="post" action="/setup/cache">
36 <input type="hidden" name="csrf" value="{{$.CSRF}}">
37 <label>{{t "setup.cachePath"}} <input name="cache_path" value="{{.path}}" placeholder="/var/cache/bluegit"></label>
38 <label>{{t "setup.cacheMax"}} <input name="max_mb" type="number" min="0" value="{{.max_mb}}" placeholder="512"></label>
39 <button type="submit">{{t "action.save"}}</button>
40 </form>
41 {{if .enabled}}<form method="post" action="/setup/cache/clear"><input type="hidden" name="csrf" value="{{$.CSRF}}"><button type="submit" class="btn-ghost">{{t "setup.cacheClear"}}</button></form>{{end}}
42 <h3>{{t "setup.locTitle"}}</h3>
43 <table class="kv">{{range .locations}}<tr><th>{{index . "what"}}</th><td><code>{{index . "path"}}</code></td><td>{{with index . "disk"}}{{.}}{{else}}?{{end}}</td></tr>{{end}}</table>
44 </section>
45 {{end}}
30 46 {{if .ShowUpdate}}{{template "updatestatus" .}}{{end}}
31 47 {{template "layout-end" .}}
32 48 {{end}}
@@ -27,6 +27,22 @@
27 </form>
27 </form>
28 <p class="muted">{{t "setup.rules" .MinFree}}</p>
28 <p class="muted">{{t "setup.rules" .MinFree}}</p>
29</section>
29</section>
30{{with .Cache}}
31<section class="panel" id="cache">
32 <h2>{{t "setup.cacheTitle"}}</h2>
33 <p class="muted">{{t "setup.cacheLead"}}</p>
34 {{if .enabled}}<p>{{t "setup.cacheUse" .bytes_h .max_h .entries .hits .misses}}</p>{{else}}<p class="muted">{{t "setup.cacheOff"}}</p>{{end}}
35 <form method="post" action="/setup/cache">
36 <input type="hidden" name="csrf" value="{{$.CSRF}}">
37 <label>{{t "setup.cachePath"}} <input name="cache_path" value="{{.path}}" placeholder="/var/cache/bluegit"></label>
38 <label>{{t "setup.cacheMax"}} <input name="max_mb" type="number" min="0" value="{{.max_mb}}" placeholder="512"></label>
39 <button type="submit">{{t "action.save"}}</button>
40 </form>
41 {{if .enabled}}<form method="post" action="/setup/cache/clear"><input type="hidden" name="csrf" value="{{$.CSRF}}"><button type="submit" class="btn-ghost">{{t "setup.cacheClear"}}</button></form>{{end}}
42 <h3>{{t "setup.locTitle"}}</h3>
43 <table class="kv">{{range .locations}}<tr><th>{{index . "what"}}</th><td><code>{{index . "path"}}</code></td><td>{{with index . "disk"}}{{.}}{{else}}?{{end}}</td></tr>{{end}}</table>
44</section>
45{{end}}
30{{if .ShowUpdate}}{{template "updatestatus" .}}{{end}}
46{{if .ShowUpdate}}{{template "updatestatus" .}}{{end}}
31{{template "layout-end" .}}
47{{template "layout-end" .}}
32{{end}}
48{{end}}
internal/state/disk.go added +57 −0
@@ -0,0 +1,57 @@
1 package state
2
3 import (
4 "bufio"
5 "os"
6 "path/filepath"
7 "strings"
8 )
9
10 // DiskKind zegt of path op een SSD ("ssd"), een draaiende schijf ("hdd")
11 // of iets onbekends ("") staat. Alleen voor weergave in Beheer.
12 func DiskKind(path string) string {
13 abs, err := filepath.Abs(path)
14 if err != nil {
15 return ""
16 }
17 if r, err := filepath.EvalSymlinks(abs); err == nil {
18 abs = r
19 }
20 f, err := os.Open("/proc/self/mounts")
21 if err != nil {
22 return ""
23 }
24 defer f.Close()
25 best, src := "", ""
26 sc := bufio.NewScanner(f)
27 for sc.Scan() {
28 parts := strings.Fields(sc.Text())
29 if len(parts) < 2 {
30 continue
31 }
32 mnt := strings.ReplaceAll(parts[1], `\040`, " ")
33 if (abs == mnt || mnt == "/" || strings.HasPrefix(abs, strings.TrimRight(mnt, "/")+"/")) && len(mnt) >= len(best) {
34 best, src = mnt, parts[0]
35 }
36 }
37 if !strings.HasPrefix(src, "/dev/") {
38 return ""
39 }
40 dev, err := filepath.EvalSymlinks(src)
41 if err != nil {
42 dev = src
43 }
44 sys, err := filepath.EvalSymlinks(filepath.Join("/sys/class/block", filepath.Base(dev)))
45 if err != nil {
46 return ""
47 }
48 for _, p := range []string{filepath.Join(sys, "queue", "rotational"), filepath.Join(sys, "..", "queue", "rotational")} {
49 if b, err := os.ReadFile(p); err == nil {
50 if strings.TrimSpace(string(b)) == "1" {
51 return "hdd"
52 }
53 return "ssd"
54 }
55 }
56 return ""
57 }
@@ -0,0 +1,57 @@
1package state
2
3import (
4 "bufio"
5 "os"
6 "path/filepath"
7 "strings"
8)
9
10// DiskKind zegt of path op een SSD ("ssd"), een draaiende schijf ("hdd")
11// of iets onbekends ("") staat. Alleen voor weergave in Beheer.
12func DiskKind(path string) string {
13 abs, err := filepath.Abs(path)
14 if err != nil {
15 return ""
16 }
17 if r, err := filepath.EvalSymlinks(abs); err == nil {
18 abs = r
19 }
20 f, err := os.Open("/proc/self/mounts")
21 if err != nil {
22 return ""
23 }
24 defer f.Close()
25 best, src := "", ""
26 sc := bufio.NewScanner(f)
27 for sc.Scan() {
28 parts := strings.Fields(sc.Text())
29 if len(parts) < 2 {
30 continue
31 }
32 mnt := strings.ReplaceAll(parts[1], `\040`, " ")
33 if (abs == mnt || mnt == "/" || strings.HasPrefix(abs, strings.TrimRight(mnt, "/")+"/")) && len(mnt) >= len(best) {
34 best, src = mnt, parts[0]
35 }
36 }
37 if !strings.HasPrefix(src, "/dev/") {
38 return ""
39 }
40 dev, err := filepath.EvalSymlinks(src)
41 if err != nil {
42 dev = src
43 }
44 sys, err := filepath.EvalSymlinks(filepath.Join("/sys/class/block", filepath.Base(dev)))
45 if err != nil {
46 return ""
47 }
48 for _, p := range []string{filepath.Join(sys, "queue", "rotational"), filepath.Join(sys, "..", "queue", "rotational")} {
49 if b, err := os.ReadFile(p); err == nil {
50 if strings.TrimSpace(string(b)) == "1" {
51 return "hdd"
52 }
53 return "ssd"
54 }
55 }
56 return ""
57}
internal/store/find_pipelines_test.go added +51 −0
@@ -0,0 +1,51 @@
1 package store
2
3 import (
4 "context"
5 "testing"
6 )
7
8 func TestFindPipelines(t *testing.T) {
9 s := openTest(t)
10 ctx := context.Background()
11 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
12 if err != nil {
13 t.Fatal(err)
14 }
15 repo, err := s.CreateRepo(ctx, "user", tim.ID, "app", "", "master", "public")
16 if err != nil {
17 t.Fatal(err)
18 }
19 jobs := []NewJob{{Name: "t", Stage: "test", Script: "true"}}
20 a := "abcdef1111111111111111111111111111111111"
21 b := "abcdef2222222222222222222222222222222222"
22 idA, _ := s.CreatePipeline(ctx, repo.ID, a, "refs/heads/master", "push", "tim", "a", jobs)
23 idB, _ := s.CreatePipeline(ctx, repo.ID, b, "refs/heads/topic", "push", "tim", "b", jobs)
24 idT, _ := s.CreatePipeline(ctx, repo.ID, a, "refs/tags/v1", "push", "tim", "tag", jobs)
25 got := func(f PipelineFilter) []int64 {
26 list, err := s.FindPipelines(ctx, repo.ID, f)
27 if err != nil {
28 t.Fatal(err)
29 }
30 var ids []int64
31 for _, p := range list {
32 ids = append(ids, p.ID)
33 }
34 return ids
35 }
36 if ids := got(PipelineFilter{}); len(ids) != 3 || ids[0] != idT {
37 t.Fatalf("alles %v", ids)
38 }
39 if ids := got(PipelineFilter{SHA: "ABCDEF1"}); len(ids) != 2 || ids[0] != idT || ids[1] != idA {
40 t.Fatalf("sha %v", ids)
41 }
42 if ids := got(PipelineFilter{Refs: []string{"refs/heads/topic"}}); len(ids) != 1 || ids[0] != idB {
43 t.Fatalf("ref %v", ids)
44 }
45 if ids := got(PipelineFilter{SHA: a[:7], Refs: []string{"refs/heads/master"}}); len(ids) != 1 || ids[0] != idA {
46 t.Fatalf("sha+ref %v", ids)
47 }
48 if ids := got(PipelineFilter{Refs: []string{"refs/heads/nope"}}); len(ids) != 0 {
49 t.Fatalf("leeg %v", ids)
50 }
51 }
@@ -0,0 +1,51 @@
1package store
2
3import (
4 "context"
5 "testing"
6)
7
8func TestFindPipelines(t *testing.T) {
9 s := openTest(t)
10 ctx := context.Background()
11 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
12 if err != nil {
13 t.Fatal(err)
14 }
15 repo, err := s.CreateRepo(ctx, "user", tim.ID, "app", "", "master", "public")
16 if err != nil {
17 t.Fatal(err)
18 }
19 jobs := []NewJob{{Name: "t", Stage: "test", Script: "true"}}
20 a := "abcdef1111111111111111111111111111111111"
21 b := "abcdef2222222222222222222222222222222222"
22 idA, _ := s.CreatePipeline(ctx, repo.ID, a, "refs/heads/master", "push", "tim", "a", jobs)
23 idB, _ := s.CreatePipeline(ctx, repo.ID, b, "refs/heads/topic", "push", "tim", "b", jobs)
24 idT, _ := s.CreatePipeline(ctx, repo.ID, a, "refs/tags/v1", "push", "tim", "tag", jobs)
25 got := func(f PipelineFilter) []int64 {
26 list, err := s.FindPipelines(ctx, repo.ID, f)
27 if err != nil {
28 t.Fatal(err)
29 }
30 var ids []int64
31 for _, p := range list {
32 ids = append(ids, p.ID)
33 }
34 return ids
35 }
36 if ids := got(PipelineFilter{}); len(ids) != 3 || ids[0] != idT {
37 t.Fatalf("alles %v", ids)
38 }
39 if ids := got(PipelineFilter{SHA: "ABCDEF1"}); len(ids) != 2 || ids[0] != idT || ids[1] != idA {
40 t.Fatalf("sha %v", ids)
41 }
42 if ids := got(PipelineFilter{Refs: []string{"refs/heads/topic"}}); len(ids) != 1 || ids[0] != idB {
43 t.Fatalf("ref %v", ids)
44 }
45 if ids := got(PipelineFilter{SHA: a[:7], Refs: []string{"refs/heads/master"}}); len(ids) != 1 || ids[0] != idA {
46 t.Fatalf("sha+ref %v", ids)
47 }
48 if ids := got(PipelineFilter{Refs: []string{"refs/heads/nope"}}); len(ids) != 0 {
49 t.Fatalf("leeg %v", ids)
50 }
51}
internal/store/flow.go +47 −0
@@ -5,6 +5,7 @@ import (
5 5 "database/sql"
6 6 "encoding/json"
7 7 "errors"
8 "strings"
8 9 "time"
9 10
10 11 "gitlab.com/Timfa/bluegit/internal/i18n"
@@ -816,6 +817,52 @@ func (s *Store) ListPipelines(ctx context.Context, repoID int64) ([]Pipeline, er
816 817 return out, rows.Err()
817 818 }
818 819
820 // PipelineFilter beperkt FindPipelines. Lege velden filteren niet.
821 // SHA mag een prefix van minstens 7 tekens zijn. Refs is een lijst van
822 // volledige refs; MRID neemt ook pipelines van de MR zelf mee.
823 type PipelineFilter struct {
824 SHA string
825 Refs []string
826 MRID int64
827 }
828
829 // FindPipelines geeft de nieuwste pipelines die bij f passen, hooguit 50.
830 func (s *Store) FindPipelines(ctx context.Context, repoID int64, f PipelineFilter) ([]Pipeline, error) {
831 q := pipelineCols + ` WHERE repo_id = ?`
832 args := []any{repoID}
833 if f.SHA != "" {
834 q += ` AND commit_sha LIKE ?`
835 args = append(args, strings.ToLower(f.SHA)+"%")
836 }
837 var alt []string
838 if f.MRID != 0 {
839 alt = append(alt, `mr_id = ?`)
840 args = append(args, f.MRID)
841 }
842 for _, ref := range f.Refs {
843 alt = append(alt, `ref = ?`)
844 args = append(args, ref)
845 }
846 if len(alt) > 0 {
847 q += ` AND (` + strings.Join(alt, ` OR `) + `)`
848 }
849 q += ` ORDER BY id DESC LIMIT 50`
850 rows, err := s.db.QueryContext(ctx, q, args...)
851 if err != nil {
852 return nil, err
853 }
854 defer rows.Close()
855 var out []Pipeline
856 for rows.Next() {
857 p, err := scanPipeline(rows)
858 if err != nil {
859 return nil, err
860 }
861 out = append(out, p)
862 }
863 return out, rows.Err()
864 }
865
819 866 const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`
820 867
821 868 func scanPipeline(rows *sql.Rows) (Pipeline, error) {
@@ -5,6 +5,7 @@ import (
5 "database/sql"
5 "database/sql"
6 "encoding/json"
6 "encoding/json"
7 "errors"
7 "errors"
8 "strings"
8 "time"
9 "time"
9
10
10 "gitlab.com/Timfa/bluegit/internal/i18n"
11 "gitlab.com/Timfa/bluegit/internal/i18n"
@@ -816,6 +817,52 @@ func (s *Store) ListPipelines(ctx context.Context, repoID int64) ([]Pipeline, er
816 return out, rows.Err()
817 return out, rows.Err()
817}
818}
818
819
820// PipelineFilter beperkt FindPipelines. Lege velden filteren niet.
821// SHA mag een prefix van minstens 7 tekens zijn. Refs is een lijst van
822// volledige refs; MRID neemt ook pipelines van de MR zelf mee.
823type PipelineFilter struct {
824 SHA string
825 Refs []string
826 MRID int64
827}
828
829// FindPipelines geeft de nieuwste pipelines die bij f passen, hooguit 50.
830func (s *Store) FindPipelines(ctx context.Context, repoID int64, f PipelineFilter) ([]Pipeline, error) {
831 q := pipelineCols + ` WHERE repo_id = ?`
832 args := []any{repoID}
833 if f.SHA != "" {
834 q += ` AND commit_sha LIKE ?`
835 args = append(args, strings.ToLower(f.SHA)+"%")
836 }
837 var alt []string
838 if f.MRID != 0 {
839 alt = append(alt, `mr_id = ?`)
840 args = append(args, f.MRID)
841 }
842 for _, ref := range f.Refs {
843 alt = append(alt, `ref = ?`)
844 args = append(args, ref)
845 }
846 if len(alt) > 0 {
847 q += ` AND (` + strings.Join(alt, ` OR `) + `)`
848 }
849 q += ` ORDER BY id DESC LIMIT 50`
850 rows, err := s.db.QueryContext(ctx, q, args...)
851 if err != nil {
852 return nil, err
853 }
854 defer rows.Close()
855 var out []Pipeline
856 for rows.Next() {
857 p, err := scanPipeline(rows)
858 if err != nil {
859 return nil, err
860 }
861 out = append(out, p)
862 }
863 return out, rows.Err()
864}
865
819const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`
866const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`
820
867
821func scanPipeline(rows *sql.Rows) (Pipeline, error) {
868func scanPipeline(rows *sql.Rows) (Pipeline, error) {
packaging/bluegit.service +2 −0
@@ -22,6 +22,8 @@ ProtectHome=true
22 22 # pad toe via packaging/storage.conf, want ProtectSystem blokkeert het anders.
23 23 StateDirectory=bluegit
24 24 ReadWritePaths=/var/lib/bluegit
25 # CacheDirectory maakt /var/cache/bluegit aan voor de optionele leescache.
26 CacheDirectory=bluegit
25 27 PrivateTmp=true
26 28
27 29 [Install]
@@ -22,6 +22,8 @@ ProtectHome=true
22# pad toe via packaging/storage.conf, want ProtectSystem blokkeert het anders.
22# pad toe via packaging/storage.conf, want ProtectSystem blokkeert het anders.
23StateDirectory=bluegit
23StateDirectory=bluegit
24ReadWritePaths=/var/lib/bluegit
24ReadWritePaths=/var/lib/bluegit
25# CacheDirectory maakt /var/cache/bluegit aan voor de optionele leescache.
26CacheDirectory=bluegit
25PrivateTmp=true
27PrivateTmp=true
26
28
27[Install]
29[Install]
packaging/bluegit.yml +3 −0
@@ -6,6 +6,9 @@ listen: 0.0.0.0:8080
6 6 state_path: /var/lib/bluegit
7 7 # storage_path blijft leeg. De installatiebeheerder kiest de datadisk
8 8 # bij de eerste aanmelding. Er is geen standaardpad.
9 # cache_path is optioneel: een leescache voor overzichtspagina's, bij
10 # voorkeur op de SSD (bijv. /var/cache/bluegit). Leeg = geen cache.
11 # Beheer (/setup) kan het ook instellen. cache_max_mb: standaard 512.
9 12 # Alleen deze adressen mogen X-Forwarded-For en X-Real-IP zetten.
10 13 # Zet hier het IP van de ingress, zoals de mini-PC die verbinding ziet.
11 14 trusted_proxies:
@@ -6,6 +6,9 @@ listen: 0.0.0.0:8080
6state_path: /var/lib/bluegit
6state_path: /var/lib/bluegit
7# storage_path blijft leeg. De installatiebeheerder kiest de datadisk
7# storage_path blijft leeg. De installatiebeheerder kiest de datadisk
8# bij de eerste aanmelding. Er is geen standaardpad.
8# bij de eerste aanmelding. Er is geen standaardpad.
9# cache_path is optioneel: een leescache voor overzichtspagina's, bij
10# voorkeur op de SSD (bijv. /var/cache/bluegit). Leeg = geen cache.
11# Beheer (/setup) kan het ook instellen. cache_max_mb: standaard 512.
9# Alleen deze adressen mogen X-Forwarded-For en X-Real-IP zetten.
12# Alleen deze adressen mogen X-Forwarded-For en X-Real-IP zetten.
10# Zet hier het IP van de ingress, zoals de mini-PC die verbinding ziet.
13# Zet hier het IP van de ingress, zoals de mini-PC die verbinding ziet.
11trusted_proxies:
14trusted_proxies:
scripts/bg added +208 −0
@@ -0,0 +1,208 @@
1 #!/usr/bin/env bash
2 # bg: kleine BlueGit-CLI voor mensen en agents. Alleen bash, curl en jq.
3 #
4 # BLUEGIT_URL standaard https://git.timfalken.com
5 # BLUEGIT_TOKEN optioneel; nodig voor schrijven en voor privé projecten.
6 # Wordt nooit geprint.
7 #
8 # Zie docs/agents.md of `bg help`.
9 set -euo pipefail
10
11 BASE="${BLUEGIT_URL:-https://git.timfalken.com}"
12 BASE="${BASE%/}"
13 API="$BASE/api/v1"
14
15 die() { echo "bg: $*" >&2; exit 64; }
16 command -v curl >/dev/null || die "curl ontbreekt"
17 command -v jq >/dev/null || die "jq ontbreekt"
18
19 # req METHOD PATH [JSON] -> body op stdout; bij HTTP >= 400 de fout op stderr en exit 1.
20 req() {
21 local method=$1 path=$2 data=${3:-} out code
22 local args=(-sS -X "$method" -w '\n%{http_code}' -H "Accept: application/json")
23 [[ -n "${BLUEGIT_TOKEN:-}" ]] && args+=(-H "Authorization: Bearer ${BLUEGIT_TOKEN}")
24 [[ -n "$data" ]] && args+=(-H "Content-Type: application/json" --data-binary "$data")
25 out=$(curl "${args[@]}" "$API$path") || { echo "bg: verbinding mislukt: $method $path" >&2; return 1; }
26 code=${out##*$'\n'}
27 out=${out%$'\n'*}
28 if (( code >= 400 )); then
29 local msg
30 msg=$(printf '%s' "$out" | jq -r '.message // .error // empty' 2>/dev/null || true)
31 echo "bg: HTTP $code bij $method $path${msg:+: $msg}" >&2
32 [[ $code == 404 ]] && echo "bg: (bestaat niet, of je hebt geen toegang)" >&2
33 return 1
34 fi
35 printf '%s' "$out"
36 }
37 get() { req GET "$1"; }
38 enc() { jq -rn --arg v "$1" '$v|@uri'; }
39 proj() { [[ "$1" == */* ]] || die "project moet owner/repo zijn: $1"; echo "/projects/$1"; }
40 issues() { echo "/repos/$1/issues"; }
41 json() { jq -cn "$@"; }
42
43 # ---------- pipelines ----------
44
45 # select_pipeline PROJECT ARGS... -> pipeline-JSON (of leeg als er nog geen is)
46 select_query() {
47 local q=""
48 while (($#)); do
49 case $1 in
50 --sha) q="sha=$2"; shift 2 ;;
51 --mr) q="mr=$2"; shift 2 ;;
52 --branch|--ref) q="ref=$(enc "$2")"; shift 2 ;;
53 --id) q="id=$2"; shift 2 ;;
54 --timeout|--interval) shift 2 ;;
55 *) die "onbekende optie: $1" ;;
56 esac
57 done
58 echo "$q"
59 }
60 find_pipeline() {
61 local p=$1 q=$2
62 if [[ $q == id=* ]]; then get "$(proj "$p")/pipelines/${q#id=}"; return; fi
63 get "$(proj "$p")/pipelines?per_page=1${q:+&$q}" | jq -c '.[0] // empty'
64 }
65 show_pipeline() {
66 local p=$1 pipe=$2
67 jq -r '"pipeline #\(.id) \(.status) \(.ref|sub("^refs/heads/";"")) \(.sha[0:8]) \(.web_url)"' <<<"$pipe"
68 get "$(proj "$p")/pipelines/$(jq -r .id <<<"$pipe")/jobs" |
69 jq -r '.[] | " \(.id)\t\(.status)\t\(.stage)/\(.name)\(if .allow_failure then " (allow_failure)" else "" end)"'
70 }
71 failed_logs() {
72 local p=$1 id=$2 jobs
73 jobs=$(get "$(proj "$p")/pipelines/$id/jobs")
74 jq -r '.[]|select(.status=="failed" and .allow_failure)|"bg: waarschuwing: \(.name) (#\(.id)) faalde (allow_failure)"' <<<"$jobs" >&2
75 for j in $(jq -r '.[]|select(.status=="failed" and (.allow_failure|not))|.id' <<<"$jobs"); do
76 echo "----- log job $j (laatste ${BG_LOG_LINES:-80} regels) -----" >&2
77 get "$(proj "$p")/pipelines/$id/jobs/$j/log" | tail -n "${BG_LOG_LINES:-80}" >&2 || true
78 done
79 }
80 cmd_pipeline() {
81 local sub=${1:-}; shift || true
82 case $sub in
83 status)
84 local p=${1:?project}; shift
85 local q pipe; q=$(select_query "$@")
86 pipe=$(find_pipeline "$p" "$q")
87 [[ -n "$pipe" ]] || { echo "geen pipeline gevonden" >&2; exit 3; }
88 show_pipeline "$p" "$pipe" ;;
89 wait)
90 local p=${1:?project}; shift
91 local timeout=1800 interval=10 a=("$@")
92 while (($#)); do case $1 in --timeout) timeout=$2; shift 2 ;; --interval) interval=$2; shift 2 ;; *) shift ;; esac; done
93 local q pipe status start=$SECONDS last=""
94 q=$(select_query "${a[@]}")
95 while :; do
96 pipe=$(find_pipeline "$p" "$q" || true)
97 status=$(jq -r '.status // empty' <<<"${pipe:-{\}}")
98 if [[ -n "$status" && "$status" != "$last" ]]; then
99 echo "pipeline #$(jq -r .id <<<"$pipe"): $status" >&2; last=$status
100 fi
101 case $status in
102 success) show_pipeline "$p" "$pipe"; failed_logs "$p" "$(jq -r .id <<<"$pipe")" 2>&1 | grep '^bg: waarschuwing' >&2 || true; exit 0 ;;
103 failed|canceled) show_pipeline "$p" "$pipe"; failed_logs "$p" "$(jq -r .id <<<"$pipe")"; exit 1 ;;
104 esac
105 (( SECONDS - start >= timeout )) && { echo "bg: timeout na ${timeout}s (status ${status:-geen pipeline})" >&2; exit 2; }
106 sleep "$interval"
107 done ;;
108 logs) get "$(proj "${1:?project}")/pipelines/${2:?pipeline-id}/jobs/${3:?job-id}/log" ;;
109 retry)
110 local mode=failed
111 [[ ${3:-} == --all ]] && mode=all
112 req POST "$(proj "${1:?project}")/pipelines/${2:?pipeline-id}/retry" "$(json --arg m "$mode" '{mode:$m}')" | jq -r '"pipeline #\(.id) \(.status) \(.web_url)"' ;;
113 list) get "$(proj "${1:?project}")/pipelines?per_page=${2:-10}" | jq -r '.[]|"\(.id)\t\(.status)\t\(.ref|sub("^refs/heads/";""))\t\(.sha[0:8])"' ;;
114 *) die "pipeline status|wait|logs|retry|list" ;;
115 esac
116 }
117
118 # ---------- merge requests ----------
119 cmd_mr() {
120 local sub=${1:-}; shift || true
121 case $sub in
122 list)
123 local p=${1:?project}; shift; local q="state=open"
124 while (($#)); do case $1 in --state) q="state=$2"; shift 2 ;; --source) q+="&source_branch=$(enc "$2")"; shift 2 ;; --target) q+="&target_branch=$(enc "$2")"; shift 2 ;; *) die "onbekende optie $1" ;; esac; done
125 get "$(proj "$p")/merge_requests?per_page=100&$q" | jq -r '.[]|"!\(.iid)\t\(.state)\t\(.source_branch) -> \(.target_branch)\t\(.title)"' ;;
126 show) get "$(proj "${1:?project}")/merge_requests/${2:?nummer}" | jq . ;;
127 create)
128 local p=${1:?project}; shift; local src="" tgt="" title="" desc="" remove=true
129 while (($#)); do case $1 in --source) src=$2; shift 2 ;; --target) tgt=$2; shift 2 ;; --title) title=$2; shift 2 ;; --description) desc=$2; shift 2 ;; --keep-source) remove=false; shift ;; *) die "onbekende optie $1" ;; esac; done
130 [[ -n $src && -n $tgt && -n $title ]] || die "mr create PROJECT --source B --target B --title T [--description D] [--keep-source]"
131 req POST "$(proj "$p")/merge_requests" "$(json --arg s "$src" --arg t "$tgt" --arg ti "$title" --arg d "$desc" --argjson r "$remove" '{source_branch:$s,target_branch:$t,title:$ti,description:$d,remove_source_branch:$r}')" | jq -r '"!\(.iid) \(.web_url)"' ;;
132 merge)
133 local remove=true; [[ ${3:-} == --keep-source ]] && remove=false
134 req POST "$(proj "${1:?project}")/merge_requests/${2:?nummer}/merge" "$(json --argjson r "$remove" '{remove_source_branch:$r}')" | jq -r '"\(.state) \(.web_url)"' ;;
135 comment) req POST "$(proj "${1:?project}")/merge_requests/${2:?nummer}/notes" "$(json --arg b "${3:?tekst}" '{body:$b}')" | jq -r '"notitie \(.id)"' ;;
136 *) die "mr list|show|create|merge|comment" ;;
137 esac
138 }
139
140 # ---------- issues ----------
141 cmd_issue() {
142 local sub=${1:-}; shift || true
143 case $sub in
144 list) get "$(issues "${1:?project}")?state=${2:-open}" | jq -r '.issues[]|"#\(.id)\t\(.status)\t\(.title)"' ;;
145 show) get "$(issues "${1:?project}")/${2:?nummer}" | jq . ;;
146 create) req POST "$(issues "${1:?project}")" "$(json --arg t "${2:?titel}" --arg d "${3:-}" '{title:$t,description:$d}')" | jq -r '"#\(.id) \(.web_url)"' ;;
147 comment) req POST "$(issues "${1:?project}")/${2:?nummer}/notes" "$(json --arg b "${3:?tekst}" '{body:$b}')" >/dev/null && echo "ok" ;;
148 close|reopen)
149 local st=closed; [[ $sub == reopen ]] && st=open
150 req PUT "$(issues "${1:?project}")/${2:?nummer}" "$(json --arg s "$st" '{state:$s}')" | jq -r '"#\(.id) \(.status)"' ;;
151 *) die "issue list|show|create|comment|close|reopen" ;;
152 esac
153 }
154
155 # ---------- overig ----------
156 cmd_branch() {
157 [[ ${1:-} == create ]] || die "branch create PROJECT NAAM [--from REF]"
158 local p=${2:?project} name=${3:?naam} from=""
159 [[ ${4:-} == --from ]] && from=${5:?ref}
160 req POST "$(proj "$p")/repository/branches" "$(json --arg n "$name" --arg f "$from" '{name:$n} + (if $f=="" then {} else {from:$f} end)')" | jq -r '.name // .'
161 }
162 cmd_release() {
163 [[ ${1:-} == create ]] || die "release create PROJECT TAG [--commit SHA] [--name N] [--body B] [--asset JOB:PAD[:NAAM]]..."
164 local p=${2:?project} tag=${3:?tag}; shift 3
165 local commit="" name="" body="" assets="[]"
166 while (($#)); do case $1 in
167 --commit) commit=$2; shift 2 ;; --name) name=$2; shift 2 ;; --body) body=$2; shift 2 ;;
168 --asset) IFS=: read -r j pth nm <<<"$2"; assets=$(jq -c --arg j "$j" --arg p "$pth" --arg n "${nm:-}" '. + [{job:$j,path:$p} + (if $n=="" then {} else {name:$n} end)]' <<<"$assets"); shift 2 ;;
169 *) die "onbekende optie $1" ;; esac; done
170 req POST "$(proj "$p")/releases" "$(json --arg t "$tag" --arg c "$commit" --arg n "$name" --arg b "$body" --argjson a "$assets" '{tag:$t,commit:$c,name:$n,body:$b,assets:$a}')" | jq -c .
171 }
172 cmd_file() {
173 [[ ${1:-} == get ]] || die "file get PROJECT PAD [--ref REF]"
174 local p=${2:?project} path=${3:?pad} ref=""
175 [[ ${4:-} == --ref ]] && ref="?ref=$(enc "${5:?ref}")"
176 local body; body=$(get "$(proj "$p")/repository/files/$(enc "$path")$ref")
177 if jq -e '.content and .encoding=="base64"' <<<"$body" >/dev/null 2>&1; then jq -r .content <<<"$body" | base64 -d
178 elif jq -e '.content' <<<"$body" >/dev/null 2>&1; then jq -r .content <<<"$body"
179 else printf '%s' "$body"; fi
180 }
181
182 usage() {
183 sed -n '2,8p' "$0" | sed 's/^# \{0,1\}//'
184 cat <<'U'
185
186 bg pipeline status PROJECT [--sha X | --mr N | --branch B | --id P]
187 bg pipeline wait PROJECT [selectie] [--timeout 1800] [--interval 10] exit 0 ok, 1 mislukt, 2 timeout
188 bg pipeline logs PROJECT PIPELINE JOB
189 bg pipeline retry PROJECT PIPELINE [--all]
190 bg pipeline list PROJECT [N]
191 bg mr list|show|create|merge|comment ...
192 bg issue list|show|create|comment|close|reopen ...
193 bg branch create PROJECT NAAM [--from REF]
194 bg release create PROJECT TAG [--commit SHA] [--asset JOB:PAD[:NAAM]]
195 bg file get PROJECT PAD [--ref REF]
196 U
197 }
198
199 case ${1:-help} in
200 pipeline|pl) shift; cmd_pipeline "$@" ;;
201 mr) shift; cmd_mr "$@" ;;
202 issue) shift; cmd_issue "$@" ;;
203 branch) shift; cmd_branch "$@" ;;
204 release) shift; cmd_release "$@" ;;
205 file) shift; cmd_file "$@" ;;
206 help|-h|--help) usage ;;
207 *) usage >&2; exit 64 ;;
208 esac
@@ -0,0 +1,208 @@
1#!/usr/bin/env bash
2# bg: kleine BlueGit-CLI voor mensen en agents. Alleen bash, curl en jq.
3#
4# BLUEGIT_URL standaard https://git.timfalken.com
5# BLUEGIT_TOKEN optioneel; nodig voor schrijven en voor privé projecten.
6# Wordt nooit geprint.
7#
8# Zie docs/agents.md of `bg help`.
9set -euo pipefail
10
11BASE="${BLUEGIT_URL:-https://git.timfalken.com}"
12BASE="${BASE%/}"
13API="$BASE/api/v1"
14
15die() { echo "bg: $*" >&2; exit 64; }
16command -v curl >/dev/null || die "curl ontbreekt"
17command -v jq >/dev/null || die "jq ontbreekt"
18
19# req METHOD PATH [JSON] -> body op stdout; bij HTTP >= 400 de fout op stderr en exit 1.
20req() {
21 local method=$1 path=$2 data=${3:-} out code
22 local args=(-sS -X "$method" -w '\n%{http_code}' -H "Accept: application/json")
23 [[ -n "${BLUEGIT_TOKEN:-}" ]] && args+=(-H "Authorization: Bearer ${BLUEGIT_TOKEN}")
24 [[ -n "$data" ]] && args+=(-H "Content-Type: application/json" --data-binary "$data")
25 out=$(curl "${args[@]}" "$API$path") || { echo "bg: verbinding mislukt: $method $path" >&2; return 1; }
26 code=${out##*$'\n'}
27 out=${out%$'\n'*}
28 if (( code >= 400 )); then
29 local msg
30 msg=$(printf '%s' "$out" | jq -r '.message // .error // empty' 2>/dev/null || true)
31 echo "bg: HTTP $code bij $method $path${msg:+: $msg}" >&2
32 [[ $code == 404 ]] && echo "bg: (bestaat niet, of je hebt geen toegang)" >&2
33 return 1
34 fi
35 printf '%s' "$out"
36}
37get() { req GET "$1"; }
38enc() { jq -rn --arg v "$1" '$v|@uri'; }
39proj() { [[ "$1" == */* ]] || die "project moet owner/repo zijn: $1"; echo "/projects/$1"; }
40issues() { echo "/repos/$1/issues"; }
41json() { jq -cn "$@"; }
42
43# ---------- pipelines ----------
44
45# select_pipeline PROJECT ARGS... -> pipeline-JSON (of leeg als er nog geen is)
46select_query() {
47 local q=""
48 while (($#)); do
49 case $1 in
50 --sha) q="sha=$2"; shift 2 ;;
51 --mr) q="mr=$2"; shift 2 ;;
52 --branch|--ref) q="ref=$(enc "$2")"; shift 2 ;;
53 --id) q="id=$2"; shift 2 ;;
54 --timeout|--interval) shift 2 ;;
55 *) die "onbekende optie: $1" ;;
56 esac
57 done
58 echo "$q"
59}
60find_pipeline() {
61 local p=$1 q=$2
62 if [[ $q == id=* ]]; then get "$(proj "$p")/pipelines/${q#id=}"; return; fi
63 get "$(proj "$p")/pipelines?per_page=1${q:+&$q}" | jq -c '.[0] // empty'
64}
65show_pipeline() {
66 local p=$1 pipe=$2
67 jq -r '"pipeline #\(.id) \(.status) \(.ref|sub("^refs/heads/";"")) \(.sha[0:8]) \(.web_url)"' <<<"$pipe"
68 get "$(proj "$p")/pipelines/$(jq -r .id <<<"$pipe")/jobs" |
69 jq -r '.[] | " \(.id)\t\(.status)\t\(.stage)/\(.name)\(if .allow_failure then " (allow_failure)" else "" end)"'
70}
71failed_logs() {
72 local p=$1 id=$2 jobs
73 jobs=$(get "$(proj "$p")/pipelines/$id/jobs")
74 jq -r '.[]|select(.status=="failed" and .allow_failure)|"bg: waarschuwing: \(.name) (#\(.id)) faalde (allow_failure)"' <<<"$jobs" >&2
75 for j in $(jq -r '.[]|select(.status=="failed" and (.allow_failure|not))|.id' <<<"$jobs"); do
76 echo "----- log job $j (laatste ${BG_LOG_LINES:-80} regels) -----" >&2
77 get "$(proj "$p")/pipelines/$id/jobs/$j/log" | tail -n "${BG_LOG_LINES:-80}" >&2 || true
78 done
79}
80cmd_pipeline() {
81 local sub=${1:-}; shift || true
82 case $sub in
83 status)
84 local p=${1:?project}; shift
85 local q pipe; q=$(select_query "$@")
86 pipe=$(find_pipeline "$p" "$q")
87 [[ -n "$pipe" ]] || { echo "geen pipeline gevonden" >&2; exit 3; }
88 show_pipeline "$p" "$pipe" ;;
89 wait)
90 local p=${1:?project}; shift
91 local timeout=1800 interval=10 a=("$@")
92 while (($#)); do case $1 in --timeout) timeout=$2; shift 2 ;; --interval) interval=$2; shift 2 ;; *) shift ;; esac; done
93 local q pipe status start=$SECONDS last=""
94 q=$(select_query "${a[@]}")
95 while :; do
96 pipe=$(find_pipeline "$p" "$q" || true)
97 status=$(jq -r '.status // empty' <<<"${pipe:-{\}}")
98 if [[ -n "$status" && "$status" != "$last" ]]; then
99 echo "pipeline #$(jq -r .id <<<"$pipe"): $status" >&2; last=$status
100 fi
101 case $status in
102 success) show_pipeline "$p" "$pipe"; failed_logs "$p" "$(jq -r .id <<<"$pipe")" 2>&1 | grep '^bg: waarschuwing' >&2 || true; exit 0 ;;
103 failed|canceled) show_pipeline "$p" "$pipe"; failed_logs "$p" "$(jq -r .id <<<"$pipe")"; exit 1 ;;
104 esac
105 (( SECONDS - start >= timeout )) && { echo "bg: timeout na ${timeout}s (status ${status:-geen pipeline})" >&2; exit 2; }
106 sleep "$interval"
107 done ;;
108 logs) get "$(proj "${1:?project}")/pipelines/${2:?pipeline-id}/jobs/${3:?job-id}/log" ;;
109 retry)
110 local mode=failed
111 [[ ${3:-} == --all ]] && mode=all
112 req POST "$(proj "${1:?project}")/pipelines/${2:?pipeline-id}/retry" "$(json --arg m "$mode" '{mode:$m}')" | jq -r '"pipeline #\(.id) \(.status) \(.web_url)"' ;;
113 list) get "$(proj "${1:?project}")/pipelines?per_page=${2:-10}" | jq -r '.[]|"\(.id)\t\(.status)\t\(.ref|sub("^refs/heads/";""))\t\(.sha[0:8])"' ;;
114 *) die "pipeline status|wait|logs|retry|list" ;;
115 esac
116}
117
118# ---------- merge requests ----------
119cmd_mr() {
120 local sub=${1:-}; shift || true
121 case $sub in
122 list)
123 local p=${1:?project}; shift; local q="state=open"
124 while (($#)); do case $1 in --state) q="state=$2"; shift 2 ;; --source) q+="&source_branch=$(enc "$2")"; shift 2 ;; --target) q+="&target_branch=$(enc "$2")"; shift 2 ;; *) die "onbekende optie $1" ;; esac; done
125 get "$(proj "$p")/merge_requests?per_page=100&$q" | jq -r '.[]|"!\(.iid)\t\(.state)\t\(.source_branch) -> \(.target_branch)\t\(.title)"' ;;
126 show) get "$(proj "${1:?project}")/merge_requests/${2:?nummer}" | jq . ;;
127 create)
128 local p=${1:?project}; shift; local src="" tgt="" title="" desc="" remove=true
129 while (($#)); do case $1 in --source) src=$2; shift 2 ;; --target) tgt=$2; shift 2 ;; --title) title=$2; shift 2 ;; --description) desc=$2; shift 2 ;; --keep-source) remove=false; shift ;; *) die "onbekende optie $1" ;; esac; done
130 [[ -n $src && -n $tgt && -n $title ]] || die "mr create PROJECT --source B --target B --title T [--description D] [--keep-source]"
131 req POST "$(proj "$p")/merge_requests" "$(json --arg s "$src" --arg t "$tgt" --arg ti "$title" --arg d "$desc" --argjson r "$remove" '{source_branch:$s,target_branch:$t,title:$ti,description:$d,remove_source_branch:$r}')" | jq -r '"!\(.iid) \(.web_url)"' ;;
132 merge)
133 local remove=true; [[ ${3:-} == --keep-source ]] && remove=false
134 req POST "$(proj "${1:?project}")/merge_requests/${2:?nummer}/merge" "$(json --argjson r "$remove" '{remove_source_branch:$r}')" | jq -r '"\(.state) \(.web_url)"' ;;
135 comment) req POST "$(proj "${1:?project}")/merge_requests/${2:?nummer}/notes" "$(json --arg b "${3:?tekst}" '{body:$b}')" | jq -r '"notitie \(.id)"' ;;
136 *) die "mr list|show|create|merge|comment" ;;
137 esac
138}
139
140# ---------- issues ----------
141cmd_issue() {
142 local sub=${1:-}; shift || true
143 case $sub in
144 list) get "$(issues "${1:?project}")?state=${2:-open}" | jq -r '.issues[]|"#\(.id)\t\(.status)\t\(.title)"' ;;
145 show) get "$(issues "${1:?project}")/${2:?nummer}" | jq . ;;
146 create) req POST "$(issues "${1:?project}")" "$(json --arg t "${2:?titel}" --arg d "${3:-}" '{title:$t,description:$d}')" | jq -r '"#\(.id) \(.web_url)"' ;;
147 comment) req POST "$(issues "${1:?project}")/${2:?nummer}/notes" "$(json --arg b "${3:?tekst}" '{body:$b}')" >/dev/null && echo "ok" ;;
148 close|reopen)
149 local st=closed; [[ $sub == reopen ]] && st=open
150 req PUT "$(issues "${1:?project}")/${2:?nummer}" "$(json --arg s "$st" '{state:$s}')" | jq -r '"#\(.id) \(.status)"' ;;
151 *) die "issue list|show|create|comment|close|reopen" ;;
152 esac
153}
154
155# ---------- overig ----------
156cmd_branch() {
157 [[ ${1:-} == create ]] || die "branch create PROJECT NAAM [--from REF]"
158 local p=${2:?project} name=${3:?naam} from=""
159 [[ ${4:-} == --from ]] && from=${5:?ref}
160 req POST "$(proj "$p")/repository/branches" "$(json --arg n "$name" --arg f "$from" '{name:$n} + (if $f=="" then {} else {from:$f} end)')" | jq -r '.name // .'
161}
162cmd_release() {
163 [[ ${1:-} == create ]] || die "release create PROJECT TAG [--commit SHA] [--name N] [--body B] [--asset JOB:PAD[:NAAM]]..."
164 local p=${2:?project} tag=${3:?tag}; shift 3
165 local commit="" name="" body="" assets="[]"
166 while (($#)); do case $1 in
167 --commit) commit=$2; shift 2 ;; --name) name=$2; shift 2 ;; --body) body=$2; shift 2 ;;
168 --asset) IFS=: read -r j pth nm <<<"$2"; assets=$(jq -c --arg j "$j" --arg p "$pth" --arg n "${nm:-}" '. + [{job:$j,path:$p} + (if $n=="" then {} else {name:$n} end)]' <<<"$assets"); shift 2 ;;
169 *) die "onbekende optie $1" ;; esac; done
170 req POST "$(proj "$p")/releases" "$(json --arg t "$tag" --arg c "$commit" --arg n "$name" --arg b "$body" --argjson a "$assets" '{tag:$t,commit:$c,name:$n,body:$b,assets:$a}')" | jq -c .
171}
172cmd_file() {
173 [[ ${1:-} == get ]] || die "file get PROJECT PAD [--ref REF]"
174 local p=${2:?project} path=${3:?pad} ref=""
175 [[ ${4:-} == --ref ]] && ref="?ref=$(enc "${5:?ref}")"
176 local body; body=$(get "$(proj "$p")/repository/files/$(enc "$path")$ref")
177 if jq -e '.content and .encoding=="base64"' <<<"$body" >/dev/null 2>&1; then jq -r .content <<<"$body" | base64 -d
178 elif jq -e '.content' <<<"$body" >/dev/null 2>&1; then jq -r .content <<<"$body"
179 else printf '%s' "$body"; fi
180}
181
182usage() {
183 sed -n '2,8p' "$0" | sed 's/^# \{0,1\}//'
184 cat <<'U'
185
186 bg pipeline status PROJECT [--sha X | --mr N | --branch B | --id P]
187 bg pipeline wait PROJECT [selectie] [--timeout 1800] [--interval 10] exit 0 ok, 1 mislukt, 2 timeout
188 bg pipeline logs PROJECT PIPELINE JOB
189 bg pipeline retry PROJECT PIPELINE [--all]
190 bg pipeline list PROJECT [N]
191 bg mr list|show|create|merge|comment ...
192 bg issue list|show|create|comment|close|reopen ...
193 bg branch create PROJECT NAAM [--from REF]
194 bg release create PROJECT TAG [--commit SHA] [--asset JOB:PAD[:NAAM]]
195 bg file get PROJECT PAD [--ref REF]
196U
197}
198
199case ${1:-help} in
200 pipeline|pl) shift; cmd_pipeline "$@" ;;
201 mr) shift; cmd_mr "$@" ;;
202 issue) shift; cmd_issue "$@" ;;
203 branch) shift; cmd_branch "$@" ;;
204 release) shift; cmd_release "$@" ;;
205 file) shift; cmd_file "$@" ;;
206 help|-h|--help) usage ;;
207 *) usage >&2; exit 64 ;;
208esac
scripts/bg_test.go added +97 −0
@@ -0,0 +1,97 @@
1 package scripts
2
3 import (
4 "bytes"
5 "errors"
6 "fmt"
7 "net/http"
8 "net/http/httptest"
9 "os"
10 "os/exec"
11 "path/filepath"
12 "strings"
13 "sync/atomic"
14 "testing"
15 )
16
17 // fakeBlueGit geeft eerst geen pipeline, dan running, dan final.
18 func fakeBlueGit(t *testing.T, final string, token string) (*httptest.Server, *atomic.Int32) {
19 var calls atomic.Int32
20 srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
21 if token != "" && r.Header.Get("Authorization") != "Bearer "+token {
22 http.Error(w, `{"message":"no"}`, 401)
23 return
24 }
25 switch {
26 case r.URL.Path == "/api/v1/projects/tim/app/pipelines":
27 if r.URL.Query().Get("sha") != "abcdef1" {
28 t.Errorf("sha-filter ontbreekt: %s", r.URL.RawQuery)
29 }
30 n := calls.Add(1)
31 switch {
32 case n == 1:
33 fmt.Fprint(w, `[]`)
34 case n == 2:
35 fmt.Fprint(w, `[{"id":9,"status":"running","ref":"refs/heads/x","sha":"abcdef1000","web_url":"u"}]`)
36 default:
37 fmt.Fprintf(w, `[{"id":9,"status":%q,"ref":"refs/heads/x","sha":"abcdef1000","web_url":"u"}]`, final)
38 }
39 case r.URL.Path == "/api/v1/projects/tim/app/pipelines/9/jobs":
40 fmt.Fprint(w, `[{"id":1,"name":"test","stage":"test","status":"failed","allow_failure":false},{"id":2,"name":"aur","stage":"release","status":"failed","allow_failure":true}]`)
41 case r.URL.Path == "/api/v1/projects/tim/app/pipelines/9/jobs/1/log":
42 fmt.Fprint(w, "regel 1\nFOUT hier\n")
43 default:
44 http.NotFound(w, r)
45 }
46 }))
47 t.Cleanup(srv.Close)
48 return srv, &calls
49 }
50
51 func runBG(t *testing.T, url, token string, args ...string) (int, string, string) {
52 t.Helper()
53 for _, bin := range []string{"bash", "curl", "jq"} {
54 if _, err := exec.LookPath(bin); err != nil {
55 t.Skip(bin + " ontbreekt")
56 }
57 }
58 cmd := exec.Command(filepath.Join("..", "scripts", "bg"), args...)
59 cmd.Env = append(os.Environ(), "BLUEGIT_URL="+url, "BLUEGIT_TOKEN="+token)
60 var out, errb bytes.Buffer
61 cmd.Stdout, cmd.Stderr = &out, &errb
62 err := cmd.Run()
63 code := 0
64 var ee *exec.ExitError
65 if errors.As(err, &ee) {
66 code = ee.ExitCode()
67 } else if err != nil {
68 t.Fatal(err)
69 }
70 return code, out.String(), errb.String()
71 }
72
73 func TestBGWaitExitCodes(t *testing.T) {
74 const tok = "bg_geheimtoken123"
75 srv, _ := fakeBlueGit(t, "success", tok)
76 code, out, errs := runBG(t, srv.URL, tok, "pipeline", "wait", "tim/app", "--sha", "abcdef1", "--interval", "0")
77 if code != 0 || !strings.Contains(out, "pipeline #9 success") {
78 t.Fatalf("success: %d %s %s", code, out, errs)
79 }
80 if strings.Contains(out+errs, tok) {
81 t.Fatal("token geprint")
82 }
83 srv, _ = fakeBlueGit(t, "failed", "")
84 code, out, errs = runBG(t, srv.URL, "", "pipeline", "wait", "tim/app", "--sha", "abcdef1", "--interval", "0")
85 if code != 1 || !strings.Contains(errs, "FOUT hier") || !strings.Contains(errs, "waarschuwing: aur") {
86 t.Fatalf("failed: %d %s %s", code, out, errs)
87 }
88 srv, _ = fakeBlueGit(t, "running", "")
89 code, _, errs = runBG(t, srv.URL, "", "pipeline", "wait", "tim/app", "--sha", "abcdef1", "--interval", "1", "--timeout", "2")
90 if code != 2 || !strings.Contains(errs, "timeout") {
91 t.Fatalf("timeout: %d %s", code, errs)
92 }
93 code, _, errs = runBG(t, srv.URL, "", "mr", "show", "tim/app", "5")
94 if code != 1 || !strings.Contains(errs, "404") {
95 t.Fatalf("404: %d %s", code, errs)
96 }
97 }
@@ -0,0 +1,97 @@
1package scripts
2
3import (
4 "bytes"
5 "errors"
6 "fmt"
7 "net/http"
8 "net/http/httptest"
9 "os"
10 "os/exec"
11 "path/filepath"
12 "strings"
13 "sync/atomic"
14 "testing"
15)
16
17// fakeBlueGit geeft eerst geen pipeline, dan running, dan final.
18func fakeBlueGit(t *testing.T, final string, token string) (*httptest.Server, *atomic.Int32) {
19 var calls atomic.Int32
20 srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
21 if token != "" && r.Header.Get("Authorization") != "Bearer "+token {
22 http.Error(w, `{"message":"no"}`, 401)
23 return
24 }
25 switch {
26 case r.URL.Path == "/api/v1/projects/tim/app/pipelines":
27 if r.URL.Query().Get("sha") != "abcdef1" {
28 t.Errorf("sha-filter ontbreekt: %s", r.URL.RawQuery)
29 }
30 n := calls.Add(1)
31 switch {
32 case n == 1:
33 fmt.Fprint(w, `[]`)
34 case n == 2:
35 fmt.Fprint(w, `[{"id":9,"status":"running","ref":"refs/heads/x","sha":"abcdef1000","web_url":"u"}]`)
36 default:
37 fmt.Fprintf(w, `[{"id":9,"status":%q,"ref":"refs/heads/x","sha":"abcdef1000","web_url":"u"}]`, final)
38 }
39 case r.URL.Path == "/api/v1/projects/tim/app/pipelines/9/jobs":
40 fmt.Fprint(w, `[{"id":1,"name":"test","stage":"test","status":"failed","allow_failure":false},{"id":2,"name":"aur","stage":"release","status":"failed","allow_failure":true}]`)
41 case r.URL.Path == "/api/v1/projects/tim/app/pipelines/9/jobs/1/log":
42 fmt.Fprint(w, "regel 1\nFOUT hier\n")
43 default:
44 http.NotFound(w, r)
45 }
46 }))
47 t.Cleanup(srv.Close)
48 return srv, &calls
49}
50
51func runBG(t *testing.T, url, token string, args ...string) (int, string, string) {
52 t.Helper()
53 for _, bin := range []string{"bash", "curl", "jq"} {
54 if _, err := exec.LookPath(bin); err != nil {
55 t.Skip(bin + " ontbreekt")
56 }
57 }
58 cmd := exec.Command(filepath.Join("..", "scripts", "bg"), args...)
59 cmd.Env = append(os.Environ(), "BLUEGIT_URL="+url, "BLUEGIT_TOKEN="+token)
60 var out, errb bytes.Buffer
61 cmd.Stdout, cmd.Stderr = &out, &errb
62 err := cmd.Run()
63 code := 0
64 var ee *exec.ExitError
65 if errors.As(err, &ee) {
66 code = ee.ExitCode()
67 } else if err != nil {
68 t.Fatal(err)
69 }
70 return code, out.String(), errb.String()
71}
72
73func TestBGWaitExitCodes(t *testing.T) {
74 const tok = "bg_geheimtoken123"
75 srv, _ := fakeBlueGit(t, "success", tok)
76 code, out, errs := runBG(t, srv.URL, tok, "pipeline", "wait", "tim/app", "--sha", "abcdef1", "--interval", "0")
77 if code != 0 || !strings.Contains(out, "pipeline #9 success") {
78 t.Fatalf("success: %d %s %s", code, out, errs)
79 }
80 if strings.Contains(out+errs, tok) {
81 t.Fatal("token geprint")
82 }
83 srv, _ = fakeBlueGit(t, "failed", "")
84 code, out, errs = runBG(t, srv.URL, "", "pipeline", "wait", "tim/app", "--sha", "abcdef1", "--interval", "0")
85 if code != 1 || !strings.Contains(errs, "FOUT hier") || !strings.Contains(errs, "waarschuwing: aur") {
86 t.Fatalf("failed: %d %s %s", code, out, errs)
87 }
88 srv, _ = fakeBlueGit(t, "running", "")
89 code, _, errs = runBG(t, srv.URL, "", "pipeline", "wait", "tim/app", "--sha", "abcdef1", "--interval", "1", "--timeout", "2")
90 if code != 2 || !strings.Contains(errs, "timeout") {
91 t.Fatalf("timeout: %d %s", code, errs)
92 }
93 code, _, errs = runBG(t, srv.URL, "", "mr", "show", "tim/app", "5")
94 if code != 1 || !strings.Contains(errs, "404") {
95 t.Fatalf("404: %d %s", code, errs)
96 }
97}