BlueGit

timfa bluegit

Merge branch 'feature/release-existing-tag' into 'stage'

timfa

Authored 2026-11-06 · 10 days ago

Hash
c50a9918c362b2928274ff057a811ddf6c5f1279
Pipeline
Passed
14 files changed +2054 −1
docs/ARCHITECTURE.md +1 −0
@@ -36,6 +36,7 @@ Onder `ProtectSystem=strict` mag het proces alleen in de state-map schrijven, to
36 36 <datadisk>/repos/eigenaar/naam.git
37 37 <datadisk>/artifacts/job-id/…
38 38 <datadisk>/releases/eigenaar/naam/tag/…
39 <datadisk>/releases/.incoming/… ongepubliceerde kopie; geen onderhoudstaak
39 40 <datadisk>/work/pipeline/job/
40 41 ```
41 42
@@ -36,6 +36,7 @@ Onder `ProtectSystem=strict` mag het proces alleen in de state-map schrijven, to
36<datadisk>/repos/eigenaar/naam.git
36<datadisk>/repos/eigenaar/naam.git
37<datadisk>/artifacts/job-id/…
37<datadisk>/artifacts/job-id/…
38<datadisk>/releases/eigenaar/naam/tag/…
38<datadisk>/releases/eigenaar/naam/tag/…
39<datadisk>/releases/.incoming/… ongepubliceerde kopie; geen onderhoudstaak
39<datadisk>/work/pipeline/job/
40<datadisk>/work/pipeline/job/
40```
41```
41
42
docs/CI.md +21 −0
@@ -138,6 +138,27 @@ Een release bestaat maar één keer: dezelfde tag nog eens publiceren geeft `409
138 138
139 139 De assets worden eerst gecontroleerd, pas daarna gaat de oude release weg. Elke vervanging schrijft een regel `release audit` met `action=replace`, repository, tag, oud en nieuw commit, job, pipeline, ref en actor in het serverlog. Zonder `replace` blijft alles zoals het was.
140 140
141 ### Release bij een tag die al bestaat
142
143 Staat de tag al in de repository, dan maakt de job hem niet opnieuw. `POST /api/v1/repos/$CI_PROJECT_PATH/releases/$CI_COMMIT_TAG/assets` publiceert één nieuwe release op die tag. Het verzoek mag geen `replace` bevatten. De server maakt, verplaatst of verwijdert de git-tag niet en start er geen pipeline door.
144
145 ```shell
146 curl -fsS -X POST "$CI_API_URL/repos/$CI_PROJECT_PATH/releases/$CI_COMMIT_TAG/assets" \
147 -H "Authorization: Bearer $CI_JOB_TOKEN" \
148 -H "Content-Type: application/json" \
149 -d '{"commit":"'"$CI_COMMIT_SHA"'","name":"1.2.3","body":"uitgave","assets":[{"job":"build","path":"dist/app","name":"app"}]}'
150 ```
151
152 Dit kan alleen met `CI_JOB_TOKEN` van een job die nog loopt. Een persoonlijk token, een sessie of een anoniem verzoek krijgt `401`. De pipeline moet een push zijn van precies `refs/tags/<tag>` in dezelfde repository, zonder merge request. De actor moet op dat moment maintainer van het project zijn, of instance-admin. Een branch, fork, web-pipeline of een andere tag krijgt `403`.
153
154 `commit` is de volledige SHA. Die moet gelijk zijn aan de pipeline, de job en de gepeelde tagcommit. Lightweight en annotated tags mogen allebei. Een symbolic ref of een tag die niet naar een commit wijst, niet. Een beschermde tag mag wel: de ref zelf blijft onaangeroerd.
155
156 Bestaat er al een release voor de tag, dan is het antwoord `409`, ook als commit, auteur of pipeline hetzelfde zijn. Een geslaagde publicatie nog eens sturen is dus ook `409`. Er is geen idempotente overwrite.
157
158 Assets komen alleen uit geslaagde jobs van dezelfde pipeline. De servernamen van de jobs en bestanden kiest de pipeline; BlueGit hardcodeert geen platformen. Een verlopen artifact, een andere pipeline, een symlink of een directory wordt geweigerd. De bestandsnaam is een basename: geen slash, backslash, `.`, `..` of stuurteken, en geen dubbele naam. Hoogstens 32 assets. De kopieën gaan naar een eigen map onder `releases/.incoming/` en worden in één transactie zichtbaar. Een fout ruimt alleen die map op. Een crash kan een ongepubliceerde map achterlaten; een bestaande release wordt daardoor niet overschreven.
159
160 Tijdens het wegschrijven houdt BlueGit `refs/tags/<tag>` vast met `git update-ref` (`start`, `verify`, `prepare`, daarna `abort`). Een gelijktijdige push kan die ref dan niet verplaatsen. De tag zelf verandert niet. Na een geslaagde databasecommit komt er één `release audit` met `action=create` en één release-event.
161
141 162 ## Wanneer een job start
142 163
143 164 `Gate` in `internal/ci` geeft `ready`, `wait`, `skip` of `fail`. `skip` volgt op een harde mislukking bij `when: on_success`, of op succes bij `when: on_failure`. `when: always` wacht tot de voorgangers klaar zijn en draait dan toch. Een job met `allow_failure` telt niet als harde mislukking. De runner verwerkt pipelines elke twee seconden, met `ci.concurrency` tegelijk (standaard 1). Jobs die al lopen vullen die slots. Een startklare job die nog wacht toont `In wachtrij (#n)` in de pipelines-lijst, de stage-popover en op de jobpagina. Het nummer is de plek in Steves globale wachtrij, over alle repositories; de eerstvolgende wachtende job is `#1`.
@@ -138,6 +138,27 @@ Een release bestaat maar één keer: dezelfde tag nog eens publiceren geeft `409
138
138
139De assets worden eerst gecontroleerd, pas daarna gaat de oude release weg. Elke vervanging schrijft een regel `release audit` met `action=replace`, repository, tag, oud en nieuw commit, job, pipeline, ref en actor in het serverlog. Zonder `replace` blijft alles zoals het was.
139De assets worden eerst gecontroleerd, pas daarna gaat de oude release weg. Elke vervanging schrijft een regel `release audit` met `action=replace`, repository, tag, oud en nieuw commit, job, pipeline, ref en actor in het serverlog. Zonder `replace` blijft alles zoals het was.
140
140
141### Release bij een tag die al bestaat
142
143Staat de tag al in de repository, dan maakt de job hem niet opnieuw. `POST /api/v1/repos/$CI_PROJECT_PATH/releases/$CI_COMMIT_TAG/assets` publiceert één nieuwe release op die tag. Het verzoek mag geen `replace` bevatten. De server maakt, verplaatst of verwijdert de git-tag niet en start er geen pipeline door.
144
145```shell
146curl -fsS -X POST "$CI_API_URL/repos/$CI_PROJECT_PATH/releases/$CI_COMMIT_TAG/assets" \
147 -H "Authorization: Bearer $CI_JOB_TOKEN" \
148 -H "Content-Type: application/json" \
149 -d '{"commit":"'"$CI_COMMIT_SHA"'","name":"1.2.3","body":"uitgave","assets":[{"job":"build","path":"dist/app","name":"app"}]}'
150```
151
152Dit kan alleen met `CI_JOB_TOKEN` van een job die nog loopt. Een persoonlijk token, een sessie of een anoniem verzoek krijgt `401`. De pipeline moet een push zijn van precies `refs/tags/<tag>` in dezelfde repository, zonder merge request. De actor moet op dat moment maintainer van het project zijn, of instance-admin. Een branch, fork, web-pipeline of een andere tag krijgt `403`.
153
154`commit` is de volledige SHA. Die moet gelijk zijn aan de pipeline, de job en de gepeelde tagcommit. Lightweight en annotated tags mogen allebei. Een symbolic ref of een tag die niet naar een commit wijst, niet. Een beschermde tag mag wel: de ref zelf blijft onaangeroerd.
155
156Bestaat er al een release voor de tag, dan is het antwoord `409`, ook als commit, auteur of pipeline hetzelfde zijn. Een geslaagde publicatie nog eens sturen is dus ook `409`. Er is geen idempotente overwrite.
157
158Assets komen alleen uit geslaagde jobs van dezelfde pipeline. De servernamen van de jobs en bestanden kiest de pipeline; BlueGit hardcodeert geen platformen. Een verlopen artifact, een andere pipeline, een symlink of een directory wordt geweigerd. De bestandsnaam is een basename: geen slash, backslash, `.`, `..` of stuurteken, en geen dubbele naam. Hoogstens 32 assets. De kopieën gaan naar een eigen map onder `releases/.incoming/` en worden in één transactie zichtbaar. Een fout ruimt alleen die map op. Een crash kan een ongepubliceerde map achterlaten; een bestaande release wordt daardoor niet overschreven.
159
160Tijdens het wegschrijven houdt BlueGit `refs/tags/<tag>` vast met `git update-ref` (`start`, `verify`, `prepare`, daarna `abort`). Een gelijktijdige push kan die ref dan niet verplaatsen. De tag zelf verandert niet. Na een geslaagde databasecommit komt er één `release audit` met `action=create` en één release-event.
161
141## Wanneer een job start
162## Wanneer een job start
142
163
143`Gate` in `internal/ci` geeft `ready`, `wait`, `skip` of `fail`. `skip` volgt op een harde mislukking bij `when: on_success`, of op succes bij `when: on_failure`. `when: always` wacht tot de voorgangers klaar zijn en draait dan toch. Een job met `allow_failure` telt niet als harde mislukking. De runner verwerkt pipelines elke twee seconden, met `ci.concurrency` tegelijk (standaard 1). Jobs die al lopen vullen die slots. Een startklare job die nog wacht toont `In wachtrij (#n)` in de pipelines-lijst, de stage-popover en op de jobpagina. Het nummer is de plek in Steves globale wachtrij, over alle repositories; de eerstvolgende wachtende job is `#1`.
164`Gate` in `internal/ci` geeft `ready`, `wait`, `skip` of `fail`. `skip` volgt op een harde mislukking bij `when: on_success`, of op succes bij `when: on_failure`. `when: always` wacht tot de voorgangers klaar zijn en draait dan toch. Een job met `allow_failure` telt niet als harde mislukking. De runner verwerkt pipelines elke twee seconden, met `ci.concurrency` tegelijk (standaard 1). Jobs die al lopen vullen die slots. Een startklare job die nog wacht toont `In wachtrij (#n)` in de pipelines-lijst, de stage-popover en op de jobpagina. Het nummer is de plek in Steves globale wachtrij, over alle repositories; de eerstvolgende wachtende job is `#1`.
docs/agents.md +4 −0
@@ -175,6 +175,10 @@ bg release delete timfa/app v1.2.0 --delete-tag # ook de git-tag weg
175 175
176 176 API: `DELETE /api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{naam}` en `DELETE /api/v1/projects/{owner}/{repo}/releases/{tag}[?delete_tag=true]`, beide `204`. Alleen eigenaar, maintainer of instance-admin; zonder token `401`, niet zichtbaar `404`, te weinig rechten `403`, onbekende release of asset `404`. De bestanden gaan ook van schijf. In de UI staan de knoppen op de Releases-pagina, met een bevestiging.
177 177
178 ## Release op een bestaande tag
179
180 `POST /api/v1/repos/{owner}/{repo}/releases/{tag}/assets` publiceert een release wanneer de git-tag al bestaat. Alleen een lopend jobtoken van de push-pipeline `refs/tags/{tag}`. Geen `replace`. De tag blijft byte voor byte dezelfde ref. Een bestaande release geeft `409`. Assets komen uit geslaagde jobs van die pipeline. Zie [CI.md](CI.md#release-bij-een-tag-die-al-bestaat).
181
178 182 ## Zichtbaarheid van releases
179 183
180 184 Releases hebben een eigen zichtbaarheid naast die van de repo: `inherit` (zelfde als de repo, standaard), `private` of `public`. Instellen kan via Instellingen van de repo, of via:
@@ -175,6 +175,10 @@ bg release delete timfa/app v1.2.0 --delete-tag # ook de git-tag weg
175
175
176API: `DELETE /api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{naam}` en `DELETE /api/v1/projects/{owner}/{repo}/releases/{tag}[?delete_tag=true]`, beide `204`. Alleen eigenaar, maintainer of instance-admin; zonder token `401`, niet zichtbaar `404`, te weinig rechten `403`, onbekende release of asset `404`. De bestanden gaan ook van schijf. In de UI staan de knoppen op de Releases-pagina, met een bevestiging.
176API: `DELETE /api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{naam}` en `DELETE /api/v1/projects/{owner}/{repo}/releases/{tag}[?delete_tag=true]`, beide `204`. Alleen eigenaar, maintainer of instance-admin; zonder token `401`, niet zichtbaar `404`, te weinig rechten `403`, onbekende release of asset `404`. De bestanden gaan ook van schijf. In de UI staan de knoppen op de Releases-pagina, met een bevestiging.
177
177
178## Release op een bestaande tag
179
180`POST /api/v1/repos/{owner}/{repo}/releases/{tag}/assets` publiceert een release wanneer de git-tag al bestaat. Alleen een lopend jobtoken van de push-pipeline `refs/tags/{tag}`. Geen `replace`. De tag blijft byte voor byte dezelfde ref. Een bestaande release geeft `409`. Assets komen uit geslaagde jobs van die pipeline. Zie [CI.md](CI.md#release-bij-een-tag-die-al-bestaat).
181
178## Zichtbaarheid van releases
182## Zichtbaarheid van releases
179
183
180Releases hebben een eigen zichtbaarheid naast die van de repo: `inherit` (zelfde als de repo, standaard), `private` of `public`. Instellen kan via Instellingen van de repo, of via:
184Releases hebben een eigen zichtbaarheid naast die van de repo: `inherit` (zelfde als de repo, standaard), `private` of `public`. Instellen kan via Instellingen van de repo, of via:
docs/runner.md +1 −1
@@ -172,7 +172,7 @@ Een job kan zonder eigen inlog bij de eigen repo, ook als die privé is:
172 172 - De runner zet via `GIT_CONFIG_COUNT`/`GIT_CONFIG_KEY_n`/`GIT_CONFIG_VALUE_n` een `http.<projecturl>.extraHeader` met `Authorization: Bearer $CI_JOB_TOKEN`, alleen voor de URL van dit project. Er komt geen bestand in de container, dus het werkt in elk image (ook `bluegit/base`) zolang git ≥ 2.31 is.
173 173 - `git ls-remote origin`, `git fetch`, `git clone $CI_PROJECT_URL.git` werken dus gewoon (scripts zoals `next-version.sh` hoeven niets te veranderen).
174 174 - De API: `GET`-routes onder `/api/v1/…/{owner}/{repo}` van de eigen repo met `Authorization: Bearer $CI_JOB_TOKEN` (of `PRIVATE-TOKEN`). Leesrechten, zoals een lid met de rol *readonly*.
175 - Niet: pushen, schrijven via de API (behalve releases publiceren, zoals voorheen), andere repo's, of iets na het einde van de job: het token vervalt dan.
175 - Niet: pushen, schrijven via de API (behalve een release publiceren: `POST /repos/…/releases` maakt een tag, `POST /repos/…/releases/{tag}/assets` publiceert bij een tag die al bestaat), andere repo's, of iets na het einde van de job: het token vervalt dan.
176 176
177 177 Pipelines hebben zo dezelfde bescherming als de repo: wie de repo niet
178 178 mag lezen, ziet ook de pipelines niet, en het job-token komt niet verder
@@ -172,7 +172,7 @@ Een job kan zonder eigen inlog bij de eigen repo, ook als die privé is:
172- De runner zet via `GIT_CONFIG_COUNT`/`GIT_CONFIG_KEY_n`/`GIT_CONFIG_VALUE_n` een `http.<projecturl>.extraHeader` met `Authorization: Bearer $CI_JOB_TOKEN`, alleen voor de URL van dit project. Er komt geen bestand in de container, dus het werkt in elk image (ook `bluegit/base`) zolang git ≥ 2.31 is.
172- De runner zet via `GIT_CONFIG_COUNT`/`GIT_CONFIG_KEY_n`/`GIT_CONFIG_VALUE_n` een `http.<projecturl>.extraHeader` met `Authorization: Bearer $CI_JOB_TOKEN`, alleen voor de URL van dit project. Er komt geen bestand in de container, dus het werkt in elk image (ook `bluegit/base`) zolang git ≥ 2.31 is.
173- `git ls-remote origin`, `git fetch`, `git clone $CI_PROJECT_URL.git` werken dus gewoon (scripts zoals `next-version.sh` hoeven niets te veranderen).
173- `git ls-remote origin`, `git fetch`, `git clone $CI_PROJECT_URL.git` werken dus gewoon (scripts zoals `next-version.sh` hoeven niets te veranderen).
174- De API: `GET`-routes onder `/api/v1/…/{owner}/{repo}` van de eigen repo met `Authorization: Bearer $CI_JOB_TOKEN` (of `PRIVATE-TOKEN`). Leesrechten, zoals een lid met de rol *readonly*.
174- De API: `GET`-routes onder `/api/v1/…/{owner}/{repo}` van de eigen repo met `Authorization: Bearer $CI_JOB_TOKEN` (of `PRIVATE-TOKEN`). Leesrechten, zoals een lid met de rol *readonly*.
175- Niet: pushen, schrijven via de API (behalve releases publiceren, zoals voorheen), andere repo's, of iets na het einde van de job: het token vervalt dan.
175- Niet: pushen, schrijven via de API (behalve een release publiceren: `POST /repos/…/releases` maakt een tag, `POST /repos/…/releases/{tag}/assets` publiceert bij een tag die al bestaat), andere repo's, of iets na het einde van de job: het token vervalt dan.
176
176
177Pipelines hebben zo dezelfde bescherming als de repo: wie de repo niet
177Pipelines hebben zo dezelfde bescherming als de repo: wie de repo niet
178mag lezen, ziet ook de pipelines niet, en het job-token komt niet verder
178mag lezen, ziet ook de pipelines niet, en het job-token komt niet verder
internal/gitrepo/refhold.go added +365 −0
@@ -0,0 +1,365 @@
1 package gitrepo
2
3 import (
4 "bytes"
5 "context"
6 "errors"
7 "io"
8 "os"
9 "os/exec"
10 "strings"
11 "sync"
12 "syscall"
13 "time"
14
15 "gitlab.com/Timfa/bluegit/internal/names"
16 )
17
18 // Fouten bij het lezen of vasthouden van refs/tags/<naam>.
19 // De tekst bevat geen pad en geen git-stderr; die staat apart in TagRefError.
20 var (
21 ErrTagMissing = errors.New("tag missing")
22 ErrTagSymbolic = errors.New("symbolic tag")
23 ErrTagNotCommit = errors.New("tag is not a commit")
24 ErrTagMoved = errors.New("tag ref changed")
25 ErrTagBusy = errors.New("tag ref busy")
26 ErrTagLock = errors.New("tag ref lock failed")
27 )
28
29 // TagRefError is een lockfout. Detail is alleen voor het serverlog.
30 type TagRefError struct {
31 Kind error
32 Detail string
33 }
34
35 func (e *TagRefError) Error() string {
36 if e == nil || e.Kind == nil {
37 return ""
38 }
39 return e.Kind.Error()
40 }
41
42 func (e *TagRefError) Unwrap() error {
43 if e == nil {
44 return nil
45 }
46 return e.Kind
47 }
48
49 // TagPoint is een directe tagref. ObjectOID is het object in de ref
50 // (commit bij lightweight, tag-object bij annotated). CommitOID is gepeeld.
51 type TagPoint struct {
52 Name string
53 Ref string
54 ObjectOID string
55 CommitOID string
56 }
57
58 // refHoldReplyTimeout begrenst één antwoord van git update-ref en het wachten
59 // op een lock die een andere publicatie vasthoudt. Tests zetten hem lager.
60 var refHoldReplyTimeout = 5 * time.Second
61
62 // ResolveTag leest refs/tags/<name> zonder een ambigue revisie.
63 // Een symbolic ref of een tag die niet naar een commit peelt wordt geweigerd.
64 func ResolveTag(ctx context.Context, gitBin, dir, name string) (TagPoint, error) {
65 if err := names.ValidTag(name); err != nil {
66 return TagPoint{}, err
67 }
68 ref := "refs/tags/" + name
69 if gitBin == "" {
70 gitBin = "git"
71 }
72 sym := exec.CommandContext(ctx, gitBin, "-C", dir, "symbolic-ref", "-q", ref)
73 sym.Env = gitExecEnv()
74 if err := sym.Run(); err == nil {
75 return TagPoint{}, ErrTagSymbolic
76 }
77 if ctx.Err() != nil {
78 return TagPoint{}, ctx.Err()
79 }
80 object, err := gitLine(ctx, gitBin, dir, "rev-parse", "--verify", "--end-of-options", ref)
81 if err != nil {
82 if ctx.Err() != nil {
83 return TagPoint{}, ctx.Err()
84 }
85 return TagPoint{}, ErrTagMissing
86 }
87 object, ok := normalizeOID(object)
88 if !ok {
89 return TagPoint{}, ErrTagMissing
90 }
91 commit, err := gitLine(ctx, gitBin, dir, "rev-parse", "--verify", "--end-of-options", ref+"^{commit}")
92 if err != nil {
93 if ctx.Err() != nil {
94 return TagPoint{}, ctx.Err()
95 }
96 return TagPoint{}, ErrTagNotCommit
97 }
98 commit, ok = normalizeOID(commit)
99 if !ok {
100 return TagPoint{}, ErrTagNotCommit
101 }
102 return TagPoint{Name: name, Ref: ref, ObjectOID: object, CommitOID: commit}, nil
103 }
104
105 // RefHold houdt refs/tags/<naam> vast via git update-ref --stdin.
106 // De transactie verifieert alleen. Release stuurt abort, zodat de tag
107 // niet verandert en git zelf de lock vrijgeeft.
108 type RefHold struct {
109 inner *refProc
110 }
111
112 // HoldTagRef neemt de ref-lock pas als git "prepare: ok" heeft geantwoord.
113 // Een lock die een andere git-proces vasthoudt, wordt begrensd afgewacht.
114 // Een andere OID is ErrTagMoved. Dit proces verwijdert nooit zelf een .lock-bestand.
115 func HoldTagRef(ctx context.Context, gitBin, dir, name, objectOID string) (*RefHold, error) {
116 if err := names.ValidTag(name); err != nil {
117 return nil, err
118 }
119 objectOID, ok := normalizeOID(objectOID)
120 if !ok {
121 return nil, ErrTagLock
122 }
123 if gitBin == "" {
124 gitBin = "git"
125 }
126 deadline := time.Now().Add(refHoldReplyTimeout)
127 for {
128 if err := ctx.Err(); err != nil {
129 return nil, err
130 }
131 hold, err := acquireTagRef(ctx, gitBin, dir, "refs/tags/"+name, objectOID)
132 if err == nil {
133 return hold, nil
134 }
135 if !errors.Is(err, ErrTagBusy) || !time.Now().Before(deadline) {
136 if errors.Is(err, ErrTagBusy) {
137 return nil, &TagRefError{Kind: ErrTagLock, Detail: detailOf(err)}
138 }
139 return nil, err
140 }
141 timer := time.NewTimer(20 * time.Millisecond)
142 select {
143 case <-ctx.Done():
144 timer.Stop()
145 return nil, ctx.Err()
146 case <-timer.C:
147 }
148 }
149 }
150
151 // Release breekt de verify-transactie af. De tag blijft gelijk.
152 // Een tweede aanroep doet niets. Bij een vastgelopen proces gaat er
153 // SIGTERM heen. SIGKILL blijft achterwege: die laat een git-lockfile staan,
154 // en die ruimen we niet zelf op.
155 func (h *RefHold) Release() error {
156 if h == nil || h.inner == nil {
157 return nil
158 }
159 return h.inner.finish(true)
160 }
161
162 type refProc struct {
163 cmd *exec.Cmd
164 stdin io.WriteCloser
165 out *os.File
166 errBuf bytes.Buffer
167 wg sync.WaitGroup
168 finished bool
169 detail string
170 abortOK bool
171 }
172
173 func acquireTagRef(ctx context.Context, gitBin, dir, ref, objectOID string) (*RefHold, error) {
174 if err := ctx.Err(); err != nil {
175 return nil, err
176 }
177 cmd := exec.Command(gitBin, "-C", dir, "update-ref", "--stdin")
178 cmd.Env = gitExecEnv()
179 stdin, err := cmd.StdinPipe()
180 if err != nil {
181 return nil, err
182 }
183 stdout, err := cmd.StdoutPipe()
184 if err != nil {
185 _ = stdin.Close()
186 return nil, err
187 }
188 out, ok := stdout.(*os.File)
189 if !ok {
190 _ = stdin.Close()
191 _ = stdout.Close()
192 return nil, ErrTagLock
193 }
194 stderr, err := cmd.StderrPipe()
195 if err != nil {
196 _ = stdin.Close()
197 _ = stdout.Close()
198 return nil, err
199 }
200 if err := cmd.Start(); err != nil {
201 _ = stdin.Close()
202 _ = stdout.Close()
203 _ = stderr.Close()
204 return nil, err
205 }
206 p := &refProc{cmd: cmd, stdin: stdin, out: out}
207 p.wg.Add(1)
208 go func() {
209 defer p.wg.Done()
210 _, _ = io.Copy(&p.errBuf, stderr)
211 }()
212 fail := func(kind error) (*RefHold, error) {
213 _ = p.finish(false)
214 return nil, &TagRefError{Kind: kind, Detail: strings.TrimSpace(p.detail)}
215 }
216 if err := p.write("option no-deref"); err != nil {
217 return fail(ErrTagLock)
218 }
219 if err := p.write("start"); err != nil {
220 return fail(ErrTagLock)
221 }
222 line, err := p.readLine()
223 if err != nil || line != "start: ok" {
224 return fail(ErrTagLock)
225 }
226 if err := p.write("verify " + ref + " " + objectOID); err != nil {
227 return fail(ErrTagLock)
228 }
229 if err := p.write("prepare"); err != nil {
230 return fail(ErrTagLock)
231 }
232 line, err = p.readLine()
233 if err != nil || line != "prepare: ok" {
234 _ = p.finish(false)
235 return nil, classifyPrepare(strings.TrimSpace(p.detail))
236 }
237 return &RefHold{inner: p}, nil
238 }
239
240 func classifyPrepare(detail string) error {
241 low := strings.ToLower(detail)
242 kind := ErrTagMoved
243 if strings.Contains(low, "file exists") || strings.Contains(low, "unable to create") {
244 kind = ErrTagBusy
245 }
246 return &TagRefError{Kind: kind, Detail: detail}
247 }
248
249 func (p *refProc) write(line string) error {
250 if strings.ContainsAny(line, "\r\n") {
251 return ErrTagLock
252 }
253 _, err := io.WriteString(p.stdin, line+"\n")
254 return err
255 }
256
257 func (p *refProc) readLine() (string, error) {
258 _ = p.out.SetReadDeadline(time.Now().Add(refHoldReplyTimeout))
259 var b strings.Builder
260 tmp := make([]byte, 1)
261 for b.Len() <= 1024 {
262 n, err := p.out.Read(tmp)
263 if n == 1 {
264 if tmp[0] == '\n' {
265 return strings.TrimSpace(b.String()), nil
266 }
267 b.WriteByte(tmp[0])
268 continue
269 }
270 if err != nil {
271 return b.String(), err
272 }
273 }
274 return b.String(), ErrTagLock
275 }
276
277 func (p *refProc) finish(tryAbort bool) error {
278 if p == nil || p.finished {
279 return nil
280 }
281 p.finished = true
282 if tryAbort && p.cmd.Process != nil {
283 _ = p.write("abort")
284 line, err := p.readLine()
285 p.abortOK = err == nil && line == "abort: ok"
286 }
287 _ = p.stdin.Close()
288 err := p.reap()
289 p.detail = strings.TrimSpace(p.errBuf.String())
290 if p.abortOK {
291 return nil
292 }
293 return err
294 }
295
296 func (p *refProc) reap() error {
297 if p.cmd.Process == nil {
298 p.wg.Wait()
299 return nil
300 }
301 done := make(chan error, 1)
302 go func() { done <- p.cmd.Wait() }()
303 select {
304 case err := <-done:
305 p.wg.Wait()
306 return err
307 case <-time.After(refHoldReplyTimeout):
308 _ = p.cmd.Process.Signal(syscall.SIGTERM)
309 }
310 select {
311 case err := <-done:
312 p.wg.Wait()
313 return err
314 case <-time.After(refHoldReplyTimeout):
315 // Niet wg.Wait(): stderr kan open blijven als SIGTERM niets deed.
316 // Geen SIGKILL en geen Remove van *.lock: een stale lock is van git.
317 return ErrTagLock
318 }
319 }
320
321 func detailOf(err error) string {
322 var te *TagRefError
323 if errors.As(err, &te) {
324 return te.Detail
325 }
326 return ""
327 }
328
329 func gitLine(ctx context.Context, gitBin, dir string, args ...string) (string, error) {
330 cmd := exec.CommandContext(ctx, gitBin, append([]string{"-C", dir}, args...)...)
331 cmd.Env = gitExecEnv()
332 var stdout, stderr bytes.Buffer
333 cmd.Stdout = &stdout
334 cmd.Stderr = &stderr
335 if err := cmd.Run(); err != nil {
336 return "", err
337 }
338 return strings.TrimSpace(stdout.String()), nil
339 }
340
341 func gitExecEnv() []string {
342 env := []string{"LC_ALL=C", "GIT_TERMINAL_PROMPT=0"}
343 if path := os.Getenv("PATH"); path != "" {
344 env = append(env, "PATH="+path)
345 } else {
346 env = append(env, "PATH=/usr/bin:/bin")
347 }
348 if home := os.Getenv("HOME"); home != "" {
349 env = append(env, "HOME="+home)
350 }
351 return env
352 }
353
354 func normalizeOID(s string) (string, bool) {
355 if len(s) != 40 && len(s) != 64 {
356 return "", false
357 }
358 s = strings.ToLower(s)
359 for _, c := range s {
360 if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
361 return "", false
362 }
363 }
364 return s, true
365 }
@@ -0,0 +1,365 @@
1package gitrepo
2
3import (
4 "bytes"
5 "context"
6 "errors"
7 "io"
8 "os"
9 "os/exec"
10 "strings"
11 "sync"
12 "syscall"
13 "time"
14
15 "gitlab.com/Timfa/bluegit/internal/names"
16)
17
18// Fouten bij het lezen of vasthouden van refs/tags/<naam>.
19// De tekst bevat geen pad en geen git-stderr; die staat apart in TagRefError.
20var (
21 ErrTagMissing = errors.New("tag missing")
22 ErrTagSymbolic = errors.New("symbolic tag")
23 ErrTagNotCommit = errors.New("tag is not a commit")
24 ErrTagMoved = errors.New("tag ref changed")
25 ErrTagBusy = errors.New("tag ref busy")
26 ErrTagLock = errors.New("tag ref lock failed")
27)
28
29// TagRefError is een lockfout. Detail is alleen voor het serverlog.
30type TagRefError struct {
31 Kind error
32 Detail string
33}
34
35func (e *TagRefError) Error() string {
36 if e == nil || e.Kind == nil {
37 return ""
38 }
39 return e.Kind.Error()
40}
41
42func (e *TagRefError) Unwrap() error {
43 if e == nil {
44 return nil
45 }
46 return e.Kind
47}
48
49// TagPoint is een directe tagref. ObjectOID is het object in de ref
50// (commit bij lightweight, tag-object bij annotated). CommitOID is gepeeld.
51type TagPoint struct {
52 Name string
53 Ref string
54 ObjectOID string
55 CommitOID string
56}
57
58// refHoldReplyTimeout begrenst één antwoord van git update-ref en het wachten
59// op een lock die een andere publicatie vasthoudt. Tests zetten hem lager.
60var refHoldReplyTimeout = 5 * time.Second
61
62// ResolveTag leest refs/tags/<name> zonder een ambigue revisie.
63// Een symbolic ref of een tag die niet naar een commit peelt wordt geweigerd.
64func ResolveTag(ctx context.Context, gitBin, dir, name string) (TagPoint, error) {
65 if err := names.ValidTag(name); err != nil {
66 return TagPoint{}, err
67 }
68 ref := "refs/tags/" + name
69 if gitBin == "" {
70 gitBin = "git"
71 }
72 sym := exec.CommandContext(ctx, gitBin, "-C", dir, "symbolic-ref", "-q", ref)
73 sym.Env = gitExecEnv()
74 if err := sym.Run(); err == nil {
75 return TagPoint{}, ErrTagSymbolic
76 }
77 if ctx.Err() != nil {
78 return TagPoint{}, ctx.Err()
79 }
80 object, err := gitLine(ctx, gitBin, dir, "rev-parse", "--verify", "--end-of-options", ref)
81 if err != nil {
82 if ctx.Err() != nil {
83 return TagPoint{}, ctx.Err()
84 }
85 return TagPoint{}, ErrTagMissing
86 }
87 object, ok := normalizeOID(object)
88 if !ok {
89 return TagPoint{}, ErrTagMissing
90 }
91 commit, err := gitLine(ctx, gitBin, dir, "rev-parse", "--verify", "--end-of-options", ref+"^{commit}")
92 if err != nil {
93 if ctx.Err() != nil {
94 return TagPoint{}, ctx.Err()
95 }
96 return TagPoint{}, ErrTagNotCommit
97 }
98 commit, ok = normalizeOID(commit)
99 if !ok {
100 return TagPoint{}, ErrTagNotCommit
101 }
102 return TagPoint{Name: name, Ref: ref, ObjectOID: object, CommitOID: commit}, nil
103}
104
105// RefHold houdt refs/tags/<naam> vast via git update-ref --stdin.
106// De transactie verifieert alleen. Release stuurt abort, zodat de tag
107// niet verandert en git zelf de lock vrijgeeft.
108type RefHold struct {
109 inner *refProc
110}
111
112// HoldTagRef neemt de ref-lock pas als git "prepare: ok" heeft geantwoord.
113// Een lock die een andere git-proces vasthoudt, wordt begrensd afgewacht.
114// Een andere OID is ErrTagMoved. Dit proces verwijdert nooit zelf een .lock-bestand.
115func HoldTagRef(ctx context.Context, gitBin, dir, name, objectOID string) (*RefHold, error) {
116 if err := names.ValidTag(name); err != nil {
117 return nil, err
118 }
119 objectOID, ok := normalizeOID(objectOID)
120 if !ok {
121 return nil, ErrTagLock
122 }
123 if gitBin == "" {
124 gitBin = "git"
125 }
126 deadline := time.Now().Add(refHoldReplyTimeout)
127 for {
128 if err := ctx.Err(); err != nil {
129 return nil, err
130 }
131 hold, err := acquireTagRef(ctx, gitBin, dir, "refs/tags/"+name, objectOID)
132 if err == nil {
133 return hold, nil
134 }
135 if !errors.Is(err, ErrTagBusy) || !time.Now().Before(deadline) {
136 if errors.Is(err, ErrTagBusy) {
137 return nil, &TagRefError{Kind: ErrTagLock, Detail: detailOf(err)}
138 }
139 return nil, err
140 }
141 timer := time.NewTimer(20 * time.Millisecond)
142 select {
143 case <-ctx.Done():
144 timer.Stop()
145 return nil, ctx.Err()
146 case <-timer.C:
147 }
148 }
149}
150
151// Release breekt de verify-transactie af. De tag blijft gelijk.
152// Een tweede aanroep doet niets. Bij een vastgelopen proces gaat er
153// SIGTERM heen. SIGKILL blijft achterwege: die laat een git-lockfile staan,
154// en die ruimen we niet zelf op.
155func (h *RefHold) Release() error {
156 if h == nil || h.inner == nil {
157 return nil
158 }
159 return h.inner.finish(true)
160}
161
162type refProc struct {
163 cmd *exec.Cmd
164 stdin io.WriteCloser
165 out *os.File
166 errBuf bytes.Buffer
167 wg sync.WaitGroup
168 finished bool
169 detail string
170 abortOK bool
171}
172
173func acquireTagRef(ctx context.Context, gitBin, dir, ref, objectOID string) (*RefHold, error) {
174 if err := ctx.Err(); err != nil {
175 return nil, err
176 }
177 cmd := exec.Command(gitBin, "-C", dir, "update-ref", "--stdin")
178 cmd.Env = gitExecEnv()
179 stdin, err := cmd.StdinPipe()
180 if err != nil {
181 return nil, err
182 }
183 stdout, err := cmd.StdoutPipe()
184 if err != nil {
185 _ = stdin.Close()
186 return nil, err
187 }
188 out, ok := stdout.(*os.File)
189 if !ok {
190 _ = stdin.Close()
191 _ = stdout.Close()
192 return nil, ErrTagLock
193 }
194 stderr, err := cmd.StderrPipe()
195 if err != nil {
196 _ = stdin.Close()
197 _ = stdout.Close()
198 return nil, err
199 }
200 if err := cmd.Start(); err != nil {
201 _ = stdin.Close()
202 _ = stdout.Close()
203 _ = stderr.Close()
204 return nil, err
205 }
206 p := &refProc{cmd: cmd, stdin: stdin, out: out}
207 p.wg.Add(1)
208 go func() {
209 defer p.wg.Done()
210 _, _ = io.Copy(&p.errBuf, stderr)
211 }()
212 fail := func(kind error) (*RefHold, error) {
213 _ = p.finish(false)
214 return nil, &TagRefError{Kind: kind, Detail: strings.TrimSpace(p.detail)}
215 }
216 if err := p.write("option no-deref"); err != nil {
217 return fail(ErrTagLock)
218 }
219 if err := p.write("start"); err != nil {
220 return fail(ErrTagLock)
221 }
222 line, err := p.readLine()
223 if err != nil || line != "start: ok" {
224 return fail(ErrTagLock)
225 }
226 if err := p.write("verify " + ref + " " + objectOID); err != nil {
227 return fail(ErrTagLock)
228 }
229 if err := p.write("prepare"); err != nil {
230 return fail(ErrTagLock)
231 }
232 line, err = p.readLine()
233 if err != nil || line != "prepare: ok" {
234 _ = p.finish(false)
235 return nil, classifyPrepare(strings.TrimSpace(p.detail))
236 }
237 return &RefHold{inner: p}, nil
238}
239
240func classifyPrepare(detail string) error {
241 low := strings.ToLower(detail)
242 kind := ErrTagMoved
243 if strings.Contains(low, "file exists") || strings.Contains(low, "unable to create") {
244 kind = ErrTagBusy
245 }
246 return &TagRefError{Kind: kind, Detail: detail}
247}
248
249func (p *refProc) write(line string) error {
250 if strings.ContainsAny(line, "\r\n") {
251 return ErrTagLock
252 }
253 _, err := io.WriteString(p.stdin, line+"\n")
254 return err
255}
256
257func (p *refProc) readLine() (string, error) {
258 _ = p.out.SetReadDeadline(time.Now().Add(refHoldReplyTimeout))
259 var b strings.Builder
260 tmp := make([]byte, 1)
261 for b.Len() <= 1024 {
262 n, err := p.out.Read(tmp)
263 if n == 1 {
264 if tmp[0] == '\n' {
265 return strings.TrimSpace(b.String()), nil
266 }
267 b.WriteByte(tmp[0])
268 continue
269 }
270 if err != nil {
271 return b.String(), err
272 }
273 }
274 return b.String(), ErrTagLock
275}
276
277func (p *refProc) finish(tryAbort bool) error {
278 if p == nil || p.finished {
279 return nil
280 }
281 p.finished = true
282 if tryAbort && p.cmd.Process != nil {
283 _ = p.write("abort")
284 line, err := p.readLine()
285 p.abortOK = err == nil && line == "abort: ok"
286 }
287 _ = p.stdin.Close()
288 err := p.reap()
289 p.detail = strings.TrimSpace(p.errBuf.String())
290 if p.abortOK {
291 return nil
292 }
293 return err
294}
295
296func (p *refProc) reap() error {
297 if p.cmd.Process == nil {
298 p.wg.Wait()
299 return nil
300 }
301 done := make(chan error, 1)
302 go func() { done <- p.cmd.Wait() }()
303 select {
304 case err := <-done:
305 p.wg.Wait()
306 return err
307 case <-time.After(refHoldReplyTimeout):
308 _ = p.cmd.Process.Signal(syscall.SIGTERM)
309 }
310 select {
311 case err := <-done:
312 p.wg.Wait()
313 return err
314 case <-time.After(refHoldReplyTimeout):
315 // Niet wg.Wait(): stderr kan open blijven als SIGTERM niets deed.
316 // Geen SIGKILL en geen Remove van *.lock: een stale lock is van git.
317 return ErrTagLock
318 }
319}
320
321func detailOf(err error) string {
322 var te *TagRefError
323 if errors.As(err, &te) {
324 return te.Detail
325 }
326 return ""
327}
328
329func gitLine(ctx context.Context, gitBin, dir string, args ...string) (string, error) {
330 cmd := exec.CommandContext(ctx, gitBin, append([]string{"-C", dir}, args...)...)
331 cmd.Env = gitExecEnv()
332 var stdout, stderr bytes.Buffer
333 cmd.Stdout = &stdout
334 cmd.Stderr = &stderr
335 if err := cmd.Run(); err != nil {
336 return "", err
337 }
338 return strings.TrimSpace(stdout.String()), nil
339}
340
341func gitExecEnv() []string {
342 env := []string{"LC_ALL=C", "GIT_TERMINAL_PROMPT=0"}
343 if path := os.Getenv("PATH"); path != "" {
344 env = append(env, "PATH="+path)
345 } else {
346 env = append(env, "PATH=/usr/bin:/bin")
347 }
348 if home := os.Getenv("HOME"); home != "" {
349 env = append(env, "HOME="+home)
350 }
351 return env
352}
353
354func normalizeOID(s string) (string, bool) {
355 if len(s) != 40 && len(s) != 64 {
356 return "", false
357 }
358 s = strings.ToLower(s)
359 for _, c := range s {
360 if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
361 return "", false
362 }
363 }
364 return s, true
365}
internal/gitrepo/refhold_test.go added +266 −0
@@ -0,0 +1,266 @@
1 package gitrepo
2
3 import (
4 "context"
5 "errors"
6 "os"
7 "os/exec"
8 "path/filepath"
9 "strings"
10 "syscall"
11 "testing"
12 "time"
13 )
14
15 func TestResolveAndHoldTagRef(t *testing.T) {
16 if _, err := exec.LookPath("git"); err != nil {
17 t.Skip("git ontbreekt")
18 }
19 ctx := context.Background()
20 bare, work := testRepo(t)
21 sha := gitSHA(t, bare, "HEAD")
22 gitOK(t, bare, "tag", "lite", sha)
23 gitOK(t, bare, "tag", "-a", "ann", sha, "-m", "notitie")
24
25 lite, err := ResolveTag(ctx, "git", bare, "lite")
26 if err != nil {
27 t.Fatal(err)
28 }
29 if lite.ObjectOID != sha || lite.CommitOID != sha || lite.Ref != "refs/tags/lite" {
30 t.Fatalf("lightweight: %+v", lite)
31 }
32 ann, err := ResolveTag(ctx, "git", bare, "ann")
33 if err != nil {
34 t.Fatal(err)
35 }
36 if ann.ObjectOID == ann.CommitOID || ann.CommitOID != sha {
37 t.Fatalf("annotated: %+v", ann)
38 }
39 if _, err := ResolveTag(ctx, "git", bare, "weg"); !errors.Is(err, ErrTagMissing) {
40 t.Fatalf("ontbrekend: %v", err)
41 }
42 gitOK(t, bare, "symbolic-ref", "refs/tags/sym", "refs/heads/master")
43 if _, err := ResolveTag(ctx, "git", bare, "sym"); !errors.Is(err, ErrTagSymbolic) {
44 t.Fatalf("symbolisch: %v", err)
45 }
46 tree := gitSHA(t, bare, "HEAD^{tree}")
47 gitOK(t, bare, "update-ref", "refs/tags/boom", tree)
48 if _, err := ResolveTag(ctx, "git", bare, "boom"); !errors.Is(err, ErrTagNotCommit) {
49 t.Fatalf("boom: %v", err)
50 }
51
52 hold, err := HoldTagRef(ctx, "git", bare, "lite", lite.ObjectOID)
53 if err != nil {
54 t.Fatal(err)
55 }
56 if err := exec.Command("git", "-C", bare, "update-ref", "refs/heads/side", sha).Run(); err != nil {
57 t.Fatalf("andere ref: %v", err)
58 }
59 del := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/lite")
60 if err := del.Run(); err == nil {
61 t.Fatal("delete moest de lock zien")
62 }
63 if got := gitSHA(t, bare, "refs/tags/lite"); got != sha {
64 t.Fatalf("tag veranderde onder de lock: %s", got)
65 }
66 if err := hold.Release(); err != nil {
67 t.Fatal(err)
68 }
69 if _, err := os.Stat(filepath.Join(bare, "refs", "tags", "lite.lock")); !os.IsNotExist(err) {
70 t.Fatalf("lock bleef na abort: %v", err)
71 }
72 if err := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/lite").Run(); err != nil {
73 t.Fatalf("delete na abort: %v", err)
74 }
75 gitOK(t, bare, "tag", "lite", sha)
76
77 moved := commitNew(t, bare, sha)
78 if _, err := HoldTagRef(ctx, "git", bare, "lite", moved); !errors.Is(err, ErrTagMoved) {
79 t.Fatalf("verkeerde oid: %v", err)
80 }
81 if got := gitSHA(t, bare, "refs/tags/lite"); got != sha {
82 t.Fatalf("verify mocht de tag niet schrijven: %s", got)
83 }
84
85 gitOK(t, bare, "update-ref", "-d", "refs/tags/sym")
86 gitOK(t, bare, "update-ref", "-d", "refs/tags/boom")
87 gitOK(t, bare, "pack-refs", "--all")
88 packed, err := os.ReadFile(filepath.Join(bare, "packed-refs"))
89 if err != nil || !strings.Contains(string(packed), "refs/tags/ann") {
90 t.Fatalf("packed-refs: %v %s", err, packed)
91 }
92 hold, err = HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
93 if err != nil {
94 t.Fatal(err)
95 }
96 if err := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/ann").Run(); err == nil {
97 t.Fatal("packed delete moest falen")
98 }
99 if err := hold.Release(); err != nil {
100 t.Fatal(err)
101 }
102 if got := gitSHA(t, bare, "refs/tags/ann"); got != ann.ObjectOID {
103 t.Fatalf("annotated object veranderde: %s", got)
104 }
105
106 // Tweede lock wacht tot de eerste abort, en krijgt hem dan.
107 first, err := HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
108 if err != nil {
109 t.Fatal(err)
110 }
111 gotHold := make(chan error, 1)
112 go func() {
113 second, err := HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
114 if err != nil {
115 gotHold <- err
116 return
117 }
118 gotHold <- second.Release()
119 }()
120 time.Sleep(40 * time.Millisecond)
121 select {
122 case err := <-gotHold:
123 t.Fatalf("tweede lock was te vroeg: %v", err)
124 default:
125 }
126 if err := first.Release(); err != nil {
127 t.Fatal(err)
128 }
129 select {
130 case err := <-gotHold:
131 if err != nil {
132 t.Fatal(err)
133 }
134 case <-time.After(3 * time.Second):
135 t.Fatal("tweede lock kwam niet vrij")
136 }
137
138 // receive-pack kan de ref niet verplaatsen zolang prepare vastzit.
139 gitOK(t, work, "fetch", "origin", "tag", "ann")
140 if err := os.WriteFile(filepath.Join(work, "f"), []byte("b\n"), 0o644); err != nil {
141 t.Fatal(err)
142 }
143 gitOK(t, work, "add", "f")
144 gitOK(t, work, "commit", "-qm", "verder")
145 hold, err = HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
146 if err != nil {
147 t.Fatal(err)
148 }
149 push := exec.Command("git", "-C", work, "push", "origin", ":refs/tags/ann")
150 if err := push.Run(); err == nil {
151 t.Fatal("receive-pack delete moest falen")
152 }
153 if got := gitSHA(t, bare, "refs/tags/ann"); got != ann.ObjectOID {
154 t.Fatalf("tijdens push: %s", got)
155 }
156 force := exec.Command("git", "-C", work, "push", "--force", "origin", "HEAD:refs/tags/ann")
157 if err := force.Run(); err == nil {
158 t.Fatal("receive-pack force moest falen")
159 }
160 if got := gitSHA(t, bare, "refs/tags/ann"); got != ann.ObjectOID {
161 t.Fatalf("na force: %s", got)
162 }
163 if err := hold.Release(); err != nil {
164 t.Fatal(err)
165 }
166 if err := exec.Command("git", "-C", work, "push", "origin", ":refs/tags/ann").Run(); err != nil {
167 t.Fatalf("delete na release: %v", err)
168 }
169 }
170
171 func TestTagRefKillDoesNotDeleteLockfile(t *testing.T) {
172 if _, err := exec.LookPath("git"); err != nil {
173 t.Skip("git ontbreekt")
174 }
175 bare, _ := testRepo(t)
176 sha := gitSHA(t, bare, "HEAD")
177 gitOK(t, bare, "tag", "v1", sha)
178 hold, err := HoldTagRef(context.Background(), "git", bare, "v1", sha)
179 if err != nil {
180 t.Fatal(err)
181 }
182 pid := hold.inner.cmd.Process.Pid
183 _ = syscall.Kill(pid, syscall.SIGKILL)
184 _ = hold.Release()
185 if _, err := os.Stat(filepath.Join(bare, "refs", "tags", "v1.lock")); err != nil {
186 t.Fatalf("SIGKILL laat de lock van git staan, die ruimen we niet zelf: %v", err)
187 }
188 }
189
190 func TestTagRefStuckProcessExits(t *testing.T) {
191 old := refHoldReplyTimeout
192 refHoldReplyTimeout = 200 * time.Millisecond
193 t.Cleanup(func() { refHoldReplyTimeout = old })
194 pidfile := filepath.Join(t.TempDir(), "pid")
195 script := filepath.Join(t.TempDir(), "git")
196 body := "#!/bin/sh\necho $$ > " + quoteSh(pidfile) + "\nexec sleep 30\n"
197 if err := os.WriteFile(script, []byte(body), 0o755); err != nil {
198 t.Fatal(err)
199 }
200 bare := t.TempDir()
201 _, err := HoldTagRef(context.Background(), script, bare, "v1", strings.Repeat("a", 40))
202 if err == nil {
203 t.Fatal("vastgelopen git moest falen")
204 }
205 raw, err := os.ReadFile(pidfile)
206 if err != nil {
207 t.Fatal(err)
208 }
209 pid := strings.TrimSpace(string(raw))
210 if err := exec.Command("ps", "-p", pid).Run(); err == nil {
211 t.Fatalf("proces %s leeft nog", pid)
212 }
213 }
214
215 func testRepo(t *testing.T) (bare, work string) {
216 t.Helper()
217 bare = filepath.Join(t.TempDir(), "repo.git")
218 if err := Init("git", bare, "master", "/bin/true"); err != nil {
219 t.Fatal(err)
220 }
221 work = t.TempDir()
222 gitOK(t, work, "init", "-q", "-b", "master")
223 if err := os.WriteFile(filepath.Join(work, "f"), []byte("a\n"), 0o644); err != nil {
224 t.Fatal(err)
225 }
226 gitOK(t, work, "add", "f")
227 gitOK(t, work, "commit", "-qm", "init")
228 gitOK(t, work, "remote", "add", "origin", bare)
229 gitOK(t, work, "push", "-q", "origin", "master")
230 return bare, work
231 }
232
233 func gitOK(t *testing.T, dir string, args ...string) {
234 t.Helper()
235 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
236 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
237 if out, err := cmd.CombinedOutput(); err != nil {
238 t.Fatalf("git %v: %v %s", args, err, out)
239 }
240 }
241
242 func gitSHA(t *testing.T, dir, rev string) string {
243 t.Helper()
244 cmd := exec.Command("git", "-C", dir, "rev-parse", "--verify", rev)
245 out, err := cmd.Output()
246 if err != nil {
247 t.Fatalf("rev-parse %s: %v", rev, err)
248 }
249 return strings.TrimSpace(string(out))
250 }
251
252 func commitNew(t *testing.T, bare, parent string) string {
253 t.Helper()
254 tree := gitSHA(t, bare, parent+"^{tree}")
255 cmd := exec.Command("git", "-C", bare, "commit-tree", tree, "-p", parent, "-m", "verder")
256 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
257 out, err := cmd.Output()
258 if err != nil {
259 t.Fatal(err)
260 }
261 return strings.TrimSpace(string(out))
262 }
263
264 func quoteSh(s string) string {
265 return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
266 }
@@ -0,0 +1,266 @@
1package gitrepo
2
3import (
4 "context"
5 "errors"
6 "os"
7 "os/exec"
8 "path/filepath"
9 "strings"
10 "syscall"
11 "testing"
12 "time"
13)
14
15func TestResolveAndHoldTagRef(t *testing.T) {
16 if _, err := exec.LookPath("git"); err != nil {
17 t.Skip("git ontbreekt")
18 }
19 ctx := context.Background()
20 bare, work := testRepo(t)
21 sha := gitSHA(t, bare, "HEAD")
22 gitOK(t, bare, "tag", "lite", sha)
23 gitOK(t, bare, "tag", "-a", "ann", sha, "-m", "notitie")
24
25 lite, err := ResolveTag(ctx, "git", bare, "lite")
26 if err != nil {
27 t.Fatal(err)
28 }
29 if lite.ObjectOID != sha || lite.CommitOID != sha || lite.Ref != "refs/tags/lite" {
30 t.Fatalf("lightweight: %+v", lite)
31 }
32 ann, err := ResolveTag(ctx, "git", bare, "ann")
33 if err != nil {
34 t.Fatal(err)
35 }
36 if ann.ObjectOID == ann.CommitOID || ann.CommitOID != sha {
37 t.Fatalf("annotated: %+v", ann)
38 }
39 if _, err := ResolveTag(ctx, "git", bare, "weg"); !errors.Is(err, ErrTagMissing) {
40 t.Fatalf("ontbrekend: %v", err)
41 }
42 gitOK(t, bare, "symbolic-ref", "refs/tags/sym", "refs/heads/master")
43 if _, err := ResolveTag(ctx, "git", bare, "sym"); !errors.Is(err, ErrTagSymbolic) {
44 t.Fatalf("symbolisch: %v", err)
45 }
46 tree := gitSHA(t, bare, "HEAD^{tree}")
47 gitOK(t, bare, "update-ref", "refs/tags/boom", tree)
48 if _, err := ResolveTag(ctx, "git", bare, "boom"); !errors.Is(err, ErrTagNotCommit) {
49 t.Fatalf("boom: %v", err)
50 }
51
52 hold, err := HoldTagRef(ctx, "git", bare, "lite", lite.ObjectOID)
53 if err != nil {
54 t.Fatal(err)
55 }
56 if err := exec.Command("git", "-C", bare, "update-ref", "refs/heads/side", sha).Run(); err != nil {
57 t.Fatalf("andere ref: %v", err)
58 }
59 del := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/lite")
60 if err := del.Run(); err == nil {
61 t.Fatal("delete moest de lock zien")
62 }
63 if got := gitSHA(t, bare, "refs/tags/lite"); got != sha {
64 t.Fatalf("tag veranderde onder de lock: %s", got)
65 }
66 if err := hold.Release(); err != nil {
67 t.Fatal(err)
68 }
69 if _, err := os.Stat(filepath.Join(bare, "refs", "tags", "lite.lock")); !os.IsNotExist(err) {
70 t.Fatalf("lock bleef na abort: %v", err)
71 }
72 if err := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/lite").Run(); err != nil {
73 t.Fatalf("delete na abort: %v", err)
74 }
75 gitOK(t, bare, "tag", "lite", sha)
76
77 moved := commitNew(t, bare, sha)
78 if _, err := HoldTagRef(ctx, "git", bare, "lite", moved); !errors.Is(err, ErrTagMoved) {
79 t.Fatalf("verkeerde oid: %v", err)
80 }
81 if got := gitSHA(t, bare, "refs/tags/lite"); got != sha {
82 t.Fatalf("verify mocht de tag niet schrijven: %s", got)
83 }
84
85 gitOK(t, bare, "update-ref", "-d", "refs/tags/sym")
86 gitOK(t, bare, "update-ref", "-d", "refs/tags/boom")
87 gitOK(t, bare, "pack-refs", "--all")
88 packed, err := os.ReadFile(filepath.Join(bare, "packed-refs"))
89 if err != nil || !strings.Contains(string(packed), "refs/tags/ann") {
90 t.Fatalf("packed-refs: %v %s", err, packed)
91 }
92 hold, err = HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
93 if err != nil {
94 t.Fatal(err)
95 }
96 if err := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/ann").Run(); err == nil {
97 t.Fatal("packed delete moest falen")
98 }
99 if err := hold.Release(); err != nil {
100 t.Fatal(err)
101 }
102 if got := gitSHA(t, bare, "refs/tags/ann"); got != ann.ObjectOID {
103 t.Fatalf("annotated object veranderde: %s", got)
104 }
105
106 // Tweede lock wacht tot de eerste abort, en krijgt hem dan.
107 first, err := HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
108 if err != nil {
109 t.Fatal(err)
110 }
111 gotHold := make(chan error, 1)
112 go func() {
113 second, err := HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
114 if err != nil {
115 gotHold <- err
116 return
117 }
118 gotHold <- second.Release()
119 }()
120 time.Sleep(40 * time.Millisecond)
121 select {
122 case err := <-gotHold:
123 t.Fatalf("tweede lock was te vroeg: %v", err)
124 default:
125 }
126 if err := first.Release(); err != nil {
127 t.Fatal(err)
128 }
129 select {
130 case err := <-gotHold:
131 if err != nil {
132 t.Fatal(err)
133 }
134 case <-time.After(3 * time.Second):
135 t.Fatal("tweede lock kwam niet vrij")
136 }
137
138 // receive-pack kan de ref niet verplaatsen zolang prepare vastzit.
139 gitOK(t, work, "fetch", "origin", "tag", "ann")
140 if err := os.WriteFile(filepath.Join(work, "f"), []byte("b\n"), 0o644); err != nil {
141 t.Fatal(err)
142 }
143 gitOK(t, work, "add", "f")
144 gitOK(t, work, "commit", "-qm", "verder")
145 hold, err = HoldTagRef(ctx, "git", bare, "ann", ann.ObjectOID)
146 if err != nil {
147 t.Fatal(err)
148 }
149 push := exec.Command("git", "-C", work, "push", "origin", ":refs/tags/ann")
150 if err := push.Run(); err == nil {
151 t.Fatal("receive-pack delete moest falen")
152 }
153 if got := gitSHA(t, bare, "refs/tags/ann"); got != ann.ObjectOID {
154 t.Fatalf("tijdens push: %s", got)
155 }
156 force := exec.Command("git", "-C", work, "push", "--force", "origin", "HEAD:refs/tags/ann")
157 if err := force.Run(); err == nil {
158 t.Fatal("receive-pack force moest falen")
159 }
160 if got := gitSHA(t, bare, "refs/tags/ann"); got != ann.ObjectOID {
161 t.Fatalf("na force: %s", got)
162 }
163 if err := hold.Release(); err != nil {
164 t.Fatal(err)
165 }
166 if err := exec.Command("git", "-C", work, "push", "origin", ":refs/tags/ann").Run(); err != nil {
167 t.Fatalf("delete na release: %v", err)
168 }
169}
170
171func TestTagRefKillDoesNotDeleteLockfile(t *testing.T) {
172 if _, err := exec.LookPath("git"); err != nil {
173 t.Skip("git ontbreekt")
174 }
175 bare, _ := testRepo(t)
176 sha := gitSHA(t, bare, "HEAD")
177 gitOK(t, bare, "tag", "v1", sha)
178 hold, err := HoldTagRef(context.Background(), "git", bare, "v1", sha)
179 if err != nil {
180 t.Fatal(err)
181 }
182 pid := hold.inner.cmd.Process.Pid
183 _ = syscall.Kill(pid, syscall.SIGKILL)
184 _ = hold.Release()
185 if _, err := os.Stat(filepath.Join(bare, "refs", "tags", "v1.lock")); err != nil {
186 t.Fatalf("SIGKILL laat de lock van git staan, die ruimen we niet zelf: %v", err)
187 }
188}
189
190func TestTagRefStuckProcessExits(t *testing.T) {
191 old := refHoldReplyTimeout
192 refHoldReplyTimeout = 200 * time.Millisecond
193 t.Cleanup(func() { refHoldReplyTimeout = old })
194 pidfile := filepath.Join(t.TempDir(), "pid")
195 script := filepath.Join(t.TempDir(), "git")
196 body := "#!/bin/sh\necho $$ > " + quoteSh(pidfile) + "\nexec sleep 30\n"
197 if err := os.WriteFile(script, []byte(body), 0o755); err != nil {
198 t.Fatal(err)
199 }
200 bare := t.TempDir()
201 _, err := HoldTagRef(context.Background(), script, bare, "v1", strings.Repeat("a", 40))
202 if err == nil {
203 t.Fatal("vastgelopen git moest falen")
204 }
205 raw, err := os.ReadFile(pidfile)
206 if err != nil {
207 t.Fatal(err)
208 }
209 pid := strings.TrimSpace(string(raw))
210 if err := exec.Command("ps", "-p", pid).Run(); err == nil {
211 t.Fatalf("proces %s leeft nog", pid)
212 }
213}
214
215func testRepo(t *testing.T) (bare, work string) {
216 t.Helper()
217 bare = filepath.Join(t.TempDir(), "repo.git")
218 if err := Init("git", bare, "master", "/bin/true"); err != nil {
219 t.Fatal(err)
220 }
221 work = t.TempDir()
222 gitOK(t, work, "init", "-q", "-b", "master")
223 if err := os.WriteFile(filepath.Join(work, "f"), []byte("a\n"), 0o644); err != nil {
224 t.Fatal(err)
225 }
226 gitOK(t, work, "add", "f")
227 gitOK(t, work, "commit", "-qm", "init")
228 gitOK(t, work, "remote", "add", "origin", bare)
229 gitOK(t, work, "push", "-q", "origin", "master")
230 return bare, work
231}
232
233func gitOK(t *testing.T, dir string, args ...string) {
234 t.Helper()
235 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
236 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
237 if out, err := cmd.CombinedOutput(); err != nil {
238 t.Fatalf("git %v: %v %s", args, err, out)
239 }
240}
241
242func gitSHA(t *testing.T, dir, rev string) string {
243 t.Helper()
244 cmd := exec.Command("git", "-C", dir, "rev-parse", "--verify", rev)
245 out, err := cmd.Output()
246 if err != nil {
247 t.Fatalf("rev-parse %s: %v", rev, err)
248 }
249 return strings.TrimSpace(string(out))
250}
251
252func commitNew(t *testing.T, bare, parent string) string {
253 t.Helper()
254 tree := gitSHA(t, bare, parent+"^{tree}")
255 cmd := exec.Command("git", "-C", bare, "commit-tree", tree, "-p", parent, "-m", "verder")
256 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
257 out, err := cmd.Output()
258 if err != nil {
259 t.Fatal(err)
260 }
261 return strings.TrimSpace(string(out))
262}
263
264func quoteSh(s string) string {
265 return "'" + strings.ReplaceAll(s, "'", `'\''`) + "'"
266}
internal/i18n/catalog.go +13 −0
@@ -536,6 +536,19 @@ var en = map[string]string{
536 536 "err.assetDisk": "artifact is missing on disk",
537 537 "err.assetCopy": "could not copy the asset",
538 538 "err.tagFail": "tag: %s",
539 "err.releaseRef": "the pipeline must be a push of this tag",
540 "err.releaseActor": "the pipeline actor must be a maintainer",
541 "err.releaseCommit": "the commit does not match the tag",
542 "err.releaseReplace": "replace is not part of this request",
543 "err.releaseTag": "this tag cannot be published",
544 "err.releaseExists": "a release for %s already exists",
545 "err.releaseMoved": "the tag changed",
546 "err.releaseFail": "could not publish the release",
547 "err.releaseMany": "too many assets",
548 "err.releaseLong": "the release text is too long",
549 "err.artifactExpired": "artifact expired: %s %s",
550 "err.assetFile": "artifact is not a regular file",
551 "err.commitFull": "commit must be the full id",
539 552 "err.json": "invalid json",
540 553 "commit.pushedBy": "pushed by %s",
541 554
@@ -536,6 +536,19 @@ var en = map[string]string{
536 "err.assetDisk": "artifact is missing on disk",
536 "err.assetDisk": "artifact is missing on disk",
537 "err.assetCopy": "could not copy the asset",
537 "err.assetCopy": "could not copy the asset",
538 "err.tagFail": "tag: %s",
538 "err.tagFail": "tag: %s",
539 "err.releaseRef": "the pipeline must be a push of this tag",
540 "err.releaseActor": "the pipeline actor must be a maintainer",
541 "err.releaseCommit": "the commit does not match the tag",
542 "err.releaseReplace": "replace is not part of this request",
543 "err.releaseTag": "this tag cannot be published",
544 "err.releaseExists": "a release for %s already exists",
545 "err.releaseMoved": "the tag changed",
546 "err.releaseFail": "could not publish the release",
547 "err.releaseMany": "too many assets",
548 "err.releaseLong": "the release text is too long",
549 "err.artifactExpired": "artifact expired: %s %s",
550 "err.assetFile": "artifact is not a regular file",
551 "err.commitFull": "commit must be the full id",
539 "err.json": "invalid json",
552 "err.json": "invalid json",
540 "commit.pushedBy": "pushed by %s",
553 "commit.pushedBy": "pushed by %s",
541
554
internal/i18n/catalog_lang.go +39 −0
@@ -532,6 +532,19 @@ var nl = map[string]string{
532 532 "err.assetDisk": "artifact ontbreekt op schijf",
533 533 "err.assetCopy": "asset kopiëren",
534 534 "err.tagFail": "tag: %s",
535 "err.releaseRef": "de pipeline moet een push van deze tag zijn",
536 "err.releaseActor": "de actor van de pipeline moet beheerder zijn",
537 "err.releaseCommit": "het commit hoort niet bij de tag",
538 "err.releaseReplace": "replace hoort niet bij dit verzoek",
539 "err.releaseTag": "deze tag kan niet gepubliceerd worden",
540 "err.releaseExists": "er is al een release voor %s",
541 "err.releaseMoved": "de tag is gewijzigd",
542 "err.releaseFail": "de release kon niet gepubliceerd worden",
543 "err.releaseMany": "te veel assets",
544 "err.releaseLong": "de releasetekst is te lang",
545 "err.artifactExpired": "artifact verlopen: %s %s",
546 "err.assetFile": "artifact is geen gewoon bestand",
547 "err.commitFull": "commit moet de volledige id zijn",
535 548 "err.json": "json",
536 549 "commit.pushedBy": "gepusht door %s",
537 550
@@ -1182,6 +1195,19 @@ var de = map[string]string{
1182 1195 "err.assetDisk": "Artefakt fehlt auf der Festplatte",
1183 1196 "err.assetCopy": "Asset konnte nicht kopiert werden",
1184 1197 "err.tagFail": "Tag: %s",
1198 "err.releaseRef": "die Pipeline muss ein Push dieses Tags sein",
1199 "err.releaseActor": "der Auslöser der Pipeline muss ein Maintainer sein",
1200 "err.releaseCommit": "der Commit passt nicht zum Tag",
1201 "err.releaseReplace": "replace gehört nicht zu dieser Anfrage",
1202 "err.releaseTag": "dieser Tag kann nicht veröffentlicht werden",
1203 "err.releaseExists": "eine Release für %s existiert bereits",
1204 "err.releaseMoved": "der Tag hat sich geändert",
1205 "err.releaseFail": "die Release konnte nicht veröffentlicht werden",
1206 "err.releaseMany": "zu viele Assets",
1207 "err.releaseLong": "der Releasetext ist zu lang",
1208 "err.artifactExpired": "Artefakt abgelaufen: %s %s",
1209 "err.assetFile": "Artefakt ist keine reguläre Datei",
1210 "err.commitFull": "der Commit muss die volle ID sein",
1185 1211 "err.json": "json",
1186 1212 "commit.pushedBy": "gepusht von %s",
1187 1213
@@ -1832,6 +1858,19 @@ var fr = map[string]string{
1832 1858 "err.assetDisk": "artefact absent du disque",
1833 1859 "err.assetCopy": "impossible de copier l’artefact",
1834 1860 "err.tagFail": "tag : %s",
1861 "err.releaseRef": "le pipeline doit être un push de ce tag",
1862 "err.releaseActor": "l’acteur du pipeline doit être mainteneur",
1863 "err.releaseCommit": "le commit ne correspond pas au tag",
1864 "err.releaseReplace": "replace n’est pas accepté ici",
1865 "err.releaseTag": "ce tag ne peut pas être publié",
1866 "err.releaseExists": "une release existe déjà pour %s",
1867 "err.releaseMoved": "le tag a changé",
1868 "err.releaseFail": "la release n’a pas pu être publiée",
1869 "err.releaseMany": "trop d’artefacts",
1870 "err.releaseLong": "le texte de la release est trop long",
1871 "err.artifactExpired": "artefact expiré : %s %s",
1872 "err.assetFile": "l’artefact n’est pas un fichier ordinaire",
1873 "err.commitFull": "le commit doit être l’identifiant complet",
1835 1874 "err.json": "json",
1836 1875 "commit.pushedBy": "poussé par %s",
1837 1876
@@ -532,6 +532,19 @@ var nl = map[string]string{
532 "err.assetDisk": "artifact ontbreekt op schijf",
532 "err.assetDisk": "artifact ontbreekt op schijf",
533 "err.assetCopy": "asset kopiëren",
533 "err.assetCopy": "asset kopiëren",
534 "err.tagFail": "tag: %s",
534 "err.tagFail": "tag: %s",
535 "err.releaseRef": "de pipeline moet een push van deze tag zijn",
536 "err.releaseActor": "de actor van de pipeline moet beheerder zijn",
537 "err.releaseCommit": "het commit hoort niet bij de tag",
538 "err.releaseReplace": "replace hoort niet bij dit verzoek",
539 "err.releaseTag": "deze tag kan niet gepubliceerd worden",
540 "err.releaseExists": "er is al een release voor %s",
541 "err.releaseMoved": "de tag is gewijzigd",
542 "err.releaseFail": "de release kon niet gepubliceerd worden",
543 "err.releaseMany": "te veel assets",
544 "err.releaseLong": "de releasetekst is te lang",
545 "err.artifactExpired": "artifact verlopen: %s %s",
546 "err.assetFile": "artifact is geen gewoon bestand",
547 "err.commitFull": "commit moet de volledige id zijn",
535 "err.json": "json",
548 "err.json": "json",
536 "commit.pushedBy": "gepusht door %s",
549 "commit.pushedBy": "gepusht door %s",
537
550
@@ -1182,6 +1195,19 @@ var de = map[string]string{
1182 "err.assetDisk": "Artefakt fehlt auf der Festplatte",
1195 "err.assetDisk": "Artefakt fehlt auf der Festplatte",
1183 "err.assetCopy": "Asset konnte nicht kopiert werden",
1196 "err.assetCopy": "Asset konnte nicht kopiert werden",
1184 "err.tagFail": "Tag: %s",
1197 "err.tagFail": "Tag: %s",
1198 "err.releaseRef": "die Pipeline muss ein Push dieses Tags sein",
1199 "err.releaseActor": "der Auslöser der Pipeline muss ein Maintainer sein",
1200 "err.releaseCommit": "der Commit passt nicht zum Tag",
1201 "err.releaseReplace": "replace gehört nicht zu dieser Anfrage",
1202 "err.releaseTag": "dieser Tag kann nicht veröffentlicht werden",
1203 "err.releaseExists": "eine Release für %s existiert bereits",
1204 "err.releaseMoved": "der Tag hat sich geändert",
1205 "err.releaseFail": "die Release konnte nicht veröffentlicht werden",
1206 "err.releaseMany": "zu viele Assets",
1207 "err.releaseLong": "der Releasetext ist zu lang",
1208 "err.artifactExpired": "Artefakt abgelaufen: %s %s",
1209 "err.assetFile": "Artefakt ist keine reguläre Datei",
1210 "err.commitFull": "der Commit muss die volle ID sein",
1185 "err.json": "json",
1211 "err.json": "json",
1186 "commit.pushedBy": "gepusht von %s",
1212 "commit.pushedBy": "gepusht von %s",
1187
1213
@@ -1832,6 +1858,19 @@ var fr = map[string]string{
1832 "err.assetDisk": "artefact absent du disque",
1858 "err.assetDisk": "artefact absent du disque",
1833 "err.assetCopy": "impossible de copier l’artefact",
1859 "err.assetCopy": "impossible de copier l’artefact",
1834 "err.tagFail": "tag : %s",
1860 "err.tagFail": "tag : %s",
1861 "err.releaseRef": "le pipeline doit être un push de ce tag",
1862 "err.releaseActor": "l’acteur du pipeline doit être mainteneur",
1863 "err.releaseCommit": "le commit ne correspond pas au tag",
1864 "err.releaseReplace": "replace n’est pas accepté ici",
1865 "err.releaseTag": "ce tag ne peut pas être publié",
1866 "err.releaseExists": "une release existe déjà pour %s",
1867 "err.releaseMoved": "le tag a changé",
1868 "err.releaseFail": "la release n’a pas pu être publiée",
1869 "err.releaseMany": "trop d’artefacts",
1870 "err.releaseLong": "le texte de la release est trop long",
1871 "err.artifactExpired": "artefact expiré : %s %s",
1872 "err.assetFile": "l’artefact n’est pas un fichier ordinaire",
1873 "err.commitFull": "le commit doit être l’identifiant complet",
1835 "err.json": "json",
1874 "err.json": "json",
1836 "commit.pushedBy": "poussé par %s",
1875 "commit.pushedBy": "poussé par %s",
1837
1876
internal/server/api_routes.go +1 −0
@@ -177,6 +177,7 @@ var apiCatalog = []apiRoute{
177 177 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/releases/{tag}", access.ScopeWrite, authUser, "Delete release (assets included). Query delete_tag=true also deletes the git tag.", (*Server).apiDeleteRelease},
178 178 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeWrite, authUser, "Delete one release asset (tag and source stay)", (*Server).apiDeleteReleaseAsset},
179 179 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases", access.ScopeWrite, authRelease, "Publish release from a job or user token. A job token may send replace=true to replace its own release: running job of this project, pipeline on an unprotected branch (not forks/*), actor maintainer or higher, existing release made by a job on the same branch. Audited.", (*Server).createRelease},
180 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases/{tag}/assets", access.ScopeWrite, authRelease, "Publish a new release for a git tag that already exists. Job token only: the running job must belong to this repository and to a push pipeline of refs/tags/{tag}. Maintainer or instance admin. Create-only, so an existing release is 409. Does not create, move, or delete the tag. Assets come from successful jobs of that same pipeline.", (*Server).publishExistingTag},
180 181
181 182 {http.MethodGet, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Read instance storage", (*Server).apiAdminStorage},
182 183 {http.MethodPut, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Set instance storage", (*Server).apiAdminStorage},
@@ -177,6 +177,7 @@ var apiCatalog = []apiRoute{
177 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/releases/{tag}", access.ScopeWrite, authUser, "Delete release (assets included). Query delete_tag=true also deletes the git tag.", (*Server).apiDeleteRelease},
177 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/releases/{tag}", access.ScopeWrite, authUser, "Delete release (assets included). Query delete_tag=true also deletes the git tag.", (*Server).apiDeleteRelease},
178 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeWrite, authUser, "Delete one release asset (tag and source stay)", (*Server).apiDeleteReleaseAsset},
178 {http.MethodDelete, "/api/v1/projects/{owner}/{repo}/releases/{tag}/assets/{name}", access.ScopeWrite, authUser, "Delete one release asset (tag and source stay)", (*Server).apiDeleteReleaseAsset},
179 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases", access.ScopeWrite, authRelease, "Publish release from a job or user token. A job token may send replace=true to replace its own release: running job of this project, pipeline on an unprotected branch (not forks/*), actor maintainer or higher, existing release made by a job on the same branch. Audited.", (*Server).createRelease},
179 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases", access.ScopeWrite, authRelease, "Publish release from a job or user token. A job token may send replace=true to replace its own release: running job of this project, pipeline on an unprotected branch (not forks/*), actor maintainer or higher, existing release made by a job on the same branch. Audited.", (*Server).createRelease},
180 {http.MethodPost, "/api/v1/repos/{owner}/{repo}/releases/{tag}/assets", access.ScopeWrite, authRelease, "Publish a new release for a git tag that already exists. Job token only: the running job must belong to this repository and to a push pipeline of refs/tags/{tag}. Maintainer or instance admin. Create-only, so an existing release is 409. Does not create, move, or delete the tag. Assets come from successful jobs of that same pipeline.", (*Server).publishExistingTag},
180
181
181 {http.MethodGet, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Read instance storage", (*Server).apiAdminStorage},
182 {http.MethodGet, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Read instance storage", (*Server).apiAdminStorage},
182 {http.MethodPut, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Set instance storage", (*Server).apiAdminStorage},
183 {http.MethodPut, "/api/v1/admin/storage", access.ScopeAdmin, authUser, "Set instance storage", (*Server).apiAdminStorage},
internal/server/api_test.go +1 −0
@@ -73,6 +73,7 @@ func TestAPICatalogCoversResources(t *testing.T) {
73 73 "GET /api/v1/ci/presets\n",
74 74 "POST /api/v1/projects/{owner}/{repo}/releases\n",
75 75 "POST /api/v1/repos/{owner}/{repo}/releases\n",
76 "POST /api/v1/repos/{owner}/{repo}/releases/{tag}/assets\n",
76 77 "PUT /api/v1/admin/storage\n",
77 78 } {
78 79 if !strings.Contains(blob, want) {
@@ -73,6 +73,7 @@ func TestAPICatalogCoversResources(t *testing.T) {
73 "GET /api/v1/ci/presets\n",
73 "GET /api/v1/ci/presets\n",
74 "POST /api/v1/projects/{owner}/{repo}/releases\n",
74 "POST /api/v1/projects/{owner}/{repo}/releases\n",
75 "POST /api/v1/repos/{owner}/{repo}/releases\n",
75 "POST /api/v1/repos/{owner}/{repo}/releases\n",
76 "POST /api/v1/repos/{owner}/{repo}/releases/{tag}/assets\n",
76 "PUT /api/v1/admin/storage\n",
77 "PUT /api/v1/admin/storage\n",
77 } {
78 } {
78 if !strings.Contains(blob, want) {
79 if !strings.Contains(blob, want) {
internal/server/release_existing_tag.go added +428 −0
@@ -0,0 +1,428 @@
1 package server
2
3 import (
4 "context"
5 "encoding/json"
6 "errors"
7 "io"
8 "net/http"
9 "os"
10 "path/filepath"
11 "strings"
12
13 "gitlab.com/Timfa/bluegit/internal/access"
14 "gitlab.com/Timfa/bluegit/internal/gitrepo"
15 "gitlab.com/Timfa/bluegit/internal/i18n"
16 "gitlab.com/Timfa/bluegit/internal/names"
17 "gitlab.com/Timfa/bluegit/internal/store"
18 )
19
20 const (
21 maxExistingReleaseAssets = 32
22 maxExistingReleaseName = 200
23 maxExistingReleaseBody = 64 << 10
24 maxArtifactKeyLen = 512
25 )
26
27 // existingTagBeforeLock draait na de eerste tagresolutie en vóór prepare.
28 // Tests verplaatsen de tag hier om de race vóór de lock te bewijzen.
29 var existingTagBeforeLock = func(ctx context.Context) error { return ctx.Err() }
30
31 // existingTagDuringLock draait terwijl git de tagref vasthoudt, vóór de insert.
32 var existingTagDuringLock = func(ctx context.Context) error { return ctx.Err() }
33
34 // releaseCopyFile kopieert één artifact. Tests laten een latere kopie falen.
35 var releaseCopyFile = copyReleaseExclusive
36
37 // releaseInsert schrijft de release. Tests injecteren een storefout.
38 var releaseInsert = func(ctx context.Context, st *store.Store, repoID int64, tag, sha, name, body string, authorID, jobID int64, assets []store.ReleaseAsset) (int64, error) {
39 return st.CreateRelease(ctx, repoID, tag, sha, name, body, authorID, jobID, assets)
40 }
41
42 type existingReleaseRequest struct {
43 Commit string `json:"commit"`
44 Name string `json:"name"`
45 Body string `json:"body"`
46 Replace json.RawMessage `json:"replace"`
47 Tag *string `json:"tag"`
48 Assets []struct {
49 Job string `json:"job"`
50 Path string `json:"path"`
51 Name string `json:"name"`
52 } `json:"assets"`
53 }
54
55 type stagedAsset struct {
56 name string
57 src string
58 size int64
59 }
60
61 // publishExistingTag publiceert een nieuwe release bij een tag die al in git staat.
62 // Alleen een lopend jobtoken van een push naar refs/tags/<tag>. De tag wordt niet
63 // aangemaakt, verplaatst of verwijderd. Een bestaande release blijft staan (409).
64 func (s *Server) publishExistingTag(w http.ResponseWriter, r *http.Request) {
65 repo, err := s.Store.RepoByPath(r.Context(), r.PathValue("owner"), r.PathValue("repo"))
66 if err != nil {
67 s.writeAPIErr(w, r, fail(404, i18n.E("err.notFound")))
68 return
69 }
70 token := tokenFrom(r)
71 if !strings.HasPrefix(token, "bgj_") {
72 s.writeAPIErr(w, r, fail(401, i18n.E("err.jobToken")))
73 return
74 }
75 var req existingReleaseRequest
76 if err := readJSON(r, &req); err != nil {
77 s.writeAPIErr(w, r, fail(400, i18n.E("err.json")))
78 return
79 }
80 id, commit, err := s.opPublishExistingTag(r.Context(), repo, token, r.PathValue("tag"), req)
81 if !s.apiOK(w, r, err) {
82 return
83 }
84 writeJSON(w, 201, map[string]any{"id": id, "tag": r.PathValue("tag"), "commit": commit})
85 }
86
87 func (s *Server) opPublishExistingTag(ctx context.Context, repo store.Repo, token, tag string, req existingReleaseRequest) (int64, string, error) {
88 if s.StoragePath() == "" {
89 return 0, "", fail(503, i18n.E("err.noStorage"))
90 }
91 if _, err := s.runningReleaseJob(ctx, repo, token); err != nil {
92 return 0, "", err
93 }
94 if len(req.Replace) > 0 {
95 return 0, "", fail(400, i18n.E("err.releaseReplace"))
96 }
97 if req.Tag != nil && *req.Tag != tag {
98 return 0, "", fail(400, i18n.E("err.releaseTag"))
99 }
100 if err := names.ValidTag(tag); err != nil {
101 return 0, "", fail(400, err)
102 }
103 if len(req.Name) > maxExistingReleaseName || len(req.Body) > maxExistingReleaseBody {
104 return 0, "", fail(400, i18n.E("err.releaseLong"))
105 }
106 if len(req.Assets) > maxExistingReleaseAssets {
107 return 0, "", fail(400, i18n.E("err.releaseMany"))
108 }
109 seen := map[string]struct{}{}
110 type named struct{ job, path, name string }
111 var wanted []named
112 for _, a := range req.Assets {
113 if !validArtifactKey(a.Path) || strings.TrimSpace(a.Job) == "" {
114 return 0, "", fail(400, i18n.E("err.assetName"))
115 }
116 name, err := releaseAssetName(a.Name, a.Path)
117 if err != nil {
118 return 0, "", fail(400, i18n.E("err.assetName"))
119 }
120 if _, dup := seen[name]; dup {
121 return 0, "", fail(400, i18n.E("err.assetName"))
122 }
123 seen[name] = struct{}{}
124 wanted = append(wanted, named{a.Job, a.Path, name})
125 }
126 auth, err := s.authorizeExistingTag(ctx, repo, token, tag, req.Commit)
127 if err != nil {
128 return 0, "", err
129 }
130 if _, err := s.Store.ReleaseByTag(ctx, repo.ID, tag); err == nil {
131 return 0, "", fail(409, i18n.E("err.releaseExists", tag))
132 } else if !store.IsNotFound(err) {
133 return 0, "", fail(500, i18n.E("err.releaseFail"))
134 }
135 var staged []stagedAsset
136 for _, a := range wanted {
137 src, err := s.Store.ArtifactForPublish(ctx, auth.job.PipelineID, a.job, a.path)
138 if errors.Is(err, store.ErrArtifactExpired) {
139 return 0, "", fail(400, i18n.E("err.artifactExpired", a.job, a.path))
140 }
141 if err != nil {
142 return 0, "", fail(400, i18n.E("err.artifactMissing", a.job, a.path))
143 }
144 info, err := os.Lstat(src)
145 if err != nil {
146 return 0, "", fail(400, i18n.E("err.assetDisk"))
147 }
148 if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() {
149 return 0, "", fail(400, i18n.E("err.assetFile"))
150 }
151 if !pathInside(s.StoragePath(), src) {
152 return 0, "", fail(400, i18n.E("err.assetFile"))
153 }
154 staged = append(staged, stagedAsset{name: a.name, src: src, size: info.Size()})
155 }
156 parent := filepath.Join(s.StoragePath(), "releases", ".incoming")
157 if err := os.MkdirAll(parent, 0o755); err != nil {
158 return 0, "", fail(500, i18n.E("err.releaseFail"))
159 }
160 stage, err := os.MkdirTemp(parent, "pub-")
161 if err != nil {
162 return 0, "", fail(500, i18n.E("err.releaseFail"))
163 }
164 published := false
165 defer func() {
166 if !published {
167 _ = os.RemoveAll(stage)
168 }
169 }()
170 var assets []store.ReleaseAsset
171 for _, sa := range staged {
172 dst := filepath.Join(stage, sa.name)
173 if !pathInside(stage, dst) {
174 return 0, "", fail(400, i18n.E("err.assetName"))
175 }
176 if err := releaseCopyFile(sa.src, dst); err != nil {
177 if s.Log != nil {
178 s.Log.Error("release kopie", "err", err, "repo", repo.FullPath(), "tag", tag)
179 }
180 return 0, "", fail(500, i18n.E("err.assetCopy"))
181 }
182 info, err := os.Lstat(dst)
183 if err != nil || !info.Mode().IsRegular() || info.Size() != sa.size {
184 return 0, "", fail(500, i18n.E("err.assetCopy"))
185 }
186 assets = append(assets, store.ReleaseAsset{Name: sa.name, StoragePath: dst, Size: info.Size()})
187 }
188 if err := syncDir(stage); err != nil {
189 return 0, "", fail(500, i18n.E("err.assetCopy"))
190 }
191 if err := existingTagBeforeLock(ctx); err != nil {
192 return 0, "", s.wrapPublishErr(err)
193 }
194 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
195 hold, err := gitrepo.HoldTagRef(ctx, s.Git, dir, tag, auth.point.ObjectOID)
196 if err != nil {
197 return 0, "", s.mapTagErr(err)
198 }
199 defer func() { _ = hold.Release() }()
200 if err := existingTagDuringLock(ctx); err != nil {
201 return 0, "", s.wrapPublishErr(err)
202 }
203 again, err := s.authorizeExistingTag(ctx, repo, token, tag, req.Commit)
204 if err != nil {
205 return 0, "", err
206 }
207 if again.point.ObjectOID != auth.point.ObjectOID || again.point.CommitOID != auth.point.CommitOID {
208 return 0, "", fail(409, i18n.E("err.releaseMoved"))
209 }
210 if _, err := s.Store.ReleaseByTag(ctx, repo.ID, tag); err == nil {
211 return 0, "", fail(409, i18n.E("err.releaseExists", tag))
212 } else if !store.IsNotFound(err) {
213 return 0, "", fail(500, i18n.E("err.releaseFail"))
214 }
215 id, err := releaseInsert(ctx, s.Store, repo.ID, tag, again.point.CommitOID, displayName(req.Name, tag), req.Body, 0, again.job.ID, assets)
216 if err != nil {
217 var ie i18n.Error
218 if errors.As(err, &ie) && ie.Key == "err.tagExists" {
219 return 0, "", fail(409, i18n.E("err.releaseExists", tag))
220 }
221 if s.Log != nil {
222 s.Log.Error("release insert", "err", err, "repo", repo.FullPath(), "tag", tag)
223 }
224 return 0, "", fail(500, i18n.E("err.releaseFail"))
225 }
226 published = true
227 // De lock moet los vóór het event: een webhook mag de ref niet vasthouden.
228 // Lukt abort niet, dan blijft de gecommitte release staan. Geen tweede insert.
229 if relErr := hold.Release(); relErr != nil && s.Log != nil {
230 s.Log.Error("release ref lock", "err", relErr, "repo", repo.FullPath(), "tag", tag)
231 }
232 if s.Log != nil {
233 s.Log.Info("release audit", "action", "create", "repo", repo.FullPath(), "tag", tag,
234 "commit", again.point.CommitOID, "job", again.job.ID, "pipeline", again.job.PipelineID, "actor", again.pipe.Actor)
235 }
236 s.Dispatch(repo.ID, s.releaseEvent(repo, tag, displayName(req.Name, tag), again.point.CommitOID, assets))
237 return id, again.point.CommitOID, nil
238 }
239
240 type existingTagAuth struct {
241 job store.Job
242 pipe store.Pipeline
243 point gitrepo.TagPoint
244 }
245
246 func (s *Server) runningReleaseJob(ctx context.Context, repo store.Repo, token string) (store.Job, error) {
247 job, err := s.Store.JobByToken(ctx, token)
248 if err != nil || job.RepoID != repo.ID || job.Status != "running" || job.PipelineID == 0 {
249 return store.Job{}, fail(401, i18n.E("err.jobToken"))
250 }
251 return job, nil
252 }
253
254 func (s *Server) authorizeExistingTag(ctx context.Context, repo store.Repo, token, tag, commit string) (existingTagAuth, error) {
255 job, err := s.runningReleaseJob(ctx, repo, token)
256 if err != nil {
257 return existingTagAuth{}, err
258 }
259 pipe, err := s.Store.PipelineByID(ctx, repo.ID, job.PipelineID)
260 if err != nil || pipe.RepoID != repo.ID || pipe.Ref != "refs/tags/"+tag || pipe.Source != "push" || pipe.MRID != 0 {
261 if err != nil && !store.IsNotFound(err) && s.Log != nil {
262 s.Log.Error("release pipeline", "err", err, "repo", repo.FullPath(), "tag", tag)
263 }
264 return existingTagAuth{}, fail(403, i18n.E("err.releaseRef"))
265 }
266 if pipe.Actor == "" {
267 return existingTagAuth{}, fail(403, i18n.E("err.releaseActor"))
268 }
269 actor, err := s.Store.UserByUsername(ctx, pipe.Actor)
270 if err != nil || actor.ID == 0 {
271 return existingTagAuth{}, fail(403, i18n.E("err.releaseActor"))
272 }
273 if !actor.InstanceAdmin {
274 role, rerr := s.Store.EffectiveRole(ctx, actor.ID, repo.ID)
275 if rerr != nil || !role.AtLeast(access.RoleMaintainer) {
276 return existingTagAuth{}, fail(403, i18n.E("err.releaseActor"))
277 }
278 }
279 if commit == "" {
280 return existingTagAuth{}, fail(400, i18n.E("err.commitAbsent"))
281 }
282 full, ok := fullCommit(commit)
283 if !ok {
284 return existingTagAuth{}, fail(400, i18n.E("err.commitFull"))
285 }
286 point, err := gitrepo.ResolveTag(ctx, s.Git, gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name), tag)
287 if err != nil {
288 return existingTagAuth{}, s.mapTagErr(err)
289 }
290 jobSHA, okJob := fullCommit(job.CommitSHA)
291 pipeSHA, okPipe := fullCommit(pipe.CommitSHA)
292 if !okJob || !okPipe || jobSHA != full || pipeSHA != full || point.CommitOID != full {
293 return existingTagAuth{}, fail(409, i18n.E("err.releaseCommit"))
294 }
295 return existingTagAuth{job: job, pipe: pipe, point: point}, nil
296 }
297
298 func (s *Server) mapTagErr(err error) error {
299 var te *gitrepo.TagRefError
300 if errors.As(err, &te) && te.Detail != "" && s.Log != nil {
301 s.Log.Error("tag ref", "err", te.Kind, "detail", te.Detail)
302 }
303 switch {
304 case errors.Is(err, gitrepo.ErrTagMissing):
305 return fail(404, i18n.E("err.notFound"))
306 case errors.Is(err, gitrepo.ErrTagSymbolic), errors.Is(err, gitrepo.ErrTagNotCommit):
307 return fail(400, i18n.E("err.releaseTag"))
308 case errors.Is(err, gitrepo.ErrTagMoved):
309 return fail(409, i18n.E("err.releaseMoved"))
310 default:
311 if s.Log != nil && te == nil {
312 s.Log.Error("tag ref", "err", err)
313 }
314 return fail(500, i18n.E("err.releaseFail"))
315 }
316 }
317
318 func (s *Server) wrapPublishErr(err error) error {
319 var oe *opError
320 if errors.As(err, &oe) {
321 return err
322 }
323 if s.Log != nil {
324 s.Log.Error("release publicatie", "err", err)
325 }
326 return fail(500, i18n.E("err.releaseFail"))
327 }
328
329 func displayName(name, tag string) string {
330 if name == "" {
331 return tag
332 }
333 return name
334 }
335
336 func fullCommit(s string) (string, bool) {
337 if len(s) != 40 && len(s) != 64 {
338 return "", false
339 }
340 s = strings.ToLower(s)
341 for _, c := range s {
342 if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
343 return "", false
344 }
345 }
346 return s, true
347 }
348
349 func validArtifactKey(p string) bool {
350 if p == "" || len(p) > maxArtifactKeyLen || strings.HasPrefix(p, "/") || strings.Contains(p, "\\") || strings.ContainsRune(p, 0) {
351 return false
352 }
353 slash := filepath.ToSlash(p)
354 if slash == "." || slash == ".." || strings.HasPrefix(slash, "../") || strings.Contains(slash, "/../") || strings.HasSuffix(slash, "/..") {
355 return false
356 }
357 for _, r := range p {
358 if r < 0x20 || r == 0x7f {
359 return false
360 }
361 }
362 return true
363 }
364
365 func releaseAssetName(name, path string) (string, error) {
366 if name == "" {
367 name = filepath.Base(filepath.ToSlash(path))
368 }
369 if name == "" || name == "." || name == ".." || len(name) > 255 {
370 return "", errors.New("name")
371 }
372 if strings.ContainsAny(name, "/\\") || strings.ContainsRune(name, 0) {
373 return "", errors.New("name")
374 }
375 for _, r := range name {
376 if r < 0x20 || r == 0x7f {
377 return "", errors.New("name")
378 }
379 }
380 return name, nil
381 }
382
383 func pathInside(root, path string) bool {
384 root, err := filepath.Abs(root)
385 if err != nil {
386 return false
387 }
388 path, err = filepath.Abs(path)
389 if err != nil {
390 return false
391 }
392 rel, err := filepath.Rel(root, path)
393 if err != nil {
394 return false
395 }
396 return rel != ".." && !strings.HasPrefix(rel, ".."+string(os.PathSeparator))
397 }
398
399 func copyReleaseExclusive(src, dst string) error {
400 in, err := os.Open(src)
401 if err != nil {
402 return err
403 }
404 defer in.Close()
405 out, err := os.OpenFile(dst, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o644)
406 if err != nil {
407 return err
408 }
409 _, copyErr := io.Copy(out, in)
410 syncErr := out.Sync()
411 closeErr := out.Close()
412 if copyErr != nil {
413 return copyErr
414 }
415 if syncErr != nil {
416 return syncErr
417 }
418 return closeErr
419 }
420
421 func syncDir(path string) error {
422 d, err := os.Open(path)
423 if err != nil {
424 return err
425 }
426 defer d.Close()
427 return d.Sync()
428 }
@@ -0,0 +1,428 @@
1package server
2
3import (
4 "context"
5 "encoding/json"
6 "errors"
7 "io"
8 "net/http"
9 "os"
10 "path/filepath"
11 "strings"
12
13 "gitlab.com/Timfa/bluegit/internal/access"
14 "gitlab.com/Timfa/bluegit/internal/gitrepo"
15 "gitlab.com/Timfa/bluegit/internal/i18n"
16 "gitlab.com/Timfa/bluegit/internal/names"
17 "gitlab.com/Timfa/bluegit/internal/store"
18)
19
20const (
21 maxExistingReleaseAssets = 32
22 maxExistingReleaseName = 200
23 maxExistingReleaseBody = 64 << 10
24 maxArtifactKeyLen = 512
25)
26
27// existingTagBeforeLock draait na de eerste tagresolutie en vóór prepare.
28// Tests verplaatsen de tag hier om de race vóór de lock te bewijzen.
29var existingTagBeforeLock = func(ctx context.Context) error { return ctx.Err() }
30
31// existingTagDuringLock draait terwijl git de tagref vasthoudt, vóór de insert.
32var existingTagDuringLock = func(ctx context.Context) error { return ctx.Err() }
33
34// releaseCopyFile kopieert één artifact. Tests laten een latere kopie falen.
35var releaseCopyFile = copyReleaseExclusive
36
37// releaseInsert schrijft de release. Tests injecteren een storefout.
38var releaseInsert = func(ctx context.Context, st *store.Store, repoID int64, tag, sha, name, body string, authorID, jobID int64, assets []store.ReleaseAsset) (int64, error) {
39 return st.CreateRelease(ctx, repoID, tag, sha, name, body, authorID, jobID, assets)
40}
41
42type existingReleaseRequest struct {
43 Commit string `json:"commit"`
44 Name string `json:"name"`
45 Body string `json:"body"`
46 Replace json.RawMessage `json:"replace"`
47 Tag *string `json:"tag"`
48 Assets []struct {
49 Job string `json:"job"`
50 Path string `json:"path"`
51 Name string `json:"name"`
52 } `json:"assets"`
53}
54
55type stagedAsset struct {
56 name string
57 src string
58 size int64
59}
60
61// publishExistingTag publiceert een nieuwe release bij een tag die al in git staat.
62// Alleen een lopend jobtoken van een push naar refs/tags/<tag>. De tag wordt niet
63// aangemaakt, verplaatst of verwijderd. Een bestaande release blijft staan (409).
64func (s *Server) publishExistingTag(w http.ResponseWriter, r *http.Request) {
65 repo, err := s.Store.RepoByPath(r.Context(), r.PathValue("owner"), r.PathValue("repo"))
66 if err != nil {
67 s.writeAPIErr(w, r, fail(404, i18n.E("err.notFound")))
68 return
69 }
70 token := tokenFrom(r)
71 if !strings.HasPrefix(token, "bgj_") {
72 s.writeAPIErr(w, r, fail(401, i18n.E("err.jobToken")))
73 return
74 }
75 var req existingReleaseRequest
76 if err := readJSON(r, &req); err != nil {
77 s.writeAPIErr(w, r, fail(400, i18n.E("err.json")))
78 return
79 }
80 id, commit, err := s.opPublishExistingTag(r.Context(), repo, token, r.PathValue("tag"), req)
81 if !s.apiOK(w, r, err) {
82 return
83 }
84 writeJSON(w, 201, map[string]any{"id": id, "tag": r.PathValue("tag"), "commit": commit})
85}
86
87func (s *Server) opPublishExistingTag(ctx context.Context, repo store.Repo, token, tag string, req existingReleaseRequest) (int64, string, error) {
88 if s.StoragePath() == "" {
89 return 0, "", fail(503, i18n.E("err.noStorage"))
90 }
91 if _, err := s.runningReleaseJob(ctx, repo, token); err != nil {
92 return 0, "", err
93 }
94 if len(req.Replace) > 0 {
95 return 0, "", fail(400, i18n.E("err.releaseReplace"))
96 }
97 if req.Tag != nil && *req.Tag != tag {
98 return 0, "", fail(400, i18n.E("err.releaseTag"))
99 }
100 if err := names.ValidTag(tag); err != nil {
101 return 0, "", fail(400, err)
102 }
103 if len(req.Name) > maxExistingReleaseName || len(req.Body) > maxExistingReleaseBody {
104 return 0, "", fail(400, i18n.E("err.releaseLong"))
105 }
106 if len(req.Assets) > maxExistingReleaseAssets {
107 return 0, "", fail(400, i18n.E("err.releaseMany"))
108 }
109 seen := map[string]struct{}{}
110 type named struct{ job, path, name string }
111 var wanted []named
112 for _, a := range req.Assets {
113 if !validArtifactKey(a.Path) || strings.TrimSpace(a.Job) == "" {
114 return 0, "", fail(400, i18n.E("err.assetName"))
115 }
116 name, err := releaseAssetName(a.Name, a.Path)
117 if err != nil {
118 return 0, "", fail(400, i18n.E("err.assetName"))
119 }
120 if _, dup := seen[name]; dup {
121 return 0, "", fail(400, i18n.E("err.assetName"))
122 }
123 seen[name] = struct{}{}
124 wanted = append(wanted, named{a.Job, a.Path, name})
125 }
126 auth, err := s.authorizeExistingTag(ctx, repo, token, tag, req.Commit)
127 if err != nil {
128 return 0, "", err
129 }
130 if _, err := s.Store.ReleaseByTag(ctx, repo.ID, tag); err == nil {
131 return 0, "", fail(409, i18n.E("err.releaseExists", tag))
132 } else if !store.IsNotFound(err) {
133 return 0, "", fail(500, i18n.E("err.releaseFail"))
134 }
135 var staged []stagedAsset
136 for _, a := range wanted {
137 src, err := s.Store.ArtifactForPublish(ctx, auth.job.PipelineID, a.job, a.path)
138 if errors.Is(err, store.ErrArtifactExpired) {
139 return 0, "", fail(400, i18n.E("err.artifactExpired", a.job, a.path))
140 }
141 if err != nil {
142 return 0, "", fail(400, i18n.E("err.artifactMissing", a.job, a.path))
143 }
144 info, err := os.Lstat(src)
145 if err != nil {
146 return 0, "", fail(400, i18n.E("err.assetDisk"))
147 }
148 if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() {
149 return 0, "", fail(400, i18n.E("err.assetFile"))
150 }
151 if !pathInside(s.StoragePath(), src) {
152 return 0, "", fail(400, i18n.E("err.assetFile"))
153 }
154 staged = append(staged, stagedAsset{name: a.name, src: src, size: info.Size()})
155 }
156 parent := filepath.Join(s.StoragePath(), "releases", ".incoming")
157 if err := os.MkdirAll(parent, 0o755); err != nil {
158 return 0, "", fail(500, i18n.E("err.releaseFail"))
159 }
160 stage, err := os.MkdirTemp(parent, "pub-")
161 if err != nil {
162 return 0, "", fail(500, i18n.E("err.releaseFail"))
163 }
164 published := false
165 defer func() {
166 if !published {
167 _ = os.RemoveAll(stage)
168 }
169 }()
170 var assets []store.ReleaseAsset
171 for _, sa := range staged {
172 dst := filepath.Join(stage, sa.name)
173 if !pathInside(stage, dst) {
174 return 0, "", fail(400, i18n.E("err.assetName"))
175 }
176 if err := releaseCopyFile(sa.src, dst); err != nil {
177 if s.Log != nil {
178 s.Log.Error("release kopie", "err", err, "repo", repo.FullPath(), "tag", tag)
179 }
180 return 0, "", fail(500, i18n.E("err.assetCopy"))
181 }
182 info, err := os.Lstat(dst)
183 if err != nil || !info.Mode().IsRegular() || info.Size() != sa.size {
184 return 0, "", fail(500, i18n.E("err.assetCopy"))
185 }
186 assets = append(assets, store.ReleaseAsset{Name: sa.name, StoragePath: dst, Size: info.Size()})
187 }
188 if err := syncDir(stage); err != nil {
189 return 0, "", fail(500, i18n.E("err.assetCopy"))
190 }
191 if err := existingTagBeforeLock(ctx); err != nil {
192 return 0, "", s.wrapPublishErr(err)
193 }
194 dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
195 hold, err := gitrepo.HoldTagRef(ctx, s.Git, dir, tag, auth.point.ObjectOID)
196 if err != nil {
197 return 0, "", s.mapTagErr(err)
198 }
199 defer func() { _ = hold.Release() }()
200 if err := existingTagDuringLock(ctx); err != nil {
201 return 0, "", s.wrapPublishErr(err)
202 }
203 again, err := s.authorizeExistingTag(ctx, repo, token, tag, req.Commit)
204 if err != nil {
205 return 0, "", err
206 }
207 if again.point.ObjectOID != auth.point.ObjectOID || again.point.CommitOID != auth.point.CommitOID {
208 return 0, "", fail(409, i18n.E("err.releaseMoved"))
209 }
210 if _, err := s.Store.ReleaseByTag(ctx, repo.ID, tag); err == nil {
211 return 0, "", fail(409, i18n.E("err.releaseExists", tag))
212 } else if !store.IsNotFound(err) {
213 return 0, "", fail(500, i18n.E("err.releaseFail"))
214 }
215 id, err := releaseInsert(ctx, s.Store, repo.ID, tag, again.point.CommitOID, displayName(req.Name, tag), req.Body, 0, again.job.ID, assets)
216 if err != nil {
217 var ie i18n.Error
218 if errors.As(err, &ie) && ie.Key == "err.tagExists" {
219 return 0, "", fail(409, i18n.E("err.releaseExists", tag))
220 }
221 if s.Log != nil {
222 s.Log.Error("release insert", "err", err, "repo", repo.FullPath(), "tag", tag)
223 }
224 return 0, "", fail(500, i18n.E("err.releaseFail"))
225 }
226 published = true
227 // De lock moet los vóór het event: een webhook mag de ref niet vasthouden.
228 // Lukt abort niet, dan blijft de gecommitte release staan. Geen tweede insert.
229 if relErr := hold.Release(); relErr != nil && s.Log != nil {
230 s.Log.Error("release ref lock", "err", relErr, "repo", repo.FullPath(), "tag", tag)
231 }
232 if s.Log != nil {
233 s.Log.Info("release audit", "action", "create", "repo", repo.FullPath(), "tag", tag,
234 "commit", again.point.CommitOID, "job", again.job.ID, "pipeline", again.job.PipelineID, "actor", again.pipe.Actor)
235 }
236 s.Dispatch(repo.ID, s.releaseEvent(repo, tag, displayName(req.Name, tag), again.point.CommitOID, assets))
237 return id, again.point.CommitOID, nil
238}
239
240type existingTagAuth struct {
241 job store.Job
242 pipe store.Pipeline
243 point gitrepo.TagPoint
244}
245
246func (s *Server) runningReleaseJob(ctx context.Context, repo store.Repo, token string) (store.Job, error) {
247 job, err := s.Store.JobByToken(ctx, token)
248 if err != nil || job.RepoID != repo.ID || job.Status != "running" || job.PipelineID == 0 {
249 return store.Job{}, fail(401, i18n.E("err.jobToken"))
250 }
251 return job, nil
252}
253
254func (s *Server) authorizeExistingTag(ctx context.Context, repo store.Repo, token, tag, commit string) (existingTagAuth, error) {
255 job, err := s.runningReleaseJob(ctx, repo, token)
256 if err != nil {
257 return existingTagAuth{}, err
258 }
259 pipe, err := s.Store.PipelineByID(ctx, repo.ID, job.PipelineID)
260 if err != nil || pipe.RepoID != repo.ID || pipe.Ref != "refs/tags/"+tag || pipe.Source != "push" || pipe.MRID != 0 {
261 if err != nil && !store.IsNotFound(err) && s.Log != nil {
262 s.Log.Error("release pipeline", "err", err, "repo", repo.FullPath(), "tag", tag)
263 }
264 return existingTagAuth{}, fail(403, i18n.E("err.releaseRef"))
265 }
266 if pipe.Actor == "" {
267 return existingTagAuth{}, fail(403, i18n.E("err.releaseActor"))
268 }
269 actor, err := s.Store.UserByUsername(ctx, pipe.Actor)
270 if err != nil || actor.ID == 0 {
271 return existingTagAuth{}, fail(403, i18n.E("err.releaseActor"))
272 }
273 if !actor.InstanceAdmin {
274 role, rerr := s.Store.EffectiveRole(ctx, actor.ID, repo.ID)
275 if rerr != nil || !role.AtLeast(access.RoleMaintainer) {
276 return existingTagAuth{}, fail(403, i18n.E("err.releaseActor"))
277 }
278 }
279 if commit == "" {
280 return existingTagAuth{}, fail(400, i18n.E("err.commitAbsent"))
281 }
282 full, ok := fullCommit(commit)
283 if !ok {
284 return existingTagAuth{}, fail(400, i18n.E("err.commitFull"))
285 }
286 point, err := gitrepo.ResolveTag(ctx, s.Git, gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name), tag)
287 if err != nil {
288 return existingTagAuth{}, s.mapTagErr(err)
289 }
290 jobSHA, okJob := fullCommit(job.CommitSHA)
291 pipeSHA, okPipe := fullCommit(pipe.CommitSHA)
292 if !okJob || !okPipe || jobSHA != full || pipeSHA != full || point.CommitOID != full {
293 return existingTagAuth{}, fail(409, i18n.E("err.releaseCommit"))
294 }
295 return existingTagAuth{job: job, pipe: pipe, point: point}, nil
296}
297
298func (s *Server) mapTagErr(err error) error {
299 var te *gitrepo.TagRefError
300 if errors.As(err, &te) && te.Detail != "" && s.Log != nil {
301 s.Log.Error("tag ref", "err", te.Kind, "detail", te.Detail)
302 }
303 switch {
304 case errors.Is(err, gitrepo.ErrTagMissing):
305 return fail(404, i18n.E("err.notFound"))
306 case errors.Is(err, gitrepo.ErrTagSymbolic), errors.Is(err, gitrepo.ErrTagNotCommit):
307 return fail(400, i18n.E("err.releaseTag"))
308 case errors.Is(err, gitrepo.ErrTagMoved):
309 return fail(409, i18n.E("err.releaseMoved"))
310 default:
311 if s.Log != nil && te == nil {
312 s.Log.Error("tag ref", "err", err)
313 }
314 return fail(500, i18n.E("err.releaseFail"))
315 }
316}
317
318func (s *Server) wrapPublishErr(err error) error {
319 var oe *opError
320 if errors.As(err, &oe) {
321 return err
322 }
323 if s.Log != nil {
324 s.Log.Error("release publicatie", "err", err)
325 }
326 return fail(500, i18n.E("err.releaseFail"))
327}
328
329func displayName(name, tag string) string {
330 if name == "" {
331 return tag
332 }
333 return name
334}
335
336func fullCommit(s string) (string, bool) {
337 if len(s) != 40 && len(s) != 64 {
338 return "", false
339 }
340 s = strings.ToLower(s)
341 for _, c := range s {
342 if (c < '0' || c > '9') && (c < 'a' || c > 'f') {
343 return "", false
344 }
345 }
346 return s, true
347}
348
349func validArtifactKey(p string) bool {
350 if p == "" || len(p) > maxArtifactKeyLen || strings.HasPrefix(p, "/") || strings.Contains(p, "\\") || strings.ContainsRune(p, 0) {
351 return false
352 }
353 slash := filepath.ToSlash(p)
354 if slash == "." || slash == ".." || strings.HasPrefix(slash, "../") || strings.Contains(slash, "/../") || strings.HasSuffix(slash, "/..") {
355 return false
356 }
357 for _, r := range p {
358 if r < 0x20 || r == 0x7f {
359 return false
360 }
361 }
362 return true
363}
364
365func releaseAssetName(name, path string) (string, error) {
366 if name == "" {
367 name = filepath.Base(filepath.ToSlash(path))
368 }
369 if name == "" || name == "." || name == ".." || len(name) > 255 {
370 return "", errors.New("name")
371 }
372 if strings.ContainsAny(name, "/\\") || strings.ContainsRune(name, 0) {
373 return "", errors.New("name")
374 }
375 for _, r := range name {
376 if r < 0x20 || r == 0x7f {
377 return "", errors.New("name")
378 }
379 }
380 return name, nil
381}
382
383func pathInside(root, path string) bool {
384 root, err := filepath.Abs(root)
385 if err != nil {
386 return false
387 }
388 path, err = filepath.Abs(path)
389 if err != nil {
390 return false
391 }
392 rel, err := filepath.Rel(root, path)
393 if err != nil {
394 return false
395 }
396 return rel != ".." && !strings.HasPrefix(rel, ".."+string(os.PathSeparator))
397}
398
399func copyReleaseExclusive(src, dst string) error {
400 in, err := os.Open(src)
401 if err != nil {
402 return err
403 }
404 defer in.Close()
405 out, err := os.OpenFile(dst, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o644)
406 if err != nil {
407 return err
408 }
409 _, copyErr := io.Copy(out, in)
410 syncErr := out.Sync()
411 closeErr := out.Close()
412 if copyErr != nil {
413 return copyErr
414 }
415 if syncErr != nil {
416 return syncErr
417 }
418 return closeErr
419}
420
421func syncDir(path string) error {
422 d, err := os.Open(path)
423 if err != nil {
424 return err
425 }
426 defer d.Close()
427 return d.Sync()
428}
internal/server/release_existing_tag_test.go added +796 −0
@@ -0,0 +1,796 @@
1 package server
2
3 import (
4 "bytes"
5 "context"
6 "fmt"
7 "io"
8 "log/slog"
9 "net/http"
10 "net/http/cookiejar"
11 "os"
12 "os/exec"
13 "path/filepath"
14 "sort"
15 "strings"
16 "sync"
17 "testing"
18 "time"
19
20 "gitlab.com/Timfa/bluegit/internal/access"
21 "gitlab.com/Timfa/bluegit/internal/gitrepo"
22 "gitlab.com/Timfa/bluegit/internal/store"
23 )
24
25 func TestPublishExistingTag(t *testing.T) {
26 restoreReleaseSeams(t)
27 e := newCIEnv(t)
28 ctx := context.Background()
29 var audits bytes.Buffer
30 e.srv.Log = slog.New(slog.NewTextHandler(&audits, nil))
31 var events int
32 var eventMu sync.Mutex
33 e.srv.hookSink = func([]byte) {
34 eventMu.Lock()
35 events++
36 eventMu.Unlock()
37 }
38 if _, err := e.st.CreateWebhook(ctx, store.Webhook{
39 ScopeKind: "repo", ScopeID: e.repo.ID, Name: "rel", Preset: "generic",
40 URLTemplate: "https://example.test/hook", Events: []string{"release_published"},
41 }); err != nil {
42 t.Fatal(err)
43 }
44 _, _ = e.st.UpsertUser(ctx, 21, "mnt", "", false)
45 _ = e.st.SetMember(ctx, "repo", e.repo.ID, "mnt", access.RoleMaintainer)
46 if err := e.st.AddProtectedTag(ctx, e.repo.ID, "v*"); err != nil {
47 t.Fatal(err)
48 }
49
50 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
51 putTag(t, bare, "v1", e.sha, false)
52 putTag(t, bare, "ann", e.sha, true)
53 beforeRefs := refList(t, bare)
54 beforePipes, _ := e.st.ListPipelines(ctx, e.repo.ID)
55
56 tok, _, jobs := e.runningTagJob("refs/tags/v1", "tim", "push", e.sha, []string{"build"})
57 body := []byte("paperdoll-bytes")
58 e.artifact(t, jobs["build"], "dist/paperdoll-renderer.AppImage", body, nil)
59 code, raw := e.postExisting(tok, "v1", map[string]any{
60 "commit": e.sha, "name": "Paperdoll Renderer v0.1.0", "body": "notities",
61 "assets": []map[string]any{{"job": "build", "path": "dist/paperdoll-renderer.AppImage", "name": "paperdoll-renderer.AppImage"}},
62 })
63 if code != 201 || !strings.Contains(raw, `"tag":"v1"`) || !strings.Contains(raw, `"commit":"`+e.sha+`"`) {
64 t.Fatalf("lightweight: %d %s", code, raw)
65 }
66 rel, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1")
67 if err != nil || rel.Name != "Paperdoll Renderer v0.1.0" || rel.Body != "notities" || rel.CommitSHA != e.sha || len(rel.Assets) != 1 {
68 t.Fatalf("release: %+v %v", rel, err)
69 }
70 if !strings.Contains(rel.Assets[0].StoragePath, string(filepath.Separator)+".incoming"+string(filepath.Separator)) {
71 t.Fatalf("pad niet privé: %s", rel.Assets[0].StoragePath)
72 }
73 det := filepath.Join(e.srv.StoragePath(), "releases", "tim", "app", "v1", "paperdoll-renderer.AppImage")
74 if _, err := os.Stat(det); !os.IsNotExist(err) {
75 t.Fatalf("vast tagpad bestaat: %v", err)
76 }
77 got, err := os.ReadFile(rel.Assets[0].StoragePath)
78 if err != nil || !bytes.Equal(got, body) {
79 t.Fatalf("bytes: %v %q", err, got)
80 }
81 owner := issue(t, e.st, e.user.ID, "api")
82 res := postJSON(t, http.MethodGet, e.ts.URL+"/api/v1/projects/tim/app/releases/v1/assets/paperdoll-renderer.AppImage", owner, nil)
83 down, _ := io.ReadAll(res.Body)
84 res.Body.Close()
85 if res.StatusCode != 200 || !bytes.Equal(down, body) {
86 t.Fatalf("download: %d %q", res.StatusCode, down)
87 }
88 if refList(t, bare) != beforeRefs {
89 t.Fatalf("refs veranderd\nwas:\n%s\nis:\n%s", beforeRefs, refList(t, bare))
90 }
91 if object, commit := tagPoint(t, bare, "v1"); object != e.sha || commit != e.sha {
92 t.Fatalf("lightweight oid %s %s", object, commit)
93 }
94 afterPipes, _ := e.st.ListPipelines(ctx, e.repo.ID)
95 if len(afterPipes) != len(beforePipes)+1 {
96 t.Fatalf("pipelines: %d naar %d", len(beforePipes), len(afterPipes))
97 }
98 if !strings.Contains(audits.String(), "action=create") || !strings.Contains(audits.String(), "tag=v1") {
99 t.Fatalf("audit: %s", audits.String())
100 }
101 if events != 1 {
102 t.Fatalf("events: %d", events)
103 }
104 code, _ = e.postExisting(tok, "v1", map[string]any{"commit": e.sha})
105 if code != 409 || events != 1 {
106 t.Fatalf("retry: %d events %d", code, events)
107 }
108 again, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "v1")
109 if again.ID != rel.ID || again.Body != rel.Body {
110 t.Fatalf("retry wijzigde de release")
111 }
112
113 tok, _, jobs = e.runningTagJob("refs/tags/ann", "mnt", "push", e.sha, []string{"build"})
114 e.artifact(t, jobs["build"], "dist/a.bin", []byte("ann"), nil)
115 code, raw = e.postExisting(tok, "ann", map[string]any{
116 "commit": strings.ToUpper(e.sha), "name": "annotated",
117 "assets": []map[string]any{{"job": "build", "path": "dist/a.bin"}},
118 })
119 if code != 201 {
120 t.Fatalf("annotated: %d %s", code, raw)
121 }
122 obj, commit := tagPoint(t, bare, "ann")
123 if obj == commit || commit != e.sha {
124 t.Fatalf("annotated tag veranderd: %s %s", obj, commit)
125 }
126 annRel, err := e.st.ReleaseByTag(ctx, e.repo.ID, "ann")
127 if err != nil || len(annRel.Assets) != 1 || annRel.Assets[0].Name != "a.bin" {
128 t.Fatalf("defaultnaam: %+v %v", annRel, err)
129 }
130
131 gitOKServer(t, bare, "pack-refs", "--all")
132 putTag(t, bare, "packed", e.sha, false)
133 // pack-refs opnieuw, zodat packed geen losse ref is.
134 gitOKServer(t, bare, "pack-refs", "--all")
135 tok, _, _ = e.runningTagJob("refs/tags/packed", "tim", "push", e.sha, nil)
136 if code, raw = e.postExisting(tok, "packed", map[string]any{"commit": e.sha, "name": "packed"}); code != 201 {
137 t.Fatalf("packed: %d %s", code, raw)
138 }
139 pats, err := e.st.ProtectedTagPatterns(ctx, e.repo.ID)
140 if err != nil || len(pats) != 1 || pats[0] != "v*" {
141 t.Fatalf("bescherming: %v %v", pats, err)
142 }
143
144 spec, err := http.Get(e.ts.URL + "/api/v1/openapi.json")
145 if err != nil {
146 t.Fatal(err)
147 }
148 specBody := read(t, spec)
149 if !strings.Contains(specBody, "/repos/{owner}/{repo}/releases/{tag}/assets") || !strings.Contains(specBody, "Job token only") {
150 t.Fatal("openapi mist de route")
151 }
152 }
153
154 func TestPublishExistingTagAuth(t *testing.T) {
155 restoreReleaseSeams(t)
156 e := newCIEnv(t)
157 ctx := context.Background()
158 _, _ = e.st.UpsertUser(ctx, 20, "dev", "", false)
159 _ = e.st.SetMember(ctx, "repo", e.repo.ID, "dev", access.RoleDeveloper)
160 _, _ = e.st.UpsertUser(ctx, 21, "mnt", "", false)
161 _ = e.st.SetMember(ctx, "repo", e.repo.ID, "mnt", access.RoleMaintainer)
162 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
163 putTag(t, bare, "v1", e.sha, false)
164 otherSHA := commitExtra(t, bare, e.sha)
165
166 okTok, _, _ := e.runningTagJob("refs/tags/v1", "tim", "push", e.sha, nil)
167 if code, raw := e.postExisting("", "v1", map[string]any{"commit": e.sha}); code != 401 {
168 t.Fatalf("geen token: %d %s", code, raw)
169 }
170 if code, raw := e.postExisting("bgj_onzin", "v1", map[string]any{"commit": e.sha}); code != 401 {
171 t.Fatalf("malformed: %d %s", code, raw)
172 }
173 pat := issue(t, e.st, e.user.ID, "api")
174 if code, raw := e.postExisting(pat, "v1", map[string]any{"commit": e.sha}); code != 401 {
175 t.Fatalf("pat: %d %s", code, raw)
176 }
177 jar, _ := cookiejar.New(nil)
178 browser := &http.Client{Jar: jar}
179 login, err := http.NewRequest(http.MethodGet, e.ts.URL+"/login", nil)
180 if err != nil {
181 t.Fatal(err)
182 }
183 res, err := browser.Do(login)
184 if err != nil {
185 t.Fatal(err)
186 }
187 res.Body.Close()
188 sessReq, _ := http.NewRequest(http.MethodPost, e.ts.URL+"/api/v1/repos/tim/app/releases/v1/assets", strings.NewReader(`{"commit":"`+e.sha+`"}`))
189 sessReq.Header.Set("Content-Type", "application/json")
190 sessRes, err := browser.Do(sessReq)
191 if err != nil {
192 t.Fatal(err)
193 }
194 sessBody := read(t, sessRes)
195 if sessRes.StatusCode != 401 {
196 t.Fatalf("sessie: %d %s", sessRes.StatusCode, sessBody)
197 }
198
199 queued := e.jobWithStatus(t, "refs/tags/v1", "tim", e.sha, "queued")
200 done := e.jobWithStatus(t, "refs/tags/v1", "tim", e.sha, "success")
201 canceled := e.jobWithStatus(t, "refs/tags/v1", "tim", e.sha, "canceled")
202 for _, tok := range []string{queued, done, canceled} {
203 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha}); code != 401 {
204 t.Fatalf("status: %d %s", code, raw)
205 }
206 }
207 other, err := e.st.CreateRepo(ctx, "user", e.user.ID, "other", "", "master", "private")
208 if err != nil {
209 t.Fatal(err)
210 }
211 otherTok, _, _ := e.runningTagJobRepo(other.ID, "refs/tags/v1", "tim", "push", e.sha, nil)
212 if code, raw := e.postExisting(otherTok, "v1", map[string]any{"commit": e.sha}); code != 401 {
213 t.Fatalf("ander project: %d %s", code, raw)
214 }
215
216 getRes := postJSON(t, http.MethodGet, e.ts.URL+"/api/v1/projects/tim/app", okTok, nil)
217 getBody := read(t, getRes)
218 if getRes.StatusCode != 200 || !strings.Contains(getBody, `"path":"tim/app"`) {
219 t.Fatalf("job GET: %d %s", getRes.StatusCode, getBody)
220 }
221 putRes := postJSON(t, http.MethodPut, e.ts.URL+"/api/v1/projects/tim/app", okTok, map[string]any{"description": "nee"})
222 putRes.Body.Close()
223 if putRes.StatusCode != 401 {
224 t.Fatalf("job PUT: %d", putRes.StatusCode)
225 }
226
227 for _, actor := range []string{"dev", "nobody", ""} {
228 tok, _, _ := e.runningTagJob("refs/tags/v1", actor, "push", e.sha, nil)
229 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha}); code != 403 {
230 t.Fatalf("actor %q: %d %s", actor, code, raw)
231 }
232 }
233 _ = e.st.RemoveMember(ctx, "repo", e.repo.ID, "mnt")
234 tok, _, _ := e.runningTagJob("refs/tags/v1", "mnt", "push", e.sha, nil)
235 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha}); code != 403 {
236 t.Fatalf("ingetrokken: %d %s", code, raw)
237 }
238 if _, err := e.st.SetInstanceAdmin(ctx, e.user.ID, mustUser(t, e.st, "dev"), true); err != nil {
239 t.Fatal(err)
240 }
241 _ = e.st.RemoveMember(ctx, "repo", e.repo.ID, "dev")
242 tok, _, _ = e.runningTagJob("refs/tags/v1", "dev", "push", e.sha, nil)
243 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "name": "admin"}); code != 201 {
244 t.Fatalf("instance-admin: %d %s", code, raw)
245 }
246 if _, err := e.st.SetInstanceAdmin(ctx, e.user.ID, mustUser(t, e.st, "dev"), false); err != nil {
247 t.Fatal(err)
248 }
249 putTag(t, bare, "vno", e.sha, false)
250 tok, _, _ = e.runningTagJob("refs/tags/vno", "dev", "push", e.sha, nil)
251 if code, raw := e.postExisting(tok, "vno", map[string]any{"commit": e.sha}); code != 403 {
252 t.Fatalf("admin ingetrokken: %d %s", code, raw)
253 }
254
255 cases := []struct {
256 ref, source, sha, url, commit string
257 mr bool
258 want int
259 }{
260 {"refs/heads/master", "push", e.sha, "v1", e.sha, false, 403},
261 {"refs/heads/forks/7/topic", "push", e.sha, "v1", e.sha, false, 403},
262 {"refs/tags/v1", "web", e.sha, "v1", e.sha, false, 403},
263 {"refs/tags/other", "push", e.sha, "v1", e.sha, false, 403},
264 {"refs/tags/v1", "push", e.sha, "v1", e.sha, true, 403},
265 {"refs/tags/v1", "push", otherSHA, "v1", e.sha, false, 409},
266 {"refs/tags/v1", "push", e.sha, "v1", otherSHA, false, 409},
267 {"refs/tags/v1", "push", e.sha, "v1", e.sha[:12], false, 400},
268 }
269 for _, tc := range cases {
270 tok, pid, _ := e.runningTagJob(tc.ref, "tim", tc.source, tc.sha, nil)
271 if tc.mr {
272 if err := e.st.AttachPipelineMR(ctx, pid, 4); err != nil {
273 t.Fatal(err)
274 }
275 }
276 if code, raw := e.postExisting(tok, tc.url, map[string]any{"commit": tc.commit}); code != tc.want {
277 t.Fatalf("%+v: %d %s", tc, code, raw)
278 }
279 }
280 missTok, _, _ := e.runningTagJob("refs/tags/missing", "tim", "push", e.sha, nil)
281 if code, raw := e.postExisting(missTok, "missing", map[string]any{"commit": e.sha}); code != 404 {
282 t.Fatalf("tag weg: %d %s", code, raw)
283 }
284 gitOKServer(t, bare, "symbolic-ref", "refs/tags/sym", "refs/heads/master")
285 tok, _, _ = e.runningTagJob("refs/tags/sym", "tim", "push", e.sha, nil)
286 if code, raw := e.postExisting(tok, "sym", map[string]any{"commit": e.sha}); code != 400 {
287 t.Fatalf("symbolisch: %d %s", code, raw)
288 }
289 tree := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "HEAD^{tree}"))
290 gitOKServer(t, bare, "update-ref", "refs/tags/boom", tree)
291 tok, _, _ = e.runningTagJob("refs/tags/boom", "tim", "push", e.sha, nil)
292 if code, raw := e.postExisting(tok, "boom", map[string]any{"commit": e.sha}); code != 400 {
293 t.Fatalf("geen commit: %d %s", code, raw)
294 }
295 putTag(t, bare, "ann", e.sha, true)
296 annObj, _ := tagPoint(t, bare, "ann")
297 tok, _, _ = e.runningTagJob("refs/tags/ann", "tim", "push", e.sha, nil)
298 if code, raw := e.postExisting(tok, "ann", map[string]any{"commit": annObj}); code != 409 {
299 t.Fatalf("tagobject als commit: %d %s", code, raw)
300 }
301 if code, raw := e.postExisting(okTok, "v1", map[string]any{}); code != 400 {
302 t.Fatalf("commit afwezig: %d %s", code, raw)
303 }
304 if code, raw := e.postExisting(okTok, "../x", map[string]any{"commit": e.sha}); code != 400 && code != 404 {
305 t.Fatalf("slechte tag: %d %s", code, raw)
306 }
307 }
308
309 func TestPublishExistingTagConflictAndLock(t *testing.T) {
310 restoreReleaseSeams(t)
311 e := newCIEnv(t)
312 ctx := context.Background()
313 var events int
314 var mu sync.Mutex
315 e.srv.hookSink = func([]byte) {
316 mu.Lock()
317 events++
318 mu.Unlock()
319 }
320 if _, err := e.st.CreateWebhook(ctx, store.Webhook{
321 ScopeKind: "repo", ScopeID: e.repo.ID, Name: "rel", Preset: "generic",
322 URLTemplate: "https://example.test/hook", Events: []string{"release_published"},
323 }); err != nil {
324 t.Fatal(err)
325 }
326 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
327 user := issue(t, e.st, e.user.ID, "api")
328 made := postJSON(t, http.MethodPost, e.ts.URL+"/api/v1/projects/tim/app/releases", user, map[string]any{"tag": "hand", "commit": e.sha, "name": "met de hand", "body": "blijft"})
329 madeBody := read(t, made)
330 if made.StatusCode != 201 {
331 t.Fatalf("handmatig: %d %s", made.StatusCode, madeBody)
332 }
333 hand, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "hand")
334 tok, _, _ := e.runningTagJob("refs/tags/hand", "tim", "push", e.sha, nil)
335 if code, raw := e.postExisting(tok, "hand", map[string]any{"commit": e.sha, "name": "nieuw"}); code != 409 {
336 t.Fatalf("hand 409: %d %s", code, raw)
337 }
338 still, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "hand")
339 if still.ID != hand.ID || still.Name != "met de hand" || still.Body != "blijft" {
340 t.Fatalf("hand gewijzigd: %+v", still)
341 }
342
343 btok, _, bjobs := e.runningTagJob("refs/heads/topic", "tim", "push", e.sha, []string{"build"})
344 payload := []byte("branch-asset")
345 e.artifact(t, bjobs["build"], "dist/a.bin", payload, nil)
346 branchRes := postJSON(t, http.MethodPost, e.ts.URL+"/api/v1/repos/tim/app/releases", btok, map[string]any{
347 "tag": "from-branch", "commit": e.sha,
348 "assets": []map[string]any{{"job": "build", "path": "dist/a.bin", "name": "a.bin"}},
349 })
350 branchBody := read(t, branchRes)
351 if branchRes.StatusCode != 201 {
352 t.Fatalf("branchrelease: %d %s", branchRes.StatusCode, branchBody)
353 }
354 // De oude API maakt zelf de tag. Die release blijft 409 voor de nieuwe route.
355 old, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "from-branch")
356 oldBytes, _ := os.ReadFile(old.Assets[0].StoragePath)
357 tok, _, _ = e.runningTagJob("refs/tags/from-branch", "tim", "push", e.sha, nil)
358 if code, raw := e.postExisting(tok, "from-branch", map[string]any{"commit": e.sha}); code != 409 {
359 t.Fatalf("branch 409: %d %s", code, raw)
360 }
361 nowBytes, _ := os.ReadFile(old.Assets[0].StoragePath)
362 kept, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "from-branch")
363 if kept.ID != old.ID || !bytes.Equal(oldBytes, nowBytes) || !bytes.Equal(nowBytes, payload) {
364 t.Fatal("bestaande asset veranderd")
365 }
366
367 // Tag wijzigt tussen resolve en prepare.
368 putTag(t, bare, "race", e.sha, false)
369 moved := commitExtra(t, bare, e.sha)
370 existingTagBeforeLock = func(context.Context) error {
371 gitOKServer(t, bare, "update-ref", "refs/tags/race", moved)
372 return nil
373 }
374 tok, _, _ = e.runningTagJob("refs/tags/race", "tim", "push", e.sha, nil)
375 if code, raw := e.postExisting(tok, "race", map[string]any{"commit": e.sha}); code != 409 {
376 t.Fatalf("verplaatst: %d %s", code, raw)
377 }
378 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "race"); !store.IsNotFound(err) {
379 t.Fatal("release na verplaatsing")
380 }
381 if got := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/race")); got != moved {
382 t.Fatalf("tag na weigering: %s", got)
383 }
384 existingTagBeforeLock = func(ctx context.Context) error { return ctx.Err() }
385
386 // Lock blokkeert receive-pack tijdens de publicatie.
387 putTag(t, bare, "held", e.sha, false)
388 work := cloneBare(t, bare)
389 existingTagDuringLock = func(context.Context) error {
390 cmd := exec.Command("git", "-C", work, "push", "origin", ":refs/tags/held")
391 if err := cmd.Run(); err == nil {
392 t.Error("push onder de lock moest falen")
393 }
394 if got := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/held")); got != e.sha {
395 t.Errorf("ref tijdens lock: %s", got)
396 }
397 return nil
398 }
399 tok, _, _ = e.runningTagJob("refs/tags/held", "tim", "push", e.sha, nil)
400 if code, raw := e.postExisting(tok, "held", map[string]any{"commit": e.sha, "name": "vast"}); code != 201 {
401 t.Fatalf("tijdens lock: %d %s", code, raw)
402 }
403 if got := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/held")); got != e.sha {
404 t.Fatalf("ref na publicatie: %s", got)
405 }
406 existingTagDuringLock = func(ctx context.Context) error { return ctx.Err() }
407
408 // Fout vóór de insert geeft de lock terug.
409 putTag(t, bare, "stop", e.sha, false)
410 tok, _, _ = e.runningTagJob("refs/tags/stop", "tim", "push", e.sha, nil)
411 ctxCancel, cancel := context.WithCancel(context.Background())
412 existingTagDuringLock = func(ctx context.Context) error {
413 cancel()
414 return ctx.Err()
415 }
416 if _, _, err := e.srv.opPublishExistingTag(ctxCancel, e.repo, tok, "stop", existingReleaseRequest{Commit: e.sha}); err == nil {
417 t.Fatal("cancel moest falen")
418 }
419 existingTagDuringLock = func(ctx context.Context) error { return ctx.Err() }
420 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "stop"); !store.IsNotFound(err) {
421 t.Fatal("release na cancel")
422 }
423 if _, err := os.Stat(filepath.Join(bare, "refs", "tags", "stop.lock")); !os.IsNotExist(err) {
424 t.Fatalf("lock na cancel: %v", err)
425 }
426 if err := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/stop").Run(); err != nil {
427 t.Fatalf("delete na cancel: %v", err)
428 }
429
430 // Twee publicaties: één wint.
431 putTag(t, bare, "duo", e.sha, false)
432 one, _, _ := e.runningTagJob("refs/tags/duo", "tim", "push", e.sha, nil)
433 two, _, _ := e.runningTagJob("refs/tags/duo", "tim", "push", e.sha, nil)
434 events = 0
435 var wg sync.WaitGroup
436 codes := make([]int, 2)
437 wg.Add(2)
438 start := make(chan struct{})
439 for i, token := range []string{one, two} {
440 go func(i int, token string) {
441 defer wg.Done()
442 <-start
443 codes[i], _ = e.postExisting(token, "duo", map[string]any{"commit": e.sha, "name": "een"})
444 }(i, token)
445 }
446 close(start)
447 wg.Wait()
448 sort.Ints(codes)
449 if codes[0] != 201 || codes[1] != 409 {
450 t.Fatalf("gelijktijdig: %v", codes)
451 }
452 list, _ := e.st.ListReleases(ctx, e.repo.ID)
453 if countTag(list, "duo") != 1 {
454 t.Fatalf("dubbele duo")
455 }
456 if events != 1 {
457 t.Fatalf("duo events: %d", events)
458 }
459 }
460
461 func TestPublishExistingTagAssets(t *testing.T) {
462 restoreReleaseSeams(t)
463 e := newCIEnv(t)
464 ctx := context.Background()
465 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
466 putTag(t, bare, "v1", e.sha, false)
467 tok, _, jobs := e.runningTagJob("refs/tags/v1", "tim", "push", e.sha, []string{"build", "other"})
468 e.artifact(t, jobs["build"], "dist/app.bin", []byte("one"), nil)
469 e.artifact(t, jobs["build"], "out/app.bin", []byte("two"), nil)
470 e.artifact(t, jobs["other"], "dist/b.bin", []byte("b"), nil)
471
472 bad := []any{
473 map[string]any{"commit": e.sha, "replace": false},
474 map[string]any{"commit": e.sha, "replace": true},
475 map[string]any{"commit": e.sha, "tag": "andere"},
476 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "../x"}}},
477 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": `a\b`}}},
478 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "."}}},
479 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "a/b"}}},
480 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "a\x00b"}}},
481 map[string]any{"commit": e.sha, "assets": []map[string]any{
482 {"job": "build", "path": "dist/app.bin", "name": "same"},
483 {"job": "other", "path": "dist/b.bin", "name": "same"},
484 }},
485 map[string]any{"commit": e.sha, "assets": []map[string]any{
486 {"job": "build", "path": "dist/app.bin"},
487 {"job": "build", "path": "out/app.bin"},
488 }},
489 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "../secret", "name": "s"}}},
490 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "nope", "path": "dist/app.bin", "name": "a"}}},
491 map[string]any{"commit": e.sha, "name": strings.Repeat("n", maxExistingReleaseName+1)},
492 }
493 for i, body := range bad {
494 if code, raw := e.postExisting(tok, "v1", body); code != 400 {
495 t.Fatalf("payload %d: %d %s", i, code, raw)
496 }
497 }
498 many := make([]map[string]any, maxExistingReleaseAssets+1)
499 for i := range many {
500 many[i] = map[string]any{"job": "build", "path": "dist/app.bin", "name": "f" + strings.Repeat("x", 0) + string(rune('a'+i%26)) + strings.Repeat("y", i)}
501 }
502 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": many}); code != 400 {
503 t.Fatalf("te veel: %d %s", code, raw)
504 }
505 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1"); !store.IsNotFound(err) {
506 t.Fatal("payload maakte een release")
507 }
508
509 past := time.Now().Add(-time.Hour)
510 e.artifact(t, jobs["build"], "dist/old.bin", []byte("old"), &past)
511 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/old.bin", "name": "old.bin"}}}); code != 400 {
512 t.Fatalf("verlopen: %d %s", code, raw)
513 }
514 _ = e.st.FinishJob(ctx, jobs["other"].ID, "failed", "")
515 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "other", "path": "dist/b.bin", "name": "b.bin"}}}); code != 400 {
516 t.Fatalf("mislukte producer: %d %s", code, raw)
517 }
518 _ = e.st.FinishJob(ctx, jobs["other"].ID, "running", "")
519 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "other", "path": "dist/b.bin", "name": "b.bin"}}}); code != 400 {
520 t.Fatalf("lopende producer: %d %s", code, raw)
521 }
522
523 link := filepath.Join(e.srv.StoragePath(), "link-src")
524 if err := os.WriteFile(link+"-real", []byte("secret"), 0o644); err != nil {
525 t.Fatal(err)
526 }
527 if err := os.Symlink(link+"-real", link); err != nil {
528 t.Fatal(err)
529 }
530 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/link", link, 6, nil); err != nil {
531 t.Fatal(err)
532 }
533 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/link", "name": "link"}}}); code != 400 {
534 t.Fatalf("symlink: %d %s", code, raw)
535 }
536 dir := filepath.Join(e.srv.StoragePath(), "a-dir")
537 if err := os.MkdirAll(dir, 0o755); err != nil {
538 t.Fatal(err)
539 }
540 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/dir", dir, 1, nil); err != nil {
541 t.Fatal(err)
542 }
543 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/dir", "name": "dir"}}}); code != 400 {
544 t.Fatalf("directory: %d %s", code, raw)
545 }
546 outside := filepath.Join(t.TempDir(), "outside")
547 if err := os.WriteFile(outside, []byte("nope"), 0o644); err != nil {
548 t.Fatal(err)
549 }
550 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/out", outside, 4, nil); err != nil {
551 t.Fatal(err)
552 }
553 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/out", "name": "out"}}}); code != 400 {
554 t.Fatalf("buiten storage: %d %s", code, raw)
555 }
556 missing := filepath.Join(e.srv.StoragePath(), "gone")
557 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/gone", missing, 1, nil); err != nil {
558 t.Fatal(err)
559 }
560 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/gone", "name": "gone"}}}); code != 400 {
561 t.Fatalf("weg: %d %s", code, raw)
562 }
563
564 otherTok, _, otherJobs := e.runningTagJob("refs/tags/v9", "tim", "push", e.sha, []string{"build"})
565 e.artifact(t, otherJobs["build"], "dist/only-other.bin", []byte("x"), nil)
566 putTag(t, bare, "v9", e.sha, false)
567 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/only-other.bin", "name": "only"}}}); code != 400 {
568 t.Fatalf("andere pipeline: %d %s", code, raw)
569 }
570 _ = otherTok
571
572 var copies int
573 releaseCopyFile = func(src, dst string) error {
574 copies++
575 if copies == 2 {
576 return os.ErrClosed
577 }
578 return copyReleaseExclusive(src, dst)
579 }
580 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{
581 {"job": "build", "path": "dist/app.bin", "name": "one.bin"},
582 {"job": "build", "path": "out/app.bin", "name": "two.bin"},
583 }}); code != 500 {
584 t.Fatalf("kopie: %d %s", code, raw)
585 }
586 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1"); !store.IsNotFound(err) {
587 t.Fatal("deelrelease na kopiefout")
588 }
589 if left := incomingPubs(t, e.srv.StoragePath()); len(left) != 0 {
590 t.Fatalf("staging bleef: %v", left)
591 }
592 releaseCopyFile = copyReleaseExclusive
593 releaseInsert = func(context.Context, *store.Store, int64, string, string, string, string, int64, int64, []store.ReleaseAsset) (int64, error) {
594 return 0, os.ErrClosed
595 }
596 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "one.bin"}}}); code != 500 {
597 t.Fatalf("db: %d %s", code, raw)
598 }
599 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1"); !store.IsNotFound(err) {
600 t.Fatal("release na db-fout")
601 }
602 if left := incomingPubs(t, e.srv.StoragePath()); len(left) != 0 {
603 t.Fatalf("staging na db: %v", left)
604 }
605 orig, _ := os.ReadFile(artifactPath(e, jobs["build"], "dist/app.bin"))
606 if !bytes.Equal(orig, []byte("one")) {
607 t.Fatal("bron beschadigd")
608 }
609 }
610
611 func restoreReleaseSeams(t *testing.T) {
612 t.Helper()
613 before, during := existingTagBeforeLock, existingTagDuringLock
614 copyFn, insert := releaseCopyFile, releaseInsert
615 t.Cleanup(func() {
616 existingTagBeforeLock, existingTagDuringLock = before, during
617 releaseCopyFile, releaseInsert = copyFn, insert
618 })
619 }
620
621 func (e *ciEnv) runningTagJob(ref, actor, source, sha string, producers []string) (string, int64, map[string]store.Job) {
622 return e.runningTagJobRepo(e.repo.ID, ref, actor, source, sha, producers)
623 }
624
625 func (e *ciEnv) runningTagJobRepo(repoID int64, ref, actor, source, sha string, producers []string) (string, int64, map[string]store.Job) {
626 e.t.Helper()
627 ctx := context.Background()
628 spec := []store.NewJob{{Name: "release", Stage: "deploy", Position: 0, Image: "alpine:3.20", Script: "true"}}
629 for i, name := range producers {
630 spec = append(spec, store.NewJob{Name: name, Stage: "build", Position: i + 1, Image: "alpine:3.20", Script: "true"})
631 }
632 pid, err := e.st.CreatePipeline(ctx, repoID, sha, ref, source, actor, "rel", spec)
633 if err != nil {
634 e.t.Fatal(err)
635 }
636 list, err := e.st.JobsByPipeline(ctx, pid)
637 if err != nil {
638 e.t.Fatal(err)
639 }
640 out := map[string]store.Job{}
641 var token string
642 for _, j := range list {
643 out[j.Name] = j
644 if j.Name == "release" {
645 if ok, err := e.st.MarkJobRunning(ctx, j.ID); err != nil || !ok {
646 e.t.Fatalf("running: %v %v", ok, err)
647 }
648 token, err = e.st.CreateJobToken(ctx, j.ID)
649 if err != nil {
650 e.t.Fatal(err)
651 }
652 continue
653 }
654 if err := e.st.FinishJob(ctx, j.ID, "success", ""); err != nil {
655 e.t.Fatal(err)
656 }
657 }
658 return token, pid, out
659 }
660
661 func (e *ciEnv) jobWithStatus(t *testing.T, ref, actor, sha, status string) string {
662 t.Helper()
663 tok, _, jobs := e.runningTagJob(ref, actor, "push", sha, nil)
664 if status == "running" {
665 return tok
666 }
667 ctx := context.Background()
668 switch status {
669 case "queued":
670 if err := e.st.FinishJob(ctx, jobs["release"].ID, "queued", ""); err != nil {
671 t.Fatal(err)
672 }
673 case "canceled":
674 if _, err := e.st.CancelJob(ctx, jobs["release"].ID); err != nil {
675 t.Fatal(err)
676 }
677 default:
678 if err := e.st.FinishJob(ctx, jobs["release"].ID, status, ""); err != nil {
679 t.Fatal(err)
680 }
681 }
682 return tok
683 }
684
685 func (e *ciEnv) artifact(t *testing.T, job store.Job, rel string, body []byte, exp *time.Time) {
686 t.Helper()
687 path := artifactPath(e, job, rel)
688 if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
689 t.Fatal(err)
690 }
691 if err := os.WriteFile(path, body, 0o644); err != nil {
692 t.Fatal(err)
693 }
694 if err := e.st.AddArtifact(context.Background(), job.ID, rel, path, int64(len(body)), exp); err != nil {
695 t.Fatal(err)
696 }
697 }
698
699 func artifactPath(e *ciEnv, job store.Job, rel string) string {
700 return filepath.Join(e.srv.StoragePath(), "artifacts", fmt.Sprintf("%d-%s", job.ID, strings.ReplaceAll(rel, "/", "_")))
701 }
702
703 func cloneBare(t *testing.T, bare string) string {
704 t.Helper()
705 work := filepath.Join(t.TempDir(), "src")
706 cmd := exec.Command("git", "clone", "-q", bare, work)
707 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
708 if out, err := cmd.CombinedOutput(); err != nil {
709 t.Fatalf("clone: %v %s", err, out)
710 }
711 return work
712 }
713
714 func (e *ciEnv) postExisting(token, tag string, body any) (int, string) {
715 e.t.Helper()
716 res := postJSON(e.t, http.MethodPost, e.ts.URL+"/api/v1/repos/tim/app/releases/"+tag+"/assets", token, body)
717 defer res.Body.Close()
718 raw, _ := io.ReadAll(res.Body)
719 return res.StatusCode, string(raw)
720 }
721
722 func putTag(t *testing.T, bare, name, sha string, annotated bool) {
723 t.Helper()
724 if annotated {
725 gitOKServer(t, bare, "tag", "-a", name, sha, "-m", "notitie")
726 return
727 }
728 gitOKServer(t, bare, "tag", name, sha)
729 }
730
731 func gitOKServer(t *testing.T, dir string, args ...string) {
732 t.Helper()
733 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
734 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
735 if out, err := cmd.CombinedOutput(); err != nil {
736 t.Fatalf("git %v: %v %s", args, err, out)
737 }
738 }
739
740 func gitOutServer(t *testing.T, dir string, args ...string) string {
741 t.Helper()
742 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
743 out, err := cmd.Output()
744 if err != nil {
745 t.Fatalf("git %v: %v", args, err)
746 }
747 return string(out)
748 }
749
750 func tagPoint(t *testing.T, bare, name string) (object, commit string) {
751 t.Helper()
752 object = strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/"+name))
753 commit = strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/"+name+"^{commit}"))
754 return object, commit
755 }
756
757 func refList(t *testing.T, bare string) string {
758 t.Helper()
759 out, err := exec.Command("git", "-C", bare, "show-ref").Output()
760 if err != nil {
761 t.Fatal(err)
762 }
763 lines := strings.Split(strings.TrimSpace(string(out)), "\n")
764 sort.Strings(lines)
765 return strings.Join(lines, "\n")
766 }
767
768 func commitExtra(t *testing.T, bare, parent string) string {
769 t.Helper()
770 tree := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", parent+"^{tree}"))
771 cmd := exec.Command("git", "-C", bare, "commit-tree", tree, "-p", parent, "-m", "extra")
772 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
773 out, err := cmd.Output()
774 if err != nil {
775 t.Fatal(err)
776 }
777 return strings.TrimSpace(string(out))
778 }
779
780 func mustUser(t *testing.T, st *store.Store, name string) int64 {
781 t.Helper()
782 u, err := st.UserByUsername(context.Background(), name)
783 if err != nil {
784 t.Fatal(err)
785 }
786 return u.ID
787 }
788
789 func incomingPubs(t *testing.T, storage string) []string {
790 t.Helper()
791 matches, err := filepath.Glob(filepath.Join(storage, "releases", ".incoming", "pub-*"))
792 if err != nil {
793 t.Fatal(err)
794 }
795 return matches
796 }
@@ -0,0 +1,796 @@
1package server
2
3import (
4 "bytes"
5 "context"
6 "fmt"
7 "io"
8 "log/slog"
9 "net/http"
10 "net/http/cookiejar"
11 "os"
12 "os/exec"
13 "path/filepath"
14 "sort"
15 "strings"
16 "sync"
17 "testing"
18 "time"
19
20 "gitlab.com/Timfa/bluegit/internal/access"
21 "gitlab.com/Timfa/bluegit/internal/gitrepo"
22 "gitlab.com/Timfa/bluegit/internal/store"
23)
24
25func TestPublishExistingTag(t *testing.T) {
26 restoreReleaseSeams(t)
27 e := newCIEnv(t)
28 ctx := context.Background()
29 var audits bytes.Buffer
30 e.srv.Log = slog.New(slog.NewTextHandler(&audits, nil))
31 var events int
32 var eventMu sync.Mutex
33 e.srv.hookSink = func([]byte) {
34 eventMu.Lock()
35 events++
36 eventMu.Unlock()
37 }
38 if _, err := e.st.CreateWebhook(ctx, store.Webhook{
39 ScopeKind: "repo", ScopeID: e.repo.ID, Name: "rel", Preset: "generic",
40 URLTemplate: "https://example.test/hook", Events: []string{"release_published"},
41 }); err != nil {
42 t.Fatal(err)
43 }
44 _, _ = e.st.UpsertUser(ctx, 21, "mnt", "", false)
45 _ = e.st.SetMember(ctx, "repo", e.repo.ID, "mnt", access.RoleMaintainer)
46 if err := e.st.AddProtectedTag(ctx, e.repo.ID, "v*"); err != nil {
47 t.Fatal(err)
48 }
49
50 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
51 putTag(t, bare, "v1", e.sha, false)
52 putTag(t, bare, "ann", e.sha, true)
53 beforeRefs := refList(t, bare)
54 beforePipes, _ := e.st.ListPipelines(ctx, e.repo.ID)
55
56 tok, _, jobs := e.runningTagJob("refs/tags/v1", "tim", "push", e.sha, []string{"build"})
57 body := []byte("paperdoll-bytes")
58 e.artifact(t, jobs["build"], "dist/paperdoll-renderer.AppImage", body, nil)
59 code, raw := e.postExisting(tok, "v1", map[string]any{
60 "commit": e.sha, "name": "Paperdoll Renderer v0.1.0", "body": "notities",
61 "assets": []map[string]any{{"job": "build", "path": "dist/paperdoll-renderer.AppImage", "name": "paperdoll-renderer.AppImage"}},
62 })
63 if code != 201 || !strings.Contains(raw, `"tag":"v1"`) || !strings.Contains(raw, `"commit":"`+e.sha+`"`) {
64 t.Fatalf("lightweight: %d %s", code, raw)
65 }
66 rel, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1")
67 if err != nil || rel.Name != "Paperdoll Renderer v0.1.0" || rel.Body != "notities" || rel.CommitSHA != e.sha || len(rel.Assets) != 1 {
68 t.Fatalf("release: %+v %v", rel, err)
69 }
70 if !strings.Contains(rel.Assets[0].StoragePath, string(filepath.Separator)+".incoming"+string(filepath.Separator)) {
71 t.Fatalf("pad niet privé: %s", rel.Assets[0].StoragePath)
72 }
73 det := filepath.Join(e.srv.StoragePath(), "releases", "tim", "app", "v1", "paperdoll-renderer.AppImage")
74 if _, err := os.Stat(det); !os.IsNotExist(err) {
75 t.Fatalf("vast tagpad bestaat: %v", err)
76 }
77 got, err := os.ReadFile(rel.Assets[0].StoragePath)
78 if err != nil || !bytes.Equal(got, body) {
79 t.Fatalf("bytes: %v %q", err, got)
80 }
81 owner := issue(t, e.st, e.user.ID, "api")
82 res := postJSON(t, http.MethodGet, e.ts.URL+"/api/v1/projects/tim/app/releases/v1/assets/paperdoll-renderer.AppImage", owner, nil)
83 down, _ := io.ReadAll(res.Body)
84 res.Body.Close()
85 if res.StatusCode != 200 || !bytes.Equal(down, body) {
86 t.Fatalf("download: %d %q", res.StatusCode, down)
87 }
88 if refList(t, bare) != beforeRefs {
89 t.Fatalf("refs veranderd\nwas:\n%s\nis:\n%s", beforeRefs, refList(t, bare))
90 }
91 if object, commit := tagPoint(t, bare, "v1"); object != e.sha || commit != e.sha {
92 t.Fatalf("lightweight oid %s %s", object, commit)
93 }
94 afterPipes, _ := e.st.ListPipelines(ctx, e.repo.ID)
95 if len(afterPipes) != len(beforePipes)+1 {
96 t.Fatalf("pipelines: %d naar %d", len(beforePipes), len(afterPipes))
97 }
98 if !strings.Contains(audits.String(), "action=create") || !strings.Contains(audits.String(), "tag=v1") {
99 t.Fatalf("audit: %s", audits.String())
100 }
101 if events != 1 {
102 t.Fatalf("events: %d", events)
103 }
104 code, _ = e.postExisting(tok, "v1", map[string]any{"commit": e.sha})
105 if code != 409 || events != 1 {
106 t.Fatalf("retry: %d events %d", code, events)
107 }
108 again, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "v1")
109 if again.ID != rel.ID || again.Body != rel.Body {
110 t.Fatalf("retry wijzigde de release")
111 }
112
113 tok, _, jobs = e.runningTagJob("refs/tags/ann", "mnt", "push", e.sha, []string{"build"})
114 e.artifact(t, jobs["build"], "dist/a.bin", []byte("ann"), nil)
115 code, raw = e.postExisting(tok, "ann", map[string]any{
116 "commit": strings.ToUpper(e.sha), "name": "annotated",
117 "assets": []map[string]any{{"job": "build", "path": "dist/a.bin"}},
118 })
119 if code != 201 {
120 t.Fatalf("annotated: %d %s", code, raw)
121 }
122 obj, commit := tagPoint(t, bare, "ann")
123 if obj == commit || commit != e.sha {
124 t.Fatalf("annotated tag veranderd: %s %s", obj, commit)
125 }
126 annRel, err := e.st.ReleaseByTag(ctx, e.repo.ID, "ann")
127 if err != nil || len(annRel.Assets) != 1 || annRel.Assets[0].Name != "a.bin" {
128 t.Fatalf("defaultnaam: %+v %v", annRel, err)
129 }
130
131 gitOKServer(t, bare, "pack-refs", "--all")
132 putTag(t, bare, "packed", e.sha, false)
133 // pack-refs opnieuw, zodat packed geen losse ref is.
134 gitOKServer(t, bare, "pack-refs", "--all")
135 tok, _, _ = e.runningTagJob("refs/tags/packed", "tim", "push", e.sha, nil)
136 if code, raw = e.postExisting(tok, "packed", map[string]any{"commit": e.sha, "name": "packed"}); code != 201 {
137 t.Fatalf("packed: %d %s", code, raw)
138 }
139 pats, err := e.st.ProtectedTagPatterns(ctx, e.repo.ID)
140 if err != nil || len(pats) != 1 || pats[0] != "v*" {
141 t.Fatalf("bescherming: %v %v", pats, err)
142 }
143
144 spec, err := http.Get(e.ts.URL + "/api/v1/openapi.json")
145 if err != nil {
146 t.Fatal(err)
147 }
148 specBody := read(t, spec)
149 if !strings.Contains(specBody, "/repos/{owner}/{repo}/releases/{tag}/assets") || !strings.Contains(specBody, "Job token only") {
150 t.Fatal("openapi mist de route")
151 }
152}
153
154func TestPublishExistingTagAuth(t *testing.T) {
155 restoreReleaseSeams(t)
156 e := newCIEnv(t)
157 ctx := context.Background()
158 _, _ = e.st.UpsertUser(ctx, 20, "dev", "", false)
159 _ = e.st.SetMember(ctx, "repo", e.repo.ID, "dev", access.RoleDeveloper)
160 _, _ = e.st.UpsertUser(ctx, 21, "mnt", "", false)
161 _ = e.st.SetMember(ctx, "repo", e.repo.ID, "mnt", access.RoleMaintainer)
162 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
163 putTag(t, bare, "v1", e.sha, false)
164 otherSHA := commitExtra(t, bare, e.sha)
165
166 okTok, _, _ := e.runningTagJob("refs/tags/v1", "tim", "push", e.sha, nil)
167 if code, raw := e.postExisting("", "v1", map[string]any{"commit": e.sha}); code != 401 {
168 t.Fatalf("geen token: %d %s", code, raw)
169 }
170 if code, raw := e.postExisting("bgj_onzin", "v1", map[string]any{"commit": e.sha}); code != 401 {
171 t.Fatalf("malformed: %d %s", code, raw)
172 }
173 pat := issue(t, e.st, e.user.ID, "api")
174 if code, raw := e.postExisting(pat, "v1", map[string]any{"commit": e.sha}); code != 401 {
175 t.Fatalf("pat: %d %s", code, raw)
176 }
177 jar, _ := cookiejar.New(nil)
178 browser := &http.Client{Jar: jar}
179 login, err := http.NewRequest(http.MethodGet, e.ts.URL+"/login", nil)
180 if err != nil {
181 t.Fatal(err)
182 }
183 res, err := browser.Do(login)
184 if err != nil {
185 t.Fatal(err)
186 }
187 res.Body.Close()
188 sessReq, _ := http.NewRequest(http.MethodPost, e.ts.URL+"/api/v1/repos/tim/app/releases/v1/assets", strings.NewReader(`{"commit":"`+e.sha+`"}`))
189 sessReq.Header.Set("Content-Type", "application/json")
190 sessRes, err := browser.Do(sessReq)
191 if err != nil {
192 t.Fatal(err)
193 }
194 sessBody := read(t, sessRes)
195 if sessRes.StatusCode != 401 {
196 t.Fatalf("sessie: %d %s", sessRes.StatusCode, sessBody)
197 }
198
199 queued := e.jobWithStatus(t, "refs/tags/v1", "tim", e.sha, "queued")
200 done := e.jobWithStatus(t, "refs/tags/v1", "tim", e.sha, "success")
201 canceled := e.jobWithStatus(t, "refs/tags/v1", "tim", e.sha, "canceled")
202 for _, tok := range []string{queued, done, canceled} {
203 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha}); code != 401 {
204 t.Fatalf("status: %d %s", code, raw)
205 }
206 }
207 other, err := e.st.CreateRepo(ctx, "user", e.user.ID, "other", "", "master", "private")
208 if err != nil {
209 t.Fatal(err)
210 }
211 otherTok, _, _ := e.runningTagJobRepo(other.ID, "refs/tags/v1", "tim", "push", e.sha, nil)
212 if code, raw := e.postExisting(otherTok, "v1", map[string]any{"commit": e.sha}); code != 401 {
213 t.Fatalf("ander project: %d %s", code, raw)
214 }
215
216 getRes := postJSON(t, http.MethodGet, e.ts.URL+"/api/v1/projects/tim/app", okTok, nil)
217 getBody := read(t, getRes)
218 if getRes.StatusCode != 200 || !strings.Contains(getBody, `"path":"tim/app"`) {
219 t.Fatalf("job GET: %d %s", getRes.StatusCode, getBody)
220 }
221 putRes := postJSON(t, http.MethodPut, e.ts.URL+"/api/v1/projects/tim/app", okTok, map[string]any{"description": "nee"})
222 putRes.Body.Close()
223 if putRes.StatusCode != 401 {
224 t.Fatalf("job PUT: %d", putRes.StatusCode)
225 }
226
227 for _, actor := range []string{"dev", "nobody", ""} {
228 tok, _, _ := e.runningTagJob("refs/tags/v1", actor, "push", e.sha, nil)
229 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha}); code != 403 {
230 t.Fatalf("actor %q: %d %s", actor, code, raw)
231 }
232 }
233 _ = e.st.RemoveMember(ctx, "repo", e.repo.ID, "mnt")
234 tok, _, _ := e.runningTagJob("refs/tags/v1", "mnt", "push", e.sha, nil)
235 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha}); code != 403 {
236 t.Fatalf("ingetrokken: %d %s", code, raw)
237 }
238 if _, err := e.st.SetInstanceAdmin(ctx, e.user.ID, mustUser(t, e.st, "dev"), true); err != nil {
239 t.Fatal(err)
240 }
241 _ = e.st.RemoveMember(ctx, "repo", e.repo.ID, "dev")
242 tok, _, _ = e.runningTagJob("refs/tags/v1", "dev", "push", e.sha, nil)
243 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "name": "admin"}); code != 201 {
244 t.Fatalf("instance-admin: %d %s", code, raw)
245 }
246 if _, err := e.st.SetInstanceAdmin(ctx, e.user.ID, mustUser(t, e.st, "dev"), false); err != nil {
247 t.Fatal(err)
248 }
249 putTag(t, bare, "vno", e.sha, false)
250 tok, _, _ = e.runningTagJob("refs/tags/vno", "dev", "push", e.sha, nil)
251 if code, raw := e.postExisting(tok, "vno", map[string]any{"commit": e.sha}); code != 403 {
252 t.Fatalf("admin ingetrokken: %d %s", code, raw)
253 }
254
255 cases := []struct {
256 ref, source, sha, url, commit string
257 mr bool
258 want int
259 }{
260 {"refs/heads/master", "push", e.sha, "v1", e.sha, false, 403},
261 {"refs/heads/forks/7/topic", "push", e.sha, "v1", e.sha, false, 403},
262 {"refs/tags/v1", "web", e.sha, "v1", e.sha, false, 403},
263 {"refs/tags/other", "push", e.sha, "v1", e.sha, false, 403},
264 {"refs/tags/v1", "push", e.sha, "v1", e.sha, true, 403},
265 {"refs/tags/v1", "push", otherSHA, "v1", e.sha, false, 409},
266 {"refs/tags/v1", "push", e.sha, "v1", otherSHA, false, 409},
267 {"refs/tags/v1", "push", e.sha, "v1", e.sha[:12], false, 400},
268 }
269 for _, tc := range cases {
270 tok, pid, _ := e.runningTagJob(tc.ref, "tim", tc.source, tc.sha, nil)
271 if tc.mr {
272 if err := e.st.AttachPipelineMR(ctx, pid, 4); err != nil {
273 t.Fatal(err)
274 }
275 }
276 if code, raw := e.postExisting(tok, tc.url, map[string]any{"commit": tc.commit}); code != tc.want {
277 t.Fatalf("%+v: %d %s", tc, code, raw)
278 }
279 }
280 missTok, _, _ := e.runningTagJob("refs/tags/missing", "tim", "push", e.sha, nil)
281 if code, raw := e.postExisting(missTok, "missing", map[string]any{"commit": e.sha}); code != 404 {
282 t.Fatalf("tag weg: %d %s", code, raw)
283 }
284 gitOKServer(t, bare, "symbolic-ref", "refs/tags/sym", "refs/heads/master")
285 tok, _, _ = e.runningTagJob("refs/tags/sym", "tim", "push", e.sha, nil)
286 if code, raw := e.postExisting(tok, "sym", map[string]any{"commit": e.sha}); code != 400 {
287 t.Fatalf("symbolisch: %d %s", code, raw)
288 }
289 tree := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "HEAD^{tree}"))
290 gitOKServer(t, bare, "update-ref", "refs/tags/boom", tree)
291 tok, _, _ = e.runningTagJob("refs/tags/boom", "tim", "push", e.sha, nil)
292 if code, raw := e.postExisting(tok, "boom", map[string]any{"commit": e.sha}); code != 400 {
293 t.Fatalf("geen commit: %d %s", code, raw)
294 }
295 putTag(t, bare, "ann", e.sha, true)
296 annObj, _ := tagPoint(t, bare, "ann")
297 tok, _, _ = e.runningTagJob("refs/tags/ann", "tim", "push", e.sha, nil)
298 if code, raw := e.postExisting(tok, "ann", map[string]any{"commit": annObj}); code != 409 {
299 t.Fatalf("tagobject als commit: %d %s", code, raw)
300 }
301 if code, raw := e.postExisting(okTok, "v1", map[string]any{}); code != 400 {
302 t.Fatalf("commit afwezig: %d %s", code, raw)
303 }
304 if code, raw := e.postExisting(okTok, "../x", map[string]any{"commit": e.sha}); code != 400 && code != 404 {
305 t.Fatalf("slechte tag: %d %s", code, raw)
306 }
307}
308
309func TestPublishExistingTagConflictAndLock(t *testing.T) {
310 restoreReleaseSeams(t)
311 e := newCIEnv(t)
312 ctx := context.Background()
313 var events int
314 var mu sync.Mutex
315 e.srv.hookSink = func([]byte) {
316 mu.Lock()
317 events++
318 mu.Unlock()
319 }
320 if _, err := e.st.CreateWebhook(ctx, store.Webhook{
321 ScopeKind: "repo", ScopeID: e.repo.ID, Name: "rel", Preset: "generic",
322 URLTemplate: "https://example.test/hook", Events: []string{"release_published"},
323 }); err != nil {
324 t.Fatal(err)
325 }
326 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
327 user := issue(t, e.st, e.user.ID, "api")
328 made := postJSON(t, http.MethodPost, e.ts.URL+"/api/v1/projects/tim/app/releases", user, map[string]any{"tag": "hand", "commit": e.sha, "name": "met de hand", "body": "blijft"})
329 madeBody := read(t, made)
330 if made.StatusCode != 201 {
331 t.Fatalf("handmatig: %d %s", made.StatusCode, madeBody)
332 }
333 hand, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "hand")
334 tok, _, _ := e.runningTagJob("refs/tags/hand", "tim", "push", e.sha, nil)
335 if code, raw := e.postExisting(tok, "hand", map[string]any{"commit": e.sha, "name": "nieuw"}); code != 409 {
336 t.Fatalf("hand 409: %d %s", code, raw)
337 }
338 still, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "hand")
339 if still.ID != hand.ID || still.Name != "met de hand" || still.Body != "blijft" {
340 t.Fatalf("hand gewijzigd: %+v", still)
341 }
342
343 btok, _, bjobs := e.runningTagJob("refs/heads/topic", "tim", "push", e.sha, []string{"build"})
344 payload := []byte("branch-asset")
345 e.artifact(t, bjobs["build"], "dist/a.bin", payload, nil)
346 branchRes := postJSON(t, http.MethodPost, e.ts.URL+"/api/v1/repos/tim/app/releases", btok, map[string]any{
347 "tag": "from-branch", "commit": e.sha,
348 "assets": []map[string]any{{"job": "build", "path": "dist/a.bin", "name": "a.bin"}},
349 })
350 branchBody := read(t, branchRes)
351 if branchRes.StatusCode != 201 {
352 t.Fatalf("branchrelease: %d %s", branchRes.StatusCode, branchBody)
353 }
354 // De oude API maakt zelf de tag. Die release blijft 409 voor de nieuwe route.
355 old, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "from-branch")
356 oldBytes, _ := os.ReadFile(old.Assets[0].StoragePath)
357 tok, _, _ = e.runningTagJob("refs/tags/from-branch", "tim", "push", e.sha, nil)
358 if code, raw := e.postExisting(tok, "from-branch", map[string]any{"commit": e.sha}); code != 409 {
359 t.Fatalf("branch 409: %d %s", code, raw)
360 }
361 nowBytes, _ := os.ReadFile(old.Assets[0].StoragePath)
362 kept, _ := e.st.ReleaseByTag(ctx, e.repo.ID, "from-branch")
363 if kept.ID != old.ID || !bytes.Equal(oldBytes, nowBytes) || !bytes.Equal(nowBytes, payload) {
364 t.Fatal("bestaande asset veranderd")
365 }
366
367 // Tag wijzigt tussen resolve en prepare.
368 putTag(t, bare, "race", e.sha, false)
369 moved := commitExtra(t, bare, e.sha)
370 existingTagBeforeLock = func(context.Context) error {
371 gitOKServer(t, bare, "update-ref", "refs/tags/race", moved)
372 return nil
373 }
374 tok, _, _ = e.runningTagJob("refs/tags/race", "tim", "push", e.sha, nil)
375 if code, raw := e.postExisting(tok, "race", map[string]any{"commit": e.sha}); code != 409 {
376 t.Fatalf("verplaatst: %d %s", code, raw)
377 }
378 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "race"); !store.IsNotFound(err) {
379 t.Fatal("release na verplaatsing")
380 }
381 if got := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/race")); got != moved {
382 t.Fatalf("tag na weigering: %s", got)
383 }
384 existingTagBeforeLock = func(ctx context.Context) error { return ctx.Err() }
385
386 // Lock blokkeert receive-pack tijdens de publicatie.
387 putTag(t, bare, "held", e.sha, false)
388 work := cloneBare(t, bare)
389 existingTagDuringLock = func(context.Context) error {
390 cmd := exec.Command("git", "-C", work, "push", "origin", ":refs/tags/held")
391 if err := cmd.Run(); err == nil {
392 t.Error("push onder de lock moest falen")
393 }
394 if got := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/held")); got != e.sha {
395 t.Errorf("ref tijdens lock: %s", got)
396 }
397 return nil
398 }
399 tok, _, _ = e.runningTagJob("refs/tags/held", "tim", "push", e.sha, nil)
400 if code, raw := e.postExisting(tok, "held", map[string]any{"commit": e.sha, "name": "vast"}); code != 201 {
401 t.Fatalf("tijdens lock: %d %s", code, raw)
402 }
403 if got := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/held")); got != e.sha {
404 t.Fatalf("ref na publicatie: %s", got)
405 }
406 existingTagDuringLock = func(ctx context.Context) error { return ctx.Err() }
407
408 // Fout vóór de insert geeft de lock terug.
409 putTag(t, bare, "stop", e.sha, false)
410 tok, _, _ = e.runningTagJob("refs/tags/stop", "tim", "push", e.sha, nil)
411 ctxCancel, cancel := context.WithCancel(context.Background())
412 existingTagDuringLock = func(ctx context.Context) error {
413 cancel()
414 return ctx.Err()
415 }
416 if _, _, err := e.srv.opPublishExistingTag(ctxCancel, e.repo, tok, "stop", existingReleaseRequest{Commit: e.sha}); err == nil {
417 t.Fatal("cancel moest falen")
418 }
419 existingTagDuringLock = func(ctx context.Context) error { return ctx.Err() }
420 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "stop"); !store.IsNotFound(err) {
421 t.Fatal("release na cancel")
422 }
423 if _, err := os.Stat(filepath.Join(bare, "refs", "tags", "stop.lock")); !os.IsNotExist(err) {
424 t.Fatalf("lock na cancel: %v", err)
425 }
426 if err := exec.Command("git", "-C", bare, "update-ref", "-d", "refs/tags/stop").Run(); err != nil {
427 t.Fatalf("delete na cancel: %v", err)
428 }
429
430 // Twee publicaties: één wint.
431 putTag(t, bare, "duo", e.sha, false)
432 one, _, _ := e.runningTagJob("refs/tags/duo", "tim", "push", e.sha, nil)
433 two, _, _ := e.runningTagJob("refs/tags/duo", "tim", "push", e.sha, nil)
434 events = 0
435 var wg sync.WaitGroup
436 codes := make([]int, 2)
437 wg.Add(2)
438 start := make(chan struct{})
439 for i, token := range []string{one, two} {
440 go func(i int, token string) {
441 defer wg.Done()
442 <-start
443 codes[i], _ = e.postExisting(token, "duo", map[string]any{"commit": e.sha, "name": "een"})
444 }(i, token)
445 }
446 close(start)
447 wg.Wait()
448 sort.Ints(codes)
449 if codes[0] != 201 || codes[1] != 409 {
450 t.Fatalf("gelijktijdig: %v", codes)
451 }
452 list, _ := e.st.ListReleases(ctx, e.repo.ID)
453 if countTag(list, "duo") != 1 {
454 t.Fatalf("dubbele duo")
455 }
456 if events != 1 {
457 t.Fatalf("duo events: %d", events)
458 }
459}
460
461func TestPublishExistingTagAssets(t *testing.T) {
462 restoreReleaseSeams(t)
463 e := newCIEnv(t)
464 ctx := context.Background()
465 bare := gitrepo.Dir(e.srv.StoragePath(), e.repo.OwnerPath, e.repo.Name)
466 putTag(t, bare, "v1", e.sha, false)
467 tok, _, jobs := e.runningTagJob("refs/tags/v1", "tim", "push", e.sha, []string{"build", "other"})
468 e.artifact(t, jobs["build"], "dist/app.bin", []byte("one"), nil)
469 e.artifact(t, jobs["build"], "out/app.bin", []byte("two"), nil)
470 e.artifact(t, jobs["other"], "dist/b.bin", []byte("b"), nil)
471
472 bad := []any{
473 map[string]any{"commit": e.sha, "replace": false},
474 map[string]any{"commit": e.sha, "replace": true},
475 map[string]any{"commit": e.sha, "tag": "andere"},
476 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "../x"}}},
477 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": `a\b`}}},
478 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "."}}},
479 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "a/b"}}},
480 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "a\x00b"}}},
481 map[string]any{"commit": e.sha, "assets": []map[string]any{
482 {"job": "build", "path": "dist/app.bin", "name": "same"},
483 {"job": "other", "path": "dist/b.bin", "name": "same"},
484 }},
485 map[string]any{"commit": e.sha, "assets": []map[string]any{
486 {"job": "build", "path": "dist/app.bin"},
487 {"job": "build", "path": "out/app.bin"},
488 }},
489 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "../secret", "name": "s"}}},
490 map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "nope", "path": "dist/app.bin", "name": "a"}}},
491 map[string]any{"commit": e.sha, "name": strings.Repeat("n", maxExistingReleaseName+1)},
492 }
493 for i, body := range bad {
494 if code, raw := e.postExisting(tok, "v1", body); code != 400 {
495 t.Fatalf("payload %d: %d %s", i, code, raw)
496 }
497 }
498 many := make([]map[string]any, maxExistingReleaseAssets+1)
499 for i := range many {
500 many[i] = map[string]any{"job": "build", "path": "dist/app.bin", "name": "f" + strings.Repeat("x", 0) + string(rune('a'+i%26)) + strings.Repeat("y", i)}
501 }
502 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": many}); code != 400 {
503 t.Fatalf("te veel: %d %s", code, raw)
504 }
505 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1"); !store.IsNotFound(err) {
506 t.Fatal("payload maakte een release")
507 }
508
509 past := time.Now().Add(-time.Hour)
510 e.artifact(t, jobs["build"], "dist/old.bin", []byte("old"), &past)
511 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/old.bin", "name": "old.bin"}}}); code != 400 {
512 t.Fatalf("verlopen: %d %s", code, raw)
513 }
514 _ = e.st.FinishJob(ctx, jobs["other"].ID, "failed", "")
515 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "other", "path": "dist/b.bin", "name": "b.bin"}}}); code != 400 {
516 t.Fatalf("mislukte producer: %d %s", code, raw)
517 }
518 _ = e.st.FinishJob(ctx, jobs["other"].ID, "running", "")
519 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "other", "path": "dist/b.bin", "name": "b.bin"}}}); code != 400 {
520 t.Fatalf("lopende producer: %d %s", code, raw)
521 }
522
523 link := filepath.Join(e.srv.StoragePath(), "link-src")
524 if err := os.WriteFile(link+"-real", []byte("secret"), 0o644); err != nil {
525 t.Fatal(err)
526 }
527 if err := os.Symlink(link+"-real", link); err != nil {
528 t.Fatal(err)
529 }
530 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/link", link, 6, nil); err != nil {
531 t.Fatal(err)
532 }
533 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/link", "name": "link"}}}); code != 400 {
534 t.Fatalf("symlink: %d %s", code, raw)
535 }
536 dir := filepath.Join(e.srv.StoragePath(), "a-dir")
537 if err := os.MkdirAll(dir, 0o755); err != nil {
538 t.Fatal(err)
539 }
540 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/dir", dir, 1, nil); err != nil {
541 t.Fatal(err)
542 }
543 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/dir", "name": "dir"}}}); code != 400 {
544 t.Fatalf("directory: %d %s", code, raw)
545 }
546 outside := filepath.Join(t.TempDir(), "outside")
547 if err := os.WriteFile(outside, []byte("nope"), 0o644); err != nil {
548 t.Fatal(err)
549 }
550 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/out", outside, 4, nil); err != nil {
551 t.Fatal(err)
552 }
553 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/out", "name": "out"}}}); code != 400 {
554 t.Fatalf("buiten storage: %d %s", code, raw)
555 }
556 missing := filepath.Join(e.srv.StoragePath(), "gone")
557 if err := e.st.AddArtifact(ctx, jobs["build"].ID, "dist/gone", missing, 1, nil); err != nil {
558 t.Fatal(err)
559 }
560 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/gone", "name": "gone"}}}); code != 400 {
561 t.Fatalf("weg: %d %s", code, raw)
562 }
563
564 otherTok, _, otherJobs := e.runningTagJob("refs/tags/v9", "tim", "push", e.sha, []string{"build"})
565 e.artifact(t, otherJobs["build"], "dist/only-other.bin", []byte("x"), nil)
566 putTag(t, bare, "v9", e.sha, false)
567 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/only-other.bin", "name": "only"}}}); code != 400 {
568 t.Fatalf("andere pipeline: %d %s", code, raw)
569 }
570 _ = otherTok
571
572 var copies int
573 releaseCopyFile = func(src, dst string) error {
574 copies++
575 if copies == 2 {
576 return os.ErrClosed
577 }
578 return copyReleaseExclusive(src, dst)
579 }
580 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{
581 {"job": "build", "path": "dist/app.bin", "name": "one.bin"},
582 {"job": "build", "path": "out/app.bin", "name": "two.bin"},
583 }}); code != 500 {
584 t.Fatalf("kopie: %d %s", code, raw)
585 }
586 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1"); !store.IsNotFound(err) {
587 t.Fatal("deelrelease na kopiefout")
588 }
589 if left := incomingPubs(t, e.srv.StoragePath()); len(left) != 0 {
590 t.Fatalf("staging bleef: %v", left)
591 }
592 releaseCopyFile = copyReleaseExclusive
593 releaseInsert = func(context.Context, *store.Store, int64, string, string, string, string, int64, int64, []store.ReleaseAsset) (int64, error) {
594 return 0, os.ErrClosed
595 }
596 if code, raw := e.postExisting(tok, "v1", map[string]any{"commit": e.sha, "assets": []map[string]any{{"job": "build", "path": "dist/app.bin", "name": "one.bin"}}}); code != 500 {
597 t.Fatalf("db: %d %s", code, raw)
598 }
599 if _, err := e.st.ReleaseByTag(ctx, e.repo.ID, "v1"); !store.IsNotFound(err) {
600 t.Fatal("release na db-fout")
601 }
602 if left := incomingPubs(t, e.srv.StoragePath()); len(left) != 0 {
603 t.Fatalf("staging na db: %v", left)
604 }
605 orig, _ := os.ReadFile(artifactPath(e, jobs["build"], "dist/app.bin"))
606 if !bytes.Equal(orig, []byte("one")) {
607 t.Fatal("bron beschadigd")
608 }
609}
610
611func restoreReleaseSeams(t *testing.T) {
612 t.Helper()
613 before, during := existingTagBeforeLock, existingTagDuringLock
614 copyFn, insert := releaseCopyFile, releaseInsert
615 t.Cleanup(func() {
616 existingTagBeforeLock, existingTagDuringLock = before, during
617 releaseCopyFile, releaseInsert = copyFn, insert
618 })
619}
620
621func (e *ciEnv) runningTagJob(ref, actor, source, sha string, producers []string) (string, int64, map[string]store.Job) {
622 return e.runningTagJobRepo(e.repo.ID, ref, actor, source, sha, producers)
623}
624
625func (e *ciEnv) runningTagJobRepo(repoID int64, ref, actor, source, sha string, producers []string) (string, int64, map[string]store.Job) {
626 e.t.Helper()
627 ctx := context.Background()
628 spec := []store.NewJob{{Name: "release", Stage: "deploy", Position: 0, Image: "alpine:3.20", Script: "true"}}
629 for i, name := range producers {
630 spec = append(spec, store.NewJob{Name: name, Stage: "build", Position: i + 1, Image: "alpine:3.20", Script: "true"})
631 }
632 pid, err := e.st.CreatePipeline(ctx, repoID, sha, ref, source, actor, "rel", spec)
633 if err != nil {
634 e.t.Fatal(err)
635 }
636 list, err := e.st.JobsByPipeline(ctx, pid)
637 if err != nil {
638 e.t.Fatal(err)
639 }
640 out := map[string]store.Job{}
641 var token string
642 for _, j := range list {
643 out[j.Name] = j
644 if j.Name == "release" {
645 if ok, err := e.st.MarkJobRunning(ctx, j.ID); err != nil || !ok {
646 e.t.Fatalf("running: %v %v", ok, err)
647 }
648 token, err = e.st.CreateJobToken(ctx, j.ID)
649 if err != nil {
650 e.t.Fatal(err)
651 }
652 continue
653 }
654 if err := e.st.FinishJob(ctx, j.ID, "success", ""); err != nil {
655 e.t.Fatal(err)
656 }
657 }
658 return token, pid, out
659}
660
661func (e *ciEnv) jobWithStatus(t *testing.T, ref, actor, sha, status string) string {
662 t.Helper()
663 tok, _, jobs := e.runningTagJob(ref, actor, "push", sha, nil)
664 if status == "running" {
665 return tok
666 }
667 ctx := context.Background()
668 switch status {
669 case "queued":
670 if err := e.st.FinishJob(ctx, jobs["release"].ID, "queued", ""); err != nil {
671 t.Fatal(err)
672 }
673 case "canceled":
674 if _, err := e.st.CancelJob(ctx, jobs["release"].ID); err != nil {
675 t.Fatal(err)
676 }
677 default:
678 if err := e.st.FinishJob(ctx, jobs["release"].ID, status, ""); err != nil {
679 t.Fatal(err)
680 }
681 }
682 return tok
683}
684
685func (e *ciEnv) artifact(t *testing.T, job store.Job, rel string, body []byte, exp *time.Time) {
686 t.Helper()
687 path := artifactPath(e, job, rel)
688 if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil {
689 t.Fatal(err)
690 }
691 if err := os.WriteFile(path, body, 0o644); err != nil {
692 t.Fatal(err)
693 }
694 if err := e.st.AddArtifact(context.Background(), job.ID, rel, path, int64(len(body)), exp); err != nil {
695 t.Fatal(err)
696 }
697}
698
699func artifactPath(e *ciEnv, job store.Job, rel string) string {
700 return filepath.Join(e.srv.StoragePath(), "artifacts", fmt.Sprintf("%d-%s", job.ID, strings.ReplaceAll(rel, "/", "_")))
701}
702
703func cloneBare(t *testing.T, bare string) string {
704 t.Helper()
705 work := filepath.Join(t.TempDir(), "src")
706 cmd := exec.Command("git", "clone", "-q", bare, work)
707 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
708 if out, err := cmd.CombinedOutput(); err != nil {
709 t.Fatalf("clone: %v %s", err, out)
710 }
711 return work
712}
713
714func (e *ciEnv) postExisting(token, tag string, body any) (int, string) {
715 e.t.Helper()
716 res := postJSON(e.t, http.MethodPost, e.ts.URL+"/api/v1/repos/tim/app/releases/"+tag+"/assets", token, body)
717 defer res.Body.Close()
718 raw, _ := io.ReadAll(res.Body)
719 return res.StatusCode, string(raw)
720}
721
722func putTag(t *testing.T, bare, name, sha string, annotated bool) {
723 t.Helper()
724 if annotated {
725 gitOKServer(t, bare, "tag", "-a", name, sha, "-m", "notitie")
726 return
727 }
728 gitOKServer(t, bare, "tag", name, sha)
729}
730
731func gitOKServer(t *testing.T, dir string, args ...string) {
732 t.Helper()
733 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
734 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
735 if out, err := cmd.CombinedOutput(); err != nil {
736 t.Fatalf("git %v: %v %s", args, err, out)
737 }
738}
739
740func gitOutServer(t *testing.T, dir string, args ...string) string {
741 t.Helper()
742 cmd := exec.Command("git", append([]string{"-C", dir}, args...)...)
743 out, err := cmd.Output()
744 if err != nil {
745 t.Fatalf("git %v: %v", args, err)
746 }
747 return string(out)
748}
749
750func tagPoint(t *testing.T, bare, name string) (object, commit string) {
751 t.Helper()
752 object = strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/"+name))
753 commit = strings.TrimSpace(gitOutServer(t, bare, "rev-parse", "refs/tags/"+name+"^{commit}"))
754 return object, commit
755}
756
757func refList(t *testing.T, bare string) string {
758 t.Helper()
759 out, err := exec.Command("git", "-C", bare, "show-ref").Output()
760 if err != nil {
761 t.Fatal(err)
762 }
763 lines := strings.Split(strings.TrimSpace(string(out)), "\n")
764 sort.Strings(lines)
765 return strings.Join(lines, "\n")
766}
767
768func commitExtra(t *testing.T, bare, parent string) string {
769 t.Helper()
770 tree := strings.TrimSpace(gitOutServer(t, bare, "rev-parse", parent+"^{tree}"))
771 cmd := exec.Command("git", "-C", bare, "commit-tree", tree, "-p", parent, "-m", "extra")
772 cmd.Env = append(os.Environ(), "GIT_AUTHOR_NAME=t", "GIT_AUTHOR_EMAIL=t@x", "GIT_COMMITTER_NAME=t", "GIT_COMMITTER_EMAIL=t@x")
773 out, err := cmd.Output()
774 if err != nil {
775 t.Fatal(err)
776 }
777 return strings.TrimSpace(string(out))
778}
779
780func mustUser(t *testing.T, st *store.Store, name string) int64 {
781 t.Helper()
782 u, err := st.UserByUsername(context.Background(), name)
783 if err != nil {
784 t.Fatal(err)
785 }
786 return u.ID
787}
788
789func incomingPubs(t *testing.T, storage string) []string {
790 t.Helper()
791 matches, err := filepath.Glob(filepath.Join(storage, "releases", ".incoming", "pub-*"))
792 if err != nil {
793 t.Fatal(err)
794 }
795 return matches
796}
internal/store/artifact_publish_test.go added +90 −0
@@ -0,0 +1,90 @@
1 package store
2
3 import (
4 "context"
5 "errors"
6 "testing"
7 "time"
8 )
9
10 func TestArtifactForPublishHonorsPipelineAndExpiry(t *testing.T) {
11 s := openTest(t)
12 ctx := context.Background()
13 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
14 if err != nil {
15 t.Fatal(err)
16 }
17 repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
18 if err != nil {
19 t.Fatal(err)
20 }
21 sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
22 jobs := []NewJob{
23 {Name: "build", Stage: "build", Image: "alpine:3.20", Script: "true"},
24 {Name: "fail", Stage: "build", Image: "alpine:3.20", Script: "false"},
25 }
26 pid, err := s.CreatePipeline(ctx, repo.ID, sha, "refs/tags/v1", "push", "tim", "rel", jobs)
27 if err != nil {
28 t.Fatal(err)
29 }
30 other, err := s.CreatePipeline(ctx, repo.ID, sha, "refs/tags/v1", "push", "tim", "rel", jobs[:1])
31 if err != nil {
32 t.Fatal(err)
33 }
34 got, err := s.JobsByPipeline(ctx, pid)
35 if err != nil {
36 t.Fatal(err)
37 }
38 var build, fail Job
39 for _, j := range got {
40 switch j.Name {
41 case "build":
42 build = j
43 case "fail":
44 fail = j
45 }
46 }
47 if err := s.FinishJob(ctx, build.ID, "success", ""); err != nil {
48 t.Fatal(err)
49 }
50 if err := s.FinishJob(ctx, fail.ID, "failed", ""); err != nil {
51 t.Fatal(err)
52 }
53 later := time.Now().Add(time.Hour)
54 past := time.Now().Add(-time.Hour)
55 if err := s.AddArtifact(ctx, build.ID, "dist/app", "/data/app", 3, &later); err != nil {
56 t.Fatal(err)
57 }
58 if err := s.AddArtifact(ctx, build.ID, "dist/old", "/data/old", 3, &past); err != nil {
59 t.Fatal(err)
60 }
61 if err := s.AddArtifact(ctx, fail.ID, "dist/bad", "/data/bad", 3, nil); err != nil {
62 t.Fatal(err)
63 }
64 otherJobs, err := s.JobsByPipeline(ctx, other)
65 if err != nil {
66 t.Fatal(err)
67 }
68 if err := s.FinishJob(ctx, otherJobs[0].ID, "success", ""); err != nil {
69 t.Fatal(err)
70 }
71 if err := s.AddArtifact(ctx, otherJobs[0].ID, "dist/other", "/data/other", 3, nil); err != nil {
72 t.Fatal(err)
73 }
74 path, err := s.ArtifactForPublish(ctx, pid, "build", "dist/app")
75 if err != nil || path != "/data/app" {
76 t.Fatalf("geldig: %q %v", path, err)
77 }
78 if _, err := s.ArtifactForPublish(ctx, pid, "build", "dist/old"); !errors.Is(err, ErrArtifactExpired) {
79 t.Fatalf("verlopen: %v", err)
80 }
81 if _, err := s.ArtifactForPublish(ctx, pid, "fail", "dist/bad"); !IsNotFound(err) {
82 t.Fatalf("mislukt: %v", err)
83 }
84 if _, err := s.ArtifactForPublish(ctx, pid, "build", "dist/other"); !IsNotFound(err) {
85 t.Fatalf("andere pipeline: %v", err)
86 }
87 if _, err := s.ArtifactPath(ctx, pid, "build", "dist/old"); err != nil {
88 t.Fatalf("ArtifactPath blijft verlopen bestanden teruggeven: %v", err)
89 }
90 }
@@ -0,0 +1,90 @@
1package store
2
3import (
4 "context"
5 "errors"
6 "testing"
7 "time"
8)
9
10func TestArtifactForPublishHonorsPipelineAndExpiry(t *testing.T) {
11 s := openTest(t)
12 ctx := context.Background()
13 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
14 if err != nil {
15 t.Fatal(err)
16 }
17 repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
18 if err != nil {
19 t.Fatal(err)
20 }
21 sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
22 jobs := []NewJob{
23 {Name: "build", Stage: "build", Image: "alpine:3.20", Script: "true"},
24 {Name: "fail", Stage: "build", Image: "alpine:3.20", Script: "false"},
25 }
26 pid, err := s.CreatePipeline(ctx, repo.ID, sha, "refs/tags/v1", "push", "tim", "rel", jobs)
27 if err != nil {
28 t.Fatal(err)
29 }
30 other, err := s.CreatePipeline(ctx, repo.ID, sha, "refs/tags/v1", "push", "tim", "rel", jobs[:1])
31 if err != nil {
32 t.Fatal(err)
33 }
34 got, err := s.JobsByPipeline(ctx, pid)
35 if err != nil {
36 t.Fatal(err)
37 }
38 var build, fail Job
39 for _, j := range got {
40 switch j.Name {
41 case "build":
42 build = j
43 case "fail":
44 fail = j
45 }
46 }
47 if err := s.FinishJob(ctx, build.ID, "success", ""); err != nil {
48 t.Fatal(err)
49 }
50 if err := s.FinishJob(ctx, fail.ID, "failed", ""); err != nil {
51 t.Fatal(err)
52 }
53 later := time.Now().Add(time.Hour)
54 past := time.Now().Add(-time.Hour)
55 if err := s.AddArtifact(ctx, build.ID, "dist/app", "/data/app", 3, &later); err != nil {
56 t.Fatal(err)
57 }
58 if err := s.AddArtifact(ctx, build.ID, "dist/old", "/data/old", 3, &past); err != nil {
59 t.Fatal(err)
60 }
61 if err := s.AddArtifact(ctx, fail.ID, "dist/bad", "/data/bad", 3, nil); err != nil {
62 t.Fatal(err)
63 }
64 otherJobs, err := s.JobsByPipeline(ctx, other)
65 if err != nil {
66 t.Fatal(err)
67 }
68 if err := s.FinishJob(ctx, otherJobs[0].ID, "success", ""); err != nil {
69 t.Fatal(err)
70 }
71 if err := s.AddArtifact(ctx, otherJobs[0].ID, "dist/other", "/data/other", 3, nil); err != nil {
72 t.Fatal(err)
73 }
74 path, err := s.ArtifactForPublish(ctx, pid, "build", "dist/app")
75 if err != nil || path != "/data/app" {
76 t.Fatalf("geldig: %q %v", path, err)
77 }
78 if _, err := s.ArtifactForPublish(ctx, pid, "build", "dist/old"); !errors.Is(err, ErrArtifactExpired) {
79 t.Fatalf("verlopen: %v", err)
80 }
81 if _, err := s.ArtifactForPublish(ctx, pid, "fail", "dist/bad"); !IsNotFound(err) {
82 t.Fatalf("mislukt: %v", err)
83 }
84 if _, err := s.ArtifactForPublish(ctx, pid, "build", "dist/other"); !IsNotFound(err) {
85 t.Fatalf("andere pipeline: %v", err)
86 }
87 if _, err := s.ArtifactPath(ctx, pid, "build", "dist/old"); err != nil {
88 t.Fatalf("ArtifactPath blijft verlopen bestanden teruggeven: %v", err)
89 }
90}
internal/store/flow.go +28 −0
@@ -1325,6 +1325,34 @@ func (s *Store) ArtifactPath(ctx context.Context, pipelineID int64, jobName, rel
1325 1325 return storage, err
1326 1326 }
1327 1327
1328 // ErrArtifactExpired: het artifact staat nog in de database, maar expires_at is voorbij.
1329 var ErrArtifactExpired = errors.New("artifact expired")
1330
1331 // ArtifactForPublish zoekt een geslaagd artifact van deze pipeline.
1332 // Een verlopen bestand telt niet. ArtifactPath blijft ongewijzigd voor de runner.
1333 func (s *Store) ArtifactForPublish(ctx context.Context, pipelineID int64, jobName, relPath string) (string, error) {
1334 var storage string
1335 var expires sql.NullString
1336 err := s.db.QueryRowContext(ctx, `SELECT a.storage_path, a.expires_at FROM artifacts a JOIN jobs j ON j.id = a.job_id
1337 WHERE j.pipeline_id = ? AND j.name = ? AND a.path = ? AND j.status = 'success'`, pipelineID, jobName, relPath).Scan(&storage, &expires)
1338 if errors.Is(err, sql.ErrNoRows) {
1339 return "", errNotFound
1340 }
1341 if err != nil {
1342 return "", err
1343 }
1344 if expires.Valid && strings.TrimSpace(expires.String) != "" {
1345 until := parseTime(expires.String)
1346 if until.IsZero() || !until.After(time.Now()) {
1347 return "", ErrArtifactExpired
1348 }
1349 }
1350 if storage == "" {
1351 return "", errNotFound
1352 }
1353 return storage, nil
1354 }
1355
1328 1356 // Release is een publicatie op een commit.
1329 1357 type Release struct {
1330 1358 ID int64
@@ -1325,6 +1325,34 @@ func (s *Store) ArtifactPath(ctx context.Context, pipelineID int64, jobName, rel
1325 return storage, err
1325 return storage, err
1326}
1326}
1327
1327
1328// ErrArtifactExpired: het artifact staat nog in de database, maar expires_at is voorbij.
1329var ErrArtifactExpired = errors.New("artifact expired")
1330
1331// ArtifactForPublish zoekt een geslaagd artifact van deze pipeline.
1332// Een verlopen bestand telt niet. ArtifactPath blijft ongewijzigd voor de runner.
1333func (s *Store) ArtifactForPublish(ctx context.Context, pipelineID int64, jobName, relPath string) (string, error) {
1334 var storage string
1335 var expires sql.NullString
1336 err := s.db.QueryRowContext(ctx, `SELECT a.storage_path, a.expires_at FROM artifacts a JOIN jobs j ON j.id = a.job_id
1337 WHERE j.pipeline_id = ? AND j.name = ? AND a.path = ? AND j.status = 'success'`, pipelineID, jobName, relPath).Scan(&storage, &expires)
1338 if errors.Is(err, sql.ErrNoRows) {
1339 return "", errNotFound
1340 }
1341 if err != nil {
1342 return "", err
1343 }
1344 if expires.Valid && strings.TrimSpace(expires.String) != "" {
1345 until := parseTime(expires.String)
1346 if until.IsZero() || !until.After(time.Now()) {
1347 return "", ErrArtifactExpired
1348 }
1349 }
1350 if storage == "" {
1351 return "", errNotFound
1352 }
1353 return storage, nil
1354}
1355
1328// Release is een publicatie op een commit.
1356// Release is een publicatie op een commit.
1329type Release struct {
1357type Release struct {
1330 ID int64
1358 ID int64