Merge branch 'cursor/pipeline-een-65da' into 'master'
fix: één pipeline wanneer push en merge request dezelfde jobs draaien See merge request Timfa/bluegit!52
@@ -74,7 +74,7 @@ $NAAM =~ $PATROON
74
74
75
75
Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_COMMIT_TIMESTAMP`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. `CI_COMMIT_TIMESTAMP` is de committer-tijd in ISO 8601. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.
76
76
77
Een merge-request-pipeline start bij het openen van de merge request, bij elke push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht. Die laatste draait nog steeds op de commit van de bronbranch, niet op een samengevoegde boom. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
77
Een merge-request-pipeline start bij het openen van de merge request, bij een push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht en de doelcommit daardoor verandert. Die draait op de commit van de bronbranch, niet op een samengevoegde boom. Dezelfde combinatie van ref, sha, bron, merge request en doelcommit start geen tweede pipeline. Kiezen de rules voor de push en voor de merge request dezelfde jobs, dan blijft één pipeline over en wordt die aan de merge request gekoppeld. Kiezen ze andere jobs, dan blijven beide bestaan en staat op de merge-request-pipeline het label Merge request met het nummer. `CI_PIPELINE_SOURCE` is `merge_request_event` op een pipeline die als merge request is gestart. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
78
78
79
79
`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.
80
80
@@ -74,7 +74,7 @@ $NAAM =~ $PATROON74
74
75
Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_COMMIT_TIMESTAMP`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. `CI_COMMIT_TIMESTAMP` is de committer-tijd in ISO 8601. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.75
Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_COMMIT_TIMESTAMP`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. `CI_COMMIT_TIMESTAMP` is de committer-tijd in ISO 8601. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.76
76
77
Een merge-request-pipeline start bij het openen van de merge request, bij elke push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht. Die laatste draait nog steeds op de commit van de bronbranch, niet op een samengevoegde boom. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.77
Een merge-request-pipeline start bij het openen van de merge request, bij een push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht en de doelcommit daardoor verandert. Die draait op de commit van de bronbranch, niet op een samengevoegde boom. Dezelfde combinatie van ref, sha, bron, merge request en doelcommit start geen tweede pipeline. Kiezen de rules voor de push en voor de merge request dezelfde jobs, dan blijft één pipeline over en wordt die aan de merge request gekoppeld. Kiezen ze andere jobs, dan blijven beide bestaan en staat op de merge-request-pipeline het label Merge request met het nummer. `CI_PIPELINE_SOURCE` is `merge_request_event` op een pipeline die als merge request is gestart. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.78
78
79
`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.79
`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.80
80
@@ -134,15 +134,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
134
134
return 0
135
135
}
136
136
title, jobs, broken := s.planPipeline(ctx, repo, sha, ref, source, before, mr)
137
mrID, target := int64(0), ""
138
if mr != nil {
139
mrID, target = mr.ID, mr.TargetSHA
140
}
137
141
if broken != nil {
138
id, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, broken.Error())
142
id, created, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, source, broken.Error(), mrID, target)
139
143
if err != nil && s.Log != nil {
140
144
s.Log.Error("pipeline", "err", err)
141
145
}
142
if mr != nil && id != 0 {
143
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
144
}
145
if id != 0 {
146
if created && id != 0 {
146
147
s.cancelOlder(ctx, repo.ID, ref, id)
147
148
}
148
149
return id
@@ -150,17 +151,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
150
151
if len(jobs) == 0 {
151
152
return 0
152
153
}
153
id, err := s.Store.CreatePipeline(ctx, repo.ID, sha, ref, source, actor, title, jobs)
154
id, created, err := s.Store.OpenPipeline(ctx, repo.ID, sha, ref, source, actor, title, target, mrID, jobs)
154
155
if err != nil {
155
156
if s.Log != nil {
156
157
s.Log.Error("pipeline", "err", err)
157
158
}
158
159
return 0
159
160
}
160
if mr != nil {
161
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
161
if created {
162
s.cancelOlder(ctx, repo.ID, ref, id)
162
163
}
163
s.cancelOlder(ctx, repo.ID, ref, id)
164
164
return id
165
165
}
166
166
@@ -134,15 +134,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a134
return 0134
return 0135
}135
}136
title, jobs, broken := s.planPipeline(ctx, repo, sha, ref, source, before, mr)136
title, jobs, broken := s.planPipeline(ctx, repo, sha, ref, source, before, mr)137
mrID, target := int64(0), ""138
if mr != nil {139
mrID, target = mr.ID, mr.TargetSHA140
}137
if broken != nil {141
if broken != nil {138
id, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, broken.Error())142
id, created, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, source, broken.Error(), mrID, target)139
if err != nil && s.Log != nil {143
if err != nil && s.Log != nil {140
s.Log.Error("pipeline", "err", err)144
s.Log.Error("pipeline", "err", err)141
}145
}142
if mr != nil && id != 0 {146
if created && id != 0 {143
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)144
}145
if id != 0 {146
s.cancelOlder(ctx, repo.ID, ref, id)147
s.cancelOlder(ctx, repo.ID, ref, id)147
}148
}148
return id149
return id@@ -150,17 +151,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a150
if len(jobs) == 0 {151
if len(jobs) == 0 {151
return 0152
return 0152
}153
}153
id, err := s.Store.CreatePipeline(ctx, repo.ID, sha, ref, source, actor, title, jobs)154
id, created, err := s.Store.OpenPipeline(ctx, repo.ID, sha, ref, source, actor, title, target, mrID, jobs)154
if err != nil {155
if err != nil {155
if s.Log != nil {156
if s.Log != nil {156
s.Log.Error("pipeline", "err", err)157
s.Log.Error("pipeline", "err", err)157
}158
}158
return 0159
return 0159
}160
}160
if mr != nil {161
if created {161
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)162
s.cancelOlder(ctx, repo.ID, ref, id)162
}163
}163
s.cancelOlder(ctx, repo.ID, ref, id)164
return id164
return id165
}165
}166
166
@@ -8,11 +8,13 @@ import (
8
8
"net/url"
9
9
"os/exec"
10
10
"path/filepath"
11
"strconv"
11
12
"strings"
12
13
"testing"
13
14
14
15
"gitlab.com/Timfa/bluegit/internal/auth"
15
16
"gitlab.com/Timfa/bluegit/internal/config"
17
"gitlab.com/Timfa/bluegit/internal/gitrepo"
16
18
"gitlab.com/Timfa/bluegit/internal/store"
17
19
)
18
20
@@ -138,6 +140,38 @@ soft:
138
140
if mrPipe.ID == 0 || mrPipe.MRID == 0 {
139
141
t.Fatalf("mr-pipeline: %+v", mrPipe)
140
142
}
143
nBefore := len(pipes)
144
user, err := st.UserByUsername(ctx, "tim")
145
if err != nil {
146
t.Fatal(err)
147
}
148
dir := gitrepo.Dir(storage, "tim", "mrpipes")
149
after, err := gitrepo.Refs(ctx, "git", dir)
150
if err != nil {
151
t.Fatal(err)
152
}
153
before := map[string]string{}
154
for ref, sha := range after {
155
before[ref] = sha
156
}
157
before["refs/heads/feature"] = strings.Repeat("a", 40)
158
srv.AfterPush(repo, user, before, after)
159
srv.rerunMergeRequestsOnTarget(ctx, repo, "master", "tim")
160
pipes, err = st.ListPipelines(ctx, repo.ID)
161
if err != nil {
162
t.Fatal(err)
163
}
164
if len(pipes) != nBefore {
165
t.Fatalf("dezelfde push nog eens: %d pipelines, was %d", len(pipes), nBefore)
166
}
167
list := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines"))
168
if !strings.Contains(list, "Merge request !1") {
169
t.Fatalf("lijst mist het mr-label:\n%s", list)
170
}
171
detail := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines/"+strconv.FormatInt(mrPipe.ID, 10)))
172
if !strings.Contains(detail, "Merge request !1") {
173
t.Fatalf("detail mist het mr-label:\n%s", detail)
174
}
141
175
res, err = tim.PostForm(ts.URL+"/tim/mrpipes/-/merge-requests/1/merge", url.Values{"csrf": {csrfOf(t, page)}})
142
176
if err != nil {
143
177
t.Fatal(err)
@@ -211,3 +245,162 @@ soft:
211
245
t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)
212
246
}
213
247
}
248
249
func TestSameJobPushAndMRShareOnePipeline(t *testing.T) {
250
if _, err := exec.LookPath("git"); err != nil {
251
t.Skip("git ontbreekt")
252
}
253
storage := t.TempDir()
254
st, err := store.Open(filepath.Join(storage, "bluegit.db"), bytes32())
255
if err != nil {
256
t.Fatal(err)
257
}
258
t.Cleanup(func() { _ = st.Close() })
259
cfg := config.Config{
260
BaseURL: "http://127.0.0.1:9", Listen: "127.0.0.1:0", StoragePath: storage,
261
Auth: config.Auth{Mode: "dev", DevUser: "tim", DevAuthID: 1, UserAgent: "test", BaseURL: "https://auth.timfalken.com"},
262
Git: config.Git{Binary: "git", DefaultBranch: "master"},
263
CI: config.CI{Enabled: true, Docker: "docker"},
264
}
265
srv := &Server{Cfg: cfg, Store: st, Auth: auth.Dev{Username: "tim", AuthID: 1}, Dev: true, Git: "git", HookBin: "/bin/true"}
266
ts := httptest.NewServer(srv.Handler())
267
t.Cleanup(ts.Close)
268
jar, _ := cookiejar.New(nil)
269
tim := &http.Client{Jar: jar}
270
res, err := tim.Get(ts.URL + "/login")
271
if err != nil {
272
t.Fatal(err)
273
}
274
page := read(t, res)
275
res, err = tim.PostForm(ts.URL+"/new/repo", url.Values{"csrf": {csrfOf(t, page)}, "name": {"samepipe"}, "visibility": {"public"}})
276
if err != nil {
277
t.Fatal(err)
278
}
279
page = read(t, res)
280
res, err = tim.PostForm(ts.URL+"/settings/tokens", url.Values{"csrf": {csrfOf(t, page)}, "name": {"laptop"}})
281
if err != nil {
282
t.Fatal(err)
283
}
284
token := between(read(t, res), `<p class="code">`, "</p>")
285
remote := strings.Replace(ts.URL, "http://", "http://tim:"+token+"@", 1) + "/tim/samepipe.git"
286
work := t.TempDir()
287
git(t, work, "init", "-b", "master")
288
git(t, work, "config", "user.email", "tim@users.git.timfalken.com")
289
git(t, work, "config", "user.name", "tim")
290
yml := `stages: [test]
291
check:
292
stage: test
293
image: alpine:3.20
294
script: ["echo check"]
295
rules:
296
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
297
- if: $CI_COMMIT_BRANCH
298
`
299
writeFile(t, filepath.Join(work, "blue-ci-cd.yml"), yml)
300
git(t, work, "add", "blue-ci-cd.yml")
301
git(t, work, "commit", "-m", "ci")
302
git(t, work, "remote", "add", "origin", remote)
303
git(t, work, "push", "-u", "origin", "master")
304
git(t, work, "checkout", "-b", "feature")
305
writeFile(t, filepath.Join(work, "README.md"), "feature\n")
306
git(t, work, "add", "README.md")
307
git(t, work, "commit", "-m", "feature")
308
git(t, work, "push", "-u", "origin", "feature")
309
first := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))
310
311
ctx := context.Background()
312
repo, err := st.RepoByPath(ctx, "tim", "samepipe")
313
if err != nil {
314
t.Fatal(err)
315
}
316
if n := len(pipelinesForSHA(t, st, repo.ID, first)); n != 1 {
317
t.Fatalf("push zonder mr: %d pipelines voor %s", n, first)
318
}
319
320
res, err = tim.Get(ts.URL + "/tim/samepipe/-/merge-requests")
321
if err != nil {
322
t.Fatal(err)
323
}
324
page = read(t, res)
325
res, err = tim.PostForm(ts.URL+"/tim/samepipe/-/merge-requests", url.Values{
326
"csrf": {csrfOf(t, page)}, "title": {"voeg samen"}, "source": {"feature"}, "target": {"master"},
327
})
328
if err != nil {
329
t.Fatal(err)
330
}
331
_ = read(t, res)
332
opened := pipelinesForSHA(t, st, repo.ID, first)
333
if len(opened) != 1 || opened[0].MRID == 0 {
334
t.Fatalf("mr openen hoort de push te hergebruiken: %+v", opened)
335
}
336
337
writeFile(t, filepath.Join(work, "README.md"), "nog een\n")
338
git(t, work, "add", "README.md")
339
git(t, work, "commit", "-m", "nog een")
340
git(t, work, "push", "origin", "feature")
341
second := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))
342
shared := pipelinesForSHA(t, st, repo.ID, second)
343
if len(shared) != 1 || shared[0].MRID == 0 {
344
t.Fatalf("push met open mr: %+v", shared)
345
}
346
user, err := st.UserByUsername(ctx, "tim")
347
if err != nil {
348
t.Fatal(err)
349
}
350
dir := gitrepo.Dir(storage, "tim", "samepipe")
351
after, err := gitrepo.Refs(ctx, "git", dir)
352
if err != nil {
353
t.Fatal(err)
354
}
355
before := map[string]string{}
356
for ref, sha := range after {
357
before[ref] = sha
358
}
359
before["refs/heads/feature"] = first
360
srv.AfterPush(repo, user, before, after)
361
if n := len(pipelinesForSHA(t, st, repo.ID, second)); n != 1 {
362
t.Fatalf("zelfde push nog eens: %d", n)
363
}
364
list := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines"))
365
if !strings.Contains(list, "Merge request !1") {
366
t.Fatalf("lijst mist het mr-label:\n%s", list)
367
}
368
detail := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines/"+strconv.FormatInt(shared[0].ID, 10)))
369
if !strings.Contains(detail, "Merge request !1") {
370
t.Fatalf("detail mist het mr-label:\n%s", detail)
371
}
372
373
git(t, work, "checkout", "master")
374
writeFile(t, filepath.Join(work, "README.md"), "target\n")
375
git(t, work, "add", "README.md")
376
git(t, work, "commit", "-m", "target")
377
git(t, work, "push", "origin", "master")
378
afterTarget := pipelinesForSHA(t, st, repo.ID, second)
379
if len(afterTarget) != 2 {
380
t.Fatalf("nieuwe doelcommit hoort een mr-pipeline toe te voegen: %+v", afterTarget)
381
}
382
var newest store.Pipeline
383
for _, p := range afterTarget {
384
if p.ID > newest.ID {
385
newest = p
386
}
387
}
388
if newest.ID <= shared[0].ID || newest.Source != "merge_request_event" || newest.MRID == 0 || newest.CommitSHA != second {
389
t.Fatalf("nieuwe mr-pipeline: %+v", newest)
390
}
391
}
392
393
func pipelinesForSHA(t *testing.T, st *store.Store, repoID int64, sha string) []store.Pipeline {
394
t.Helper()
395
all, err := st.ListPipelines(context.Background(), repoID)
396
if err != nil {
397
t.Fatal(err)
398
}
399
var out []store.Pipeline
400
for _, p := range all {
401
if p.CommitSHA == sha {
402
out = append(out, p)
403
}
404
}
405
return out
406
}
@@ -8,11 +8,13 @@ import (8
"net/url"8
"net/url"9
"os/exec"9
"os/exec"10
"path/filepath"10
"path/filepath"11
"strconv"11
"strings"12
"strings"12
"testing"13
"testing"13
14
14
"gitlab.com/Timfa/bluegit/internal/auth"15
"gitlab.com/Timfa/bluegit/internal/auth"15
"gitlab.com/Timfa/bluegit/internal/config"16
"gitlab.com/Timfa/bluegit/internal/config"17
"gitlab.com/Timfa/bluegit/internal/gitrepo"16
"gitlab.com/Timfa/bluegit/internal/store"18
"gitlab.com/Timfa/bluegit/internal/store"17
)19
)18
20
@@ -138,6 +140,38 @@ soft:138
if mrPipe.ID == 0 || mrPipe.MRID == 0 {140
if mrPipe.ID == 0 || mrPipe.MRID == 0 {139
t.Fatalf("mr-pipeline: %+v", mrPipe)141
t.Fatalf("mr-pipeline: %+v", mrPipe)140
}142
}143
nBefore := len(pipes)144
user, err := st.UserByUsername(ctx, "tim")145
if err != nil {146
t.Fatal(err)147
}148
dir := gitrepo.Dir(storage, "tim", "mrpipes")149
after, err := gitrepo.Refs(ctx, "git", dir)150
if err != nil {151
t.Fatal(err)152
}153
before := map[string]string{}154
for ref, sha := range after {155
before[ref] = sha156
}157
before["refs/heads/feature"] = strings.Repeat("a", 40)158
srv.AfterPush(repo, user, before, after)159
srv.rerunMergeRequestsOnTarget(ctx, repo, "master", "tim")160
pipes, err = st.ListPipelines(ctx, repo.ID)161
if err != nil {162
t.Fatal(err)163
}164
if len(pipes) != nBefore {165
t.Fatalf("dezelfde push nog eens: %d pipelines, was %d", len(pipes), nBefore)166
}167
list := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines"))168
if !strings.Contains(list, "Merge request !1") {169
t.Fatalf("lijst mist het mr-label:\n%s", list)170
}171
detail := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines/"+strconv.FormatInt(mrPipe.ID, 10)))172
if !strings.Contains(detail, "Merge request !1") {173
t.Fatalf("detail mist het mr-label:\n%s", detail)174
}141
res, err = tim.PostForm(ts.URL+"/tim/mrpipes/-/merge-requests/1/merge", url.Values{"csrf": {csrfOf(t, page)}})175
res, err = tim.PostForm(ts.URL+"/tim/mrpipes/-/merge-requests/1/merge", url.Values{"csrf": {csrfOf(t, page)}})142
if err != nil {176
if err != nil {143
t.Fatal(err)177
t.Fatal(err)@@ -211,3 +245,162 @@ soft:211
t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)245
t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)212
}246
}213
}247
}248
249
func TestSameJobPushAndMRShareOnePipeline(t *testing.T) {250
if _, err := exec.LookPath("git"); err != nil {251
t.Skip("git ontbreekt")252
}253
storage := t.TempDir()254
st, err := store.Open(filepath.Join(storage, "bluegit.db"), bytes32())255
if err != nil {256
t.Fatal(err)257
}258
t.Cleanup(func() { _ = st.Close() })259
cfg := config.Config{260
BaseURL: "http://127.0.0.1:9", Listen: "127.0.0.1:0", StoragePath: storage,261
Auth: config.Auth{Mode: "dev", DevUser: "tim", DevAuthID: 1, UserAgent: "test", BaseURL: "https://auth.timfalken.com"},262
Git: config.Git{Binary: "git", DefaultBranch: "master"},263
CI: config.CI{Enabled: true, Docker: "docker"},264
}265
srv := &Server{Cfg: cfg, Store: st, Auth: auth.Dev{Username: "tim", AuthID: 1}, Dev: true, Git: "git", HookBin: "/bin/true"}266
ts := httptest.NewServer(srv.Handler())267
t.Cleanup(ts.Close)268
jar, _ := cookiejar.New(nil)269
tim := &http.Client{Jar: jar}270
res, err := tim.Get(ts.URL + "/login")271
if err != nil {272
t.Fatal(err)273
}274
page := read(t, res)275
res, err = tim.PostForm(ts.URL+"/new/repo", url.Values{"csrf": {csrfOf(t, page)}, "name": {"samepipe"}, "visibility": {"public"}})276
if err != nil {277
t.Fatal(err)278
}279
page = read(t, res)280
res, err = tim.PostForm(ts.URL+"/settings/tokens", url.Values{"csrf": {csrfOf(t, page)}, "name": {"laptop"}})281
if err != nil {282
t.Fatal(err)283
}284
token := between(read(t, res), `<p class="code">`, "</p>")285
remote := strings.Replace(ts.URL, "http://", "http://tim:"+token+"@", 1) + "/tim/samepipe.git"286
work := t.TempDir()287
git(t, work, "init", "-b", "master")288
git(t, work, "config", "user.email", "tim@users.git.timfalken.com")289
git(t, work, "config", "user.name", "tim")290
yml := `stages: [test]291
check:292
stage: test293
image: alpine:3.20294
script: ["echo check"]295
rules:296
- if: $CI_PIPELINE_SOURCE == "merge_request_event"297
- if: $CI_COMMIT_BRANCH298
`299
writeFile(t, filepath.Join(work, "blue-ci-cd.yml"), yml)300
git(t, work, "add", "blue-ci-cd.yml")301
git(t, work, "commit", "-m", "ci")302
git(t, work, "remote", "add", "origin", remote)303
git(t, work, "push", "-u", "origin", "master")304
git(t, work, "checkout", "-b", "feature")305
writeFile(t, filepath.Join(work, "README.md"), "feature\n")306
git(t, work, "add", "README.md")307
git(t, work, "commit", "-m", "feature")308
git(t, work, "push", "-u", "origin", "feature")309
first := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))310
311
ctx := context.Background()312
repo, err := st.RepoByPath(ctx, "tim", "samepipe")313
if err != nil {314
t.Fatal(err)315
}316
if n := len(pipelinesForSHA(t, st, repo.ID, first)); n != 1 {317
t.Fatalf("push zonder mr: %d pipelines voor %s", n, first)318
}319
320
res, err = tim.Get(ts.URL + "/tim/samepipe/-/merge-requests")321
if err != nil {322
t.Fatal(err)323
}324
page = read(t, res)325
res, err = tim.PostForm(ts.URL+"/tim/samepipe/-/merge-requests", url.Values{326
"csrf": {csrfOf(t, page)}, "title": {"voeg samen"}, "source": {"feature"}, "target": {"master"},327
})328
if err != nil {329
t.Fatal(err)330
}331
_ = read(t, res)332
opened := pipelinesForSHA(t, st, repo.ID, first)333
if len(opened) != 1 || opened[0].MRID == 0 {334
t.Fatalf("mr openen hoort de push te hergebruiken: %+v", opened)335
}336
337
writeFile(t, filepath.Join(work, "README.md"), "nog een\n")338
git(t, work, "add", "README.md")339
git(t, work, "commit", "-m", "nog een")340
git(t, work, "push", "origin", "feature")341
second := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))342
shared := pipelinesForSHA(t, st, repo.ID, second)343
if len(shared) != 1 || shared[0].MRID == 0 {344
t.Fatalf("push met open mr: %+v", shared)345
}346
user, err := st.UserByUsername(ctx, "tim")347
if err != nil {348
t.Fatal(err)349
}350
dir := gitrepo.Dir(storage, "tim", "samepipe")351
after, err := gitrepo.Refs(ctx, "git", dir)352
if err != nil {353
t.Fatal(err)354
}355
before := map[string]string{}356
for ref, sha := range after {357
before[ref] = sha358
}359
before["refs/heads/feature"] = first360
srv.AfterPush(repo, user, before, after)361
if n := len(pipelinesForSHA(t, st, repo.ID, second)); n != 1 {362
t.Fatalf("zelfde push nog eens: %d", n)363
}364
list := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines"))365
if !strings.Contains(list, "Merge request !1") {366
t.Fatalf("lijst mist het mr-label:\n%s", list)367
}368
detail := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines/"+strconv.FormatInt(shared[0].ID, 10)))369
if !strings.Contains(detail, "Merge request !1") {370
t.Fatalf("detail mist het mr-label:\n%s", detail)371
}372
373
git(t, work, "checkout", "master")374
writeFile(t, filepath.Join(work, "README.md"), "target\n")375
git(t, work, "add", "README.md")376
git(t, work, "commit", "-m", "target")377
git(t, work, "push", "origin", "master")378
afterTarget := pipelinesForSHA(t, st, repo.ID, second)379
if len(afterTarget) != 2 {380
t.Fatalf("nieuwe doelcommit hoort een mr-pipeline toe te voegen: %+v", afterTarget)381
}382
var newest store.Pipeline383
for _, p := range afterTarget {384
if p.ID > newest.ID {385
newest = p386
}387
}388
if newest.ID <= shared[0].ID || newest.Source != "merge_request_event" || newest.MRID == 0 || newest.CommitSHA != second {389
t.Fatalf("nieuwe mr-pipeline: %+v", newest)390
}391
}392
393
func pipelinesForSHA(t *testing.T, st *store.Store, repoID int64, sha string) []store.Pipeline {394
t.Helper()395
all, err := st.ListPipelines(context.Background(), repoID)396
if err != nil {397
t.Fatal(err)398
}399
var out []store.Pipeline400
for _, p := range all {401
if p.CommitSHA == sha {402
out = append(out, p)403
}404
}405
return out406
}@@ -17,20 +17,23 @@ import (
17
17
18
18
// PipeRow is één rij in het pipeline-overzicht.
19
19
type PipeRow struct {
20
ID int64
21
Status string
22
Label string
23
Duration string
24
When string
25
Title string
26
SHA string
27
Short string
28
Ref string
29
RefKind string
30
Actor string
31
Avatar string
32
Initial string
33
Stages []StageView
20
ID int64
21
Status string
22
Label string
23
Duration string
24
When string
25
Title string
26
SHA string
27
Short string
28
Ref string
29
RefKind string
30
Source string
31
SourceLabel string
32
MRNumber int64
33
Actor string
34
Avatar string
35
Initial string
36
Stages []StageView
34
37
}
35
38
36
39
// StageView is één bolletje in de stages-kolom.
@@ -154,15 +157,50 @@ func (s *Server) pipeRow(r *http.Request, lang i18n.Lang, repo store.Repo, p sto
154
157
if actor != "" {
155
158
initial = strings.ToUpper(actor[:1])
156
159
}
160
sourceLabel, mrNumber := s.pipelineSourceLabel(r, repo, p)
157
161
return PipeRow{
158
162
ID: p.ID, Status: status, Label: s.tr(r, "pipe."+status),
159
163
Duration: pipelineClock(p), When: i18n.Ago(lang, p.CreatedAt.Unix()),
160
164
Title: p.CommitTitle, SHA: p.CommitSHA, Short: shortSHA(p.CommitSHA),
161
Ref: name, RefKind: kind, Actor: actor, Avatar: avatar, Initial: initial,
165
Ref: name, RefKind: kind, Source: p.Source, SourceLabel: sourceLabel, MRNumber: mrNumber,
166
Actor: actor, Avatar: avatar, Initial: initial,
162
167
Stages: stageViews(jobs, ranks, func(key string, args ...any) string { return s.tr(r, key, args...) }),
163
168
}
164
169
}
165
170
171
// pipelineSourceLabel benoemt een pipeline die bij een merge request hoort,
172
// ook wanneer de rij als push is gestart en daarna gekoppeld is.
173
// Een gewone push zonder merge request krijgt geen extra label.
174
func (s *Server) pipelineSourceLabel(r *http.Request, repo store.Repo, p store.Pipeline) (string, int64) {
175
if p.MRID != 0 {
176
var number int64
177
if mr, err := s.Store.MergeRequest(r.Context(), repo.ID, p.MRID); err == nil {
178
number = mr.Number
179
}
180
return s.tr(r, "pipe.src.merge_request_event"), number
181
}
182
if p.Source == "" || p.Source == "push" {
183
return "", 0
184
}
185
key := "pipe.src." + p.Source
186
label := s.tr(r, key)
187
if label == key {
188
label = p.Source
189
}
190
return label, 0
191
}
192
193
func (s *Server) pipelineSourceText(r *http.Request, repo store.Repo, p store.Pipeline) string {
194
label, number := s.pipelineSourceLabel(r, repo, p)
195
if label == "" {
196
return ""
197
}
198
if number == 0 {
199
return label
200
}
201
return label + " !" + strconv.FormatInt(number, 10)
202
}
203
166
204
func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.User) (string, string) {
167
205
name = strings.TrimSpace(name)
168
206
if name == "" {
@@ -183,7 +221,7 @@ func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.Us
183
221
}
184
222
185
223
func pipeMatches(row PipeRow, q string) bool {
186
bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, strconv.FormatInt(row.ID, 10), row.Status}
224
bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, row.SourceLabel, strconv.FormatInt(row.ID, 10), row.Status}
187
225
for _, st := range row.Stages {
188
226
bits = append(bits, st.Name)
189
227
for _, j := range st.Jobs {
@@ -17,20 +17,23 @@ import (17
17
18
// PipeRow is één rij in het pipeline-overzicht.18
// PipeRow is één rij in het pipeline-overzicht.19
type PipeRow struct {19
type PipeRow struct {20
ID int6420
ID int6421
Status string21
Status string22
Label string22
Label string23
Duration string23
Duration string24
When string24
When string25
Title string25
Title string26
SHA string26
SHA string27
Short string27
Short string28
Ref string28
Ref string29
RefKind string29
RefKind string30
Actor string30
Source string31
Avatar string31
SourceLabel string32
Initial string32
MRNumber int6433
Stages []StageView33
Actor string34
Avatar string35
Initial string36
Stages []StageView34
}37
}35
38
36
// StageView is één bolletje in de stages-kolom.39
// StageView is één bolletje in de stages-kolom.@@ -154,15 +157,50 @@ func (s *Server) pipeRow(r *http.Request, lang i18n.Lang, repo store.Repo, p sto154
if actor != "" {157
if actor != "" {155
initial = strings.ToUpper(actor[:1])158
initial = strings.ToUpper(actor[:1])156
}159
}160
sourceLabel, mrNumber := s.pipelineSourceLabel(r, repo, p)157
return PipeRow{161
return PipeRow{158
ID: p.ID, Status: status, Label: s.tr(r, "pipe."+status),162
ID: p.ID, Status: status, Label: s.tr(r, "pipe."+status),159
Duration: pipelineClock(p), When: i18n.Ago(lang, p.CreatedAt.Unix()),163
Duration: pipelineClock(p), When: i18n.Ago(lang, p.CreatedAt.Unix()),160
Title: p.CommitTitle, SHA: p.CommitSHA, Short: shortSHA(p.CommitSHA),164
Title: p.CommitTitle, SHA: p.CommitSHA, Short: shortSHA(p.CommitSHA),161
Ref: name, RefKind: kind, Actor: actor, Avatar: avatar, Initial: initial,165
Ref: name, RefKind: kind, Source: p.Source, SourceLabel: sourceLabel, MRNumber: mrNumber,166
Actor: actor, Avatar: avatar, Initial: initial,162
Stages: stageViews(jobs, ranks, func(key string, args ...any) string { return s.tr(r, key, args...) }),167
Stages: stageViews(jobs, ranks, func(key string, args ...any) string { return s.tr(r, key, args...) }),163
}168
}164
}169
}165
170
171
// pipelineSourceLabel benoemt een pipeline die bij een merge request hoort,172
// ook wanneer de rij als push is gestart en daarna gekoppeld is.173
// Een gewone push zonder merge request krijgt geen extra label.174
func (s *Server) pipelineSourceLabel(r *http.Request, repo store.Repo, p store.Pipeline) (string, int64) {175
if p.MRID != 0 {176
var number int64177
if mr, err := s.Store.MergeRequest(r.Context(), repo.ID, p.MRID); err == nil {178
number = mr.Number179
}180
return s.tr(r, "pipe.src.merge_request_event"), number181
}182
if p.Source == "" || p.Source == "push" {183
return "", 0184
}185
key := "pipe.src." + p.Source186
label := s.tr(r, key)187
if label == key {188
label = p.Source189
}190
return label, 0191
}192
193
func (s *Server) pipelineSourceText(r *http.Request, repo store.Repo, p store.Pipeline) string {194
label, number := s.pipelineSourceLabel(r, repo, p)195
if label == "" {196
return ""197
}198
if number == 0 {199
return label200
}201
return label + " !" + strconv.FormatInt(number, 10)202
}203
166
func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.User) (string, string) {204
func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.User) (string, string) {167
name = strings.TrimSpace(name)205
name = strings.TrimSpace(name)168
if name == "" {206
if name == "" {@@ -183,7 +221,7 @@ func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.Us183
}221
}184
222
185
func pipeMatches(row PipeRow, q string) bool {223
func pipeMatches(row PipeRow, q string) bool {186
bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, strconv.FormatInt(row.ID, 10), row.Status}224
bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, row.SourceLabel, strconv.FormatInt(row.ID, 10), row.Status}187
for _, st := range row.Stages {225
for _, st := range row.Stages {188
bits = append(bits, st.Name)226
bits = append(bits, st.Name)189
for _, j := range st.Jobs {227
for _, j := range st.Jobs {@@ -367,6 +367,7 @@ func (s *Server) pipeline(w http.ResponseWriter, r *http.Request) {
367
367
tr := func(key string, args ...any) string { return s.tr(r, key, args...) }
368
368
v.PipeStatus = displayStatus(p.Status, jobs)
369
369
v.PipeLabel = tr("pipe." + v.PipeStatus)
370
v.PipeSource = s.pipelineSourceText(r, repo, p)
370
371
v.PipeJobs = linksFor(jobs, ranks, tr)
371
372
v.PipeStages = stageViews(jobs, ranks, tr)
372
373
v.Live = jobsLive(p.Status, jobs)
@@ -367,6 +367,7 @@ func (s *Server) pipeline(w http.ResponseWriter, r *http.Request) {367
tr := func(key string, args ...any) string { return s.tr(r, key, args...) }367
tr := func(key string, args ...any) string { return s.tr(r, key, args...) }368
v.PipeStatus = displayStatus(p.Status, jobs)368
v.PipeStatus = displayStatus(p.Status, jobs)369
v.PipeLabel = tr("pipe." + v.PipeStatus)369
v.PipeLabel = tr("pipe." + v.PipeStatus)370
v.PipeSource = s.pipelineSourceText(r, repo, p)370
v.PipeJobs = linksFor(jobs, ranks, tr)371
v.PipeJobs = linksFor(jobs, ranks, tr)371
v.PipeStages = stageViews(jobs, ranks, tr)372
v.PipeStages = stageViews(jobs, ranks, tr)372
v.Live = jobsLive(p.Status, jobs)373
v.Live = jobsLive(p.Status, jobs)@@ -540,6 +540,7 @@ type View struct {
540
540
Pipes []store.Pipeline
541
541
PipeRows []PipeRow
542
542
Pipe *store.Pipeline
543
PipeSource string
543
544
PipeStatus string
544
545
PipeLabel string
545
546
PipeJobs []JobLink
@@ -540,6 +540,7 @@ type View struct {540
Pipes []store.Pipeline540
Pipes []store.Pipeline541
PipeRows []PipeRow541
PipeRows []PipeRow542
Pipe *store.Pipeline542
Pipe *store.Pipeline543
PipeSource string543
PipeStatus string544
PipeStatus string544
PipeLabel string545
PipeLabel string545
PipeJobs []JobLink546
PipeJobs []JobLink@@ -253,7 +253,7 @@
253
253
{{template "layout-start" .}}
254
254
<header class="pagehead"><h1>{{t "pipe.one"}} #{{.Pipe.ID}}</h1></header>
255
255
{{template "pipenav" .}}
256
<p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
256
<p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span>{{if .PipeSource}} <span class="pill">{{.PipeSource}}</span>{{end}} {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
257
257
<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>
258
258
<ul class="clean"{{if .Live}} data-live="{{.LiveURL}}" data-live-on="1"{{end}}>
259
259
{{range .PipeJobs}}<li data-job="{{.ID}}"><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="status {{.Status}}" data-job-status><span class="mark"></span><span data-job-label>{{.Label}}</span></span>{{if .Queue}} <span class="queue" data-queue>{{.Queue}}</span>{{end}} <span class="muted">{{.Stage}}</span></li>{{end}}
@@ -277,6 +277,7 @@
277
277
<div class="chips">
278
278
{{if .Ref}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commits?rev={{.Ref}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true">{{if eq .RefKind "tag"}}<path d="M2.2 2.2h5.2l6.2 6.2-5.2 5.2-6.2-6.2V2.2z" fill="none" stroke="currentColor" stroke-width="1.3"/><circle cx="5.2" cy="5.2" r="1" fill="currentColor"/>{{else}}<circle cx="4" cy="3.2" r="1.5" fill="currentColor"/><circle cx="4" cy="12.8" r="1.5" fill="currentColor"/><circle cx="12" cy="6.4" r="1.5" fill="currentColor"/><path d="M4 4.7v6.6M4 6.4c2.4 0 4.2.2 6 .2 1.2 0 2 1 2 1.8" fill="none" stroke="currentColor" stroke-width="1.3"/>{{end}}</svg>{{.Ref}}</a>{{end}}
279
279
{{if .Short}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commit/{{.SHA}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true"><circle cx="8" cy="8" r="2.1" fill="currentColor"/><path d="M8 1.8v3.4M8 10.8v3.4" stroke="currentColor" stroke-width="1.4"/></svg>{{.Short}}</a>{{end}}
280
{{if .SourceLabel}}<span class="chip">{{.SourceLabel}}{{if .MRNumber}} !{{.MRNumber}}{{end}}</span>{{end}}
280
281
{{if .Actor}}{{if .Avatar}}<img class="avatar sm" src="{{.Avatar}}" alt="">{{else}}<span class="avatar sm">{{.Initial}}</span>{{end}}{{end}}
281
282
</div>
282
283
</td>
@@ -253,7 +253,7 @@253
{{template "layout-start" .}}253
{{template "layout-start" .}}254
<header class="pagehead"><h1>{{t "pipe.one"}} #{{.Pipe.ID}}</h1></header>254
<header class="pagehead"><h1>{{t "pipe.one"}} #{{.Pipe.ID}}</h1></header>255
{{template "pipenav" .}}255
{{template "pipenav" .}}256
<p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>256
<p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span>{{if .PipeSource}} <span class="pill">{{.PipeSource}}</span>{{end}} {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>257
<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>257
<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>258
<ul class="clean"{{if .Live}} data-live="{{.LiveURL}}" data-live-on="1"{{end}}>258
<ul class="clean"{{if .Live}} data-live="{{.LiveURL}}" data-live-on="1"{{end}}>259
{{range .PipeJobs}}<li data-job="{{.ID}}"><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="status {{.Status}}" data-job-status><span class="mark"></span><span data-job-label>{{.Label}}</span></span>{{if .Queue}} <span class="queue" data-queue>{{.Queue}}</span>{{end}} <span class="muted">{{.Stage}}</span></li>{{end}}259
{{range .PipeJobs}}<li data-job="{{.ID}}"><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="status {{.Status}}" data-job-status><span class="mark"></span><span data-job-label>{{.Label}}</span></span>{{if .Queue}} <span class="queue" data-queue>{{.Queue}}</span>{{end}} <span class="muted">{{.Stage}}</span></li>{{end}}@@ -277,6 +277,7 @@277
<div class="chips">277
<div class="chips">278
{{if .Ref}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commits?rev={{.Ref}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true">{{if eq .RefKind "tag"}}<path d="M2.2 2.2h5.2l6.2 6.2-5.2 5.2-6.2-6.2V2.2z" fill="none" stroke="currentColor" stroke-width="1.3"/><circle cx="5.2" cy="5.2" r="1" fill="currentColor"/>{{else}}<circle cx="4" cy="3.2" r="1.5" fill="currentColor"/><circle cx="4" cy="12.8" r="1.5" fill="currentColor"/><circle cx="12" cy="6.4" r="1.5" fill="currentColor"/><path d="M4 4.7v6.6M4 6.4c2.4 0 4.2.2 6 .2 1.2 0 2 1 2 1.8" fill="none" stroke="currentColor" stroke-width="1.3"/>{{end}}</svg>{{.Ref}}</a>{{end}}278
{{if .Ref}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commits?rev={{.Ref}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true">{{if eq .RefKind "tag"}}<path d="M2.2 2.2h5.2l6.2 6.2-5.2 5.2-6.2-6.2V2.2z" fill="none" stroke="currentColor" stroke-width="1.3"/><circle cx="5.2" cy="5.2" r="1" fill="currentColor"/>{{else}}<circle cx="4" cy="3.2" r="1.5" fill="currentColor"/><circle cx="4" cy="12.8" r="1.5" fill="currentColor"/><circle cx="12" cy="6.4" r="1.5" fill="currentColor"/><path d="M4 4.7v6.6M4 6.4c2.4 0 4.2.2 6 .2 1.2 0 2 1 2 1.8" fill="none" stroke="currentColor" stroke-width="1.3"/>{{end}}</svg>{{.Ref}}</a>{{end}}279
{{if .Short}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commit/{{.SHA}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true"><circle cx="8" cy="8" r="2.1" fill="currentColor"/><path d="M8 1.8v3.4M8 10.8v3.4" stroke="currentColor" stroke-width="1.4"/></svg>{{.Short}}</a>{{end}}279
{{if .Short}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commit/{{.SHA}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true"><circle cx="8" cy="8" r="2.1" fill="currentColor"/><path d="M8 1.8v3.4M8 10.8v3.4" stroke="currentColor" stroke-width="1.4"/></svg>{{.Short}}</a>{{end}}280
{{if .SourceLabel}}<span class="chip">{{.SourceLabel}}{{if .MRNumber}} !{{.MRNumber}}{{end}}</span>{{end}}280
{{if .Actor}}{{if .Avatar}}<img class="avatar sm" src="{{.Avatar}}" alt="">{{else}}<span class="avatar sm">{{.Initial}}</span>{{end}}{{end}}281
{{if .Actor}}{{if .Avatar}}<img class="avatar sm" src="{{.Avatar}}" alt="">{{else}}<span class="avatar sm">{{.Initial}}</span>{{end}}{{end}}281
</div>282
</div>282
</td>283
</td>@@ -399,6 +399,10 @@ type Pipeline struct {
399
399
StartedAt time.Time
400
400
FinishedAt time.Time
401
401
MRID int64
402
// TargetSHA is de doelcommit van een merge-request-pipeline.
403
// Leeg zolang de rij nog niet aan een merge request hangt.
404
// Een nieuwe doelcommit start een nieuwe pipeline.
405
TargetSHA string
402
406
}
403
407
404
408
// Job is één container in een pipeline.
@@ -442,30 +446,71 @@ type Job struct {
442
446
}
443
447
444
448
// RecordConfigFailure bewaart een pipeline die niet kon starten.
445
func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, message string) (int64, error) {
449
// Dezelfde repo, ref, sha, bron, merge request en doel-sha levert de
450
// bestaande pipeline terug. Een kapotte push die al bestaat wordt aan
451
// de merge request gekoppeld in plaats van een tweede mislukte rij.
452
func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, source, message string, mrID int64, targetSHA string) (int64, bool, error) {
453
if source == "" {
454
source = "push"
455
}
446
456
tx, err := s.db.BeginTx(ctx, nil)
447
457
if err != nil {
448
return 0, err
458
return 0, false, err
449
459
}
450
460
defer func() { _ = tx.Rollback() }()
461
if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {
462
return 0, false, err
463
} else if ok {
464
return finishExisting(ctx, tx, id)
465
}
466
if source == "push" && mrID == 0 {
467
if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {
468
return 0, false, err
469
} else if ok {
470
return finishExisting(ctx, tx, id)
471
}
472
}
473
if mrID != 0 {
474
if id, ok, err := configFailurePush(ctx, tx, repoID, sha, ref, mrID, targetSHA); err != nil {
475
return 0, false, err
476
} else if ok {
477
if err := bindPipelineMR(ctx, tx, id, mrID, targetSHA); err != nil {
478
return 0, false, err
479
}
480
return finishExisting(ctx, tx, id)
481
}
482
}
451
483
ts := now()
452
res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, created_at, finished_at) VALUES (?, ?, ?, 'failed', ?, ?)`, repoID, sha, ref, ts, ts)
484
res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, created_at, finished_at, mr_id, target_sha)
485
SELECT ?, ?, ?, 'failed', ?, ?, ?, ?, ?
486
WHERE NOT EXISTS (
487
SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?
488
)`, repoID, sha, ref, source, ts, ts, mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA)
453
489
if err != nil {
454
return 0, err
490
return 0, false, err
491
}
492
if n, _ := res.RowsAffected(); n == 0 {
493
id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
494
if err != nil {
495
return 0, false, err
496
}
497
return finishExisting(ctx, tx, id)
455
498
}
456
499
id, _ := res.LastInsertId()
457
500
_, err = tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, finished_at)
458
501
VALUES (?, ?, 'blue-ci-cd', 'test', 0, 'failed', 'alpine:3.20', 'exit 1', '[]', 0, '{}', '[]', 0, 0, ?, ?)`, id, repoID, message, ts)
459
502
if err != nil {
460
return 0, err
503
return 0, false, err
461
504
}
462
505
if err := tx.Commit(); err != nil {
463
return 0, err
506
return 0, false, err
464
507
}
465
return id, nil
508
return id, true, nil
466
509
}
467
510
468
511
// CreatePipeline maakt een pipeline met jobs in de wachtrij.
512
// Elke aanroep voegt een rij toe. Starten via een push of merge request
513
// gaat via OpenPipeline, zodat een dubbele hook dezelfde run teruggeeft.
469
514
func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title string, jobs []NewJob) (int64, error) {
470
515
tx, err := s.db.BeginTx(ctx, nil)
471
516
if err != nil {
@@ -477,6 +522,238 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
477
522
return 0, err
478
523
}
479
524
id, _ := res.LastInsertId()
525
if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {
526
return 0, err
527
}
528
if err := tx.Commit(); err != nil {
529
return 0, err
530
}
531
return id, nil
532
}
533
534
// OpenPipeline maakt een pipeline, of geeft de bestaande terug.
535
// De sleutel is repo, ref, sha, bron, merge request en doel-sha.
536
// Een push voor dezelfde sha blijft één rij, ook nadat een merge request
537
// hem heeft gekoppeld. Kiezen push en merge request dezelfde jobs, dan
538
// koppelt deze aanroep de push-rij en maakt hij geen tweede pipeline.
539
// Een andere jobset of een nieuwe doel-sha start wel een eigen rij.
540
// created is false wanneer een eerdere trigger de run al had.
541
func (s *Store) OpenPipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title, targetSHA string, mrID int64, jobs []NewJob) (int64, bool, error) {
542
if source == "" {
543
source = "push"
544
}
545
tx, err := s.db.BeginTx(ctx, nil)
546
if err != nil {
547
return 0, false, err
548
}
549
defer func() { _ = tx.Rollback() }()
550
if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {
551
return 0, false, err
552
} else if ok {
553
return finishExisting(ctx, tx, id)
554
}
555
if source == "push" && mrID == 0 {
556
if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {
557
return 0, false, err
558
} else if ok {
559
return finishExisting(ctx, tx, id)
560
}
561
}
562
if mrID != 0 {
563
if id, ok, err := claimMatchingPush(ctx, tx, repoID, sha, ref, mrID, targetSHA, jobs); err != nil {
564
return 0, false, err
565
} else if ok {
566
return finishExisting(ctx, tx, id)
567
}
568
}
569
notExists := `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?`
570
args := []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA}
571
if source == "push" && mrID == 0 {
572
notExists = `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'`
573
args = []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref}
574
}
575
res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, actor, commit_title, created_at, mr_id, target_sha)
576
SELECT ?, ?, ?, 'queued', ?, ?, ?, ?, ?, ?
577
WHERE NOT EXISTS (`+notExists+`)`, args...)
578
if err != nil {
579
return 0, false, err
580
}
581
if n, _ := res.RowsAffected(); n == 0 {
582
id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
583
if err != nil {
584
return 0, false, err
585
}
586
if !ok && source == "push" && mrID == 0 {
587
id, ok, err = newestPush(ctx, tx, repoID, sha, ref)
588
if err != nil {
589
return 0, false, err
590
}
591
}
592
if !ok {
593
return 0, false, errors.New("pipeline ontbreekt na een lege insert")
594
}
595
return finishExisting(ctx, tx, id)
596
}
597
id, _ := res.LastInsertId()
598
if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {
599
return 0, false, err
600
}
601
if err := tx.Commit(); err != nil {
602
return 0, false, err
603
}
604
return id, true, nil
605
}
606
607
func finishExisting(ctx context.Context, tx *sql.Tx, id int64) (int64, bool, error) {
608
if err := tx.Commit(); err != nil {
609
return 0, false, err
610
}
611
return id, false, nil
612
}
613
614
func existingPipeline(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, bool, error) {
615
id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
616
if errors.Is(err, sql.ErrNoRows) {
617
return 0, false, nil
618
}
619
if err != nil {
620
return 0, false, err
621
}
622
return id, true, nil
623
}
624
625
func pipelineKeyID(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, error) {
626
var id int64
627
err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ? ORDER BY id LIMIT 1`,
628
repoID, sha, ref, source, mrID, targetSHA).Scan(&id)
629
return id, err
630
}
631
632
func newestPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string) (int64, bool, error) {
633
var id int64
634
err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push' ORDER BY id DESC LIMIT 1`,
635
repoID, sha, ref).Scan(&id)
636
if errors.Is(err, sql.ErrNoRows) {
637
return 0, false, nil
638
}
639
if err != nil {
640
return 0, false, err
641
}
642
return id, true, nil
643
}
644
645
// configFailurePush zoekt een mislukte push die alleen de kapotte yaml bewaart.
646
func configFailurePush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string) (int64, bool, error) {
647
var id int64
648
err := tx.QueryRowContext(ctx, `SELECT p.id FROM pipelines p
649
WHERE p.repo_id = ? AND p.commit_sha = ? AND p.ref = ? AND p.source = 'push' AND p.status = 'failed'
650
AND (p.mr_id = 0 OR p.mr_id = ?) AND (p.target_sha = '' OR p.target_sha = ?)
651
AND EXISTS (SELECT 1 FROM jobs j WHERE j.pipeline_id = p.id AND j.name = 'blue-ci-cd')
652
ORDER BY p.id DESC LIMIT 1`, repoID, sha, ref, mrID, targetSHA).Scan(&id)
653
if errors.Is(err, sql.ErrNoRows) {
654
return 0, false, nil
655
}
656
if err != nil {
657
return 0, false, err
658
}
659
return id, true, nil
660
}
661
662
type pushCandidate struct {
663
id int64
664
mrID int64
665
target string
666
}
667
668
// claimMatchingPush koppelt een push-pipeline met dezelfde jobs aan de merge request.
669
func claimMatchingPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string, jobs []NewJob) (int64, bool, error) {
670
rows, err := tx.QueryContext(ctx, `SELECT id, mr_id, target_sha FROM pipelines
671
WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'
672
AND (mr_id = 0 OR mr_id = ?) AND (target_sha = '' OR target_sha = ?)
673
ORDER BY id DESC`, repoID, sha, ref, mrID, targetSHA)
674
if err != nil {
675
return 0, false, err
676
}
677
var cands []pushCandidate
678
for rows.Next() {
679
var c pushCandidate
680
if err := rows.Scan(&c.id, &c.mrID, &c.target); err != nil {
681
rows.Close()
682
return 0, false, err
683
}
684
cands = append(cands, c)
685
}
686
if err := rows.Err(); err != nil {
687
rows.Close()
688
return 0, false, err
689
}
690
rows.Close()
691
for _, c := range cands {
692
if c.mrID == mrID && c.target == targetSHA {
693
return c.id, true, nil
694
}
695
got, err := loadJobSigs(ctx, tx, c.id)
696
if err != nil {
697
return 0, false, err
698
}
699
if !sameJobSigs(got, jobs) {
700
continue
701
}
702
if err := bindPipelineMR(ctx, tx, c.id, mrID, targetSHA); err != nil {
703
return 0, false, err
704
}
705
return c.id, true, nil
706
}
707
return 0, false, nil
708
}
709
710
func bindPipelineMR(ctx context.Context, tx *sql.Tx, id, mrID int64, targetSHA string) error {
711
_, err := tx.ExecContext(ctx, `UPDATE pipelines SET mr_id = ?, target_sha = ? WHERE id = ?`, mrID, targetSHA, id)
712
return err
713
}
714
715
type jobSig struct {
716
name, stage, image, script, when string
717
allow bool
718
}
719
720
func loadJobSigs(ctx context.Context, tx *sql.Tx, pipelineID int64) ([]jobSig, error) {
721
rows, err := tx.QueryContext(ctx, `SELECT name, stage, image, script, when_mode, allow_failure FROM jobs WHERE pipeline_id = ? ORDER BY position, id`, pipelineID)
722
if err != nil {
723
return nil, err
724
}
725
defer rows.Close()
726
var out []jobSig
727
for rows.Next() {
728
var s jobSig
729
var allow int
730
if err := rows.Scan(&s.name, &s.stage, &s.image, &s.script, &s.when, &allow); err != nil {
731
return nil, err
732
}
733
s.allow = allow == 1
734
out = append(out, s)
735
}
736
return out, rows.Err()
737
}
738
739
func sameJobSigs(got []jobSig, jobs []NewJob) bool {
740
if len(got) != len(jobs) {
741
return false
742
}
743
for i, j := range jobs {
744
when := j.When
745
if when == "" {
746
when = "on_success"
747
}
748
g := got[i]
749
if g.name != j.Name || g.stage != j.Stage || g.image != j.Image || g.script != j.Script || g.when != when || g.allow != j.AllowFailure {
750
return false
751
}
752
}
753
return true
754
}
755
756
func insertPipelineJobs(ctx context.Context, tx *sql.Tx, id, repoID int64, jobs []NewJob) error {
480
757
for _, j := range jobs {
481
758
if j.Variables == nil {
482
759
j.Variables = map[string]string{}
@@ -515,13 +792,10 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
515
792
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),
516
793
j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))
517
794
if err != nil {
518
return 0, err
795
return err
519
796
}
520
797
}
521
if err := tx.Commit(); err != nil {
522
return 0, err
523
}
524
return id, nil
798
return nil
525
799
}
526
800
527
801
// ListPipelines geeft de nieuwste pipelines.
@@ -542,7 +816,7 @@ func (s *Store) ListPipelines(ctx context.Context, repoID int64) ([]Pipeline, er
542
816
return out, rows.Err()
543
817
}
544
818
545
const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id FROM pipelines`
819
const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`
546
820
547
821
func scanPipeline(rows *sql.Rows) (Pipeline, error) {
548
822
return scanPipelineRow(rows)
@@ -551,7 +825,7 @@ func scanPipeline(rows *sql.Rows) (Pipeline, error) {
551
825
func scanPipelineRow(sc scanner) (Pipeline, error) {
552
826
var p Pipeline
553
827
var created, finished, started sql.NullString
554
if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID); err != nil {
828
if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID, &p.TargetSHA); err != nil {
555
829
return Pipeline{}, err
556
830
}
557
831
p.CreatedAt = parseTime(created.String)
@@ -399,6 +399,10 @@ type Pipeline struct {399
StartedAt time.Time399
StartedAt time.Time400
FinishedAt time.Time400
FinishedAt time.Time401
MRID int64401
MRID int64402
// TargetSHA is de doelcommit van een merge-request-pipeline.403
// Leeg zolang de rij nog niet aan een merge request hangt.404
// Een nieuwe doelcommit start een nieuwe pipeline.405
TargetSHA string402
}406
}403
407
404
// Job is één container in een pipeline.408
// Job is één container in een pipeline.@@ -442,30 +446,71 @@ type Job struct {442
}446
}443
447
444
// RecordConfigFailure bewaart een pipeline die niet kon starten.448
// RecordConfigFailure bewaart een pipeline die niet kon starten.445
func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, message string) (int64, error) {449
// Dezelfde repo, ref, sha, bron, merge request en doel-sha levert de450
// bestaande pipeline terug. Een kapotte push die al bestaat wordt aan451
// de merge request gekoppeld in plaats van een tweede mislukte rij.452
func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, source, message string, mrID int64, targetSHA string) (int64, bool, error) {453
if source == "" {454
source = "push"455
}446
tx, err := s.db.BeginTx(ctx, nil)456
tx, err := s.db.BeginTx(ctx, nil)447
if err != nil {457
if err != nil {448
return 0, err458
return 0, false, err449
}459
}450
defer func() { _ = tx.Rollback() }()460
defer func() { _ = tx.Rollback() }()461
if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {462
return 0, false, err463
} else if ok {464
return finishExisting(ctx, tx, id)465
}466
if source == "push" && mrID == 0 {467
if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {468
return 0, false, err469
} else if ok {470
return finishExisting(ctx, tx, id)471
}472
}473
if mrID != 0 {474
if id, ok, err := configFailurePush(ctx, tx, repoID, sha, ref, mrID, targetSHA); err != nil {475
return 0, false, err476
} else if ok {477
if err := bindPipelineMR(ctx, tx, id, mrID, targetSHA); err != nil {478
return 0, false, err479
}480
return finishExisting(ctx, tx, id)481
}482
}451
ts := now()483
ts := now()452
res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, created_at, finished_at) VALUES (?, ?, ?, 'failed', ?, ?)`, repoID, sha, ref, ts, ts)484
res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, created_at, finished_at, mr_id, target_sha)485
SELECT ?, ?, ?, 'failed', ?, ?, ?, ?, ?486
WHERE NOT EXISTS (487
SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?488
)`, repoID, sha, ref, source, ts, ts, mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA)453
if err != nil {489
if err != nil {454
return 0, err490
return 0, false, err491
}492
if n, _ := res.RowsAffected(); n == 0 {493
id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)494
if err != nil {495
return 0, false, err496
}497
return finishExisting(ctx, tx, id)455
}498
}456
id, _ := res.LastInsertId()499
id, _ := res.LastInsertId()457
_, err = tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, finished_at)500
_, err = tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, finished_at)458
VALUES (?, ?, 'blue-ci-cd', 'test', 0, 'failed', 'alpine:3.20', 'exit 1', '[]', 0, '{}', '[]', 0, 0, ?, ?)`, id, repoID, message, ts)501
VALUES (?, ?, 'blue-ci-cd', 'test', 0, 'failed', 'alpine:3.20', 'exit 1', '[]', 0, '{}', '[]', 0, 0, ?, ?)`, id, repoID, message, ts)459
if err != nil {502
if err != nil {460
return 0, err503
return 0, false, err461
}504
}462
if err := tx.Commit(); err != nil {505
if err := tx.Commit(); err != nil {463
return 0, err506
return 0, false, err464
}507
}465
return id, nil508
return id, true, nil466
}509
}467
510
468
// CreatePipeline maakt een pipeline met jobs in de wachtrij.511
// CreatePipeline maakt een pipeline met jobs in de wachtrij.512
// Elke aanroep voegt een rij toe. Starten via een push of merge request513
// gaat via OpenPipeline, zodat een dubbele hook dezelfde run teruggeeft.469
func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title string, jobs []NewJob) (int64, error) {514
func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title string, jobs []NewJob) (int64, error) {470
tx, err := s.db.BeginTx(ctx, nil)515
tx, err := s.db.BeginTx(ctx, nil)471
if err != nil {516
if err != nil {@@ -477,6 +522,238 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour477
return 0, err522
return 0, err478
}523
}479
id, _ := res.LastInsertId()524
id, _ := res.LastInsertId()525
if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {526
return 0, err527
}528
if err := tx.Commit(); err != nil {529
return 0, err530
}531
return id, nil532
}533
534
// OpenPipeline maakt een pipeline, of geeft de bestaande terug.535
// De sleutel is repo, ref, sha, bron, merge request en doel-sha.536
// Een push voor dezelfde sha blijft één rij, ook nadat een merge request537
// hem heeft gekoppeld. Kiezen push en merge request dezelfde jobs, dan538
// koppelt deze aanroep de push-rij en maakt hij geen tweede pipeline.539
// Een andere jobset of een nieuwe doel-sha start wel een eigen rij.540
// created is false wanneer een eerdere trigger de run al had.541
func (s *Store) OpenPipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title, targetSHA string, mrID int64, jobs []NewJob) (int64, bool, error) {542
if source == "" {543
source = "push"544
}545
tx, err := s.db.BeginTx(ctx, nil)546
if err != nil {547
return 0, false, err548
}549
defer func() { _ = tx.Rollback() }()550
if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {551
return 0, false, err552
} else if ok {553
return finishExisting(ctx, tx, id)554
}555
if source == "push" && mrID == 0 {556
if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {557
return 0, false, err558
} else if ok {559
return finishExisting(ctx, tx, id)560
}561
}562
if mrID != 0 {563
if id, ok, err := claimMatchingPush(ctx, tx, repoID, sha, ref, mrID, targetSHA, jobs); err != nil {564
return 0, false, err565
} else if ok {566
return finishExisting(ctx, tx, id)567
}568
}569
notExists := `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?`570
args := []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA}571
if source == "push" && mrID == 0 {572
notExists = `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'`573
args = []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref}574
}575
res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, actor, commit_title, created_at, mr_id, target_sha)576
SELECT ?, ?, ?, 'queued', ?, ?, ?, ?, ?, ?577
WHERE NOT EXISTS (`+notExists+`)`, args...)578
if err != nil {579
return 0, false, err580
}581
if n, _ := res.RowsAffected(); n == 0 {582
id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)583
if err != nil {584
return 0, false, err585
}586
if !ok && source == "push" && mrID == 0 {587
id, ok, err = newestPush(ctx, tx, repoID, sha, ref)588
if err != nil {589
return 0, false, err590
}591
}592
if !ok {593
return 0, false, errors.New("pipeline ontbreekt na een lege insert")594
}595
return finishExisting(ctx, tx, id)596
}597
id, _ := res.LastInsertId()598
if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {599
return 0, false, err600
}601
if err := tx.Commit(); err != nil {602
return 0, false, err603
}604
return id, true, nil605
}606
607
func finishExisting(ctx context.Context, tx *sql.Tx, id int64) (int64, bool, error) {608
if err := tx.Commit(); err != nil {609
return 0, false, err610
}611
return id, false, nil612
}613
614
func existingPipeline(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, bool, error) {615
id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)616
if errors.Is(err, sql.ErrNoRows) {617
return 0, false, nil618
}619
if err != nil {620
return 0, false, err621
}622
return id, true, nil623
}624
625
func pipelineKeyID(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, error) {626
var id int64627
err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ? ORDER BY id LIMIT 1`,628
repoID, sha, ref, source, mrID, targetSHA).Scan(&id)629
return id, err630
}631
632
func newestPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string) (int64, bool, error) {633
var id int64634
err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push' ORDER BY id DESC LIMIT 1`,635
repoID, sha, ref).Scan(&id)636
if errors.Is(err, sql.ErrNoRows) {637
return 0, false, nil638
}639
if err != nil {640
return 0, false, err641
}642
return id, true, nil643
}644
645
// configFailurePush zoekt een mislukte push die alleen de kapotte yaml bewaart.646
func configFailurePush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string) (int64, bool, error) {647
var id int64648
err := tx.QueryRowContext(ctx, `SELECT p.id FROM pipelines p649
WHERE p.repo_id = ? AND p.commit_sha = ? AND p.ref = ? AND p.source = 'push' AND p.status = 'failed'650
AND (p.mr_id = 0 OR p.mr_id = ?) AND (p.target_sha = '' OR p.target_sha = ?)651
AND EXISTS (SELECT 1 FROM jobs j WHERE j.pipeline_id = p.id AND j.name = 'blue-ci-cd')652
ORDER BY p.id DESC LIMIT 1`, repoID, sha, ref, mrID, targetSHA).Scan(&id)653
if errors.Is(err, sql.ErrNoRows) {654
return 0, false, nil655
}656
if err != nil {657
return 0, false, err658
}659
return id, true, nil660
}661
662
type pushCandidate struct {663
id int64664
mrID int64665
target string666
}667
668
// claimMatchingPush koppelt een push-pipeline met dezelfde jobs aan de merge request.669
func claimMatchingPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string, jobs []NewJob) (int64, bool, error) {670
rows, err := tx.QueryContext(ctx, `SELECT id, mr_id, target_sha FROM pipelines671
WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'672
AND (mr_id = 0 OR mr_id = ?) AND (target_sha = '' OR target_sha = ?)673
ORDER BY id DESC`, repoID, sha, ref, mrID, targetSHA)674
if err != nil {675
return 0, false, err676
}677
var cands []pushCandidate678
for rows.Next() {679
var c pushCandidate680
if err := rows.Scan(&c.id, &c.mrID, &c.target); err != nil {681
rows.Close()682
return 0, false, err683
}684
cands = append(cands, c)685
}686
if err := rows.Err(); err != nil {687
rows.Close()688
return 0, false, err689
}690
rows.Close()691
for _, c := range cands {692
if c.mrID == mrID && c.target == targetSHA {693
return c.id, true, nil694
}695
got, err := loadJobSigs(ctx, tx, c.id)696
if err != nil {697
return 0, false, err698
}699
if !sameJobSigs(got, jobs) {700
continue701
}702
if err := bindPipelineMR(ctx, tx, c.id, mrID, targetSHA); err != nil {703
return 0, false, err704
}705
return c.id, true, nil706
}707
return 0, false, nil708
}709
710
func bindPipelineMR(ctx context.Context, tx *sql.Tx, id, mrID int64, targetSHA string) error {711
_, err := tx.ExecContext(ctx, `UPDATE pipelines SET mr_id = ?, target_sha = ? WHERE id = ?`, mrID, targetSHA, id)712
return err713
}714
715
type jobSig struct {716
name, stage, image, script, when string717
allow bool718
}719
720
func loadJobSigs(ctx context.Context, tx *sql.Tx, pipelineID int64) ([]jobSig, error) {721
rows, err := tx.QueryContext(ctx, `SELECT name, stage, image, script, when_mode, allow_failure FROM jobs WHERE pipeline_id = ? ORDER BY position, id`, pipelineID)722
if err != nil {723
return nil, err724
}725
defer rows.Close()726
var out []jobSig727
for rows.Next() {728
var s jobSig729
var allow int730
if err := rows.Scan(&s.name, &s.stage, &s.image, &s.script, &s.when, &allow); err != nil {731
return nil, err732
}733
s.allow = allow == 1734
out = append(out, s)735
}736
return out, rows.Err()737
}738
739
func sameJobSigs(got []jobSig, jobs []NewJob) bool {740
if len(got) != len(jobs) {741
return false742
}743
for i, j := range jobs {744
when := j.When745
if when == "" {746
when = "on_success"747
}748
g := got[i]749
if g.name != j.Name || g.stage != j.Stage || g.image != j.Image || g.script != j.Script || g.when != when || g.allow != j.AllowFailure {750
return false751
}752
}753
return true754
}755
756
func insertPipelineJobs(ctx context.Context, tx *sql.Tx, id, repoID int64, jobs []NewJob) error {480
for _, j := range jobs {757
for _, j := range jobs {481
if j.Variables == nil {758
if j.Variables == nil {482
j.Variables = map[string]string{}759
j.Variables = map[string]string{}@@ -515,13 +792,10 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour515
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),792
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),516
j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))793
j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))517
if err != nil {794
if err != nil {518
return 0, err795
return err519
}796
}520
}797
}521
if err := tx.Commit(); err != nil {798
return nil522
return 0, err523
}524
return id, nil525
}799
}526
800
527
// ListPipelines geeft de nieuwste pipelines.801
// ListPipelines geeft de nieuwste pipelines.@@ -542,7 +816,7 @@ func (s *Store) ListPipelines(ctx context.Context, repoID int64) ([]Pipeline, er542
return out, rows.Err()816
return out, rows.Err()543
}817
}544
818
545
const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id FROM pipelines`819
const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`546
820
547
func scanPipeline(rows *sql.Rows) (Pipeline, error) {821
func scanPipeline(rows *sql.Rows) (Pipeline, error) {548
return scanPipelineRow(rows)822
return scanPipelineRow(rows)@@ -551,7 +825,7 @@ func scanPipeline(rows *sql.Rows) (Pipeline, error) {551
func scanPipelineRow(sc scanner) (Pipeline, error) {825
func scanPipelineRow(sc scanner) (Pipeline, error) {552
var p Pipeline826
var p Pipeline553
var created, finished, started sql.NullString827
var created, finished, started sql.NullString554
if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID); err != nil {828
if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID, &p.TargetSHA); err != nil {555
return Pipeline{}, err829
return Pipeline{}, err556
}830
}557
p.CreatedAt = parseTime(created.String)831
p.CreatedAt = parseTime(created.String)@@ -0,0 +1,127 @@
1
package store
2
3
import (
4
"context"
5
"testing"
6
)
7
8
func TestOpenPipelineKeepsOnePerSource(t *testing.T) {
9
s := openTest(t)
10
ctx := context.Background()
11
tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
12
if err != nil {
13
t.Fatal(err)
14
}
15
repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
16
if err != nil {
17
t.Fatal(err)
18
}
19
sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
20
ref := "refs/heads/gitlab-sync"
21
jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}
22
23
push, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
24
if err != nil || !created || push == 0 {
25
t.Fatalf("push: %d %v %v", push, created, err)
26
}
27
again, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
28
if err != nil || created || again != push {
29
t.Fatalf("tweede push: %d created=%v %v", again, created, err)
30
}
31
got, err := s.JobsByPipeline(ctx, push)
32
if err != nil || len(got) != 1 {
33
t.Fatalf("jobs verdubbeld: %d %v", len(got), err)
34
}
35
36
mr, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)
37
if err != nil || created || mr != push {
38
t.Fatalf("zelfde jobs horen de push te claimen: %d created=%v %v", mr, created, err)
39
}
40
linked, err := s.PipelineByID(ctx, repo.ID, push)
41
if err != nil || linked.MRID != 7 || linked.TargetSHA != "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" || linked.Source != "push" {
42
t.Fatalf("gekoppelde push: %+v %v", linked, err)
43
}
44
if jobs, err := s.JobsByPipeline(ctx, push); err != nil || len(jobs) != 1 {
45
t.Fatalf("claim voegt jobs toe: %d %v", len(jobs), err)
46
}
47
same, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)
48
if err != nil || created || same != push {
49
t.Fatalf("zelfde doel: %d created=%v %v", same, created, err)
50
}
51
replay, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
52
if err != nil || created || replay != push {
53
t.Fatalf("push na koppeling: %d created=%v %v", replay, created, err)
54
}
55
moved, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "cccccccccccccccccccccccccccccccccccccccc", 7, jobs)
56
if err != nil || !created || moved == push {
57
t.Fatalf("nieuwe doel-sha: %d created=%v %v", moved, created, err)
58
}
59
fresh, err := s.PipelineByID(ctx, repo.ID, moved)
60
if err != nil || fresh.Source != "merge_request_event" || fresh.MRID != 7 || fresh.TargetSHA != "cccccccccccccccccccccccccccccccccccccccc" {
61
t.Fatalf("nieuwe mr-pipeline: %+v %v", fresh, err)
62
}
63
other, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "web", "tim", "hand", "", 0, jobs)
64
if err != nil || !created || other == push {
65
t.Fatalf("web blijft een eigen bron: %d %v %v", other, created, err)
66
}
67
68
diff := []NewJob{{Name: "mr_job", Stage: "test", Image: "alpine:3.20", Script: "echo mr"}}
69
otherSHA := "dddddddddddddddddddddddddddddddddddddddd"
70
pushOnly, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "push", "tim", "anders", "", 0, jobs)
71
if err != nil || !created {
72
t.Fatalf("andere sha: %d %v %v", pushOnly, created, err)
73
}
74
apart, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "merge_request_event", "tim", "anders", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, diff)
75
if err != nil || !created || apart == pushOnly {
76
t.Fatalf("andere jobs blijven apart: %d created=%v %v", apart, created, err)
77
}
78
79
failID, created, err := s.RecordConfigFailure(ctx, repo.ID, sha, ref, "push", "kapot", 0, "")
80
if err != nil || created || failID != push {
81
t.Fatalf("kapotte yaml bovenop een push: %d created=%v %v", failID, created, err)
82
}
83
broken := "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
84
first, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")
85
if err != nil || !created || first == 0 {
86
t.Fatalf("eerste fout: %d %v %v", first, created, err)
87
}
88
second, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")
89
if err != nil || created || second != first {
90
t.Fatalf("tweede fout: %d created=%v %v", second, created, err)
91
}
92
claimed, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "merge_request_event", "kapot", 9, "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
93
if err != nil || created || claimed != first {
94
t.Fatalf("mr-fout claimt de push-fout: %d created=%v %v", claimed, created, err)
95
}
96
linked, err = s.PipelineByID(ctx, repo.ID, first)
97
if err != nil || linked.MRID != 9 {
98
t.Fatalf("fout gekoppeld: %+v %v", linked, err)
99
}
100
}
101
102
func TestCreatePipelineStaysDuplicating(t *testing.T) {
103
s := openTest(t)
104
ctx := context.Background()
105
tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
106
if err != nil {
107
t.Fatal(err)
108
}
109
repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
110
if err != nil {
111
t.Fatal(err)
112
}
113
jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}
114
sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
115
ref := "refs/heads/master"
116
first, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "een", jobs)
117
if err != nil {
118
t.Fatal(err)
119
}
120
second, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "twee", jobs)
121
if err != nil {
122
t.Fatal(err)
123
}
124
if first == 0 || second == 0 || first == second {
125
t.Fatalf("CreatePipeline hoort twee rijen te maken: %d %d", first, second)
126
}
127
}
@@ -0,0 +1,127 @@1
package store2
3
import (4
"context"5
"testing"6
)7
8
func TestOpenPipelineKeepsOnePerSource(t *testing.T) {9
s := openTest(t)10
ctx := context.Background()11
tim, err := s.UpsertUser(ctx, 1, "tim", "", false)12
if err != nil {13
t.Fatal(err)14
}15
repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")16
if err != nil {17
t.Fatal(err)18
}19
sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"20
ref := "refs/heads/gitlab-sync"21
jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}22
23
push, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)24
if err != nil || !created || push == 0 {25
t.Fatalf("push: %d %v %v", push, created, err)26
}27
again, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)28
if err != nil || created || again != push {29
t.Fatalf("tweede push: %d created=%v %v", again, created, err)30
}31
got, err := s.JobsByPipeline(ctx, push)32
if err != nil || len(got) != 1 {33
t.Fatalf("jobs verdubbeld: %d %v", len(got), err)34
}35
36
mr, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)37
if err != nil || created || mr != push {38
t.Fatalf("zelfde jobs horen de push te claimen: %d created=%v %v", mr, created, err)39
}40
linked, err := s.PipelineByID(ctx, repo.ID, push)41
if err != nil || linked.MRID != 7 || linked.TargetSHA != "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" || linked.Source != "push" {42
t.Fatalf("gekoppelde push: %+v %v", linked, err)43
}44
if jobs, err := s.JobsByPipeline(ctx, push); err != nil || len(jobs) != 1 {45
t.Fatalf("claim voegt jobs toe: %d %v", len(jobs), err)46
}47
same, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)48
if err != nil || created || same != push {49
t.Fatalf("zelfde doel: %d created=%v %v", same, created, err)50
}51
replay, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)52
if err != nil || created || replay != push {53
t.Fatalf("push na koppeling: %d created=%v %v", replay, created, err)54
}55
moved, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "cccccccccccccccccccccccccccccccccccccccc", 7, jobs)56
if err != nil || !created || moved == push {57
t.Fatalf("nieuwe doel-sha: %d created=%v %v", moved, created, err)58
}59
fresh, err := s.PipelineByID(ctx, repo.ID, moved)60
if err != nil || fresh.Source != "merge_request_event" || fresh.MRID != 7 || fresh.TargetSHA != "cccccccccccccccccccccccccccccccccccccccc" {61
t.Fatalf("nieuwe mr-pipeline: %+v %v", fresh, err)62
}63
other, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "web", "tim", "hand", "", 0, jobs)64
if err != nil || !created || other == push {65
t.Fatalf("web blijft een eigen bron: %d %v %v", other, created, err)66
}67
68
diff := []NewJob{{Name: "mr_job", Stage: "test", Image: "alpine:3.20", Script: "echo mr"}}69
otherSHA := "dddddddddddddddddddddddddddddddddddddddd"70
pushOnly, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "push", "tim", "anders", "", 0, jobs)71
if err != nil || !created {72
t.Fatalf("andere sha: %d %v %v", pushOnly, created, err)73
}74
apart, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "merge_request_event", "tim", "anders", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, diff)75
if err != nil || !created || apart == pushOnly {76
t.Fatalf("andere jobs blijven apart: %d created=%v %v", apart, created, err)77
}78
79
failID, created, err := s.RecordConfigFailure(ctx, repo.ID, sha, ref, "push", "kapot", 0, "")80
if err != nil || created || failID != push {81
t.Fatalf("kapotte yaml bovenop een push: %d created=%v %v", failID, created, err)82
}83
broken := "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"84
first, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")85
if err != nil || !created || first == 0 {86
t.Fatalf("eerste fout: %d %v %v", first, created, err)87
}88
second, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")89
if err != nil || created || second != first {90
t.Fatalf("tweede fout: %d created=%v %v", second, created, err)91
}92
claimed, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "merge_request_event", "kapot", 9, "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")93
if err != nil || created || claimed != first {94
t.Fatalf("mr-fout claimt de push-fout: %d created=%v %v", claimed, created, err)95
}96
linked, err = s.PipelineByID(ctx, repo.ID, first)97
if err != nil || linked.MRID != 9 {98
t.Fatalf("fout gekoppeld: %+v %v", linked, err)99
}100
}101
102
func TestCreatePipelineStaysDuplicating(t *testing.T) {103
s := openTest(t)104
ctx := context.Background()105
tim, err := s.UpsertUser(ctx, 1, "tim", "", false)106
if err != nil {107
t.Fatal(err)108
}109
repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")110
if err != nil {111
t.Fatal(err)112
}113
jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}114
sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"115
ref := "refs/heads/master"116
first, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "een", jobs)117
if err != nil {118
t.Fatal(err)119
}120
second, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "twee", jobs)121
if err != nil {122
t.Fatal(err)123
}124
if first == 0 || second == 0 || first == second {125
t.Fatalf("CreatePipeline hoort twee rijen te maken: %d %d", first, second)126
}127
}@@ -99,6 +99,7 @@ func (s *Store) migrate() error {
99
99
`ALTER TABLE secrets ADD COLUMN invisible INTEGER NOT NULL DEFAULT 0`,
100
100
`ALTER TABLE webhooks ADD COLUMN fields_json TEXT NOT NULL DEFAULT ''`,
101
101
`ALTER TABLE access_tokens ADD COLUMN scopes TEXT NOT NULL DEFAULT 'api'`,
102
`ALTER TABLE pipelines ADD COLUMN target_sha TEXT NOT NULL DEFAULT ''`,
102
103
}
103
104
for _, q := range alters {
104
105
if _, err := s.db.Exec(q); err != nil && !strings.Contains(strings.ToLower(err.Error()), "duplicate column") {
@@ -99,6 +99,7 @@ func (s *Store) migrate() error {99
`ALTER TABLE secrets ADD COLUMN invisible INTEGER NOT NULL DEFAULT 0`,99
`ALTER TABLE secrets ADD COLUMN invisible INTEGER NOT NULL DEFAULT 0`,100
`ALTER TABLE webhooks ADD COLUMN fields_json TEXT NOT NULL DEFAULT ''`,100
`ALTER TABLE webhooks ADD COLUMN fields_json TEXT NOT NULL DEFAULT ''`,101
`ALTER TABLE access_tokens ADD COLUMN scopes TEXT NOT NULL DEFAULT 'api'`,101
`ALTER TABLE access_tokens ADD COLUMN scopes TEXT NOT NULL DEFAULT 'api'`,102
`ALTER TABLE pipelines ADD COLUMN target_sha TEXT NOT NULL DEFAULT ''`,102
}103
}103
for _, q := range alters {104
for _, q := range alters {104
if _, err := s.db.Exec(q); err != nil && !strings.Contains(strings.ToLower(err.Error()), "duplicate column") {105
if _, err := s.db.Exec(q); err != nil && !strings.Contains(strings.ToLower(err.Error()), "duplicate column") {