BlueGit

timfa bluegit

fix: start per sha geen dubbele push- en mr-pipeline

Een push en een merge request die dezelfde jobs zouden draaien delen
nu één pipeline. Die blijft aan de merge request hangen en is in de
lijst gelabeld. Afwijkende jobs blijven twee pipelines, en een nieuwe
doelcommit start opnieuw een merge-request-pipeline.

Co-authored-by: tim <tim@timfalken.com>

timfa project_87923614_bot_3aff574f6f98414b945a8799374f9092 · pushed by timfa

Authored 2026-10-23 · 3 weeks ago

Hash
6a8f3828204413ba250b8329e2b8d3b848d9ab38
Parent
6d21d25c
Pipeline
Passed
10 files changed +677 −41
docs/CI.md +1 −1
@@ -74,7 +74,7 @@ $NAAM =~ $PATROON
74 74
75 75 Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_COMMIT_TIMESTAMP`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. `CI_COMMIT_TIMESTAMP` is de committer-tijd in ISO 8601. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.
76 76
77 Een merge-request-pipeline start bij het openen van de merge request, bij elke push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht. Die laatste draait nog steeds op de commit van de bronbranch, niet op een samengevoegde boom. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
77 Een merge-request-pipeline start bij het openen van de merge request, bij een push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht en de doelcommit daardoor verandert. Die draait op de commit van de bronbranch, niet op een samengevoegde boom. Dezelfde combinatie van ref, sha, bron, merge request en doelcommit start geen tweede pipeline. Kiezen de rules voor de push en voor de merge request dezelfde jobs, dan blijft één pipeline over en wordt die aan de merge request gekoppeld. Kiezen ze andere jobs, dan blijven beide bestaan en staat op de merge-request-pipeline het label Merge request met het nummer. `CI_PIPELINE_SOURCE` is `merge_request_event` op een pipeline die als merge request is gestart. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
78 78
79 79 `CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.
80 80
@@ -74,7 +74,7 @@ $NAAM =~ $PATROON
74
74
75Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_COMMIT_TIMESTAMP`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. `CI_COMMIT_TIMESTAMP` is de committer-tijd in ISO 8601. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.
75Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_COMMIT_TIMESTAMP`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. `CI_COMMIT_TIMESTAMP` is de committer-tijd in ISO 8601. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.
76
76
77Een merge-request-pipeline start bij het openen van de merge request, bij elke push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht. Die laatste draait nog steeds op de commit van de bronbranch, niet op een samengevoegde boom. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
77Een merge-request-pipeline start bij het openen van de merge request, bij een push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht en de doelcommit daardoor verandert. Die draait op de commit van de bronbranch, niet op een samengevoegde boom. Dezelfde combinatie van ref, sha, bron, merge request en doelcommit start geen tweede pipeline. Kiezen de rules voor de push en voor de merge request dezelfde jobs, dan blijft één pipeline over en wordt die aan de merge request gekoppeld. Kiezen ze andere jobs, dan blijven beide bestaan en staat op de merge-request-pipeline het label Merge request met het nummer. `CI_PIPELINE_SOURCE` is `merge_request_event` op een pipeline die als merge request is gestart. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
78
78
79`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.
79`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.
80
80
internal/server/events.go +9 −9
@@ -134,15 +134,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
134 134 return 0
135 135 }
136 136 title, jobs, broken := s.planPipeline(ctx, repo, sha, ref, source, before, mr)
137 mrID, target := int64(0), ""
138 if mr != nil {
139 mrID, target = mr.ID, mr.TargetSHA
140 }
137 141 if broken != nil {
138 id, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, broken.Error())
142 id, created, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, source, broken.Error(), mrID, target)
139 143 if err != nil && s.Log != nil {
140 144 s.Log.Error("pipeline", "err", err)
141 145 }
142 if mr != nil && id != 0 {
143 _ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
144 }
145 if id != 0 {
146 if created && id != 0 {
146 147 s.cancelOlder(ctx, repo.ID, ref, id)
147 148 }
148 149 return id
@@ -150,17 +151,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
150 151 if len(jobs) == 0 {
151 152 return 0
152 153 }
153 id, err := s.Store.CreatePipeline(ctx, repo.ID, sha, ref, source, actor, title, jobs)
154 id, created, err := s.Store.OpenPipeline(ctx, repo.ID, sha, ref, source, actor, title, target, mrID, jobs)
154 155 if err != nil {
155 156 if s.Log != nil {
156 157 s.Log.Error("pipeline", "err", err)
157 158 }
158 159 return 0
159 160 }
160 if mr != nil {
161 _ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
161 if created {
162 s.cancelOlder(ctx, repo.ID, ref, id)
162 163 }
163 s.cancelOlder(ctx, repo.ID, ref, id)
164 164 return id
165 165 }
166 166
@@ -134,15 +134,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
134 return 0
134 return 0
135 }
135 }
136 title, jobs, broken := s.planPipeline(ctx, repo, sha, ref, source, before, mr)
136 title, jobs, broken := s.planPipeline(ctx, repo, sha, ref, source, before, mr)
137 mrID, target := int64(0), ""
138 if mr != nil {
139 mrID, target = mr.ID, mr.TargetSHA
140 }
137 if broken != nil {
141 if broken != nil {
138 id, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, broken.Error())
142 id, created, err := s.Store.RecordConfigFailure(ctx, repo.ID, sha, ref, source, broken.Error(), mrID, target)
139 if err != nil && s.Log != nil {
143 if err != nil && s.Log != nil {
140 s.Log.Error("pipeline", "err", err)
144 s.Log.Error("pipeline", "err", err)
141 }
145 }
142 if mr != nil && id != 0 {
146 if created && id != 0 {
143 _ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
144 }
145 if id != 0 {
146 s.cancelOlder(ctx, repo.ID, ref, id)
147 s.cancelOlder(ctx, repo.ID, ref, id)
147 }
148 }
148 return id
149 return id
@@ -150,17 +151,16 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
150 if len(jobs) == 0 {
151 if len(jobs) == 0 {
151 return 0
152 return 0
152 }
153 }
153 id, err := s.Store.CreatePipeline(ctx, repo.ID, sha, ref, source, actor, title, jobs)
154 id, created, err := s.Store.OpenPipeline(ctx, repo.ID, sha, ref, source, actor, title, target, mrID, jobs)
154 if err != nil {
155 if err != nil {
155 if s.Log != nil {
156 if s.Log != nil {
156 s.Log.Error("pipeline", "err", err)
157 s.Log.Error("pipeline", "err", err)
157 }
158 }
158 return 0
159 return 0
159 }
160 }
160 if mr != nil {
161 if created {
161 _ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
162 s.cancelOlder(ctx, repo.ID, ref, id)
162 }
163 }
163 s.cancelOlder(ctx, repo.ID, ref, id)
164 return id
164 return id
165}
165}
166
166
internal/server/mr_pipeline_test.go +193 −0
@@ -8,11 +8,13 @@ import (
8 8 "net/url"
9 9 "os/exec"
10 10 "path/filepath"
11 "strconv"
11 12 "strings"
12 13 "testing"
13 14
14 15 "gitlab.com/Timfa/bluegit/internal/auth"
15 16 "gitlab.com/Timfa/bluegit/internal/config"
17 "gitlab.com/Timfa/bluegit/internal/gitrepo"
16 18 "gitlab.com/Timfa/bluegit/internal/store"
17 19 )
18 20
@@ -138,6 +140,38 @@ soft:
138 140 if mrPipe.ID == 0 || mrPipe.MRID == 0 {
139 141 t.Fatalf("mr-pipeline: %+v", mrPipe)
140 142 }
143 nBefore := len(pipes)
144 user, err := st.UserByUsername(ctx, "tim")
145 if err != nil {
146 t.Fatal(err)
147 }
148 dir := gitrepo.Dir(storage, "tim", "mrpipes")
149 after, err := gitrepo.Refs(ctx, "git", dir)
150 if err != nil {
151 t.Fatal(err)
152 }
153 before := map[string]string{}
154 for ref, sha := range after {
155 before[ref] = sha
156 }
157 before["refs/heads/feature"] = strings.Repeat("a", 40)
158 srv.AfterPush(repo, user, before, after)
159 srv.rerunMergeRequestsOnTarget(ctx, repo, "master", "tim")
160 pipes, err = st.ListPipelines(ctx, repo.ID)
161 if err != nil {
162 t.Fatal(err)
163 }
164 if len(pipes) != nBefore {
165 t.Fatalf("dezelfde push nog eens: %d pipelines, was %d", len(pipes), nBefore)
166 }
167 list := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines"))
168 if !strings.Contains(list, "Merge request !1") {
169 t.Fatalf("lijst mist het mr-label:\n%s", list)
170 }
171 detail := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines/"+strconv.FormatInt(mrPipe.ID, 10)))
172 if !strings.Contains(detail, "Merge request !1") {
173 t.Fatalf("detail mist het mr-label:\n%s", detail)
174 }
141 175 res, err = tim.PostForm(ts.URL+"/tim/mrpipes/-/merge-requests/1/merge", url.Values{"csrf": {csrfOf(t, page)}})
142 176 if err != nil {
143 177 t.Fatal(err)
@@ -211,3 +245,162 @@ soft:
211 245 t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)
212 246 }
213 247 }
248
249 func TestSameJobPushAndMRShareOnePipeline(t *testing.T) {
250 if _, err := exec.LookPath("git"); err != nil {
251 t.Skip("git ontbreekt")
252 }
253 storage := t.TempDir()
254 st, err := store.Open(filepath.Join(storage, "bluegit.db"), bytes32())
255 if err != nil {
256 t.Fatal(err)
257 }
258 t.Cleanup(func() { _ = st.Close() })
259 cfg := config.Config{
260 BaseURL: "http://127.0.0.1:9", Listen: "127.0.0.1:0", StoragePath: storage,
261 Auth: config.Auth{Mode: "dev", DevUser: "tim", DevAuthID: 1, UserAgent: "test", BaseURL: "https://auth.timfalken.com"},
262 Git: config.Git{Binary: "git", DefaultBranch: "master"},
263 CI: config.CI{Enabled: true, Docker: "docker"},
264 }
265 srv := &Server{Cfg: cfg, Store: st, Auth: auth.Dev{Username: "tim", AuthID: 1}, Dev: true, Git: "git", HookBin: "/bin/true"}
266 ts := httptest.NewServer(srv.Handler())
267 t.Cleanup(ts.Close)
268 jar, _ := cookiejar.New(nil)
269 tim := &http.Client{Jar: jar}
270 res, err := tim.Get(ts.URL + "/login")
271 if err != nil {
272 t.Fatal(err)
273 }
274 page := read(t, res)
275 res, err = tim.PostForm(ts.URL+"/new/repo", url.Values{"csrf": {csrfOf(t, page)}, "name": {"samepipe"}, "visibility": {"public"}})
276 if err != nil {
277 t.Fatal(err)
278 }
279 page = read(t, res)
280 res, err = tim.PostForm(ts.URL+"/settings/tokens", url.Values{"csrf": {csrfOf(t, page)}, "name": {"laptop"}})
281 if err != nil {
282 t.Fatal(err)
283 }
284 token := between(read(t, res), `<p class="code">`, "</p>")
285 remote := strings.Replace(ts.URL, "http://", "http://tim:"+token+"@", 1) + "/tim/samepipe.git"
286 work := t.TempDir()
287 git(t, work, "init", "-b", "master")
288 git(t, work, "config", "user.email", "tim@users.git.timfalken.com")
289 git(t, work, "config", "user.name", "tim")
290 yml := `stages: [test]
291 check:
292 stage: test
293 image: alpine:3.20
294 script: ["echo check"]
295 rules:
296 - if: $CI_PIPELINE_SOURCE == "merge_request_event"
297 - if: $CI_COMMIT_BRANCH
298 `
299 writeFile(t, filepath.Join(work, "blue-ci-cd.yml"), yml)
300 git(t, work, "add", "blue-ci-cd.yml")
301 git(t, work, "commit", "-m", "ci")
302 git(t, work, "remote", "add", "origin", remote)
303 git(t, work, "push", "-u", "origin", "master")
304 git(t, work, "checkout", "-b", "feature")
305 writeFile(t, filepath.Join(work, "README.md"), "feature\n")
306 git(t, work, "add", "README.md")
307 git(t, work, "commit", "-m", "feature")
308 git(t, work, "push", "-u", "origin", "feature")
309 first := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))
310
311 ctx := context.Background()
312 repo, err := st.RepoByPath(ctx, "tim", "samepipe")
313 if err != nil {
314 t.Fatal(err)
315 }
316 if n := len(pipelinesForSHA(t, st, repo.ID, first)); n != 1 {
317 t.Fatalf("push zonder mr: %d pipelines voor %s", n, first)
318 }
319
320 res, err = tim.Get(ts.URL + "/tim/samepipe/-/merge-requests")
321 if err != nil {
322 t.Fatal(err)
323 }
324 page = read(t, res)
325 res, err = tim.PostForm(ts.URL+"/tim/samepipe/-/merge-requests", url.Values{
326 "csrf": {csrfOf(t, page)}, "title": {"voeg samen"}, "source": {"feature"}, "target": {"master"},
327 })
328 if err != nil {
329 t.Fatal(err)
330 }
331 _ = read(t, res)
332 opened := pipelinesForSHA(t, st, repo.ID, first)
333 if len(opened) != 1 || opened[0].MRID == 0 {
334 t.Fatalf("mr openen hoort de push te hergebruiken: %+v", opened)
335 }
336
337 writeFile(t, filepath.Join(work, "README.md"), "nog een\n")
338 git(t, work, "add", "README.md")
339 git(t, work, "commit", "-m", "nog een")
340 git(t, work, "push", "origin", "feature")
341 second := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))
342 shared := pipelinesForSHA(t, st, repo.ID, second)
343 if len(shared) != 1 || shared[0].MRID == 0 {
344 t.Fatalf("push met open mr: %+v", shared)
345 }
346 user, err := st.UserByUsername(ctx, "tim")
347 if err != nil {
348 t.Fatal(err)
349 }
350 dir := gitrepo.Dir(storage, "tim", "samepipe")
351 after, err := gitrepo.Refs(ctx, "git", dir)
352 if err != nil {
353 t.Fatal(err)
354 }
355 before := map[string]string{}
356 for ref, sha := range after {
357 before[ref] = sha
358 }
359 before["refs/heads/feature"] = first
360 srv.AfterPush(repo, user, before, after)
361 if n := len(pipelinesForSHA(t, st, repo.ID, second)); n != 1 {
362 t.Fatalf("zelfde push nog eens: %d", n)
363 }
364 list := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines"))
365 if !strings.Contains(list, "Merge request !1") {
366 t.Fatalf("lijst mist het mr-label:\n%s", list)
367 }
368 detail := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines/"+strconv.FormatInt(shared[0].ID, 10)))
369 if !strings.Contains(detail, "Merge request !1") {
370 t.Fatalf("detail mist het mr-label:\n%s", detail)
371 }
372
373 git(t, work, "checkout", "master")
374 writeFile(t, filepath.Join(work, "README.md"), "target\n")
375 git(t, work, "add", "README.md")
376 git(t, work, "commit", "-m", "target")
377 git(t, work, "push", "origin", "master")
378 afterTarget := pipelinesForSHA(t, st, repo.ID, second)
379 if len(afterTarget) != 2 {
380 t.Fatalf("nieuwe doelcommit hoort een mr-pipeline toe te voegen: %+v", afterTarget)
381 }
382 var newest store.Pipeline
383 for _, p := range afterTarget {
384 if p.ID > newest.ID {
385 newest = p
386 }
387 }
388 if newest.ID <= shared[0].ID || newest.Source != "merge_request_event" || newest.MRID == 0 || newest.CommitSHA != second {
389 t.Fatalf("nieuwe mr-pipeline: %+v", newest)
390 }
391 }
392
393 func pipelinesForSHA(t *testing.T, st *store.Store, repoID int64, sha string) []store.Pipeline {
394 t.Helper()
395 all, err := st.ListPipelines(context.Background(), repoID)
396 if err != nil {
397 t.Fatal(err)
398 }
399 var out []store.Pipeline
400 for _, p := range all {
401 if p.CommitSHA == sha {
402 out = append(out, p)
403 }
404 }
405 return out
406 }
@@ -8,11 +8,13 @@ import (
8 "net/url"
8 "net/url"
9 "os/exec"
9 "os/exec"
10 "path/filepath"
10 "path/filepath"
11 "strconv"
11 "strings"
12 "strings"
12 "testing"
13 "testing"
13
14
14 "gitlab.com/Timfa/bluegit/internal/auth"
15 "gitlab.com/Timfa/bluegit/internal/auth"
15 "gitlab.com/Timfa/bluegit/internal/config"
16 "gitlab.com/Timfa/bluegit/internal/config"
17 "gitlab.com/Timfa/bluegit/internal/gitrepo"
16 "gitlab.com/Timfa/bluegit/internal/store"
18 "gitlab.com/Timfa/bluegit/internal/store"
17)
19)
18
20
@@ -138,6 +140,38 @@ soft:
138 if mrPipe.ID == 0 || mrPipe.MRID == 0 {
140 if mrPipe.ID == 0 || mrPipe.MRID == 0 {
139 t.Fatalf("mr-pipeline: %+v", mrPipe)
141 t.Fatalf("mr-pipeline: %+v", mrPipe)
140 }
142 }
143 nBefore := len(pipes)
144 user, err := st.UserByUsername(ctx, "tim")
145 if err != nil {
146 t.Fatal(err)
147 }
148 dir := gitrepo.Dir(storage, "tim", "mrpipes")
149 after, err := gitrepo.Refs(ctx, "git", dir)
150 if err != nil {
151 t.Fatal(err)
152 }
153 before := map[string]string{}
154 for ref, sha := range after {
155 before[ref] = sha
156 }
157 before["refs/heads/feature"] = strings.Repeat("a", 40)
158 srv.AfterPush(repo, user, before, after)
159 srv.rerunMergeRequestsOnTarget(ctx, repo, "master", "tim")
160 pipes, err = st.ListPipelines(ctx, repo.ID)
161 if err != nil {
162 t.Fatal(err)
163 }
164 if len(pipes) != nBefore {
165 t.Fatalf("dezelfde push nog eens: %d pipelines, was %d", len(pipes), nBefore)
166 }
167 list := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines"))
168 if !strings.Contains(list, "Merge request !1") {
169 t.Fatalf("lijst mist het mr-label:\n%s", list)
170 }
171 detail := read(t, mustGet(t, tim, ts.URL+"/tim/mrpipes/-/pipelines/"+strconv.FormatInt(mrPipe.ID, 10)))
172 if !strings.Contains(detail, "Merge request !1") {
173 t.Fatalf("detail mist het mr-label:\n%s", detail)
174 }
141 res, err = tim.PostForm(ts.URL+"/tim/mrpipes/-/merge-requests/1/merge", url.Values{"csrf": {csrfOf(t, page)}})
175 res, err = tim.PostForm(ts.URL+"/tim/mrpipes/-/merge-requests/1/merge", url.Values{"csrf": {csrfOf(t, page)}})
142 if err != nil {
176 if err != nil {
143 t.Fatal(err)
177 t.Fatal(err)
@@ -211,3 +245,162 @@ soft:
211 t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)
245 t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)
212 }
246 }
213}
247}
248
249func TestSameJobPushAndMRShareOnePipeline(t *testing.T) {
250 if _, err := exec.LookPath("git"); err != nil {
251 t.Skip("git ontbreekt")
252 }
253 storage := t.TempDir()
254 st, err := store.Open(filepath.Join(storage, "bluegit.db"), bytes32())
255 if err != nil {
256 t.Fatal(err)
257 }
258 t.Cleanup(func() { _ = st.Close() })
259 cfg := config.Config{
260 BaseURL: "http://127.0.0.1:9", Listen: "127.0.0.1:0", StoragePath: storage,
261 Auth: config.Auth{Mode: "dev", DevUser: "tim", DevAuthID: 1, UserAgent: "test", BaseURL: "https://auth.timfalken.com"},
262 Git: config.Git{Binary: "git", DefaultBranch: "master"},
263 CI: config.CI{Enabled: true, Docker: "docker"},
264 }
265 srv := &Server{Cfg: cfg, Store: st, Auth: auth.Dev{Username: "tim", AuthID: 1}, Dev: true, Git: "git", HookBin: "/bin/true"}
266 ts := httptest.NewServer(srv.Handler())
267 t.Cleanup(ts.Close)
268 jar, _ := cookiejar.New(nil)
269 tim := &http.Client{Jar: jar}
270 res, err := tim.Get(ts.URL + "/login")
271 if err != nil {
272 t.Fatal(err)
273 }
274 page := read(t, res)
275 res, err = tim.PostForm(ts.URL+"/new/repo", url.Values{"csrf": {csrfOf(t, page)}, "name": {"samepipe"}, "visibility": {"public"}})
276 if err != nil {
277 t.Fatal(err)
278 }
279 page = read(t, res)
280 res, err = tim.PostForm(ts.URL+"/settings/tokens", url.Values{"csrf": {csrfOf(t, page)}, "name": {"laptop"}})
281 if err != nil {
282 t.Fatal(err)
283 }
284 token := between(read(t, res), `<p class="code">`, "</p>")
285 remote := strings.Replace(ts.URL, "http://", "http://tim:"+token+"@", 1) + "/tim/samepipe.git"
286 work := t.TempDir()
287 git(t, work, "init", "-b", "master")
288 git(t, work, "config", "user.email", "tim@users.git.timfalken.com")
289 git(t, work, "config", "user.name", "tim")
290 yml := `stages: [test]
291check:
292 stage: test
293 image: alpine:3.20
294 script: ["echo check"]
295 rules:
296 - if: $CI_PIPELINE_SOURCE == "merge_request_event"
297 - if: $CI_COMMIT_BRANCH
298`
299 writeFile(t, filepath.Join(work, "blue-ci-cd.yml"), yml)
300 git(t, work, "add", "blue-ci-cd.yml")
301 git(t, work, "commit", "-m", "ci")
302 git(t, work, "remote", "add", "origin", remote)
303 git(t, work, "push", "-u", "origin", "master")
304 git(t, work, "checkout", "-b", "feature")
305 writeFile(t, filepath.Join(work, "README.md"), "feature\n")
306 git(t, work, "add", "README.md")
307 git(t, work, "commit", "-m", "feature")
308 git(t, work, "push", "-u", "origin", "feature")
309 first := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))
310
311 ctx := context.Background()
312 repo, err := st.RepoByPath(ctx, "tim", "samepipe")
313 if err != nil {
314 t.Fatal(err)
315 }
316 if n := len(pipelinesForSHA(t, st, repo.ID, first)); n != 1 {
317 t.Fatalf("push zonder mr: %d pipelines voor %s", n, first)
318 }
319
320 res, err = tim.Get(ts.URL + "/tim/samepipe/-/merge-requests")
321 if err != nil {
322 t.Fatal(err)
323 }
324 page = read(t, res)
325 res, err = tim.PostForm(ts.URL+"/tim/samepipe/-/merge-requests", url.Values{
326 "csrf": {csrfOf(t, page)}, "title": {"voeg samen"}, "source": {"feature"}, "target": {"master"},
327 })
328 if err != nil {
329 t.Fatal(err)
330 }
331 _ = read(t, res)
332 opened := pipelinesForSHA(t, st, repo.ID, first)
333 if len(opened) != 1 || opened[0].MRID == 0 {
334 t.Fatalf("mr openen hoort de push te hergebruiken: %+v", opened)
335 }
336
337 writeFile(t, filepath.Join(work, "README.md"), "nog een\n")
338 git(t, work, "add", "README.md")
339 git(t, work, "commit", "-m", "nog een")
340 git(t, work, "push", "origin", "feature")
341 second := strings.TrimSpace(gitOutOK(t, work, "rev-parse", "HEAD"))
342 shared := pipelinesForSHA(t, st, repo.ID, second)
343 if len(shared) != 1 || shared[0].MRID == 0 {
344 t.Fatalf("push met open mr: %+v", shared)
345 }
346 user, err := st.UserByUsername(ctx, "tim")
347 if err != nil {
348 t.Fatal(err)
349 }
350 dir := gitrepo.Dir(storage, "tim", "samepipe")
351 after, err := gitrepo.Refs(ctx, "git", dir)
352 if err != nil {
353 t.Fatal(err)
354 }
355 before := map[string]string{}
356 for ref, sha := range after {
357 before[ref] = sha
358 }
359 before["refs/heads/feature"] = first
360 srv.AfterPush(repo, user, before, after)
361 if n := len(pipelinesForSHA(t, st, repo.ID, second)); n != 1 {
362 t.Fatalf("zelfde push nog eens: %d", n)
363 }
364 list := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines"))
365 if !strings.Contains(list, "Merge request !1") {
366 t.Fatalf("lijst mist het mr-label:\n%s", list)
367 }
368 detail := read(t, mustGet(t, tim, ts.URL+"/tim/samepipe/-/pipelines/"+strconv.FormatInt(shared[0].ID, 10)))
369 if !strings.Contains(detail, "Merge request !1") {
370 t.Fatalf("detail mist het mr-label:\n%s", detail)
371 }
372
373 git(t, work, "checkout", "master")
374 writeFile(t, filepath.Join(work, "README.md"), "target\n")
375 git(t, work, "add", "README.md")
376 git(t, work, "commit", "-m", "target")
377 git(t, work, "push", "origin", "master")
378 afterTarget := pipelinesForSHA(t, st, repo.ID, second)
379 if len(afterTarget) != 2 {
380 t.Fatalf("nieuwe doelcommit hoort een mr-pipeline toe te voegen: %+v", afterTarget)
381 }
382 var newest store.Pipeline
383 for _, p := range afterTarget {
384 if p.ID > newest.ID {
385 newest = p
386 }
387 }
388 if newest.ID <= shared[0].ID || newest.Source != "merge_request_event" || newest.MRID == 0 || newest.CommitSHA != second {
389 t.Fatalf("nieuwe mr-pipeline: %+v", newest)
390 }
391}
392
393func pipelinesForSHA(t *testing.T, st *store.Store, repoID int64, sha string) []store.Pipeline {
394 t.Helper()
395 all, err := st.ListPipelines(context.Background(), repoID)
396 if err != nil {
397 t.Fatal(err)
398 }
399 var out []store.Pipeline
400 for _, p := range all {
401 if p.CommitSHA == sha {
402 out = append(out, p)
403 }
404 }
405 return out
406}
internal/server/pipeline_page.go +54 −16
@@ -17,20 +17,23 @@ import (
17 17
18 18 // PipeRow is één rij in het pipeline-overzicht.
19 19 type PipeRow struct {
20 ID int64
21 Status string
22 Label string
23 Duration string
24 When string
25 Title string
26 SHA string
27 Short string
28 Ref string
29 RefKind string
30 Actor string
31 Avatar string
32 Initial string
33 Stages []StageView
20 ID int64
21 Status string
22 Label string
23 Duration string
24 When string
25 Title string
26 SHA string
27 Short string
28 Ref string
29 RefKind string
30 Source string
31 SourceLabel string
32 MRNumber int64
33 Actor string
34 Avatar string
35 Initial string
36 Stages []StageView
34 37 }
35 38
36 39 // StageView is één bolletje in de stages-kolom.
@@ -154,15 +157,50 @@ func (s *Server) pipeRow(r *http.Request, lang i18n.Lang, repo store.Repo, p sto
154 157 if actor != "" {
155 158 initial = strings.ToUpper(actor[:1])
156 159 }
160 sourceLabel, mrNumber := s.pipelineSourceLabel(r, repo, p)
157 161 return PipeRow{
158 162 ID: p.ID, Status: status, Label: s.tr(r, "pipe."+status),
159 163 Duration: pipelineClock(p), When: i18n.Ago(lang, p.CreatedAt.Unix()),
160 164 Title: p.CommitTitle, SHA: p.CommitSHA, Short: shortSHA(p.CommitSHA),
161 Ref: name, RefKind: kind, Actor: actor, Avatar: avatar, Initial: initial,
165 Ref: name, RefKind: kind, Source: p.Source, SourceLabel: sourceLabel, MRNumber: mrNumber,
166 Actor: actor, Avatar: avatar, Initial: initial,
162 167 Stages: stageViews(jobs, ranks, func(key string, args ...any) string { return s.tr(r, key, args...) }),
163 168 }
164 169 }
165 170
171 // pipelineSourceLabel benoemt een pipeline die bij een merge request hoort,
172 // ook wanneer de rij als push is gestart en daarna gekoppeld is.
173 // Een gewone push zonder merge request krijgt geen extra label.
174 func (s *Server) pipelineSourceLabel(r *http.Request, repo store.Repo, p store.Pipeline) (string, int64) {
175 if p.MRID != 0 {
176 var number int64
177 if mr, err := s.Store.MergeRequest(r.Context(), repo.ID, p.MRID); err == nil {
178 number = mr.Number
179 }
180 return s.tr(r, "pipe.src.merge_request_event"), number
181 }
182 if p.Source == "" || p.Source == "push" {
183 return "", 0
184 }
185 key := "pipe.src." + p.Source
186 label := s.tr(r, key)
187 if label == key {
188 label = p.Source
189 }
190 return label, 0
191 }
192
193 func (s *Server) pipelineSourceText(r *http.Request, repo store.Repo, p store.Pipeline) string {
194 label, number := s.pipelineSourceLabel(r, repo, p)
195 if label == "" {
196 return ""
197 }
198 if number == 0 {
199 return label
200 }
201 return label + " !" + strconv.FormatInt(number, 10)
202 }
203
166 204 func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.User) (string, string) {
167 205 name = strings.TrimSpace(name)
168 206 if name == "" {
@@ -183,7 +221,7 @@ func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.Us
183 221 }
184 222
185 223 func pipeMatches(row PipeRow, q string) bool {
186 bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, strconv.FormatInt(row.ID, 10), row.Status}
224 bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, row.SourceLabel, strconv.FormatInt(row.ID, 10), row.Status}
187 225 for _, st := range row.Stages {
188 226 bits = append(bits, st.Name)
189 227 for _, j := range st.Jobs {
@@ -17,20 +17,23 @@ import (
17
17
18// PipeRow is één rij in het pipeline-overzicht.
18// PipeRow is één rij in het pipeline-overzicht.
19type PipeRow struct {
19type PipeRow struct {
20 ID int64
20 ID int64
21 Status string
21 Status string
22 Label string
22 Label string
23 Duration string
23 Duration string
24 When string
24 When string
25 Title string
25 Title string
26 SHA string
26 SHA string
27 Short string
27 Short string
28 Ref string
28 Ref string
29 RefKind string
29 RefKind string
30 Actor string
30 Source string
31 Avatar string
31 SourceLabel string
32 Initial string
32 MRNumber int64
33 Stages []StageView
33 Actor string
34 Avatar string
35 Initial string
36 Stages []StageView
34}
37}
35
38
36// StageView is één bolletje in de stages-kolom.
39// StageView is één bolletje in de stages-kolom.
@@ -154,15 +157,50 @@ func (s *Server) pipeRow(r *http.Request, lang i18n.Lang, repo store.Repo, p sto
154 if actor != "" {
157 if actor != "" {
155 initial = strings.ToUpper(actor[:1])
158 initial = strings.ToUpper(actor[:1])
156 }
159 }
160 sourceLabel, mrNumber := s.pipelineSourceLabel(r, repo, p)
157 return PipeRow{
161 return PipeRow{
158 ID: p.ID, Status: status, Label: s.tr(r, "pipe."+status),
162 ID: p.ID, Status: status, Label: s.tr(r, "pipe."+status),
159 Duration: pipelineClock(p), When: i18n.Ago(lang, p.CreatedAt.Unix()),
163 Duration: pipelineClock(p), When: i18n.Ago(lang, p.CreatedAt.Unix()),
160 Title: p.CommitTitle, SHA: p.CommitSHA, Short: shortSHA(p.CommitSHA),
164 Title: p.CommitTitle, SHA: p.CommitSHA, Short: shortSHA(p.CommitSHA),
161 Ref: name, RefKind: kind, Actor: actor, Avatar: avatar, Initial: initial,
165 Ref: name, RefKind: kind, Source: p.Source, SourceLabel: sourceLabel, MRNumber: mrNumber,
166 Actor: actor, Avatar: avatar, Initial: initial,
162 Stages: stageViews(jobs, ranks, func(key string, args ...any) string { return s.tr(r, key, args...) }),
167 Stages: stageViews(jobs, ranks, func(key string, args ...any) string { return s.tr(r, key, args...) }),
163 }
168 }
164}
169}
165
170
171// pipelineSourceLabel benoemt een pipeline die bij een merge request hoort,
172// ook wanneer de rij als push is gestart en daarna gekoppeld is.
173// Een gewone push zonder merge request krijgt geen extra label.
174func (s *Server) pipelineSourceLabel(r *http.Request, repo store.Repo, p store.Pipeline) (string, int64) {
175 if p.MRID != 0 {
176 var number int64
177 if mr, err := s.Store.MergeRequest(r.Context(), repo.ID, p.MRID); err == nil {
178 number = mr.Number
179 }
180 return s.tr(r, "pipe.src.merge_request_event"), number
181 }
182 if p.Source == "" || p.Source == "push" {
183 return "", 0
184 }
185 key := "pipe.src." + p.Source
186 label := s.tr(r, key)
187 if label == key {
188 label = p.Source
189 }
190 return label, 0
191}
192
193func (s *Server) pipelineSourceText(r *http.Request, repo store.Repo, p store.Pipeline) string {
194 label, number := s.pipelineSourceLabel(r, repo, p)
195 if label == "" {
196 return ""
197 }
198 if number == 0 {
199 return label
200 }
201 return label + " !" + strconv.FormatInt(number, 10)
202}
203
166func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.User) (string, string) {
204func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.User) (string, string) {
167 name = strings.TrimSpace(name)
205 name = strings.TrimSpace(name)
168 if name == "" {
206 if name == "" {
@@ -183,7 +221,7 @@ func (s *Server) actorOf(r *http.Request, name string, cache map[string]store.Us
183}
221}
184
222
185func pipeMatches(row PipeRow, q string) bool {
223func pipeMatches(row PipeRow, q string) bool {
186 bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, strconv.FormatInt(row.ID, 10), row.Status}
224 bits := []string{row.Label, row.Title, row.SHA, row.Short, row.Ref, row.Actor, row.SourceLabel, strconv.FormatInt(row.ID, 10), row.Status}
187 for _, st := range row.Stages {
225 for _, st := range row.Stages {
188 bits = append(bits, st.Name)
226 bits = append(bits, st.Name)
189 for _, j := range st.Jobs {
227 for _, j := range st.Jobs {
internal/server/repo_handlers.go +1 −0
@@ -367,6 +367,7 @@ func (s *Server) pipeline(w http.ResponseWriter, r *http.Request) {
367 367 tr := func(key string, args ...any) string { return s.tr(r, key, args...) }
368 368 v.PipeStatus = displayStatus(p.Status, jobs)
369 369 v.PipeLabel = tr("pipe." + v.PipeStatus)
370 v.PipeSource = s.pipelineSourceText(r, repo, p)
370 371 v.PipeJobs = linksFor(jobs, ranks, tr)
371 372 v.PipeStages = stageViews(jobs, ranks, tr)
372 373 v.Live = jobsLive(p.Status, jobs)
@@ -367,6 +367,7 @@ func (s *Server) pipeline(w http.ResponseWriter, r *http.Request) {
367 tr := func(key string, args ...any) string { return s.tr(r, key, args...) }
367 tr := func(key string, args ...any) string { return s.tr(r, key, args...) }
368 v.PipeStatus = displayStatus(p.Status, jobs)
368 v.PipeStatus = displayStatus(p.Status, jobs)
369 v.PipeLabel = tr("pipe." + v.PipeStatus)
369 v.PipeLabel = tr("pipe." + v.PipeStatus)
370 v.PipeSource = s.pipelineSourceText(r, repo, p)
370 v.PipeJobs = linksFor(jobs, ranks, tr)
371 v.PipeJobs = linksFor(jobs, ranks, tr)
371 v.PipeStages = stageViews(jobs, ranks, tr)
372 v.PipeStages = stageViews(jobs, ranks, tr)
372 v.Live = jobsLive(p.Status, jobs)
373 v.Live = jobsLive(p.Status, jobs)
internal/server/server.go +1 −0
@@ -540,6 +540,7 @@ type View struct {
540 540 Pipes []store.Pipeline
541 541 PipeRows []PipeRow
542 542 Pipe *store.Pipeline
543 PipeSource string
543 544 PipeStatus string
544 545 PipeLabel string
545 546 PipeJobs []JobLink
@@ -540,6 +540,7 @@ type View struct {
540 Pipes []store.Pipeline
540 Pipes []store.Pipeline
541 PipeRows []PipeRow
541 PipeRows []PipeRow
542 Pipe *store.Pipeline
542 Pipe *store.Pipeline
543 PipeSource string
543 PipeStatus string
544 PipeStatus string
544 PipeLabel string
545 PipeLabel string
545 PipeJobs []JobLink
546 PipeJobs []JobLink
internal/server/templates/collab.html +2 −1
@@ -253,7 +253,7 @@
253 253 {{template "layout-start" .}}
254 254 <header class="pagehead"><h1>{{t "pipe.one"}} #{{.Pipe.ID}}</h1></header>
255 255 {{template "pipenav" .}}
256 <p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
256 <p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span>{{if .PipeSource}} <span class="pill">{{.PipeSource}}</span>{{end}} {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
257 257 <p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>
258 258 <ul class="clean"{{if .Live}} data-live="{{.LiveURL}}" data-live-on="1"{{end}}>
259 259 {{range .PipeJobs}}<li data-job="{{.ID}}"><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="status {{.Status}}" data-job-status><span class="mark"></span><span data-job-label>{{.Label}}</span></span>{{if .Queue}} <span class="queue" data-queue>{{.Queue}}</span>{{end}} <span class="muted">{{.Stage}}</span></li>{{end}}
@@ -277,6 +277,7 @@
277 277 <div class="chips">
278 278 {{if .Ref}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commits?rev={{.Ref}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true">{{if eq .RefKind "tag"}}<path d="M2.2 2.2h5.2l6.2 6.2-5.2 5.2-6.2-6.2V2.2z" fill="none" stroke="currentColor" stroke-width="1.3"/><circle cx="5.2" cy="5.2" r="1" fill="currentColor"/>{{else}}<circle cx="4" cy="3.2" r="1.5" fill="currentColor"/><circle cx="4" cy="12.8" r="1.5" fill="currentColor"/><circle cx="12" cy="6.4" r="1.5" fill="currentColor"/><path d="M4 4.7v6.6M4 6.4c2.4 0 4.2.2 6 .2 1.2 0 2 1 2 1.8" fill="none" stroke="currentColor" stroke-width="1.3"/>{{end}}</svg>{{.Ref}}</a>{{end}}
279 279 {{if .Short}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commit/{{.SHA}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true"><circle cx="8" cy="8" r="2.1" fill="currentColor"/><path d="M8 1.8v3.4M8 10.8v3.4" stroke="currentColor" stroke-width="1.4"/></svg>{{.Short}}</a>{{end}}
280 {{if .SourceLabel}}<span class="chip">{{.SourceLabel}}{{if .MRNumber}} !{{.MRNumber}}{{end}}</span>{{end}}
280 281 {{if .Actor}}{{if .Avatar}}<img class="avatar sm" src="{{.Avatar}}" alt="">{{else}}<span class="avatar sm">{{.Initial}}</span>{{end}}{{end}}
281 282 </div>
282 283 </td>
@@ -253,7 +253,7 @@
253{{template "layout-start" .}}
253{{template "layout-start" .}}
254<header class="pagehead"><h1>{{t "pipe.one"}} #{{.Pipe.ID}}</h1></header>
254<header class="pagehead"><h1>{{t "pipe.one"}} #{{.Pipe.ID}}</h1></header>
255{{template "pipenav" .}}
255{{template "pipenav" .}}
256<p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
256<p><span class="status {{.PipeStatus}}" data-pipe-status><span class="mark"></span><span data-pipe-label>{{.PipeLabel}}</span></span>{{if .PipeSource}} <span class="pill">{{.PipeSource}}</span>{{end}} {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
257<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>
257<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>
258<ul class="clean"{{if .Live}} data-live="{{.LiveURL}}" data-live-on="1"{{end}}>
258<ul class="clean"{{if .Live}} data-live="{{.LiveURL}}" data-live-on="1"{{end}}>
259{{range .PipeJobs}}<li data-job="{{.ID}}"><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="status {{.Status}}" data-job-status><span class="mark"></span><span data-job-label>{{.Label}}</span></span>{{if .Queue}} <span class="queue" data-queue>{{.Queue}}</span>{{end}} <span class="muted">{{.Stage}}</span></li>{{end}}
259{{range .PipeJobs}}<li data-job="{{.ID}}"><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="status {{.Status}}" data-job-status><span class="mark"></span><span data-job-label>{{.Label}}</span></span>{{if .Queue}} <span class="queue" data-queue>{{.Queue}}</span>{{end}} <span class="muted">{{.Stage}}</span></li>{{end}}
@@ -277,6 +277,7 @@
277 <div class="chips">
277 <div class="chips">
278 {{if .Ref}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commits?rev={{.Ref}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true">{{if eq .RefKind "tag"}}<path d="M2.2 2.2h5.2l6.2 6.2-5.2 5.2-6.2-6.2V2.2z" fill="none" stroke="currentColor" stroke-width="1.3"/><circle cx="5.2" cy="5.2" r="1" fill="currentColor"/>{{else}}<circle cx="4" cy="3.2" r="1.5" fill="currentColor"/><circle cx="4" cy="12.8" r="1.5" fill="currentColor"/><circle cx="12" cy="6.4" r="1.5" fill="currentColor"/><path d="M4 4.7v6.6M4 6.4c2.4 0 4.2.2 6 .2 1.2 0 2 1 2 1.8" fill="none" stroke="currentColor" stroke-width="1.3"/>{{end}}</svg>{{.Ref}}</a>{{end}}
278 {{if .Ref}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commits?rev={{.Ref}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true">{{if eq .RefKind "tag"}}<path d="M2.2 2.2h5.2l6.2 6.2-5.2 5.2-6.2-6.2V2.2z" fill="none" stroke="currentColor" stroke-width="1.3"/><circle cx="5.2" cy="5.2" r="1" fill="currentColor"/>{{else}}<circle cx="4" cy="3.2" r="1.5" fill="currentColor"/><circle cx="4" cy="12.8" r="1.5" fill="currentColor"/><circle cx="12" cy="6.4" r="1.5" fill="currentColor"/><path d="M4 4.7v6.6M4 6.4c2.4 0 4.2.2 6 .2 1.2 0 2 1 2 1.8" fill="none" stroke="currentColor" stroke-width="1.3"/>{{end}}</svg>{{.Ref}}</a>{{end}}
279 {{if .Short}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commit/{{.SHA}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true"><circle cx="8" cy="8" r="2.1" fill="currentColor"/><path d="M8 1.8v3.4M8 10.8v3.4" stroke="currentColor" stroke-width="1.4"/></svg>{{.Short}}</a>{{end}}
279 {{if .Short}}<a class="chip" href="/{{$.Repo.FullPath}}/-/commit/{{.SHA}}"><svg class="ico" viewBox="0 0 16 16" aria-hidden="true"><circle cx="8" cy="8" r="2.1" fill="currentColor"/><path d="M8 1.8v3.4M8 10.8v3.4" stroke="currentColor" stroke-width="1.4"/></svg>{{.Short}}</a>{{end}}
280 {{if .SourceLabel}}<span class="chip">{{.SourceLabel}}{{if .MRNumber}} !{{.MRNumber}}{{end}}</span>{{end}}
280 {{if .Actor}}{{if .Avatar}}<img class="avatar sm" src="{{.Avatar}}" alt="">{{else}}<span class="avatar sm">{{.Initial}}</span>{{end}}{{end}}
281 {{if .Actor}}{{if .Avatar}}<img class="avatar sm" src="{{.Avatar}}" alt="">{{else}}<span class="avatar sm">{{.Initial}}</span>{{end}}{{end}}
281 </div>
282 </div>
282 </td>
283 </td>
internal/store/flow.go +288 −14
@@ -399,6 +399,10 @@ type Pipeline struct {
399 399 StartedAt time.Time
400 400 FinishedAt time.Time
401 401 MRID int64
402 // TargetSHA is de doelcommit van een merge-request-pipeline.
403 // Leeg zolang de rij nog niet aan een merge request hangt.
404 // Een nieuwe doelcommit start een nieuwe pipeline.
405 TargetSHA string
402 406 }
403 407
404 408 // Job is één container in een pipeline.
@@ -442,30 +446,71 @@ type Job struct {
442 446 }
443 447
444 448 // RecordConfigFailure bewaart een pipeline die niet kon starten.
445 func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, message string) (int64, error) {
449 // Dezelfde repo, ref, sha, bron, merge request en doel-sha levert de
450 // bestaande pipeline terug. Een kapotte push die al bestaat wordt aan
451 // de merge request gekoppeld in plaats van een tweede mislukte rij.
452 func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, source, message string, mrID int64, targetSHA string) (int64, bool, error) {
453 if source == "" {
454 source = "push"
455 }
446 456 tx, err := s.db.BeginTx(ctx, nil)
447 457 if err != nil {
448 return 0, err
458 return 0, false, err
449 459 }
450 460 defer func() { _ = tx.Rollback() }()
461 if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {
462 return 0, false, err
463 } else if ok {
464 return finishExisting(ctx, tx, id)
465 }
466 if source == "push" && mrID == 0 {
467 if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {
468 return 0, false, err
469 } else if ok {
470 return finishExisting(ctx, tx, id)
471 }
472 }
473 if mrID != 0 {
474 if id, ok, err := configFailurePush(ctx, tx, repoID, sha, ref, mrID, targetSHA); err != nil {
475 return 0, false, err
476 } else if ok {
477 if err := bindPipelineMR(ctx, tx, id, mrID, targetSHA); err != nil {
478 return 0, false, err
479 }
480 return finishExisting(ctx, tx, id)
481 }
482 }
451 483 ts := now()
452 res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, created_at, finished_at) VALUES (?, ?, ?, 'failed', ?, ?)`, repoID, sha, ref, ts, ts)
484 res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, created_at, finished_at, mr_id, target_sha)
485 SELECT ?, ?, ?, 'failed', ?, ?, ?, ?, ?
486 WHERE NOT EXISTS (
487 SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?
488 )`, repoID, sha, ref, source, ts, ts, mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA)
453 489 if err != nil {
454 return 0, err
490 return 0, false, err
491 }
492 if n, _ := res.RowsAffected(); n == 0 {
493 id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
494 if err != nil {
495 return 0, false, err
496 }
497 return finishExisting(ctx, tx, id)
455 498 }
456 499 id, _ := res.LastInsertId()
457 500 _, err = tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, finished_at)
458 501 VALUES (?, ?, 'blue-ci-cd', 'test', 0, 'failed', 'alpine:3.20', 'exit 1', '[]', 0, '{}', '[]', 0, 0, ?, ?)`, id, repoID, message, ts)
459 502 if err != nil {
460 return 0, err
503 return 0, false, err
461 504 }
462 505 if err := tx.Commit(); err != nil {
463 return 0, err
506 return 0, false, err
464 507 }
465 return id, nil
508 return id, true, nil
466 509 }
467 510
468 511 // CreatePipeline maakt een pipeline met jobs in de wachtrij.
512 // Elke aanroep voegt een rij toe. Starten via een push of merge request
513 // gaat via OpenPipeline, zodat een dubbele hook dezelfde run teruggeeft.
469 514 func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title string, jobs []NewJob) (int64, error) {
470 515 tx, err := s.db.BeginTx(ctx, nil)
471 516 if err != nil {
@@ -477,6 +522,238 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
477 522 return 0, err
478 523 }
479 524 id, _ := res.LastInsertId()
525 if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {
526 return 0, err
527 }
528 if err := tx.Commit(); err != nil {
529 return 0, err
530 }
531 return id, nil
532 }
533
534 // OpenPipeline maakt een pipeline, of geeft de bestaande terug.
535 // De sleutel is repo, ref, sha, bron, merge request en doel-sha.
536 // Een push voor dezelfde sha blijft één rij, ook nadat een merge request
537 // hem heeft gekoppeld. Kiezen push en merge request dezelfde jobs, dan
538 // koppelt deze aanroep de push-rij en maakt hij geen tweede pipeline.
539 // Een andere jobset of een nieuwe doel-sha start wel een eigen rij.
540 // created is false wanneer een eerdere trigger de run al had.
541 func (s *Store) OpenPipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title, targetSHA string, mrID int64, jobs []NewJob) (int64, bool, error) {
542 if source == "" {
543 source = "push"
544 }
545 tx, err := s.db.BeginTx(ctx, nil)
546 if err != nil {
547 return 0, false, err
548 }
549 defer func() { _ = tx.Rollback() }()
550 if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {
551 return 0, false, err
552 } else if ok {
553 return finishExisting(ctx, tx, id)
554 }
555 if source == "push" && mrID == 0 {
556 if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {
557 return 0, false, err
558 } else if ok {
559 return finishExisting(ctx, tx, id)
560 }
561 }
562 if mrID != 0 {
563 if id, ok, err := claimMatchingPush(ctx, tx, repoID, sha, ref, mrID, targetSHA, jobs); err != nil {
564 return 0, false, err
565 } else if ok {
566 return finishExisting(ctx, tx, id)
567 }
568 }
569 notExists := `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?`
570 args := []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA}
571 if source == "push" && mrID == 0 {
572 notExists = `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'`
573 args = []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref}
574 }
575 res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, actor, commit_title, created_at, mr_id, target_sha)
576 SELECT ?, ?, ?, 'queued', ?, ?, ?, ?, ?, ?
577 WHERE NOT EXISTS (`+notExists+`)`, args...)
578 if err != nil {
579 return 0, false, err
580 }
581 if n, _ := res.RowsAffected(); n == 0 {
582 id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
583 if err != nil {
584 return 0, false, err
585 }
586 if !ok && source == "push" && mrID == 0 {
587 id, ok, err = newestPush(ctx, tx, repoID, sha, ref)
588 if err != nil {
589 return 0, false, err
590 }
591 }
592 if !ok {
593 return 0, false, errors.New("pipeline ontbreekt na een lege insert")
594 }
595 return finishExisting(ctx, tx, id)
596 }
597 id, _ := res.LastInsertId()
598 if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {
599 return 0, false, err
600 }
601 if err := tx.Commit(); err != nil {
602 return 0, false, err
603 }
604 return id, true, nil
605 }
606
607 func finishExisting(ctx context.Context, tx *sql.Tx, id int64) (int64, bool, error) {
608 if err := tx.Commit(); err != nil {
609 return 0, false, err
610 }
611 return id, false, nil
612 }
613
614 func existingPipeline(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, bool, error) {
615 id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
616 if errors.Is(err, sql.ErrNoRows) {
617 return 0, false, nil
618 }
619 if err != nil {
620 return 0, false, err
621 }
622 return id, true, nil
623 }
624
625 func pipelineKeyID(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, error) {
626 var id int64
627 err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ? ORDER BY id LIMIT 1`,
628 repoID, sha, ref, source, mrID, targetSHA).Scan(&id)
629 return id, err
630 }
631
632 func newestPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string) (int64, bool, error) {
633 var id int64
634 err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push' ORDER BY id DESC LIMIT 1`,
635 repoID, sha, ref).Scan(&id)
636 if errors.Is(err, sql.ErrNoRows) {
637 return 0, false, nil
638 }
639 if err != nil {
640 return 0, false, err
641 }
642 return id, true, nil
643 }
644
645 // configFailurePush zoekt een mislukte push die alleen de kapotte yaml bewaart.
646 func configFailurePush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string) (int64, bool, error) {
647 var id int64
648 err := tx.QueryRowContext(ctx, `SELECT p.id FROM pipelines p
649 WHERE p.repo_id = ? AND p.commit_sha = ? AND p.ref = ? AND p.source = 'push' AND p.status = 'failed'
650 AND (p.mr_id = 0 OR p.mr_id = ?) AND (p.target_sha = '' OR p.target_sha = ?)
651 AND EXISTS (SELECT 1 FROM jobs j WHERE j.pipeline_id = p.id AND j.name = 'blue-ci-cd')
652 ORDER BY p.id DESC LIMIT 1`, repoID, sha, ref, mrID, targetSHA).Scan(&id)
653 if errors.Is(err, sql.ErrNoRows) {
654 return 0, false, nil
655 }
656 if err != nil {
657 return 0, false, err
658 }
659 return id, true, nil
660 }
661
662 type pushCandidate struct {
663 id int64
664 mrID int64
665 target string
666 }
667
668 // claimMatchingPush koppelt een push-pipeline met dezelfde jobs aan de merge request.
669 func claimMatchingPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string, jobs []NewJob) (int64, bool, error) {
670 rows, err := tx.QueryContext(ctx, `SELECT id, mr_id, target_sha FROM pipelines
671 WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'
672 AND (mr_id = 0 OR mr_id = ?) AND (target_sha = '' OR target_sha = ?)
673 ORDER BY id DESC`, repoID, sha, ref, mrID, targetSHA)
674 if err != nil {
675 return 0, false, err
676 }
677 var cands []pushCandidate
678 for rows.Next() {
679 var c pushCandidate
680 if err := rows.Scan(&c.id, &c.mrID, &c.target); err != nil {
681 rows.Close()
682 return 0, false, err
683 }
684 cands = append(cands, c)
685 }
686 if err := rows.Err(); err != nil {
687 rows.Close()
688 return 0, false, err
689 }
690 rows.Close()
691 for _, c := range cands {
692 if c.mrID == mrID && c.target == targetSHA {
693 return c.id, true, nil
694 }
695 got, err := loadJobSigs(ctx, tx, c.id)
696 if err != nil {
697 return 0, false, err
698 }
699 if !sameJobSigs(got, jobs) {
700 continue
701 }
702 if err := bindPipelineMR(ctx, tx, c.id, mrID, targetSHA); err != nil {
703 return 0, false, err
704 }
705 return c.id, true, nil
706 }
707 return 0, false, nil
708 }
709
710 func bindPipelineMR(ctx context.Context, tx *sql.Tx, id, mrID int64, targetSHA string) error {
711 _, err := tx.ExecContext(ctx, `UPDATE pipelines SET mr_id = ?, target_sha = ? WHERE id = ?`, mrID, targetSHA, id)
712 return err
713 }
714
715 type jobSig struct {
716 name, stage, image, script, when string
717 allow bool
718 }
719
720 func loadJobSigs(ctx context.Context, tx *sql.Tx, pipelineID int64) ([]jobSig, error) {
721 rows, err := tx.QueryContext(ctx, `SELECT name, stage, image, script, when_mode, allow_failure FROM jobs WHERE pipeline_id = ? ORDER BY position, id`, pipelineID)
722 if err != nil {
723 return nil, err
724 }
725 defer rows.Close()
726 var out []jobSig
727 for rows.Next() {
728 var s jobSig
729 var allow int
730 if err := rows.Scan(&s.name, &s.stage, &s.image, &s.script, &s.when, &allow); err != nil {
731 return nil, err
732 }
733 s.allow = allow == 1
734 out = append(out, s)
735 }
736 return out, rows.Err()
737 }
738
739 func sameJobSigs(got []jobSig, jobs []NewJob) bool {
740 if len(got) != len(jobs) {
741 return false
742 }
743 for i, j := range jobs {
744 when := j.When
745 if when == "" {
746 when = "on_success"
747 }
748 g := got[i]
749 if g.name != j.Name || g.stage != j.Stage || g.image != j.Image || g.script != j.Script || g.when != when || g.allow != j.AllowFailure {
750 return false
751 }
752 }
753 return true
754 }
755
756 func insertPipelineJobs(ctx context.Context, tx *sql.Tx, id, repoID int64, jobs []NewJob) error {
480 757 for _, j := range jobs {
481 758 if j.Variables == nil {
482 759 j.Variables = map[string]string{}
@@ -515,13 +792,10 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
515 792 j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),
516 793 j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))
517 794 if err != nil {
518 return 0, err
795 return err
519 796 }
520 797 }
521 if err := tx.Commit(); err != nil {
522 return 0, err
523 }
524 return id, nil
798 return nil
525 799 }
526 800
527 801 // ListPipelines geeft de nieuwste pipelines.
@@ -542,7 +816,7 @@ func (s *Store) ListPipelines(ctx context.Context, repoID int64) ([]Pipeline, er
542 816 return out, rows.Err()
543 817 }
544 818
545 const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id FROM pipelines`
819 const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`
546 820
547 821 func scanPipeline(rows *sql.Rows) (Pipeline, error) {
548 822 return scanPipelineRow(rows)
@@ -551,7 +825,7 @@ func scanPipeline(rows *sql.Rows) (Pipeline, error) {
551 825 func scanPipelineRow(sc scanner) (Pipeline, error) {
552 826 var p Pipeline
553 827 var created, finished, started sql.NullString
554 if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID); err != nil {
828 if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID, &p.TargetSHA); err != nil {
555 829 return Pipeline{}, err
556 830 }
557 831 p.CreatedAt = parseTime(created.String)
@@ -399,6 +399,10 @@ type Pipeline struct {
399 StartedAt time.Time
399 StartedAt time.Time
400 FinishedAt time.Time
400 FinishedAt time.Time
401 MRID int64
401 MRID int64
402 // TargetSHA is de doelcommit van een merge-request-pipeline.
403 // Leeg zolang de rij nog niet aan een merge request hangt.
404 // Een nieuwe doelcommit start een nieuwe pipeline.
405 TargetSHA string
402}
406}
403
407
404// Job is één container in een pipeline.
408// Job is één container in een pipeline.
@@ -442,30 +446,71 @@ type Job struct {
442}
446}
443
447
444// RecordConfigFailure bewaart een pipeline die niet kon starten.
448// RecordConfigFailure bewaart een pipeline die niet kon starten.
445func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, message string) (int64, error) {
449// Dezelfde repo, ref, sha, bron, merge request en doel-sha levert de
450// bestaande pipeline terug. Een kapotte push die al bestaat wordt aan
451// de merge request gekoppeld in plaats van een tweede mislukte rij.
452func (s *Store) RecordConfigFailure(ctx context.Context, repoID int64, sha, ref, source, message string, mrID int64, targetSHA string) (int64, bool, error) {
453 if source == "" {
454 source = "push"
455 }
446 tx, err := s.db.BeginTx(ctx, nil)
456 tx, err := s.db.BeginTx(ctx, nil)
447 if err != nil {
457 if err != nil {
448 return 0, err
458 return 0, false, err
449 }
459 }
450 defer func() { _ = tx.Rollback() }()
460 defer func() { _ = tx.Rollback() }()
461 if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {
462 return 0, false, err
463 } else if ok {
464 return finishExisting(ctx, tx, id)
465 }
466 if source == "push" && mrID == 0 {
467 if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {
468 return 0, false, err
469 } else if ok {
470 return finishExisting(ctx, tx, id)
471 }
472 }
473 if mrID != 0 {
474 if id, ok, err := configFailurePush(ctx, tx, repoID, sha, ref, mrID, targetSHA); err != nil {
475 return 0, false, err
476 } else if ok {
477 if err := bindPipelineMR(ctx, tx, id, mrID, targetSHA); err != nil {
478 return 0, false, err
479 }
480 return finishExisting(ctx, tx, id)
481 }
482 }
451 ts := now()
483 ts := now()
452 res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, created_at, finished_at) VALUES (?, ?, ?, 'failed', ?, ?)`, repoID, sha, ref, ts, ts)
484 res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, created_at, finished_at, mr_id, target_sha)
485 SELECT ?, ?, ?, 'failed', ?, ?, ?, ?, ?
486 WHERE NOT EXISTS (
487 SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?
488 )`, repoID, sha, ref, source, ts, ts, mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA)
453 if err != nil {
489 if err != nil {
454 return 0, err
490 return 0, false, err
491 }
492 if n, _ := res.RowsAffected(); n == 0 {
493 id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
494 if err != nil {
495 return 0, false, err
496 }
497 return finishExisting(ctx, tx, id)
455 }
498 }
456 id, _ := res.LastInsertId()
499 id, _ := res.LastInsertId()
457 _, err = tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, finished_at)
500 _, err = tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, finished_at)
458 VALUES (?, ?, 'blue-ci-cd', 'test', 0, 'failed', 'alpine:3.20', 'exit 1', '[]', 0, '{}', '[]', 0, 0, ?, ?)`, id, repoID, message, ts)
501 VALUES (?, ?, 'blue-ci-cd', 'test', 0, 'failed', 'alpine:3.20', 'exit 1', '[]', 0, '{}', '[]', 0, 0, ?, ?)`, id, repoID, message, ts)
459 if err != nil {
502 if err != nil {
460 return 0, err
503 return 0, false, err
461 }
504 }
462 if err := tx.Commit(); err != nil {
505 if err := tx.Commit(); err != nil {
463 return 0, err
506 return 0, false, err
464 }
507 }
465 return id, nil
508 return id, true, nil
466}
509}
467
510
468// CreatePipeline maakt een pipeline met jobs in de wachtrij.
511// CreatePipeline maakt een pipeline met jobs in de wachtrij.
512// Elke aanroep voegt een rij toe. Starten via een push of merge request
513// gaat via OpenPipeline, zodat een dubbele hook dezelfde run teruggeeft.
469func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title string, jobs []NewJob) (int64, error) {
514func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title string, jobs []NewJob) (int64, error) {
470 tx, err := s.db.BeginTx(ctx, nil)
515 tx, err := s.db.BeginTx(ctx, nil)
471 if err != nil {
516 if err != nil {
@@ -477,6 +522,238 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
477 return 0, err
522 return 0, err
478 }
523 }
479 id, _ := res.LastInsertId()
524 id, _ := res.LastInsertId()
525 if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {
526 return 0, err
527 }
528 if err := tx.Commit(); err != nil {
529 return 0, err
530 }
531 return id, nil
532}
533
534// OpenPipeline maakt een pipeline, of geeft de bestaande terug.
535// De sleutel is repo, ref, sha, bron, merge request en doel-sha.
536// Een push voor dezelfde sha blijft één rij, ook nadat een merge request
537// hem heeft gekoppeld. Kiezen push en merge request dezelfde jobs, dan
538// koppelt deze aanroep de push-rij en maakt hij geen tweede pipeline.
539// Een andere jobset of een nieuwe doel-sha start wel een eigen rij.
540// created is false wanneer een eerdere trigger de run al had.
541func (s *Store) OpenPipeline(ctx context.Context, repoID int64, sha, ref, source, actor, title, targetSHA string, mrID int64, jobs []NewJob) (int64, bool, error) {
542 if source == "" {
543 source = "push"
544 }
545 tx, err := s.db.BeginTx(ctx, nil)
546 if err != nil {
547 return 0, false, err
548 }
549 defer func() { _ = tx.Rollback() }()
550 if id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA); err != nil {
551 return 0, false, err
552 } else if ok {
553 return finishExisting(ctx, tx, id)
554 }
555 if source == "push" && mrID == 0 {
556 if id, ok, err := newestPush(ctx, tx, repoID, sha, ref); err != nil {
557 return 0, false, err
558 } else if ok {
559 return finishExisting(ctx, tx, id)
560 }
561 }
562 if mrID != 0 {
563 if id, ok, err := claimMatchingPush(ctx, tx, repoID, sha, ref, mrID, targetSHA, jobs); err != nil {
564 return 0, false, err
565 } else if ok {
566 return finishExisting(ctx, tx, id)
567 }
568 }
569 notExists := `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ?`
570 args := []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref, source, mrID, targetSHA}
571 if source == "push" && mrID == 0 {
572 notExists = `SELECT 1 FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'`
573 args = []any{repoID, sha, ref, source, actor, title, now(), mrID, targetSHA, repoID, sha, ref}
574 }
575 res, err := tx.ExecContext(ctx, `INSERT INTO pipelines (repo_id, commit_sha, ref, status, source, actor, commit_title, created_at, mr_id, target_sha)
576 SELECT ?, ?, ?, 'queued', ?, ?, ?, ?, ?, ?
577 WHERE NOT EXISTS (`+notExists+`)`, args...)
578 if err != nil {
579 return 0, false, err
580 }
581 if n, _ := res.RowsAffected(); n == 0 {
582 id, ok, err := existingPipeline(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
583 if err != nil {
584 return 0, false, err
585 }
586 if !ok && source == "push" && mrID == 0 {
587 id, ok, err = newestPush(ctx, tx, repoID, sha, ref)
588 if err != nil {
589 return 0, false, err
590 }
591 }
592 if !ok {
593 return 0, false, errors.New("pipeline ontbreekt na een lege insert")
594 }
595 return finishExisting(ctx, tx, id)
596 }
597 id, _ := res.LastInsertId()
598 if err := insertPipelineJobs(ctx, tx, id, repoID, jobs); err != nil {
599 return 0, false, err
600 }
601 if err := tx.Commit(); err != nil {
602 return 0, false, err
603 }
604 return id, true, nil
605}
606
607func finishExisting(ctx context.Context, tx *sql.Tx, id int64) (int64, bool, error) {
608 if err := tx.Commit(); err != nil {
609 return 0, false, err
610 }
611 return id, false, nil
612}
613
614func existingPipeline(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, bool, error) {
615 id, err := pipelineKeyID(ctx, tx, repoID, sha, ref, source, mrID, targetSHA)
616 if errors.Is(err, sql.ErrNoRows) {
617 return 0, false, nil
618 }
619 if err != nil {
620 return 0, false, err
621 }
622 return id, true, nil
623}
624
625func pipelineKeyID(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref, source string, mrID int64, targetSHA string) (int64, error) {
626 var id int64
627 err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = ? AND mr_id = ? AND target_sha = ? ORDER BY id LIMIT 1`,
628 repoID, sha, ref, source, mrID, targetSHA).Scan(&id)
629 return id, err
630}
631
632func newestPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string) (int64, bool, error) {
633 var id int64
634 err := tx.QueryRowContext(ctx, `SELECT id FROM pipelines WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push' ORDER BY id DESC LIMIT 1`,
635 repoID, sha, ref).Scan(&id)
636 if errors.Is(err, sql.ErrNoRows) {
637 return 0, false, nil
638 }
639 if err != nil {
640 return 0, false, err
641 }
642 return id, true, nil
643}
644
645// configFailurePush zoekt een mislukte push die alleen de kapotte yaml bewaart.
646func configFailurePush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string) (int64, bool, error) {
647 var id int64
648 err := tx.QueryRowContext(ctx, `SELECT p.id FROM pipelines p
649 WHERE p.repo_id = ? AND p.commit_sha = ? AND p.ref = ? AND p.source = 'push' AND p.status = 'failed'
650 AND (p.mr_id = 0 OR p.mr_id = ?) AND (p.target_sha = '' OR p.target_sha = ?)
651 AND EXISTS (SELECT 1 FROM jobs j WHERE j.pipeline_id = p.id AND j.name = 'blue-ci-cd')
652 ORDER BY p.id DESC LIMIT 1`, repoID, sha, ref, mrID, targetSHA).Scan(&id)
653 if errors.Is(err, sql.ErrNoRows) {
654 return 0, false, nil
655 }
656 if err != nil {
657 return 0, false, err
658 }
659 return id, true, nil
660}
661
662type pushCandidate struct {
663 id int64
664 mrID int64
665 target string
666}
667
668// claimMatchingPush koppelt een push-pipeline met dezelfde jobs aan de merge request.
669func claimMatchingPush(ctx context.Context, tx *sql.Tx, repoID int64, sha, ref string, mrID int64, targetSHA string, jobs []NewJob) (int64, bool, error) {
670 rows, err := tx.QueryContext(ctx, `SELECT id, mr_id, target_sha FROM pipelines
671 WHERE repo_id = ? AND commit_sha = ? AND ref = ? AND source = 'push'
672 AND (mr_id = 0 OR mr_id = ?) AND (target_sha = '' OR target_sha = ?)
673 ORDER BY id DESC`, repoID, sha, ref, mrID, targetSHA)
674 if err != nil {
675 return 0, false, err
676 }
677 var cands []pushCandidate
678 for rows.Next() {
679 var c pushCandidate
680 if err := rows.Scan(&c.id, &c.mrID, &c.target); err != nil {
681 rows.Close()
682 return 0, false, err
683 }
684 cands = append(cands, c)
685 }
686 if err := rows.Err(); err != nil {
687 rows.Close()
688 return 0, false, err
689 }
690 rows.Close()
691 for _, c := range cands {
692 if c.mrID == mrID && c.target == targetSHA {
693 return c.id, true, nil
694 }
695 got, err := loadJobSigs(ctx, tx, c.id)
696 if err != nil {
697 return 0, false, err
698 }
699 if !sameJobSigs(got, jobs) {
700 continue
701 }
702 if err := bindPipelineMR(ctx, tx, c.id, mrID, targetSHA); err != nil {
703 return 0, false, err
704 }
705 return c.id, true, nil
706 }
707 return 0, false, nil
708}
709
710func bindPipelineMR(ctx context.Context, tx *sql.Tx, id, mrID int64, targetSHA string) error {
711 _, err := tx.ExecContext(ctx, `UPDATE pipelines SET mr_id = ?, target_sha = ? WHERE id = ?`, mrID, targetSHA, id)
712 return err
713}
714
715type jobSig struct {
716 name, stage, image, script, when string
717 allow bool
718}
719
720func loadJobSigs(ctx context.Context, tx *sql.Tx, pipelineID int64) ([]jobSig, error) {
721 rows, err := tx.QueryContext(ctx, `SELECT name, stage, image, script, when_mode, allow_failure FROM jobs WHERE pipeline_id = ? ORDER BY position, id`, pipelineID)
722 if err != nil {
723 return nil, err
724 }
725 defer rows.Close()
726 var out []jobSig
727 for rows.Next() {
728 var s jobSig
729 var allow int
730 if err := rows.Scan(&s.name, &s.stage, &s.image, &s.script, &s.when, &allow); err != nil {
731 return nil, err
732 }
733 s.allow = allow == 1
734 out = append(out, s)
735 }
736 return out, rows.Err()
737}
738
739func sameJobSigs(got []jobSig, jobs []NewJob) bool {
740 if len(got) != len(jobs) {
741 return false
742 }
743 for i, j := range jobs {
744 when := j.When
745 if when == "" {
746 when = "on_success"
747 }
748 g := got[i]
749 if g.name != j.Name || g.stage != j.Stage || g.image != j.Image || g.script != j.Script || g.when != when || g.allow != j.AllowFailure {
750 return false
751 }
752 }
753 return true
754}
755
756func insertPipelineJobs(ctx context.Context, tx *sql.Tx, id, repoID int64, jobs []NewJob) error {
480 for _, j := range jobs {
757 for _, j := range jobs {
481 if j.Variables == nil {
758 if j.Variables == nil {
482 j.Variables = map[string]string{}
759 j.Variables = map[string]string{}
@@ -515,13 +792,10 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
515 j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),
792 j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),
516 j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))
793 j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))
517 if err != nil {
794 if err != nil {
518 return 0, err
795 return err
519 }
796 }
520 }
797 }
521 if err := tx.Commit(); err != nil {
798 return nil
522 return 0, err
523 }
524 return id, nil
525}
799}
526
800
527// ListPipelines geeft de nieuwste pipelines.
801// ListPipelines geeft de nieuwste pipelines.
@@ -542,7 +816,7 @@ func (s *Store) ListPipelines(ctx context.Context, repoID int64) ([]Pipeline, er
542 return out, rows.Err()
816 return out, rows.Err()
543}
817}
544
818
545const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id FROM pipelines`
819const pipelineCols = `SELECT id, repo_id, commit_sha, ref, status, created_at, finished_at, source, actor, commit_title, started_at, mr_id, target_sha FROM pipelines`
546
820
547func scanPipeline(rows *sql.Rows) (Pipeline, error) {
821func scanPipeline(rows *sql.Rows) (Pipeline, error) {
548 return scanPipelineRow(rows)
822 return scanPipelineRow(rows)
@@ -551,7 +825,7 @@ func scanPipeline(rows *sql.Rows) (Pipeline, error) {
551func scanPipelineRow(sc scanner) (Pipeline, error) {
825func scanPipelineRow(sc scanner) (Pipeline, error) {
552 var p Pipeline
826 var p Pipeline
553 var created, finished, started sql.NullString
827 var created, finished, started sql.NullString
554 if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID); err != nil {
828 if err := sc.Scan(&p.ID, &p.RepoID, &p.CommitSHA, &p.Ref, &p.Status, &created, &finished, &p.Source, &p.Actor, &p.CommitTitle, &started, &p.MRID, &p.TargetSHA); err != nil {
555 return Pipeline{}, err
829 return Pipeline{}, err
556 }
830 }
557 p.CreatedAt = parseTime(created.String)
831 p.CreatedAt = parseTime(created.String)
internal/store/pipeline_dedupe_test.go added +127 −0
@@ -0,0 +1,127 @@
1 package store
2
3 import (
4 "context"
5 "testing"
6 )
7
8 func TestOpenPipelineKeepsOnePerSource(t *testing.T) {
9 s := openTest(t)
10 ctx := context.Background()
11 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
12 if err != nil {
13 t.Fatal(err)
14 }
15 repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
16 if err != nil {
17 t.Fatal(err)
18 }
19 sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
20 ref := "refs/heads/gitlab-sync"
21 jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}
22
23 push, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
24 if err != nil || !created || push == 0 {
25 t.Fatalf("push: %d %v %v", push, created, err)
26 }
27 again, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
28 if err != nil || created || again != push {
29 t.Fatalf("tweede push: %d created=%v %v", again, created, err)
30 }
31 got, err := s.JobsByPipeline(ctx, push)
32 if err != nil || len(got) != 1 {
33 t.Fatalf("jobs verdubbeld: %d %v", len(got), err)
34 }
35
36 mr, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)
37 if err != nil || created || mr != push {
38 t.Fatalf("zelfde jobs horen de push te claimen: %d created=%v %v", mr, created, err)
39 }
40 linked, err := s.PipelineByID(ctx, repo.ID, push)
41 if err != nil || linked.MRID != 7 || linked.TargetSHA != "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" || linked.Source != "push" {
42 t.Fatalf("gekoppelde push: %+v %v", linked, err)
43 }
44 if jobs, err := s.JobsByPipeline(ctx, push); err != nil || len(jobs) != 1 {
45 t.Fatalf("claim voegt jobs toe: %d %v", len(jobs), err)
46 }
47 same, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)
48 if err != nil || created || same != push {
49 t.Fatalf("zelfde doel: %d created=%v %v", same, created, err)
50 }
51 replay, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
52 if err != nil || created || replay != push {
53 t.Fatalf("push na koppeling: %d created=%v %v", replay, created, err)
54 }
55 moved, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "cccccccccccccccccccccccccccccccccccccccc", 7, jobs)
56 if err != nil || !created || moved == push {
57 t.Fatalf("nieuwe doel-sha: %d created=%v %v", moved, created, err)
58 }
59 fresh, err := s.PipelineByID(ctx, repo.ID, moved)
60 if err != nil || fresh.Source != "merge_request_event" || fresh.MRID != 7 || fresh.TargetSHA != "cccccccccccccccccccccccccccccccccccccccc" {
61 t.Fatalf("nieuwe mr-pipeline: %+v %v", fresh, err)
62 }
63 other, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "web", "tim", "hand", "", 0, jobs)
64 if err != nil || !created || other == push {
65 t.Fatalf("web blijft een eigen bron: %d %v %v", other, created, err)
66 }
67
68 diff := []NewJob{{Name: "mr_job", Stage: "test", Image: "alpine:3.20", Script: "echo mr"}}
69 otherSHA := "dddddddddddddddddddddddddddddddddddddddd"
70 pushOnly, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "push", "tim", "anders", "", 0, jobs)
71 if err != nil || !created {
72 t.Fatalf("andere sha: %d %v %v", pushOnly, created, err)
73 }
74 apart, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "merge_request_event", "tim", "anders", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, diff)
75 if err != nil || !created || apart == pushOnly {
76 t.Fatalf("andere jobs blijven apart: %d created=%v %v", apart, created, err)
77 }
78
79 failID, created, err := s.RecordConfigFailure(ctx, repo.ID, sha, ref, "push", "kapot", 0, "")
80 if err != nil || created || failID != push {
81 t.Fatalf("kapotte yaml bovenop een push: %d created=%v %v", failID, created, err)
82 }
83 broken := "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
84 first, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")
85 if err != nil || !created || first == 0 {
86 t.Fatalf("eerste fout: %d %v %v", first, created, err)
87 }
88 second, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")
89 if err != nil || created || second != first {
90 t.Fatalf("tweede fout: %d created=%v %v", second, created, err)
91 }
92 claimed, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "merge_request_event", "kapot", 9, "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
93 if err != nil || created || claimed != first {
94 t.Fatalf("mr-fout claimt de push-fout: %d created=%v %v", claimed, created, err)
95 }
96 linked, err = s.PipelineByID(ctx, repo.ID, first)
97 if err != nil || linked.MRID != 9 {
98 t.Fatalf("fout gekoppeld: %+v %v", linked, err)
99 }
100 }
101
102 func TestCreatePipelineStaysDuplicating(t *testing.T) {
103 s := openTest(t)
104 ctx := context.Background()
105 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
106 if err != nil {
107 t.Fatal(err)
108 }
109 repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
110 if err != nil {
111 t.Fatal(err)
112 }
113 jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}
114 sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
115 ref := "refs/heads/master"
116 first, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "een", jobs)
117 if err != nil {
118 t.Fatal(err)
119 }
120 second, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "twee", jobs)
121 if err != nil {
122 t.Fatal(err)
123 }
124 if first == 0 || second == 0 || first == second {
125 t.Fatalf("CreatePipeline hoort twee rijen te maken: %d %d", first, second)
126 }
127 }
@@ -0,0 +1,127 @@
1package store
2
3import (
4 "context"
5 "testing"
6)
7
8func TestOpenPipelineKeepsOnePerSource(t *testing.T) {
9 s := openTest(t)
10 ctx := context.Background()
11 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
12 if err != nil {
13 t.Fatal(err)
14 }
15 repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
16 if err != nil {
17 t.Fatal(err)
18 }
19 sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
20 ref := "refs/heads/gitlab-sync"
21 jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}
22
23 push, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
24 if err != nil || !created || push == 0 {
25 t.Fatalf("push: %d %v %v", push, created, err)
26 }
27 again, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
28 if err != nil || created || again != push {
29 t.Fatalf("tweede push: %d created=%v %v", again, created, err)
30 }
31 got, err := s.JobsByPipeline(ctx, push)
32 if err != nil || len(got) != 1 {
33 t.Fatalf("jobs verdubbeld: %d %v", len(got), err)
34 }
35
36 mr, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)
37 if err != nil || created || mr != push {
38 t.Fatalf("zelfde jobs horen de push te claimen: %d created=%v %v", mr, created, err)
39 }
40 linked, err := s.PipelineByID(ctx, repo.ID, push)
41 if err != nil || linked.MRID != 7 || linked.TargetSHA != "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" || linked.Source != "push" {
42 t.Fatalf("gekoppelde push: %+v %v", linked, err)
43 }
44 if jobs, err := s.JobsByPipeline(ctx, push); err != nil || len(jobs) != 1 {
45 t.Fatalf("claim voegt jobs toe: %d %v", len(jobs), err)
46 }
47 same, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, jobs)
48 if err != nil || created || same != push {
49 t.Fatalf("zelfde doel: %d created=%v %v", same, created, err)
50 }
51 replay, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "push", "tim", "sync", "", 0, jobs)
52 if err != nil || created || replay != push {
53 t.Fatalf("push na koppeling: %d created=%v %v", replay, created, err)
54 }
55 moved, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "merge_request_event", "tim", "sync", "cccccccccccccccccccccccccccccccccccccccc", 7, jobs)
56 if err != nil || !created || moved == push {
57 t.Fatalf("nieuwe doel-sha: %d created=%v %v", moved, created, err)
58 }
59 fresh, err := s.PipelineByID(ctx, repo.ID, moved)
60 if err != nil || fresh.Source != "merge_request_event" || fresh.MRID != 7 || fresh.TargetSHA != "cccccccccccccccccccccccccccccccccccccccc" {
61 t.Fatalf("nieuwe mr-pipeline: %+v %v", fresh, err)
62 }
63 other, created, err := s.OpenPipeline(ctx, repo.ID, sha, ref, "web", "tim", "hand", "", 0, jobs)
64 if err != nil || !created || other == push {
65 t.Fatalf("web blijft een eigen bron: %d %v %v", other, created, err)
66 }
67
68 diff := []NewJob{{Name: "mr_job", Stage: "test", Image: "alpine:3.20", Script: "echo mr"}}
69 otherSHA := "dddddddddddddddddddddddddddddddddddddddd"
70 pushOnly, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "push", "tim", "anders", "", 0, jobs)
71 if err != nil || !created {
72 t.Fatalf("andere sha: %d %v %v", pushOnly, created, err)
73 }
74 apart, created, err := s.OpenPipeline(ctx, repo.ID, otherSHA, ref, "merge_request_event", "tim", "anders", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", 7, diff)
75 if err != nil || !created || apart == pushOnly {
76 t.Fatalf("andere jobs blijven apart: %d created=%v %v", apart, created, err)
77 }
78
79 failID, created, err := s.RecordConfigFailure(ctx, repo.ID, sha, ref, "push", "kapot", 0, "")
80 if err != nil || created || failID != push {
81 t.Fatalf("kapotte yaml bovenop een push: %d created=%v %v", failID, created, err)
82 }
83 broken := "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
84 first, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")
85 if err != nil || !created || first == 0 {
86 t.Fatalf("eerste fout: %d %v %v", first, created, err)
87 }
88 second, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "push", "kapot", 0, "")
89 if err != nil || created || second != first {
90 t.Fatalf("tweede fout: %d created=%v %v", second, created, err)
91 }
92 claimed, created, err := s.RecordConfigFailure(ctx, repo.ID, broken, ref, "merge_request_event", "kapot", 9, "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb")
93 if err != nil || created || claimed != first {
94 t.Fatalf("mr-fout claimt de push-fout: %d created=%v %v", claimed, created, err)
95 }
96 linked, err = s.PipelineByID(ctx, repo.ID, first)
97 if err != nil || linked.MRID != 9 {
98 t.Fatalf("fout gekoppeld: %+v %v", linked, err)
99 }
100}
101
102func TestCreatePipelineStaysDuplicating(t *testing.T) {
103 s := openTest(t)
104 ctx := context.Background()
105 tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
106 if err != nil {
107 t.Fatal(err)
108 }
109 repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
110 if err != nil {
111 t.Fatal(err)
112 }
113 jobs := []NewJob{{Name: "unit", Stage: "test", Image: "alpine:3.20", Script: "echo ok"}}
114 sha := "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
115 ref := "refs/heads/master"
116 first, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "een", jobs)
117 if err != nil {
118 t.Fatal(err)
119 }
120 second, err := s.CreatePipeline(ctx, repo.ID, sha, ref, "push", "tim", "twee", jobs)
121 if err != nil {
122 t.Fatal(err)
123 }
124 if first == 0 || second == 0 || first == second {
125 t.Fatalf("CreatePipeline hoort twee rijen te maken: %d %d", first, second)
126 }
127}
internal/store/store.go +1 −0
@@ -99,6 +99,7 @@ func (s *Store) migrate() error {
99 99 `ALTER TABLE secrets ADD COLUMN invisible INTEGER NOT NULL DEFAULT 0`,
100 100 `ALTER TABLE webhooks ADD COLUMN fields_json TEXT NOT NULL DEFAULT ''`,
101 101 `ALTER TABLE access_tokens ADD COLUMN scopes TEXT NOT NULL DEFAULT 'api'`,
102 `ALTER TABLE pipelines ADD COLUMN target_sha TEXT NOT NULL DEFAULT ''`,
102 103 }
103 104 for _, q := range alters {
104 105 if _, err := s.db.Exec(q); err != nil && !strings.Contains(strings.ToLower(err.Error()), "duplicate column") {
@@ -99,6 +99,7 @@ func (s *Store) migrate() error {
99 `ALTER TABLE secrets ADD COLUMN invisible INTEGER NOT NULL DEFAULT 0`,
99 `ALTER TABLE secrets ADD COLUMN invisible INTEGER NOT NULL DEFAULT 0`,
100 `ALTER TABLE webhooks ADD COLUMN fields_json TEXT NOT NULL DEFAULT ''`,
100 `ALTER TABLE webhooks ADD COLUMN fields_json TEXT NOT NULL DEFAULT ''`,
101 `ALTER TABLE access_tokens ADD COLUMN scopes TEXT NOT NULL DEFAULT 'api'`,
101 `ALTER TABLE access_tokens ADD COLUMN scopes TEXT NOT NULL DEFAULT 'api'`,
102 `ALTER TABLE pipelines ADD COLUMN target_sha TEXT NOT NULL DEFAULT ''`,
102 }
103 }
103 for _, q := range alters {
104 for _, q := range alters {
104 if _, err := s.db.Exec(q); err != nil && !strings.Contains(strings.ToLower(err.Error()), "duplicate column") {
105 if _, err := s.db.Exec(q); err != nil && !strings.Contains(strings.ToLower(err.Error()), "duplicate column") {