Merge branch 'cursor/ci-uitbreiding-65da' into 'master'
CI: rules met &&/||, workflow, retry, parallel, services en interruptible See merge request Timfa/bluegit!15
@@ -37,7 +37,8 @@ jobs:
37
37
| `include` | Alleen `local`. Een job met dezelfde naam wordt samengevoegd; het buitenste bestand wint per veld. |
38
38
| `variables` | Tekst, of `{value: ...}`. Een job overschrijft dezelfde naam. |
39
39
| `before_script`, `script`, `after_script` | `after_script` draait ook als `script` mislukt. |
40
| `needs` | Namen, of `{job, optional}`. `needs: []` start meteen. Weggelaten: wacht op eerdere stages. |
40
| `workflow` | `rules` zoals bij een job. Geen passende rule, of `when: never`, maakt geen pipeline. |
41
| `needs` | Namen, of `{job, optional}`. `needs: []` start meteen. Weggelaten: wacht op eerdere stages. Een `needs` op een `parallel`-job wacht op elke instantie. |
41
42
| `rules` | De eerste rule die klopt wint. `if`, `changes`, `exists` en `when`. Zonder rules gelden `only` en `except`. |
42
43
| `only`, `except` | Taknaam, `branches`, `tags`, of `/regex/`. |
43
44
| `when` | `on_success` (standaard), `on_failure`, `always`, `manual`. In een rule ook `never`. |
@@ -45,30 +46,41 @@ jobs:
45
46
| `artifacts` | `paths` en `expire_in` (`30s`, `30m`, `12h`, `14d`, `2w`, `never`). |
46
47
| `cache` | `paths` en een tekstuele `key`. Lokaal per repository, niet gedeeld met een andere machine. |
47
48
| `resource_group` | Jobs met dezelfde naam in één repository draaien niet tegelijk. |
48
| `timeout` | Overschrijft `ci.timeout_seconds` voor die job. |
49
| `timeout` | Overschrijft `ci.timeout_seconds` voor die job. De runner stopt de container als de tijd om is. |
50
| `retry` | Een getal, of `{max, when}`. `max` is 0 tot 2; hoger wordt afgekapt. `retry: 2` herhaalt elke mislukking behalve een annulering. `when` noemt `script_failure`, `runner_system_failure`, `job_execution_timeout`, `stuck_or_timeout_failure` of `always`. |
51
| `parallel` | Een getal, hoogstens 50. De jobs heten `naam 1/N` en krijgen `CI_NODE_INDEX` en `CI_NODE_TOTAL`. |
52
| `parallel:matrix` | Per lijstitem het cartesisch product, daarna aan elkaar. De job heet `naam: [waarde, waarde]`. De sleutels worden variabelen, in de volgorde uit het bestand. |
53
| `services` | Sidecar-containers op een eigen Docker-netwerk met de job. Een tekst, of `name`, `alias`, `command`, `entrypoint` en `variables`. Zonder alias is dat het imagenaam zonder registry en tag. |
54
| `environment` | Een naam, of `{name, url}`. Zet `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL`. Geen deployment en geen `on_stop`. |
55
| `interruptible` | Standaard uit. Een nieuwe pipeline op dezelfde ref annuleert oudere jobs die dit aan hebben en nog in de wachtrij staan of lopen. |
49
56
| `tags` | Bewaard bij de job. De ingebouwde runner draait elke job; er is geen tweede runner om op tags te kiezen. |
50
57
| `extends` | Eén naam of een lijst. Een job waarvan de naam met `.` begint is een sjabloon en draait niet. |
51
58
52
Een `if` is één vergelijking, zonder `&&` of `||`:
59
Een `if` volgt de GitLab-vorm. `&&` bindt sterker dan `||`. Haakjes groeperen. Een lege variabele is `null`.
53
60
54
61
```text
55
$NAAM == "waarde"
56
$NAAM == $ANDERE
57
$NAAM == null
62
$NAAM == "waarde" && ($ANDERE == "x" || $ANDERE == null)
58
63
$NAAM != "waarde"
59
64
$NAAM =~ /reguliere-expressie/
65
$NAAM =~ /^pad\/met\/slash/
66
$NAAM =~ /hoofdletters/i
60
67
$NAAM !~ /reguliere-expressie/
68
$NAAM =~ $PATROON
61
69
```
62
70
71
`=~` en `!~` gebruiken RE2, zonder anker tot je `^` en `$` zelf zet. Een `/` in het patroon schrijf je als `\/`. De vlag `i` staat direct achter de sluithaak. Een patroon in een variabele moet bij het beoordelen een geldige expressie zijn.
72
63
73
`changes` kijkt naar de diff van de push. Is die diff niet te bepalen, dan geldt de voorwaarde. `exists` eist dat elk pad in de commit staat.
64
74
65
75
Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.
66
76
67
Een merge-request-pipeline start bij het openen van de merge request en bij elke push naar de bronbranch. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat of op manual staat, houdt de merge tegen.
77
Een merge-request-pipeline start bij het openen van de merge request, bij elke push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht. Die laatste draait nog steeds op de commit van de bronbranch, niet op een samengevoegde boom. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.
78
79
`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.
68
80
69
81
## Bewust niet
70
82
71
`services`, `pages`, `workflow`, `spec`, een include van een ander project of van een URL, `parallel`/`matrix`, trigger- en child-pipelines, `environment`, `retry`, `coverage`, `id_tokens`, `inherit` en `hooks`. Die sleutels geven een waarschuwing in de editor en houden de rest van het bestand niet tegen. Er is één runner, in het BlueGit-proces, die Docker op dezelfde host aanroept.
83
`pages`, `spec`, een include van een ander project of van een URL, trigger- en child-pipelines, `coverage`, `id_tokens`, `inherit` en `hooks`. Die sleutels geven een waarschuwing in de editor en houden de rest van het bestand niet tegen. Er is één runner, in het BlueGit-proces, die Docker op dezelfde host aanroept. Zijn naam is `runner.name` in de configuratie, standaard Steve. Die naam staat bij de runner op de jobpagina, op de pipelinedetails en boven het joblog. `tags` kiezen die runner niet. `services` delen één netwerk met de job; er is geen healthcheck verder dan een gestarte container.
72
84
73
85
## Editor
74
86
@@ -37,7 +37,8 @@ jobs:37
| `include` | Alleen `local`. Een job met dezelfde naam wordt samengevoegd; het buitenste bestand wint per veld. |37
| `include` | Alleen `local`. Een job met dezelfde naam wordt samengevoegd; het buitenste bestand wint per veld. |38
| `variables` | Tekst, of `{value: ...}`. Een job overschrijft dezelfde naam. |38
| `variables` | Tekst, of `{value: ...}`. Een job overschrijft dezelfde naam. |39
| `before_script`, `script`, `after_script` | `after_script` draait ook als `script` mislukt. |39
| `before_script`, `script`, `after_script` | `after_script` draait ook als `script` mislukt. |40
| `needs` | Namen, of `{job, optional}`. `needs: []` start meteen. Weggelaten: wacht op eerdere stages. |40
| `workflow` | `rules` zoals bij een job. Geen passende rule, of `when: never`, maakt geen pipeline. |41
| `needs` | Namen, of `{job, optional}`. `needs: []` start meteen. Weggelaten: wacht op eerdere stages. Een `needs` op een `parallel`-job wacht op elke instantie. |41
| `rules` | De eerste rule die klopt wint. `if`, `changes`, `exists` en `when`. Zonder rules gelden `only` en `except`. |42
| `rules` | De eerste rule die klopt wint. `if`, `changes`, `exists` en `when`. Zonder rules gelden `only` en `except`. |42
| `only`, `except` | Taknaam, `branches`, `tags`, of `/regex/`. |43
| `only`, `except` | Taknaam, `branches`, `tags`, of `/regex/`. |43
| `when` | `on_success` (standaard), `on_failure`, `always`, `manual`. In een rule ook `never`. |44
| `when` | `on_success` (standaard), `on_failure`, `always`, `manual`. In een rule ook `never`. |@@ -45,30 +46,41 @@ jobs:45
| `artifacts` | `paths` en `expire_in` (`30s`, `30m`, `12h`, `14d`, `2w`, `never`). |46
| `artifacts` | `paths` en `expire_in` (`30s`, `30m`, `12h`, `14d`, `2w`, `never`). |46
| `cache` | `paths` en een tekstuele `key`. Lokaal per repository, niet gedeeld met een andere machine. |47
| `cache` | `paths` en een tekstuele `key`. Lokaal per repository, niet gedeeld met een andere machine. |47
| `resource_group` | Jobs met dezelfde naam in één repository draaien niet tegelijk. |48
| `resource_group` | Jobs met dezelfde naam in één repository draaien niet tegelijk. |48
| `timeout` | Overschrijft `ci.timeout_seconds` voor die job. |49
| `timeout` | Overschrijft `ci.timeout_seconds` voor die job. De runner stopt de container als de tijd om is. |50
| `retry` | Een getal, of `{max, when}`. `max` is 0 tot 2; hoger wordt afgekapt. `retry: 2` herhaalt elke mislukking behalve een annulering. `when` noemt `script_failure`, `runner_system_failure`, `job_execution_timeout`, `stuck_or_timeout_failure` of `always`. |51
| `parallel` | Een getal, hoogstens 50. De jobs heten `naam 1/N` en krijgen `CI_NODE_INDEX` en `CI_NODE_TOTAL`. |52
| `parallel:matrix` | Per lijstitem het cartesisch product, daarna aan elkaar. De job heet `naam: [waarde, waarde]`. De sleutels worden variabelen, in de volgorde uit het bestand. |53
| `services` | Sidecar-containers op een eigen Docker-netwerk met de job. Een tekst, of `name`, `alias`, `command`, `entrypoint` en `variables`. Zonder alias is dat het imagenaam zonder registry en tag. |54
| `environment` | Een naam, of `{name, url}`. Zet `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL`. Geen deployment en geen `on_stop`. |55
| `interruptible` | Standaard uit. Een nieuwe pipeline op dezelfde ref annuleert oudere jobs die dit aan hebben en nog in de wachtrij staan of lopen. |49
| `tags` | Bewaard bij de job. De ingebouwde runner draait elke job; er is geen tweede runner om op tags te kiezen. |56
| `tags` | Bewaard bij de job. De ingebouwde runner draait elke job; er is geen tweede runner om op tags te kiezen. |50
| `extends` | Eén naam of een lijst. Een job waarvan de naam met `.` begint is een sjabloon en draait niet. |57
| `extends` | Eén naam of een lijst. Een job waarvan de naam met `.` begint is een sjabloon en draait niet. |51
58
52
Een `if` is één vergelijking, zonder `&&` of `||`:59
Een `if` volgt de GitLab-vorm. `&&` bindt sterker dan `||`. Haakjes groeperen. Een lege variabele is `null`.53
60
54
```text61
```text55
$NAAM == "waarde"62
$NAAM == "waarde" && ($ANDERE == "x" || $ANDERE == null)56
$NAAM == $ANDERE57
$NAAM == null58
$NAAM != "waarde"63
$NAAM != "waarde"59
$NAAM =~ /reguliere-expressie/64
$NAAM =~ /reguliere-expressie/65
$NAAM =~ /^pad\/met\/slash/66
$NAAM =~ /hoofdletters/i60
$NAAM !~ /reguliere-expressie/67
$NAAM !~ /reguliere-expressie/68
$NAAM =~ $PATROON61
```69
```62
70
71
`=~` en `!~` gebruiken RE2, zonder anker tot je `^` en `$` zelf zet. Een `/` in het patroon schrijf je als `\/`. De vlag `i` staat direct achter de sluithaak. Een patroon in een variabele moet bij het beoordelen een geldige expressie zijn.72
63
`changes` kijkt naar de diff van de push. Is die diff niet te bepalen, dan geldt de voorwaarde. `exists` eist dat elk pad in de commit staat.73
`changes` kijkt naar de diff van de push. Is die diff niet te bepalen, dan geldt de voorwaarde. `exists` eist dat elk pad in de commit staat.64
74
65
Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.75
Voorgedefinieerd, in rules en in de container: `CI`, `CI_COMMIT_SHA`, `CI_COMMIT_SHORT_SHA`, `CI_COMMIT_BRANCH`, `CI_COMMIT_TAG`, `CI_COMMIT_REF`, `CI_COMMIT_REF_NAME`, `CI_COMMIT_REF_SLUG`, `CI_COMMIT_TITLE`, `CI_DEFAULT_BRANCH`, `CI_PIPELINE_ID`, `CI_PIPELINE_SOURCE`, `CI_PIPELINE_URL`, `CI_PROJECT_PATH`, `CI_PROJECT_NAME`, `CI_PROJECT_NAMESPACE`, `CI_PROJECT_URL`, `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_API_URL`, `CI_CONFIG_PATH`, `CI_SERVER_NAME`. Op een tag is `CI_COMMIT_BRANCH` leeg. `CI_PIPELINE_SOURCE` is `push`, `web` of `merge_request_event`.66
76
67
Een merge-request-pipeline start bij het openen van de merge request en bij elke push naar de bronbranch. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat of op manual staat, houdt de merge tegen.77
Een merge-request-pipeline start bij het openen van de merge request, bij elke push naar de bronbranch, en opnieuw wanneer iemand naar de doelbranch pusht. Die laatste draait nog steeds op de commit van de bronbranch, niet op een samengevoegde boom. `CI_PIPELINE_SOURCE` is dan `merge_request_event`. Daarbij horen `CI_MERGE_REQUEST_ID`, `CI_MERGE_REQUEST_IID`, `CI_MERGE_REQUEST_TITLE`, `CI_MERGE_REQUEST_SOURCE_BRANCH_NAME`, `CI_MERGE_REQUEST_SOURCE_BRANCH_SHA`, `CI_MERGE_REQUEST_TARGET_BRANCH_NAME`, `CI_MERGE_REQUEST_TARGET_BRANCH_SHA`, `CI_MERGE_REQUEST_PROJECT_PATH` en `CI_MERGE_REQUEST_PROJECT_URL`, in de rules én in de container. De pipeline draait op de commit van de bronbranch. Jobs zonder `rules` en zonder `only: merge_requests` doen daar niet aan mee. `only: merge_requests` en `except: merge_requests` werken wel. Zodra zo'n bestand jobs voor de merge request kiest, mag er pas gemerged worden als de nieuwste merge-request-pipeline geslaagd is. Een job met `allow_failure` mag mislukken, maar moet wel af zijn. Een job die nog loopt, in de wachtrij staat, op manual staat of geannuleerd is, houdt de merge tegen.78
79
`CI_PIPELINE_ID` is tijdens het beoordelen van rules nog 0. `CI_JOB_ID`, `CI_JOB_NAME`, `CI_JOB_STAGE`, `CI_JOB_URL`, `CI_JOB_TOKEN`, `CI_NODE_INDEX`, `CI_NODE_TOTAL`, `CI_ENVIRONMENT_NAME` en `CI_ENVIRONMENT_URL` bestaan pas in de container.68
80
69
## Bewust niet81
## Bewust niet70
82
71
`services`, `pages`, `workflow`, `spec`, een include van een ander project of van een URL, `parallel`/`matrix`, trigger- en child-pipelines, `environment`, `retry`, `coverage`, `id_tokens`, `inherit` en `hooks`. Die sleutels geven een waarschuwing in de editor en houden de rest van het bestand niet tegen. Er is één runner, in het BlueGit-proces, die Docker op dezelfde host aanroept.83
`pages`, `spec`, een include van een ander project of van een URL, trigger- en child-pipelines, `coverage`, `id_tokens`, `inherit` en `hooks`. Die sleutels geven een waarschuwing in de editor en houden de rest van het bestand niet tegen. Er is één runner, in het BlueGit-proces, die Docker op dezelfde host aanroept. Zijn naam is `runner.name` in de configuratie, standaard Steve. Die naam staat bij de runner op de jobpagina, op de pipelinedetails en boven het joblog. `tags` kiezen die runner niet. `services` delen één netwerk met de job; er is geen healthcheck verder dan een gestarte container.72
84
73
## Editor85
## Editor74
86
@@ -17,7 +17,7 @@ Deze basis is genoeg om repositories te bewaren, te klonen, een pipeline te draa
17
17
1. **Productie aanzetten.** Domein registreren, de schijf mounten, `ReadWritePaths` voor dat pad zetten, de ingress uit [INGRESS.md](INGRESS.md) aanzetten, eerste aanmelding en in `/setup` het pad bevestigen, `master` beschermen, deploy-geheimen zetten, één keer met de hand deployen, daarna de pipeline.
18
18
2. **SSH in de praktijk.** Hostkey publiceren en `known_hosts` vullen. De publieke poort is `ssh.public_port` op de ingress; op de mini-PC blijft `ssh.listen` configureerbaar. Foutmeldingen van de hook gaan al via stderr naar de git-client; de UX van een geweigerde push kan scherper.
19
19
3. **Merge requests verder.** Opmerkingen, een echte diff in plaats van `git log --oneline`, en een pipeline die groen moet zijn voor het samenvoegen.
20
4. **CI.** Services naast een job (een database voor tests), een cache, live logs in plaats van het log na afloop, en opruimen van verlopen artifacts. Een tweede runner op een andere machine als de mini-PC het niet trekt.
20
4. **CI.** Live logs tijdens een job, en opruimen van verlopen artifacts. Een tweede runner op een andere machine als de mini-PC het niet trekt. `workflow`, `retry`, `parallel`, `services` en `environment` horen bij de ingebouwde runner; zie [CI.md](CI.md). `tags` sturen nog geen aparte runner aan.
21
21
5. **Webhooks.** Privénetwerken zijn dicht. Nog te doen: opnieuw proberen, en een knop die een testgebeurtenis stuurt.
22
22
6. **Git.** LFS als de 4 TB-schijf grote bestanden moet dragen, signed commits, en mirrors. Force-push op een beschermde branch is nu toegestaan voor een beheerder; een aparte schakelaar hoort daarbij.
23
23
7. **Account.** Uitnodigen voordat iemand ooit is aangemeld werkt al op gebruikersnaam. Een zoekfunctie ontbreekt, omdat de auth-server geen lijst van alle gebruikers aan een site geeft. E-mail naar een gebruiker kan later via `POST /api/email` van de auth-server, zonder het adres te zien.
@@ -17,7 +17,7 @@ Deze basis is genoeg om repositories te bewaren, te klonen, een pipeline te draa17
1. **Productie aanzetten.** Domein registreren, de schijf mounten, `ReadWritePaths` voor dat pad zetten, de ingress uit [INGRESS.md](INGRESS.md) aanzetten, eerste aanmelding en in `/setup` het pad bevestigen, `master` beschermen, deploy-geheimen zetten, één keer met de hand deployen, daarna de pipeline.17
1. **Productie aanzetten.** Domein registreren, de schijf mounten, `ReadWritePaths` voor dat pad zetten, de ingress uit [INGRESS.md](INGRESS.md) aanzetten, eerste aanmelding en in `/setup` het pad bevestigen, `master` beschermen, deploy-geheimen zetten, één keer met de hand deployen, daarna de pipeline.18
2. **SSH in de praktijk.** Hostkey publiceren en `known_hosts` vullen. De publieke poort is `ssh.public_port` op de ingress; op de mini-PC blijft `ssh.listen` configureerbaar. Foutmeldingen van de hook gaan al via stderr naar de git-client; de UX van een geweigerde push kan scherper.18
2. **SSH in de praktijk.** Hostkey publiceren en `known_hosts` vullen. De publieke poort is `ssh.public_port` op de ingress; op de mini-PC blijft `ssh.listen` configureerbaar. Foutmeldingen van de hook gaan al via stderr naar de git-client; de UX van een geweigerde push kan scherper.19
3. **Merge requests verder.** Opmerkingen, een echte diff in plaats van `git log --oneline`, en een pipeline die groen moet zijn voor het samenvoegen.19
3. **Merge requests verder.** Opmerkingen, een echte diff in plaats van `git log --oneline`, en een pipeline die groen moet zijn voor het samenvoegen.20
4. **CI.** Services naast een job (een database voor tests), een cache, live logs in plaats van het log na afloop, en opruimen van verlopen artifacts. Een tweede runner op een andere machine als de mini-PC het niet trekt.20
4. **CI.** Live logs tijdens een job, en opruimen van verlopen artifacts. Een tweede runner op een andere machine als de mini-PC het niet trekt. `workflow`, `retry`, `parallel`, `services` en `environment` horen bij de ingebouwde runner; zie [CI.md](CI.md). `tags` sturen nog geen aparte runner aan.21
5. **Webhooks.** Privénetwerken zijn dicht. Nog te doen: opnieuw proberen, en een knop die een testgebeurtenis stuurt.21
5. **Webhooks.** Privénetwerken zijn dicht. Nog te doen: opnieuw proberen, en een knop die een testgebeurtenis stuurt.22
6. **Git.** LFS als de 4 TB-schijf grote bestanden moet dragen, signed commits, en mirrors. Force-push op een beschermde branch is nu toegestaan voor een beheerder; een aparte schakelaar hoort daarbij.22
6. **Git.** LFS als de 4 TB-schijf grote bestanden moet dragen, signed commits, en mirrors. Force-push op een beschermde branch is nu toegestaan voor een beheerder; een aparte schakelaar hoort daarbij.23
7. **Account.** Uitnodigen voordat iemand ooit is aangemeld werkt al op gebruikersnaam. Een zoekfunctie ontbreekt, omdat de auth-server geen lijst van alle gebruikers aan een site geeft. E-mail naar een gebruiker kan later via `POST /api/email` van de auth-server, zonder het adres te zien.23
7. **Account.** Uitnodigen voordat iemand ooit is aangemeld werkt al op gebruikersnaam. Een zoekfunctie ontbreekt, omdat de auth-server geen lijst van alle gebruikers aan een site geeft. E-mail naar een gebruiker kan later via `POST /api/email` van de auth-server, zonder het adres te zien.@@ -4,6 +4,7 @@ import (
4
4
"fmt"
5
5
"path"
6
6
"regexp"
7
"strconv"
7
8
"strings"
8
9
9
10
"gitlab.com/Timfa/bluegit/internal/i18n"
@@ -147,7 +148,21 @@ func (p Pipeline) Select(ev Eval) ([]Job, error) {
147
148
pending[i].Needs = needs
148
149
out = append(out, pending[i])
149
150
}
150
return out, nil
151
return expandJobs(out)
152
}
153
154
// Allow zegt of workflow:rules deze pipeline toelaat.
155
// Zonder workflow wordt de pipeline gemaakt. Een passende rule met when:never
156
// of geen enkele passende rule slaat de pipeline over.
157
func (p Pipeline) Allow(ev Eval) (bool, error) {
158
if len(p.Workflow) == 0 {
159
return true, nil
160
}
161
if ev.Env == nil {
162
ev.Env = map[string]string{}
163
}
164
ok, _, err := (Job{Rules: p.Workflow, When: "always"}).matches(ev)
165
return ok, err
151
166
}
152
167
153
168
// ActiveJobs is Select met alleen variabelen. changes en exists gelden dan als waar.
@@ -398,7 +413,7 @@ func Gate(jobs []Job, status map[string]string, job Job) string {
398
413
switch status[d.Name] {
399
414
case "success", "skipped":
400
415
continue
401
case "failed":
416
case "failed", "canceled":
402
417
failedAny = true
403
418
if !d.AllowFailure {
404
419
failedHard = true
@@ -432,75 +447,184 @@ func Gate(jobs []Job, status map[string]string, job Job) string {
432
447
}
433
448
}
434
449
435
var (
436
reEq = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*==\s*(?:"([^"]*)"|\$([A-Za-z_][A-Za-z0-9_]*)|null)$`)
437
reNe = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*!=\s*(?:"([^"]*)"|\$([A-Za-z_][A-Za-z0-9_]*)|null)$`)
438
reRe = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*=~\s*/(.*)/$`)
439
reNRe = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*!~\s*/(.*)/$`)
440
)
450
func evalRule(expr string, env map[string]string) (bool, error) {
451
p := &ruleParser{s: expr, env: env}
452
ok, err := p.parseOr()
453
if err != nil {
454
return false, err
455
}
456
p.skip()
457
if p.i < len(p.s) {
458
return false, i18n.E("ci.err.ruleExpr", strings.TrimSpace(expr))
459
}
460
return ok, nil
461
}
441
462
442
type compiledRule struct {
443
kind string
444
name string
445
arg string
446
other string
447
null bool
448
re *regexp.Regexp
463
const maxParallel = 50
464
465
func expandJobs(jobs []Job) ([]Job, error) {
466
expanded := map[string][]string{}
467
var out []Job
468
seen := map[string]bool{}
469
for _, j := range jobs {
470
instances, err := expandOne(j)
471
if err != nil {
472
return nil, fmt.Errorf("job %s: %w", j.Name, err)
473
}
474
names := make([]string, len(instances))
475
for i, inst := range instances {
476
if seen[inst.Name] {
477
return nil, i18n.E("ci.err.jobDup", inst.Name)
478
}
479
seen[inst.Name] = true
480
names[i] = inst.Name
481
out = append(out, inst)
482
}
483
expanded[j.Name] = names
484
}
485
for i := range out {
486
out[i].Needs = rewriteNeeds(out[i].Needs, expanded)
487
out[i].Optional = rewriteNeeds(out[i].Optional, expanded)
488
}
489
return out, nil
449
490
}
450
491
451
func compileRule(expr string) (compiledRule, error) {
452
expr = strings.TrimSpace(expr)
453
if strings.Contains(expr, "&&") || strings.Contains(expr, "||") {
454
return compiledRule{}, i18n.E("ci.err.ruleCombine", expr)
492
func expandOne(j Job) ([]Job, error) {
493
if len(j.Matrix) > 0 {
494
var combos []matrixCombo
495
for _, spec := range j.Matrix {
496
combos = append(combos, cartesian(spec)...)
497
}
498
if len(combos) > maxParallel {
499
return nil, i18n.E("ci.err.parallel", len(combos), maxParallel)
500
}
501
if len(combos) == 0 {
502
return nil, i18n.E("ci.err.parallel", 0, maxParallel)
503
}
504
var out []Job
505
for i, combo := range combos {
506
cp := copyJob(j)
507
cp.Name = j.Name + ": [" + strings.Join(combo.values, ", ") + "]"
508
cp.Parallel = 0
509
cp.Matrix = nil
510
for k, v := range combo.vars {
511
cp.Variables[k] = v
512
}
513
cp.Variables["CI_NODE_INDEX"] = strconv.Itoa(i + 1)
514
cp.Variables["CI_NODE_TOTAL"] = strconv.Itoa(len(combos))
515
out = append(out, cp)
516
}
517
return out, nil
455
518
}
456
if m := reEq.FindStringSubmatch(expr); m != nil {
457
return compiledRule{kind: "eq", name: m[1], arg: m[2], other: m[3], null: m[2] == "" && m[3] == "" && strings.HasSuffix(strings.TrimSpace(expr), "null")}, nil
519
if j.Parallel <= 1 {
520
return []Job{j}, nil
458
521
}
459
if m := reNe.FindStringSubmatch(expr); m != nil {
460
return compiledRule{kind: "ne", name: m[1], arg: m[2], other: m[3], null: m[2] == "" && m[3] == "" && strings.HasSuffix(strings.TrimSpace(expr), "null")}, nil
522
if j.Parallel > maxParallel {
523
return nil, i18n.E("ci.err.parallel", j.Parallel, maxParallel)
461
524
}
462
if m := reRe.FindStringSubmatch(expr); m != nil {
463
re, err := regexp.Compile(m[2])
464
if err != nil {
465
return compiledRule{}, i18n.E("ci.err.regex", expr, err.Error())
466
}
467
return compiledRule{kind: "re", name: m[1], re: re}, nil
525
var out []Job
526
for i := 1; i <= j.Parallel; i++ {
527
cp := copyJob(j)
528
cp.Name = fmt.Sprintf("%s %d/%d", j.Name, i, j.Parallel)
529
cp.Parallel = 0
530
cp.Variables["CI_NODE_INDEX"] = strconv.Itoa(i)
531
cp.Variables["CI_NODE_TOTAL"] = strconv.Itoa(j.Parallel)
532
out = append(out, cp)
468
533
}
469
if m := reNRe.FindStringSubmatch(expr); m != nil {
470
re, err := regexp.Compile(m[2])
471
if err != nil {
472
return compiledRule{}, i18n.E("ci.err.regex", expr, err.Error())
534
return out, nil
535
}
536
537
type matrixCombo struct {
538
values []string
539
vars map[string]string
540
}
541
542
func cartesian(spec MatrixSpec) []matrixCombo {
543
if len(spec.Keys) == 0 {
544
return nil
545
}
546
var out []matrixCombo
547
var walk func(i int, vals []string, vars map[string]string)
548
walk = func(i int, vals []string, vars map[string]string) {
549
if i == len(spec.Keys) {
550
cp := map[string]string{}
551
for k, v := range vars {
552
cp[k] = v
553
}
554
out = append(out, matrixCombo{values: append([]string{}, vals...), vars: cp})
555
return
556
}
557
for _, v := range spec.Vals[i] {
558
vars[spec.Keys[i]] = v
559
walk(i+1, append(vals, v), vars)
473
560
}
474
return compiledRule{kind: "nre", name: m[1], re: re}, nil
475
561
}
476
return compiledRule{}, i18n.E("ci.err.ruleExpr", expr)
562
walk(0, nil, map[string]string{})
563
return out
477
564
}
478
565
479
func evalRule(expr string, env map[string]string) (bool, error) {
480
c, err := compileRule(expr)
481
if err != nil {
482
return false, err
566
func copyJob(j Job) Job {
567
cp := j
568
cp.Variables = map[string]string{}
569
for k, v := range j.Variables {
570
cp.Variables[k] = v
571
}
572
cp.Needs = append([]string{}, j.Needs...)
573
cp.Optional = append([]string{}, j.Optional...)
574
cp.Services = append([]Service{}, j.Services...)
575
return cp
576
}
577
578
func rewriteNeeds(list []string, expanded map[string][]string) []string {
579
if list == nil {
580
return nil
581
}
582
var out []string
583
for _, n := range list {
584
names := expanded[n]
585
if len(names) == 0 || (len(names) == 1 && names[0] == n) {
586
out = append(out, n)
587
continue
588
}
589
out = append(out, names...)
483
590
}
484
val := env[c.name]
485
right := c.arg
486
if c.other != "" {
487
right = env[c.other]
488
}
489
if c.null {
490
right = ""
491
}
492
switch c.kind {
493
case "eq":
494
return val == right, nil
495
case "ne":
496
return val != right, nil
497
case "re":
498
return c.re.MatchString(val), nil
499
case "nre":
500
return !c.re.MatchString(val), nil
501
default:
502
return false, i18n.E("ci.err.ruleExpr", expr)
591
if out == nil {
592
return []string{}
593
}
594
return out
595
}
596
597
// Expand vervangt $NAAM en ${NAAM} in een tekst, bijvoorbeeld environment:url.
598
func Expand(s string, env map[string]string) string {
599
var b strings.Builder
600
for i := 0; i < len(s); i++ {
601
if s[i] != '$' || i+1 >= len(s) {
602
b.WriteByte(s[i])
603
continue
604
}
605
if s[i+1] == '{' {
606
end := strings.IndexByte(s[i+2:], '}')
607
if end < 0 {
608
b.WriteByte('$')
609
continue
610
}
611
name := s[i+2 : i+2+end]
612
b.WriteString(env[name])
613
i += 2 + end
614
continue
615
}
616
j := i + 1
617
for j < len(s) && isIdent(s[j]) {
618
j++
619
}
620
if j == i+1 {
621
b.WriteByte('$')
622
continue
623
}
624
b.WriteString(env[s[i+1:j]])
625
i = j - 1
503
626
}
627
return b.String()
504
628
}
505
629
506
630
func contains(list []string, s string) bool {
@@ -4,6 +4,7 @@ import (4
"fmt"4
"fmt"5
"path"5
"path"6
"regexp"6
"regexp"7
"strconv"7
"strings"8
"strings"8
9
9
"gitlab.com/Timfa/bluegit/internal/i18n"10
"gitlab.com/Timfa/bluegit/internal/i18n"@@ -147,7 +148,21 @@ func (p Pipeline) Select(ev Eval) ([]Job, error) {147
pending[i].Needs = needs148
pending[i].Needs = needs148
out = append(out, pending[i])149
out = append(out, pending[i])149
}150
}150
return out, nil151
return expandJobs(out)152
}153
154
// Allow zegt of workflow:rules deze pipeline toelaat.155
// Zonder workflow wordt de pipeline gemaakt. Een passende rule met when:never156
// of geen enkele passende rule slaat de pipeline over.157
func (p Pipeline) Allow(ev Eval) (bool, error) {158
if len(p.Workflow) == 0 {159
return true, nil160
}161
if ev.Env == nil {162
ev.Env = map[string]string{}163
}164
ok, _, err := (Job{Rules: p.Workflow, When: "always"}).matches(ev)165
return ok, err151
}166
}152
167
153
// ActiveJobs is Select met alleen variabelen. changes en exists gelden dan als waar.168
// ActiveJobs is Select met alleen variabelen. changes en exists gelden dan als waar.@@ -398,7 +413,7 @@ func Gate(jobs []Job, status map[string]string, job Job) string {398
switch status[d.Name] {413
switch status[d.Name] {399
case "success", "skipped":414
case "success", "skipped":400
continue415
continue401
case "failed":416
case "failed", "canceled":402
failedAny = true417
failedAny = true403
if !d.AllowFailure {418
if !d.AllowFailure {404
failedHard = true419
failedHard = true@@ -432,75 +447,184 @@ func Gate(jobs []Job, status map[string]string, job Job) string {432
}447
}433
}448
}434
449
435
var (450
func evalRule(expr string, env map[string]string) (bool, error) {436
reEq = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*==\s*(?:"([^"]*)"|\$([A-Za-z_][A-Za-z0-9_]*)|null)$`)451
p := &ruleParser{s: expr, env: env}437
reNe = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*!=\s*(?:"([^"]*)"|\$([A-Za-z_][A-Za-z0-9_]*)|null)$`)452
ok, err := p.parseOr()438
reRe = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*=~\s*/(.*)/$`)453
if err != nil {439
reNRe = regexp.MustCompile(`^\$([A-Za-z_][A-Za-z0-9_]*)\s*!~\s*/(.*)/$`)454
return false, err440
)455
}456
p.skip()457
if p.i < len(p.s) {458
return false, i18n.E("ci.err.ruleExpr", strings.TrimSpace(expr))459
}460
return ok, nil461
}441
462
442
type compiledRule struct {463
const maxParallel = 50443
kind string464
444
name string465
func expandJobs(jobs []Job) ([]Job, error) {445
arg string466
expanded := map[string][]string{}446
other string467
var out []Job447
null bool468
seen := map[string]bool{}448
re *regexp.Regexp469
for _, j := range jobs {470
instances, err := expandOne(j)471
if err != nil {472
return nil, fmt.Errorf("job %s: %w", j.Name, err)473
}474
names := make([]string, len(instances))475
for i, inst := range instances {476
if seen[inst.Name] {477
return nil, i18n.E("ci.err.jobDup", inst.Name)478
}479
seen[inst.Name] = true480
names[i] = inst.Name481
out = append(out, inst)482
}483
expanded[j.Name] = names484
}485
for i := range out {486
out[i].Needs = rewriteNeeds(out[i].Needs, expanded)487
out[i].Optional = rewriteNeeds(out[i].Optional, expanded)488
}489
return out, nil449
}490
}450
491
451
func compileRule(expr string) (compiledRule, error) {492
func expandOne(j Job) ([]Job, error) {452
expr = strings.TrimSpace(expr)493
if len(j.Matrix) > 0 {453
if strings.Contains(expr, "&&") || strings.Contains(expr, "||") {494
var combos []matrixCombo454
return compiledRule{}, i18n.E("ci.err.ruleCombine", expr)495
for _, spec := range j.Matrix {496
combos = append(combos, cartesian(spec)...)497
}498
if len(combos) > maxParallel {499
return nil, i18n.E("ci.err.parallel", len(combos), maxParallel)500
}501
if len(combos) == 0 {502
return nil, i18n.E("ci.err.parallel", 0, maxParallel)503
}504
var out []Job505
for i, combo := range combos {506
cp := copyJob(j)507
cp.Name = j.Name + ": [" + strings.Join(combo.values, ", ") + "]"508
cp.Parallel = 0509
cp.Matrix = nil510
for k, v := range combo.vars {511
cp.Variables[k] = v512
}513
cp.Variables["CI_NODE_INDEX"] = strconv.Itoa(i + 1)514
cp.Variables["CI_NODE_TOTAL"] = strconv.Itoa(len(combos))515
out = append(out, cp)516
}517
return out, nil455
}518
}456
if m := reEq.FindStringSubmatch(expr); m != nil {519
if j.Parallel <= 1 {457
return compiledRule{kind: "eq", name: m[1], arg: m[2], other: m[3], null: m[2] == "" && m[3] == "" && strings.HasSuffix(strings.TrimSpace(expr), "null")}, nil520
return []Job{j}, nil458
}521
}459
if m := reNe.FindStringSubmatch(expr); m != nil {522
if j.Parallel > maxParallel {460
return compiledRule{kind: "ne", name: m[1], arg: m[2], other: m[3], null: m[2] == "" && m[3] == "" && strings.HasSuffix(strings.TrimSpace(expr), "null")}, nil523
return nil, i18n.E("ci.err.parallel", j.Parallel, maxParallel)461
}524
}462
if m := reRe.FindStringSubmatch(expr); m != nil {525
var out []Job463
re, err := regexp.Compile(m[2])526
for i := 1; i <= j.Parallel; i++ {464
if err != nil {527
cp := copyJob(j)465
return compiledRule{}, i18n.E("ci.err.regex", expr, err.Error())528
cp.Name = fmt.Sprintf("%s %d/%d", j.Name, i, j.Parallel)466
}529
cp.Parallel = 0467
return compiledRule{kind: "re", name: m[1], re: re}, nil530
cp.Variables["CI_NODE_INDEX"] = strconv.Itoa(i)531
cp.Variables["CI_NODE_TOTAL"] = strconv.Itoa(j.Parallel)532
out = append(out, cp)468
}533
}469
if m := reNRe.FindStringSubmatch(expr); m != nil {534
return out, nil470
re, err := regexp.Compile(m[2])535
}471
if err != nil {536
472
return compiledRule{}, i18n.E("ci.err.regex", expr, err.Error())537
type matrixCombo struct {538
values []string539
vars map[string]string540
}541
542
func cartesian(spec MatrixSpec) []matrixCombo {543
if len(spec.Keys) == 0 {544
return nil545
}546
var out []matrixCombo547
var walk func(i int, vals []string, vars map[string]string)548
walk = func(i int, vals []string, vars map[string]string) {549
if i == len(spec.Keys) {550
cp := map[string]string{}551
for k, v := range vars {552
cp[k] = v553
}554
out = append(out, matrixCombo{values: append([]string{}, vals...), vars: cp})555
return556
}557
for _, v := range spec.Vals[i] {558
vars[spec.Keys[i]] = v559
walk(i+1, append(vals, v), vars)473
}560
}474
return compiledRule{kind: "nre", name: m[1], re: re}, nil475
}561
}476
return compiledRule{}, i18n.E("ci.err.ruleExpr", expr)562
walk(0, nil, map[string]string{})563
return out477
}564
}478
565
479
func evalRule(expr string, env map[string]string) (bool, error) {566
func copyJob(j Job) Job {480
c, err := compileRule(expr)567
cp := j481
if err != nil {568
cp.Variables = map[string]string{}482
return false, err569
for k, v := range j.Variables {570
cp.Variables[k] = v571
}572
cp.Needs = append([]string{}, j.Needs...)573
cp.Optional = append([]string{}, j.Optional...)574
cp.Services = append([]Service{}, j.Services...)575
return cp576
}577
578
func rewriteNeeds(list []string, expanded map[string][]string) []string {579
if list == nil {580
return nil581
}582
var out []string583
for _, n := range list {584
names := expanded[n]585
if len(names) == 0 || (len(names) == 1 && names[0] == n) {586
out = append(out, n)587
continue588
}589
out = append(out, names...)483
}590
}484
val := env[c.name]591
if out == nil {485
right := c.arg592
return []string{}486
if c.other != "" {593
}487
right = env[c.other]594
return out488
}595
}489
if c.null {596
490
right = ""597
// Expand vervangt $NAAM en ${NAAM} in een tekst, bijvoorbeeld environment:url.491
}598
func Expand(s string, env map[string]string) string {492
switch c.kind {599
var b strings.Builder493
case "eq":600
for i := 0; i < len(s); i++ {494
return val == right, nil601
if s[i] != '$' || i+1 >= len(s) {495
case "ne":602
b.WriteByte(s[i])496
return val != right, nil603
continue497
case "re":604
}498
return c.re.MatchString(val), nil605
if s[i+1] == '{' {499
case "nre":606
end := strings.IndexByte(s[i+2:], '}')500
return !c.re.MatchString(val), nil607
if end < 0 {501
default:608
b.WriteByte('$')502
return false, i18n.E("ci.err.ruleExpr", expr)609
continue610
}611
name := s[i+2 : i+2+end]612
b.WriteString(env[name])613
i += 2 + end614
continue615
}616
j := i + 1617
for j < len(s) && isIdent(s[j]) {618
j++619
}620
if j == i+1 {621
b.WriteByte('$')622
continue623
}624
b.WriteString(env[s[i+1:j]])625
i = j - 1503
}626
}627
return b.String()504
}628
}505
629
506
func contains(list []string, s string) bool {630
func contains(list []string, s string) bool {@@ -0,0 +1,337 @@
1
package ci
2
3
import (
4
"regexp"
5
"strings"
6
"unicode"
7
8
"gitlab.com/Timfa/bluegit/internal/i18n"
9
)
10
11
// ruleParser leest een GitLab-achtige rules:if.
12
// && bindt sterker dan ||. Haakjes groeperen. Een lege waarde is null.
13
type ruleParser struct {
14
s string
15
i int
16
env map[string]string
17
syntax bool
18
}
19
20
func (p *ruleParser) parseOr() (bool, error) {
21
left, err := p.parseAnd()
22
if err != nil {
23
return false, err
24
}
25
for {
26
p.skip()
27
if !p.take("||") {
28
return left, nil
29
}
30
right, err := p.parseAnd()
31
if err != nil {
32
return false, err
33
}
34
left = left || right
35
}
36
}
37
38
func (p *ruleParser) parseAnd() (bool, error) {
39
left, err := p.parseCmp()
40
if err != nil {
41
return false, err
42
}
43
for {
44
p.skip()
45
if !p.take("&&") {
46
return left, nil
47
}
48
right, err := p.parseCmp()
49
if err != nil {
50
return false, err
51
}
52
left = left && right
53
}
54
}
55
56
func (p *ruleParser) parseCmp() (bool, error) {
57
p.skip()
58
if p.peek() == '(' {
59
p.i++
60
inner, err := p.parseOr()
61
if err != nil {
62
return false, err
63
}
64
p.skip()
65
if p.peek() != ')' {
66
return false, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
67
}
68
p.i++
69
return inner, nil
70
}
71
left, err := p.value()
72
if err != nil {
73
return false, err
74
}
75
p.skip()
76
op := p.operator()
77
if op == "" {
78
return false, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
79
}
80
right, err := p.value()
81
if err != nil {
82
return false, err
83
}
84
return compare(op, left, right, p.syntax)
85
}
86
87
type ruleValue struct {
88
text string
89
null bool
90
re *regexp.Regexp
91
}
92
93
func (p *ruleParser) value() (ruleValue, error) {
94
p.skip()
95
if p.i >= len(p.s) {
96
return ruleValue{}, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
97
}
98
switch p.s[p.i] {
99
case '$':
100
name, err := p.variable()
101
if err != nil {
102
return ruleValue{}, err
103
}
104
text := p.env[name]
105
return ruleValue{text: text, null: text == ""}, nil
106
case '"', '\'':
107
text, err := p.quoted()
108
if err != nil {
109
return ruleValue{}, err
110
}
111
return ruleValue{text: text, null: text == ""}, nil
112
case '/':
113
re, err := p.regex()
114
if err != nil {
115
return ruleValue{}, err
116
}
117
return ruleValue{re: re}, nil
118
default:
119
if p.takeWord("null") {
120
return ruleValue{null: true}, nil
121
}
122
return ruleValue{}, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
123
}
124
}
125
126
func (p *ruleParser) variable() (string, error) {
127
p.i++
128
if p.i < len(p.s) && p.s[p.i] == '{' {
129
p.i++
130
start := p.i
131
for p.i < len(p.s) && p.s[p.i] != '}' {
132
p.i++
133
}
134
if p.i >= len(p.s) || start == p.i {
135
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
136
}
137
name := p.s[start:p.i]
138
p.i++
139
if !ident(name) {
140
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
141
}
142
return name, nil
143
}
144
start := p.i
145
for p.i < len(p.s) && (isIdent(p.s[p.i])) {
146
p.i++
147
}
148
if start == p.i {
149
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
150
}
151
return p.s[start:p.i], nil
152
}
153
154
func (p *ruleParser) quoted() (string, error) {
155
quote := p.s[p.i]
156
p.i++
157
var b strings.Builder
158
for p.i < len(p.s) {
159
c := p.s[p.i]
160
if c == quote {
161
p.i++
162
return b.String(), nil
163
}
164
if c == '\\' && quote == '"' && p.i+1 < len(p.s) {
165
p.i++
166
b.WriteByte(p.s[p.i])
167
p.i++
168
continue
169
}
170
if c == '$' && quote == '"' {
171
name, err := p.variable()
172
if err != nil {
173
return "", err
174
}
175
b.WriteString(p.env[name])
176
continue
177
}
178
b.WriteByte(c)
179
p.i++
180
}
181
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
182
}
183
184
func (p *ruleParser) regex() (*regexp.Regexp, error) {
185
p.i++
186
var b strings.Builder
187
for p.i < len(p.s) {
188
c := p.s[p.i]
189
if c == '\\' && p.i+1 < len(p.s) {
190
next := p.s[p.i+1]
191
if next == '/' {
192
b.WriteByte('/')
193
} else {
194
b.WriteByte('\\')
195
b.WriteByte(next)
196
}
197
p.i += 2
198
continue
199
}
200
if c == '/' {
201
p.i++
202
pat := b.String()
203
if p.i < len(p.s) && (p.s[p.i] == 'i' || p.s[p.i] == 'I') {
204
pat = "(?i)" + pat
205
p.i++
206
}
207
re, err := regexp.Compile(pat)
208
if err != nil {
209
return nil, i18n.E("ci.err.regex", strings.TrimSpace(p.s), err.Error())
210
}
211
return re, nil
212
}
213
b.WriteByte(c)
214
p.i++
215
}
216
return nil, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))
217
}
218
219
func (p *ruleParser) operator() string {
220
p.skip()
221
for _, op := range []string{"=~", "!~", "==", "!="} {
222
if p.take(op) {
223
return op
224
}
225
}
226
return ""
227
}
228
229
func (p *ruleParser) take(lit string) bool {
230
if strings.HasPrefix(p.s[p.i:], lit) {
231
p.i += len(lit)
232
return true
233
}
234
return false
235
}
236
237
func (p *ruleParser) takeWord(word string) bool {
238
if !strings.HasPrefix(p.s[p.i:], word) {
239
return false
240
}
241
end := p.i + len(word)
242
if end < len(p.s) && isIdent(p.s[end]) {
243
return false
244
}
245
p.i = end
246
return true
247
}
248
249
func (p *ruleParser) peek() byte {
250
p.skip()
251
if p.i >= len(p.s) {
252
return 0
253
}
254
return p.s[p.i]
255
}
256
257
func (p *ruleParser) skip() {
258
for p.i < len(p.s) && unicode.IsSpace(rune(p.s[p.i])) {
259
p.i++
260
}
261
}
262
263
func compare(op string, left, right ruleValue, syntax bool) (bool, error) {
264
switch op {
265
case "==":
266
if left.re != nil || right.re != nil {
267
return false, i18n.E("ci.err.ruleExpr", op)
268
}
269
if left.null || right.null {
270
return left.null && right.null, nil
271
}
272
return left.text == right.text, nil
273
case "!=":
274
if left.re != nil || right.re != nil {
275
return false, i18n.E("ci.err.ruleExpr", op)
276
}
277
if left.null || right.null {
278
return left.null != right.null, nil
279
}
280
return left.text != right.text, nil
281
case "=~", "!~":
282
if left.re != nil {
283
return false, i18n.E("ci.err.ruleExpr", op)
284
}
285
re := right.re
286
if re == nil && !right.null {
287
compiled, err := regexp.Compile(right.text)
288
if err != nil {
289
return false, i18n.E("ci.err.regex", right.text, err.Error())
290
}
291
re = compiled
292
}
293
if re == nil {
294
if syntax && right.null && right.text == "" && right.re == nil {
295
return false, nil
296
}
297
return false, i18n.E("ci.err.ruleExpr", op)
298
}
299
ok := re.MatchString(left.text)
300
if op == "!~" {
301
ok = !ok
302
}
303
return ok, nil
304
default:
305
return false, i18n.E("ci.err.ruleExpr", op)
306
}
307
}
308
309
// checkRule keurt de vorm goed zonder variabelen. Een patroon in een
310
// variabele wordt pas bij het draaien gecompileerd.
311
func checkRule(expr string) error {
312
p := &ruleParser{s: expr, env: map[string]string{}, syntax: true}
313
if _, err := p.parseOr(); err != nil {
314
return err
315
}
316
p.skip()
317
if p.i < len(p.s) {
318
return i18n.E("ci.err.ruleExpr", strings.TrimSpace(expr))
319
}
320
return nil
321
}
322
323
func ident(s string) bool {
324
if s == "" {
325
return false
326
}
327
for i := 0; i < len(s); i++ {
328
if !isIdent(s[i]) {
329
return false
330
}
331
}
332
return true
333
}
334
335
func isIdent(c byte) bool {
336
return c == '_' || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9')
337
}
@@ -0,0 +1,337 @@1
package ci2
3
import (4
"regexp"5
"strings"6
"unicode"7
8
"gitlab.com/Timfa/bluegit/internal/i18n"9
)10
11
// ruleParser leest een GitLab-achtige rules:if.12
// && bindt sterker dan ||. Haakjes groeperen. Een lege waarde is null.13
type ruleParser struct {14
s string15
i int16
env map[string]string17
syntax bool18
}19
20
func (p *ruleParser) parseOr() (bool, error) {21
left, err := p.parseAnd()22
if err != nil {23
return false, err24
}25
for {26
p.skip()27
if !p.take("||") {28
return left, nil29
}30
right, err := p.parseAnd()31
if err != nil {32
return false, err33
}34
left = left || right35
}36
}37
38
func (p *ruleParser) parseAnd() (bool, error) {39
left, err := p.parseCmp()40
if err != nil {41
return false, err42
}43
for {44
p.skip()45
if !p.take("&&") {46
return left, nil47
}48
right, err := p.parseCmp()49
if err != nil {50
return false, err51
}52
left = left && right53
}54
}55
56
func (p *ruleParser) parseCmp() (bool, error) {57
p.skip()58
if p.peek() == '(' {59
p.i++60
inner, err := p.parseOr()61
if err != nil {62
return false, err63
}64
p.skip()65
if p.peek() != ')' {66
return false, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))67
}68
p.i++69
return inner, nil70
}71
left, err := p.value()72
if err != nil {73
return false, err74
}75
p.skip()76
op := p.operator()77
if op == "" {78
return false, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))79
}80
right, err := p.value()81
if err != nil {82
return false, err83
}84
return compare(op, left, right, p.syntax)85
}86
87
type ruleValue struct {88
text string89
null bool90
re *regexp.Regexp91
}92
93
func (p *ruleParser) value() (ruleValue, error) {94
p.skip()95
if p.i >= len(p.s) {96
return ruleValue{}, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))97
}98
switch p.s[p.i] {99
case '$':100
name, err := p.variable()101
if err != nil {102
return ruleValue{}, err103
}104
text := p.env[name]105
return ruleValue{text: text, null: text == ""}, nil106
case '"', '\'':107
text, err := p.quoted()108
if err != nil {109
return ruleValue{}, err110
}111
return ruleValue{text: text, null: text == ""}, nil112
case '/':113
re, err := p.regex()114
if err != nil {115
return ruleValue{}, err116
}117
return ruleValue{re: re}, nil118
default:119
if p.takeWord("null") {120
return ruleValue{null: true}, nil121
}122
return ruleValue{}, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))123
}124
}125
126
func (p *ruleParser) variable() (string, error) {127
p.i++128
if p.i < len(p.s) && p.s[p.i] == '{' {129
p.i++130
start := p.i131
for p.i < len(p.s) && p.s[p.i] != '}' {132
p.i++133
}134
if p.i >= len(p.s) || start == p.i {135
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))136
}137
name := p.s[start:p.i]138
p.i++139
if !ident(name) {140
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))141
}142
return name, nil143
}144
start := p.i145
for p.i < len(p.s) && (isIdent(p.s[p.i])) {146
p.i++147
}148
if start == p.i {149
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))150
}151
return p.s[start:p.i], nil152
}153
154
func (p *ruleParser) quoted() (string, error) {155
quote := p.s[p.i]156
p.i++157
var b strings.Builder158
for p.i < len(p.s) {159
c := p.s[p.i]160
if c == quote {161
p.i++162
return b.String(), nil163
}164
if c == '\\' && quote == '"' && p.i+1 < len(p.s) {165
p.i++166
b.WriteByte(p.s[p.i])167
p.i++168
continue169
}170
if c == '$' && quote == '"' {171
name, err := p.variable()172
if err != nil {173
return "", err174
}175
b.WriteString(p.env[name])176
continue177
}178
b.WriteByte(c)179
p.i++180
}181
return "", i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))182
}183
184
func (p *ruleParser) regex() (*regexp.Regexp, error) {185
p.i++186
var b strings.Builder187
for p.i < len(p.s) {188
c := p.s[p.i]189
if c == '\\' && p.i+1 < len(p.s) {190
next := p.s[p.i+1]191
if next == '/' {192
b.WriteByte('/')193
} else {194
b.WriteByte('\\')195
b.WriteByte(next)196
}197
p.i += 2198
continue199
}200
if c == '/' {201
p.i++202
pat := b.String()203
if p.i < len(p.s) && (p.s[p.i] == 'i' || p.s[p.i] == 'I') {204
pat = "(?i)" + pat205
p.i++206
}207
re, err := regexp.Compile(pat)208
if err != nil {209
return nil, i18n.E("ci.err.regex", strings.TrimSpace(p.s), err.Error())210
}211
return re, nil212
}213
b.WriteByte(c)214
p.i++215
}216
return nil, i18n.E("ci.err.ruleExpr", strings.TrimSpace(p.s))217
}218
219
func (p *ruleParser) operator() string {220
p.skip()221
for _, op := range []string{"=~", "!~", "==", "!="} {222
if p.take(op) {223
return op224
}225
}226
return ""227
}228
229
func (p *ruleParser) take(lit string) bool {230
if strings.HasPrefix(p.s[p.i:], lit) {231
p.i += len(lit)232
return true233
}234
return false235
}236
237
func (p *ruleParser) takeWord(word string) bool {238
if !strings.HasPrefix(p.s[p.i:], word) {239
return false240
}241
end := p.i + len(word)242
if end < len(p.s) && isIdent(p.s[end]) {243
return false244
}245
p.i = end246
return true247
}248
249
func (p *ruleParser) peek() byte {250
p.skip()251
if p.i >= len(p.s) {252
return 0253
}254
return p.s[p.i]255
}256
257
func (p *ruleParser) skip() {258
for p.i < len(p.s) && unicode.IsSpace(rune(p.s[p.i])) {259
p.i++260
}261
}262
263
func compare(op string, left, right ruleValue, syntax bool) (bool, error) {264
switch op {265
case "==":266
if left.re != nil || right.re != nil {267
return false, i18n.E("ci.err.ruleExpr", op)268
}269
if left.null || right.null {270
return left.null && right.null, nil271
}272
return left.text == right.text, nil273
case "!=":274
if left.re != nil || right.re != nil {275
return false, i18n.E("ci.err.ruleExpr", op)276
}277
if left.null || right.null {278
return left.null != right.null, nil279
}280
return left.text != right.text, nil281
case "=~", "!~":282
if left.re != nil {283
return false, i18n.E("ci.err.ruleExpr", op)284
}285
re := right.re286
if re == nil && !right.null {287
compiled, err := regexp.Compile(right.text)288
if err != nil {289
return false, i18n.E("ci.err.regex", right.text, err.Error())290
}291
re = compiled292
}293
if re == nil {294
if syntax && right.null && right.text == "" && right.re == nil {295
return false, nil296
}297
return false, i18n.E("ci.err.ruleExpr", op)298
}299
ok := re.MatchString(left.text)300
if op == "!~" {301
ok = !ok302
}303
return ok, nil304
default:305
return false, i18n.E("ci.err.ruleExpr", op)306
}307
}308
309
// checkRule keurt de vorm goed zonder variabelen. Een patroon in een310
// variabele wordt pas bij het draaien gecompileerd.311
func checkRule(expr string) error {312
p := &ruleParser{s: expr, env: map[string]string{}, syntax: true}313
if _, err := p.parseOr(); err != nil {314
return err315
}316
p.skip()317
if p.i < len(p.s) {318
return i18n.E("ci.err.ruleExpr", strings.TrimSpace(expr))319
}320
return nil321
}322
323
func ident(s string) bool {324
if s == "" {325
return false326
}327
for i := 0; i < len(s); i++ {328
if !isIdent(s[i]) {329
return false330
}331
}332
return true333
}334
335
func isIdent(c byte) bool {336
return c == '_' || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9')337
}@@ -0,0 +1,162 @@
1
package ci
2
3
import "testing"
4
5
func TestRuleExpr(t *testing.T) {
6
env := map[string]string{
7
"CI_COMMIT_BRANCH": "master",
8
"CI_DEFAULT_BRANCH": "master",
9
"CI_COMMIT_TAG": "",
10
"CI_PIPELINE_SOURCE": "push",
11
"PATTERN": `^release/.*`,
12
"MSG": "Hello.World",
13
}
14
cases := []struct {
15
expr string
16
ok bool
17
}{
18
{`$CI_COMMIT_BRANCH == "master"`, true},
19
{`$CI_COMMIT_BRANCH == "feature"`, false},
20
{`$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "push"`, true},
21
{`$CI_COMMIT_BRANCH == "nope" || $CI_PIPELINE_SOURCE == "push"`, true},
22
{`($CI_COMMIT_BRANCH == "nope" || $CI_COMMIT_BRANCH == "master") && $CI_COMMIT_TAG == null`, true},
23
{`$CI_COMMIT_TAG == null`, true},
24
{`$CI_COMMIT_BRANCH != null`, true},
25
{`$CI_COMMIT_BRANCH =~ /^master$/`, true},
26
{`$CI_COMMIT_BRANCH =~ /feature/`, false},
27
{`$CI_COMMIT_BRANCH !~ /feature/`, true},
28
{`$CI_COMMIT_BRANCH =~ /^MASTER$/i`, true},
29
{`$MSG =~ /Hello\.World/`, true},
30
{`$MSG =~ /Hello.World/`, true},
31
{`$CI_COMMIT_BRANCH =~ $PATTERN`, false},
32
{`$CI_COMMIT_BRANCH =~ /^release\/.*/ || $CI_COMMIT_BRANCH == "master"`, true},
33
{`"$CI_COMMIT_BRANCH" == "master"`, true},
34
{`${CI_COMMIT_BRANCH} == 'master'`, true},
35
}
36
for _, tc := range cases {
37
ok, err := evalRule(tc.expr, env)
38
if err != nil {
39
t.Fatalf("%s: %v", tc.expr, err)
40
}
41
if ok != tc.ok {
42
t.Fatalf("%s: got %v", tc.expr, ok)
43
}
44
}
45
if err := checkRule(`$FOO =~ $PATTERN`); err != nil {
46
t.Fatal(err)
47
}
48
if _, err := evalRule(`$CI_COMMIT_BRANCH ==`, env); err == nil {
49
t.Fatal("kapotte expressie")
50
}
51
if _, err := evalRule(`$MSG =~ /(/`, env); err == nil {
52
t.Fatal("kapotte regex")
53
}
54
if _, err := evalRule(`($CI_COMMIT_BRANCH == "master"`, env); err == nil {
55
t.Fatal("haakje")
56
}
57
}
58
59
func TestWorkflowParallelServices(t *testing.T) {
60
raw := []byte(`
61
workflow:
62
rules:
63
- if: $CI_COMMIT_BRANCH == "master" && $CI_PIPELINE_SOURCE == "push"
64
- when: never
65
default:
66
interruptible: true
67
retry: 2
68
build:
69
image: alpine:3.20
70
script: ["echo $CI_NODE_INDEX"]
71
parallel: 2
72
environment:
73
name: review
74
url: https://example.test/$CI_COMMIT_REF_SLUG
75
pack:
76
image: alpine:3.20
77
script: ["echo pack"]
78
needs: [build]
79
retry:
80
max: 5
81
when: [runner_system_failure, stuck_or_timeout_failure]
82
services:
83
- name: postgres:16
84
alias: db
85
variables:
86
POSTGRES_PASSWORD: secret
87
- redis:7
88
interruptible: false
89
matrix:
90
image: alpine:3.20
91
script: ["echo matrix"]
92
parallel:
93
matrix:
94
- PROVIDER: [aws, gcp]
95
STACK: app
96
- PROVIDER: aws
97
STACK: [backup]
98
`)
99
p, err := Parse(raw)
100
if err != nil {
101
t.Fatal(err)
102
}
103
if len(p.Workflow) != 2 {
104
t.Fatalf("workflow %d", len(p.Workflow))
105
}
106
ok, err := p.Allow(Eval{Env: map[string]string{"CI_COMMIT_BRANCH": "feature", "CI_PIPELINE_SOURCE": "push"}})
107
if err != nil || ok {
108
t.Fatalf("feature hoort geen pipeline: %v %v", ok, err)
109
}
110
ok, err = p.Allow(Eval{Env: map[string]string{"CI_COMMIT_BRANCH": "master", "CI_PIPELINE_SOURCE": "push"}})
111
if err != nil || !ok {
112
t.Fatalf("master hoort wel: %v %v", ok, err)
113
}
114
jobs, err := p.Select(Eval{Env: map[string]string{"CI_COMMIT_BRANCH": "master", "CI_PIPELINE_SOURCE": "push"}})
115
if err != nil {
116
t.Fatal(err)
117
}
118
byName := map[string]Job{}
119
for _, j := range jobs {
120
byName[j.Name] = j
121
}
122
if _, ok := byName["build 1/2"]; !ok || byName["build 1/2"].Variables["CI_NODE_TOTAL"] != "2" {
123
t.Fatalf("parallel: %v", namesOf(jobs))
124
}
125
if !byName["build 1/2"].Interruptible || byName["build 1/2"].Retry.Max != 2 {
126
t.Fatalf("default: %+v", byName["build 1/2"])
127
}
128
if byName["build 1/2"].Environment.Name != "review" || byName["build 1/2"].Environment.URL == "" {
129
t.Fatalf("environment: %+v", byName["build 1/2"].Environment)
130
}
131
pack := byName["pack"]
132
if pack.Interruptible || pack.Retry.Max != 2 || len(pack.Retry.When) != 2 {
133
t.Fatalf("retry geklemd: %+v", pack.Retry)
134
}
135
if len(pack.Needs) != 2 || pack.Needs[0] != "build 1/2" || pack.Needs[1] != "build 2/2" {
136
t.Fatalf("needs: %v", pack.Needs)
137
}
138
if len(pack.Services) != 2 || pack.Services[0].Alias != "db" || pack.Services[1].Alias != "redis" {
139
t.Fatalf("services: %+v", pack.Services)
140
}
141
if _, ok := byName["matrix: [aws, app]"]; !ok {
142
t.Fatalf("matrix: %v", namesOf(jobs))
143
}
144
if byName["matrix: [gcp, app]"].Variables["PROVIDER"] != "gcp" || byName["matrix: [aws, backup]"].Variables["STACK"] != "backup" {
145
t.Fatalf("matrix vars")
146
}
147
if got := Expand("https://example.test/$CI_COMMIT_REF_SLUG", map[string]string{"CI_COMMIT_REF_SLUG": "master"}); got != "https://example.test/master" {
148
t.Fatal(got)
149
}
150
}
151
152
func TestGateCanceled(t *testing.T) {
153
jobs := []Job{{Name: "a", Stage: "test"}, {Name: "b", Stage: "deploy"}}
154
status := map[string]string{"a": "canceled", "b": "queued"}
155
if got := Gate(jobs, status, jobs[1]); got != "skip" {
156
t.Fatalf("canceled houdt de volgende tegen: %s", got)
157
}
158
jobs[0].AllowFailure = true
159
if got := Gate(jobs, status, jobs[1]); got != "ready" {
160
t.Fatalf("allow_failure canceled: %s", got)
161
}
162
}
@@ -0,0 +1,162 @@1
package ci2
3
import "testing"4
5
func TestRuleExpr(t *testing.T) {6
env := map[string]string{7
"CI_COMMIT_BRANCH": "master",8
"CI_DEFAULT_BRANCH": "master",9
"CI_COMMIT_TAG": "",10
"CI_PIPELINE_SOURCE": "push",11
"PATTERN": `^release/.*`,12
"MSG": "Hello.World",13
}14
cases := []struct {15
expr string16
ok bool17
}{18
{`$CI_COMMIT_BRANCH == "master"`, true},19
{`$CI_COMMIT_BRANCH == "feature"`, false},20
{`$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH && $CI_PIPELINE_SOURCE == "push"`, true},21
{`$CI_COMMIT_BRANCH == "nope" || $CI_PIPELINE_SOURCE == "push"`, true},22
{`($CI_COMMIT_BRANCH == "nope" || $CI_COMMIT_BRANCH == "master") && $CI_COMMIT_TAG == null`, true},23
{`$CI_COMMIT_TAG == null`, true},24
{`$CI_COMMIT_BRANCH != null`, true},25
{`$CI_COMMIT_BRANCH =~ /^master$/`, true},26
{`$CI_COMMIT_BRANCH =~ /feature/`, false},27
{`$CI_COMMIT_BRANCH !~ /feature/`, true},28
{`$CI_COMMIT_BRANCH =~ /^MASTER$/i`, true},29
{`$MSG =~ /Hello\.World/`, true},30
{`$MSG =~ /Hello.World/`, true},31
{`$CI_COMMIT_BRANCH =~ $PATTERN`, false},32
{`$CI_COMMIT_BRANCH =~ /^release\/.*/ || $CI_COMMIT_BRANCH == "master"`, true},33
{`"$CI_COMMIT_BRANCH" == "master"`, true},34
{`${CI_COMMIT_BRANCH} == 'master'`, true},35
}36
for _, tc := range cases {37
ok, err := evalRule(tc.expr, env)38
if err != nil {39
t.Fatalf("%s: %v", tc.expr, err)40
}41
if ok != tc.ok {42
t.Fatalf("%s: got %v", tc.expr, ok)43
}44
}45
if err := checkRule(`$FOO =~ $PATTERN`); err != nil {46
t.Fatal(err)47
}48
if _, err := evalRule(`$CI_COMMIT_BRANCH ==`, env); err == nil {49
t.Fatal("kapotte expressie")50
}51
if _, err := evalRule(`$MSG =~ /(/`, env); err == nil {52
t.Fatal("kapotte regex")53
}54
if _, err := evalRule(`($CI_COMMIT_BRANCH == "master"`, env); err == nil {55
t.Fatal("haakje")56
}57
}58
59
func TestWorkflowParallelServices(t *testing.T) {60
raw := []byte(`61
workflow:62
rules:63
- if: $CI_COMMIT_BRANCH == "master" && $CI_PIPELINE_SOURCE == "push"64
- when: never65
default:66
interruptible: true67
retry: 268
build:69
image: alpine:3.2070
script: ["echo $CI_NODE_INDEX"]71
parallel: 272
environment:73
name: review74
url: https://example.test/$CI_COMMIT_REF_SLUG75
pack:76
image: alpine:3.2077
script: ["echo pack"]78
needs: [build]79
retry:80
max: 581
when: [runner_system_failure, stuck_or_timeout_failure]82
services:83
- name: postgres:1684
alias: db85
variables:86
POSTGRES_PASSWORD: secret87
- redis:788
interruptible: false89
matrix:90
image: alpine:3.2091
script: ["echo matrix"]92
parallel:93
matrix:94
- PROVIDER: [aws, gcp]95
STACK: app96
- PROVIDER: aws97
STACK: [backup]98
`)99
p, err := Parse(raw)100
if err != nil {101
t.Fatal(err)102
}103
if len(p.Workflow) != 2 {104
t.Fatalf("workflow %d", len(p.Workflow))105
}106
ok, err := p.Allow(Eval{Env: map[string]string{"CI_COMMIT_BRANCH": "feature", "CI_PIPELINE_SOURCE": "push"}})107
if err != nil || ok {108
t.Fatalf("feature hoort geen pipeline: %v %v", ok, err)109
}110
ok, err = p.Allow(Eval{Env: map[string]string{"CI_COMMIT_BRANCH": "master", "CI_PIPELINE_SOURCE": "push"}})111
if err != nil || !ok {112
t.Fatalf("master hoort wel: %v %v", ok, err)113
}114
jobs, err := p.Select(Eval{Env: map[string]string{"CI_COMMIT_BRANCH": "master", "CI_PIPELINE_SOURCE": "push"}})115
if err != nil {116
t.Fatal(err)117
}118
byName := map[string]Job{}119
for _, j := range jobs {120
byName[j.Name] = j121
}122
if _, ok := byName["build 1/2"]; !ok || byName["build 1/2"].Variables["CI_NODE_TOTAL"] != "2" {123
t.Fatalf("parallel: %v", namesOf(jobs))124
}125
if !byName["build 1/2"].Interruptible || byName["build 1/2"].Retry.Max != 2 {126
t.Fatalf("default: %+v", byName["build 1/2"])127
}128
if byName["build 1/2"].Environment.Name != "review" || byName["build 1/2"].Environment.URL == "" {129
t.Fatalf("environment: %+v", byName["build 1/2"].Environment)130
}131
pack := byName["pack"]132
if pack.Interruptible || pack.Retry.Max != 2 || len(pack.Retry.When) != 2 {133
t.Fatalf("retry geklemd: %+v", pack.Retry)134
}135
if len(pack.Needs) != 2 || pack.Needs[0] != "build 1/2" || pack.Needs[1] != "build 2/2" {136
t.Fatalf("needs: %v", pack.Needs)137
}138
if len(pack.Services) != 2 || pack.Services[0].Alias != "db" || pack.Services[1].Alias != "redis" {139
t.Fatalf("services: %+v", pack.Services)140
}141
if _, ok := byName["matrix: [aws, app]"]; !ok {142
t.Fatalf("matrix: %v", namesOf(jobs))143
}144
if byName["matrix: [gcp, app]"].Variables["PROVIDER"] != "gcp" || byName["matrix: [aws, backup]"].Variables["STACK"] != "backup" {145
t.Fatalf("matrix vars")146
}147
if got := Expand("https://example.test/$CI_COMMIT_REF_SLUG", map[string]string{"CI_COMMIT_REF_SLUG": "master"}); got != "https://example.test/master" {148
t.Fatal(got)149
}150
}151
152
func TestGateCanceled(t *testing.T) {153
jobs := []Job{{Name: "a", Stage: "test"}, {Name: "b", Stage: "deploy"}}154
status := map[string]string{"a": "canceled", "b": "queued"}155
if got := Gate(jobs, status, jobs[1]); got != "skip" {156
t.Fatalf("canceled houdt de volgende tegen: %s", got)157
}158
jobs[0].AllowFailure = true159
if got := Gate(jobs, status, jobs[1]); got != "ready" {160
t.Fatalf("allow_failure canceled: %s", got)161
}162
}@@ -0,0 +1,278 @@
1
package ci
2
3
import (
4
"strconv"
5
"strings"
6
7
"gopkg.in/yaml.v3"
8
)
9
10
func parseWorkflow(node *yaml.Node) ([]Rule, []Issue) {
11
if node.Kind != yaml.MappingNode {
12
return nil, []Issue{{Line: node.Line, Key: "ci.err.map"}}
13
}
14
var rules []Rule
15
var issues []Issue
16
for i := 0; i < len(node.Content); i += 2 {
17
key := node.Content[i].Value
18
val := node.Content[i+1]
19
if key == "rules" {
20
parsed, iss := parseRules(val)
21
issues = append(issues, iss...)
22
rules = parsed
23
continue
24
}
25
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"workflow." + key}, Warning: true})
26
}
27
return rules, issues
28
}
29
30
func parseRetry(node *yaml.Node) (Retry, []Issue) {
31
switch node.Kind {
32
case yaml.ScalarNode:
33
n, iss := retryMax(node)
34
if len(iss) > 0 && n == 0 && iss[0].Key != "ci.warn.retryMax" {
35
return Retry{}, iss
36
}
37
return Retry{Max: n}, iss
38
case yaml.MappingNode:
39
var retry Retry
40
var issues []Issue
41
for i := 0; i < len(node.Content); i += 2 {
42
key := node.Content[i].Value
43
val := node.Content[i+1]
44
switch key {
45
case "max":
46
n, iss := retryMax(val)
47
issues = append(issues, iss...)
48
retry.Max = n
49
case "when":
50
list, iss := retryWhen(val)
51
issues = append(issues, iss...)
52
retry.When = list
53
default:
54
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"retry." + key}, Warning: true})
55
}
56
}
57
return retry, issues
58
default:
59
return Retry{}, []Issue{{Line: node.Line, Key: "ci.err.retry"}}
60
}
61
}
62
63
func retryMax(node *yaml.Node) (int, []Issue) {
64
if node.Kind != yaml.ScalarNode {
65
return 0, []Issue{{Line: node.Line, Key: "ci.err.retry"}}
66
}
67
n, err := strconv.Atoi(strings.TrimSpace(node.Value))
68
if err != nil || n < 0 {
69
return 0, []Issue{{Line: node.Line, Key: "ci.err.retry"}}
70
}
71
if n > 2 {
72
return 2, []Issue{{Line: node.Line, Key: "ci.warn.retryMax", Args: []any{n, 2}, Warning: true}}
73
}
74
return n, nil
75
}
76
77
func retryWhen(node *yaml.Node) ([]string, []Issue) {
78
if node.Kind == yaml.ScalarNode {
79
if strings.TrimSpace(node.Value) == "" {
80
return nil, []Issue{{Line: node.Line, Key: "ci.err.retry"}}
81
}
82
return []string{node.Value}, nil
83
}
84
return scalarList(node, "ci.err.list")
85
}
86
87
func parseParallel(node *yaml.Node) (int, []MatrixSpec, []Issue) {
88
if node.Kind == yaml.ScalarNode {
89
n, err := strconv.Atoi(strings.TrimSpace(node.Value))
90
if err != nil || n < 1 {
91
return 0, nil, []Issue{{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}}}
92
}
93
if n > maxParallel {
94
return 0, nil, []Issue{{Line: node.Line, Key: "ci.err.parallel", Args: []any{n, maxParallel}}}
95
}
96
return n, nil, nil
97
}
98
if node.Kind != yaml.MappingNode {
99
return 0, nil, []Issue{{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}}}
100
}
101
var specs []MatrixSpec
102
var issues []Issue
103
seen := false
104
for i := 0; i < len(node.Content); i += 2 {
105
key := node.Content[i].Value
106
val := node.Content[i+1]
107
if key != "matrix" {
108
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"parallel." + key}, Warning: true})
109
continue
110
}
111
seen = true
112
parsed, iss := parseMatrix(val)
113
issues = append(issues, iss...)
114
specs = parsed
115
}
116
if !seen {
117
issues = append(issues, Issue{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}})
118
}
119
count := 0
120
for _, spec := range specs {
121
count += len(cartesian(spec))
122
}
123
if count > maxParallel {
124
issues = append(issues, Issue{Line: node.Line, Key: "ci.err.parallel", Args: []any{count, maxParallel}})
125
}
126
return 0, specs, issues
127
}
128
129
func parseMatrix(node *yaml.Node) ([]MatrixSpec, []Issue) {
130
if node.Kind != yaml.SequenceNode || len(node.Content) == 0 {
131
return nil, []Issue{{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}}}
132
}
133
var specs []MatrixSpec
134
var issues []Issue
135
for _, item := range node.Content {
136
if item.Kind != yaml.MappingNode {
137
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}})
138
continue
139
}
140
spec := MatrixSpec{}
141
for i := 0; i < len(item.Content); i += 2 {
142
key := item.Content[i].Value
143
vals, iss := matrixValues(item.Content[i+1])
144
issues = append(issues, iss...)
145
if len(vals) == 0 {
146
continue
147
}
148
spec.Keys = append(spec.Keys, key)
149
spec.Vals = append(spec.Vals, vals)
150
}
151
if len(spec.Keys) == 0 {
152
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}})
153
continue
154
}
155
specs = append(specs, spec)
156
}
157
return specs, issues
158
}
159
160
func matrixValues(node *yaml.Node) ([]string, []Issue) {
161
if node.Kind == yaml.ScalarNode {
162
return []string{node.Value}, nil
163
}
164
return scalarList(node, "ci.err.list")
165
}
166
167
func parseServices(node *yaml.Node) ([]Service, []Issue) {
168
var items []*yaml.Node
169
switch node.Kind {
170
case yaml.SequenceNode:
171
items = node.Content
172
default:
173
items = []*yaml.Node{node}
174
}
175
var out []Service
176
var issues []Issue
177
for _, item := range items {
178
switch item.Kind {
179
case yaml.ScalarNode:
180
if strings.TrimSpace(item.Value) == "" {
181
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.service"})
182
continue
183
}
184
out = append(out, Service{Name: item.Value, Alias: serviceAlias(item.Value)})
185
case yaml.MappingNode:
186
svc := Service{Variables: map[string]string{}}
187
for i := 0; i < len(item.Content); i += 2 {
188
key := item.Content[i].Value
189
val := item.Content[i+1]
190
switch key {
191
case "name":
192
if val.Kind == yaml.ScalarNode {
193
svc.Name = val.Value
194
}
195
case "alias":
196
if val.Kind == yaml.ScalarNode {
197
svc.Alias = val.Value
198
}
199
case "command":
200
list, iss := scriptList(val)
201
issues = append(issues, iss...)
202
svc.Command = list
203
case "entrypoint":
204
list, iss := scriptList(val)
205
issues = append(issues, iss...)
206
svc.Entrypoint = list
207
case "variables":
208
vars, iss := scalarMap(val)
209
issues = append(issues, iss...)
210
svc.Variables = vars
211
default:
212
issues = append(issues, Issue{Line: item.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"services." + key}, Warning: true})
213
}
214
}
215
if strings.TrimSpace(svc.Name) == "" {
216
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.service"})
217
continue
218
}
219
if svc.Alias == "" {
220
svc.Alias = serviceAlias(svc.Name)
221
}
222
out = append(out, svc)
223
default:
224
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.service"})
225
}
226
}
227
return out, issues
228
}
229
230
func serviceAlias(image string) string {
231
name := image
232
if i := strings.LastIndex(name, "/"); i >= 0 {
233
name = name[i+1:]
234
}
235
if i := strings.IndexAny(name, ":@"); i >= 0 {
236
name = name[:i]
237
}
238
name = strings.TrimSpace(name)
239
if name == "" {
240
return "service"
241
}
242
return name
243
}
244
245
func parseEnvironment(node *yaml.Node) (Environment, []Issue) {
246
switch node.Kind {
247
case yaml.ScalarNode:
248
if strings.TrimSpace(node.Value) == "" {
249
return Environment{}, []Issue{{Line: node.Line, Key: "ci.err.environment"}}
250
}
251
return Environment{Name: node.Value}, nil
252
case yaml.MappingNode:
253
var env Environment
254
var issues []Issue
255
for i := 0; i < len(node.Content); i += 2 {
256
key := node.Content[i].Value
257
val := node.Content[i+1]
258
switch key {
259
case "name":
260
if val.Kind == yaml.ScalarNode {
261
env.Name = val.Value
262
}
263
case "url":
264
if val.Kind == yaml.ScalarNode {
265
env.URL = val.Value
266
}
267
default:
268
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"environment." + key}, Warning: true})
269
}
270
}
271
if strings.TrimSpace(env.Name) == "" {
272
issues = append(issues, Issue{Line: node.Line, Key: "ci.err.environment"})
273
}
274
return env, issues
275
default:
276
return Environment{}, []Issue{{Line: node.Line, Key: "ci.err.environment"}}
277
}
278
}
@@ -0,0 +1,278 @@1
package ci2
3
import (4
"strconv"5
"strings"6
7
"gopkg.in/yaml.v3"8
)9
10
func parseWorkflow(node *yaml.Node) ([]Rule, []Issue) {11
if node.Kind != yaml.MappingNode {12
return nil, []Issue{{Line: node.Line, Key: "ci.err.map"}}13
}14
var rules []Rule15
var issues []Issue16
for i := 0; i < len(node.Content); i += 2 {17
key := node.Content[i].Value18
val := node.Content[i+1]19
if key == "rules" {20
parsed, iss := parseRules(val)21
issues = append(issues, iss...)22
rules = parsed23
continue24
}25
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"workflow." + key}, Warning: true})26
}27
return rules, issues28
}29
30
func parseRetry(node *yaml.Node) (Retry, []Issue) {31
switch node.Kind {32
case yaml.ScalarNode:33
n, iss := retryMax(node)34
if len(iss) > 0 && n == 0 && iss[0].Key != "ci.warn.retryMax" {35
return Retry{}, iss36
}37
return Retry{Max: n}, iss38
case yaml.MappingNode:39
var retry Retry40
var issues []Issue41
for i := 0; i < len(node.Content); i += 2 {42
key := node.Content[i].Value43
val := node.Content[i+1]44
switch key {45
case "max":46
n, iss := retryMax(val)47
issues = append(issues, iss...)48
retry.Max = n49
case "when":50
list, iss := retryWhen(val)51
issues = append(issues, iss...)52
retry.When = list53
default:54
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"retry." + key}, Warning: true})55
}56
}57
return retry, issues58
default:59
return Retry{}, []Issue{{Line: node.Line, Key: "ci.err.retry"}}60
}61
}62
63
func retryMax(node *yaml.Node) (int, []Issue) {64
if node.Kind != yaml.ScalarNode {65
return 0, []Issue{{Line: node.Line, Key: "ci.err.retry"}}66
}67
n, err := strconv.Atoi(strings.TrimSpace(node.Value))68
if err != nil || n < 0 {69
return 0, []Issue{{Line: node.Line, Key: "ci.err.retry"}}70
}71
if n > 2 {72
return 2, []Issue{{Line: node.Line, Key: "ci.warn.retryMax", Args: []any{n, 2}, Warning: true}}73
}74
return n, nil75
}76
77
func retryWhen(node *yaml.Node) ([]string, []Issue) {78
if node.Kind == yaml.ScalarNode {79
if strings.TrimSpace(node.Value) == "" {80
return nil, []Issue{{Line: node.Line, Key: "ci.err.retry"}}81
}82
return []string{node.Value}, nil83
}84
return scalarList(node, "ci.err.list")85
}86
87
func parseParallel(node *yaml.Node) (int, []MatrixSpec, []Issue) {88
if node.Kind == yaml.ScalarNode {89
n, err := strconv.Atoi(strings.TrimSpace(node.Value))90
if err != nil || n < 1 {91
return 0, nil, []Issue{{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}}}92
}93
if n > maxParallel {94
return 0, nil, []Issue{{Line: node.Line, Key: "ci.err.parallel", Args: []any{n, maxParallel}}}95
}96
return n, nil, nil97
}98
if node.Kind != yaml.MappingNode {99
return 0, nil, []Issue{{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}}}100
}101
var specs []MatrixSpec102
var issues []Issue103
seen := false104
for i := 0; i < len(node.Content); i += 2 {105
key := node.Content[i].Value106
val := node.Content[i+1]107
if key != "matrix" {108
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"parallel." + key}, Warning: true})109
continue110
}111
seen = true112
parsed, iss := parseMatrix(val)113
issues = append(issues, iss...)114
specs = parsed115
}116
if !seen {117
issues = append(issues, Issue{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}})118
}119
count := 0120
for _, spec := range specs {121
count += len(cartesian(spec))122
}123
if count > maxParallel {124
issues = append(issues, Issue{Line: node.Line, Key: "ci.err.parallel", Args: []any{count, maxParallel}})125
}126
return 0, specs, issues127
}128
129
func parseMatrix(node *yaml.Node) ([]MatrixSpec, []Issue) {130
if node.Kind != yaml.SequenceNode || len(node.Content) == 0 {131
return nil, []Issue{{Line: node.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}}}132
}133
var specs []MatrixSpec134
var issues []Issue135
for _, item := range node.Content {136
if item.Kind != yaml.MappingNode {137
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}})138
continue139
}140
spec := MatrixSpec{}141
for i := 0; i < len(item.Content); i += 2 {142
key := item.Content[i].Value143
vals, iss := matrixValues(item.Content[i+1])144
issues = append(issues, iss...)145
if len(vals) == 0 {146
continue147
}148
spec.Keys = append(spec.Keys, key)149
spec.Vals = append(spec.Vals, vals)150
}151
if len(spec.Keys) == 0 {152
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.parallelForm", Args: []any{maxParallel}})153
continue154
}155
specs = append(specs, spec)156
}157
return specs, issues158
}159
160
func matrixValues(node *yaml.Node) ([]string, []Issue) {161
if node.Kind == yaml.ScalarNode {162
return []string{node.Value}, nil163
}164
return scalarList(node, "ci.err.list")165
}166
167
func parseServices(node *yaml.Node) ([]Service, []Issue) {168
var items []*yaml.Node169
switch node.Kind {170
case yaml.SequenceNode:171
items = node.Content172
default:173
items = []*yaml.Node{node}174
}175
var out []Service176
var issues []Issue177
for _, item := range items {178
switch item.Kind {179
case yaml.ScalarNode:180
if strings.TrimSpace(item.Value) == "" {181
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.service"})182
continue183
}184
out = append(out, Service{Name: item.Value, Alias: serviceAlias(item.Value)})185
case yaml.MappingNode:186
svc := Service{Variables: map[string]string{}}187
for i := 0; i < len(item.Content); i += 2 {188
key := item.Content[i].Value189
val := item.Content[i+1]190
switch key {191
case "name":192
if val.Kind == yaml.ScalarNode {193
svc.Name = val.Value194
}195
case "alias":196
if val.Kind == yaml.ScalarNode {197
svc.Alias = val.Value198
}199
case "command":200
list, iss := scriptList(val)201
issues = append(issues, iss...)202
svc.Command = list203
case "entrypoint":204
list, iss := scriptList(val)205
issues = append(issues, iss...)206
svc.Entrypoint = list207
case "variables":208
vars, iss := scalarMap(val)209
issues = append(issues, iss...)210
svc.Variables = vars211
default:212
issues = append(issues, Issue{Line: item.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"services." + key}, Warning: true})213
}214
}215
if strings.TrimSpace(svc.Name) == "" {216
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.service"})217
continue218
}219
if svc.Alias == "" {220
svc.Alias = serviceAlias(svc.Name)221
}222
out = append(out, svc)223
default:224
issues = append(issues, Issue{Line: item.Line, Key: "ci.err.service"})225
}226
}227
return out, issues228
}229
230
func serviceAlias(image string) string {231
name := image232
if i := strings.LastIndex(name, "/"); i >= 0 {233
name = name[i+1:]234
}235
if i := strings.IndexAny(name, ":@"); i >= 0 {236
name = name[:i]237
}238
name = strings.TrimSpace(name)239
if name == "" {240
return "service"241
}242
return name243
}244
245
func parseEnvironment(node *yaml.Node) (Environment, []Issue) {246
switch node.Kind {247
case yaml.ScalarNode:248
if strings.TrimSpace(node.Value) == "" {249
return Environment{}, []Issue{{Line: node.Line, Key: "ci.err.environment"}}250
}251
return Environment{Name: node.Value}, nil252
case yaml.MappingNode:253
var env Environment254
var issues []Issue255
for i := 0; i < len(node.Content); i += 2 {256
key := node.Content[i].Value257
val := node.Content[i+1]258
switch key {259
case "name":260
if val.Kind == yaml.ScalarNode {261
env.Name = val.Value262
}263
case "url":264
if val.Kind == yaml.ScalarNode {265
env.URL = val.Value266
}267
default:268
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{"environment." + key}, Warning: true})269
}270
}271
if strings.TrimSpace(env.Name) == "" {272
issues = append(issues, Issue{Line: node.Line, Key: "ci.err.environment"})273
}274
return env, issues275
default:276
return Environment{}, []Issue{{Line: node.Line, Key: "ci.err.environment"}}277
}278
}@@ -14,6 +14,7 @@ type Pipeline struct {
14
14
Tags []string
15
15
Jobs []Job
16
16
Warnings []Issue
17
Workflow []Rule
17
18
}
18
19
19
20
// Job is één containerstap.
@@ -38,6 +39,40 @@ type Job struct {
38
39
ResourceGroup string
39
40
Only []string
40
41
Except []string
42
Retry Retry
43
Parallel int
44
Matrix []MatrixSpec
45
Services []Service
46
Environment Environment
47
Interruptible bool
48
}
49
50
// Retry is het aantal extra pogingen na een mislukking. Max is hoogstens 2.
51
// Een lege When betekent elke mislukking behalve een annulering.
52
type Retry struct {
53
Max int
54
When []string
55
}
56
57
// MatrixSpec is één item uit parallel:matrix. Keys staan in YAML-volgorde.
58
type MatrixSpec struct {
59
Keys []string
60
Vals [][]string
61
}
62
63
// Service is een sidecar-container op hetzelfde Docker-netwerk als de job.
64
type Service struct {
65
Name string
66
Alias string
67
Command []string
68
Entrypoint []string
69
Variables map[string]string
70
}
71
72
// Environment is de naam en een optionele URL. Er is geen deployment.
73
type Environment struct {
74
Name string
75
URL string
41
76
}
42
77
43
78
// Artifacts zijn paden die na een geslaagde job bewaard blijven.
@@ -14,6 +14,7 @@ type Pipeline struct {14
Tags []string14
Tags []string15
Jobs []Job15
Jobs []Job16
Warnings []Issue16
Warnings []Issue17
Workflow []Rule17
}18
}18
19
19
// Job is één containerstap.20
// Job is één containerstap.@@ -38,6 +39,40 @@ type Job struct {38
ResourceGroup string39
ResourceGroup string39
Only []string40
Only []string40
Except []string41
Except []string42
Retry Retry43
Parallel int44
Matrix []MatrixSpec45
Services []Service46
Environment Environment47
Interruptible bool48
}49
50
// Retry is het aantal extra pogingen na een mislukking. Max is hoogstens 2.51
// Een lege When betekent elke mislukking behalve een annulering.52
type Retry struct {53
Max int54
When []string55
}56
57
// MatrixSpec is één item uit parallel:matrix. Keys staan in YAML-volgorde.58
type MatrixSpec struct {59
Keys []string60
Vals [][]string61
}62
63
// Service is een sidecar-container op hetzelfde Docker-netwerk als de job.64
type Service struct {65
Name string66
Alias string67
Command []string68
Entrypoint []string69
Variables map[string]string70
}71
72
// Environment is de naam en een optionele URL. Er is geen deployment.73
type Environment struct {74
Name string75
URL string41
}76
}42
77
43
// Artifacts zijn paden die na een geslaagde job bewaard blijven.78
// Artifacts zijn paden die na een geslaagde job bewaard blijven.@@ -153,7 +153,11 @@ func parseBytes(data []byte, read func(string) ([]byte, error), stack []string)
153
153
d, iss := parseDefault(val)
154
154
issues = append(issues, iss...)
155
155
def = overlayDefault(def, d)
156
case "workflow", "services", "spec", "pages":
156
case "workflow":
157
rules, iss := parseWorkflow(val)
158
issues = append(issues, iss...)
159
p.Workflow = rules
160
case "services", "spec", "pages":
157
161
issues = append(issues, Issue{Line: keyNode.Line, Key: "ci.warn.ignored", Args: []any{key}, Warning: true})
158
162
case "jobs":
159
163
seenLegacy = true
@@ -347,6 +351,16 @@ func parseDefault(node *yaml.Node) (draft, []Issue) {
347
351
issues = append(issues, iss...)
348
352
d.job.Timeout = dur
349
353
d.present["timeout"] = true
354
case "retry":
355
retry, iss := parseRetry(val)
356
issues = append(issues, iss...)
357
d.job.Retry = retry
358
d.present["retry"] = true
359
case "interruptible":
360
b, iss := parseBool(val)
361
issues = append(issues, iss...)
362
d.job.Interruptible = b
363
d.present["interruptible"] = true
350
364
case "cache":
351
365
c, iss := parseCache(val)
352
366
issues = append(issues, iss...)
@@ -385,6 +399,14 @@ func overlayJob(base Job, d draft) Job {
385
399
set("resource_group", func() { base.ResourceGroup = d.job.ResourceGroup })
386
400
set("only", func() { base.Only = d.job.Only })
387
401
set("except", func() { base.Except = d.job.Except })
402
set("retry", func() { base.Retry = d.job.Retry })
403
set("parallel", func() {
404
base.Parallel = d.job.Parallel
405
base.Matrix = d.job.Matrix
406
})
407
set("services", func() { base.Services = d.job.Services })
408
set("environment", func() { base.Environment = d.job.Environment })
409
set("interruptible", func() { base.Interruptible = d.job.Interruptible })
388
410
if d.present["variables"] {
389
411
mergedVars := map[string]string{}
390
412
for k, v := range base.Variables {
@@ -420,6 +442,10 @@ func overlayDefault(base, extra draft) draft {
420
442
base.job.Timeout = extra.job.Timeout
421
443
case "cache":
422
444
base.job.Cache = extra.job.Cache
445
case "retry":
446
base.job.Retry = extra.job.Retry
447
case "interruptible":
448
base.job.Interruptible = extra.job.Interruptible
423
449
}
424
450
}
425
451
return base
@@ -530,6 +556,27 @@ func parseJob(name string, node *yaml.Node) (draft, []Issue) {
530
556
list, iss := extendsOf(val)
531
557
issues = append(issues, iss...)
532
558
d.extends = list
559
case "retry":
560
retry, iss := parseRetry(val)
561
issues = append(issues, iss...)
562
d.job.Retry = retry
563
case "parallel":
564
n, matrix, iss := parseParallel(val)
565
issues = append(issues, iss...)
566
d.job.Parallel = n
567
d.job.Matrix = matrix
568
case "services":
569
svcs, iss := parseServices(val)
570
issues = append(issues, iss...)
571
d.job.Services = svcs
572
case "environment":
573
env, iss := parseEnvironment(val)
574
issues = append(issues, iss...)
575
d.job.Environment = env
576
case "interruptible":
577
b, iss := parseBool(val)
578
issues = append(issues, iss...)
579
d.job.Interruptible = b
533
580
default:
534
581
if _, ok := reserved[key]; ok {
535
582
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{key}, Warning: true})
@@ -616,6 +663,14 @@ func inherit(child, parent *draft) {
616
663
copyField("resource_group", func() { child.job.ResourceGroup = parent.job.ResourceGroup })
617
664
copyField("only", func() { child.job.Only = parent.job.Only })
618
665
copyField("except", func() { child.job.Except = parent.job.Except })
666
copyField("retry", func() { child.job.Retry = parent.job.Retry })
667
copyField("parallel", func() {
668
child.job.Parallel = parent.job.Parallel
669
child.job.Matrix = parent.job.Matrix
670
})
671
copyField("services", func() { child.job.Services = parent.job.Services })
672
copyField("environment", func() { child.job.Environment = parent.job.Environment })
673
copyField("interruptible", func() { child.job.Interruptible = parent.job.Interruptible })
619
674
if len(parent.job.Variables) > 0 {
620
675
merged := map[string]string{}
621
676
for k, v := range parent.job.Variables {
@@ -835,7 +890,7 @@ func parseRules(node *yaml.Node) ([]Rule, []Issue) {
835
890
continue
836
891
}
837
892
r.If = strings.TrimSpace(val.Value)
838
if _, err := compileRule(r.If); err != nil {
893
if err := checkRule(r.If); err != nil {
839
894
if ie, ok := err.(i18n.Error); ok {
840
895
issues = append(issues, Issue{Line: val.Line, Key: ie.Key, Args: ie.Args})
841
896
} else {
@@ -153,7 +153,11 @@ func parseBytes(data []byte, read func(string) ([]byte, error), stack []string)153
d, iss := parseDefault(val)153
d, iss := parseDefault(val)154
issues = append(issues, iss...)154
issues = append(issues, iss...)155
def = overlayDefault(def, d)155
def = overlayDefault(def, d)156
case "workflow", "services", "spec", "pages":156
case "workflow":157
rules, iss := parseWorkflow(val)158
issues = append(issues, iss...)159
p.Workflow = rules160
case "services", "spec", "pages":157
issues = append(issues, Issue{Line: keyNode.Line, Key: "ci.warn.ignored", Args: []any{key}, Warning: true})161
issues = append(issues, Issue{Line: keyNode.Line, Key: "ci.warn.ignored", Args: []any{key}, Warning: true})158
case "jobs":162
case "jobs":159
seenLegacy = true163
seenLegacy = true@@ -347,6 +351,16 @@ func parseDefault(node *yaml.Node) (draft, []Issue) {347
issues = append(issues, iss...)351
issues = append(issues, iss...)348
d.job.Timeout = dur352
d.job.Timeout = dur349
d.present["timeout"] = true353
d.present["timeout"] = true354
case "retry":355
retry, iss := parseRetry(val)356
issues = append(issues, iss...)357
d.job.Retry = retry358
d.present["retry"] = true359
case "interruptible":360
b, iss := parseBool(val)361
issues = append(issues, iss...)362
d.job.Interruptible = b363
d.present["interruptible"] = true350
case "cache":364
case "cache":351
c, iss := parseCache(val)365
c, iss := parseCache(val)352
issues = append(issues, iss...)366
issues = append(issues, iss...)@@ -385,6 +399,14 @@ func overlayJob(base Job, d draft) Job {385
set("resource_group", func() { base.ResourceGroup = d.job.ResourceGroup })399
set("resource_group", func() { base.ResourceGroup = d.job.ResourceGroup })386
set("only", func() { base.Only = d.job.Only })400
set("only", func() { base.Only = d.job.Only })387
set("except", func() { base.Except = d.job.Except })401
set("except", func() { base.Except = d.job.Except })402
set("retry", func() { base.Retry = d.job.Retry })403
set("parallel", func() {404
base.Parallel = d.job.Parallel405
base.Matrix = d.job.Matrix406
})407
set("services", func() { base.Services = d.job.Services })408
set("environment", func() { base.Environment = d.job.Environment })409
set("interruptible", func() { base.Interruptible = d.job.Interruptible })388
if d.present["variables"] {410
if d.present["variables"] {389
mergedVars := map[string]string{}411
mergedVars := map[string]string{}390
for k, v := range base.Variables {412
for k, v := range base.Variables {@@ -420,6 +442,10 @@ func overlayDefault(base, extra draft) draft {420
base.job.Timeout = extra.job.Timeout442
base.job.Timeout = extra.job.Timeout421
case "cache":443
case "cache":422
base.job.Cache = extra.job.Cache444
base.job.Cache = extra.job.Cache445
case "retry":446
base.job.Retry = extra.job.Retry447
case "interruptible":448
base.job.Interruptible = extra.job.Interruptible423
}449
}424
}450
}425
return base451
return base@@ -530,6 +556,27 @@ func parseJob(name string, node *yaml.Node) (draft, []Issue) {530
list, iss := extendsOf(val)556
list, iss := extendsOf(val)531
issues = append(issues, iss...)557
issues = append(issues, iss...)532
d.extends = list558
d.extends = list559
case "retry":560
retry, iss := parseRetry(val)561
issues = append(issues, iss...)562
d.job.Retry = retry563
case "parallel":564
n, matrix, iss := parseParallel(val)565
issues = append(issues, iss...)566
d.job.Parallel = n567
d.job.Matrix = matrix568
case "services":569
svcs, iss := parseServices(val)570
issues = append(issues, iss...)571
d.job.Services = svcs572
case "environment":573
env, iss := parseEnvironment(val)574
issues = append(issues, iss...)575
d.job.Environment = env576
case "interruptible":577
b, iss := parseBool(val)578
issues = append(issues, iss...)579
d.job.Interruptible = b533
default:580
default:534
if _, ok := reserved[key]; ok {581
if _, ok := reserved[key]; ok {535
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{key}, Warning: true})582
issues = append(issues, Issue{Line: node.Content[i].Line, Key: "ci.warn.ignored", Args: []any{key}, Warning: true})@@ -616,6 +663,14 @@ func inherit(child, parent *draft) {616
copyField("resource_group", func() { child.job.ResourceGroup = parent.job.ResourceGroup })663
copyField("resource_group", func() { child.job.ResourceGroup = parent.job.ResourceGroup })617
copyField("only", func() { child.job.Only = parent.job.Only })664
copyField("only", func() { child.job.Only = parent.job.Only })618
copyField("except", func() { child.job.Except = parent.job.Except })665
copyField("except", func() { child.job.Except = parent.job.Except })666
copyField("retry", func() { child.job.Retry = parent.job.Retry })667
copyField("parallel", func() {668
child.job.Parallel = parent.job.Parallel669
child.job.Matrix = parent.job.Matrix670
})671
copyField("services", func() { child.job.Services = parent.job.Services })672
copyField("environment", func() { child.job.Environment = parent.job.Environment })673
copyField("interruptible", func() { child.job.Interruptible = parent.job.Interruptible })619
if len(parent.job.Variables) > 0 {674
if len(parent.job.Variables) > 0 {620
merged := map[string]string{}675
merged := map[string]string{}621
for k, v := range parent.job.Variables {676
for k, v := range parent.job.Variables {@@ -835,7 +890,7 @@ func parseRules(node *yaml.Node) ([]Rule, []Issue) {835
continue890
continue836
}891
}837
r.If = strings.TrimSpace(val.Value)892
r.If = strings.TrimSpace(val.Value)838
if _, err := compileRule(r.If); err != nil {893
if err := checkRule(r.If); err != nil {839
if ie, ok := err.(i18n.Error); ok {894
if ie, ok := err.(i18n.Error); ok {840
issues = append(issues, Issue{Line: val.Line, Key: ie.Key, Args: ie.Args})895
issues = append(issues, Issue{Line: val.Line, Key: ie.Key, Args: ie.Args})841
} else {896
} else {@@ -26,11 +26,12 @@ type Config struct {
26
26
TrustedProxies []string `yaml:"trusted_proxies"`
27
27
// BehindProxy is het oude veld. Het wordt geweigerd: één boolean
28
28
// vertrouwt ofwel iedereen, ofwel niemand. De lijst hierboven beslist.
29
BehindProxy bool `yaml:"behind_proxy"`
30
Auth Auth `yaml:"auth"`
31
Git Git `yaml:"git"`
32
SSH SSH `yaml:"ssh"`
33
CI CI `yaml:"ci"`
29
BehindProxy bool `yaml:"behind_proxy"`
30
Auth Auth `yaml:"auth"`
31
Git Git `yaml:"git"`
32
SSH SSH `yaml:"ssh"`
33
Runner Runner `yaml:"runner"`
34
CI CI `yaml:"ci"`
34
35
35
36
proxies *realip.Set
36
37
}
@@ -65,6 +66,11 @@ type SSH struct {
65
66
PublicPort int `yaml:"public_port"`
66
67
}
67
68
69
// Runner is de ingebouwde Docker-runner.
70
type Runner struct {
71
Name string `yaml:"name"`
72
}
73
68
74
// CI stuurt de Docker-runner.
69
75
type CI struct {
70
76
Enabled bool `yaml:"enabled"`
@@ -156,6 +162,11 @@ func (c *Config) applyDefaults() {
156
162
if c.CI.TimeoutSec <= 0 {
157
163
c.CI.TimeoutSec = 1800
158
164
}
165
if strings.TrimSpace(c.Runner.Name) == "" {
166
c.Runner.Name = "Steve"
167
} else {
168
c.Runner.Name = strings.TrimSpace(c.Runner.Name)
169
}
159
170
c.Auth.BaseURL = strings.TrimRight(c.Auth.BaseURL, "/")
160
171
c.BaseURL = strings.TrimRight(c.BaseURL, "/")
161
172
}
@@ -301,6 +312,9 @@ ssh:
301
312
public_host: git.timfalken.com
302
313
public_port: 22
303
314
315
runner:
316
name: Steve
317
304
318
ci:
305
319
enabled: true
306
320
docker_binary: docker
@@ -26,11 +26,12 @@ type Config struct {26
TrustedProxies []string `yaml:"trusted_proxies"`26
TrustedProxies []string `yaml:"trusted_proxies"`27
// BehindProxy is het oude veld. Het wordt geweigerd: één boolean27
// BehindProxy is het oude veld. Het wordt geweigerd: één boolean28
// vertrouwt ofwel iedereen, ofwel niemand. De lijst hierboven beslist.28
// vertrouwt ofwel iedereen, ofwel niemand. De lijst hierboven beslist.29
BehindProxy bool `yaml:"behind_proxy"`29
BehindProxy bool `yaml:"behind_proxy"`30
Auth Auth `yaml:"auth"`30
Auth Auth `yaml:"auth"`31
Git Git `yaml:"git"`31
Git Git `yaml:"git"`32
SSH SSH `yaml:"ssh"`32
SSH SSH `yaml:"ssh"`33
CI CI `yaml:"ci"`33
Runner Runner `yaml:"runner"`34
CI CI `yaml:"ci"`34
35
35
proxies *realip.Set36
proxies *realip.Set36
}37
}@@ -65,6 +66,11 @@ type SSH struct {65
PublicPort int `yaml:"public_port"`66
PublicPort int `yaml:"public_port"`66
}67
}67
68
69
// Runner is de ingebouwde Docker-runner.70
type Runner struct {71
Name string `yaml:"name"`72
}73
68
// CI stuurt de Docker-runner.74
// CI stuurt de Docker-runner.69
type CI struct {75
type CI struct {70
Enabled bool `yaml:"enabled"`76
Enabled bool `yaml:"enabled"`@@ -156,6 +162,11 @@ func (c *Config) applyDefaults() {156
if c.CI.TimeoutSec <= 0 {162
if c.CI.TimeoutSec <= 0 {157
c.CI.TimeoutSec = 1800163
c.CI.TimeoutSec = 1800158
}164
}165
if strings.TrimSpace(c.Runner.Name) == "" {166
c.Runner.Name = "Steve"167
} else {168
c.Runner.Name = strings.TrimSpace(c.Runner.Name)169
}159
c.Auth.BaseURL = strings.TrimRight(c.Auth.BaseURL, "/")170
c.Auth.BaseURL = strings.TrimRight(c.Auth.BaseURL, "/")160
c.BaseURL = strings.TrimRight(c.BaseURL, "/")171
c.BaseURL = strings.TrimRight(c.BaseURL, "/")161
}172
}@@ -301,6 +312,9 @@ ssh:301
public_host: git.timfalken.com312
public_host: git.timfalken.com302
public_port: 22313
public_port: 22303
314
315
runner:316
name: Steve317
304
ci:318
ci:305
enabled: true319
enabled: true306
docker_binary: docker320
docker_binary: docker@@ -7,6 +7,34 @@ import (
7
7
"testing"
8
8
)
9
9
10
func TestRunnerName(t *testing.T) {
11
dir := t.TempDir()
12
path := filepath.Join(dir, "config.yml")
13
body := []byte("base_url: https://git.example.test\nauth:\n mode: oauth\n client_id: git.example.test\n")
14
if err := os.WriteFile(path, body, 0o600); err != nil {
15
t.Fatal(err)
16
}
17
t.Setenv("BLUEGIT_AUTH_CLIENT_SECRET", "geheim")
18
cfg, err := Load(path)
19
if err != nil {
20
t.Fatal(err)
21
}
22
if cfg.Runner.Name != "Steve" {
23
t.Fatalf("standaard %q", cfg.Runner.Name)
24
}
25
body = []byte("base_url: https://git.example.test\nrunner:\n name: \" Ada \"\nauth:\n mode: oauth\n client_id: git.example.test\n")
26
if err := os.WriteFile(path, body, 0o600); err != nil {
27
t.Fatal(err)
28
}
29
cfg, err = Load(path)
30
if err != nil {
31
t.Fatal(err)
32
}
33
if cfg.Runner.Name != "Ada" {
34
t.Fatalf("naam %q", cfg.Runner.Name)
35
}
36
}
37
10
38
func TestStoragePathOptional(t *testing.T) {
11
39
dir := t.TempDir()
12
40
path := filepath.Join(dir, "config.yml")
@@ -7,6 +7,34 @@ import (7
"testing"7
"testing"8
)8
)9
9
10
func TestRunnerName(t *testing.T) {11
dir := t.TempDir()12
path := filepath.Join(dir, "config.yml")13
body := []byte("base_url: https://git.example.test\nauth:\n mode: oauth\n client_id: git.example.test\n")14
if err := os.WriteFile(path, body, 0o600); err != nil {15
t.Fatal(err)16
}17
t.Setenv("BLUEGIT_AUTH_CLIENT_SECRET", "geheim")18
cfg, err := Load(path)19
if err != nil {20
t.Fatal(err)21
}22
if cfg.Runner.Name != "Steve" {23
t.Fatalf("standaard %q", cfg.Runner.Name)24
}25
body = []byte("base_url: https://git.example.test\nrunner:\n name: \" Ada \"\nauth:\n mode: oauth\n client_id: git.example.test\n")26
if err := os.WriteFile(path, body, 0o600); err != nil {27
t.Fatal(err)28
}29
cfg, err = Load(path)30
if err != nil {31
t.Fatal(err)32
}33
if cfg.Runner.Name != "Ada" {34
t.Fatalf("naam %q", cfg.Runner.Name)35
}36
}37
10
func TestStoragePathOptional(t *testing.T) {38
func TestStoragePathOptional(t *testing.T) {11
dir := t.TempDir()39
dir := t.TempDir()12
path := filepath.Join(dir, "config.yml")40
path := filepath.Join(dir, "config.yml")@@ -207,14 +207,16 @@ var en = map[string]string{
207
207
"pipe.pending": "Pending",
208
208
"pipe.manual": "Manual",
209
209
"pipe.skipped": "Skipped",
210
"pipe.canceled": "Canceled",
210
211
"pipe.warning": "Warning",
211
212
"pipe.stage": "Stage",
212
213
"pipe.duration": "Duration",
213
214
"pipe.finished": "Finished",
214
215
"pipe.queued": "Queued",
215
216
"pipe.timeout": "Timeout",
217
"pipe.environment": "Environment",
216
218
"pipe.runner": "Runner",
217
"pipe.runnerHere": "bluegit #1",
219
"pipe.runningOn": "Running on %s",
218
220
"pipe.source": "Source",
219
221
"pipe.commit": "Commit",
220
222
"pipe.for": "for",
@@ -562,6 +564,12 @@ var en = map[string]string{
562
564
"ci.err.ruleExpr": "rule %q is not understood",
563
565
"ci.err.ruleCombine": "rule %q cannot combine conditions with && or ||",
564
566
"ci.err.regex": "invalid regex in %q: %s",
567
"ci.err.parallel": "parallel creates %d jobs; the maximum is %d",
568
"ci.err.parallelForm": "parallel must be a number from 1 to %d, or a matrix",
569
"ci.err.retry": "retry must be a number from 0 to 2, or max and when",
570
"ci.err.service": "a service needs an image name",
571
"ci.err.environment": "environment needs a name",
572
"ci.warn.retryMax": "retry max %d is limited to %d",
565
573
"ci.err.when": "when %q is not supported",
566
574
"ci.err.timeout": "timeout %q is invalid",
567
575
"ci.err.extendsList": "extends must be a job name or a list",
@@ -207,14 +207,16 @@ var en = map[string]string{207
"pipe.pending": "Pending",207
"pipe.pending": "Pending",208
"pipe.manual": "Manual",208
"pipe.manual": "Manual",209
"pipe.skipped": "Skipped",209
"pipe.skipped": "Skipped",210
"pipe.canceled": "Canceled",210
"pipe.warning": "Warning",211
"pipe.warning": "Warning",211
"pipe.stage": "Stage",212
"pipe.stage": "Stage",212
"pipe.duration": "Duration",213
"pipe.duration": "Duration",213
"pipe.finished": "Finished",214
"pipe.finished": "Finished",214
"pipe.queued": "Queued",215
"pipe.queued": "Queued",215
"pipe.timeout": "Timeout",216
"pipe.timeout": "Timeout",217
"pipe.environment": "Environment",216
"pipe.runner": "Runner",218
"pipe.runner": "Runner",217
"pipe.runnerHere": "bluegit #1",219
"pipe.runningOn": "Running on %s",218
"pipe.source": "Source",220
"pipe.source": "Source",219
"pipe.commit": "Commit",221
"pipe.commit": "Commit",220
"pipe.for": "for",222
"pipe.for": "for",@@ -562,6 +564,12 @@ var en = map[string]string{562
"ci.err.ruleExpr": "rule %q is not understood",564
"ci.err.ruleExpr": "rule %q is not understood",563
"ci.err.ruleCombine": "rule %q cannot combine conditions with && or ||",565
"ci.err.ruleCombine": "rule %q cannot combine conditions with && or ||",564
"ci.err.regex": "invalid regex in %q: %s",566
"ci.err.regex": "invalid regex in %q: %s",567
"ci.err.parallel": "parallel creates %d jobs; the maximum is %d",568
"ci.err.parallelForm": "parallel must be a number from 1 to %d, or a matrix",569
"ci.err.retry": "retry must be a number from 0 to 2, or max and when",570
"ci.err.service": "a service needs an image name",571
"ci.err.environment": "environment needs a name",572
"ci.warn.retryMax": "retry max %d is limited to %d",565
"ci.err.when": "when %q is not supported",573
"ci.err.when": "when %q is not supported",566
"ci.err.timeout": "timeout %q is invalid",574
"ci.err.timeout": "timeout %q is invalid",567
"ci.err.extendsList": "extends must be a job name or a list",575
"ci.err.extendsList": "extends must be a job name or a list",@@ -214,14 +214,16 @@ var nl = map[string]string{
214
214
"pipe.pending": "Wachtend",
215
215
"pipe.manual": "Handmatig",
216
216
"pipe.skipped": "Overgeslagen",
217
"pipe.canceled": "Geannuleerd",
217
218
"pipe.warning": "Waarschuwing",
218
219
"pipe.stage": "Stage",
219
220
"pipe.duration": "Duur",
220
221
"pipe.finished": "Klaar",
221
222
"pipe.queued": "Wachtrij",
222
223
"pipe.timeout": "Timeout",
224
"pipe.environment": "Omgeving",
223
225
"pipe.runner": "Runner",
224
"pipe.runnerHere": "bluegit #1",
226
"pipe.runningOn": "Draait op %s",
225
227
"pipe.source": "Bron",
226
228
"pipe.commit": "Commit",
227
229
"pipe.for": "voor",
@@ -557,6 +559,12 @@ var nl = map[string]string{
557
559
"ci.err.ruleExpr": "rule %q wordt niet begrepen",
558
560
"ci.err.ruleCombine": "rule %q kan geen voorwaarden combineren met && of ||",
559
561
"ci.err.regex": "ongeldige regex in %q: %s",
562
"ci.err.parallel": "parallel maakt %d jobs; het maximum is %d",
563
"ci.err.parallelForm": "parallel is een getal van 1 tot %d, of een matrix",
564
"ci.err.retry": "retry is een getal van 0 tot 2, of max en when",
565
"ci.err.service": "een service heeft een imagenaam nodig",
566
"ci.err.environment": "environment heeft een naam nodig",
567
"ci.warn.retryMax": "retry-maximum %d is beperkt tot %d",
560
568
"ci.err.when": "when %q wordt niet ondersteund",
561
569
"ci.err.timeout": "timeout %q is ongeldig",
562
570
"ci.err.extendsList": "extends moet een jobnaam of een lijst zijn",
@@ -780,14 +788,16 @@ var de = map[string]string{
780
788
"pipe.pending": "Wartend",
781
789
"pipe.manual": "Manuell",
782
790
"pipe.skipped": "Übersprungen",
791
"pipe.canceled": "Abgebrochen",
783
792
"pipe.warning": "Warnung",
784
793
"pipe.stage": "Stage",
785
794
"pipe.duration": "Dauer",
786
795
"pipe.finished": "Beendet",
787
796
"pipe.queued": "Warteschlange",
788
797
"pipe.timeout": "Timeout",
798
"pipe.environment": "Umgebung",
789
799
"pipe.runner": "Runner",
790
"pipe.runnerHere": "bluegit #1",
800
"pipe.runningOn": "Läuft auf %s",
791
801
"pipe.source": "Quelle",
792
802
"pipe.commit": "Commit",
793
803
"pipe.for": "für",
@@ -1123,6 +1133,12 @@ var de = map[string]string{
1123
1133
"ci.err.ruleExpr": "Rule %q wird nicht verstanden",
1124
1134
"ci.err.ruleCombine": "Rule %q kann Bedingungen nicht mit && oder || verbinden",
1125
1135
"ci.err.regex": "ungültiger regulärer Ausdruck in %q: %s",
1136
"ci.err.parallel": "parallel erzeugt %d Jobs; das Maximum ist %d",
1137
"ci.err.parallelForm": "parallel ist eine Zahl von 1 bis %d, oder eine Matrix",
1138
"ci.err.retry": "retry ist eine Zahl von 0 bis 2, oder max und when",
1139
"ci.err.service": "ein Service braucht einen Image-Namen",
1140
"ci.err.environment": "environment braucht einen Namen",
1141
"ci.warn.retryMax": "retry-Maximum %d ist auf %d begrenzt",
1126
1142
"ci.err.when": "when %q wird nicht unterstützt",
1127
1143
"ci.err.timeout": "timeout %q ist ungültig",
1128
1144
"ci.err.extendsList": "extends muss ein Jobname oder eine Liste sein",
@@ -1346,14 +1362,16 @@ var fr = map[string]string{
1346
1362
"pipe.pending": "En attente",
1347
1363
"pipe.manual": "Manuel",
1348
1364
"pipe.skipped": "Ignoré",
1365
"pipe.canceled": "Annulé",
1349
1366
"pipe.warning": "Avertissement",
1350
1367
"pipe.stage": "Stage",
1351
1368
"pipe.duration": "Durée",
1352
1369
"pipe.finished": "Terminé",
1353
1370
"pipe.queued": "File",
1354
1371
"pipe.timeout": "Délai",
1372
"pipe.environment": "Environnement",
1355
1373
"pipe.runner": "Runner",
1356
"pipe.runnerHere": "bluegit #1",
1374
"pipe.runningOn": "S'exécute sur %s",
1357
1375
"pipe.source": "Source",
1358
1376
"pipe.commit": "Commit",
1359
1377
"pipe.for": "pour",
@@ -1689,6 +1707,12 @@ var fr = map[string]string{
1689
1707
"ci.err.ruleExpr": "la règle %q n'est pas comprise",
1690
1708
"ci.err.ruleCombine": "la règle %q ne peut pas combiner des conditions avec && ou ||",
1691
1709
"ci.err.regex": "expression régulière invalide dans %q : %s",
1710
"ci.err.parallel": "parallel crée %d jobs ; le maximum est %d",
1711
"ci.err.parallelForm": "parallel est un nombre de 1 à %d, ou une matrice",
1712
"ci.err.retry": "retry est un nombre de 0 à 2, ou max et when",
1713
"ci.err.service": "un service a besoin d'un nom d'image",
1714
"ci.err.environment": "environment a besoin d'un nom",
1715
"ci.warn.retryMax": "le maximum de retry %d est limité à %d",
1692
1716
"ci.err.when": "when %q n'est pas pris en charge",
1693
1717
"ci.err.timeout": "timeout %q est invalide",
1694
1718
"ci.err.extendsList": "extends doit être un nom de job ou une liste",
@@ -214,14 +214,16 @@ var nl = map[string]string{214
"pipe.pending": "Wachtend",214
"pipe.pending": "Wachtend",215
"pipe.manual": "Handmatig",215
"pipe.manual": "Handmatig",216
"pipe.skipped": "Overgeslagen",216
"pipe.skipped": "Overgeslagen",217
"pipe.canceled": "Geannuleerd",217
"pipe.warning": "Waarschuwing",218
"pipe.warning": "Waarschuwing",218
"pipe.stage": "Stage",219
"pipe.stage": "Stage",219
"pipe.duration": "Duur",220
"pipe.duration": "Duur",220
"pipe.finished": "Klaar",221
"pipe.finished": "Klaar",221
"pipe.queued": "Wachtrij",222
"pipe.queued": "Wachtrij",222
"pipe.timeout": "Timeout",223
"pipe.timeout": "Timeout",224
"pipe.environment": "Omgeving",223
"pipe.runner": "Runner",225
"pipe.runner": "Runner",224
"pipe.runnerHere": "bluegit #1",226
"pipe.runningOn": "Draait op %s",225
"pipe.source": "Bron",227
"pipe.source": "Bron",226
"pipe.commit": "Commit",228
"pipe.commit": "Commit",227
"pipe.for": "voor",229
"pipe.for": "voor",@@ -557,6 +559,12 @@ var nl = map[string]string{557
"ci.err.ruleExpr": "rule %q wordt niet begrepen",559
"ci.err.ruleExpr": "rule %q wordt niet begrepen",558
"ci.err.ruleCombine": "rule %q kan geen voorwaarden combineren met && of ||",560
"ci.err.ruleCombine": "rule %q kan geen voorwaarden combineren met && of ||",559
"ci.err.regex": "ongeldige regex in %q: %s",561
"ci.err.regex": "ongeldige regex in %q: %s",562
"ci.err.parallel": "parallel maakt %d jobs; het maximum is %d",563
"ci.err.parallelForm": "parallel is een getal van 1 tot %d, of een matrix",564
"ci.err.retry": "retry is een getal van 0 tot 2, of max en when",565
"ci.err.service": "een service heeft een imagenaam nodig",566
"ci.err.environment": "environment heeft een naam nodig",567
"ci.warn.retryMax": "retry-maximum %d is beperkt tot %d",560
"ci.err.when": "when %q wordt niet ondersteund",568
"ci.err.when": "when %q wordt niet ondersteund",561
"ci.err.timeout": "timeout %q is ongeldig",569
"ci.err.timeout": "timeout %q is ongeldig",562
"ci.err.extendsList": "extends moet een jobnaam of een lijst zijn",570
"ci.err.extendsList": "extends moet een jobnaam of een lijst zijn",@@ -780,14 +788,16 @@ var de = map[string]string{780
"pipe.pending": "Wartend",788
"pipe.pending": "Wartend",781
"pipe.manual": "Manuell",789
"pipe.manual": "Manuell",782
"pipe.skipped": "Übersprungen",790
"pipe.skipped": "Übersprungen",791
"pipe.canceled": "Abgebrochen",783
"pipe.warning": "Warnung",792
"pipe.warning": "Warnung",784
"pipe.stage": "Stage",793
"pipe.stage": "Stage",785
"pipe.duration": "Dauer",794
"pipe.duration": "Dauer",786
"pipe.finished": "Beendet",795
"pipe.finished": "Beendet",787
"pipe.queued": "Warteschlange",796
"pipe.queued": "Warteschlange",788
"pipe.timeout": "Timeout",797
"pipe.timeout": "Timeout",798
"pipe.environment": "Umgebung",789
"pipe.runner": "Runner",799
"pipe.runner": "Runner",790
"pipe.runnerHere": "bluegit #1",800
"pipe.runningOn": "Läuft auf %s",791
"pipe.source": "Quelle",801
"pipe.source": "Quelle",792
"pipe.commit": "Commit",802
"pipe.commit": "Commit",793
"pipe.for": "für",803
"pipe.for": "für",@@ -1123,6 +1133,12 @@ var de = map[string]string{1123
"ci.err.ruleExpr": "Rule %q wird nicht verstanden",1133
"ci.err.ruleExpr": "Rule %q wird nicht verstanden",1124
"ci.err.ruleCombine": "Rule %q kann Bedingungen nicht mit && oder || verbinden",1134
"ci.err.ruleCombine": "Rule %q kann Bedingungen nicht mit && oder || verbinden",1125
"ci.err.regex": "ungültiger regulärer Ausdruck in %q: %s",1135
"ci.err.regex": "ungültiger regulärer Ausdruck in %q: %s",1136
"ci.err.parallel": "parallel erzeugt %d Jobs; das Maximum ist %d",1137
"ci.err.parallelForm": "parallel ist eine Zahl von 1 bis %d, oder eine Matrix",1138
"ci.err.retry": "retry ist eine Zahl von 0 bis 2, oder max und when",1139
"ci.err.service": "ein Service braucht einen Image-Namen",1140
"ci.err.environment": "environment braucht einen Namen",1141
"ci.warn.retryMax": "retry-Maximum %d ist auf %d begrenzt",1126
"ci.err.when": "when %q wird nicht unterstützt",1142
"ci.err.when": "when %q wird nicht unterstützt",1127
"ci.err.timeout": "timeout %q ist ungültig",1143
"ci.err.timeout": "timeout %q ist ungültig",1128
"ci.err.extendsList": "extends muss ein Jobname oder eine Liste sein",1144
"ci.err.extendsList": "extends muss ein Jobname oder eine Liste sein",@@ -1346,14 +1362,16 @@ var fr = map[string]string{1346
"pipe.pending": "En attente",1362
"pipe.pending": "En attente",1347
"pipe.manual": "Manuel",1363
"pipe.manual": "Manuel",1348
"pipe.skipped": "Ignoré",1364
"pipe.skipped": "Ignoré",1365
"pipe.canceled": "Annulé",1349
"pipe.warning": "Avertissement",1366
"pipe.warning": "Avertissement",1350
"pipe.stage": "Stage",1367
"pipe.stage": "Stage",1351
"pipe.duration": "Durée",1368
"pipe.duration": "Durée",1352
"pipe.finished": "Terminé",1369
"pipe.finished": "Terminé",1353
"pipe.queued": "File",1370
"pipe.queued": "File",1354
"pipe.timeout": "Délai",1371
"pipe.timeout": "Délai",1372
"pipe.environment": "Environnement",1355
"pipe.runner": "Runner",1373
"pipe.runner": "Runner",1356
"pipe.runnerHere": "bluegit #1",1374
"pipe.runningOn": "S'exécute sur %s",1357
"pipe.source": "Source",1375
"pipe.source": "Source",1358
"pipe.commit": "Commit",1376
"pipe.commit": "Commit",1359
"pipe.for": "pour",1377
"pipe.for": "pour",@@ -1689,6 +1707,12 @@ var fr = map[string]string{1689
"ci.err.ruleExpr": "la règle %q n'est pas comprise",1707
"ci.err.ruleExpr": "la règle %q n'est pas comprise",1690
"ci.err.ruleCombine": "la règle %q ne peut pas combiner des conditions avec && ou ||",1708
"ci.err.ruleCombine": "la règle %q ne peut pas combiner des conditions avec && ou ||",1691
"ci.err.regex": "expression régulière invalide dans %q : %s",1709
"ci.err.regex": "expression régulière invalide dans %q : %s",1710
"ci.err.parallel": "parallel crée %d jobs ; le maximum est %d",1711
"ci.err.parallelForm": "parallel est un nombre de 1 à %d, ou une matrice",1712
"ci.err.retry": "retry est un nombre de 0 à 2, ou max et when",1713
"ci.err.service": "un service a besoin d'un nom d'image",1714
"ci.err.environment": "environment a besoin d'un nom",1715
"ci.warn.retryMax": "le maximum de retry %d est limité à %d",1692
"ci.err.when": "when %q n'est pas pris en charge",1716
"ci.err.when": "when %q n'est pas pris en charge",1693
"ci.err.timeout": "timeout %q est invalide",1717
"ci.err.timeout": "timeout %q est invalide",1694
"ci.err.extendsList": "extends doit être un nom de job ou une liste",1718
"ci.err.extendsList": "extends doit être un nom de job ou une liste",@@ -0,0 +1,29 @@
1
package runner
2
3
import (
4
"testing"
5
6
"gitlab.com/Timfa/bluegit/internal/store"
7
)
8
9
func TestShouldRetry(t *testing.T) {
10
job := store.Job{RetryMax: 2}
11
if !shouldRetry(job, "script_failure") {
12
t.Fatal("lege when herhaalt")
13
}
14
job.Attempt = 2
15
if shouldRetry(job, "script_failure") {
16
t.Fatal("max bereikt")
17
}
18
job.Attempt = 0
19
job.RetryWhen = []string{"runner_system_failure", "stuck_or_timeout_failure"}
20
if shouldRetry(job, "script_failure") {
21
t.Fatal("script hoort niet")
22
}
23
if !shouldRetry(job, "runner_system_failure") || !shouldRetry(job, "job_execution_timeout") {
24
t.Fatal("systeem en timeout")
25
}
26
if safeSegment("build 1/3") != "build_1_3" {
27
t.Fatal(safeSegment("build 1/3"))
28
}
29
}
@@ -0,0 +1,29 @@1
package runner2
3
import (4
"testing"5
6
"gitlab.com/Timfa/bluegit/internal/store"7
)8
9
func TestShouldRetry(t *testing.T) {10
job := store.Job{RetryMax: 2}11
if !shouldRetry(job, "script_failure") {12
t.Fatal("lege when herhaalt")13
}14
job.Attempt = 215
if shouldRetry(job, "script_failure") {16
t.Fatal("max bereikt")17
}18
job.Attempt = 019
job.RetryWhen = []string{"runner_system_failure", "stuck_or_timeout_failure"}20
if shouldRetry(job, "script_failure") {21
t.Fatal("script hoort niet")22
}23
if !shouldRetry(job, "runner_system_failure") || !shouldRetry(job, "job_execution_timeout") {24
t.Fatal("systeem en timeout")25
}26
if safeSegment("build 1/3") != "build_1_3" {27
t.Fatal(safeSegment("build 1/3"))28
}29
}@@ -4,6 +4,7 @@ package runner
4
4
import (
5
5
"bytes"
6
6
"context"
7
"errors"
7
8
"fmt"
8
9
"io"
9
10
"log/slog"
@@ -23,17 +24,45 @@ import (
23
24
24
25
// Runner plant jobs en stuurt het resultaat door naar webhooks.
25
26
type Runner struct {
26
Store *store.Store
27
GitBin string
28
Docker string
29
Storage string
30
BaseURL string
31
Network string
32
Timeout time.Duration
33
Log *slog.Logger
34
OnEvent func(repoID int64, ev webhook.Event)
35
slots chan struct{}
36
mu sync.RWMutex
27
Store *store.Store
28
GitBin string
29
Docker string
30
Storage string
31
BaseURL string
32
Network string
33
Timeout time.Duration
34
Log *slog.Logger
35
OnEvent func(repoID int64, ev webhook.Event)
36
slots chan struct{}
37
mu sync.RWMutex
38
cancelMu sync.Mutex
39
cancel map[int64]context.CancelFunc
40
}
41
42
// Interrupt stopt lopende jobs waarvan een nieuwere pipeline de beurt overneemt.
43
func (r *Runner) Interrupt(ids []int64) {
44
r.cancelMu.Lock()
45
defer r.cancelMu.Unlock()
46
for _, id := range ids {
47
if c := r.cancel[id]; c != nil {
48
c()
49
}
50
}
51
}
52
53
func (r *Runner) track(id int64, cancel context.CancelFunc) {
54
r.cancelMu.Lock()
55
defer r.cancelMu.Unlock()
56
if r.cancel == nil {
57
r.cancel = map[int64]context.CancelFunc{}
58
}
59
r.cancel[id] = cancel
60
}
61
62
func (r *Runner) untrack(id int64) {
63
r.cancelMu.Lock()
64
defer r.cancelMu.Unlock()
65
delete(r.cancel, id)
37
66
}
38
67
39
68
// SetStorage wijzigt het pad waarop jobs hun clone en artifacts zetten.
@@ -186,13 +215,52 @@ func (r *Runner) execute(ctx context.Context, p store.Pipeline, job store.Job) {
186
215
defer func() { <-r.slots }()
187
216
}
188
217
before, _ := r.Store.PipelineByID(context.Background(), p.RepoID, p.ID)
189
mask, err := r.runOne(ctx, p, job)
218
base, stop := context.WithCancel(ctx)
219
r.track(job.ID, stop)
220
defer func() {
221
r.untrack(job.ID)
222
stop()
223
}()
224
runCtx := base
225
limit := r.Timeout
226
if job.Timeout > 0 {
227
limit = job.Timeout
228
}
229
if limit > 0 {
230
var cancel context.CancelFunc
231
runCtx, cancel = context.WithTimeout(base, limit)
232
defer cancel()
233
}
234
mask, err := r.runOne(runCtx, p, job)
190
235
status := "success"
191
236
extra := ""
192
if err != nil {
237
kind := ""
238
switch {
239
case err == nil:
240
case ctx.Err() == nil && base.Err() != nil:
241
status = "canceled"
242
extra = stamp("canceled because a newer pipeline started for the same ref")
243
case errors.Is(runCtx.Err(), context.DeadlineExceeded):
193
244
status = "failed"
245
kind = "job_execution_timeout"
246
extra = stamp(ci.Mask(err.Error(), mask))
247
default:
248
status = "failed"
249
kind = classifyFailure(err)
194
250
extra = stamp(ci.Mask(err.Error(), mask))
195
251
}
252
if status == "failed" && shouldRetry(job, kind) {
253
note := stamp(fmt.Sprintf("retrying (%d/%d) after %s", job.Attempt+1, job.RetryMax, kind))
254
if ferr := r.Store.RequeueJob(context.Background(), job.ID, job.Attempt+1, note+extra); ferr != nil && r.Log != nil {
255
r.Log.Error("job opnieuw", "err", ferr)
256
} else if ferr == nil {
257
r.settle(context.Background(), p.ID)
258
if _, rerr := r.Store.RefreshPipelineStatus(context.Background(), p.ID); rerr != nil && r.Log != nil {
259
r.Log.Error("pipelinestatus", "err", rerr)
260
}
261
return
262
}
263
}
196
264
if ferr := r.Store.FinishJob(context.Background(), job.ID, status, extra); ferr != nil {
197
265
r.Log.Error("job afronden", "err", ferr)
198
266
}
@@ -230,22 +298,13 @@ func (r *Runner) execute(ctx context.Context, p store.Pipeline, job store.Job) {
230
298
}
231
299
232
300
func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([]string, error) {
233
limit := r.Timeout
234
if job.Timeout > 0 {
235
limit = job.Timeout
236
}
237
if limit > 0 {
238
var cancel context.CancelFunc
239
ctx, cancel = context.WithTimeout(ctx, limit)
240
defer cancel()
241
}
242
301
repo, err := r.Store.RepoByID(ctx, p.RepoID)
243
302
if err != nil {
244
303
return nil, err
245
304
}
246
305
sink := &logSink{store: r.Store, jobID: job.ID}
247
306
sink.section("prepare", "Preparing the docker executor")
248
work := filepath.Join(r.storage(), "work", fmt.Sprint(p.ID), job.Name)
307
work := filepath.Join(r.storage(), "work", fmt.Sprint(p.ID), safeSegment(job.Name))
249
308
_ = os.RemoveAll(work)
250
309
if err := os.MkdirAll(work, 0o755); err != nil {
251
310
sink.closeSection()
@@ -286,13 +345,23 @@ func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([
286
345
sink.secrets = mask
287
346
if _, err := exec.LookPath(r.Docker); err != nil {
288
347
sink.closeSection()
289
return mask, fmt.Errorf("docker (%s) is not available; the job only runs in a container", r.Docker)
348
return mask, fmt.Errorf("runner_system_failure: docker (%s) is not available; the job only runs in a container", r.Docker)
349
}
350
network := r.Network
351
if len(job.Services) > 0 {
352
net, cleanup, serr := r.startServices(ctx, job)
353
if serr != nil {
354
sink.closeSection()
355
return mask, serr
356
}
357
defer cleanup()
358
network = net
290
359
}
291
360
sink.closeSection()
292
361
// -e NAAM zonder waarde laat Docker de variabele uit de omgeving van
293
362
// deze client overnemen. Zo blijft een meerregelige sleutel heel en
294
363
// staat de waarde niet in de proceslijst.
295
args := []string{"run", "--rm", "--network", r.Network, "-v", work + ":/workspace", "-w", "/workspace"}
364
args := []string{"run", "--rm", "--network", network, "-v", work + ":/workspace", "-w", "/workspace"}
296
365
cmdEnv := append([]string{}, os.Environ()...)
297
366
for k, v := range env {
298
367
args = append(args, "-e", k)
@@ -451,11 +520,27 @@ func (r *Runner) environment(ctx context.Context, repo store.Repo, p store.Pipel
451
520
env[k] = v
452
521
}
453
522
base := strings.TrimRight(r.BaseURL, "/")
523
var info *ci.MRInfo
524
if p.MRID != 0 {
525
if m, merr := r.Store.MergeRequest(ctx, repo.ID, p.MRID); merr == nil {
526
target := ""
527
dir := gitrepo.Dir(r.storage(), repo.OwnerPath, repo.Name)
528
if out, gerr := exec.CommandContext(ctx, r.GitBin, "-C", dir, "rev-parse", "--verify", "refs/heads/"+m.TargetBranch).Output(); gerr == nil {
529
target = strings.TrimSpace(string(out))
530
}
531
info = &ci.MRInfo{
532
ID: m.ID, IID: m.Number, Title: m.Title,
533
SourceBranch: m.SourceBranch, TargetBranch: m.TargetBranch,
534
ProjectPath: repo.FullPath(), ProjectURL: base + "/" + repo.FullPath(),
535
TargetSHA: target,
536
}
537
}
538
}
454
539
pre := ci.Variables(ci.Ref{
455
540
SHA: p.CommitSHA, FullRef: p.Ref, Title: p.CommitTitle,
456
541
DefaultBranch: repo.DefaultBranch, Source: p.Source,
457
542
ProjectPath: repo.FullPath(), ProjectURL: base + "/" + repo.FullPath(),
458
APIURL: base + "/api/v1", PipelineID: p.ID,
543
APIURL: base + "/api/v1", PipelineID: p.ID, MR: info,
459
544
})
460
545
for k, v := range pre {
461
546
env[k] = v
@@ -465,10 +550,119 @@ func (r *Runner) environment(ctx context.Context, repo store.Repo, p store.Pipel
465
550
env["CI_JOB_STAGE"] = job.Stage
466
551
env["CI_JOB_URL"] = base + "/" + repo.FullPath() + "/-/pipelines/" + fmt.Sprint(p.ID) + "/jobs/" + fmt.Sprint(job.ID)
467
552
env["CI_JOB_TOKEN"] = token
553
if job.Environment != "" {
554
env["CI_ENVIRONMENT_NAME"] = job.Environment
555
if job.EnvironmentURL != "" {
556
env["CI_ENVIRONMENT_URL"] = ci.Expand(job.EnvironmentURL, env)
557
}
558
}
559
for k, v := range job.Variables {
560
if strings.HasPrefix(k, "CI_NODE_") {
561
env[k] = v
562
}
563
}
468
564
mask = append(mask, token)
469
565
return env, mask, nil
470
566
}
471
567
568
func (r *Runner) startServices(ctx context.Context, job store.Job) (string, func(), error) {
569
netName := fmt.Sprintf("bluegit-%d", job.ID)
570
if _, err := execOut(ctx, r.Docker, "network", "create", netName); err != nil {
571
return "", func() {}, fmt.Errorf("runner_system_failure: docker network: %w", err)
572
}
573
var names []string
574
cleanup := func() {
575
cctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
576
defer cancel()
577
for _, name := range names {
578
_, _ = execOut(cctx, r.Docker, "rm", "-f", name)
579
}
580
_, _ = execOut(cctx, r.Docker, "network", "rm", netName)
581
}
582
for i, svc := range job.Services {
583
cname := fmt.Sprintf("bluegit-svc-%d-%d", job.ID, i)
584
args := []string{"run", "-d", "--name", cname, "--network", netName}
585
if svc.Alias != "" {
586
args = append(args, "--network-alias", svc.Alias)
587
}
588
cmdEnv := append([]string{}, os.Environ()...)
589
for k, v := range svc.Variables {
590
args = append(args, "-e", k)
591
cmdEnv = append(cmdEnv, k+"="+v)
592
}
593
if len(svc.Entrypoint) > 0 {
594
args = append(args, "--entrypoint", svc.Entrypoint[0])
595
}
596
args = append(args, svc.Name)
597
if len(svc.Entrypoint) > 1 {
598
args = append(args, svc.Entrypoint[1:]...)
599
}
600
args = append(args, svc.Command...)
601
cmd := exec.CommandContext(ctx, r.Docker, args...)
602
cmd.Env = cmdEnv
603
var stderr bytes.Buffer
604
cmd.Stderr = &stderr
605
if err := cmd.Run(); err != nil {
606
cleanup()
607
msg := strings.TrimSpace(stderr.String())
608
if msg == "" {
609
msg = err.Error()
610
}
611
return "", func() {}, fmt.Errorf("runner_system_failure: service %s: %s", svc.Name, msg)
612
}
613
names = append(names, cname)
614
}
615
return netName, cleanup, nil
616
}
617
618
func shouldRetry(job store.Job, kind string) bool {
619
if job.RetryMax <= 0 || job.Attempt >= job.RetryMax || kind == "" {
620
return false
621
}
622
if len(job.RetryWhen) == 0 || containsStr(job.RetryWhen, "always") {
623
return true
624
}
625
if containsStr(job.RetryWhen, kind) {
626
return true
627
}
628
return kind == "job_execution_timeout" && containsStr(job.RetryWhen, "stuck_or_timeout_failure")
629
}
630
631
func classifyFailure(err error) string {
632
msg := err.Error()
633
if strings.Contains(msg, "runner_system_failure") || strings.Contains(msg, "is not available") {
634
return "runner_system_failure"
635
}
636
if errors.Is(err, context.DeadlineExceeded) || strings.Contains(msg, "deadline exceeded") {
637
return "job_execution_timeout"
638
}
639
return "script_failure"
640
}
641
642
func containsStr(list []string, s string) bool {
643
for _, item := range list {
644
if item == s {
645
return true
646
}
647
}
648
return false
649
}
650
651
func safeSegment(name string) string {
652
var b strings.Builder
653
for _, r := range name {
654
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '.' || r == '_' || r == '-' {
655
b.WriteRune(r)
656
} else {
657
b.WriteByte('_')
658
}
659
}
660
if b.Len() == 0 {
661
return "job"
662
}
663
return b.String()
664
}
665
472
666
func (r *Runner) restoreCache(repoID int64, job store.Job, work string) error {
473
667
if len(job.CachePaths) == 0 {
474
668
return nil
@@ -4,6 +4,7 @@ package runner4
import (4
import (5
"bytes"5
"bytes"6
"context"6
"context"7
"errors"7
"fmt"8
"fmt"8
"io"9
"io"9
"log/slog"10
"log/slog"@@ -23,17 +24,45 @@ import (23
24
24
// Runner plant jobs en stuurt het resultaat door naar webhooks.25
// Runner plant jobs en stuurt het resultaat door naar webhooks.25
type Runner struct {26
type Runner struct {26
Store *store.Store27
Store *store.Store27
GitBin string28
GitBin string28
Docker string29
Docker string29
Storage string30
Storage string30
BaseURL string31
BaseURL string31
Network string32
Network string32
Timeout time.Duration33
Timeout time.Duration33
Log *slog.Logger34
Log *slog.Logger34
OnEvent func(repoID int64, ev webhook.Event)35
OnEvent func(repoID int64, ev webhook.Event)35
slots chan struct{}36
slots chan struct{}36
mu sync.RWMutex37
mu sync.RWMutex38
cancelMu sync.Mutex39
cancel map[int64]context.CancelFunc40
}41
42
// Interrupt stopt lopende jobs waarvan een nieuwere pipeline de beurt overneemt.43
func (r *Runner) Interrupt(ids []int64) {44
r.cancelMu.Lock()45
defer r.cancelMu.Unlock()46
for _, id := range ids {47
if c := r.cancel[id]; c != nil {48
c()49
}50
}51
}52
53
func (r *Runner) track(id int64, cancel context.CancelFunc) {54
r.cancelMu.Lock()55
defer r.cancelMu.Unlock()56
if r.cancel == nil {57
r.cancel = map[int64]context.CancelFunc{}58
}59
r.cancel[id] = cancel60
}61
62
func (r *Runner) untrack(id int64) {63
r.cancelMu.Lock()64
defer r.cancelMu.Unlock()65
delete(r.cancel, id)37
}66
}38
67
39
// SetStorage wijzigt het pad waarop jobs hun clone en artifacts zetten.68
// SetStorage wijzigt het pad waarop jobs hun clone en artifacts zetten.@@ -186,13 +215,52 @@ func (r *Runner) execute(ctx context.Context, p store.Pipeline, job store.Job) {186
defer func() { <-r.slots }()215
defer func() { <-r.slots }()187
}216
}188
before, _ := r.Store.PipelineByID(context.Background(), p.RepoID, p.ID)217
before, _ := r.Store.PipelineByID(context.Background(), p.RepoID, p.ID)189
mask, err := r.runOne(ctx, p, job)218
base, stop := context.WithCancel(ctx)219
r.track(job.ID, stop)220
defer func() {221
r.untrack(job.ID)222
stop()223
}()224
runCtx := base225
limit := r.Timeout226
if job.Timeout > 0 {227
limit = job.Timeout228
}229
if limit > 0 {230
var cancel context.CancelFunc231
runCtx, cancel = context.WithTimeout(base, limit)232
defer cancel()233
}234
mask, err := r.runOne(runCtx, p, job)190
status := "success"235
status := "success"191
extra := ""236
extra := ""192
if err != nil {237
kind := ""238
switch {239
case err == nil:240
case ctx.Err() == nil && base.Err() != nil:241
status = "canceled"242
extra = stamp("canceled because a newer pipeline started for the same ref")243
case errors.Is(runCtx.Err(), context.DeadlineExceeded):193
status = "failed"244
status = "failed"245
kind = "job_execution_timeout"246
extra = stamp(ci.Mask(err.Error(), mask))247
default:248
status = "failed"249
kind = classifyFailure(err)194
extra = stamp(ci.Mask(err.Error(), mask))250
extra = stamp(ci.Mask(err.Error(), mask))195
}251
}252
if status == "failed" && shouldRetry(job, kind) {253
note := stamp(fmt.Sprintf("retrying (%d/%d) after %s", job.Attempt+1, job.RetryMax, kind))254
if ferr := r.Store.RequeueJob(context.Background(), job.ID, job.Attempt+1, note+extra); ferr != nil && r.Log != nil {255
r.Log.Error("job opnieuw", "err", ferr)256
} else if ferr == nil {257
r.settle(context.Background(), p.ID)258
if _, rerr := r.Store.RefreshPipelineStatus(context.Background(), p.ID); rerr != nil && r.Log != nil {259
r.Log.Error("pipelinestatus", "err", rerr)260
}261
return262
}263
}196
if ferr := r.Store.FinishJob(context.Background(), job.ID, status, extra); ferr != nil {264
if ferr := r.Store.FinishJob(context.Background(), job.ID, status, extra); ferr != nil {197
r.Log.Error("job afronden", "err", ferr)265
r.Log.Error("job afronden", "err", ferr)198
}266
}@@ -230,22 +298,13 @@ func (r *Runner) execute(ctx context.Context, p store.Pipeline, job store.Job) {230
}298
}231
299
232
func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([]string, error) {300
func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([]string, error) {233
limit := r.Timeout234
if job.Timeout > 0 {235
limit = job.Timeout236
}237
if limit > 0 {238
var cancel context.CancelFunc239
ctx, cancel = context.WithTimeout(ctx, limit)240
defer cancel()241
}242
repo, err := r.Store.RepoByID(ctx, p.RepoID)301
repo, err := r.Store.RepoByID(ctx, p.RepoID)243
if err != nil {302
if err != nil {244
return nil, err303
return nil, err245
}304
}246
sink := &logSink{store: r.Store, jobID: job.ID}305
sink := &logSink{store: r.Store, jobID: job.ID}247
sink.section("prepare", "Preparing the docker executor")306
sink.section("prepare", "Preparing the docker executor")248
work := filepath.Join(r.storage(), "work", fmt.Sprint(p.ID), job.Name)307
work := filepath.Join(r.storage(), "work", fmt.Sprint(p.ID), safeSegment(job.Name))249
_ = os.RemoveAll(work)308
_ = os.RemoveAll(work)250
if err := os.MkdirAll(work, 0o755); err != nil {309
if err := os.MkdirAll(work, 0o755); err != nil {251
sink.closeSection()310
sink.closeSection()@@ -286,13 +345,23 @@ func (r *Runner) runOne(ctx context.Context, p store.Pipeline, job store.Job) ([286
sink.secrets = mask345
sink.secrets = mask287
if _, err := exec.LookPath(r.Docker); err != nil {346
if _, err := exec.LookPath(r.Docker); err != nil {288
sink.closeSection()347
sink.closeSection()289
return mask, fmt.Errorf("docker (%s) is not available; the job only runs in a container", r.Docker)348
return mask, fmt.Errorf("runner_system_failure: docker (%s) is not available; the job only runs in a container", r.Docker)349
}350
network := r.Network351
if len(job.Services) > 0 {352
net, cleanup, serr := r.startServices(ctx, job)353
if serr != nil {354
sink.closeSection()355
return mask, serr356
}357
defer cleanup()358
network = net290
}359
}291
sink.closeSection()360
sink.closeSection()292
// -e NAAM zonder waarde laat Docker de variabele uit de omgeving van361
// -e NAAM zonder waarde laat Docker de variabele uit de omgeving van293
// deze client overnemen. Zo blijft een meerregelige sleutel heel en362
// deze client overnemen. Zo blijft een meerregelige sleutel heel en294
// staat de waarde niet in de proceslijst.363
// staat de waarde niet in de proceslijst.295
args := []string{"run", "--rm", "--network", r.Network, "-v", work + ":/workspace", "-w", "/workspace"}364
args := []string{"run", "--rm", "--network", network, "-v", work + ":/workspace", "-w", "/workspace"}296
cmdEnv := append([]string{}, os.Environ()...)365
cmdEnv := append([]string{}, os.Environ()...)297
for k, v := range env {366
for k, v := range env {298
args = append(args, "-e", k)367
args = append(args, "-e", k)@@ -451,11 +520,27 @@ func (r *Runner) environment(ctx context.Context, repo store.Repo, p store.Pipel451
env[k] = v520
env[k] = v452
}521
}453
base := strings.TrimRight(r.BaseURL, "/")522
base := strings.TrimRight(r.BaseURL, "/")523
var info *ci.MRInfo524
if p.MRID != 0 {525
if m, merr := r.Store.MergeRequest(ctx, repo.ID, p.MRID); merr == nil {526
target := ""527
dir := gitrepo.Dir(r.storage(), repo.OwnerPath, repo.Name)528
if out, gerr := exec.CommandContext(ctx, r.GitBin, "-C", dir, "rev-parse", "--verify", "refs/heads/"+m.TargetBranch).Output(); gerr == nil {529
target = strings.TrimSpace(string(out))530
}531
info = &ci.MRInfo{532
ID: m.ID, IID: m.Number, Title: m.Title,533
SourceBranch: m.SourceBranch, TargetBranch: m.TargetBranch,534
ProjectPath: repo.FullPath(), ProjectURL: base + "/" + repo.FullPath(),535
TargetSHA: target,536
}537
}538
}454
pre := ci.Variables(ci.Ref{539
pre := ci.Variables(ci.Ref{455
SHA: p.CommitSHA, FullRef: p.Ref, Title: p.CommitTitle,540
SHA: p.CommitSHA, FullRef: p.Ref, Title: p.CommitTitle,456
DefaultBranch: repo.DefaultBranch, Source: p.Source,541
DefaultBranch: repo.DefaultBranch, Source: p.Source,457
ProjectPath: repo.FullPath(), ProjectURL: base + "/" + repo.FullPath(),542
ProjectPath: repo.FullPath(), ProjectURL: base + "/" + repo.FullPath(),458
APIURL: base + "/api/v1", PipelineID: p.ID,543
APIURL: base + "/api/v1", PipelineID: p.ID, MR: info,459
})544
})460
for k, v := range pre {545
for k, v := range pre {461
env[k] = v546
env[k] = v@@ -465,10 +550,119 @@ func (r *Runner) environment(ctx context.Context, repo store.Repo, p store.Pipel465
env["CI_JOB_STAGE"] = job.Stage550
env["CI_JOB_STAGE"] = job.Stage466
env["CI_JOB_URL"] = base + "/" + repo.FullPath() + "/-/pipelines/" + fmt.Sprint(p.ID) + "/jobs/" + fmt.Sprint(job.ID)551
env["CI_JOB_URL"] = base + "/" + repo.FullPath() + "/-/pipelines/" + fmt.Sprint(p.ID) + "/jobs/" + fmt.Sprint(job.ID)467
env["CI_JOB_TOKEN"] = token552
env["CI_JOB_TOKEN"] = token553
if job.Environment != "" {554
env["CI_ENVIRONMENT_NAME"] = job.Environment555
if job.EnvironmentURL != "" {556
env["CI_ENVIRONMENT_URL"] = ci.Expand(job.EnvironmentURL, env)557
}558
}559
for k, v := range job.Variables {560
if strings.HasPrefix(k, "CI_NODE_") {561
env[k] = v562
}563
}468
mask = append(mask, token)564
mask = append(mask, token)469
return env, mask, nil565
return env, mask, nil470
}566
}471
567
568
func (r *Runner) startServices(ctx context.Context, job store.Job) (string, func(), error) {569
netName := fmt.Sprintf("bluegit-%d", job.ID)570
if _, err := execOut(ctx, r.Docker, "network", "create", netName); err != nil {571
return "", func() {}, fmt.Errorf("runner_system_failure: docker network: %w", err)572
}573
var names []string574
cleanup := func() {575
cctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)576
defer cancel()577
for _, name := range names {578
_, _ = execOut(cctx, r.Docker, "rm", "-f", name)579
}580
_, _ = execOut(cctx, r.Docker, "network", "rm", netName)581
}582
for i, svc := range job.Services {583
cname := fmt.Sprintf("bluegit-svc-%d-%d", job.ID, i)584
args := []string{"run", "-d", "--name", cname, "--network", netName}585
if svc.Alias != "" {586
args = append(args, "--network-alias", svc.Alias)587
}588
cmdEnv := append([]string{}, os.Environ()...)589
for k, v := range svc.Variables {590
args = append(args, "-e", k)591
cmdEnv = append(cmdEnv, k+"="+v)592
}593
if len(svc.Entrypoint) > 0 {594
args = append(args, "--entrypoint", svc.Entrypoint[0])595
}596
args = append(args, svc.Name)597
if len(svc.Entrypoint) > 1 {598
args = append(args, svc.Entrypoint[1:]...)599
}600
args = append(args, svc.Command...)601
cmd := exec.CommandContext(ctx, r.Docker, args...)602
cmd.Env = cmdEnv603
var stderr bytes.Buffer604
cmd.Stderr = &stderr605
if err := cmd.Run(); err != nil {606
cleanup()607
msg := strings.TrimSpace(stderr.String())608
if msg == "" {609
msg = err.Error()610
}611
return "", func() {}, fmt.Errorf("runner_system_failure: service %s: %s", svc.Name, msg)612
}613
names = append(names, cname)614
}615
return netName, cleanup, nil616
}617
618
func shouldRetry(job store.Job, kind string) bool {619
if job.RetryMax <= 0 || job.Attempt >= job.RetryMax || kind == "" {620
return false621
}622
if len(job.RetryWhen) == 0 || containsStr(job.RetryWhen, "always") {623
return true624
}625
if containsStr(job.RetryWhen, kind) {626
return true627
}628
return kind == "job_execution_timeout" && containsStr(job.RetryWhen, "stuck_or_timeout_failure")629
}630
631
func classifyFailure(err error) string {632
msg := err.Error()633
if strings.Contains(msg, "runner_system_failure") || strings.Contains(msg, "is not available") {634
return "runner_system_failure"635
}636
if errors.Is(err, context.DeadlineExceeded) || strings.Contains(msg, "deadline exceeded") {637
return "job_execution_timeout"638
}639
return "script_failure"640
}641
642
func containsStr(list []string, s string) bool {643
for _, item := range list {644
if item == s {645
return true646
}647
}648
return false649
}650
651
func safeSegment(name string) string {652
var b strings.Builder653
for _, r := range name {654
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '.' || r == '_' || r == '-' {655
b.WriteRune(r)656
} else {657
b.WriteByte('_')658
}659
}660
if b.Len() == 0 {661
return "job"662
}663
return b.String()664
}665
472
func (r *Runner) restoreCache(repoID int64, job store.Job, work string) error {666
func (r *Runner) restoreCache(repoID int64, job store.Job, work string) error {473
if len(job.CachePaths) == 0 {667
if len(job.CachePaths) == 0 {474
return nil668
return nil@@ -42,6 +42,7 @@ func (s *Server) AfterPush(repo store.Repo, user store.User, before, after map[s
42
42
s.recordPushContributions(repo, user, map[string]string{ref: before[ref]}, map[string]string{ref: sha})
43
43
s.startPipeline(context.Background(), repo, sha, ref, actor, "push", before[ref], nil)
44
44
s.startMergeRequestPipelines(context.Background(), repo, sha, branch, actor, before[ref])
45
s.rerunMergeRequestsOnTarget(context.Background(), repo, branch, actor)
45
46
web := strings.TrimRight(s.Cfg.BaseURL, "/") + "/" + repo.FullPath()
46
47
s.Dispatch(repo.ID, webhook.Event{
47
48
Type: webhook.EventPush,
@@ -72,6 +73,9 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
72
73
if mr != nil && id != 0 {
73
74
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
74
75
}
76
if id != 0 {
77
s.cancelOlder(ctx, repo.ID, ref, id)
78
}
75
79
return id
76
80
}
77
81
if len(jobs) == 0 {
@@ -87,9 +91,28 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a
87
91
if mr != nil {
88
92
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)
89
93
}
94
s.cancelOlder(ctx, repo.ID, ref, id)
90
95
return id
91
96
}
92
97
98
func (s *Server) cancelOlder(ctx context.Context, repoID int64, ref string, keep int64) {
99
running, pipes, err := s.Store.CancelInterruptible(ctx, repoID, ref, keep)
100
if err != nil {
101
if s.Log != nil {
102
s.Log.Error("interruptible", "err", err)
103
}
104
return
105
}
106
if s.Runner != nil {
107
s.Runner.Interrupt(running)
108
}
109
for _, id := range pipes {
110
if _, err := s.Store.RefreshPipelineStatus(ctx, id); err != nil && s.Log != nil {
111
s.Log.Error("pipelinestatus", "err", err)
112
}
113
}
114
}
115
93
116
func gitOne(ctx context.Context, gitBin, dir string, args ...string) string {
94
117
out, err := exec.CommandContext(ctx, gitBin, append([]string{"-C", dir}, args...)...).Output()
95
118
if err != nil {
@@ -42,6 +42,7 @@ func (s *Server) AfterPush(repo store.Repo, user store.User, before, after map[s42
s.recordPushContributions(repo, user, map[string]string{ref: before[ref]}, map[string]string{ref: sha})42
s.recordPushContributions(repo, user, map[string]string{ref: before[ref]}, map[string]string{ref: sha})43
s.startPipeline(context.Background(), repo, sha, ref, actor, "push", before[ref], nil)43
s.startPipeline(context.Background(), repo, sha, ref, actor, "push", before[ref], nil)44
s.startMergeRequestPipelines(context.Background(), repo, sha, branch, actor, before[ref])44
s.startMergeRequestPipelines(context.Background(), repo, sha, branch, actor, before[ref])45
s.rerunMergeRequestsOnTarget(context.Background(), repo, branch, actor)45
web := strings.TrimRight(s.Cfg.BaseURL, "/") + "/" + repo.FullPath()46
web := strings.TrimRight(s.Cfg.BaseURL, "/") + "/" + repo.FullPath()46
s.Dispatch(repo.ID, webhook.Event{47
s.Dispatch(repo.ID, webhook.Event{47
Type: webhook.EventPush,48
Type: webhook.EventPush,@@ -72,6 +73,9 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a72
if mr != nil && id != 0 {73
if mr != nil && id != 0 {73
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)74
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)74
}75
}76
if id != 0 {77
s.cancelOlder(ctx, repo.ID, ref, id)78
}75
return id79
return id76
}80
}77
if len(jobs) == 0 {81
if len(jobs) == 0 {@@ -87,9 +91,28 @@ func (s *Server) startPipeline(ctx context.Context, repo store.Repo, sha, ref, a87
if mr != nil {91
if mr != nil {88
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)92
_ = s.Store.AttachPipelineMR(ctx, id, mr.ID)89
}93
}94
s.cancelOlder(ctx, repo.ID, ref, id)90
return id95
return id91
}96
}92
97
98
func (s *Server) cancelOlder(ctx context.Context, repoID int64, ref string, keep int64) {99
running, pipes, err := s.Store.CancelInterruptible(ctx, repoID, ref, keep)100
if err != nil {101
if s.Log != nil {102
s.Log.Error("interruptible", "err", err)103
}104
return105
}106
if s.Runner != nil {107
s.Runner.Interrupt(running)108
}109
for _, id := range pipes {110
if _, err := s.Store.RefreshPipelineStatus(ctx, id); err != nil && s.Log != nil {111
s.Log.Error("pipelinestatus", "err", err)112
}113
}114
}115
93
func gitOne(ctx context.Context, gitBin, dir string, args ...string) string {116
func gitOne(ctx context.Context, gitBin, dir string, args ...string) string {94
out, err := exec.CommandContext(ctx, gitBin, append([]string{"-C", dir}, args...)...).Output()117
out, err := exec.CommandContext(ctx, gitBin, append([]string{"-C", dir}, args...)...).Output()95
if err != nil {118
if err != nil {@@ -38,7 +38,15 @@ func (s *Server) planPipeline(ctx context.Context, repo store.Repo, sha, ref, so
38
38
DefaultBranch: repo.DefaultBranch, Source: source, ProjectPath: repo.FullPath(),
39
39
ProjectURL: base + "/" + repo.FullPath(), APIURL: base + "/api/v1", MR: mr,
40
40
})
41
active, err := spec.Select(ci.Eval{Env: env, Changed: changed, Files: files})
41
ev := ci.Eval{Env: env, Changed: changed, Files: files}
42
allow, err := spec.Allow(ev)
43
if err != nil {
44
return "", nil, err
45
}
46
if !allow {
47
return title, nil, nil
48
}
49
active, err := spec.Select(ev)
42
50
if err != nil {
43
51
return "", nil, err
44
52
}
@@ -59,6 +67,12 @@ func (s *Server) planPipeline(ctx context.Context, repo store.Repo, sha, ref, so
59
67
if paths == nil {
60
68
paths = []string{}
61
69
}
70
var services []store.Service
71
for _, svc := range j.Services {
72
services = append(services, store.Service{
73
Name: svc.Name, Alias: svc.Alias, Command: svc.Command, Entrypoint: svc.Entrypoint, Variables: svc.Variables,
74
})
75
}
62
76
jobs = append(jobs, store.NewJob{
63
77
Name: j.Name, Stage: j.Stage, Position: i, Image: j.Image,
64
78
Before: strings.Join(j.Before, "\n"), Script: strings.Join(j.Script, "\n"), After: strings.Join(j.After, "\n"),
@@ -66,6 +80,8 @@ func (s *Server) planPipeline(ctx context.Context, repo store.Repo, sha, ref, so
66
80
Variables: vars, ArtifactPaths: paths, ArtifactExpire: j.Artifacts.ExpireIn,
67
81
AllowFailure: j.AllowFailure, When: j.When, Timeout: j.Timeout, Tags: j.Tags,
68
82
CachePaths: j.Cache.Paths, CacheKey: j.Cache.Key, ResourceGroup: j.ResourceGroup,
83
RetryMax: j.Retry.Max, RetryWhen: j.Retry.When, Services: services,
84
Environment: j.Environment.Name, EnvironmentURL: j.Environment.URL, Interruptible: j.Interruptible,
69
85
})
70
86
}
71
87
return title, jobs, nil
@@ -88,6 +104,23 @@ func (s *Server) startMergeRequestPipelines(ctx context.Context, repo store.Repo
88
104
}
89
105
}
90
106
107
// rerunMergeRequestsOnTarget start een nieuwe merge-request-pipeline op de
108
// broncommit wanneer de doelbranch zelf een push krijgt.
109
func (s *Server) rerunMergeRequestsOnTarget(ctx context.Context, repo store.Repo, branch, actor string) {
110
mrs, err := s.Store.OpenMergeRequestsByTarget(ctx, repo.ID, branch)
111
if err != nil {
112
return
113
}
114
dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
115
for _, mr := range mrs {
116
sha := gitOne(ctx, s.Git, dir, "rev-parse", "--verify", "refs/heads/"+mr.SourceBranch)
117
if sha == "" {
118
continue
119
}
120
s.startOneMergePipeline(ctx, repo, sha, actor, "", mr)
121
}
122
}
123
91
124
func (s *Server) startOneMergePipeline(ctx context.Context, repo store.Repo, sha, actor, before string, mr store.MergeRequest) {
92
125
dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)
93
126
base := strings.TrimRight(s.Cfg.BaseURL, "/")
@@ -38,7 +38,15 @@ func (s *Server) planPipeline(ctx context.Context, repo store.Repo, sha, ref, so38
DefaultBranch: repo.DefaultBranch, Source: source, ProjectPath: repo.FullPath(),38
DefaultBranch: repo.DefaultBranch, Source: source, ProjectPath: repo.FullPath(),39
ProjectURL: base + "/" + repo.FullPath(), APIURL: base + "/api/v1", MR: mr,39
ProjectURL: base + "/" + repo.FullPath(), APIURL: base + "/api/v1", MR: mr,40
})40
})41
active, err := spec.Select(ci.Eval{Env: env, Changed: changed, Files: files})41
ev := ci.Eval{Env: env, Changed: changed, Files: files}42
allow, err := spec.Allow(ev)43
if err != nil {44
return "", nil, err45
}46
if !allow {47
return title, nil, nil48
}49
active, err := spec.Select(ev)42
if err != nil {50
if err != nil {43
return "", nil, err51
return "", nil, err44
}52
}@@ -59,6 +67,12 @@ func (s *Server) planPipeline(ctx context.Context, repo store.Repo, sha, ref, so59
if paths == nil {67
if paths == nil {60
paths = []string{}68
paths = []string{}61
}69
}70
var services []store.Service71
for _, svc := range j.Services {72
services = append(services, store.Service{73
Name: svc.Name, Alias: svc.Alias, Command: svc.Command, Entrypoint: svc.Entrypoint, Variables: svc.Variables,74
})75
}62
jobs = append(jobs, store.NewJob{76
jobs = append(jobs, store.NewJob{63
Name: j.Name, Stage: j.Stage, Position: i, Image: j.Image,77
Name: j.Name, Stage: j.Stage, Position: i, Image: j.Image,64
Before: strings.Join(j.Before, "\n"), Script: strings.Join(j.Script, "\n"), After: strings.Join(j.After, "\n"),78
Before: strings.Join(j.Before, "\n"), Script: strings.Join(j.Script, "\n"), After: strings.Join(j.After, "\n"),@@ -66,6 +80,8 @@ func (s *Server) planPipeline(ctx context.Context, repo store.Repo, sha, ref, so66
Variables: vars, ArtifactPaths: paths, ArtifactExpire: j.Artifacts.ExpireIn,80
Variables: vars, ArtifactPaths: paths, ArtifactExpire: j.Artifacts.ExpireIn,67
AllowFailure: j.AllowFailure, When: j.When, Timeout: j.Timeout, Tags: j.Tags,81
AllowFailure: j.AllowFailure, When: j.When, Timeout: j.Timeout, Tags: j.Tags,68
CachePaths: j.Cache.Paths, CacheKey: j.Cache.Key, ResourceGroup: j.ResourceGroup,82
CachePaths: j.Cache.Paths, CacheKey: j.Cache.Key, ResourceGroup: j.ResourceGroup,83
RetryMax: j.Retry.Max, RetryWhen: j.Retry.When, Services: services,84
Environment: j.Environment.Name, EnvironmentURL: j.Environment.URL, Interruptible: j.Interruptible,69
})85
})70
}86
}71
return title, jobs, nil87
return title, jobs, nil@@ -88,6 +104,23 @@ func (s *Server) startMergeRequestPipelines(ctx context.Context, repo store.Repo88
}104
}89
}105
}90
106
107
// rerunMergeRequestsOnTarget start een nieuwe merge-request-pipeline op de108
// broncommit wanneer de doelbranch zelf een push krijgt.109
func (s *Server) rerunMergeRequestsOnTarget(ctx context.Context, repo store.Repo, branch, actor string) {110
mrs, err := s.Store.OpenMergeRequestsByTarget(ctx, repo.ID, branch)111
if err != nil {112
return113
}114
dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)115
for _, mr := range mrs {116
sha := gitOne(ctx, s.Git, dir, "rev-parse", "--verify", "refs/heads/"+mr.SourceBranch)117
if sha == "" {118
continue119
}120
s.startOneMergePipeline(ctx, repo, sha, actor, "", mr)121
}122
}123
91
func (s *Server) startOneMergePipeline(ctx context.Context, repo store.Repo, sha, actor, before string, mr store.MergeRequest) {124
func (s *Server) startOneMergePipeline(ctx context.Context, repo store.Repo, sha, actor, before string, mr store.MergeRequest) {92
dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)125
dir := gitrepo.Dir(s.StoragePath(), repo.OwnerPath, repo.Name)93
base := strings.TrimRight(s.Cfg.BaseURL, "/")126
base := strings.TrimRight(s.Cfg.BaseURL, "/")@@ -187,4 +187,27 @@ soft:
187
187
if !strings.Contains(page, "Warning") && !strings.Contains(page, "Passed") {
188
188
t.Fatalf("status na allow_failure:\n%s", page)
189
189
}
190
191
git(t, work, "checkout", "master")
192
writeFile(t, filepath.Join(work, "README.md"), "target\n")
193
git(t, work, "add", "README.md")
194
git(t, work, "commit", "-m", "target")
195
git(t, work, "push", "origin", "master")
196
pipes, err = st.ListPipelines(ctx, repo.ID)
197
if err != nil {
198
t.Fatal(err)
199
}
200
var newest int64
201
for _, p := range pipes {
202
if p.Source == "merge_request_event" && p.ID > newest {
203
newest = p.ID
204
if p.CommitSHA == mrPipe.CommitSHA && p.MRID == mrPipe.MRID {
205
continue
206
}
207
t.Fatalf("nieuwe mr-pipeline hoort op de broncommit: %+v", p)
208
}
209
}
210
if newest <= mrPipe.ID {
211
t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)
212
}
190
213
}
@@ -187,4 +187,27 @@ soft:187
if !strings.Contains(page, "Warning") && !strings.Contains(page, "Passed") {187
if !strings.Contains(page, "Warning") && !strings.Contains(page, "Passed") {188
t.Fatalf("status na allow_failure:\n%s", page)188
t.Fatalf("status na allow_failure:\n%s", page)189
}189
}190
191
git(t, work, "checkout", "master")192
writeFile(t, filepath.Join(work, "README.md"), "target\n")193
git(t, work, "add", "README.md")194
git(t, work, "commit", "-m", "target")195
git(t, work, "push", "origin", "master")196
pipes, err = st.ListPipelines(ctx, repo.ID)197
if err != nil {198
t.Fatal(err)199
}200
var newest int64201
for _, p := range pipes {202
if p.Source == "merge_request_event" && p.ID > newest {203
newest = p.ID204
if p.CommitSHA == mrPipe.CommitSHA && p.MRID == mrPipe.MRID {205
continue206
}207
t.Fatalf("nieuwe mr-pipeline hoort op de broncommit: %+v", p)208
}209
}210
if newest <= mrPipe.ID {211
t.Fatalf("push naar de doelbranch start geen nieuwe mr-pipeline: %d", newest)212
}190
}213
}@@ -52,33 +52,36 @@ type JobLink struct {
52
52
53
53
// JobPage is de console van één job.
54
54
type JobPage struct {
55
Job store.Job
56
Pipe store.Pipeline
57
Name string
58
Status string
59
Label string
60
Actor string
61
Avatar string
62
Initial string
63
When string
64
Duration string
65
Finished string
66
Queued string
67
Timeout string
68
Source string
69
Ref string
70
RefKind string
71
Short string
72
Title string
73
Stage string
74
PipeStatus string
75
PipeLabel string
76
Stages []string
77
Related []JobLink
78
Blocks []LogBlock
79
Running bool
80
LogBytes int
81
CanPlay bool
55
Job store.Job
56
Pipe store.Pipeline
57
Name string
58
Status string
59
Label string
60
Actor string
61
Avatar string
62
Initial string
63
When string
64
Duration string
65
Finished string
66
Queued string
67
Timeout string
68
Source string
69
Ref string
70
RefKind string
71
Short string
72
Title string
73
Stage string
74
PipeStatus string
75
PipeLabel string
76
Stages []string
77
Related []JobLink
78
Blocks []LogBlock
79
Running bool
80
LogBytes int
81
CanPlay bool
82
Environment string
83
EnvironmentURL string
84
EnvironmentLink string
82
85
}
83
86
84
87
// LogBlock is een losse regel of een inklapbare sectie.
@@ -183,6 +186,8 @@ func displayStatus(pipe string, jobs []store.Job) string {
183
186
return "manual"
184
187
case "queued":
185
188
return "pending"
189
case "canceled":
190
return "canceled"
186
191
}
187
192
warn := false
188
193
for _, j := range jobs {
@@ -228,7 +233,7 @@ func stageViews(lang i18n.Lang, jobs []store.Job, tr func(string) string) []Stag
228
233
}
229
234
230
235
func stageStatus(jobs []store.Job) string {
231
pending, failed, warning, running := false, false, false, false
236
pending, failed, warning, running, canceled := false, false, false, false, false
232
237
seen := false
233
238
for _, j := range jobs {
234
239
seen = true
@@ -237,6 +242,8 @@ func stageStatus(jobs []store.Job) string {
237
242
running = true
238
243
case j.Status == "failed" && !j.AllowFailure:
239
244
failed = true
245
case j.Status == "canceled" && !j.AllowFailure:
246
canceled = true
240
247
case j.Status == "failed" && j.AllowFailure:
241
248
warning = true
242
249
case j.Status == "queued" || j.Status == "manual":
@@ -252,6 +259,9 @@ func stageStatus(jobs []store.Job) string {
252
259
if failed {
253
260
return "failed"
254
261
}
262
if canceled {
263
return "canceled"
264
}
255
265
if warning {
256
266
return "warning"
257
267
}
@@ -451,7 +461,18 @@ func (s *Server) jobPage(r *http.Request, repo store.Repo, p store.Pipeline, job
451
461
PipeStatus: pipeStatus, PipeLabel: s.tr(r, "pipe."+pipeStatus),
452
462
Stages: stages, Related: related, Blocks: parseLog(job.Log), Running: job.Status == "running" || job.Status == "queued",
453
463
LogBytes: len(job.Log), CanPlay: job.Status == "manual" && role.CanWriteCode(),
464
Environment: job.Environment, EnvironmentURL: job.EnvironmentURL, EnvironmentLink: safeEnvironmentURL(job.EnvironmentURL),
465
}
466
}
467
468
func safeEnvironmentURL(raw string) string {
469
if strings.Contains(raw, "$") {
470
return ""
471
}
472
if strings.HasPrefix(raw, "https://") || strings.HasPrefix(raw, "http://") {
473
return raw
454
474
}
475
return ""
455
476
}
456
477
457
478
func parseLog(raw string) []LogBlock {
@@ -52,33 +52,36 @@ type JobLink struct {52
52
53
// JobPage is de console van één job.53
// JobPage is de console van één job.54
type JobPage struct {54
type JobPage struct {55
Job store.Job55
Job store.Job56
Pipe store.Pipeline56
Pipe store.Pipeline57
Name string57
Name string58
Status string58
Status string59
Label string59
Label string60
Actor string60
Actor string61
Avatar string61
Avatar string62
Initial string62
Initial string63
When string63
When string64
Duration string64
Duration string65
Finished string65
Finished string66
Queued string66
Queued string67
Timeout string67
Timeout string68
Source string68
Source string69
Ref string69
Ref string70
RefKind string70
RefKind string71
Short string71
Short string72
Title string72
Title string73
Stage string73
Stage string74
PipeStatus string74
PipeStatus string75
PipeLabel string75
PipeLabel string76
Stages []string76
Stages []string77
Related []JobLink77
Related []JobLink78
Blocks []LogBlock78
Blocks []LogBlock79
Running bool79
Running bool80
LogBytes int80
LogBytes int81
CanPlay bool81
CanPlay bool82
Environment string83
EnvironmentURL string84
EnvironmentLink string82
}85
}83
86
84
// LogBlock is een losse regel of een inklapbare sectie.87
// LogBlock is een losse regel of een inklapbare sectie.@@ -183,6 +186,8 @@ func displayStatus(pipe string, jobs []store.Job) string {183
return "manual"186
return "manual"184
case "queued":187
case "queued":185
return "pending"188
return "pending"189
case "canceled":190
return "canceled"186
}191
}187
warn := false192
warn := false188
for _, j := range jobs {193
for _, j := range jobs {@@ -228,7 +233,7 @@ func stageViews(lang i18n.Lang, jobs []store.Job, tr func(string) string) []Stag228
}233
}229
234
230
func stageStatus(jobs []store.Job) string {235
func stageStatus(jobs []store.Job) string {231
pending, failed, warning, running := false, false, false, false236
pending, failed, warning, running, canceled := false, false, false, false, false232
seen := false237
seen := false233
for _, j := range jobs {238
for _, j := range jobs {234
seen = true239
seen = true@@ -237,6 +242,8 @@ func stageStatus(jobs []store.Job) string {237
running = true242
running = true238
case j.Status == "failed" && !j.AllowFailure:243
case j.Status == "failed" && !j.AllowFailure:239
failed = true244
failed = true245
case j.Status == "canceled" && !j.AllowFailure:246
canceled = true240
case j.Status == "failed" && j.AllowFailure:247
case j.Status == "failed" && j.AllowFailure:241
warning = true248
warning = true242
case j.Status == "queued" || j.Status == "manual":249
case j.Status == "queued" || j.Status == "manual":@@ -252,6 +259,9 @@ func stageStatus(jobs []store.Job) string {252
if failed {259
if failed {253
return "failed"260
return "failed"254
}261
}262
if canceled {263
return "canceled"264
}255
if warning {265
if warning {256
return "warning"266
return "warning"257
}267
}@@ -451,7 +461,18 @@ func (s *Server) jobPage(r *http.Request, repo store.Repo, p store.Pipeline, job451
PipeStatus: pipeStatus, PipeLabel: s.tr(r, "pipe."+pipeStatus),461
PipeStatus: pipeStatus, PipeLabel: s.tr(r, "pipe."+pipeStatus),452
Stages: stages, Related: related, Blocks: parseLog(job.Log), Running: job.Status == "running" || job.Status == "queued",462
Stages: stages, Related: related, Blocks: parseLog(job.Log), Running: job.Status == "running" || job.Status == "queued",453
LogBytes: len(job.Log), CanPlay: job.Status == "manual" && role.CanWriteCode(),463
LogBytes: len(job.Log), CanPlay: job.Status == "manual" && role.CanWriteCode(),464
Environment: job.Environment, EnvironmentURL: job.EnvironmentURL, EnvironmentLink: safeEnvironmentURL(job.EnvironmentURL),465
}466
}467
468
func safeEnvironmentURL(raw string) string {469
if strings.Contains(raw, "$") {470
return ""471
}472
if strings.HasPrefix(raw, "https://") || strings.HasPrefix(raw, "http://") {473
return raw454
}474
}475
return ""455
}476
}456
477
457
func parseLog(raw string) []LogBlock {478
func parseLog(raw string) []LogBlock {@@ -165,6 +165,9 @@ func TestPipelinePages(t *testing.T) {
165
165
if !strings.HasSuffix(res.Request.URL.Path, "/pipelines/"+strconv.FormatInt(failID, 10)) {
166
166
t.Fatalf("show-id ging naar %s", res.Request.URL.Path)
167
167
}
168
if detail := read(t, res); !strings.Contains(detail, "Running on Steve") {
169
t.Fatalf("pipelinedetails:\n%s", detail)
170
}
168
171
169
172
jobURL := ts.URL + "/tim/demo/-/pipelines/" + strconv.FormatInt(failID, 10) + "/jobs/" + strconv.FormatInt(buildID, 10)
170
173
res, err = client.Get(jobURL)
@@ -174,7 +177,7 @@ func TestPipelinePages(t *testing.T) {
174
177
jobPage := read(t, res)
175
178
for _, want := range []string{
176
179
"Running script", "hello from the job", "00:08",
177
"Log timestamps in UTC.", "Related jobs", "bluegit #1", "Push",
180
"Running on Steve", "Log timestamps in UTC.", "Related jobs", ">Steve<", "Push",
178
181
`data-stage="deploy" hidden`, `value="test" selected`,
179
182
"for", "v0.0.26",
180
183
} {
@@ -165,6 +165,9 @@ func TestPipelinePages(t *testing.T) {165
if !strings.HasSuffix(res.Request.URL.Path, "/pipelines/"+strconv.FormatInt(failID, 10)) {165
if !strings.HasSuffix(res.Request.URL.Path, "/pipelines/"+strconv.FormatInt(failID, 10)) {166
t.Fatalf("show-id ging naar %s", res.Request.URL.Path)166
t.Fatalf("show-id ging naar %s", res.Request.URL.Path)167
}167
}168
if detail := read(t, res); !strings.Contains(detail, "Running on Steve") {169
t.Fatalf("pipelinedetails:\n%s", detail)170
}168
171
169
jobURL := ts.URL + "/tim/demo/-/pipelines/" + strconv.FormatInt(failID, 10) + "/jobs/" + strconv.FormatInt(buildID, 10)172
jobURL := ts.URL + "/tim/demo/-/pipelines/" + strconv.FormatInt(failID, 10) + "/jobs/" + strconv.FormatInt(buildID, 10)170
res, err = client.Get(jobURL)173
res, err = client.Get(jobURL)@@ -174,7 +177,7 @@ func TestPipelinePages(t *testing.T) {174
jobPage := read(t, res)177
jobPage := read(t, res)175
for _, want := range []string{178
for _, want := range []string{176
"Running script", "hello from the job", "00:08",179
"Running script", "hello from the job", "00:08",177
"Log timestamps in UTC.", "Related jobs", "bluegit #1", "Push",180
"Running on Steve", "Log timestamps in UTC.", "Related jobs", ">Steve<", "Push",178
`data-stage="deploy" hidden`, `value="test" selected`,181
`data-stage="deploy" hidden`, `value="test" selected`,179
"for", "v0.0.26",182
"for", "v0.0.26",180
} {183
} {@@ -369,8 +369,12 @@ func (s *Server) view(w http.ResponseWriter, r *http.Request, title string) View
369
369
if strings.HasPrefix(title, "title.") {
370
370
title = i18n.T(lang, title)
371
371
}
372
name := strings.TrimSpace(s.Cfg.Runner.Name)
373
if name == "" {
374
name = "Steve"
375
}
372
376
v := View{
373
Title: title, BaseURL: s.Cfg.BaseURL, Notice: notice, Err: problem,
377
Title: title, BaseURL: s.Cfg.BaseURL, RunnerName: name, Notice: notice, Err: problem,
374
378
StorageUnset: s.StoragePath() == "", Lang: string(lang), HTMLLang: htmlLang(lang),
375
379
Cunei: lang == i18n.Cuneiform,
376
380
}
@@ -421,6 +425,7 @@ func randomState() (string, error) {
421
425
type View struct {
422
426
Title string
423
427
BaseURL string
428
RunnerName string
424
429
User *store.User
425
430
CSRF string
426
431
Notice string
@@ -369,8 +369,12 @@ func (s *Server) view(w http.ResponseWriter, r *http.Request, title string) View369
if strings.HasPrefix(title, "title.") {369
if strings.HasPrefix(title, "title.") {370
title = i18n.T(lang, title)370
title = i18n.T(lang, title)371
}371
}372
name := strings.TrimSpace(s.Cfg.Runner.Name)373
if name == "" {374
name = "Steve"375
}372
v := View{376
v := View{373
Title: title, BaseURL: s.Cfg.BaseURL, Notice: notice, Err: problem,377
Title: title, BaseURL: s.Cfg.BaseURL, RunnerName: name, Notice: notice, Err: problem,374
StorageUnset: s.StoragePath() == "", Lang: string(lang), HTMLLang: htmlLang(lang),378
StorageUnset: s.StoragePath() == "", Lang: string(lang), HTMLLang: htmlLang(lang),375
Cunei: lang == i18n.Cuneiform,379
Cunei: lang == i18n.Cuneiform,376
}380
}@@ -421,6 +425,7 @@ func randomState() (string, error) {421
type View struct {425
type View struct {422
Title string426
Title string423
BaseURL string427
BaseURL string428
RunnerName string424
User *store.User429
User *store.User425
CSRF string430
CSRF string426
Notice string431
Notice string@@ -346,11 +346,13 @@ svg.ico { width: 14px; height: 14px; flex: none; }
346
346
.status.failed { background: #fde8e8; color: #9f2d2d; }
347
347
.status.warning { background: #fff4e0; color: #8a5a00; }
348
348
.status.running { background: #e7f0ff; color: #1f6feb; }
349
.status.canceled { background: #f3f0f0; color: #57606a; }
349
350
.mark { position: relative; width: 16px; height: 16px; border-radius: 50%; display: inline-block; flex: none; background: #8c959f; }
350
351
.status.passed .mark, .stage-dot.passed .mark { background: #1aaa55; }
351
352
.status.failed .mark, .stage-dot.failed .mark { background: #dd2b0e; }
352
353
.status.warning .mark, .stage-dot.warning .mark { background: #c17d10; }
353
354
.status.running .mark, .stage-dot.running .mark { background: #1f75cb; }
355
.status.canceled .mark, .stage-dot.canceled .mark { background: #8c959f; }
354
356
.passed .mark::after {
355
357
content: ""; position: absolute; left: 5px; top: 2px; width: 4px; height: 8px;
356
358
border: solid #fff; border-width: 0 2px 2px 0; transform: rotate(45deg);
@@ -366,7 +368,7 @@ svg.ico { width: 14px; height: 14px; flex: none; }
366
368
.warning .mark::after {
367
369
content: ""; position: absolute; left: 7px; top: 11px; width: 2px; height: 2px; background: #fff;
368
370
}
369
.skipped .mark::after {
371
.skipped .mark::after, .canceled .mark::after {
370
372
content: ""; position: absolute; left: 4px; top: 7px; width: 8px; height: 2px; background: #fff;
371
373
}
372
374
.running .mark::after {
@@ -346,11 +346,13 @@ svg.ico { width: 14px; height: 14px; flex: none; }346
.status.failed { background: #fde8e8; color: #9f2d2d; }346
.status.failed { background: #fde8e8; color: #9f2d2d; }347
.status.warning { background: #fff4e0; color: #8a5a00; }347
.status.warning { background: #fff4e0; color: #8a5a00; }348
.status.running { background: #e7f0ff; color: #1f6feb; }348
.status.running { background: #e7f0ff; color: #1f6feb; }349
.status.canceled { background: #f3f0f0; color: #57606a; }349
.mark { position: relative; width: 16px; height: 16px; border-radius: 50%; display: inline-block; flex: none; background: #8c959f; }350
.mark { position: relative; width: 16px; height: 16px; border-radius: 50%; display: inline-block; flex: none; background: #8c959f; }350
.status.passed .mark, .stage-dot.passed .mark { background: #1aaa55; }351
.status.passed .mark, .stage-dot.passed .mark { background: #1aaa55; }351
.status.failed .mark, .stage-dot.failed .mark { background: #dd2b0e; }352
.status.failed .mark, .stage-dot.failed .mark { background: #dd2b0e; }352
.status.warning .mark, .stage-dot.warning .mark { background: #c17d10; }353
.status.warning .mark, .stage-dot.warning .mark { background: #c17d10; }353
.status.running .mark, .stage-dot.running .mark { background: #1f75cb; }354
.status.running .mark, .stage-dot.running .mark { background: #1f75cb; }355
.status.canceled .mark, .stage-dot.canceled .mark { background: #8c959f; }354
.passed .mark::after {356
.passed .mark::after {355
content: ""; position: absolute; left: 5px; top: 2px; width: 4px; height: 8px;357
content: ""; position: absolute; left: 5px; top: 2px; width: 4px; height: 8px;356
border: solid #fff; border-width: 0 2px 2px 0; transform: rotate(45deg);358
border: solid #fff; border-width: 0 2px 2px 0; transform: rotate(45deg);@@ -366,7 +368,7 @@ svg.ico { width: 14px; height: 14px; flex: none; }366
.warning .mark::after {368
.warning .mark::after {367
content: ""; position: absolute; left: 7px; top: 11px; width: 2px; height: 2px; background: #fff;369
content: ""; position: absolute; left: 7px; top: 11px; width: 2px; height: 2px; background: #fff;368
}370
}369
.skipped .mark::after {371
.skipped .mark::after, .canceled .mark::after {370
content: ""; position: absolute; left: 4px; top: 7px; width: 8px; height: 2px; background: #fff;372
content: ""; position: absolute; left: 4px; top: 7px; width: 8px; height: 2px; background: #fff;371
}373
}372
.running .mark::after {374
.running .mark::after {@@ -239,6 +239,7 @@
239
239
{{template "repotabs" .}}
240
240
{{template "pipenav" .}}
241
241
<p><span class="pill">{{.Pipe.Status}}</span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>
242
<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>
242
243
<ul class="clean">
243
244
{{range .Jobs}}<li><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="pill">{{.Status}}</span> <span class="muted">{{.Stage}}</span></li>{{end}}
244
245
</ul>
@@ -273,6 +274,7 @@
273
274
</div>
274
275
</div>
275
276
<div class="console" data-log>
277
<p class="lognote">{{t "pipe.runningOn" $.RunnerName}}</p>
276
278
<p class="lognote">{{t "pipe.utc"}}</p>
277
279
{{range .Blocks}}{{if .Section}}
278
280
<section class="logsec{{if .Open}} open{{end}}">
@@ -291,7 +293,8 @@
291
293
<dt>{{t "pipe.finished"}}</dt><dd>{{if .Finished}}{{.Finished}}{{else}}—{{end}}</dd>
292
294
<dt>{{t "pipe.queued"}}</dt><dd>{{if .Queued}}{{.Queued}}{{else}}—{{end}}</dd>
293
295
<dt>{{t "pipe.timeout"}}</dt><dd>{{.Timeout}}</dd>
294
<dt>{{t "pipe.runner"}}</dt><dd>{{t "pipe.runnerHere"}}</dd>
296
{{if .Environment}}<dt>{{t "pipe.environment"}}</dt><dd>{{.Environment}}{{if .EnvironmentLink}} <a href="{{.EnvironmentLink}}">{{.EnvironmentURL}}</a>{{else if .EnvironmentURL}} {{.EnvironmentURL}}{{end}}</dd>{{end}}
297
<dt>{{t "pipe.runner"}}</dt><dd>{{$.RunnerName}}</dd>
295
298
<dt>{{t "pipe.source"}}</dt><dd>{{.Source}}</dd>
296
299
<dt>{{t "pipe.commit"}}</dt><dd><code>{{.Short}}</code> <button type="button" class="iconbtn" data-copy="{{.Pipe.CommitSHA}}" aria-label="{{t "action.copy"}}">{{t "action.copy"}}</button>{{if .Title}}<div>{{.Title}}</div>{{end}}</dd>
297
300
<dt>{{t "pipe.one"}}</dt><dd><a href="/{{$.Repo.FullPath}}/-/pipelines/{{.Pipe.ID}}">#{{.Pipe.ID}}</a> <span class="status {{.PipeStatus}}"><span class="mark"></span>{{.PipeLabel}}</span> <span class="muted">{{t "pipe.for"}} {{.Ref}}</span></dd>
@@ -239,6 +239,7 @@239
{{template "repotabs" .}}239
{{template "repotabs" .}}240
{{template "pipenav" .}}240
{{template "pipenav" .}}241
<p><span class="pill">{{.Pipe.Status}}</span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>241
<p><span class="pill">{{.Pipe.Status}}</span> {{.Pipe.Ref}} <a href="/{{.Repo.FullPath}}/-/commit/{{.Pipe.CommitSHA}}"><code>{{.Pipe.CommitSHA}}</code></a></p>242
<p class="muted">{{t "pipe.runningOn" .RunnerName}}</p>242
<ul class="clean">243
<ul class="clean">243
{{range .Jobs}}<li><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="pill">{{.Status}}</span> <span class="muted">{{.Stage}}</span></li>{{end}}244
{{range .Jobs}}<li><a href="/{{$.Repo.FullPath}}/-/pipelines/{{$.Pipe.ID}}/jobs/{{.ID}}">{{.Name}}</a> <span class="pill">{{.Status}}</span> <span class="muted">{{.Stage}}</span></li>{{end}}244
</ul>245
</ul>@@ -273,6 +274,7 @@273
</div>274
</div>274
</div>275
</div>275
<div class="console" data-log>276
<div class="console" data-log>277
<p class="lognote">{{t "pipe.runningOn" $.RunnerName}}</p>276
<p class="lognote">{{t "pipe.utc"}}</p>278
<p class="lognote">{{t "pipe.utc"}}</p>277
{{range .Blocks}}{{if .Section}}279
{{range .Blocks}}{{if .Section}}278
<section class="logsec{{if .Open}} open{{end}}">280
<section class="logsec{{if .Open}} open{{end}}">@@ -291,7 +293,8 @@291
<dt>{{t "pipe.finished"}}</dt><dd>{{if .Finished}}{{.Finished}}{{else}}—{{end}}</dd>293
<dt>{{t "pipe.finished"}}</dt><dd>{{if .Finished}}{{.Finished}}{{else}}—{{end}}</dd>292
<dt>{{t "pipe.queued"}}</dt><dd>{{if .Queued}}{{.Queued}}{{else}}—{{end}}</dd>294
<dt>{{t "pipe.queued"}}</dt><dd>{{if .Queued}}{{.Queued}}{{else}}—{{end}}</dd>293
<dt>{{t "pipe.timeout"}}</dt><dd>{{.Timeout}}</dd>295
<dt>{{t "pipe.timeout"}}</dt><dd>{{.Timeout}}</dd>294
<dt>{{t "pipe.runner"}}</dt><dd>{{t "pipe.runnerHere"}}</dd>296
{{if .Environment}}<dt>{{t "pipe.environment"}}</dt><dd>{{.Environment}}{{if .EnvironmentLink}} <a href="{{.EnvironmentLink}}">{{.EnvironmentURL}}</a>{{else if .EnvironmentURL}} {{.EnvironmentURL}}{{end}}</dd>{{end}}297
<dt>{{t "pipe.runner"}}</dt><dd>{{$.RunnerName}}</dd>295
<dt>{{t "pipe.source"}}</dt><dd>{{.Source}}</dd>298
<dt>{{t "pipe.source"}}</dt><dd>{{.Source}}</dd>296
<dt>{{t "pipe.commit"}}</dt><dd><code>{{.Short}}</code> <button type="button" class="iconbtn" data-copy="{{.Pipe.CommitSHA}}" aria-label="{{t "action.copy"}}">{{t "action.copy"}}</button>{{if .Title}}<div>{{.Title}}</div>{{end}}</dd>299
<dt>{{t "pipe.commit"}}</dt><dd><code>{{.Short}}</code> <button type="button" class="iconbtn" data-copy="{{.Pipe.CommitSHA}}" aria-label="{{t "action.copy"}}">{{t "action.copy"}}</button>{{if .Title}}<div>{{.Title}}</div>{{end}}</dd>297
<dt>{{t "pipe.one"}}</dt><dd><a href="/{{$.Repo.FullPath}}/-/pipelines/{{.Pipe.ID}}">#{{.Pipe.ID}}</a> <span class="status {{.PipeStatus}}"><span class="mark"></span>{{.PipeLabel}}</span> <span class="muted">{{t "pipe.for"}} {{.Ref}}</span></dd>300
<dt>{{t "pipe.one"}}</dt><dd><a href="/{{$.Repo.FullPath}}/-/pipelines/{{.Pipe.ID}}">#{{.Pipe.ID}}</a> <span class="status {{.PipeStatus}}"><span class="mark"></span>{{.PipeLabel}}</span> <span class="muted">{{t "pipe.for"}} {{.Ref}}</span></dd>@@ -231,7 +231,19 @@ func (s *Store) SetMergeRequestStatus(ctx context.Context, id int64, status stri
231
231
232
232
// OpenMergeRequestsBySource geeft open verzoeken waarvan de bronbranch deze is.
233
233
func (s *Store) OpenMergeRequestsBySource(ctx context.Context, repoID int64, branch string) ([]MergeRequest, error) {
234
rows, err := s.db.QueryContext(ctx, mrSelect+` WHERE m.repo_id = ? AND m.source_branch = ? AND m.status = 'open' ORDER BY m.number`, repoID, branch)
234
return s.openMergeRequests(ctx, repoID, "source_branch", branch)
235
}
236
237
// OpenMergeRequestsByTarget geeft open verzoeken waarvan de doelbranch deze is.
238
func (s *Store) OpenMergeRequestsByTarget(ctx context.Context, repoID int64, branch string) ([]MergeRequest, error) {
239
return s.openMergeRequests(ctx, repoID, "target_branch", branch)
240
}
241
242
func (s *Store) openMergeRequests(ctx context.Context, repoID int64, column, branch string) ([]MergeRequest, error) {
243
if column != "source_branch" && column != "target_branch" {
244
return nil, errNotFound
245
}
246
rows, err := s.db.QueryContext(ctx, mrSelect+` WHERE m.repo_id = ? AND m.`+column+` = ? AND m.status = 'open' ORDER BY m.number`, repoID, branch)
235
247
if err != nil {
236
248
return nil, err
237
249
}
@@ -280,7 +292,7 @@ func PipelineBlocksMerge(jobs []Job) string {
280
292
for _, j := range jobs {
281
293
switch j.Status {
282
294
case "success", "skipped":
283
case "failed":
295
case "failed", "canceled":
284
296
if !j.AllowFailure {
285
297
hard = true
286
298
}
@@ -319,6 +331,21 @@ type NewJob struct {
319
331
CachePaths []string
320
332
CacheKey string
321
333
ResourceGroup string
334
RetryMax int
335
RetryWhen []string
336
Services []Service
337
Environment string
338
EnvironmentURL string
339
Interruptible bool
340
}
341
342
// Service is een sidecar die de runner naast de job start.
343
type Service struct {
344
Name string `json:"name"`
345
Alias string `json:"alias"`
346
Command []string `json:"command,omitempty"`
347
Entrypoint []string `json:"entrypoint,omitempty"`
348
Variables map[string]string `json:"variables,omitempty"`
322
349
}
323
350
324
351
// Pipeline is een draai van blue-ci-cd.yml.
@@ -363,6 +390,13 @@ type Job struct {
363
390
CachePaths []string
364
391
CacheKey string
365
392
ResourceGroup string
393
RetryMax int
394
RetryWhen []string
395
Attempt int
396
Services []Service
397
Environment string
398
EnvironmentURL string
399
Interruptible bool
366
400
Log string
367
401
CommitSHA string
368
402
Ref string
@@ -428,14 +462,21 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour
428
462
if j.When == "" {
429
463
j.When = "on_success"
430
464
}
465
if j.RetryWhen == nil {
466
j.RetryWhen = []string{}
467
}
468
if j.Services == nil {
469
j.Services = []Service{}
470
}
431
471
status := "queued"
432
472
if j.When == "manual" {
433
473
status = "manual"
434
474
}
435
_, err := tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, before_script, after_script, when_mode, timeout_seconds, tags_json, cache_paths, cache_key, resource_group, optional_json)
436
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, '', ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
475
_, err := tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, before_script, after_script, when_mode, timeout_seconds, tags_json, cache_paths, cache_key, resource_group, optional_json, retry_max, retry_when, services_json, environment_name, environment_url, interruptible)
476
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, '', ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,
437
477
id, repoID, j.Name, j.Stage, j.Position, status, j.Image, j.Script, mustJSON(j.Needs), boolInt(j.NeedsSet), mustJSON(j.Variables), mustJSON(j.ArtifactPaths), int(j.ArtifactExpire/time.Second), boolInt(j.AllowFailure),
438
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional))
478
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),
479
j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))
439
480
if err != nil {
440
481
return 0, err
441
482
}
@@ -513,19 +554,20 @@ func (s *Store) JobsByPipeline(ctx context.Context, pipelineID int64) ([]Job, er
513
554
return out, rows.Err()
514
555
}
515
556
516
const jobSelect = `SELECT j.id, j.pipeline_id, j.repo_id, j.name, j.stage, j.position, j.status, j.image, j.script, j.needs_json, j.needs_set, j.variables_json, j.artifact_paths, j.artifact_expire_seconds, j.allow_failure, j.log, p.commit_sha, p.ref, j.before_script, j.after_script, j.when_mode, j.timeout_seconds, j.tags_json, j.cache_paths, j.cache_key, j.resource_group, j.optional_json, j.started_at, j.finished_at
557
const jobSelect = `SELECT j.id, j.pipeline_id, j.repo_id, j.name, j.stage, j.position, j.status, j.image, j.script, j.needs_json, j.needs_set, j.variables_json, j.artifact_paths, j.artifact_expire_seconds, j.allow_failure, j.log, p.commit_sha, p.ref, j.before_script, j.after_script, j.when_mode, j.timeout_seconds, j.tags_json, j.cache_paths, j.cache_key, j.resource_group, j.optional_json, j.started_at, j.finished_at, j.retry_max, j.retry_when, j.attempt, j.services_json, j.environment_name, j.environment_url, j.interruptible
517
558
FROM jobs j JOIN pipelines p ON p.id = j.pipeline_id`
518
559
519
560
func scanJob(rows *sql.Rows) (Job, error) {
520
561
var j Job
521
var needs, vars, paths, tags, cache, optional string
522
var needsSet, allow, expire, timeout int
562
var needs, vars, paths, tags, cache, optional, retryWhen, services string
563
var needsSet, allow, expire, timeout, interruptible int
523
564
var started, finished sql.NullString
524
if err := rows.Scan(&j.ID, &j.PipelineID, &j.RepoID, &j.Name, &j.Stage, &j.Position, &j.Status, &j.Image, &j.Script, &needs, &needsSet, &vars, &paths, &expire, &allow, &j.Log, &j.CommitSHA, &j.Ref, &j.Before, &j.After, &j.When, &timeout, &tags, &cache, &j.CacheKey, &j.ResourceGroup, &optional, &started, &finished); err != nil {
565
if err := rows.Scan(&j.ID, &j.PipelineID, &j.RepoID, &j.Name, &j.Stage, &j.Position, &j.Status, &j.Image, &j.Script, &needs, &needsSet, &vars, &paths, &expire, &allow, &j.Log, &j.CommitSHA, &j.Ref, &j.Before, &j.After, &j.When, &timeout, &tags, &cache, &j.CacheKey, &j.ResourceGroup, &optional, &started, &finished, &j.RetryMax, &retryWhen, &j.Attempt, &services, &j.Environment, &j.EnvironmentURL, &interruptible); err != nil {
525
566
return Job{}, err
526
567
}
527
568
j.NeedsSet = needsSet == 1
528
569
j.AllowFailure = allow == 1
570
j.Interruptible = interruptible == 1
529
571
j.ArtifactExpire = time.Duration(expire) * time.Second
530
572
j.Timeout = time.Duration(timeout) * time.Second
531
573
j.StartedAt = parseTime(started.String)
@@ -536,6 +578,8 @@ func scanJob(rows *sql.Rows) (Job, error) {
536
578
_ = json.Unmarshal([]byte(tags), &j.Tags)
537
579
_ = json.Unmarshal([]byte(cache), &j.CachePaths)
538
580
_ = json.Unmarshal([]byte(optional), &j.Optional)
581
_ = json.Unmarshal([]byte(retryWhen), &j.RetryWhen)
582
_ = json.Unmarshal([]byte(services), &j.Services)
539
583
return j, nil
540
584
}
541
585
@@ -582,6 +626,55 @@ func (s *Store) AppendJobLog(ctx context.Context, id int64, chunk string) error
582
626
return err
583
627
}
584
628
629
// RequeueJob zet een mislukte job terug in de wachtrij voor een nieuwe poging.
630
func (s *Store) RequeueJob(ctx context.Context, id int64, attempt int, note string) error {
631
_, err := s.db.ExecContext(ctx, `UPDATE jobs SET status = 'queued', attempt = ?, log = log || ?, finished_at = NULL, started_at = NULL WHERE id = ?`, attempt, note, id)
632
return err
633
}
634
635
// CancelInterruptible annuleert onderbreekbare jobs van oudere pipelines op dezelfde ref.
636
// Lopende jobs komen terug zodat de runner hun context kan stoppen.
637
func (s *Store) CancelInterruptible(ctx context.Context, repoID int64, ref string, keep int64) (running []int64, pipes []int64, err error) {
638
rows, err := s.db.QueryContext(ctx, `SELECT j.id, j.pipeline_id, j.status FROM jobs j
639
JOIN pipelines p ON p.id = j.pipeline_id
640
WHERE p.repo_id = ? AND p.ref = ? AND p.id != ? AND j.interruptible = 1 AND j.status IN ('queued', 'running', 'manual')`, repoID, ref, keep)
641
if err != nil {
642
return nil, nil, err
643
}
644
defer rows.Close()
645
type row struct {
646
id, pipe int64
647
status string
648
}
649
var found []row
650
seenPipe := map[int64]bool{}
651
for rows.Next() {
652
var item row
653
if err := rows.Scan(&item.id, &item.pipe, &item.status); err != nil {
654
return nil, nil, err
655
}
656
found = append(found, item)
657
if !seenPipe[item.pipe] {
658
seenPipe[item.pipe] = true
659
pipes = append(pipes, item.pipe)
660
}
661
}
662
if err := rows.Err(); err != nil {
663
return nil, nil, err
664
}
665
note := time.Now().UTC().Format(time.RFC3339) + "\tcanceled because a newer pipeline started for the same ref\n"
666
for _, item := range found {
667
if item.status == "running" {
668
running = append(running, item.id)
669
continue
670
}
671
if _, err := s.db.ExecContext(ctx, `UPDATE jobs SET status = 'canceled', log = log || ?, finished_at = ? WHERE id = ? AND status IN ('queued', 'manual')`, note, now(), item.id); err != nil {
672
return running, pipes, err
673
}
674
}
675
return running, pipes, nil
676
}
677
585
678
// RefreshPipelineStatus berekent de pipelinestatus uit de jobs.
586
679
func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (string, error) {
587
680
jobs, err := s.JobsByPipeline(ctx, pipelineID)
@@ -591,12 +684,17 @@ func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (st
591
684
status := "success"
592
685
open := false
593
686
manual := false
687
canceled := false
594
688
for _, j := range jobs {
595
689
switch j.Status {
596
690
case "failed":
597
691
if !j.AllowFailure {
598
692
status = "failed"
599
693
}
694
case "canceled":
695
if !j.AllowFailure {
696
canceled = true
697
}
600
698
case "queued", "running":
601
699
open = true
602
700
case "manual":
@@ -611,6 +709,9 @@ func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (st
611
709
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'running', finished_at = NULL WHERE id = ?`, pipelineID)
612
710
case status == "failed":
613
711
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'failed', finished_at = ? WHERE id = ?`, now(), pipelineID)
712
case canceled:
713
status = "canceled"
714
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'canceled', finished_at = ? WHERE id = ?`, now(), pipelineID)
614
715
case manual:
615
716
status = "manual"
616
717
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'manual', finished_at = NULL WHERE id = ?`, pipelineID)
@@ -231,7 +231,19 @@ func (s *Store) SetMergeRequestStatus(ctx context.Context, id int64, status stri231
231
232
// OpenMergeRequestsBySource geeft open verzoeken waarvan de bronbranch deze is.232
// OpenMergeRequestsBySource geeft open verzoeken waarvan de bronbranch deze is.233
func (s *Store) OpenMergeRequestsBySource(ctx context.Context, repoID int64, branch string) ([]MergeRequest, error) {233
func (s *Store) OpenMergeRequestsBySource(ctx context.Context, repoID int64, branch string) ([]MergeRequest, error) {234
rows, err := s.db.QueryContext(ctx, mrSelect+` WHERE m.repo_id = ? AND m.source_branch = ? AND m.status = 'open' ORDER BY m.number`, repoID, branch)234
return s.openMergeRequests(ctx, repoID, "source_branch", branch)235
}236
237
// OpenMergeRequestsByTarget geeft open verzoeken waarvan de doelbranch deze is.238
func (s *Store) OpenMergeRequestsByTarget(ctx context.Context, repoID int64, branch string) ([]MergeRequest, error) {239
return s.openMergeRequests(ctx, repoID, "target_branch", branch)240
}241
242
func (s *Store) openMergeRequests(ctx context.Context, repoID int64, column, branch string) ([]MergeRequest, error) {243
if column != "source_branch" && column != "target_branch" {244
return nil, errNotFound245
}246
rows, err := s.db.QueryContext(ctx, mrSelect+` WHERE m.repo_id = ? AND m.`+column+` = ? AND m.status = 'open' ORDER BY m.number`, repoID, branch)235
if err != nil {247
if err != nil {236
return nil, err248
return nil, err237
}249
}@@ -280,7 +292,7 @@ func PipelineBlocksMerge(jobs []Job) string {280
for _, j := range jobs {292
for _, j := range jobs {281
switch j.Status {293
switch j.Status {282
case "success", "skipped":294
case "success", "skipped":283
case "failed":295
case "failed", "canceled":284
if !j.AllowFailure {296
if !j.AllowFailure {285
hard = true297
hard = true286
}298
}@@ -319,6 +331,21 @@ type NewJob struct {319
CachePaths []string331
CachePaths []string320
CacheKey string332
CacheKey string321
ResourceGroup string333
ResourceGroup string334
RetryMax int335
RetryWhen []string336
Services []Service337
Environment string338
EnvironmentURL string339
Interruptible bool340
}341
342
// Service is een sidecar die de runner naast de job start.343
type Service struct {344
Name string `json:"name"`345
Alias string `json:"alias"`346
Command []string `json:"command,omitempty"`347
Entrypoint []string `json:"entrypoint,omitempty"`348
Variables map[string]string `json:"variables,omitempty"`322
}349
}323
350
324
// Pipeline is een draai van blue-ci-cd.yml.351
// Pipeline is een draai van blue-ci-cd.yml.@@ -363,6 +390,13 @@ type Job struct {363
CachePaths []string390
CachePaths []string364
CacheKey string391
CacheKey string365
ResourceGroup string392
ResourceGroup string393
RetryMax int394
RetryWhen []string395
Attempt int396
Services []Service397
Environment string398
EnvironmentURL string399
Interruptible bool366
Log string400
Log string367
CommitSHA string401
CommitSHA string368
Ref string402
Ref string@@ -428,14 +462,21 @@ func (s *Store) CreatePipeline(ctx context.Context, repoID int64, sha, ref, sour428
if j.When == "" {462
if j.When == "" {429
j.When = "on_success"463
j.When = "on_success"430
}464
}465
if j.RetryWhen == nil {466
j.RetryWhen = []string{}467
}468
if j.Services == nil {469
j.Services = []Service{}470
}431
status := "queued"471
status := "queued"432
if j.When == "manual" {472
if j.When == "manual" {433
status = "manual"473
status = "manual"434
}474
}435
_, err := tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, before_script, after_script, when_mode, timeout_seconds, tags_json, cache_paths, cache_key, resource_group, optional_json)475
_, err := tx.ExecContext(ctx, `INSERT INTO jobs (pipeline_id, repo_id, name, stage, position, status, image, script, needs_json, needs_set, variables_json, artifact_paths, artifact_expire_seconds, allow_failure, log, before_script, after_script, when_mode, timeout_seconds, tags_json, cache_paths, cache_key, resource_group, optional_json, retry_max, retry_when, services_json, environment_name, environment_url, interruptible)436
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, '', ?, ?, ?, ?, ?, ?, ?, ?, ?)`,476
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, '', ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)`,437
id, repoID, j.Name, j.Stage, j.Position, status, j.Image, j.Script, mustJSON(j.Needs), boolInt(j.NeedsSet), mustJSON(j.Variables), mustJSON(j.ArtifactPaths), int(j.ArtifactExpire/time.Second), boolInt(j.AllowFailure),477
id, repoID, j.Name, j.Stage, j.Position, status, j.Image, j.Script, mustJSON(j.Needs), boolInt(j.NeedsSet), mustJSON(j.Variables), mustJSON(j.ArtifactPaths), int(j.ArtifactExpire/time.Second), boolInt(j.AllowFailure),438
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional))478
j.Before, j.After, j.When, int(j.Timeout/time.Second), mustJSON(j.Tags), mustJSON(j.CachePaths), j.CacheKey, j.ResourceGroup, mustJSON(j.Optional),479
j.RetryMax, mustJSON(j.RetryWhen), mustJSON(j.Services), j.Environment, j.EnvironmentURL, boolInt(j.Interruptible))439
if err != nil {480
if err != nil {440
return 0, err481
return 0, err441
}482
}@@ -513,19 +554,20 @@ func (s *Store) JobsByPipeline(ctx context.Context, pipelineID int64) ([]Job, er513
return out, rows.Err()554
return out, rows.Err()514
}555
}515
556
516
const jobSelect = `SELECT j.id, j.pipeline_id, j.repo_id, j.name, j.stage, j.position, j.status, j.image, j.script, j.needs_json, j.needs_set, j.variables_json, j.artifact_paths, j.artifact_expire_seconds, j.allow_failure, j.log, p.commit_sha, p.ref, j.before_script, j.after_script, j.when_mode, j.timeout_seconds, j.tags_json, j.cache_paths, j.cache_key, j.resource_group, j.optional_json, j.started_at, j.finished_at557
const jobSelect = `SELECT j.id, j.pipeline_id, j.repo_id, j.name, j.stage, j.position, j.status, j.image, j.script, j.needs_json, j.needs_set, j.variables_json, j.artifact_paths, j.artifact_expire_seconds, j.allow_failure, j.log, p.commit_sha, p.ref, j.before_script, j.after_script, j.when_mode, j.timeout_seconds, j.tags_json, j.cache_paths, j.cache_key, j.resource_group, j.optional_json, j.started_at, j.finished_at, j.retry_max, j.retry_when, j.attempt, j.services_json, j.environment_name, j.environment_url, j.interruptible517
FROM jobs j JOIN pipelines p ON p.id = j.pipeline_id`558
FROM jobs j JOIN pipelines p ON p.id = j.pipeline_id`518
559
519
func scanJob(rows *sql.Rows) (Job, error) {560
func scanJob(rows *sql.Rows) (Job, error) {520
var j Job561
var j Job521
var needs, vars, paths, tags, cache, optional string562
var needs, vars, paths, tags, cache, optional, retryWhen, services string522
var needsSet, allow, expire, timeout int563
var needsSet, allow, expire, timeout, interruptible int523
var started, finished sql.NullString564
var started, finished sql.NullString524
if err := rows.Scan(&j.ID, &j.PipelineID, &j.RepoID, &j.Name, &j.Stage, &j.Position, &j.Status, &j.Image, &j.Script, &needs, &needsSet, &vars, &paths, &expire, &allow, &j.Log, &j.CommitSHA, &j.Ref, &j.Before, &j.After, &j.When, &timeout, &tags, &cache, &j.CacheKey, &j.ResourceGroup, &optional, &started, &finished); err != nil {565
if err := rows.Scan(&j.ID, &j.PipelineID, &j.RepoID, &j.Name, &j.Stage, &j.Position, &j.Status, &j.Image, &j.Script, &needs, &needsSet, &vars, &paths, &expire, &allow, &j.Log, &j.CommitSHA, &j.Ref, &j.Before, &j.After, &j.When, &timeout, &tags, &cache, &j.CacheKey, &j.ResourceGroup, &optional, &started, &finished, &j.RetryMax, &retryWhen, &j.Attempt, &services, &j.Environment, &j.EnvironmentURL, &interruptible); err != nil {525
return Job{}, err566
return Job{}, err526
}567
}527
j.NeedsSet = needsSet == 1568
j.NeedsSet = needsSet == 1528
j.AllowFailure = allow == 1569
j.AllowFailure = allow == 1570
j.Interruptible = interruptible == 1529
j.ArtifactExpire = time.Duration(expire) * time.Second571
j.ArtifactExpire = time.Duration(expire) * time.Second530
j.Timeout = time.Duration(timeout) * time.Second572
j.Timeout = time.Duration(timeout) * time.Second531
j.StartedAt = parseTime(started.String)573
j.StartedAt = parseTime(started.String)@@ -536,6 +578,8 @@ func scanJob(rows *sql.Rows) (Job, error) {536
_ = json.Unmarshal([]byte(tags), &j.Tags)578
_ = json.Unmarshal([]byte(tags), &j.Tags)537
_ = json.Unmarshal([]byte(cache), &j.CachePaths)579
_ = json.Unmarshal([]byte(cache), &j.CachePaths)538
_ = json.Unmarshal([]byte(optional), &j.Optional)580
_ = json.Unmarshal([]byte(optional), &j.Optional)581
_ = json.Unmarshal([]byte(retryWhen), &j.RetryWhen)582
_ = json.Unmarshal([]byte(services), &j.Services)539
return j, nil583
return j, nil540
}584
}541
585
@@ -582,6 +626,55 @@ func (s *Store) AppendJobLog(ctx context.Context, id int64, chunk string) error582
return err626
return err583
}627
}584
628
629
// RequeueJob zet een mislukte job terug in de wachtrij voor een nieuwe poging.630
func (s *Store) RequeueJob(ctx context.Context, id int64, attempt int, note string) error {631
_, err := s.db.ExecContext(ctx, `UPDATE jobs SET status = 'queued', attempt = ?, log = log || ?, finished_at = NULL, started_at = NULL WHERE id = ?`, attempt, note, id)632
return err633
}634
635
// CancelInterruptible annuleert onderbreekbare jobs van oudere pipelines op dezelfde ref.636
// Lopende jobs komen terug zodat de runner hun context kan stoppen.637
func (s *Store) CancelInterruptible(ctx context.Context, repoID int64, ref string, keep int64) (running []int64, pipes []int64, err error) {638
rows, err := s.db.QueryContext(ctx, `SELECT j.id, j.pipeline_id, j.status FROM jobs j639
JOIN pipelines p ON p.id = j.pipeline_id640
WHERE p.repo_id = ? AND p.ref = ? AND p.id != ? AND j.interruptible = 1 AND j.status IN ('queued', 'running', 'manual')`, repoID, ref, keep)641
if err != nil {642
return nil, nil, err643
}644
defer rows.Close()645
type row struct {646
id, pipe int64647
status string648
}649
var found []row650
seenPipe := map[int64]bool{}651
for rows.Next() {652
var item row653
if err := rows.Scan(&item.id, &item.pipe, &item.status); err != nil {654
return nil, nil, err655
}656
found = append(found, item)657
if !seenPipe[item.pipe] {658
seenPipe[item.pipe] = true659
pipes = append(pipes, item.pipe)660
}661
}662
if err := rows.Err(); err != nil {663
return nil, nil, err664
}665
note := time.Now().UTC().Format(time.RFC3339) + "\tcanceled because a newer pipeline started for the same ref\n"666
for _, item := range found {667
if item.status == "running" {668
running = append(running, item.id)669
continue670
}671
if _, err := s.db.ExecContext(ctx, `UPDATE jobs SET status = 'canceled', log = log || ?, finished_at = ? WHERE id = ? AND status IN ('queued', 'manual')`, note, now(), item.id); err != nil {672
return running, pipes, err673
}674
}675
return running, pipes, nil676
}677
585
// RefreshPipelineStatus berekent de pipelinestatus uit de jobs.678
// RefreshPipelineStatus berekent de pipelinestatus uit de jobs.586
func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (string, error) {679
func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (string, error) {587
jobs, err := s.JobsByPipeline(ctx, pipelineID)680
jobs, err := s.JobsByPipeline(ctx, pipelineID)@@ -591,12 +684,17 @@ func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (st591
status := "success"684
status := "success"592
open := false685
open := false593
manual := false686
manual := false687
canceled := false594
for _, j := range jobs {688
for _, j := range jobs {595
switch j.Status {689
switch j.Status {596
case "failed":690
case "failed":597
if !j.AllowFailure {691
if !j.AllowFailure {598
status = "failed"692
status = "failed"599
}693
}694
case "canceled":695
if !j.AllowFailure {696
canceled = true697
}600
case "queued", "running":698
case "queued", "running":601
open = true699
open = true602
case "manual":700
case "manual":@@ -611,6 +709,9 @@ func (s *Store) RefreshPipelineStatus(ctx context.Context, pipelineID int64) (st611
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'running', finished_at = NULL WHERE id = ?`, pipelineID)709
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'running', finished_at = NULL WHERE id = ?`, pipelineID)612
case status == "failed":710
case status == "failed":613
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'failed', finished_at = ? WHERE id = ?`, now(), pipelineID)711
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'failed', finished_at = ? WHERE id = ?`, now(), pipelineID)712
case canceled:713
status = "canceled"714
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'canceled', finished_at = ? WHERE id = ?`, now(), pipelineID)614
case manual:715
case manual:615
status = "manual"716
status = "manual"616
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'manual', finished_at = NULL WHERE id = ?`, pipelineID)717
_, err = s.db.ExecContext(ctx, `UPDATE pipelines SET status = 'manual', finished_at = NULL WHERE id = ?`, pipelineID)@@ -21,6 +21,9 @@ func TestPipelineBlocksMerge(t *testing.T) {
21
21
if PipelineBlocksMerge([]Job{{Status: "failed"}}) != "failed" {
22
22
t.Fatal("harde fout")
23
23
}
24
if PipelineBlocksMerge([]Job{{Status: "canceled"}}) != "failed" {
25
t.Fatal("geannuleerd houdt de merge tegen")
26
}
24
27
}
25
28
26
29
func TestRefreshWaitsForQueuedJobs(t *testing.T) {
@@ -68,3 +71,81 @@ func TestRefreshWaitsForQueuedJobs(t *testing.T) {
68
71
t.Fatalf("na het overslaan: %s %v", status, err)
69
72
}
70
73
}
74
75
func TestCancelInterruptibleAndRequeue(t *testing.T) {
76
s := openTest(t)
77
ctx := context.Background()
78
tim, err := s.UpsertUser(ctx, 1, "tim", "", false)
79
if err != nil {
80
t.Fatal(err)
81
}
82
repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")
83
if err != nil {
84
t.Fatal(err)
85
}
86
first, err := s.CreatePipeline(ctx, repo.ID, "abc", "refs/heads/master", "push", "tim", "eerste", []NewJob{
87
{Name: "stop", Stage: "test", Script: "echo", Interruptible: true},
88
{Name: "keep", Stage: "test", Script: "echo"},
89
})
90
if err != nil {
91
t.Fatal(err)
92
}
93
second, err := s.CreatePipeline(ctx, repo.ID, "def", "refs/heads/master", "push", "tim", "tweede", []NewJob{
94
{Name: "next", Stage: "test", Script: "echo", Interruptible: true},
95
})
96
if err != nil {
97
t.Fatal(err)
98
}
99
running, pipes, err := s.CancelInterruptible(ctx, repo.ID, "refs/heads/master", second)
100
if err != nil {
101
t.Fatal(err)
102
}
103
if len(running) != 0 || len(pipes) != 1 || pipes[0] != first {
104
t.Fatalf("running %v pipes %v", running, pipes)
105
}
106
jobs, err := s.JobsByPipeline(ctx, first)
107
if err != nil {
108
t.Fatal(err)
109
}
110
for _, j := range jobs {
111
if j.Name == "stop" && j.Status != "canceled" {
112
t.Fatalf("stop: %s", j.Status)
113
}
114
if j.Name == "keep" && j.Status != "queued" {
115
t.Fatalf("keep: %s", j.Status)
116
}
117
}
118
if err := s.FinishJob(ctx, jobs[0].ID, "failed", "nope\n"); err != nil {
119
t.Fatal(err)
120
}
121
if err := s.RequeueJob(ctx, jobs[0].ID, 1, "retry\n"); err != nil {
122
t.Fatal(err)
123
}
124
again, err := s.JobsByPipeline(ctx, first)
125
if err != nil {
126
t.Fatal(err)
127
}
128
if again[0].Status != "queued" || again[0].Attempt != 1 {
129
t.Fatalf("%s attempt %d", again[0].Status, again[0].Attempt)
130
}
131
status, err := s.RefreshPipelineStatus(ctx, first)
132
if err != nil || status != "running" {
133
t.Fatalf("na retry: %s %v", status, err)
134
}
135
for _, j := range again {
136
if j.Name == "keep" {
137
if err := s.SkipJob(ctx, j.ID, "skip"); err != nil {
138
t.Fatal(err)
139
}
140
}
141
if j.Name == "stop" {
142
if err := s.FinishJob(ctx, j.ID, "canceled", ""); err != nil {
143
t.Fatal(err)
144
}
145
}
146
}
147
status, err = s.RefreshPipelineStatus(ctx, first)
148
if err != nil || status != "canceled" {
149
t.Fatalf("pipeline: %s %v", status, err)
150
}
151
}
@@ -21,6 +21,9 @@ func TestPipelineBlocksMerge(t *testing.T) {21
if PipelineBlocksMerge([]Job{{Status: "failed"}}) != "failed" {21
if PipelineBlocksMerge([]Job{{Status: "failed"}}) != "failed" {22
t.Fatal("harde fout")22
t.Fatal("harde fout")23
}23
}24
if PipelineBlocksMerge([]Job{{Status: "canceled"}}) != "failed" {25
t.Fatal("geannuleerd houdt de merge tegen")26
}24
}27
}25
28
26
func TestRefreshWaitsForQueuedJobs(t *testing.T) {29
func TestRefreshWaitsForQueuedJobs(t *testing.T) {@@ -68,3 +71,81 @@ func TestRefreshWaitsForQueuedJobs(t *testing.T) {68
t.Fatalf("na het overslaan: %s %v", status, err)71
t.Fatalf("na het overslaan: %s %v", status, err)69
}72
}70
}73
}74
75
func TestCancelInterruptibleAndRequeue(t *testing.T) {76
s := openTest(t)77
ctx := context.Background()78
tim, err := s.UpsertUser(ctx, 1, "tim", "", false)79
if err != nil {80
t.Fatal(err)81
}82
repo, err := s.CreateRepo(ctx, "user", tim.ID, "ci", "", "master", "private")83
if err != nil {84
t.Fatal(err)85
}86
first, err := s.CreatePipeline(ctx, repo.ID, "abc", "refs/heads/master", "push", "tim", "eerste", []NewJob{87
{Name: "stop", Stage: "test", Script: "echo", Interruptible: true},88
{Name: "keep", Stage: "test", Script: "echo"},89
})90
if err != nil {91
t.Fatal(err)92
}93
second, err := s.CreatePipeline(ctx, repo.ID, "def", "refs/heads/master", "push", "tim", "tweede", []NewJob{94
{Name: "next", Stage: "test", Script: "echo", Interruptible: true},95
})96
if err != nil {97
t.Fatal(err)98
}99
running, pipes, err := s.CancelInterruptible(ctx, repo.ID, "refs/heads/master", second)100
if err != nil {101
t.Fatal(err)102
}103
if len(running) != 0 || len(pipes) != 1 || pipes[0] != first {104
t.Fatalf("running %v pipes %v", running, pipes)105
}106
jobs, err := s.JobsByPipeline(ctx, first)107
if err != nil {108
t.Fatal(err)109
}110
for _, j := range jobs {111
if j.Name == "stop" && j.Status != "canceled" {112
t.Fatalf("stop: %s", j.Status)113
}114
if j.Name == "keep" && j.Status != "queued" {115
t.Fatalf("keep: %s", j.Status)116
}117
}118
if err := s.FinishJob(ctx, jobs[0].ID, "failed", "nope\n"); err != nil {119
t.Fatal(err)120
}121
if err := s.RequeueJob(ctx, jobs[0].ID, 1, "retry\n"); err != nil {122
t.Fatal(err)123
}124
again, err := s.JobsByPipeline(ctx, first)125
if err != nil {126
t.Fatal(err)127
}128
if again[0].Status != "queued" || again[0].Attempt != 1 {129
t.Fatalf("%s attempt %d", again[0].Status, again[0].Attempt)130
}131
status, err := s.RefreshPipelineStatus(ctx, first)132
if err != nil || status != "running" {133
t.Fatalf("na retry: %s %v", status, err)134
}135
for _, j := range again {136
if j.Name == "keep" {137
if err := s.SkipJob(ctx, j.ID, "skip"); err != nil {138
t.Fatal(err)139
}140
}141
if j.Name == "stop" {142
if err := s.FinishJob(ctx, j.ID, "canceled", ""); err != nil {143
t.Fatal(err)144
}145
}146
}147
status, err = s.RefreshPipelineStatus(ctx, first)148
if err != nil || status != "canceled" {149
t.Fatalf("pipeline: %s %v", status, err)150
}151
}@@ -81,6 +81,13 @@ func (s *Store) migrate() error {
81
81
`ALTER TABLE jobs ADD COLUMN cache_key TEXT NOT NULL DEFAULT ''`,
82
82
`ALTER TABLE jobs ADD COLUMN resource_group TEXT NOT NULL DEFAULT ''`,
83
83
`ALTER TABLE jobs ADD COLUMN optional_json TEXT NOT NULL DEFAULT '[]'`,
84
`ALTER TABLE jobs ADD COLUMN retry_max INTEGER NOT NULL DEFAULT 0`,
85
`ALTER TABLE jobs ADD COLUMN retry_when TEXT NOT NULL DEFAULT '[]'`,
86
`ALTER TABLE jobs ADD COLUMN attempt INTEGER NOT NULL DEFAULT 0`,
87
`ALTER TABLE jobs ADD COLUMN services_json TEXT NOT NULL DEFAULT '[]'`,
88
`ALTER TABLE jobs ADD COLUMN environment_name TEXT NOT NULL DEFAULT ''`,
89
`ALTER TABLE jobs ADD COLUMN environment_url TEXT NOT NULL DEFAULT ''`,
90
`ALTER TABLE jobs ADD COLUMN interruptible INTEGER NOT NULL DEFAULT 0`,
84
91
`ALTER TABLE pipelines ADD COLUMN mr_id INTEGER NOT NULL DEFAULT 0`,
85
92
`ALTER TABLE groups ADD COLUMN owner_id INTEGER NOT NULL DEFAULT 0`,
86
93
}
@@ -271,6 +278,13 @@ CREATE TABLE IF NOT EXISTS jobs (
271
278
cache_key TEXT NOT NULL DEFAULT '',
272
279
resource_group TEXT NOT NULL DEFAULT '',
273
280
optional_json TEXT NOT NULL DEFAULT '[]',
281
retry_max INTEGER NOT NULL DEFAULT 0,
282
retry_when TEXT NOT NULL DEFAULT '[]',
283
attempt INTEGER NOT NULL DEFAULT 0,
284
services_json TEXT NOT NULL DEFAULT '[]',
285
environment_name TEXT NOT NULL DEFAULT '',
286
environment_url TEXT NOT NULL DEFAULT '',
287
interruptible INTEGER NOT NULL DEFAULT 0,
274
288
UNIQUE(pipeline_id, name)
275
289
);
276
290
CREATE TABLE IF NOT EXISTS artifacts (
@@ -81,6 +81,13 @@ func (s *Store) migrate() error {81
`ALTER TABLE jobs ADD COLUMN cache_key TEXT NOT NULL DEFAULT ''`,81
`ALTER TABLE jobs ADD COLUMN cache_key TEXT NOT NULL DEFAULT ''`,82
`ALTER TABLE jobs ADD COLUMN resource_group TEXT NOT NULL DEFAULT ''`,82
`ALTER TABLE jobs ADD COLUMN resource_group TEXT NOT NULL DEFAULT ''`,83
`ALTER TABLE jobs ADD COLUMN optional_json TEXT NOT NULL DEFAULT '[]'`,83
`ALTER TABLE jobs ADD COLUMN optional_json TEXT NOT NULL DEFAULT '[]'`,84
`ALTER TABLE jobs ADD COLUMN retry_max INTEGER NOT NULL DEFAULT 0`,85
`ALTER TABLE jobs ADD COLUMN retry_when TEXT NOT NULL DEFAULT '[]'`,86
`ALTER TABLE jobs ADD COLUMN attempt INTEGER NOT NULL DEFAULT 0`,87
`ALTER TABLE jobs ADD COLUMN services_json TEXT NOT NULL DEFAULT '[]'`,88
`ALTER TABLE jobs ADD COLUMN environment_name TEXT NOT NULL DEFAULT ''`,89
`ALTER TABLE jobs ADD COLUMN environment_url TEXT NOT NULL DEFAULT ''`,90
`ALTER TABLE jobs ADD COLUMN interruptible INTEGER NOT NULL DEFAULT 0`,84
`ALTER TABLE pipelines ADD COLUMN mr_id INTEGER NOT NULL DEFAULT 0`,91
`ALTER TABLE pipelines ADD COLUMN mr_id INTEGER NOT NULL DEFAULT 0`,85
`ALTER TABLE groups ADD COLUMN owner_id INTEGER NOT NULL DEFAULT 0`,92
`ALTER TABLE groups ADD COLUMN owner_id INTEGER NOT NULL DEFAULT 0`,86
}93
}@@ -271,6 +278,13 @@ CREATE TABLE IF NOT EXISTS jobs (271
cache_key TEXT NOT NULL DEFAULT '',278
cache_key TEXT NOT NULL DEFAULT '',272
resource_group TEXT NOT NULL DEFAULT '',279
resource_group TEXT NOT NULL DEFAULT '',273
optional_json TEXT NOT NULL DEFAULT '[]',280
optional_json TEXT NOT NULL DEFAULT '[]',281
retry_max INTEGER NOT NULL DEFAULT 0,282
retry_when TEXT NOT NULL DEFAULT '[]',283
attempt INTEGER NOT NULL DEFAULT 0,284
services_json TEXT NOT NULL DEFAULT '[]',285
environment_name TEXT NOT NULL DEFAULT '',286
environment_url TEXT NOT NULL DEFAULT '',287
interruptible INTEGER NOT NULL DEFAULT 0,274
UNIQUE(pipeline_id, name)288
UNIQUE(pipeline_id, name)275
);289
);276
CREATE TABLE IF NOT EXISTS artifacts (290
CREATE TABLE IF NOT EXISTS artifacts (@@ -33,6 +33,9 @@ ssh:
33
33
public_host: git.timfalken.com
34
34
public_port: 22
35
35
36
runner:
37
name: Steve
38
36
39
ci:
37
40
enabled: true
38
41
docker_binary: docker
@@ -33,6 +33,9 @@ ssh:33
public_host: git.timfalken.com33
public_host: git.timfalken.com34
public_port: 2234
public_port: 2235
35
36
runner:37
name: Steve38
36
ci:39
ci:37
enabled: true40
enabled: true38
docker_binary: docker41
docker_binary: docker