Prechádzať zdrojové kódy

Uat流水线优化。

dujian 9 hodín pred
rodič
commit
d1c5ea590c

+ 8 - 6
docs/jenkins/README-UAT-HARBOR-PUSH.md

@@ -20,7 +20,7 @@ UAT Jenkins Job(88:30003)
   1. Checkout(拉 alien_cloud 预生产分支)
   2. Maven Build(mvn clean package)
   3. Push images to Harbor(PUSH_TO_HARBOR=true 时)
-       若已有 uat-latest:pull → 再打 tag uat-build-<本次构建号> → push
+       若已有 uat-latest:Harbor API 给旧 artifact 追加 tag uat-build-<N>(同 digest,不重传层)
        docker build → push ...:uat-latest
   4. Deploy Services(拷 jar 到 /app_deploy_uat + docker restart,与现网一致)
 
@@ -34,7 +34,7 @@ Harbor Web(153.68)gateway 等仓库有 uat-latest + 若干 uat-build-*
 
 | 步骤 | 说明 |
 |------|------|
-| 推送前 | 若存在 `uat-latest`,将其归档为 `uat-build-49`(与本次 Jenkins 构建号一致) |
+| 推送前 | 若存在 `uat-latest`,Harbor API 将其 artifact 追加 tag `uat-build-49`(同 digest) |
 | 推送后 | 新镜像仅打 `uat-latest` |
 | 生产晋升 | `SOURCE_TAG=uat-latest` 即「当前 UAT 最新」 |
 
@@ -127,7 +127,8 @@ echo '<TOKEN>' | docker login 39.105.153.68 -u 'robot$alien_cloud+jenkins-k8s' -
 构建成功后日志末尾应有:
 
 ```text
->>> archive previous uat-latest -> uat-build-<N>   # 非首次构建才有
+>>> archive previous uat-latest -> uat-build-<N> (gateway, Harbor API)   # 非首次构建才有
+>>> archived gateway:uat-build-<N> (same digest as prior uat-latest)
 >>> pushed 39.105.153.68/alien_cloud/gateway:uat-latest
 >>> Prod promote: SOURCE_TAG=uat-latest
 ```
@@ -170,16 +171,16 @@ curl -s -u 'robot$alien_cloud+jenkins-k8s:<TOKEN>' \
 
 ### 旧镜像清理说明
 
-- 每次 push 更新 **`uat-latest`**;被顶替的旧 `uat-latest` 归档为 **`uat-build-<本次 Jenkins 构建号>`**。
+- 每次 push 更新 **`uat-latest`**;被顶替的旧 `uat-latest` 经 **Harbor API** 归档为 **`uat-build-<本次 Jenkins 构建号>`**(同一 digest,无需 docker pull/push 归档)
 - 脚本在 push 后调 Harbor API 删除最老的 `uat-build-*`,**保留最近 N 个**;**永不删除 `uat-latest`**。
-- 机器人需有 **删除制品** 权限;若删除失败,日志会有 `WARN: delete failed`。
+- 机器人需有 **推送/创建 tag** 权限;prune 删旧 tag 还需 **删除制品** 权限(若删除失败,日志会有 `WARN: delete failed`
 - **不会**删除 `base/openjdk8-ffmpeg` 等基础镜像。
 - Jenkins 本机 `docker images` 可用 `docker image prune` 另行清理,与 Harbor 无关。
 
 | 环境变量 | 说明 |
 |----------|------|
 | `UAT_HARBOR_LATEST_TAG` | 固定 `uat-latest` |
-| `UAT_HARBOR_BUILD_TAG` | 本次归档名 `uat-build-${BUILD_NUMBER}`(仅在有旧 `uat-latest` 时 push) |
+| `UAT_HARBOR_BUILD_TAG` | 本次归档名 `uat-build-${BUILD_NUMBER}`(Harbor API 追加到旧 latest 的 artifact) |
 
 ---
 
@@ -193,6 +194,7 @@ curl -s -u 'robot$alien_cloud+jenkins-k8s:<TOKEN>' \
 | Harbor 仍无 gateway | 是否勾选了 `PUSH_TO_HARBOR`;Maven 是否打出 `alien-gateway/target/*.jar` |
 | 生产 Whole `uat-latest: not found` | 先跑一轮 UAT 推 Harbor;`SOURCE_TAG` 填 **`uat-latest`** |
 | 需要回滚到上一版 UAT | 在 Harbor 查 `uat-build-<N>`,生产 Job 填该 tag 作 `SOURCE_TAG` |
+| `Harbor archive ... HTTP 403/404` | 机器人缺 **创建 tag** 权限,或尚无 `uat-latest`(404 可忽略) |
 | `no space left on device`(docker build) | Jenkins 节点磁盘满;见下文 **磁盘不足** |
 
 ### 磁盘不足(`no space left on device`)

+ 51 - 11
docs/jenkins/uat/Jenkinsfile

@@ -6,8 +6,9 @@
  *
  * Harbor (153.68): when PUSH_TO_HARBOR=true, push e.g.
  *   39.105.153.68/alien_cloud/gateway:uat-latest
- * Before push: existing uat-latest is archived as uat-build-<BUILD_NUMBER> (same digest).
- * New image is pushed only as uat-latest. Prod promote: SOURCE_TAG=uat-latest.
+ * Before push: existing uat-latest is archived as uat-build-<BUILD_NUMBER> via Harbor API
+ * (same digest, no docker pull/tag/push). New image is pushed only as uat-latest.
+ * Prod promote: SOURCE_TAG=uat-latest.
  */
 
 /** Normalize GIT_BRANCH: uat-20260202 (not origin/uat-20260202 or refs/heads/...) */
@@ -42,13 +43,52 @@ def filterHarborPushScope(List allServices, String scope) {
     error("Unknown HARBOR_PUSH_SCOPE: ${scope}")
 }
 
+/** Archive current uat-latest as uat-build-N via Harbor API (same digest, no layer re-upload). */
+def archiveHarborLatestViaApi(def script, String reg, String proj, String repo, String latestTag, String buildTag) {
+    script.sh """
+        set -e
+        REG='${reg}'
+        PROJ='${proj}'
+        REPO='${repo}'
+        LATEST='${latestTag}'
+        BUILD_TAG='${buildTag}'
+        if ! command -v jq >/dev/null 2>&1; then
+          echo '>>> WARN: jq missing, skip Harbor API archive for '\${REPO}
+          exit 0
+        fi
+        enc_repo=\$(printf '%s' "\${REPO}" | jq -sRr @uri)
+        if ! curl -fsS -u "\${HARBOR_USER}:\${HARBOR_PASS}" \\
+          "http://\${REG}/api/v2.0/projects/\${PROJ}/repositories/\${enc_repo}/artifacts/\${LATEST}" >/dev/null 2>&1; then
+          echo ">>> no prior \${LATEST} for \${REPO}, skip archive"
+          exit 0
+        fi
+        echo ">>> archive previous \${LATEST} -> \${BUILD_TAG} (\${REPO}, Harbor API)"
+        archive_code=\$(curl -sS -o /tmp/harbor_archive_\${REPO}.json -w '%{http_code}' \\
+          -X POST -u "\${HARBOR_USER}:\${HARBOR_PASS}" \\
+          "http://\${REG}/api/v2.0/projects/\${PROJ}/repositories/\${enc_repo}/artifacts/\${LATEST}/tags" \\
+          -H 'Content-Type: application/json' \\
+          -d "{\\"name\\":\\"\${BUILD_TAG}\\"}")
+        case "\${archive_code}" in
+          201|200)
+            echo ">>> archived \${REPO}:\${BUILD_TAG} (same digest as prior \${LATEST})"
+            ;;
+          409)
+            echo ">>> WARN: \${REPO}:\${BUILD_TAG} already exists, continue"
+            ;;
+          *)
+            echo ">>> ERROR: Harbor archive \${REPO} HTTP \${archive_code}"
+            cat /tmp/harbor_archive_\${REPO}.json 2>/dev/null || true
+            exit 1
+            ;;
+        esac
+    """
+}
+
 def pushOneHarborImage(def script, Map svc, String reg, String proj, String latestTag, String buildTag,
                       String baseImage, String dockerfile, String workspace) {
     def jarName = "${svc.module}-1.0.0.jar"
     def imageLatest = "${reg}/${proj}/${svc.repo}:${latestTag}"
-    def imageBuild = "${reg}/${proj}/${svc.repo}:${buildTag}"
     def withLibFlag = svc.withLib ? 'true' : 'false'
-    // Parallel branches share one Docker daemon; serialize docker ops to avoid containerd CreateDiff/lease races.
     script.sh """
         set -e
         test -f ${workspace}/${svc.module}/target/${jarName}
@@ -60,16 +100,16 @@ def pushOneHarborImage(def script, Map svc, String reg, String proj, String late
         else
           touch .jenkins_docker_ctx/lib/.keep
         fi
+    """
+    // Harbor API archive runs outside flock (HTTP only, safe to parallelize).
+    archiveHarborLatestViaApi(script, reg, proj, svc.repo, latestTag, buildTag)
+    // Parallel branches share one Docker daemon; serialize docker build/push to avoid containerd races.
+    script.sh """
+        set -e
         DOCKER_LOCK=/tmp/jenkins-alien-cloud-docker.lock
         flock "\${DOCKER_LOCK}" sh -c '
           set -e
           cd ${workspace}/${svc.module}/.jenkins_docker_ctx
-          if docker pull ${imageLatest} 2>/dev/null; then
-            echo ">>> archive previous ${latestTag} -> ${buildTag}"
-            docker tag ${imageLatest} ${imageBuild}
-            docker push ${imageBuild}
-            docker rmi ${imageBuild} ${imageLatest} 2>/dev/null || true
-          fi
           build_ok=0
           for attempt in 1 2 3; do
             if docker build -f ${workspace}/${dockerfile} \\
@@ -228,7 +268,7 @@ pipeline {
         booleanParam(
                 name: 'HARBOR_PUSH_PARALLEL',
                 defaultValue: true,
-                description: 'Parallel per service (context prep); docker build/push serialized via flock on agent'
+                description: 'Parallel per service (context prep + Harbor API archive); docker build/push serialized via flock'
         )
     }
 

+ 2 - 1
docs/jenkins/uat/README.md

@@ -23,7 +23,8 @@
 | 构建前删 BOM | `rm -rf spring-cloud-dependencies` | 无 |
 | `FORCE_UPDATE` | 默认 **true** | 默认 **false** |
 | Maven 并行 | 无 | `-T 1C -Dmaven.artifact.threads=8` |
-| Harbor push | 串行 | 上下文准备可并行;`docker build/push` 经 `flock` 串行(防 containerd 竞态) |
+| Harbor 归档 | docker pull/tag/push | **Harbor API 打 tag**(同 digest,可并行;与生产 promote 相同机制) |
+| Harbor push | 串行 | 上下文准备 + API 归档可并行;`docker build/push` 经 `flock` 串行 |
 | Docker build | BuildKit(需 buildx) | 经典 builder + 失败重试 3 次(节点无 buildx 时稳定) |
 | Deploy | 串行 7 次 `sh` | `parallel` |
 | 并发构建 | 允许(产生 `@2` 工作区) | `disableConcurrentBuilds()` |