From: Aric Gardner <agardner@linuxfoundation.org>
Date: Mon, 5 Aug 2019 17:32:04 +0000 (-0400)
Subject: Exit 0 if release job has already tagged a repo.
X-Git-Tag: v0.40.4~4
X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=commitdiff_plain;h=refs%2Fchanges%2F41%2F16441%2F14;p=releng%2Fglobal-jjb.git

Exit 0 if release job has already tagged a repo.

The release merge job is a one way operation.
Given this, Release jobs should only exist in the format
{project-name}-release-verify
and
{project-name}-release-merge

They should trigger from a change to any branch/**
As a developer working on {example-branch} will not want to
check out master to push a new release file.

Edge Cases:
Inevitably a release file will be pulled in from master to {example-branch}
or a remerge will be requested
This will again trigger the release jobs.
since in this case the repo is already tagged, the job should not report
a failure.

This is solved by having the verfiy and merge exit 0 when the repo is already tagged

if git tag -v "$VERSION"; then
  echo "Repo already tagged $VERSION"
  echo "This job has already run exit 0"
  exit 0
if

Example Workflow - Milestone 0 is released from master:
releases/1.4.0.yaml (on branch master)
tag: 1.4.0
log_dir: 'optf-osdf-maven-stage-master/97/'

Repo is tagged and artifact is pushed!

Milestone 3 is released from dublin:
releases/1.4.3.yaml (on branch dublin)
tag: 1.4.3
log_dir: 'optf-osdf-maven-stage-dublin/20/'

Repo is tagged and artifact is pushed!

Note:
the ref is determined by the log file.
So We could check if this ref can be found on the {branch}
to reduce human errors.
example:
odlparent-maven-stage-4.0.x/152/patches/taglist.log.gz
has:
odlparent c8964051cf81e2bd5d831fe6e5d568c5eedeeb8e

Psudo code:
STREAM=${STREAM:-'nostream'}
echo "--> Checking if $ref is on stable/$STREAM"
if ! (git branch -a --contains $ref | grep "stable/$STREAM"); then
    echo "--> ERROR: $ref for $repo is not on stable/$STREAM!"
fi

Also in this patch:
To reduce possible errors:
Don't get project name from the release file:
PROJECT="$(niet ".project" "$release_file")"
we can just use ${PROJECT//\//-} when we need
the /'s changed to -'s

It will allways be correct when passed to jenkins
from gerrit

Also remove quiet from wget,
if there is a failure I want to see it on the console.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I1e95e864bc2b49a62b3f7016cb2f1292a6687d9a
---

diff --git a/jjb/lf-release-jobs.yaml b/jjb/lf-release-jobs.yaml
index 3f226db4..00acc009 100644
--- a/jjb/lf-release-jobs.yaml
+++ b/jjb/lf-release-jobs.yaml
@@ -36,8 +36,8 @@
     parameters:
       - lf-infra-parameters:
           project: "{project}"
-          branch: "master"
-          stream: "{stream}"
+          branch: "$GERRIT_BRANCH"
+          stream: "$GERRIT_BRANCH"
 
     wrappers:
       - lf-infra-wrappers:
@@ -58,8 +58,6 @@
     # Default parameters #
     ######################
 
-    branch: master
-    stream: master
     build-days-to-keep: 7
     build-timeout: 15
     disable-job: false
@@ -98,7 +96,7 @@
       - lf-release
 
 - job-template:
-    name: "{project-name}-release-verify-{stream}"
+    name: "{project-name}-release-verify"
     id: gerrit-release-verify
     <<: *lf_release_common
     # yamllint disable-line rule:key-duplicates
@@ -108,7 +106,7 @@
       - lf-infra-gerrit-scm:
           git-url: "{git-url}"
           refspec: "$GERRIT_REFSPEC"
-          branch: "master"
+          branch: "$GERRIT_BRANCH"
           submodule-disable: true
           submodule-recursive: false
           submodule-timeout: 10
@@ -124,7 +122,7 @@
               project-pattern: "{project}"
               branches:
                 - branch-compare-type: "ANT"
-                  branch-pattern: "**/{branch}"
+                  branch-pattern: "**"
               file-paths: "{obj:gerrit_trigger_file_paths}"
           skip-vote:
             successful: "{gerrit-skip-vote}"
@@ -143,8 +141,6 @@
     # Default parameters #
     ######################
 
-    branch: master
-    stream: master
     build-days-to-keep: 7
     build-timeout: 15
     disable-job: false
@@ -178,7 +174,7 @@
       - lf-release
 
 - job-template:
-    name: "{project-name}-release-merge-{stream}"
+    name: "{project-name}-release-merge"
     id: gerrit-release-merge
     <<: *lf_release_common
     # yamllint disable-line rule:key-duplicates
@@ -204,5 +200,5 @@
               project-pattern: "{project}"
               branches:
                 - branch-compare-type: "ANT"
-                  branch-pattern: "**/{branch}"
+                  branch-pattern: "**"
               file-paths: "{obj:gerrit_trigger_file_paths}"
diff --git a/releasenotes/notes/lf-release-jobs-c95bd5f491ba43b2.yaml b/releasenotes/notes/lf-release-jobs-c95bd5f491ba43b2.yaml
new file mode 100644
index 00000000..ab751431
--- /dev/null
+++ b/releasenotes/notes/lf-release-jobs-c95bd5f491ba43b2.yaml
@@ -0,0 +1,21 @@
+---
+fixes:
+  - |
+    1.
+    The release merge job is a one way operation.
+    Given this, Release jobs should only exist in the format
+    {project-name}-release-verify and
+    {project-name}-release-merge
+    As these jobs trigger from a change to any branch/**
+    These jobs Must Exit 0 if release job has already tagged a repo.
+    Inevitably a release file will be pulled in from master to branch
+    or a remerge will be requested This will again trigger the release jobs.
+    since in this case the repo is already tagged, the job should not report
+    a failure. This is solved by having the verfiy and merge exit 0 when the
+    repo is already tagged
+    2.
+    Rather than use project as defined in the release file use
+    ${PROJECT//\//-}
+    This changes PROJECT="optf/osdf/foo/bar" to
+    optf-osdf-foo-bar so that we can fetch the log files.
+    by changing /'s in the project names to -'s
diff --git a/shell/release-job.sh b/shell/release-job.sh
index d9880cd3..ad118ded 100644
--- a/shell/release-job.sh
+++ b/shell/release-job.sh
@@ -55,14 +55,13 @@ echo "--> Verifying $release_file schema."
 lftools schema verify $release_file release-schema.yaml
 
 VERSION="$(niet ".version" "$release_file")"
-PROJECT="$(niet ".project" "$release_file")"
 LOG_DIR="$(niet ".log_dir" "$release_file")"
-
 NEXUS_PATH="${SILO}/${JENKINS_HOSTNAME}/"
 LOGS_URL="${LOGS_SERVER}/${NEXUS_PATH}${LOG_DIR}"
 PATCH_DIR="$(mktemp -d)"
 
 LOGS_URL=${LOGS_URL%/}  # strip any trailing '/'
+echo "wget -P "$PATCH_DIR" "${LOGS_URL}/"staging-repo.txt.gz"
 wget -P "$PATCH_DIR" "${LOGS_URL}/"staging-repo.txt.gz
 
 nexus_release(){
@@ -91,12 +90,14 @@ echo "NEXUSPROXY: $NEXUSPROXY"
 echo "JENKINS_HOSTNAME: $JENKINS_HOSTNAME"
 echo "SILO: $SILO"
 echo "PROJECT: $PROJECT"
+echo "PROJECT-DASHED: ${PROJECT//\//-}"
+
 echo "VERSION: $VERSION"
-echo "PROJECT: $PROJECT"
 echo "LOG DIR: $LOG_DIR"
 
 pushd "$PATCH_DIR"
-  wget --quiet  "${LOGS_URL}"/patches/{"${PROJECT}".bundle,taglist.log.gz}
+  echo "wget "${LOGS_URL}"/patches/{"${PROJECT//\//-}".bundle,taglist.log.gz}"
+  wget "${LOGS_URL}"/patches/{"${PROJECT//\//-}".bundle,taglist.log.gz}
   gunzip taglist.log.gz
   cat "$PATCH_DIR"/taglist.log
 popd
@@ -111,10 +112,16 @@ if [[ ! $VERSION =~ $allowed_version_regex ]]; then
   exit 1
 fi
 
+if git tag -v "$VERSION"; then
+  echo "Repo already tagged $VERSION"
+  echo "This job has already run exit 0"
+  exit 0
+if
+
 git checkout "$(awk '{print $NF}' "$PATCH_DIR/taglist.log")"
 git fetch "$PATCH_DIR/$PROJECT.bundle"
 git merge --ff-only FETCH_HEAD
-git tag -am "$PROJECT $VERSION" "$VERSION"
+git tag -am "${PROJECT//\//-} $VERSION" "$VERSION"
 sigul --batch -c "$SIGUL_CONFIG" sign-git-tag "$SIGUL_KEY" "$VERSION" < "$SIGUL_PASSWORD"
 
 echo "Showing latest signature for $PROJECT:"
@@ -126,7 +133,7 @@ git tag -v "$VERSION"
 if [[ "$JOB_NAME" =~ "merge" ]]; then
   echo "Running merge"
   gerrit_ssh=$(echo "$GERRIT_URL" | awk -F"/" '{print $3}')
-  git remote set-url origin ssh://"$RELEASE_USERNAME"@"$gerrit_ssh":29418/$GERRIT_PROJECT
+  git remote set-url origin ssh://"$RELEASE_USERNAME"@"$gerrit_ssh":29418/"$PROJECT"
   git config user.name "$RELEASE_USERNAME"
   git config user.email "$RELEASE_EMAIL"
   echo -e "Host $gerrit_ssh\n\tStrictHostKeyChecking no\n" >> ~/.ssh/config