From: Anil Belur <abelur@linuxfoundation.org>
Date: Wed, 5 Dec 2018 09:19:33 +0000 (+1000)
Subject: Compress and upload the jjb-verify XML files
X-Git-Tag: v0.28.3~5
X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=commitdiff_plain;h=acb1db1cf0efbeea7540866b500657d3dec935ce;p=releng%2Fglobal-jjb.git

Compress and upload the jjb-verify XML files

The present jjb-verify job that calls lftools maven
deploy archives for releng/builder which takes about ~8 mins to
complete uploading the archives. This is because the job generates
around ~2.3K XML files (small files) which is uploaded to Nexus.
This is possibly because the Nexus Unpack plugin in the Nexus
end unpacking the zip file we upload takes longer.

It's best to compress this into a single file and upload
to Nexus, potentially minimizing the Nexus logs from
bloating up since we have been seeing higher IOPs
and load managing these files in the backend cron jobs.

Compressing the job files before hand shaves off around
6-8 mins from the upload time, therefore reducing the
job time overall significantly.

Tested the change here:
https://jenkins.opendaylight.org/releng/job/builder-jjb-verify/4325

Change-Id: Ifb3ea6e61d28b234c79088ca128294a037ec511f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
---

diff --git a/releasenotes/notes/jjv-verify-compress-xml-files-71ae17abdfc79c2f.yaml b/releasenotes/notes/jjv-verify-compress-xml-files-71ae17abdfc79c2f.yaml
new file mode 100644
index 00000000..f28ea8b7
--- /dev/null
+++ b/releasenotes/notes/jjv-verify-compress-xml-files-71ae17abdfc79c2f.yaml
@@ -0,0 +1,9 @@
+---
+fixes:
+  - |
+    Compress and upload all jjb-verify XML files to Nexus, to ease out the IOPs
+    on cron jobs that manage the logs on Nexus and optimize job performace by
+    ~8 mins. This is because the job generates around ~2.3K XML files (small
+    files) which is uploaded to Nexus in every run of jjb-verify. Doing this
+    is faster as compared to the Nexus Unpack plugin in the Nexus end unpacking
+    the zip file we upload takes longer.
diff --git a/shell/jjb-verify-job.sh b/shell/jjb-verify-job.sh
index f38685f2..add39378 100644
--- a/shell/jjb-verify-job.sh
+++ b/shell/jjb-verify-job.sh
@@ -28,3 +28,8 @@ do
     fi
 done
 popd
+
+if [ ! -z "$(ls -A archives/job-configs)" ]; then
+    tar cvzf archives/job-configs.tar.gz archives/job-configs
+    rm -rf archives/job-configs
+fi