From: Anil Belur Date: Wed, 5 Dec 2018 09:19:33 +0000 (+1000) Subject: Compress and upload the jjb-verify XML files X-Git-Tag: v0.28.3~5 X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=commitdiff_plain;h=acb1db1cf0efbeea7540866b500657d3dec935ce;p=releng%2Fglobal-jjb.git Compress and upload the jjb-verify XML files The present jjb-verify job that calls lftools maven deploy archives for releng/builder which takes about ~8 mins to complete uploading the archives. This is because the job generates around ~2.3K XML files (small files) which is uploaded to Nexus. This is possibly because the Nexus Unpack plugin in the Nexus end unpacking the zip file we upload takes longer. It's best to compress this into a single file and upload to Nexus, potentially minimizing the Nexus logs from bloating up since we have been seeing higher IOPs and load managing these files in the backend cron jobs. Compressing the job files before hand shaves off around 6-8 mins from the upload time, therefore reducing the job time overall significantly. Tested the change here: https://jenkins.opendaylight.org/releng/job/builder-jjb-verify/4325 Change-Id: Ifb3ea6e61d28b234c79088ca128294a037ec511f Signed-off-by: Anil Belur --- diff --git a/releasenotes/notes/jjv-verify-compress-xml-files-71ae17abdfc79c2f.yaml b/releasenotes/notes/jjv-verify-compress-xml-files-71ae17abdfc79c2f.yaml new file mode 100644 index 00000000..f28ea8b7 --- /dev/null +++ b/releasenotes/notes/jjv-verify-compress-xml-files-71ae17abdfc79c2f.yaml @@ -0,0 +1,9 @@ +--- +fixes: + - | + Compress and upload all jjb-verify XML files to Nexus, to ease out the IOPs + on cron jobs that manage the logs on Nexus and optimize job performace by + ~8 mins. This is because the job generates around ~2.3K XML files (small + files) which is uploaded to Nexus in every run of jjb-verify. Doing this + is faster as compared to the Nexus Unpack plugin in the Nexus end unpacking + the zip file we upload takes longer. diff --git a/shell/jjb-verify-job.sh b/shell/jjb-verify-job.sh index f38685f2..add39378 100644 --- a/shell/jjb-verify-job.sh +++ b/shell/jjb-verify-job.sh @@ -28,3 +28,8 @@ do fi done popd + +if [ ! -z "$(ls -A archives/job-configs)" ]; then + tar cvzf archives/job-configs.tar.gz archives/job-configs + rm -rf archives/job-configs +fi