Use official CentOS & Ubuntu images via prodId

The previous filter method could result in getting an unexpected image
even one that's entirely different than what is intended. This patch
uses a product-code search of the marketplace for the official upstream
supported images rather than the custom ones provided by Amazon.

This patch is based on Zack William's suggestion on the lf-releng
mailinglist.

Reference:
https://lists.linuxfoundation.org/pipermail/lf-releng/2020-February/000135.html

Change-Id: Ia6208bd7a61036eb210d07533e3ceedbbc787678
Co-authored-by: Zack Williams <zdw@opennetworking.org>
Signed-off-by: Thanh Ha <zxiiro@gmail.com>

diff --git a/README b/README
index cae4abf..6715e78 100644 (file)
--- a/README
+++ b/README
@@ -1,5 +1,52 @@
+# Common Packer
+
 A collection of common packer scripts and baseline used by Linux Foundation
 projects as a central way to maintain and manage common VM configuration.
 
 Documentation for common packer is available at this link
 https://docs.releng.linuxfoundation.org/projects/common-packer
+
+## Finding source AMI images
+
+Source OS images published by upstream projects like Ubuntu and CentOS need to
+be well specified, so that the correct images are used. Anyone can list in
+the marketplace, so care should be taken to use the correct images.
+
+This is done in Packer using
+[source_ami_filter](https://packer.io/docs/builders/amazon-ebs.html#source_ami_filter)
+which is parameterized on the image `name`, `owner`, and `product-code` within
+the `packer/vars/<os_name>.json` files that define the source images.
+
+Upstream docs that specify AMIs:
+
+- [CentOS](https://wiki.centos.org/Cloud/AWS)
+- [Ubuntu](https://cloud-images.ubuntu.com/locator/ec2/)
+
+Unfortunately these filter parameters have conflicts - images with the official
+Ubuntu owner (`099720109477`) doesn't specify a `product-code` field.
+
+As an alternative, `aws-marketplace` owner is used, which also has the same
+images. To find the `product-code`, go to the [AWS
+Marketplace](https://aws.amazon.com/marketplace) and find the image you want,
+then click the button to launch the image. In the URL there will be a
+`productId` UUID parameter - find this, and then use it search for a product
+code using the [aws command
+line](https://docs.aws.amazon.com/cli/latest/index.html):
+
+    aws ec2 describe-images \
+        --owners aws-marketplace \
+        --filters "Name=name,Values=*aced0818-eef1-427a-9e04-8ba38bada306*"
+
+Then look at the output for the `ProductCodeId` - this is what should go in
+the OS json file in the `source_ami_filter_product_code` field.
+
+Once you've determined the correct settings, the Packer filter can be tested
+with this command:
+
+    aws ec2 describe-images \
+        --owners aws-marketplace \
+        --filters "Name=name,Values=*ubuntu*20.04*" \
+                  "Name=product-code,Values=a8jyynf4hjutohctm41o2z18m" \
+                  "Name=architecture,Values=x86_64" \
+                  "Name=virtualization-type,Values=hvm" \
+                  "Name=root-device-type,Values=ebs"

diff --git a/templates/builder-aws.json b/templates/builder-aws.json
index c4a43c2..f5a0f01 100644 (file)
--- a/templates/builder-aws.json
+++ b/templates/builder-aws.json
@@ -12,6 +12,7 @@
     "security_group_id": null,
     "source_ami_filter_name": null,
     "source_ami_filter_owner": null,
+    "source_ami_filter_product_code": null,
     "subnet_id": null,
     "ssh_user": null,
     "ssh_proxy_host": "",
@@ -29,6 +30,7 @@
       "source_ami_filter": {
         "filters": {
           "name": "{{user `source_ami_filter_name`}}",
+          "product-code": "{{user `source_ami_filter_product_code`}}",
           "root-device-type": "ebs",
           "virtualization-type": "hvm"
         },

diff --git a/vars/centos-7.json b/vars/centos-7.json
index efdd3b5..151b7bf 100644 (file)
--- a/vars/centos-7.json
+++ b/vars/centos-7.json
@@ -1,6 +1,7 @@
 {
-  "source_ami_filter_name": "*CentOS Linux 7*HVM*",
-  "source_ami_filter_owner": "679593333241",
+  "source_ami_filter_name": "*CentOS-7*",
+  "source_ami_filter_owner": "aws-marketplace",
+  "source_ami_filter_product_code": "cvugziknvmxgqna9noibqnnsy",
   "base_image": "LF - CentOS 7.2003 (2020-04-22)",
   "distro": "CentOS 7",
   "ssh_user": "centos",

diff --git a/vars/centos-8.json b/vars/centos-8.json
index bd82e9d..6c7ff10 100644 (file)
--- a/vars/centos-8.json
+++ b/vars/centos-8.json
@@ -1,6 +1,7 @@
 {
-  "source_ami_filter_name": "*CentOS Linux 8*HVM*",
-  "source_ami_filter_owner": "679593333241",
+  "source_ami_filter_name": "*CentOS-8*",
+  "source_ami_filter_owner": "aws-marketplace",
+  "source_ami_filter_product_code": "47k9ia2igxpcce2bzo8u3kj03",
   "base_image": "CentOS 8.2 (x86_64) [2020-06-11]",
   "distro": "CentOS 8",
   "ssh_user": "centos",

diff --git a/vars/ubuntu-16.04.json b/vars/ubuntu-16.04.json
index 0c25018..3289e4b 100644 (file)
--- a/vars/ubuntu-16.04.json
+++ b/vars/ubuntu-16.04.json
@@ -1,6 +1,7 @@
 {
-  "source_ami_filter_name": "*ubuntu*16.04*",
-  "source_ami_filter_owner": "099720109477",
+  "source_ami_filter_name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server*",
+  "source_ami_filter_owner": "aws-marketplace",
+  "source_ami_filter_product_code": "csv6h7oyg29b7epjzg7qdr7no",
   "base_image": "LF - Ubuntu 16.04 LTS (2019-12-11)",
   "distro": "Ubuntu 16.04",
   "ssh_user": "ubuntu",

diff --git a/vars/ubuntu-18.04.json b/vars/ubuntu-18.04.json
index 19f03a5..67c4d40 100644 (file)
--- a/vars/ubuntu-18.04.json
+++ b/vars/ubuntu-18.04.json
@@ -1,6 +1,7 @@
 {
-  "source_ami_filter_name": "*ubuntu*18.04*",
-  "source_ami_filter_owner": "099720109477",
+  "source_ami_filter_name": "ubuntu/images/*ubuntu-bionic-18.04-amd64-server-*",
+  "source_ami_filter_owner": "aws-marketplace",
+  "source_ami_filter_product_code": "3iplms73etrdhxdepv72l6ywj",
   "base_image": "LF - Ubuntu 18.04 LTS (2020-07-29)",
   "distro": "Ubuntu 18.04",
   "ssh_user": "ubuntu",