Feature: ServiceCatalog

[davidconnor-ont]

This is an early draft of the service catalog API endpoint.

These endpoints should provide enough coverage to run the core provisioning tasks of creating portfolios, products, linking them and provisioning a stack through the moto cloudformation module.

I'm providing this draft PR as it's already ballooned into quite a large PR even without the full unit test coverage needed for testing exception responses. I wanted to get any early feedback that might be needed before going down too many wrong paths. So far the code is not particularly complex as it's mostly managed the session state of the created resources.

Initial implementation covering:

• associate_product_with_portfolio
• create_constraint
• create_portfolio
• create_product
• describe_portfolio
• describe_product
• describe_product_as_admin
• describe_provisioned_product
• get_provisioned_product_outputs
• list_launch_paths
• list_portfolios
• list_portfolios_for_product
• list_provisioning_artifacts
• provision_product
• search_products
• search_provisioned_products
• terminate_provisioned_product
• update_portfolio
• update_product

The aim of the initial version is to be able to run some of the simpler tutorials like this one from AWS:
https://www.youtube.com/watch?v=egZroVhDF4M

This terraform script should run completely and create all resources correctly including provisioning from the cloudformation template provided (EDIT: the provison_product step fails at the moment which is the part I'm currently working on.)

resource "aws_s3_bucket" "cf_templates" {
    bucket = "cf_templates"
}
locals {
  object_source = "${path.module}/simple.yaml"
}

resource "aws_s3_object" "file_upload" {
  bucket = aws_s3_bucket.cf_templates.id
  key         = "main_template"
  source      = local.object_source
  source_hash = filemd5(local.object_source)
}

resource "aws_servicecatalog_portfolio" "organization_portfolio" {
  name          = "Organization Portfolio"
  description   = "List of organization's services"
  provider_name = "CCOE"
}

resource "aws_servicecatalog_product" "s3" {
  name  = "S3 Bucket"
  owner = "CCOE"
  type  = "CLOUD_FORMATION_TEMPLATE"

  provisioning_artifact_parameters {
    template_url = "https://${aws_s3_bucket.cf_templates.id}.s3.${aws_s3_bucket.cf_templates.region}.amazonaws.com/${aws_s3_object.file_upload.key}"
    name = "Simple S3 Bucket"
    description = "v1.0"
    type = "CLOUD_FORMATION_TEMPLATE"
  }
}

resource "aws_servicecatalog_product_portfolio_association" "product_portfolio_association" {
  portfolio_id = aws_servicecatalog_portfolio.organization_portfolio.id
  product_id   = aws_servicecatalog_product.s3.id
}

resource "aws_servicecatalog_principal_portfolio_association" "associate_role" {
  portfolio_id  = aws_servicecatalog_portfolio.organization_portfolio.id
  principal_arn = "arn:aws:iam::444422324444:role/Admin-OneClick"
}

data "aws_servicecatalog_launch_paths" "s3_product_path" {
  product_id = aws_servicecatalog_product.s3.id
}

resource "aws_servicecatalog_provisioned_product" "provisioned_s3" {
  name = "Simple S3"
  product_id = aws_servicecatalog_product.s3.id
  path_id = data.aws_servicecatalog_launch_paths.s3_product_path.summaries[0].path_id
  provisioning_artifact_name = aws_servicecatalog_product.s3.provisioning_artifact_parameters[0].name
}

cloudformation template:

Resources:
  LocalBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: cfn-quickstart-bucket

[bblommers]

Hi @davidconnor-ont, welcome to Moto!

This all looks good to me so far. It all follows the same approach/setup as our other services, which is quite important to keep things readable.

Same goes for the tests - they only test a single operation, making them very readable, so I'm happy with that.

One possible suggestion:
There seem to be a lot of tests that share the same setup. Is it worth converting them to separate TestClass-es, so that there is less deduplication?
Example: https://github.com/getmoto/moto/blob/master/tests/test_autoscaling/test_autoscaling_groups.py

Don't worry about the size of the PR.
it's indeed ballooning, but because it's mostly session state management, its all easy to review.

[davidconnor-ont]

Thanks for the feedback.

One possible suggestion: There seem to be a lot of tests that share the same setup. Is it worth converting them to separate TestClass-es, so that there is less deduplication? Example: https://github.com/getmoto/moto/blob/master/tests/test_autoscaling/test_autoscaling_groups.py

Definitely. I hadn't had the chance to refactor the tests into a more clean setup - it was repetitive out of laziness/convenience to get the functions semi-working. I'll take a look at that example when I'm updating them.

[davidconnor-ont]

@bblommers Service catalog seems to implement filters mostly the same as EC2 and other filters that have the pattern:
[{"filterName":["filterValue"]}]
but has a slightly different inputFormat.

{"SearchQuery":["field:value", "anotherfield:value,value2]}

The docs indicate that "SearchQuery" is the only key, so really the values in SearchQuery map to the EC2 way of filtering under-the-hood.

I've added a commit where I map the input to match the excepted input of generic_filter and have implemented get_filter_value on the specific object.

Before I go any further:

1. is there a standard way of filtering objects in moto like there is for tagging?
2. If not, is importing the generic_filter code from EC2 ok across the service boundry (servicecatalog importing from ec2).

[bblommers]

Hi @davidconnor-ont,

There is no standard way to filter things - it's all a bit disjointed unfortunately

It sounds like the best solution to use generic_filter, so I'm OK with importing this from EC2 for now.

Importing EC2 does come at a price - it will add another 2/300 ms to the import time, because it automatically generates all kinds of default objects (AMI's, VPC's, etc.). So later on I will just refactor the filter-methods into a separate service-agnostic utilities-file to remedy this.

[davidconnor-ont]

Hello, I'm still working on this. I'm testing it with our real world usage of service catalog so it's being adjusted as I add the unit testing to our service catalog calls. The main area I need to address is the provisioning a product when a passed in path_id is used - those are the artificially failing test placeholders in the last test run.

[bblommers]

@davidconnor-ont We've just released Moto v5 with quite a few breaking changes. Let me know if you need any help bringing this PR up-to-date, I'd be happy to help out.