Skip to content

feat(ecs-patterns): enable Backlog per instance scaling in QueueProcessingFargateService #34202

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

Abdul-pitodia
Copy link

Issue #34072

Closes #34072.

Reason for this change

AWS Documentation recommends a Backlog per instance scaling strategy for Queue based workloads Scale SQS queue using custom metric

Currently QueueProcessingFargateService does not natively support this type of target tracking scaling strategy. The only options currently are to depend on CPU Based Scaling or scaling based on SQS metric called ApproximateNumberOfMessagesVisible.

This PR aims to add the necessary scaling strategy to QueueProcessingFargateService which will allow the service to scale based on the backlog per instance.

Note: ContainerInsights must be enabled on the cluster to publish the metric RunningTaskCount

Description of changes

  • Added support for using MathExpression in TargetTrackingScalingPolicy under aws-applicationauatoscaling
  • Added enableBacklogPerInstanceBasedScaling and backlogPerInstanceTargetValue properties for enabling and defining the Backlog scaling
  • Updated configureAutoscalingForService method to work with the above properties to enable the scaling strategy in QueueProcessingServiceBase. Used Official documentation for scaling strategy implementation

Describe any new or updated permissions being added

N/A – no new permissions are introduced in this change.

Description of how you validated changes

  • Added unit test for testing Target tracking using Math Expression Construct in packages/aws-cdk-lib/aws-applicationautoscaling/test/target-tracking.test.ts
  • Added unit test for testing Backlog per instance scaling in packages/aws-cdk-lib/aws-ecs-patterns/test/fargate/queue-processing-fargate-service.test.ts
  • Added new integration test packages/@aws-cdk-testing/framework-integ/test/aws-ecs-patterns/test/fargate/integ.queue-processing-fargate-service-backlog-per-instance-scaling.ts
  • Tested by deploying the integration test App, attached images.
SCR-20250419-krqe SCR-20250419-krui SCR-20250419-krvq SCR-20250419-lblq SCR-20250419-kroh

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added the beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK label Apr 20, 2025
@aws-cdk-automation aws-cdk-automation requested a review from a team April 20, 2025 13:35
@github-actions github-actions bot added effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 labels Apr 20, 2025
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(This review is outdated)

@Abdul-pitodia Abdul-pitodia changed the title feat(ecs-patterns): Enable Backlog per instance scaling in QueueProcessingFargateService feat(ecs-patterns): enable Backlog per instance scaling in QueueProcessingFargateService Apr 20, 2025
@aws-cdk-automation aws-cdk-automation dismissed their stale review April 20, 2025 13:39

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@Abdul-pitodia
Copy link
Author

Abdul-pitodia commented Apr 20, 2025

I was getting a error from the Code build job complaining about the version mismatch in the manifest.json

I have followed the integration test guideline to generate the snapshot, which outputted the version to 43.0.0

@aws-cdk-testing/framework-integ:   ERROR      aws-ecs-patterns/test/fargate/integ.queue-processing-fargate-service-backlog-per-instance-scaling 0.804s
@aws-cdk-testing/framework-integ:       Cannot read integ manifest 'test/aws-ecs-patterns/test/fargate/integ.queue-processing-fargate-service-backlog-per-instance-scaling.js.snapshot/manifest.json': Cloud assembly schema version mismatch: Maximum schema version supported is 42.x.x, but found 43.0.0. You need at least CLI version 2.1008.0 to read this manifest.

I have updated the version manually to 42.0.0 in the manifest.json in this commit

Please let me know if its fine, or any other way to fix this issue

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: e5bd600
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Apr 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beginning-contributor [Pilot] contributed between 0-2 PRs to the CDK effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2 pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member.
Projects
None yet
2 participants