feat(aws-cdk): surface custom resource Lambda logs on deployment failure

[iankhou]

Summary

When a CloudFormation custom resource fails during deployment, the CLI today shows only the raw status reason — typically a cfn-response pointer like "See the details in CloudWatch Log Stream: …" — leaving the developer to go search for the lambda logs by hand. This PR fetches the backing Lambda's CloudWatch logs and surfaces them inline in the failure diagnosis, mirroring the existing ECS service investigation. The result is the actual handler error, where the failure is reported.

Applies to Custom::* and AWS::CloudFormation::CustomResource. The whole investigation is best-effort: every AWS call is wrapped so a failure is logged at debug level and never breaks the deployment error path.

Future work - explore Cloudtrail events following deployment failure. This will come in another PR! Cloudtrail events aren't available in console until 5-15 minutes after they occur, so will only be available on cdk diagnose runs after failure.

Before

❌  MyStack failed: ... Received response status [FAILED] from custom resource.
   Message returned: See the details in CloudWatch Log Stream: 2026/.../[$LATEST]abc123

After

MyStack/MyResource  (AWS::CloudFormation::CustomResource MyResource)
  Received response status [FAILED] from custom resource. ...

   Logs from /aws/lambda/MyStack-Handler-XXXX:
   ERROR Boom: the handler blew up
   INFO  Response body: {"Status":"FAILED", ...}
   Logs: https://<region>.console.aws.amazon.com/cloudwatch/...

How it works

1. Find the backing Lambda. The CloudFormation event doesn't name the function — only the resource's ServiceToken does, and that lives in the template. We fetch the template once and read ServiceToken, resolving:

• a literal Lambda ARN,
• an Fn::GetAtt (JSON array form and the YAML string short-form "Fn.Arn"), or
• a Ref, via describeStackResources.

For an intrinsic reference we confirm the target is actually AWS::Lambda::Function before treating its physical ID as a function name (a Ref to, say, an SNS topic is ignored rather than producing a bogus /aws/lambda/<name> lookup).

2. Resolve the log group — rollback-proof on stack CREATE. A first-deploy failure rolls back and deletes the backing Lambda, so getFunctionConfiguration is unavailable exactly when we need it. We therefore derive the log group without the live function wherever possible:

• the /aws/lambda/<function-name> convention (covers the vast majority);
• for advanced logging controls (custom LoggingConfig.LogGroup), read it from the template — including the common CDK shape where it's a Ref to an AWS::Logs::LogGroup whose physical name CloudFormation generates (resolved via describeStackResources, which still returns RETAINed resources after rollback);
• getFunctionConfiguration is used only as a last-resort fallback.

3. Target the right invocation. cfn-response writes the failing log stream name into the status reason, so we query that exact stream. Because that stream is pinned to the original create invocation, on update/rollback failures we bound the query to a time window around the failure event and fall back to a group-wide scan, so a stale stream can't hide the real logs. This required adding an optional timestamp to ResourceError (populated from stack events).

4. Readable output. Lambda log events are normalized before display (Lambda-only — ECS logs are arbitrary container output and left untouched):

• text format → strip the per-line timestamp/requestId prefix, render aligned LEVEL message;
• JSON format → render level + message (and error envelopes' errorMessage/stackTrace) instead of dumping raw JSON;
• Lambda platform boilerplate (INIT_START/START/END/REPORT, JSON platform.*) is dropped;
• application output is never dropped by level — failure detail often rides in INFO lines (e.g. the cfn-response "Response body") — and anything unrecognized passes through verbatim.

When logs genuinely can't be retrieved, the message distinguishes an empty group ("No log events found…") from a failed fetch ("Could not fetch logs… the credentials may lack logs:FilterLogEvents"), and the console deep-link is always emitted so the full logs are one click away.

Changes

• lib/api/diagnosing/resource-investigation.ts — custom-resource investigation, log-group resolution, stream targeting, and log normalization (shares the log-trimming helper with the ECS path).
• lib/api/stack-events/resource-errors.ts — optional ResourceError.timestamp, populated from stack events.
• lib/api/aws-auth/sdk.ts — getFunctionConfiguration on the Lambda client (fallback log-group discovery).
• test/api/diagnosing/resource-investigation.test.ts — unit tests for ServiceToken resolution (ARN / GetAtt array+string / Ref), log-group resolution (convention / template literal / template Ref / generated name), stream targeting + timestamp window, log normalization (text/JSON/platform/malformed), and the empty-vs-error messaging.

Testing

• Unit tests as above; new behaviors are mutation-verified (each fix's test fails when the fix is reverted).
• Verified end-to-end against live deployments: a cfn-response custom resource failing on create and on update (iankhou/cdk-app-with-problems@d733eb6), and an advanced-logging variant (iankhou/cdk-app-with-problems@571c33d) (custom log group, function deleted by rollback) — confirming the template-sourced log-group resolution surfaces the real logs without the live function.

Notes

• Limitation: if a custom log group is itself a CFN-managed resource deleted by rollback without a retain policy, the log data is gone (the resolution still names the right group). The default /aws/lambda/<fn> group's data persists regardless.

Checklist

• This change contains a major version upgrade for a dependency and I confirm all breaking changes are addressed
  • Release notes for the new version:

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

[iankhou]

More thorough log examples:

cfn-response module - Output of cdk deploy with JSON lambda logs, logging to default lambda log group

Logs from testing against the following project: https://github.com/iankhou/cdk-app-with-problems/tree/iankhou-cr-main on commit iankhou/cdk-app-with-problems@d733eb6

TEXT lambda logs also yields basically the same result

FailingCustomResourceStack: deploying... [1/1]
4:02:19 PM | CREATE_FAILED           | AWS::CloudFor
mation::CustomResource | MyFailingResource
Received response status [FAILED] from custom resour
ce. Message returned: See the details in CloudWatch
Log Stream: 2026/06/19/[$LATEST]2e517e59d4c0461cb4f3
f52e4a9064f7 (RequestId: 9f3d93cb-f86b-41dc-b2e2-7b0
a00033b82)

❌  FailingCustomResourceStack failed: DeploymentError: Resource updates failed:
FailingCustomResourceStack/MyFailingResource/Default  (AWS::CloudFormation::CustomResource MyFailingResource)
  Received response status [FAILED] from custom resource. Message returned: See the details in CloudWatch Log Stream:
  2026/06/19/[$LATEST]2e517e59d4c0461cb4f3f52e4a9064f7 (RequestId: 9f3d93cb-f86b-41dc-b2e2-7b0a00033b82)

   Logs from /aws/lambda/FailingCustomResourceStack-CrHandler8F2F7DBF-pq4JFc9Ri647:
   INFO  request type: "Create"
   ERROR Boom: simulated custom resource failure on Create
   INFO  Response body:
 {"Status":"FAILED","Reason":"See the details in CloudWatch Log Stream: 2026/06/19/[$LATEST]2e517e59d4c0461cb4f3f52e4a9064f7","PhysicalResourceId":"2026/06/19/[$LATEST]2e517e59d4c0461cb4f3f52e4a9064f7","StackId":"arn:aws:cloudformation:us-east-1:053108159643:stack/FailingCustomResourceStack/acb17810-6c19-11f1-9f58-0e525b316913","RequestId":"9f3d93cb-f86b-41dc-b2e2-7b0a00033b82","LogicalResourceId":"MyFailingResource","NoEcho":false,"Data":{"error":"simulated"}}
   INFO  Status code: 200
   INFO  Status message: OK
   INFO  request type: "Delete"
   INFO  Response body:
 {"Status":"SUCCESS","Reason":"See the details in CloudWatch Log Stream: 2026/06/19/[$LATEST]2e517e59d4c0461cb4f3f52e4a9064f7","PhysicalResourceId":"2026/06/19/[$LATEST]2e517e59d4c0461cb4f3f52e4a9064f7","StackId":"arn:aws:cloudformation:us-east-1:053108159643:stack/FailingCustomResourceStack/acb17810-6c19-11f1-9f58-0e525b316913","RequestId":"25da2754-c695-48c4-ba84-41199cf5128e","LogicalResourceId":"MyFailingResource","NoEcho":false,"Data":{}}
   INFO  Status code: 200
   INFO  Status message: OK
   Logs: https://us-east-1.console.aws.amazon.com/cloudwatch/home?region=us-east-1#logsV2:log-groups/log-group/$252Faws$252Flambda$252FFailingCustomResourceStack-CrHandler8F2F7DBF-pq4JFc9Ri647

[github-actions]

Total lines changed 1204 is greater than 1000. Please consider breaking this PR down.

[iankhou]

Sorry for the big PR! Includes some refactoring to move util functions out of resource-investigation.ts, which was getting bloated.

[rix0rrr]

<3 it!

Couple small nits.

[rix0rrr]

The documentation says you are normalizing to LEVEL message, so that's 2 fields.

The return type here only has message, so either that's a string containing both, or the comment is wrong.

• If the return type is a combined string, just return string[]
• Otherwise add level: string and leave it to the caller to format these.

[iankhou]

Will correct the documentation and adjust the field names the prevent confusion.

[rix0rrr]

Let's not to <tab>s, I don't trust em.

Pad with spaces to the desired length.

[iankhou]

It was actually using spaces, the documentation here was wrong. Will make the correction, thanks for the catch!

[rix0rrr]

I think these functions should probably go into their own files as well. investigate-custom-resource.ts, investigate-ecs-service.ts ?

[iankhou]

Will do