Skip to content

Conversation

@baburciu
Copy link

@baburciu baburciu commented Oct 27, 2025

Issue #, if available:

Description of changes:

This fix resolves a message confusion where launch lifecycle events were misleadingly logged as "drain" operations when they're actually doing the opposite.

  • Updated logging interface: renamed RequestingInstanceDrain() to ProcessingInterruptionEvent()
  • Added event-type awareness: the logging function now uses a switch statement based on event.Kind

Before the fix: Everything logged "Requesting instance drain"

{
    "level": "info",
    "event": {
        "EventID": "asg-lifecycle-term-65343736363839362d346666352d303536352d656661622d303432326331343631666665",
        "Kind": "ASG_LAUNCH_LIFECYCLE",
        "Monitor": "SQS_MONITOR",
        "Description": "ASG Lifecycle Launch event received. Instance was started at 2025-10-27 12:10:53.217 +0000 UTC \n",
        "State": "",
        "AutoScalingGroupName": "on-demand-huge-workers-01-2024112121084029580000002c",
        "NodeName": "ip-10-31-16-222.eu-west-1.compute.internal",
        "NodeLabels": null,
        "Pods": null,
        "InstanceID": "i-070c03698c54561a9",
        "ProviderID": "aws:///eu-west-1a/i-070c03698c54561a9",
        "InstanceType": "",
        "IsManaged": true,
        "StartTime": "2025-10-27T12:10:53.217Z",
        "EndTime": "0001-01-01T00:00:00Z",
        "NodeProcessed": false,
        "InProgress": false
    },
    "time": "2025-10-27T12:11:16Z",
    "message": "Adding new event to the event store"
}
{
    "level": "info",
    "event-id": "asg-lifecycle-term-65343736363839362d346666352d303536352d656661622d303432326331343631666665",
    "kind": "ASG_LAUNCH_LIFECYCLE",
    "node-name": "ip-10-31-16-222.eu-west-1.compute.internal",
    "instance-id": "i-070c03698c54561a9",
    "provider-id": "aws:///eu-west-1a/i-070c03698c54561a9",
    "time": "2025-10-27T12:11:17Z",
    "message": "Requesting instance drain"  #<==
}

After the fix:

  • Only launch lifecycle events get the specific message "Waiting for node to be ready before completing ASG launch lifecycle"
  • Everything else (including actual drain operations) still gets "Requesting instance drain"
{
    "level": "info",
    "event": {
        "EventID": "asg-lifecycle-term-32346536363839392d326162342d376165612d323936332d376438383166333631323932",
        "Kind": "ASG_LAUNCH_LIFECYCLE",
        "Monitor": "SQS_MONITOR",
        "Description": "ASG Lifecycle Launch event received. Instance was started at 2025-10-27 15:30:25.519 +0000 UTC \n",
        "State": "",
        "AutoScalingGroupName": "on-demand-huge-workers-01-2024112121084029580000002c",
        "NodeName": "ip-10-31-26-73.eu-west-1.compute.internal",
        "NodeLabels": null,
        "Pods": null,
        "InstanceID": "i-069246571a8ee057a",
        "ProviderID": "aws:///eu-west-1c/i-069246571a8ee057a",
        "InstanceType": "",
        "IsManaged": true,
        "StartTime": "2025-10-27T15:30:25.519Z",
        "EndTime": "0001-01-01T00:00:00Z",
        "NodeProcessed": false,
        "InProgress": false
    },
    "time": "2025-10-27T15:30:25Z",
    "message": "Adding new event to the event store"
}
{
    "level": "info",
    "event-id": "asg-lifecycle-term-32346536363839392d326162342d376165612d323936332d376438383166333631323932",
    "kind": "ASG_LAUNCH_LIFECYCLE",
    "node-name": "ip-10-31-26-73.eu-west-1.compute.internal",
    "instance-id": "i-069246571a8ee057a",
    "provider-id": "aws:///eu-west-1c/i-069246571a8ee057a",
    "time": "2025-10-27T15:30:26Z",
    "message": "Waiting for node to be ready before completing ASG launch lifecycle"  #<==
}

How you tested your changes:
Environment (Linux / Windows): Linux
Kubernetes Version: v1.31.13

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@baburciu baburciu marked this pull request as ready for review October 27, 2025 14:40
@baburciu baburciu requested a review from a team as a code owner October 27, 2025 14:40
@baburciu baburciu changed the title Fix misleading logging for launch lifecycle events logged as drain operations Fix misleading message for launch lifecycle events logged as drain operations Oct 27, 2025
@baburciu baburciu force-pushed the bb/fix-launch-event-message branch from 380d9ee to 3a1915d Compare October 27, 2025 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant