Note To Self: How To Delete AWS SageMaker's Endpoint With MonitoringSchedule

The Story

I have recently been deep diving into AWS SageMaker. I will document my journey in another blog post stick around!

This short post will show you how to delete an endpoint with a monitoring schedule. For some reason, this isn't possible with the AWS console; which I find very odd.

If you are here and have no idea what Endpoint with Monitoring Schedule is, you can read this Amazon SageMaker Model Monitor docs. If like me, you rather read the shorter version and were lazy to read the AWS docs, here is the short version:
With SageMaker Model Monitor, you can do the following:

• Monitor data quality and model accuracy drift.
• Monitor bias in your model's predictions.
• Monitor drift in feature attribution.

This post will walk you through how to delete an endpoint with a monitoring schedule.

TL;DR

Delete an endpoint with a monitoring schedule via AWS CLI.

The Walk-through

If like me, you have tried to delete an endpoint with a monitoring schedule, you will have noticed that it is not possible. See the dreaded and cryptic error message below!

Fear not I have a solution.
We first need to delete the MonitoringSchedules configured with the endpoint via the AWS CLI tool.

On the SageMaker terminal, run the following command:

1. SageMaker instances do not come pre-installed with jq, so first things first install it.

  # Ref: https://stedolan.github.io/jq/
  sudo yum install jq

1. Let's get the region of the endpoint we want to delete.

  $ REGION=$(python -c 'import boto3; print(boto3.Session().region_name)')
  $ echo "REGION: $REGION"

  REGION: us-east-1

1. Get the list of MonitoringSchedules available

  $ aws sagemaker list-monitoring-schedules --region $REGION | jq '.'
  {
    "MonitoringScheduleSummaries": [
      {
        "MonitoringScheduleName": "my-monitoring-schedule",
        "MonitoringScheduleArn": "arn:aws:sagemaker:us-east-1:853052508252:monitoring-schedule/my-monitoring-schedule",
        "CreationTime": 1635378407.474,
        "LastModifiedTime": 1635476955.122,
        "MonitoringScheduleStatus": "Scheduled",
        "EndpointName": "xgboost-2021-10-27-23-31-41-439",
        "MonitoringJobDefinitionName": "data-quality-job-definition-2021-10-27-23-46-47-211",
        "MonitoringType": "DataQuality"
      }
    ]
  }

1. Get the name of your MonitoringSchedules and pass it to delete the monitoring schedule

  MON_NAME=$(aws sagemaker list-monitoring-schedules --region $REGION | jq -r '.MonitoringScheduleSummaries[].MonitoringScheduleName')
  aws sagemaker delete-monitoring-schedule --monitoring-schedule-name $MON_NAME --region $REGION

1. Now we can delete the Endpoint with no issues, first get the endpoint name you want to delete

  $ aws sagemaker list-endpoints --region $REGION | jq "."
  {
      "Endpoints": [
          {
              "EndpointName": "xgboost-2021-10-27-23-31-41-439",
              "EndpointArn": "arn:aws:sagemaker:us-east-1:853052508252:endpoint/xgboost-2021-10-27-23-31-41-439",
              "CreationTime": 1635377502.453,
              "LastModifiedTime": 1635378004.108,
              "EndpointStatus": "InService"
          }
      ]
  }

1. With the endpoint name, delete the endpoint

  $ ENDPOINT_NAME=$(aws sagemaker list-endpoints --region $REGION | jq -r ".Endpoints[].EndpointName")
  $ aws sagemaker delete-endpoint --endpoint-name $ENDPOINT_NAME --region $REGION
  $ aws sagemaker list-endpoints --region $REGION
  {
      "Endpoints": []
  }

NB: Always, make sure to delete the endpoint and other resources after you are done to avoid cost!