aws/agent-toolkit-for-aws

running-release-tests

- Run automated release testing (UI or API) via the AWS DevOps Agent using a pre-configured test profile.

Voir la source

Mode d’emploi

  1. Copiez la commande et examinez le paquet avant de l’installer dans votre environnement agent.
  2. Consultez le document original pour les outils, identifiants et environnements requis.
  3. Décrivez les entrées et le livrable souhaité ; comparez la sortie aux sources.
npx skills add https://github.com/aws/agent-toolkit-for-aws --skill running-release-tests

Avant de commencer

  • L’installation n’inclut ni abonnement au modèle, ni quota API, ni accès aux fichiers.
  • Étoiles et installations reflètent l’activité, pas un test indépendant de qualité ou de sécurité.
Document Skill original

Rendu depuis le dépôt source en conservant titres, exemples, code, tableaux, liens et images.

Release Testing

AgentSpace routing (SigV4 only): If list_agent_spaces is available in your tool list and the multi-space orchestration skill has NOT been invoked yet this session, invoke it first to determine which agent_space_id to use. Then pass agent_space_id on all tool calls below. For bearer token auth this is unnecessary — the token is already scoped to one space.

Run automated release testing in the cloud via the AWS DevOps Agent's Release Testing Agent. Supports UI testing (browser-based) and API testing (OpenAPI spec-based). Uses pre-existing test profiles that define target URL, agent type, personas, and credentials.

Input is a test profile — the test profile already contains the target URL, agent type (UI or API), test personas, and credentials. Do NOT ask the user for a URL directly; the URL is defined in the test profile.

Prerequisites

  • A pre-existing test profile (Knowledge Item ID like ki-12345) created from the AWS DevOps Agent console

Gathering test parameters

Before starting any workflow, you MUST gather the following parameters. Do NOT proceed to job creation until answered.

Step 1 — Test profile (required)

Ask the user which test profile to use. The test profile already contains the target URL, agent type (UI or API), test personas, and credentials configuration — these do NOT need to be gathered separately.

Note: A pre-existing test profile is a prerequisite. Test profiles are created using the AWS DevOps Agent console or API, not through this tool. If the user asks whether one can be created here, inform them it must already exist.

Step 2 — Test requirement (optional)

If the user has not already mentioned a test focus, ask:

"Do you have a specific test requirement or focus area? If not, I'll run a full exploratory test."

Wait for the user's response. If they provide one, use it as the test_requirement. If they say no or skip, proceed without it.

IMPORTANT: You MUST wait for the user to respond before proceeding to job creation.

Core workflow

1. Select Agent Space

List available agent spaces:

aws devops-agent list-agent-spaces --region us-east-1

Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected agentSpaceId as SPACE_ID in all subsequent calls.

2. Check tool availability

Verify that the following tools are available: aws_devops_agent__create_release_testing_job, aws_devops_agent__get_task, aws_devops_agent__list_journal_records, aws_devops_agent__get_release_ui_testing_report, aws_devops_agent__get_release_api_testing_report. These tools are NOT deferred/lazy-loaded — if they do not appear in your tool list, they are unavailable. Do NOT search for them via ToolSearch. If any are missing, skip the remaining steps in this section and use the "Fallback (aws-mcp)" path below instead.

3. Start the Job

aws_devops_agent__create_release_testing_job(
    test_profile_id="ki-12345",
    webhook_event_message="<optional test requirement>"
)
→ {"taskId": "...", "executionId": "...", "status": "started"}

Record the taskId and executionId from the response.

4. Poll for Status

Call aws_devops_agent__get_task(task_id=TASK_ID) every 30 seconds until the status transitions to IN_PROGRESS or a terminal state.

5. Monitor Until Completion

Once IN_PROGRESS, poll for progress in a loop:

  1. Call aws_devops_agent__list_journal_records(execution_id=EXEC_ID, order="ASC") to fetch new findings.
  2. Present each record to the user with a friendly progress update.
  3. Use next_token from the response to fetch only new records on subsequent polls.
  4. Wait 20 seconds between each poll iteration.
  5. Check aws_devops_agent__get_task(task_id=TASK_ID) periodically — stop when terminal status (COMPLETED, FAILED, CANCELED, TIMED_OUT).

6. Present Results

Once the job reaches a terminal status:

  • If COMPLETED:
  1. Determine the report type from the test profile's agent type (UI or API). Call aws_devops_agent__get_release_ui_testing_report(execution_id=EXEC_ID) for UI profiles or aws_devops_agent__get_release_api_testing_report(execution_id=EXEC_ID) for API profiles.
  2. Write the report contents to a markdown file:
     release-testing-report-<YYYY-MM-DD-HHmmss>.md
  1. Inform the user that the report was saved, including the file path.
  • If FAILED or TIMED_OUT: Present the error information and suggest next steps.
  • If CANCELED: Inform the user the job was canceled and no report is available.

Cancelling a job

aws_devops_agent__cancel_release_testing_job(task_id=TASK_ID)

Error handling

  1. If the task status changes to FAILED, stop the workflow and report the error.
  2. If the task does not reach IN_PROGRESS within 5 minutes, cancel it using cancel_release_testing_job.
  3. If any output contains "NoCredentialsError", "ExpiredTokenException", or auth failures, suggest the user refresh their credentials or check the bearer token.
  4. If throttled (429 or ThrottlingException), wait 30 seconds before retrying. After 3 retries, inform the user.

Fallback (aws-mcp)

If the aws-devops-agent remote server is unavailable, use the AWS CLI directly:

Tell the user: "Remote server unavailable — using direct AWS API fallback."

1. Select Agent Space

List available agent spaces:

aws devops-agent list-agent-spaces --region us-east-1

Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected agentSpaceId as SPACE_ID in all subsequent calls.

2. Start the Job

aws devops-agent create-backlog-task \
  --agent-space-id SPACE_ID \
  --task-type RELEASE_TESTING \
  --title 'Release Testing' \
  --priority MEDIUM \
  --description '{\"testProfileId\": \"<PROFILE_ID>\", \"webhookEventMessage\": \"<REQUIREMENT>\"}' \
  --region us-east-1

If the user provided a test requirement, include it as webhookEventMessage. If not, omit the field or leave it empty.

3. Poll for Status

aws devops-agent get-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --region us-east-1

Poll every 30 seconds until the status transitions to IN_PROGRESS or a terminal state (COMPLETED, FAILED, CANCELED, TIMED_OUT).

4. Monitor Until Completion

Once IN_PROGRESS, poll for progress in a loop:

aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --order ASC \
  --region us-east-1
  1. Present each record to the user with a friendly progress update.
  2. Use next_token from the response to fetch only new records on subsequent polls.
  3. Wait 20 seconds between each poll iteration.
  4. Check get-backlog-task periodically — stop when terminal status (COMPLETED, FAILED, CANCELED, TIMED_OUT).

5. Present Results

Once the job reaches a terminal status:

  • If COMPLETED:
  1. Retrieve the report using the appropriate record type:
  • UI testing: --record-type qa_ui_testing_report
  • API testing: --record-type qa_api_testing_report
     aws devops-agent list-journal-records \
       --agent-space-id SPACE_ID \
       --execution-id EXEC_ID \
       --record-type qa_ui_testing_report \
       --order ASC \
       --region us-east-1
  1. Write the report contents to a markdown file:
     release-testing-report-<YYYY-MM-DD-HHmmss>.md
  1. Inform the user that the report was saved, including the file path.
  • If FAILED or TIMED_OUT: Present the error information and suggest next steps.
  • If CANCELED: Inform the user the job was canceled and no report is available.

Cancelling (fallback)

aws devops-agent update-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --task-status CANCELED \
  --region us-east-1

Tâches connexes

du même dépôt

Autres Skills

Tous les Skills
aws
Officiel

deploying-containers-to-amazon-lightsail

Deploys, hosts, or runs a container or Docker app on Amazon Lightsail Container Service and returns a live, managed HTTPS URL at a flat monthly price — no cluster, VPC, load balancer, or registry to set up. Used when deploying a container image to Lightsail from a local Docker build, a public registry image, or a GitHub repo, or when a Lightsail container deployment fails or won't serve — including when the request doesn't name Lightsail but asks to run one always-on container behind a public UR

Installations / 1h
4
GitHub Stars
3,5 k
Mis à jour
20 nov.
aws
Officiel

managing-data-asset-access

Grants, requests, audits and removes access to data assets in an AWS data catalog (the Amazon SageMaker Catalog supported by Amazon DataZone) through catalog subscriptions over published listings. Covers requesting a subscription, approving or rejecting requests with a recorded decision, sharing a listing with a project, revoking one or many subscriptions, and confirming grant settlement. Triggers on: give another team access to a table, request access to a dataset, approve or reject an access r

Installations / 1h
4
GitHub Stars
3,5 k
Mis à jour
20 nov.
aws
Officiel

managing-data-catalog

Writes and curates business metadata in an AWS data catalog: the Amazon SageMaker Catalog (Amazon DataZone), or the AWS Glue Data Catalog. Detects which catalog platform an account uses before writing. Triggers on: add a description to a table, document a dataset, write a README, set an owner, add column comments, tag with a glossary term, create a business glossary, create a metadata form, publish an asset, create a data product, run the data source crawl, delete a catalog asset, attach a conve

Installations / 1h
4
GitHub Stars
3,5 k
Mis à jour
20 nov.
aws
Officiel

aws-billing-and-cost-management

Analyze AWS costs, find savings, manage budgets, evaluate Savings Plans and Reserved Instances, right-size EC2/Lambda/RDS/EBS with Compute Optimizer, look up service pricing, query CUR with Athena, detect cost anomalies, scope costs to billing views, and monitor Free Tier usage. Triggers on: AWS bill, cost analysis, reduce spend, savings plan, reserved instance, right-size, budget alert, cost optimization, pricing, free tier, cost anomaly, CUR, cost audit, billing view, billing view ARN.

Installations / 1h
2
GitHub Stars
3,5 k
Mis à jour
19 nov.