ANSIBLE LINK

RESTful API for executing Ansible playbooks remotely

Features

• Playbook Execution Asynchronous playbook executions with real-time status updates.
• Playbook History Keep track of playbook executions and their status.
• API Documentation Swagger UI documentation for easy exploration of the API endpoints.
• Metrics Exposes Prometheus metrics for playbook runs, durations, and active jobs.
• Webhook Notifications Send notifications to Slack, Discord, or custom webhooks for job events.

NOTE Project is usable but still in early development.

Motivation

Searched for a way to run our playbooks automated without the need of AWX or other big projects while still being more stable and less error-prone than custom bash scripts. So I made Ansible-Link. This projects aims to be a KISS way to run ansible jobs remotely. Essentially a RESTful API sitting on top of ansible-runner.

Prerequisites

• Ansible CLI installed
• Your playbooks and inventory files

Installation

The fastest way to set up Ansible-Link is by using the provided install.sh script:

Download and run the install script:

wget https://raw.githubusercontent.com/lfkdev/ansible-link/main/install.sh
sudo bash install.sh

This script will:

• Check for necessary dependencies and install them if missing.
• Download and install Ansible-Link.
• Set up a Python virtual environment.
• Configure a systemd service for Ansible-Link.

After the installation, you can start using Ansible-Link immediately. You probably need to change some config values for your ansible environment /opt/ansible-link/config.yml

playbook_dir: '/etc/ansible/'
inventory_file: '/etc/ansible/environments/hosts'
...

To add more workers or change the user, modify /etc/systemd/system/ansible-link.service

⚠️ Note: Currently, only Ubuntu versions 16.04 and higher, or Debian versions 9 and higher are officially supported. Other operating systems might also work but have not been tested. You can clone the repository and perform a manual installation if you are using a different OS.

API Documentation

The API documentation is available via the Swagger UI.

API Endpoints

• POST /ansible/playbook: Execute a playbook
• GET /ansible/jobs: List all jobs
• GET /ansible/job/<job_id>: Get job status
• GET /ansible/job/<job_id>/output: Get job output
• GET /health: Health check endpoint

Configuration

The API configuration is stored in the config.yml file. If you move your config to a different location you can use ANSIBLE_LINK_CONFIG_PATH

$ export ANSIBLE_LINK_CONFIG_PATH='/etc/ansible-link/config.yml'

You can customize the following settings:

# webhook
# webhook:
#   url: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"
#   type: "slack" # "slack", "discord" or "generic" supported
#   timeout: 5  # optional, default 5 seconds

# flask
host: '127.0.0.1'
port: 5001
debug: false

# ansible-runner
suppress_ansible_output: false
omit_event_data: false
only_failed_event_data: false

# promtetheus
metrics_port: 9090

# general
playbook_dir: '/etc/ansible/'
inventory_file: '/etc/ansible/environments/hosts'
job_storage_dir: '/var/lib/ansible-link/job-storage'
log_level: 'INFO'

# ansible-link
playbook_whitelist: []
# playbook_whitelist:
#   - monitoring.yml
#   - mariadb.yml

The whitelist supports full regex, you could go wild:

playbook_whitelist:
  # Allow all playbooks in the 'test' directory
  - ^test/.*\.ya?ml$

  # Allow playbooks starting with 'prod_' or 'dev_'
  - ^(prod|dev)_.*\.ya?ml$

  # Allow specific playbooks
  - ^(backup|restore|maintenance)\.ya?ml$

Leave empty to allow all playbooks. This is for the backend, you could also use the limit arg from ansible-runner in the request directly.

Prod environment

You can use the install script install.sh to get a production-ready environment for Ansible-Link.

The install script will:

• Set up a Python VENV
• Configure a systemd service to manage Ansible-Link
• Utilize Gunicorn as the WSGI server
• Start Ansible-Link to liste only on localhost

You can use a webserver like Caddy to add Basic Authentication and TLS to your Ansible-Link setup.

Also, if you're using a webserver, you can change Gunicorn to use sockets instead. For example:

ExecStart=$VENV_DIR/bin/gunicorn --workers 1 --bind unix:$INSTALL_DIR/ansible_link.sock -m 007 wsgi:application

unitD example

[Unit]
Description=Ansible Link Service
After=network.target

[Service]
ExecStart=$VENV_DIR/bin/gunicorn --workers 1 --bind 127.0.0.1:$ANSIBLE_LINK_PORT wsgi:application
WorkingDirectory=$INSTALL_DIR
Restart=always
User=root

[Install]
WantedBy=multi-user.target

Example setup:

├── etc/
│   └── ansible/
│       ├── playbooks/
│       │   └── some_playbooks.yml
│       └── inventory/
│           ├── production
│           └── staging
│
├── opt/
│   └── ansible-link/
│       ├── ansible-link.py
│       └── config.yml
│
└── var/
    └── lib/
        └── ansible-link/
            └── job-storage/
                └── playbook_name_20230624_130000_job_id.json

Webhook Configuration

Ansible-Link supports sending webhook notifications for job events. You can configure webhooks for Slack, Discord, or a generic endpoint. Add the following to your config.yml:

webhook:
  url: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"
  type: "slack"  # Options: slack, discord, generic
  timeout: 5  # Optional, defaults to 5 seconds

or leave it commented out to disable webhooks

• url The webhook URL for your chosen platform.
• type The type of webhook (slack, discord, or generic).
• timeout The timeout for webhook requests in seconds (optional, default is 5 seconds).

Only Slack and Discord are supported for now, you can also use generic which will send the base JSON payload:

{
    "event_type": event_type,
    "job_id": job_data['job_id'],
    "playbook": job_data['playbook'],
    "status": job_data['status'],
    "timestamp": datetime.now().isoformat()
}

The following notifcations are sent:

• Job Started
• Job Completed (success or failure)
• Job Error

View webhook.py for more info.

Usage

Below are examples demonstrating how to use ansible-link API compared to Ansible CLI.

---

$ ansible-playbook site.yml

{
  "playbook": "site.yml"
}

curl -X POST http://your-ansible-link-server/ansible/playbook \
  -H "Content-Type: application/json" \
  -d '{"playbook": "site.yml"}'

---

$ ansible-playbook deploy.yml -e version=1.5.0 environment=staging

{
  "playbook": "deploy.yml",
  "vars": {
    "version": "1.5.0",
    "environment": "staging"
  }
}

---

$ ansible-playbook site.yml --tags "update,packages" -vv

{
  "playbook": "site.yml",
  "tags": "update,packages",
  "verbosity": 2
}

---

$ ansible-playbook restore.yml --limit "databases" --forks 3

{
  "playbook": "restore.yml",
  "limit": "databases",
  "forks": 3
}

---

$ ansible-playbook site.yml -i custom_inventory.ini -e '{"key1": "value1", "key2": "value2"}' --tags "provision,configure" --skip-tags "cleanup" --limit "webservers:&staged" --forks 10 -vvv

{
  "playbook": "site.yml",
  "inventory": "custom_inventory.ini",
  "vars": {
    "key1": "value1",
    "key2": "value2"
  },
  "tags": "provision,configure",
  "skip_tags": "cleanup",
  "limit": "webservers:&staged",
  "forks": 10,
  "verbosity": 3
}

---

$ ansible-playbook site.yml -i custom_inventory.ini -e environment=production --diff --check

{
  "playbook": "site.yml",
  "inventory": "custom_inventory.ini",
  "vars": {
    "environment": "production"
  },
  "cmdline": "--diff --check"
}

Ansible-Link supports the following native parameters:

• playbook: The name of the playbook to run (required)
• inventory: Path to the inventory file
• vars (extravars): A dictionary of additional variables to pass to the playbook
• limit: A host pattern to further constrain the list of hosts
• verbosity: Control the output level of ansible-playbook
• forks: Specify number of parallel processes to use
• tags: Only run plays and tasks tagged with these values
• skip_tags: Only run plays and tasks whose tags do not match these values
• cmdline: Any additional command line options to pass to ansible-playbook

Which means you can always use cmdline if your arg is not natively supported, like:

{
  "playbook": "site.yml",
  "cmdline": "--diff --check -e environment=production -i /etc/ansible/test/custom_inventory.ini"
}

Output

Ansible-Link will save each job as .json with the following info (from ansible-runner):

{
  "status": "successfull",
  "playbook": "<playbook_name>",
  "inventory": null,
  "vars": {
    "customer": "emind"
  },
 "start_time": "2024-06-24T15:32:35.380662",
  "stdout": "<ANSIBLE PLAYBOOK OUTPUT>",
  "stderr": "",
  "stats": {
    "skipped": {
      "<playbook_name>": 8
    },
    "ok": {
      "<playbook_name>": 28
    },
    "dark": {},
    "failures": {
      "<playbook_name>": 1
    },
    "ignored": {},
    "rescued": {},
    "processed": {
      "<playbook_name>": 1
    },
    "changed": {}
  }
}

essentially showing everything ansible-playbook would display.

Note After submitting a request to the API, you will receive a job ID. You can use this job ID to check the status and retrieve the output of the playbook run using the /ansible/job/<job_id> and /ansible/job/<job_id>/output endpoints respectively.

Metrics

Ansible-Link exposes the following metrics:

PLAYBOOK_RUNS = Counter('ansible_link_playbook_runs_total', 'Total number of playbook runs', ['playbook', 'status'])
PLAYBOOK_DURATION = Histogram('ansible_link_playbook_duration_seconds', 'Duration of playbook runs in seconds', ['playbook'])
ACTIVE_JOBS = Gauge('ansible_link_active_jobs', 'Number of currently active jobs')

The metrics can be used to set alerts, track the history of jobs, monitor performance and so on

Security Considerations

• Use TLS in production
• Add basic auth

Contributing

Contributions are always welcome - if you find any issues or have suggestions for improvements, please open an issue or submit a pull request.

License

This project is licensed under the MPL2 License. See the LICENSE file for more information.