add linode as one of cloud providers (#1590)

* add linode as one of cloud providers

* add Linode into cloud provider list

* fix code style

* install requirements of ansible linode module

* Update prompts.yml

- Make the regions list more readable
- Assign us-east as the default region

* remove prompt of asking root password

* roles/common: Add sshd tasks

* cloud-linode/tasks: Fix LINODE_API_TOKEN env lookup

* docs: Add Linode to Ansible deploy docs

* docs: Add cloud-linode

* config: Use Ubuntu 20.04 on Linode

* README: syntax

* Linode stackscript support

* Linode stackscript fix

* linting

Co-authored-by: Jack Ivanov <17044561+jackivanov@users.noreply.github.com>
Co-authored-by: William Woodruff <william@yossarian.net>
Co-authored-by: William Woodruff <william.woodruff@trailofbits.com>
Co-authored-by: Jack Ivanov <e601809@gmail.com>

README.md

@@ -16,7 +16,7 @@ Algo VPN is a set of Ansible scripts that simplify the setup of a personal WireG
 * Blocks ads with a local DNS resolver (optional)
 * Sets up limited SSH users for tunneling traffic (optional)
 * Based on current versions of Ubuntu and strongSwan
-* Installs to DigitalOcean, Amazon Lightsail, Amazon EC2, Vultr, Microsoft Azure, Google Compute Engine, Scaleway, OpenStack, CloudStack, Hetzner Cloud, or [your own Ubuntu server (for more advanced users)](docs/deploy-to-ubuntu.md)
+* Installs to DigitalOcean, Amazon Lightsail, Amazon EC2, Vultr, Microsoft Azure, Google Compute Engine, Scaleway, OpenStack, CloudStack, Hetzner Cloud, Linode, or [your own Ubuntu server (for more advanced users)](docs/deploy-to-ubuntu.md)
 
 ## Anti-features
 
@@ -30,7 +30,7 @@ Algo VPN is a set of Ansible scripts that simplify the setup of a personal WireG
 
 The easiest way to get an Algo server running is to run it on your local system or from [Google Cloud Shell](docs/deploy-from-cloudshell.md) and let it set up a _new_ virtual machine in the cloud for you.
 
-1. **Setup an account on a cloud hosting provider.** Algo supports [DigitalOcean](https://m.do.co/c/4d7f4ff9cfe4) (most user friendly), [Amazon Lightsail](https://aws.amazon.com/lightsail/), [Amazon EC2](https://aws.amazon.com/), [Vultr](https://www.vultr.com/), [Microsoft Azure](https://azure.microsoft.com/), [Google Compute Engine](https://cloud.google.com/compute/), [Scaleway](https://www.scaleway.com/), [DreamCompute](https://www.dreamhost.com/cloud/computing/) or other OpenStack-based cloud hosting, [Exoscale](https://www.exoscale.com) or other CloudStack-based cloud hosting,  or [Hetzner Cloud](https://www.hetzner.com/).
+1. **Setup an account on a cloud hosting provider.** Algo supports [DigitalOcean](https://m.do.co/c/4d7f4ff9cfe4) (most user friendly), [Amazon Lightsail](https://aws.amazon.com/lightsail/), [Amazon EC2](https://aws.amazon.com/), [Vultr](https://www.vultr.com/), [Microsoft Azure](https://azure.microsoft.com/), [Google Compute Engine](https://cloud.google.com/compute/), [Scaleway](https://www.scaleway.com/), [DreamCompute](https://www.dreamhost.com/cloud/computing/), [Linode](https://www.linode.com), or other OpenStack-based cloud hosting, [Exoscale](https://www.exoscale.com) or other CloudStack-based cloud hosting,  or [Hetzner Cloud](https://www.hetzner.com/).
 
 2. **Get a copy of Algo.** The Algo scripts will be installed on your local system. There are two ways to get a copy:
 

config.cfg

@@ -198,6 +198,9 @@ cloud_providers:
   vultr:
     os: Ubuntu 20.04 x64
     size: 1024 MB RAM,25 GB SSD,1.00 TB BW
+  linode:
+    type: g6-nanode-1
+    image: linode/ubuntu20.04
   local:
 
 fail_hint:

docs/cloud-linode.md

@@ -0,0 +1,8 @@
+## API Token
+
+Sign into the Linode Manager and go to the
+[tokens management page](https://cloud.linode.com/profile/tokens).
+
+Click `Add a Personal Access Token`. Label your new token and select *at least* the
+`Linodes` read/write permission. Press `Submit` and make sure to copy the displayed token
+as it won't be shown again.

docs/deploy-from-ansible.md

@@ -51,6 +51,7 @@ Cloud roles:
 - role: cloud-openstack,    [provider: openstack](#openstack)
 - role: cloud-cloudstack,   [provider: cloudstack](#cloudstack)
 - role: cloud-hetzner,      [provider: hetzner](#hetzner)
+- role: cloud-linode,       [provider: linode](#linode)
 
 Server roles:
 
@@ -264,6 +265,13 @@ Required variables:
 - hcloud_token: Your [API token](https://trailofbits.github.io/algo/cloud-hetzner.html#api-token) - can also be defined in the environment as HCLOUD_TOKEN
 - region: e.g. `nbg1`
 
+### Linode
+
+Required variables:
+
+- linode_token: Your [API token](https://trailofbits.github.io/algo/cloud-linode.html#api-token) - can also be defined in the environment as LINODE_TOKEN
+- region: e.g. `us-east`
+
 ### Update users
 
 Playbook:

input.yml

@@ -21,6 +21,7 @@
       - { name: Scaleway, alias: scaleway}
       - { name: OpenStack (DreamCompute optimised), alias: openstack }
       - { name: CloudStack (Exoscale optimised), alias: cloudstack }
+      - { name: Linode, alias: linode }
       - { name: "Install to existing Ubuntu 18.04 or 20.04 server (for more advanced users)", alias: local }
   vars_files:
     - config.cfg

library/linode_stackscript_v4.py

@@ -0,0 +1,113 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import traceback
+
+from ansible.module_utils.basic import AnsibleModule, env_fallback, missing_required_lib
+from ansible.module_utils.linode import get_user_agent
+
+LINODE_IMP_ERR = None
+try:
+    from linode_api4 import StackScript, LinodeClient
+    HAS_LINODE_DEPENDENCY = True
+except ImportError:
+    LINODE_IMP_ERR = traceback.format_exc()
+    HAS_LINODE_DEPENDENCY = False
+
+
+def create_stackscript(module, client, **kwargs):
+    """Creates a stackscript and handles return format."""
+    try:
+        response = client.linode.stackscript_create(**kwargs)
+        return response._raw_json
+    except Exception as exception:
+        module.fail_json(msg='Unable to query the Linode API. Saw: %s' % exception)
+
+
+def stackscript_available(module, client):
+    """Try to retrieve a stackscript."""
+    try:
+        label = module.params['label']
+        desc = module.params['description']
+
+        result = client.linode.stackscripts(StackScript.label == label,
+                                            StackScript.description == desc,
+                                            mine_only=True
+                                          )
+        return result[0]
+    except IndexError:
+        return None
+    except Exception as exception:
+        module.fail_json(msg='Unable to query the Linode API. Saw: %s' % exception)
+
+
+def initialise_module():
+    """Initialise the module parameter specification."""
+    return AnsibleModule(
+        argument_spec=dict(
+            label=dict(type='str', required=True),
+            state=dict(
+                type='str',
+                required=True,
+                choices=['present', 'absent']
+            ),
+            access_token=dict(
+                type='str',
+                required=True,
+                no_log=True,
+                fallback=(env_fallback, ['LINODE_ACCESS_TOKEN']),
+            ),
+            script=dict(type='str', required=True),
+            images=dict(type='list', required=True),
+            description=dict(type='str', required=False),
+            public=dict(type='bool', required=False, default=False),
+        ),
+        supports_check_mode=False
+    )
+
+
+def build_client(module):
+    """Build a LinodeClient."""
+    return LinodeClient(
+        module.params['access_token'],
+        user_agent=get_user_agent('linode_v4_module')
+    )
+
+
+def main():
+    """Module entrypoint."""
+    module = initialise_module()
+
+    if not HAS_LINODE_DEPENDENCY:
+        module.fail_json(msg=missing_required_lib('linode-api4'), exception=LINODE_IMP_ERR)
+
+    client = build_client(module)
+    stackscript = stackscript_available(module, client)
+
+    if module.params['state'] == 'present' and stackscript is not None:
+        module.exit_json(changed=False, stackscript=stackscript._raw_json)
+
+    elif module.params['state'] == 'present' and stackscript is None:
+        stackscript_json = create_stackscript(
+            module, client,
+            label=module.params['label'],
+            script=module.params['script'],
+            images=module.params['images'],
+            desc=module.params['description'],
+            public=module.params['public'],
+        )
+        module.exit_json(changed=True, stackscript=stackscript_json)
+
+    elif module.params['state'] == 'absent' and stackscript is not None:
+        stackscript.delete()
+        module.exit_json(changed=True, stackscript=stackscript._raw_json)
+
+    elif module.params['state'] == 'absent' and stackscript is None:
+        module.exit_json(changed=False, stackscript={})
+
+
+if __name__ == "__main__":
+    main()

library/linode_v4.py

@@ -0,0 +1,142 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+
+# Copyright (c) 2017 Ansible Project
+# GNU General Public License v3.0+
+# (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
+
+from __future__ import absolute_import, division, print_function
+__metaclass__ = type
+
+import traceback
+
+from ansible.module_utils.basic import AnsibleModule, env_fallback, missing_required_lib
+from ansible.module_utils.linode import get_user_agent
+
+LINODE_IMP_ERR = None
+try:
+    from linode_api4 import Instance, LinodeClient
+    HAS_LINODE_DEPENDENCY = True
+except ImportError:
+    LINODE_IMP_ERR = traceback.format_exc()
+    HAS_LINODE_DEPENDENCY = False
+
+
+def create_linode(module, client, **kwargs):
+    """Creates a Linode instance and handles return format."""
+    if kwargs['root_pass'] is None:
+        kwargs.pop('root_pass')
+
+    try:
+        response = client.linode.instance_create(**kwargs)
+    except Exception as exception:
+        module.fail_json(msg='Unable to query the Linode API. Saw: %s' % exception)
+
+    try:
+        if isinstance(response, tuple):
+            instance, root_pass = response
+            instance_json = instance._raw_json
+            instance_json.update({'root_pass': root_pass})
+            return instance_json
+        else:
+            return response._raw_json
+    except TypeError:
+        module.fail_json(msg='Unable to parse Linode instance creation'
+                             ' response. Please raise a bug against this'
+                             ' module on https://github.com/ansible/ansible/issues'
+                         )
+
+
+def maybe_instance_from_label(module, client):
+    """Try to retrieve an instance based on a label."""
+    try:
+        label = module.params['label']
+        result = client.linode.instances(Instance.label == label)
+        return result[0]
+    except IndexError:
+        return None
+    except Exception as exception:
+        module.fail_json(msg='Unable to query the Linode API. Saw: %s' % exception)
+
+
+def initialise_module():
+    """Initialise the module parameter specification."""
+    return AnsibleModule(
+        argument_spec=dict(
+            label=dict(type='str', required=True),
+            state=dict(
+                type='str',
+                required=True,
+                choices=['present', 'absent']
+            ),
+            access_token=dict(
+                type='str',
+                required=True,
+                no_log=True,
+                fallback=(env_fallback, ['LINODE_ACCESS_TOKEN']),
+            ),
+            authorized_keys=dict(type='list', required=False),
+            group=dict(type='str', required=False),
+            image=dict(type='str', required=False),
+            region=dict(type='str', required=False),
+            root_pass=dict(type='str', required=False, no_log=True),
+            tags=dict(type='list', required=False),
+            type=dict(type='str', required=False),
+            stackscript_id=dict(type='int', required=False),
+        ),
+        supports_check_mode=False,
+        required_one_of=(
+            ['state', 'label'],
+        ),
+        required_together=(
+            ['region', 'image', 'type'],
+        )
+    )
+
+
+def build_client(module):
+    """Build a LinodeClient."""
+    return LinodeClient(
+        module.params['access_token'],
+        user_agent=get_user_agent('linode_v4_module')
+    )
+
+
+def main():
+    """Module entrypoint."""
+    module = initialise_module()
+
+    if not HAS_LINODE_DEPENDENCY:
+        module.fail_json(msg=missing_required_lib('linode-api4'), exception=LINODE_IMP_ERR)
+
+    client = build_client(module)
+    instance = maybe_instance_from_label(module, client)
+
+    if module.params['state'] == 'present' and instance is not None:
+        module.exit_json(changed=False, instance=instance._raw_json)
+
+    elif module.params['state'] == 'present' and instance is None:
+        instance_json = create_linode(
+            module, client,
+            authorized_keys=module.params['authorized_keys'],
+            group=module.params['group'],
+            image=module.params['image'],
+            label=module.params['label'],
+            region=module.params['region'],
+            root_pass=module.params['root_pass'],
+            tags=module.params['tags'],
+            ltype=module.params['type'],
+            stackscript_id=module.params['stackscript_id'],
+        )
+        module.exit_json(changed=True, instance=instance_json)
+
+    elif module.params['state'] == 'absent' and instance is not None:
+        instance.delete()
+        module.exit_json(changed=True, instance=instance._raw_json)
+
+    elif module.params['state'] == 'absent' and instance is None:
+        module.exit_json(changed=False, instance={})
+
+
+if __name__ == "__main__":
+    main()

roles/cloud-linode/defaults/main.yml

@@ -0,0 +1,2 @@
+---
+linode_venv: "{{ playbook_dir }}/configs/.venvs/linode"

roles/cloud-linode/tasks/main.yml

@@ -0,0 +1,56 @@
+---
+- name: Build python virtual environment
+  import_tasks: venv.yml
+
+- name: Include prompts
+  import_tasks: prompts.yml
+
+- name: Set facts
+  set_fact:
+    stackscript: |
+      {{ lookup('template', 'files/cloud-init/base.sh') }}
+      mkdir -p /var/lib/cloud/data/ || true
+      touch /var/lib/cloud/data/result.json
+
+- name: Create a stackscript
+  linode_stackscript_v4:
+    access_token: "{{ algo_linode_token }}"
+    label: "{{ algo_server_name }}"
+    state: present
+    description: Environment:Algo
+    images:
+      - "{{ cloud_providers.linode.image }}"
+    script: |
+      {{ stackscript }}
+  register: _linode_stackscript
+
+- name: Update the stackscript
+  uri:
+    url: "https://api.linode.com/v4/linode/stackscripts/{{ _linode_stackscript.stackscript.id }}"
+    method: PUT
+    body_format: json
+    body:
+      script: |
+        {{ stackscript }}
+    headers:
+      Content-Type: application/json
+      Authorization: "Bearer {{ algo_linode_token }}"
+  when: (_linode_stackscript.stackscript.script | hash('md5')) != (stackscript | hash('md5'))
+
+- name: "Creating an instance..."
+  linode_v4:
+    access_token: "{{ algo_linode_token }}"
+    label: "{{ algo_server_name }}"
+    state: present
+    region: "{{ algo_linode_region }}"
+    image: "{{ cloud_providers.linode.image }}"
+    type: "{{ cloud_providers.linode.type }}"
+    authorized_keys: "{{ public_key }}"
+    stackscript_id: "{{ _linode_stackscript.stackscript.id }}"
+  register: _linode
+
+- set_fact:
+    cloud_instance_ip: "{{ _linode.instance.ipv4[0] }}"
+    ansible_ssh_user: algo
+    ansible_ssh_port: "{{ ssh_port }}"
+    cloudinit: true

roles/cloud-linode/tasks/prompts.yml

@@ -0,0 +1,51 @@
+---
+- pause:
+    prompt: |
+      Enter your ACCESS token. (https://developers.linode.com/api/v4/#access-and-authentication):
+    echo: false
+  register: _linode_token
+  when:
+    - linode_token is undefined
+    - lookup('env','LINODE_API_TOKEN')|length <= 0
+
+- name: Set the token as a fact
+  set_fact:
+    algo_linode_token: "{{ linode_token | default(_linode_token.user_input|default(None)) | default(lookup('env','LINODE_API_TOKEN'), true) }}"
+
+- name: Get regions
+  uri:
+    url: https://api.linode.com/v4/regions
+    method: GET
+    status_code: 200
+  register: _linode_regions
+
+- name: Set facts about the regions
+  set_fact:
+    linode_regions: "{{ _linode_regions.json.data | sort(attribute='id') }}"
+
+- name: Set default region
+  set_fact:
+    default_region: >-
+      {% for r in linode_regions %}
+      {%- if r['id'] == "us-east" %}{{ loop.index }}{% endif %}
+      {%- endfor %}
+
+- pause:
+    prompt: |
+      What region should the server be located in?
+        {% for r in linode_regions %}
+        {{ loop.index }}. {{ r['id'] }}
+        {% endfor %}
+
+      Enter the number of your desired region
+      [{{ default_region }}]
+  register: _algo_region
+  when: region is undefined
+
+- name: Set additional facts
+  set_fact:
+    algo_linode_region: >-
+      {% if region is defined %}{{ region }}
+      {%- elif _algo_region.user_input %}{{ linode_regions[_algo_region.user_input | int -1 ]['id'] }}
+      {%- else %}{{ linode_regions[default_region | int - 1]['id'] }}{% endif %}
+    public_key: "{{ lookup('file', '{{ SSH_keys.public }}') }}"

roles/cloud-linode/tasks/venv.yml

@@ -0,0 +1,7 @@
+---
+- name: Install requirements
+  pip:
+    name:
+      - linode_api4
+    state: latest
+    virtualenv_python: python3