feat: Add AWS credentials file support (#14778)

* feat: Add AWS credentials file support

- Automatically reads AWS credentials from ~/.aws/credentials
- Supports AWS_PROFILE and AWS_SHARED_CREDENTIALS_FILE environment variables
- Adds support for temporary credentials with session tokens
- Maintains backward compatibility with existing credential methods
- Follows standard AWS credential precedence order

Based on PR #14460 by @lefth with the following improvements:
- Fixed variable naming to match existing code (access_key vs aws_access_key)
- Added session token support for temporary credentials
- Integrated credential discovery directly into prompts.yml
- Added comprehensive tests
- Added documentation

Closes #14382

* fix ansible lint

---------

Co-authored-by: Jack Ivanov <17044561+jackivanov@users.noreply.github.com>

docs/aws-credentials.md

@@ -0,0 +1,61 @@
+# AWS Credential Configuration
+
+Algo supports multiple methods for providing AWS credentials, following standard AWS practices:
+
+## Methods (in order of precedence)
+
+1. **Command-line variables** (highest priority)
+   ```bash
+   ./algo -e "aws_access_key=YOUR_KEY aws_secret_key=YOUR_SECRET"
+   ```
+
+2. **Environment variables**
+   ```bash
+   export AWS_ACCESS_KEY_ID=YOUR_KEY
+   export AWS_SECRET_ACCESS_KEY=YOUR_SECRET
+   export AWS_SESSION_TOKEN=YOUR_TOKEN  # Optional, for temporary credentials
+   ./algo
+   ```
+
+3. **AWS credentials file** (lowest priority)
+   - Default location: `~/.aws/credentials`
+   - Custom location: Set `AWS_SHARED_CREDENTIALS_FILE` environment variable
+   - Profile selection: Set `AWS_PROFILE` environment variable (defaults to "default")
+
+## Using AWS Credentials File
+
+After running `aws configure` or manually creating `~/.aws/credentials`:
+
+```ini
+[default]
+aws_access_key_id = YOUR_KEY_ID
+aws_secret_access_key = YOUR_SECRET_KEY
+
+[work]
+aws_access_key_id = WORK_KEY_ID
+aws_secret_access_key = WORK_SECRET_KEY
+aws_session_token = TEMPORARY_TOKEN  # Optional
+```
+
+To use a specific profile:
+```bash
+AWS_PROFILE=work ./algo
+```
+
+## Security Considerations
+
+- Credentials files should have restricted permissions (600)
+- Consider using AWS IAM roles or temporary credentials when possible
+- Tools like [aws-vault](https://github.com/99designs/aws-vault) can provide additional security by storing credentials encrypted
+
+## Troubleshooting
+
+If Algo isn't finding your credentials:
+
+1. Check file permissions: `ls -la ~/.aws/credentials`
+2. Verify the profile name matches: `AWS_PROFILE=your-profile`
+3. Test with AWS CLI: `aws sts get-caller-identity`
+
+If credentials are found but authentication fails:
+- Ensure your IAM user has the required permissions (see [EC2 deployment guide](deploy-from-ansible.md))
+- Check if you need session tokens for temporary credentials

docs/cloud-amazon-ec2.md

@@ -81,7 +81,14 @@ Enter the number of your desired provider
 : 3
 ```
 
-Next, you will be asked for the AWS Access Key (Access Key ID) and AWS Secret Key (Secret Access Key) that you received in  the CSV file when you setup the account (don't worry if you don't see your text entered in the console; the key input is  hidden here by Algo).
+Next, Algo will need your AWS credentials. If you have already configured AWS CLI with `aws configure`, Algo will automatically use those credentials. Otherwise, you will be asked for the AWS Access Key (Access Key ID) and AWS Secret Key (Secret Access Key) that you received in the CSV file when you setup the account (don't worry if you don't see your text entered in the console; the key input is hidden here by Algo).
+
+**Automatic credential detection**: Algo will check for credentials in this order:
+1. Command-line variables
+2. Environment variables (`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`)
+3. AWS credentials file (`~/.aws/credentials`)
+
+If none are found, you'll see these prompts:
 
 ```
 Enter your aws_access_key (http://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html)
@@ -94,6 +101,8 @@ Enter your aws_secret_key (http://docs.aws.amazon.com/general/latest/gr/managing
 [ABCD...]:
 ```
 
+For more details on credential configuration, see the [AWS Credentials guide](aws-credentials.md).
+
 You will be prompted for the server name to enter. Feel free to leave this as the default ("algo") if you are not certain how this will affect your setup. Here we chose to call it "algovpn".
 
 ```

roles/cloud-ec2/tasks/cloudformation.yml

@@ -3,6 +3,7 @@
   cloudformation:
     aws_access_key: "{{ access_key }}"
     aws_secret_key: "{{ secret_key }}"
+    aws_session_token: "{{ session_token if session_token else omit }}"
     stack_name: "{{ stack_name }}"
     state: present
     region: "{{ algo_region }}"

roles/cloud-ec2/tasks/prompts.yml

@@ -1,4 +1,30 @@
 ---
+# Discover AWS credentials from standard locations
+- name: Set AWS credentials file path
+  set_fact:
+    aws_credentials_path: "{{ lookup('env', 'AWS_SHARED_CREDENTIALS_FILE') | default(lookup('env', 'HOME') + '/.aws/credentials', true) }}"
+    aws_profile: "{{ lookup('env', 'AWS_PROFILE') | default('default', true) }}"
+
+# Try to read credentials from file if not already provided
+- block:
+    - name: Check if AWS credentials file exists
+      stat:
+        path: "{{ aws_credentials_path }}"
+      register: aws_creds_file
+      delegate_to: localhost
+
+    - name: Read AWS credentials from file
+      set_fact:
+        _file_access_key: "{{ lookup('ini', 'aws_access_key_id', section=aws_profile, file=aws_credentials_path, errors='ignore') | default('', true) }}"
+        _file_secret_key: "{{ lookup('ini', 'aws_secret_access_key', section=aws_profile, file=aws_credentials_path, errors='ignore') | default('', true) }}"
+        _file_session_token: "{{ lookup('ini', 'aws_session_token', section=aws_profile, file=aws_credentials_path, errors='ignore') | default('', true) }}"
+      when: aws_creds_file.stat.exists
+      no_log: true
+  when:
+    - aws_access_key is undefined
+    - lookup('env','AWS_ACCESS_KEY_ID')|length <= 0
+
+# Prompt for credentials if still not available
 - pause:
     prompt: |
       Enter your AWS Access Key ID (http://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html)
@@ -8,6 +34,7 @@
   when:
     - aws_access_key is undefined
     - lookup('env','AWS_ACCESS_KEY_ID')|length <= 0
+    - _file_access_key is undefined or _file_access_key|length <= 0
 
 - pause:
     prompt: |
@@ -17,16 +44,33 @@
   when:
     - aws_secret_key is undefined
     - lookup('env','AWS_SECRET_ACCESS_KEY')|length <= 0
+    - _file_secret_key is undefined or _file_secret_key|length <= 0
 
+# Set final credentials with proper precedence
 - set_fact:
-    access_key: "{{ aws_access_key | default(_aws_access_key.user_input|default(None)) | default(lookup('env','AWS_ACCESS_KEY_ID'), true) }}"
+    access_key: >-
-    secret_key: "{{ aws_secret_key | default(_aws_secret_key.user_input|default(None)) | default(lookup('env','AWS_SECRET_ACCESS_KEY'), true) }}"
+      {{ aws_access_key
+         | default(lookup('env', 'AWS_ACCESS_KEY_ID'))
+         | default(_file_access_key)
+         | default(_aws_access_key.user_input | default(None)) }}
+    secret_key: >-
+      {{ aws_secret_key
+         | default(lookup('env', 'AWS_SECRET_ACCESS_KEY'))
+         | default(_file_secret_key)
+         | default(_aws_secret_key.user_input | default(None)) }}
+    session_token: >-
+      {{ aws_session_token
+         | default(lookup('env', 'AWS_SESSION_TOKEN'))
+         | default(_file_session_token)
+         | default('') }}
+  no_log: true
 
 - block:
     - name: Get regions
       aws_region_info:
         aws_access_key: "{{ access_key }}"
         aws_secret_key: "{{ secret_key }}"
+        aws_session_token: "{{ session_token if session_token else omit }}"
         region: us-east-1
       register: _aws_regions
 
@@ -67,6 +111,7 @@
       ec2_eip_info:
         aws_access_key: "{{ access_key }}"
         aws_secret_key: "{{ secret_key }}"
+        aws_session_token: "{{ session_token if session_token else omit }}"
         region: "{{ algo_region }}"
       register: raw_eip_addresses
 

tests/test-aws-credentials.yml

@@ -0,0 +1,107 @@
+---
+# Test AWS credential reading from files
+# Run with: ansible-playbook tests/test-aws-credentials.yml
+
+- name: Test AWS credential file reading
+  hosts: localhost
+  gather_facts: no
+  vars:
+    # These would normally come from config.cfg
+    cloud_providers:
+      ec2:
+        use_existing_eip: false
+    
+  tasks:
+    - name: Test with environment variables
+      block:
+        - include_tasks: ../roles/cloud-ec2/tasks/prompts.yml
+          vars:
+            algo_server_name: test-server
+        
+        - assert:
+            that:
+              - access_key == "test_env_key"
+              - secret_key == "test_env_secret"
+            msg: "Environment variables should take precedence"
+      vars:
+        AWS_ACCESS_KEY_ID: "test_env_key"
+        AWS_SECRET_ACCESS_KEY: "test_env_secret"
+      environment:
+        AWS_ACCESS_KEY_ID: "test_env_key"
+        AWS_SECRET_ACCESS_KEY: "test_env_secret"
+
+    - name: Test with command line variables
+      block:
+        - include_tasks: ../roles/cloud-ec2/tasks/prompts.yml
+          vars:
+            aws_access_key: "test_cli_key"
+            aws_secret_key: "test_cli_secret"
+            algo_server_name: test-server
+            region: "us-east-1"
+        
+        - assert:
+            that:
+              - access_key == "test_cli_key"
+              - secret_key == "test_cli_secret"
+            msg: "Command line variables should take precedence over everything"
+
+    - name: Test reading from credentials file
+      block:
+        - name: Create test credentials directory
+          file:
+            path: /tmp/test-aws
+            state: directory
+            mode: '0700'
+        
+        - name: Create test credentials file
+          copy:
+            dest: /tmp/test-aws/credentials
+            mode: '0600'
+            content: |
+              [default]
+              aws_access_key_id = test_file_key
+              aws_secret_access_key = test_file_secret
+              
+              [test-profile]
+              aws_access_key_id = test_profile_key
+              aws_secret_access_key = test_profile_secret
+              aws_session_token = test_session_token
+        
+        - name: Test default profile
+          include_tasks: ../roles/cloud-ec2/tasks/prompts.yml
+          vars:
+            algo_server_name: test-server
+            region: "us-east-1"
+          environment:
+            HOME: /tmp/test-aws
+            AWS_ACCESS_KEY_ID: ""
+            AWS_SECRET_ACCESS_KEY: ""
+        
+        - assert:
+            that:
+              - access_key == "test_file_key"
+              - secret_key == "test_file_secret"
+            msg: "Should read from default profile"
+        
+        - name: Test custom profile
+          include_tasks: ../roles/cloud-ec2/tasks/prompts.yml
+          vars:
+            algo_server_name: test-server
+            region: "us-east-1"
+          environment:
+            HOME: /tmp/test-aws
+            AWS_PROFILE: "test-profile"
+            AWS_ACCESS_KEY_ID: ""
+            AWS_SECRET_ACCESS_KEY: ""
+        
+        - assert:
+            that:
+              - access_key == "test_profile_key"
+              - secret_key == "test_profile_secret"
+              - session_token == "test_session_token"
+            msg: "Should read from custom profile with session token"
+        
+        - name: Cleanup test directory
+          file:
+            path: /tmp/test-aws
+            state: absent