Better documentation (#459)

* Closes #443

* Remove numbers

* context

* split up local and scripted

* Closes #458

* .

* better layout

* Closes #451

* do this later

* grammar

* typo
This commit is contained in:
Dan Guido 2017-04-20 17:56:03 -04:00 committed by GitHub
parent 0b05ea19bc
commit 019d729fe6
9 changed files with 138 additions and 80 deletions

View file

@ -198,10 +198,22 @@ The Algo VPN server now contains only the users listed in the `config.cfg` file.
## Additional Documentation
* [Advanced Usage](docs/advanced-usage.md) describes how to deploy an Algo VPN server directly from Ansible.
* [FAQ](docs/faq.md) includes answers to common questions.
* [Roles](docs/ansible-roles.md) includes a description of optional Algo VPN server features.
* [Troubleshooting](docs/troubleshooting.md) includes answers to common technical issues.
* Setup instructions
- Documentation for avaialble [Ansible roles](setup-roles.md)
- Deploy from [RedHat/CentOS 6.x](setup-redhat-centos6.md)
* Client setup
- Setup [Windows](client-windows.md) clients
- Setup [Android](client-android.md) clients
- Setup [Generic/Linux](client-generic.md) clients with Ansible
* Cloud setup
- Configure [Azure](cloud-azure.md)
- Deploy to an [unsupported cloud provider](cloud-unsupported.md)
* Advanced Deployment
- Deploy to local [FreeBSD](deploy-to-freebsd.md) servers
- Deploy to local [Ubuntu 16.04](deploy-to-ubuntu.md) servers
- Deploy with [Ansible](deploy-with-ansible.md)
* [FAQ](faq.md)
* [Troubleshooting](troubleshooting.md)
## Endorsements

20
docs/cloud-unsupported.md Normal file
View file

@ -0,0 +1,20 @@
# Unsupported Cloud Providers
Algo officially supports DigitalOcean, Amazon Web Services, Microsoft Azure, and Google Cloud Engine. If you want to deploy Algo on another virtual hosting provider, that provider must support:
1. the base operating system image that Algo uses (Ubuntu 16.04), and
2. a minimum of certain kernel modules required for the strongSwan IPsec server.
Please see the [Required Kernel Modules](https://wiki.strongswan.org/projects/strongswan/wiki/KernelModules) documentation from strongSwan for a list of the specific required modules and a script to check for them. As a first step, we recommend running their shell script to determine initial compatibility with your new hosting provider.
If you want Algo to officially support your new cloud provider then it must have an Ansible [cloud module](https://docs.ansible.com/ansible/list_of_cloud_modules.html) available. If no module is available for your provider, search Ansible's [open issues](https://github.com/ansible/ansible/issues) and [pull requests](https://github.com/ansible/ansible/pulls) for existing efforts to add it. If none are available, then you may want to develop the module yourself. Reference the [Ansible module developer documentation](https://docs.ansible.com/ansible/dev_guide/developing_modules.html) and the API documentation for your hosting provider.
## IPsec in userland
Hosting providers that rely on OpenVZ or Docker cannot be used by Algo since they cannot load the required kernel modules or access the required network interfaces. For more information, see the strongSwan documentation on [Cloud Platforms](https://wiki.strongswan.org/projects/strongswan/wiki/Cloudplatforms).
In order to address this issue, strongSwan has developed the [kernel-libipsec](https://wiki.strongswan.org/projects/strongswan/wiki/Kernel-libipsec) plugin which provides an IPsec backend that works entirely in userland. `libipsec` bundles its own IPsec implementation and uses TUN devices to route packets. For example, `libipsec` is used by the Android strongSwan app to address Adnroid's lack of a functional IPsec stack.
Use of `libipsec` is not supported by Algo. It has known performance issues since it buffers each packet in memory. On certain systems with insufficient processor power, such as many cloud hosting providers, using `libipsec` can lead to an out of memory conditions, crash the charon daemon, or lock up the entire host.
Further, `libipsec` introduces unknown security risks. The code in `libipsec` has not been scrutinized to the same level as the code in the Linux or FreeBSD kernel that it replaces. This additional code introduces new complexity to the Algo server that we want to avoid at this time. We recommend moving to a hosting provider that does not require libipsec.

View file

@ -1,6 +1,6 @@
# FreeBSD / HardenedBSD server setup
It is only possible to install Algo on existing systems only. We support only 11 version for now.
FreeBSD server support is a work in progress. For now, it is only possible to install Algo on existing FreeBSD 11 systems.
## System preparation

14
docs/deploy-to-ubuntu.md Normal file
View file

@ -0,0 +1,14 @@
# Local deployment
It is possible to download the Algo scripts to your own Ubuntu server and run the scripts locally. You need to install Ansible to run Algo on Ubuntu. Installing ansible via pip requires pulling in a lot of dependencies, including a full compiler suite. It would be easier to use apt, however, Ubuntu 16.04 only comes with Ansible 2.0.0.2. Therefore, to use apt you must use the ansible PPA, and using a PPA requires installing `software-properties-common`.
tl;dr:
```shell
sudo apt-get install software-properties-common && sudo apt-add-repository ppa:ansible/ansible
sudo apt-get update && sudo apt-get install ansible
git clone https://github.com/trailofbits/algo
cd algo && ./algo
```
**Warning**: If you run Algo on your existing server, the iptables rules will be overwritten. If you don't want to overwrite the rules, you must deploy via `ansible-playbook` and skip the `iptables` tag as described below.

View file

@ -1,23 +1,6 @@
# Advanced Usage
# Scripted Deployment
Make sure you have installed all the dependencies necessary for your operating system as described in the [README](../README.md).
## Local deployment
It is possible to download the Algo scripts to your own Ubuntu server and run the scripts locally. You need to install Ansible to run Algo on Ubuntu. Installing ansible via pip requires pulling in a lot of dependencies, including a full compiler suite. It would be easier to use apt, however, Ubuntu 16.04 only comes with Ansible 2.0.0.2. Therefore, to use apt you must use the ansible PPA, and using a PPA requires installing `software-properties-common`.
tl;dr:
```shell
sudo apt-get install software-properties-common && sudo apt-add-repository ppa:ansible/ansible
sudo apt-get update && sudo apt-get install ansible
git clone https://github.com/trailofbits/algo
cd algo && ./algo
```
**Warning**: If you run Algo on your existing server, the iptables rules will be overwritten. If you don't want to overwrite the rules, you must deploy via `ansible-playbook` and skip the `iptables` tag as described below.
## Scripted deployment
Before you begin, make sure you have installed all the dependencies necessary for your operating system as described in the [README](../README.md).
You can deploy Algo non-interactively by running the Ansible playbooks directly with `ansible-playbook`.
@ -47,7 +30,6 @@ Server roles:
- role: vpn, tags: vpn
- role: dns_adblocking, tags: dns, adblock
- role: proxy, tags: proxy, adblock
- role: security, tags: security
- role: ssh_tunneling, tags: ssh_tunneling

View file

@ -1,15 +1,19 @@
# Algo VPN documentation
* [Advanced usage](advanced-usage.md)
* [Ansible roles](ansible-roles.md)
* Setup instructions
- Documentation for avaialble [Ansible roles](setup-roles.md)
- Deploy from [RedHat/CentOS 6.x](setup-redhat-centos6.md)
* Client setup
- [Windows](client-windows.md)
- [Android](client-android.md)
- [Generic/Linux](client-generic.md)
- Setup [Windows](client-windows.md) clients
- Setup [Android](client-android.md) clients
- Setup [Generic/Linux](client-generic.md) clients with Ansible
* Cloud setup
- [Azure](cloud-azure.md)
* Server setup
- [RedHat/CentOS 6.x](server-redhat-centos6.md)
- [FreeBSD](server-freebsd.md)
* [Troubleshooting](troubleshooting.md)
- Configure [Azure](cloud-azure.md)
- Deploy to an [unsupported cloud provider](cloud-unsupported.md)
* Advanced Deployment
- Deploy to local [FreeBSD](deploy-to-freebsd.md) servers
- Deploy to local [Ubuntu 16.04](deploy-to-ubuntu.md) servers
- Deploy with [Ansible](deploy-with-ansible.md)
* [FAQ](faq.md)
* [Troubleshooting](troubleshooting.md)

View file

@ -1,23 +1,10 @@
# Troubleshooting
## Table of Contents
## Installation Problems
1. [Error: "You have not agreed to the Xcode license agreements"](#1-error-you-have-not-agreed-to-the-xcode-license-agreements)
2. [Error: "fatal error: 'openssl/opensslv.h' file not found"](#2-error-fatal-error-opensslopensslvh-file-not-found)
3. [Error: "TypeError: must be str, not bytes"](#3-error-typeerror-must-be-str-not-bytes)
4. [Error: "ansible-playbook: command not found"](#4-error-ansible-playbook-command-not-found)
5. [Bad owner or permissions on .ssh](#5-bad-owner-or-permissions-on-ssh)
6. [Little Snitch is broken when connected to the VPN](#6-little-snitch-is-broken-when-connected-to-the-vpn)
7. [Various websites appear to be offline through the VPN](#7-various-websites-appear-to-be-offline-through-the-vpn)
8. [The region you want is not available](#8-the-region-you-want-is-not-available)
9. [I want to change the list of trusted Wifi networks on my Apple device](#9-i-want-to-change-the-list-of-trusted-wifi-networks-on-my-apple-device)
10. [Error: "The VPN Service payload could not be installed"](#10-error-the-vpn-service-payload-could-not-be-installed)
11. [I can't get my router to connect to the Algo server](#11-i-cant-get-my-router-to-connect-to-the-algo-server)
12. [I can't get Network Manager to connect to the Algo Server](#12-i-cant-get-network-manager-to-connect-to-the-algo-server)
13. [IKEAUTH request never makes it to the server](#13-ikeauth-request-never-makes-it-to-the-server)
14. [I have a problem not covered here](#i-have-a-problem-not-covered-here)
Look here if you have a problem running the installer to set up a new Algo server.
### 1. Error: "You have not agreed to the Xcode license agreements"
### Error: "You have not agreed to the Xcode license agreements"
On macOS, you tried to install the dependencies with pip and encountered the following error:
@ -41,7 +28,33 @@ Storing debug log for failure in /Users/algore/Library/Logs/pip.log
The Xcode compiler is installed but requires you to accept its license agreement prior to using it. Run `xcodebuild -license` to agree and then retry installing the dependencies.
### 2. Error: "fatal error: 'openssl/opensslv.h' file not found"
### Error: checking whether the C compiler works... no
On macOS, you tried to install the dependencies with pip and encountered the following error:
```
Failed building wheel for pycrypto
Running setup.py clean for pycrypto
Failed to build pycrypto
...
copying lib/Crypto/Signature/PKCS1_v1_5.py -> build/lib.macosx-10.6-intel-2.7/Crypto/Signature
running build_ext
running build_configure
checking for gcc... gcc
checking whether the C compiler works... no
configure: error: in '/private/var/folders/3f/q33hl6_x6_nfyjg29fcl9qdr0000gp/T/pip-build-DB5VZp/pycrypto': configure: error: C compiler cannot create executables See config.log for more details
Traceback (most recent call last):
File "", line 1, in
...
cmd_obj.run()
File "/private/var/folders/3f/q33hl6_x6_nfyjg29fcl9qdr0000gp/T/pip-build-DB5VZp/pycrypto/setup.py", line 278, in run
raise RuntimeError("autoconf error")
RuntimeError: autoconf error
```
You don't have a working compiler installed. You should install the XCode compiler by opening your terminal and running `xcode-select --install`.
### Error: "fatal error: 'openssl/opensslv.h' file not found"
On macOS, you tried to install pycrypto and encountered the following error:
@ -62,9 +75,9 @@ Command /usr/bin/python -c "import setuptools, tokenize;__file__='/private/tmp/p
Storing debug log for failure in /Users/algore/Library/Logs/pip.log
```
You are running an old version of `pip` that cannot build the `pycrypto` dependency. Upgrade to a new version of `pip` by running `sudo pip install -U pip`.
You are running an old version of `pip` that cannot build the `pycrypto` dependency. Upgrade to a new version of `pip` by running `sudo pip install -U pip`.
### 3. Error: "TypeError: must be str, not bytes"
### Error: "TypeError: must be str, not bytes"
You tried to install Algo and you see many repeated errors referencing `TypeError`, such as `TypeError: '>=' not supported between instances of 'TypeError' and 'int'` and `TypeError: must be str, not bytes`. For example:
@ -76,13 +89,13 @@ fatal: [localhost -> localhost]: FAILED! => {"changed": false, "failed": true, "
You may be trying to run Algo with Python3. Algo uses [Ansible](https://github.com/ansible/ansible) which has issues with Python3, although this situation is improving over time. Try running Algo with Python2 to fix this issue. Open your terminal and `cd` to the directory with Algo, then run: ``virtualenv -p `which python2.7` env && source env/bin/activate && pip install -r requirements.txt``
### 4. Error: "ansible-playbook: command not found"
### Error: "ansible-playbook: command not found"
You tried to install Algo and you see an error that reads "ansible-playbook: command not found."
You did not finish step 4 in the installation instructions, "[Install Algo's remaining dependencies](https://github.com/trailofbits/algo#deploy-the-algo-server)." Algo depends on [Ansible](https://github.com/ansible/ansible), an automation framework, and this error indicates that you do not have Ansible installed. Ansible is installed by `pip` when you run `python -m pip install -r requirements.txt`. You must complete the installation instructions to run the Algo server deployment process.
### 5. Bad owner or permissions on .ssh
### Bad owner or permissions on .ssh
You tried to run Algo and it quickly exits with an error about a bad owner or permissions:
@ -92,11 +105,44 @@ fatal: [104.236.2.94]: UNREACHABLE! => {"changed": false, "msg": "Failed to conn
You need to reset the permissions on your `.ssh` directory. Run `chmod 700 /home/user/.ssh` and then `chmod 600 /home/user/.ssh/config`. You may need to repeat this for other files mentioned in the error message.
### 6. Little Snitch is broken when connected to the VPN
### The region you want is not available
You want to install Algo to a specific region in a cloud provider, but that region is not available in the list given by the installer. In that case, you should [file an issue](https://github.com/trailofbits/algo/issues/new). Cloud providers add new regions on a regular basis and we don't always keep up. File an issue and give us information about what region is missing and we'll add it.
## Connection Problems
Look here if you deployed an Algo server but now have a problem connecting to it with a client.
### I'm blocked or get CAPTCHAs when I access certain websites
This is normal.
When you deploy a Algo to a new cloud server, the address you are given may have been used before. In some cases, a malicious individual may have attacked others with that address and had it added to "IP reputation" feeds or simply a blacklist. In order to regain the trust for that address, you may be asked to enter CAPTCHAs to prove that you are a human, and not a Denial of Service (DoS) bot trying to attack others. This happens most frequently with Google. You can try entering the CAPTCHAs or you can try redeploying your Algo server to a new IP to resolve this issue.
In some cases, a website will block any visitors accessing their site through a cloud hosting provider due to previous, frequent DoS attacks originating from them. In these cases, there is not much you can do except deploy Algo to your own server or another IP that the website has not outright blocked.
### I want to change the list of trusted Wifi networks on my Apple device
This setting is enforced on your client device via the Apple profile you put on it. You can edit the profile with new settings, then load it on your device to change the settings. You can use the [Apple Configurator](https://itunes.apple.com/us/app/apple-configurator-2/id1037126344?mt=12) to edit and resave the profile. Advanced users can edit the file directly in a text editor. Use the [Configuration Profile Reference](https://developer.apple.com/library/content/featuredarticles/iPhoneConfigurationProfileRef/Introduction/Introduction.html) for information about the file format and other available options. If you're not comfortable editing the profile, you can also simply redeploy a new Algo server with different settings to receive a new auto-generated profile.
### Error: "The VPN Service payload could not be installed."
You tried to install the Apple profile on one of your devices and you received an error stating `The "VPN Service" payload could not be installed. The VPN service could not be created.` Client support for Algo VPN is limited to modern operating systems, e.g. macOS 10.11+, iOS 9+. Please upgrade your operating system and try again.
### Little Snitch is broken when connected to the VPN
Little Snitch is not compatible with IPSEC VPNs due to a known bug in macOS and there is no solution. The Little Snitch "filter" does not get incoming packets from IPSEC VPNs and, therefore, cannot evaluate any rules over them. Their developers have filed a bug report with Apple but there has been no response. There is nothing they or Algo can do to resolve this problem on their own. You can read more about this problem in [issue #134](https://github.com/trailofbits/algo/issues/134).
### 7. Various websites appear to be offline through the VPN
### I can't get my router to connect to the Algo server
In order to connect to the Algo VPN server, your router must support IKEv2, ECC certificate-based authentication, and the cipher suite we use. See the ipsec.conf files we generate in the `config` folder for more information. Note that we do not officially support routers as clients for Algo VPN at this time, though patches and documentation for them are welcome (for example, see open issues for [Ubiquiti](https://github.com/trailofbits/algo/issues/307) and [pfSense](https://github.com/trailofbits/algo/issues/292)).
### I can't get Network Manager to connect to the Algo server
You're trying to connect Ubuntu or Debian to the Algo server through the Network Manager GUI but it's not working. Many versions of Ubuntu and some older versions of Debian bundle a [broken version of Network Manager](https://github.com/trailofbits/algo/issues/263) without support for modern standards or the strongSwan server. You must upgrade to Ubuntu 17.04 or Debian 9 Stretch, each of which contain the required minimum version of Network Manager.
### Various websites appear to be offline through the VPN
This issue appears intermittently due to issues with MTU size. If you experience this issue, we recommend [filing an issue](https://github.com/trailofbits/algo/issues/new) for assistance. Advanced users can troubleshoot the correct MTU size by retrying `ping` with the "don't fragment" bit set, then decreasing packet size until it works. This will determine the correct MTU size for your network, which you then need to update on your network adapter.
@ -111,32 +157,12 @@ Then, set the MTU size on your network adapter (wlan0 or eth0):
$ sudo ifconfig wlan0 mtu 1438
```
### 8. The region you want is not available
You want to install Algo to a specific region in a cloud provider, but that region is not available in the list given by the installer. In that case, you should [file an issue](https://github.com/trailofbits/algo/issues/new). Cloud providers add new regions on a regular basis and we don't always keep up. File an issue and give us information about what region is missing and we'll add it.
### 9. I want to change the list of trusted Wifi networks on my Apple device
This setting is enforced on your client device via the Apple profile you put on it. You can edit the profile with new settings, then load it on your device to change the settings. You can use the [Apple Configurator](https://itunes.apple.com/us/app/apple-configurator-2/id1037126344?mt=12) to edit and resave the profile. Advanced users can edit the file directly in a text editor. Use the [Configuration Profile Reference](https://developer.apple.com/library/content/featuredarticles/iPhoneConfigurationProfileRef/Introduction/Introduction.html) for information about the file format and other available options. If you're not comfortable editing the profile, you can also simply redeploy a new Algo server with different settings to receive a new auto-generated profile.
### 10. Error: "The VPN Service payload could not be installed."
You tried to install the Apple profile on one of your devices and you received an error stating `The "VPN Service" payload could not be installed. The VPN service could not be created.` Client support for Algo VPN is limited to modern operating systems, e.g. macOS 10.11+, iOS 9+. Please upgrade your operating system and try again.
### 11. I can't get my router to connect to the Algo server
In order to connect to the Algo VPN server, your router must support IKEv2, ECC certificate-based authentication, and the cipher suite we use. See the ipsec.conf files we generate in the `config` folder for more information. Note that we do not officially support routers as clients for Algo VPN at this time, though patches and documentation for them are welcome (for example, see open issues for [Ubiquiti](https://github.com/trailofbits/algo/issues/307) and [pfSense](https://github.com/trailofbits/algo/issues/292)).
### 12. I can't get Network Manager to connect to the Algo server
You're trying to connect Ubuntu or Debian to the Algo server through the Network Manager GUI but it's not working. Many versions of Ubuntu and some older versions of Debian bundle a [broken version of Network Manager](https://github.com/trailofbits/algo/issues/263) without support for modern standards or the strongSwan server. You must upgrade to Ubuntu 17.04 or Debian 9 Stretch, each of which contain the required minimum version of Network Manager.
### 13. "Error 809" or IKE_AUTH requests that never make it to the server
### "Error 809" or IKE_AUTH requests that never make it to the server
On Windows, this issue may manifest with an error message that says "The network connection between your computer and the VPN server could not be established because the remote server is not responding... This is Error 809." On other operating systems, you may try to debug the issue by capturing packets with tcpdump and notice that, while IKE_SA_INIT request and responses are exchanged between the client and server, IKE_AUTH requests never make it to the server.
It is possible that the IKE_AUTH payload is too big to fit in a single IP datagram, and so is fragmented. Many consumer routers and cable modems ship with 'Block Fragmented IP packets'. Many consumer routers and cable modems ship with a feature that blocks "fragmented IP packets." Try logging into your router and disabling any firewall settings related to blocking or dropping fragmented IP packets. For more information, see [Issue #305](https://github.com/trailofbits/algo/issues/305).
### I have a problem not covered here
## I have a problem not covered here
If you have an issue that you cannot solve with the guidance here, [file an issue](https://github.com/trailofbits/algo/issues/new) that describes the problem and we'll do our best to help you. You can also [join our Slack](https://empireslacking.herokuapp.com/) and ask for help in the **#algo-support** channel.