* Bump version in most places.
* Update official release steps for desktop
* Version bump a bunch of Rust packages.
* Windows fix for disabling MT properly.
* Release notes.
In nixpkgs we use coreutils and do not have PWD available.
This change have any downsides on normal macOS systems, but helps
nixpkgs packaging a bit.
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
Update the minimal compilers in documentation. As `std=c++17` is used the minimal got quite bumped from last update. For gcc 7.x would seem possibly enough, but some of the dependencies (eg. libpqxx) need 8.x at least, so setting that as documented minimum.
cpp-httplib sets IPV6_V6ONLY to false on it's sockets.
On FreeBSD, this makes all ipv4 addresses get get prefixed with ::ffff:
it makes them IPv6 addresses mapped to v4.
This is a partial fix for #2151. The cli will work again.
Something should probably also be adjusted with the httplib.
If you want to, for example, use the `allowManagementFrom` option in
local.conf
you will need to prefix it with "::ffff:", "::ffff:1.2.3.4"
which is a little surprising and inconsistent between BSD and other OSs.
Want to show it in UIs. So need to get it via API.
It's also a pain to look up. You have to go to external docs.
zerotier-cli info -j
```json
"config": {
"settings": {
"allowTcpFallbackRelay": true,
"forceTcpRelay": true,
"homeDir": "/Library/Application Support/ZeroTier/One",
```
```sh
curl -s -X POST "http://localhost:9993/controller/network/abcdabcdabcdabcd/member/1122334455"
```
Would return 200 and ZT_HOME/controller.d/abcdabcdabcdabcd/members/1122334455
would be created. Without a ZT_HOME/controller.d/abcdabcdabcdabcd.json
Then other parts of the system mistakenly think a abcdabcdabcdabcd
network sorta kinda exists and then fail in weird ways.
There was as Windows directory traversal bug in the static
file server feature. We don't use that feature, so we are ok.
We are going to use that feature soon, so we are
taking the opportunity to update.
Add /unstable/controller/network endpoint
Similar to /unstable/controller/network/{id}/member, it returns actual network objects,
instead of just network ids.
Also includes the total network count,
and each network has it's member counts in meta{}.
The current api at /controller/network/1111111111767f2f/member
Lists only the members' ID and revision number.
If you want details, you have to query each specific member.
So if you want to make a members list, and you have
10000 members on a network, you need to make
10000 http requests.
It's also in a hard to specify and use shape
{ [member-id-1]: 13, [member-id-2]: 14, ... }
GET http://localhost:9993/unstable/controller/network/1111111111767f2f/member ->
```
{
data: [ {...member1}, {...member2}, ...],
meta: { totalCount: 4, authorizedCount: 3 }
}
```
It wasn't ignoring separator characters such as the colon and hyphen.
The rules compiler automatically add a colon to separate bytes, which is
not compatible with how they are parsed.
* Better compatibility with LLVM toolchain where clang -c doesn't
support the flag, but the linker does. LLD already defaults to
noexecstack, but adding it in the linker phase will avoid errors about
unsupported options.
Signed-off-by: Alfred Wingate <parona@protonmail.com>
This makes switching between physical networks
with full tunnel mode enabled more reliable.
There were issues with the physical default route or device
changing.
* 1.10.6 merge to main (#1930)
* add note about forceTcpRelay
* Create a sample systemd unit for tcp proxy
* set gitattributes for rust & cargo so hashes dont conflict on Windows
* Revert "set gitattributes for rust & cargo so hashes dont conflict on Windows"
This reverts commit 032dc5c108.
* Turn off autocrlf for rust source
Doesn't appear to play nice well when it comes to git and vendored cargo package hashes
* Fix#1883 (#1886)
Still unknown as to why, but the call to `nc->GetProperties()` can fail
when setting a friendly name on the Windows virtual ethernet adapter.
Ensure that `ncp` is not null before continuing and accessing the device
GUID.
* Don't vendor packages for zeroidc (#1885)
* Added docker environment way to join networks (#1871)
* add StringUtils
* fix headers
use recommended headers and remove unused headers
* move extern "C"
only JNI functions need to be exported
* cleanup
* fix ANDROID-50: RESULT_ERROR_BAD_PARAMETER typo
* fix typo in log message
* fix typos in JNI method signatures
* fix typo
* fix ANDROID-51: fieldName is uninitialized
* fix ANDROID-35: memory leak
* fix missing DeleteLocalRef in loops
* update to use unique error codes
* add GETENV macro
* add LOG_TAG defines
* ANDROID-48: add ZT_jnicache.cpp
* ANDROID-48: use ZT_jnicache.cpp and remove ZT_jnilookup.cpp and ZT_jniarray.cpp
* add Event.fromInt
* add PeerRole.fromInt
* add ResultCode.fromInt
* fix ANDROID-36: issues with ResultCode
* add VirtualNetworkConfigOperation.fromInt
* fix ANDROID-40: VirtualNetworkConfigOperation out-of-sync with ZT_VirtualNetworkConfigOperation enum
* add VirtualNetworkStatus.fromInt
* fix ANDROID-37: VirtualNetworkStatus out-of-sync with ZT_VirtualNetworkStatus enum
* add VirtualNetworkType.fromInt
* make NodeStatus a plain data class
* fix ANDROID-52: synchronization bug with nodeMap
* Node init work: separate Node construction and init
* add Node.toString
* make PeerPhysicalPath a plain data class
* remove unused PeerPhysicalPath.fixed
* add array functions
* make Peer a plain data class
* make Version a plain data class
* fix ANDROID-42: copy/paste error
* fix ANDROID-49: VirtualNetworkConfig.equals is wrong
* reimplement VirtualNetworkConfig.equals
* reimplement VirtualNetworkConfig.compareTo
* add VirtualNetworkConfig.hashCode
* make VirtualNetworkConfig a plain data class
* remove unused VirtualNetworkConfig.enabled
* reimplement VirtualNetworkDNS.equals
* add VirtualNetworkDNS.hashCode
* make VirtualNetworkDNS a plain data class
* reimplement VirtualNetworkRoute.equals
* reimplement VirtualNetworkRoute.compareTo
* reimplement VirtualNetworkRoute.toString
* add VirtualNetworkRoute.hashCode
* make VirtualNetworkRoute a plain data class
* add isSocketAddressEmpty
* add addressPort
* add fromSocketAddressObject
* invert logic in a couple of places and return early
* newInetAddress and newInetSocketAddress work
allow newInetSocketAddress to return NULL if given empty address
* fix ANDROID-38: stack corruption in onSendPacketRequested
* use GETENV macro
* JniRef work
JniRef does not use callbacks struct, so remove
fix NewGlobalRef / DeleteGlobalRef mismatch
* use PRId64 macros
* switch statement work
* comments and logging
* Modifier 'public' is redundant for interface members
* NodeException can be made a checked Exception
* 'NodeException' does not define a 'serialVersionUID' field
* 'finalize()' should not be overridden
this is fine to do because ZeroTierOneService calls close() when it is done
* error handling, error reporting, asserts, logging
* simplify loadLibrary
* rename Node.networks -> Node.networkConfigs
* Windows file permissions fix (#1887)
* Allow macOS interfaces to use multiple IP addresses (#1879)
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Fix condition where full HELLOs might not be sent when necessary (#1877)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* 1.10.4 version bumps
* Add security policy to repo (#1889)
* [+] add e2k64 arch (#1890)
* temp fix for ANDROID-56: crash inside newNetworkConfig from too many args
* 1.10.4 release notes
* Windows 1.10.4 Advanced Installer bump
* Revert "temp fix for ANDROID-56: crash inside newNetworkConfig from too many args"
This reverts commit dd627cd7f4.
* actual fix for ANDROID-56: crash inside newNetworkConfig
cast all arguments to varargs functions as good style
* Fix addIp being called with applied ips (#1897)
This was getting called outside of the check for existing ips
Because of the added ifdef and a brace getting moved to the
wrong place.
```
if (! n.tap()->addIp(*ip)) {
fprintf(stderr, "ERROR: unable to add ip address %s" ZT_EOL_S, ip->toString(ipbuf));
}
WinFWHelper::newICMPRule(*ip, n.config().nwid);
```
* 1.10.5 (#1905)
* 1.10.5 bump
* 1.10.5 for Windows
* 1.10.5
* Prevent path-learning loops (#1914)
* Prevent path-learning loops
* Only allow new overwrite if not bonded
* fix binding temporary ipv6 addresses on macos (#1910)
The check code wasn't running.
I don't know why !defined(TARGET_OS_IOS) would exclude code on
desktop macOS. I did a quick search and changed it to defined(TARGET_OS_MAC).
Not 100% sure what the most correct solution there is.
You can verify the old and new versions with
`ifconfig | grep temporary`
plus
`zerotier-cli info -j` -> listeningOn
* 1.10.6 (#1929)
* 1.10.5 bump
* 1.10.6
* 1.10.6 AIP for Windows.
---------
Co-authored-by: travis laduke <travisladuke@gmail.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Joseph Henry <joseph-henry@users.noreply.github.com>
Co-authored-by: Roman Peshkichev <roman.peshkichev@gmail.com>
* 1.12.0 merge to main (#2104)
* add note about forceTcpRelay
* Create a sample systemd unit for tcp proxy
* set gitattributes for rust & cargo so hashes dont conflict on Windows
* Revert "set gitattributes for rust & cargo so hashes dont conflict on Windows"
This reverts commit 032dc5c108.
* Turn off autocrlf for rust source
Doesn't appear to play nice well when it comes to git and vendored cargo package hashes
* Fix#1883 (#1886)
Still unknown as to why, but the call to `nc->GetProperties()` can fail
when setting a friendly name on the Windows virtual ethernet adapter.
Ensure that `ncp` is not null before continuing and accessing the device
GUID.
* Don't vendor packages for zeroidc (#1885)
* Added docker environment way to join networks (#1871)
* add StringUtils
* fix headers
use recommended headers and remove unused headers
* move extern "C"
only JNI functions need to be exported
* cleanup
* fix ANDROID-50: RESULT_ERROR_BAD_PARAMETER typo
* fix typo in log message
* fix typos in JNI method signatures
* fix typo
* fix ANDROID-51: fieldName is uninitialized
* fix ANDROID-35: memory leak
* fix missing DeleteLocalRef in loops
* update to use unique error codes
* add GETENV macro
* add LOG_TAG defines
* ANDROID-48: add ZT_jnicache.cpp
* ANDROID-48: use ZT_jnicache.cpp and remove ZT_jnilookup.cpp and ZT_jniarray.cpp
* add Event.fromInt
* add PeerRole.fromInt
* add ResultCode.fromInt
* fix ANDROID-36: issues with ResultCode
* add VirtualNetworkConfigOperation.fromInt
* fix ANDROID-40: VirtualNetworkConfigOperation out-of-sync with ZT_VirtualNetworkConfigOperation enum
* add VirtualNetworkStatus.fromInt
* fix ANDROID-37: VirtualNetworkStatus out-of-sync with ZT_VirtualNetworkStatus enum
* add VirtualNetworkType.fromInt
* make NodeStatus a plain data class
* fix ANDROID-52: synchronization bug with nodeMap
* Node init work: separate Node construction and init
* add Node.toString
* make PeerPhysicalPath a plain data class
* remove unused PeerPhysicalPath.fixed
* add array functions
* make Peer a plain data class
* make Version a plain data class
* fix ANDROID-42: copy/paste error
* fix ANDROID-49: VirtualNetworkConfig.equals is wrong
* reimplement VirtualNetworkConfig.equals
* reimplement VirtualNetworkConfig.compareTo
* add VirtualNetworkConfig.hashCode
* make VirtualNetworkConfig a plain data class
* remove unused VirtualNetworkConfig.enabled
* reimplement VirtualNetworkDNS.equals
* add VirtualNetworkDNS.hashCode
* make VirtualNetworkDNS a plain data class
* reimplement VirtualNetworkRoute.equals
* reimplement VirtualNetworkRoute.compareTo
* reimplement VirtualNetworkRoute.toString
* add VirtualNetworkRoute.hashCode
* make VirtualNetworkRoute a plain data class
* add isSocketAddressEmpty
* add addressPort
* add fromSocketAddressObject
* invert logic in a couple of places and return early
* newInetAddress and newInetSocketAddress work
allow newInetSocketAddress to return NULL if given empty address
* fix ANDROID-38: stack corruption in onSendPacketRequested
* use GETENV macro
* JniRef work
JniRef does not use callbacks struct, so remove
fix NewGlobalRef / DeleteGlobalRef mismatch
* use PRId64 macros
* switch statement work
* comments and logging
* Modifier 'public' is redundant for interface members
* NodeException can be made a checked Exception
* 'NodeException' does not define a 'serialVersionUID' field
* 'finalize()' should not be overridden
this is fine to do because ZeroTierOneService calls close() when it is done
* error handling, error reporting, asserts, logging
* simplify loadLibrary
* rename Node.networks -> Node.networkConfigs
* Windows file permissions fix (#1887)
* Allow macOS interfaces to use multiple IP addresses (#1879)
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Fix condition where full HELLOs might not be sent when necessary (#1877)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* 1.10.4 version bumps
* Add security policy to repo (#1889)
* [+] add e2k64 arch (#1890)
* temp fix for ANDROID-56: crash inside newNetworkConfig from too many args
* 1.10.4 release notes
* Windows 1.10.4 Advanced Installer bump
* Revert "temp fix for ANDROID-56: crash inside newNetworkConfig from too many args"
This reverts commit dd627cd7f4.
* actual fix for ANDROID-56: crash inside newNetworkConfig
cast all arguments to varargs functions as good style
* Fix addIp being called with applied ips (#1897)
This was getting called outside of the check for existing ips
Because of the added ifdef and a brace getting moved to the
wrong place.
```
if (! n.tap()->addIp(*ip)) {
fprintf(stderr, "ERROR: unable to add ip address %s" ZT_EOL_S, ip->toString(ipbuf));
}
WinFWHelper::newICMPRule(*ip, n.config().nwid);
```
* 1.10.5 (#1905)
* 1.10.5 bump
* 1.10.5 for Windows
* 1.10.5
* Prevent path-learning loops (#1914)
* Prevent path-learning loops
* Only allow new overwrite if not bonded
* fix binding temporary ipv6 addresses on macos (#1910)
The check code wasn't running.
I don't know why !defined(TARGET_OS_IOS) would exclude code on
desktop macOS. I did a quick search and changed it to defined(TARGET_OS_MAC).
Not 100% sure what the most correct solution there is.
You can verify the old and new versions with
`ifconfig | grep temporary`
plus
`zerotier-cli info -j` -> listeningOn
* 1.10.6 (#1929)
* 1.10.5 bump
* 1.10.6
* 1.10.6 AIP for Windows.
* Release notes for 1.10.6 (#1931)
* Minor tweak to Synology Docker image script (#1936)
* Change if_def again so ios can build (#1937)
All apple's variables are "defined"
but sometimes they are defined as "0"
* move begin/commit into try/catch block (#1932)
Thread was exiting in some cases
* Bump openssl from 0.10.45 to 0.10.48 in /zeroidc (#1938)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.45 to 0.10.48.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.45...openssl-v0.10.48)
---
updated-dependencies:
- dependency-name: openssl
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* new drone bits
* Fix multiple network join from environment entrypoint.sh.release (#1961)
* _bond_m guards _bond, not _paths_m (#1965)
* Fix: warning: mutex '_aqm_m' is not held on every path through here [-Wthread-safety-analysis] (#1964)
* Bump h2 from 0.3.16 to 0.3.17 in /zeroidc (#1963)
Bumps [h2](https://github.com/hyperium/h2) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/h2/compare/v0.3.16...v0.3.17)
---
updated-dependencies:
- dependency-name: h2
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Add note that binutils is required on FreeBSD (#1968)
* Add prometheus metrics for Central controllers (#1969)
* add header-only prometheus lib to ext
* rename folder
* Undo rename directory
* prometheus simpleapi included on mac & linux
* wip
* wire up some controller stats
* Get windows building with prometheus
* bsd build flags for prometheus
* Fix multiple network join from environment entrypoint.sh.release (#1961)
* _bond_m guards _bond, not _paths_m (#1965)
* Fix: warning: mutex '_aqm_m' is not held on every path through here [-Wthread-safety-analysis] (#1964)
* Serve prom metrics from /metrics endpoint
* Add prom metrics for Central controller specific things
* reorganize metric initialization
* testing out a labled gauge on Networks
* increment error counter on throw
* Consolidate metrics definitions
Put all metric definitions into node/Metrics.hpp. Accessed as needed
from there.
* Revert "testing out a labled gauge on Networks"
This reverts commit 499ed6d95e.
* still blows up but adding to the record for completeness right now
* Fix runtime issues with metrics
* Add metrics files to visual studio project
* Missed an "extern"
* add copyright headers to new files
* Add metrics for sent/received bytes (total)
* put /metrics endpoint behind auth
* sendto returns int on Win32
---------
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
* Central startup update (#1973)
* allow specifying authtoken in central startup
* set allowManagedFrom
* move redis_mem_notification to the correct place
* add node checkins metric
* wire up min/max connection pool size metrics
* x86_64-unknown-linux-gnu on ubuntu runner (#1975)
* adding incoming zt packet type metrics (#1976)
* use cpp-httplib for HTTP control plane (#1979)
refactored the old control plane code to use [cpp-httplib](https://github.com/yhirose/cpp-httplib) instead of a hand rolled HTTP server. Makes the control plane code much more legible. Also no longer randomly stops responding.
* Outgoing Packet Metrics (#1980)
add tx/rx labels to packet counters and add metrics for outgoing packets
* Add short-term validation test workflow (#1974)
Add short-term validation test workflow
* Brenton/curly braces (#1971)
* fix formatting
* properly adjust various lines
breakup multiple statements onto multiple lines
* insert {} around if, for, etc.
* Fix rust dependency caching (#1983)
* fun with rust caching
* kick
* comment out invalid yaml keys for now
* Caching should now work
* re-add/rename key directives
* bump
* bump
* bump
* Don't force rebuild on Windows build GH Action (#1985)
Switching `/t:ZeroTierOne:Rebuild` to just `/t:ZeroTierOne` allows the Windows build to use the rust cache. `/t:ZeroTierOne:Rebuild` cleared the cache before building.
* More packet metrics (#1982)
* found path negotation sends that weren't accounted for
* Fix histogram so it will actually compile
* Found more places for packet metrics
* separate the bind & listen calls on the http backplane (#1988)
* fix memory leak (#1992)
* fix a couple of metrics (#1989)
* More aggressive CLI spamming (#1993)
* fix type signatures (#1991)
* Network-metrics (#1994)
* Add a couple quick functions for converting a uint64_t network ID/node ID into std::string
* Network metrics
* Peer metrics (#1995)
* Adding peer metrics
still need to be wired up for use
* per peer packet metrics
* Fix crash from bad instantiation of histogram
* separate alive & dead path counts
* Add peer metric update block
* add peer latency values in doPingAndKeepalive
* prevent deadlock
* peer latency histogram actually works now
* cleanup
* capture counts of packets to specific peers
---------
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
* Metrics consolidation (#1997)
* Rename zt_packet_incoming -> zt_packet
Also consolidate zt_peer_packets into a single metric with tx and rx labels. Same for ztc_tcp_data and ztc_udp_data
* Further collapse tcp & udp into metric labels for zt_data
* Fix zt_data metric description
* zt_peer_packets description fix
* Consolidate incoming/outgoing network packets to a single metric
* zt_incoming_packet_error -> zt_packet_error
* Disable peer metrics for central controllers
Can change in the future if needed, but given the traffic our controllers serve, that's going to be a *lot* of data
* Disable peer metrics for controllers pt 2
* Update readme files for metrics (#2000)
* Controller Metrics & Network Config Request Fix (#2003)
* add new metrics for network config request queue size and sso expirations
* move sso expiration to its own thread in the controller
* fix potential undefined behavior when modifying a set
* Enable RTTI in Windows build
The new prometheus histogram stuff needs it.
Access violation - no RTTI data!INVALID packet 636ebd9ee8cac6c0 from cafe9efeb9(2605:9880:200:1200:30:571:e34:51/9993) (unexpected exception in tryDecode())
* Don't re-apply routes on BSD
See issue #1986
* Capture setContent by-value instead of by-reference (#2006)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* fix typos (#2010)
* central controller metrics & request path updates (#2012)
* internal db metrics
* use shared mutexes for read/write locks
* remove this lock. only used for a metric
* more metrics
* remove exploratory metrics
place controller request benchmarks behind ifdef
* Improve validation test (#2013)
* fix init order for EmbeddedNetworkController (#2014)
* add constant for getifaddrs cache time
* cache getifaddrs - mac
* cache getifaddrs - linux
* cache getifaddrs - bsd
* cache getifaddrs - windows
* Fix oidc client lookup query
join condition referenced the wrong table. Worked fine unless there were multiple identical client IDs
* Fix udp sent metric
was only incrementing by 1 for each packet sent
* Allow sending all surface addresses to peer in low-bandwidth mode
* allow enabling of low bandwidth mode on controllers
* don't unborrow bad connections
pool will clean them up later
* Multi-arch controller container (#2037)
create arm64 & amd64 images for central controller
* Update README.md
issue #2009
* docker tags change
* fix oidc auth url memory leak (#2031)
getAuthURL() was not calling zeroidc::free_cstr(url);
the only place authAuthURL is called, the url can be retrieved
from the network config instead.
You could alternatively copy the string and call free_cstr in getAuthURL.
If that's better we can change the PR.
Since now there are no callers of getAuthURL I deleted it.
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Bump openssl from 0.10.48 to 0.10.55 in /zeroidc (#2034)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.48 to 0.10.55.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.48...openssl-v0.10.55)
---
updated-dependencies:
- dependency-name: openssl
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* zeroidc cargo warnings (#2029)
* fix unused struct member cargo warning
* fix unused import cargo warning
* fix unused return value cargo warning
---------
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* fix memory leak in macos ipv6/dns helper (#2030)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Consider ZEROTIER_JOIN_NETWORKS in healthcheck (#1978)
* Add a 2nd auth token only for access to /metrics (#2043)
* Add a 2nd auth token for /metrics
Allows administrators to distribute a token that only has access to read
metrics and nothing else.
Also added support for using bearer auth tokens for both types of tokens
Separate endpoint for metrics #2041
* Update readme
* fix a couple of cases of writing the wrong token
* Add warning to cli for allow default on FreeBSD
It doesn't work.
Not possible to fix with deficient network
stack and APIs.
ZeroTierOne-freebsd # zerotier-cli set 9bee8941b5xxxxxx allowDefault=1
400 set Allow Default does not work properly on FreeBSD. See #580
root@freebsd13-a:~/ZeroTierOne-freebsd # zerotier-cli get 9bee8941b5xxxxxx allowDefault
1
* ARM64 Support for TapDriver6 (#1949)
* Release memory previously allocated by UPNP_GetValidIGD
* Fix ifdef that breaks libzt on iOS (#2050)
* less drone (#2060)
* Exit if loading an invalid identity from disk (#2058)
* Exit if loading an invalid identity from disk
Previously, if an invalid identity was loaded from disk, ZeroTier would
generate a new identity & chug along and generate a brand new identity
as if nothing happened. When running in containers, this introduces the
possibility for key matter loss; especially when running in containers
where the identity files are mounted in the container read only. In
this case, ZT will continue chugging along with a brand new identity
with no possibility of recovering the private key.
ZeroTier should exit upon loading of invalid identity.public/identity.secret #2056
* add validation test for #2056
* tcp-proxy: fix build
* Adjust tcp-proxy makefile to support metrics
There's no way to get the metrics yet. Someone will
have to add the http service.
* remove ZT_NO_METRIC ifdef
* Implement recvmmsg() for Linux to reduce syscalls. (#2046)
Between 5% and 40% speed improvement on Linux, depending on system configuration and load.
* suppress warnings: comparison of integers of different signs: 'int64_t' (aka 'long') and 'uint64_t' (aka 'unsigned long') [-Wsign-compare] (#2063)
* fix warning: 'OS_STRING' macro redefined [-Wmacro-redefined] (#2064)
Even though this is in ext, these particular chunks of code were added
by us, so are ok to modify.
* Apply default route a different way - macOS
The original way we applied default route, by forking
0.0.0.0/0 into 0/1 and 128/1 works, but if mac os has any networking
hiccups -if you change SSIDs or sleep/wake- macos erases the system default route.
And then all networking on the computer is broken.
to summarize the new way:
allowDefault=1
```
sudo route delete default 192.168.82.1
sudo route add default 10.2.0.2
sudo route add -ifscope en1 default 192.168.82.1
```
gives us this routing table
```
Destination Gateway RT_IFA Flags Refs Use Mtu Netif Expire rtt(ms) rttvar(ms)
default 10.2.0.2 10.2.0.18 UGScg 90 1 2800 feth4823
default 192.168.82.1 192.168.82.217 UGScIg
```
allowDefault=0
```
sudo route delete default
sudo route delete -ifscope en1 default
sudo route add default 192.168.82.1
```
Notice the I flag, for -ifscope, on the physical default route.
route change does not seem to work reliably.
* fix docker tag for controllers (#2066)
* Update build.sh (#2068)
fix mkwork compilation errors
* Fix network DNS on macOS
It stopped working for ipv4 only networks in Monterey.
See #1696
We add some config like so to System Configuration
```
scutil
show State:/Network/Service/9bee8941b5xxxxxx/IPv4
<dictionary> {
Addresses : <array> {
0 : 10.2.1.36
}
InterfaceName : feth4823
Router : 10.2.1.36
ServerAddress : 127.0.0.1
}
```
* Add search domain to macos dns configuration
Stumbled upon this while debugging something else.
If we add search domain to our system configuration for
network DNS, then search domains work:
```
ping server1 ~
PING server1.my.domain (10.123.3.1): 56 data bytes
64 bytes from 10.123.3.1
```
* Fix reporting of secondaryPort and tertiaryPort See: #2039
* Fix typos (#2075)
* Disable executable stacks on assembly objects (#2071)
Add `--noexecstack` to the assembler flags so the resulting binary
will link with a non-executable stack.
Fixeszerotier/ZeroTierOne#1179
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
* Test that starting zerotier before internet works
* Don't skip hellos when there are no paths available
working on #2082
* Update validate-1m-linux.sh
* Save zt node log files on abort
* Separate test and summary step in validator script
* Don't apply default route until zerotier is "online"
I was running into issues with restarting the zerotier service while
"full tunnel" mode is enabled.
When zerotier first boots, it gets network state from the cache
on disk. So it immediately applies all the routes it knew about
before it shutdown.
The network config may have change in this time.
If it has, then your default route is via a route
you are blocked from talking on. So you can't get the current
network config, so your internet does not work.
Other options include
- don't use cached network state on boot
- find a better criteria than "online"
* Fix node time-to-online counter in validator script
* Export variables so that they are accessible by exit function
* Fix PortMapper issue on ZeroTier startup
See issue #2082
We use a call to libnatpmp::ininatpp to make sure the computer
has working network sockets before we go into the main
nat-pmp/upnp logic.
With basic exponenetial delay up to 30 seconds.
* testing
* Comment out PortMapper debug
this got left turned on in a confusing merge previously
* fix macos default route again
see commit fb6af1971 * Fix network DNS on macOS
adding that stuff to System Config causes this extra route to be added
which breaks ipv4 default route.
We figured out a weird System Coniguration setting
that works.
--- old
couldn't figure out how to fix it in SystemConfiguration
so here we are# Please enter the commit message for your changes. Lines starting
We also moved the dns setter to before the syncIps stuff
to help with a race condition. It didn't always work when
you re-joined a network with default route enabled.
* Catch all conditions in switch statement, remove trailing whitespaces
* Add setmtu command, fix bond lifetime issue
* Basic cleanups
* Check if null is passed to VirtualNetworkConfig.equals and name fixes
* ANDROID-96: Simplify and use return code from node_init directly
* Windows arm64 (#2099)
* ARM64 changes for 1.12
* 1.12 Windows advanced installer updates and updates for ARM64
* 1.12.0
* Linux build fixes for old distros.
* release notes
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: travis laduke <travisladuke@gmail.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Joseph Henry <joseph-henry@users.noreply.github.com>
Co-authored-by: Roman Peshkichev <roman.peshkichev@gmail.com>
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>
Co-authored-by: Jake Vis <jakevis@outlook.com>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: lison <imlison@foxmail.com>
Co-authored-by: Kenny MacDermid <kenny@macdermid.ca>
* Fix primary port binding issue in 1.12 (#2107)
* Add test for primary port bindings to validator - See #2105
* Add delay to binding test
* Remove TCP binding logic from Binder to fix#2105
* add second control plane socket for ipv6
* fix controller network post endpoint
* exit if we can't bind at least one of IPV4 or IPV6 for control plane port
---------
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
* Version bump, Linux version stuff, Debian dependencies from 1.12.0 rebuild, release notes.
* macOS version bump in installer
* Windows version bump.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: travis laduke <travisladuke@gmail.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Joseph Henry <joseph-henry@users.noreply.github.com>
Co-authored-by: Roman Peshkichev <roman.peshkichev@gmail.com>
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>
Co-authored-by: Jake Vis <jakevis@outlook.com>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: lison <imlison@foxmail.com>
Co-authored-by: Kenny MacDermid <kenny@macdermid.ca>
* Fix primary port binding issue in 1.12 (#2107)
* Add test for primary port bindings to validator - See #2105
* Add delay to binding test
* Remove TCP binding logic from Binder to fix#2105
* add second control plane socket for ipv6
* fix controller network post endpoint
* exit if we can't bind at least one of IPV4 or IPV6 for control plane port
---------
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
* Version bump, Linux version stuff, Debian dependencies from 1.12.0 rebuild, release notes.
* macOS version bump in installer
* Windows version bump.
---------
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
* Add test for primary port bindings to validator - See #2105
* Add delay to binding test
* Remove TCP binding logic from Binder to fix#2105
* add second control plane socket for ipv6
* fix controller network post endpoint
* exit if we can't bind at least one of IPV4 or IPV6 for control plane port
---------
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
* add note about forceTcpRelay
* Create a sample systemd unit for tcp proxy
* set gitattributes for rust & cargo so hashes dont conflict on Windows
* Revert "set gitattributes for rust & cargo so hashes dont conflict on Windows"
This reverts commit 032dc5c108.
* Turn off autocrlf for rust source
Doesn't appear to play nice well when it comes to git and vendored cargo package hashes
* Fix#1883 (#1886)
Still unknown as to why, but the call to `nc->GetProperties()` can fail
when setting a friendly name on the Windows virtual ethernet adapter.
Ensure that `ncp` is not null before continuing and accessing the device
GUID.
* Don't vendor packages for zeroidc (#1885)
* Added docker environment way to join networks (#1871)
* add StringUtils
* fix headers
use recommended headers and remove unused headers
* move extern "C"
only JNI functions need to be exported
* cleanup
* fix ANDROID-50: RESULT_ERROR_BAD_PARAMETER typo
* fix typo in log message
* fix typos in JNI method signatures
* fix typo
* fix ANDROID-51: fieldName is uninitialized
* fix ANDROID-35: memory leak
* fix missing DeleteLocalRef in loops
* update to use unique error codes
* add GETENV macro
* add LOG_TAG defines
* ANDROID-48: add ZT_jnicache.cpp
* ANDROID-48: use ZT_jnicache.cpp and remove ZT_jnilookup.cpp and ZT_jniarray.cpp
* add Event.fromInt
* add PeerRole.fromInt
* add ResultCode.fromInt
* fix ANDROID-36: issues with ResultCode
* add VirtualNetworkConfigOperation.fromInt
* fix ANDROID-40: VirtualNetworkConfigOperation out-of-sync with ZT_VirtualNetworkConfigOperation enum
* add VirtualNetworkStatus.fromInt
* fix ANDROID-37: VirtualNetworkStatus out-of-sync with ZT_VirtualNetworkStatus enum
* add VirtualNetworkType.fromInt
* make NodeStatus a plain data class
* fix ANDROID-52: synchronization bug with nodeMap
* Node init work: separate Node construction and init
* add Node.toString
* make PeerPhysicalPath a plain data class
* remove unused PeerPhysicalPath.fixed
* add array functions
* make Peer a plain data class
* make Version a plain data class
* fix ANDROID-42: copy/paste error
* fix ANDROID-49: VirtualNetworkConfig.equals is wrong
* reimplement VirtualNetworkConfig.equals
* reimplement VirtualNetworkConfig.compareTo
* add VirtualNetworkConfig.hashCode
* make VirtualNetworkConfig a plain data class
* remove unused VirtualNetworkConfig.enabled
* reimplement VirtualNetworkDNS.equals
* add VirtualNetworkDNS.hashCode
* make VirtualNetworkDNS a plain data class
* reimplement VirtualNetworkRoute.equals
* reimplement VirtualNetworkRoute.compareTo
* reimplement VirtualNetworkRoute.toString
* add VirtualNetworkRoute.hashCode
* make VirtualNetworkRoute a plain data class
* add isSocketAddressEmpty
* add addressPort
* add fromSocketAddressObject
* invert logic in a couple of places and return early
* newInetAddress and newInetSocketAddress work
allow newInetSocketAddress to return NULL if given empty address
* fix ANDROID-38: stack corruption in onSendPacketRequested
* use GETENV macro
* JniRef work
JniRef does not use callbacks struct, so remove
fix NewGlobalRef / DeleteGlobalRef mismatch
* use PRId64 macros
* switch statement work
* comments and logging
* Modifier 'public' is redundant for interface members
* NodeException can be made a checked Exception
* 'NodeException' does not define a 'serialVersionUID' field
* 'finalize()' should not be overridden
this is fine to do because ZeroTierOneService calls close() when it is done
* error handling, error reporting, asserts, logging
* simplify loadLibrary
* rename Node.networks -> Node.networkConfigs
* Windows file permissions fix (#1887)
* Allow macOS interfaces to use multiple IP addresses (#1879)
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Fix condition where full HELLOs might not be sent when necessary (#1877)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* 1.10.4 version bumps
* Add security policy to repo (#1889)
* [+] add e2k64 arch (#1890)
* temp fix for ANDROID-56: crash inside newNetworkConfig from too many args
* 1.10.4 release notes
* Windows 1.10.4 Advanced Installer bump
* Revert "temp fix for ANDROID-56: crash inside newNetworkConfig from too many args"
This reverts commit dd627cd7f4.
* actual fix for ANDROID-56: crash inside newNetworkConfig
cast all arguments to varargs functions as good style
* Fix addIp being called with applied ips (#1897)
This was getting called outside of the check for existing ips
Because of the added ifdef and a brace getting moved to the
wrong place.
```
if (! n.tap()->addIp(*ip)) {
fprintf(stderr, "ERROR: unable to add ip address %s" ZT_EOL_S, ip->toString(ipbuf));
}
WinFWHelper::newICMPRule(*ip, n.config().nwid);
```
* 1.10.5 (#1905)
* 1.10.5 bump
* 1.10.5 for Windows
* 1.10.5
* Prevent path-learning loops (#1914)
* Prevent path-learning loops
* Only allow new overwrite if not bonded
* fix binding temporary ipv6 addresses on macos (#1910)
The check code wasn't running.
I don't know why !defined(TARGET_OS_IOS) would exclude code on
desktop macOS. I did a quick search and changed it to defined(TARGET_OS_MAC).
Not 100% sure what the most correct solution there is.
You can verify the old and new versions with
`ifconfig | grep temporary`
plus
`zerotier-cli info -j` -> listeningOn
* 1.10.6 (#1929)
* 1.10.5 bump
* 1.10.6
* 1.10.6 AIP for Windows.
* Release notes for 1.10.6 (#1931)
* Minor tweak to Synology Docker image script (#1936)
* Change if_def again so ios can build (#1937)
All apple's variables are "defined"
but sometimes they are defined as "0"
* move begin/commit into try/catch block (#1932)
Thread was exiting in some cases
* Bump openssl from 0.10.45 to 0.10.48 in /zeroidc (#1938)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.45 to 0.10.48.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.45...openssl-v0.10.48)
---
updated-dependencies:
- dependency-name: openssl
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* new drone bits
* Fix multiple network join from environment entrypoint.sh.release (#1961)
* _bond_m guards _bond, not _paths_m (#1965)
* Fix: warning: mutex '_aqm_m' is not held on every path through here [-Wthread-safety-analysis] (#1964)
* Bump h2 from 0.3.16 to 0.3.17 in /zeroidc (#1963)
Bumps [h2](https://github.com/hyperium/h2) from 0.3.16 to 0.3.17.
- [Release notes](https://github.com/hyperium/h2/releases)
- [Changelog](https://github.com/hyperium/h2/blob/master/CHANGELOG.md)
- [Commits](https://github.com/hyperium/h2/compare/v0.3.16...v0.3.17)
---
updated-dependencies:
- dependency-name: h2
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Add note that binutils is required on FreeBSD (#1968)
* Add prometheus metrics for Central controllers (#1969)
* add header-only prometheus lib to ext
* rename folder
* Undo rename directory
* prometheus simpleapi included on mac & linux
* wip
* wire up some controller stats
* Get windows building with prometheus
* bsd build flags for prometheus
* Fix multiple network join from environment entrypoint.sh.release (#1961)
* _bond_m guards _bond, not _paths_m (#1965)
* Fix: warning: mutex '_aqm_m' is not held on every path through here [-Wthread-safety-analysis] (#1964)
* Serve prom metrics from /metrics endpoint
* Add prom metrics for Central controller specific things
* reorganize metric initialization
* testing out a labled gauge on Networks
* increment error counter on throw
* Consolidate metrics definitions
Put all metric definitions into node/Metrics.hpp. Accessed as needed
from there.
* Revert "testing out a labled gauge on Networks"
This reverts commit 499ed6d95e.
* still blows up but adding to the record for completeness right now
* Fix runtime issues with metrics
* Add metrics files to visual studio project
* Missed an "extern"
* add copyright headers to new files
* Add metrics for sent/received bytes (total)
* put /metrics endpoint behind auth
* sendto returns int on Win32
---------
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
* Central startup update (#1973)
* allow specifying authtoken in central startup
* set allowManagedFrom
* move redis_mem_notification to the correct place
* add node checkins metric
* wire up min/max connection pool size metrics
* x86_64-unknown-linux-gnu on ubuntu runner (#1975)
* adding incoming zt packet type metrics (#1976)
* use cpp-httplib for HTTP control plane (#1979)
refactored the old control plane code to use [cpp-httplib](https://github.com/yhirose/cpp-httplib) instead of a hand rolled HTTP server. Makes the control plane code much more legible. Also no longer randomly stops responding.
* Outgoing Packet Metrics (#1980)
add tx/rx labels to packet counters and add metrics for outgoing packets
* Add short-term validation test workflow (#1974)
Add short-term validation test workflow
* Brenton/curly braces (#1971)
* fix formatting
* properly adjust various lines
breakup multiple statements onto multiple lines
* insert {} around if, for, etc.
* Fix rust dependency caching (#1983)
* fun with rust caching
* kick
* comment out invalid yaml keys for now
* Caching should now work
* re-add/rename key directives
* bump
* bump
* bump
* Don't force rebuild on Windows build GH Action (#1985)
Switching `/t:ZeroTierOne:Rebuild` to just `/t:ZeroTierOne` allows the Windows build to use the rust cache. `/t:ZeroTierOne:Rebuild` cleared the cache before building.
* More packet metrics (#1982)
* found path negotation sends that weren't accounted for
* Fix histogram so it will actually compile
* Found more places for packet metrics
* separate the bind & listen calls on the http backplane (#1988)
* fix memory leak (#1992)
* fix a couple of metrics (#1989)
* More aggressive CLI spamming (#1993)
* fix type signatures (#1991)
* Network-metrics (#1994)
* Add a couple quick functions for converting a uint64_t network ID/node ID into std::string
* Network metrics
* Peer metrics (#1995)
* Adding peer metrics
still need to be wired up for use
* per peer packet metrics
* Fix crash from bad instantiation of histogram
* separate alive & dead path counts
* Add peer metric update block
* add peer latency values in doPingAndKeepalive
* prevent deadlock
* peer latency histogram actually works now
* cleanup
* capture counts of packets to specific peers
---------
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
* Metrics consolidation (#1997)
* Rename zt_packet_incoming -> zt_packet
Also consolidate zt_peer_packets into a single metric with tx and rx labels. Same for ztc_tcp_data and ztc_udp_data
* Further collapse tcp & udp into metric labels for zt_data
* Fix zt_data metric description
* zt_peer_packets description fix
* Consolidate incoming/outgoing network packets to a single metric
* zt_incoming_packet_error -> zt_packet_error
* Disable peer metrics for central controllers
Can change in the future if needed, but given the traffic our controllers serve, that's going to be a *lot* of data
* Disable peer metrics for controllers pt 2
* Update readme files for metrics (#2000)
* Controller Metrics & Network Config Request Fix (#2003)
* add new metrics for network config request queue size and sso expirations
* move sso expiration to its own thread in the controller
* fix potential undefined behavior when modifying a set
* Enable RTTI in Windows build
The new prometheus histogram stuff needs it.
Access violation - no RTTI data!INVALID packet 636ebd9ee8cac6c0 from cafe9efeb9(2605:9880:200:1200:30:571:e34:51/9993) (unexpected exception in tryDecode())
* Don't re-apply routes on BSD
See issue #1986
* Capture setContent by-value instead of by-reference (#2006)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* fix typos (#2010)
* central controller metrics & request path updates (#2012)
* internal db metrics
* use shared mutexes for read/write locks
* remove this lock. only used for a metric
* more metrics
* remove exploratory metrics
place controller request benchmarks behind ifdef
* Improve validation test (#2013)
* fix init order for EmbeddedNetworkController (#2014)
* add constant for getifaddrs cache time
* cache getifaddrs - mac
* cache getifaddrs - linux
* cache getifaddrs - bsd
* cache getifaddrs - windows
* Fix oidc client lookup query
join condition referenced the wrong table. Worked fine unless there were multiple identical client IDs
* Fix udp sent metric
was only incrementing by 1 for each packet sent
* Allow sending all surface addresses to peer in low-bandwidth mode
* allow enabling of low bandwidth mode on controllers
* don't unborrow bad connections
pool will clean them up later
* Multi-arch controller container (#2037)
create arm64 & amd64 images for central controller
* Update README.md
issue #2009
* docker tags change
* fix oidc auth url memory leak (#2031)
getAuthURL() was not calling zeroidc::free_cstr(url);
the only place authAuthURL is called, the url can be retrieved
from the network config instead.
You could alternatively copy the string and call free_cstr in getAuthURL.
If that's better we can change the PR.
Since now there are no callers of getAuthURL I deleted it.
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Bump openssl from 0.10.48 to 0.10.55 in /zeroidc (#2034)
Bumps [openssl](https://github.com/sfackler/rust-openssl) from 0.10.48 to 0.10.55.
- [Release notes](https://github.com/sfackler/rust-openssl/releases)
- [Commits](https://github.com/sfackler/rust-openssl/compare/openssl-v0.10.48...openssl-v0.10.55)
---
updated-dependencies:
- dependency-name: openssl
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* zeroidc cargo warnings (#2029)
* fix unused struct member cargo warning
* fix unused import cargo warning
* fix unused return value cargo warning
---------
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* fix memory leak in macos ipv6/dns helper (#2030)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Consider ZEROTIER_JOIN_NETWORKS in healthcheck (#1978)
* Add a 2nd auth token only for access to /metrics (#2043)
* Add a 2nd auth token for /metrics
Allows administrators to distribute a token that only has access to read
metrics and nothing else.
Also added support for using bearer auth tokens for both types of tokens
Separate endpoint for metrics #2041
* Update readme
* fix a couple of cases of writing the wrong token
* Add warning to cli for allow default on FreeBSD
It doesn't work.
Not possible to fix with deficient network
stack and APIs.
ZeroTierOne-freebsd # zerotier-cli set 9bee8941b5xxxxxx allowDefault=1
400 set Allow Default does not work properly on FreeBSD. See #580
root@freebsd13-a:~/ZeroTierOne-freebsd # zerotier-cli get 9bee8941b5xxxxxx allowDefault
1
* ARM64 Support for TapDriver6 (#1949)
* Release memory previously allocated by UPNP_GetValidIGD
* Fix ifdef that breaks libzt on iOS (#2050)
* less drone (#2060)
* Exit if loading an invalid identity from disk (#2058)
* Exit if loading an invalid identity from disk
Previously, if an invalid identity was loaded from disk, ZeroTier would
generate a new identity & chug along and generate a brand new identity
as if nothing happened. When running in containers, this introduces the
possibility for key matter loss; especially when running in containers
where the identity files are mounted in the container read only. In
this case, ZT will continue chugging along with a brand new identity
with no possibility of recovering the private key.
ZeroTier should exit upon loading of invalid identity.public/identity.secret #2056
* add validation test for #2056
* tcp-proxy: fix build
* Adjust tcp-proxy makefile to support metrics
There's no way to get the metrics yet. Someone will
have to add the http service.
* remove ZT_NO_METRIC ifdef
* Implement recvmmsg() for Linux to reduce syscalls. (#2046)
Between 5% and 40% speed improvement on Linux, depending on system configuration and load.
* suppress warnings: comparison of integers of different signs: 'int64_t' (aka 'long') and 'uint64_t' (aka 'unsigned long') [-Wsign-compare] (#2063)
* fix warning: 'OS_STRING' macro redefined [-Wmacro-redefined] (#2064)
Even though this is in ext, these particular chunks of code were added
by us, so are ok to modify.
* Apply default route a different way - macOS
The original way we applied default route, by forking
0.0.0.0/0 into 0/1 and 128/1 works, but if mac os has any networking
hiccups -if you change SSIDs or sleep/wake- macos erases the system default route.
And then all networking on the computer is broken.
to summarize the new way:
allowDefault=1
```
sudo route delete default 192.168.82.1
sudo route add default 10.2.0.2
sudo route add -ifscope en1 default 192.168.82.1
```
gives us this routing table
```
Destination Gateway RT_IFA Flags Refs Use Mtu Netif Expire rtt(ms) rttvar(ms)
default 10.2.0.2 10.2.0.18 UGScg 90 1 2800 feth4823
default 192.168.82.1 192.168.82.217 UGScIg
```
allowDefault=0
```
sudo route delete default
sudo route delete -ifscope en1 default
sudo route add default 192.168.82.1
```
Notice the I flag, for -ifscope, on the physical default route.
route change does not seem to work reliably.
* fix docker tag for controllers (#2066)
* Update build.sh (#2068)
fix mkwork compilation errors
* Fix network DNS on macOS
It stopped working for ipv4 only networks in Monterey.
See #1696
We add some config like so to System Configuration
```
scutil
show State:/Network/Service/9bee8941b5xxxxxx/IPv4
<dictionary> {
Addresses : <array> {
0 : 10.2.1.36
}
InterfaceName : feth4823
Router : 10.2.1.36
ServerAddress : 127.0.0.1
}
```
* Add search domain to macos dns configuration
Stumbled upon this while debugging something else.
If we add search domain to our system configuration for
network DNS, then search domains work:
```
ping server1 ~
PING server1.my.domain (10.123.3.1): 56 data bytes
64 bytes from 10.123.3.1
```
* Fix reporting of secondaryPort and tertiaryPort See: #2039
* Fix typos (#2075)
* Disable executable stacks on assembly objects (#2071)
Add `--noexecstack` to the assembler flags so the resulting binary
will link with a non-executable stack.
Fixeszerotier/ZeroTierOne#1179
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
* Test that starting zerotier before internet works
* Don't skip hellos when there are no paths available
working on #2082
* Update validate-1m-linux.sh
* Save zt node log files on abort
* Separate test and summary step in validator script
* Don't apply default route until zerotier is "online"
I was running into issues with restarting the zerotier service while
"full tunnel" mode is enabled.
When zerotier first boots, it gets network state from the cache
on disk. So it immediately applies all the routes it knew about
before it shutdown.
The network config may have change in this time.
If it has, then your default route is via a route
you are blocked from talking on. So you can't get the current
network config, so your internet does not work.
Other options include
- don't use cached network state on boot
- find a better criteria than "online"
* Fix node time-to-online counter in validator script
* Export variables so that they are accessible by exit function
* Fix PortMapper issue on ZeroTier startup
See issue #2082
We use a call to libnatpmp::ininatpp to make sure the computer
has working network sockets before we go into the main
nat-pmp/upnp logic.
With basic exponenetial delay up to 30 seconds.
* testing
* Comment out PortMapper debug
this got left turned on in a confusing merge previously
* fix macos default route again
see commit fb6af1971 * Fix network DNS on macOS
adding that stuff to System Config causes this extra route to be added
which breaks ipv4 default route.
We figured out a weird System Coniguration setting
that works.
--- old
couldn't figure out how to fix it in SystemConfiguration
so here we are# Please enter the commit message for your changes. Lines starting
We also moved the dns setter to before the syncIps stuff
to help with a race condition. It didn't always work when
you re-joined a network with default route enabled.
* Catch all conditions in switch statement, remove trailing whitespaces
* Add setmtu command, fix bond lifetime issue
* Basic cleanups
* Check if null is passed to VirtualNetworkConfig.equals and name fixes
* ANDROID-96: Simplify and use return code from node_init directly
* Windows arm64 (#2099)
* ARM64 changes for 1.12
* 1.12 Windows advanced installer updates and updates for ARM64
* 1.12.0
* Linux build fixes for old distros.
* release notes
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: travis laduke <travisladuke@gmail.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Joseph Henry <joseph-henry@users.noreply.github.com>
Co-authored-by: Roman Peshkichev <roman.peshkichev@gmail.com>
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Stavros Kois <47820033+stavros-k@users.noreply.github.com>
Co-authored-by: Jake Vis <jakevis@outlook.com>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
Co-authored-by: lison <imlison@foxmail.com>
Co-authored-by: Kenny MacDermid <kenny@macdermid.ca>
see commit fb6af1971 * Fix network DNS on macOS
adding that stuff to System Config causes this extra route to be added
which breaks ipv4 default route.
We figured out a weird System Coniguration setting
that works.
--- old
couldn't figure out how to fix it in SystemConfiguration
so here we are# Please enter the commit message for your changes. Lines starting
We also moved the dns setter to before the syncIps stuff
to help with a race condition. It didn't always work when
you re-joined a network with default route enabled.
See issue #2082
We use a call to libnatpmp::ininatpp to make sure the computer
has working network sockets before we go into the main
nat-pmp/upnp logic.
With basic exponenetial delay up to 30 seconds.
I was running into issues with restarting the zerotier service while
"full tunnel" mode is enabled.
When zerotier first boots, it gets network state from the cache
on disk. So it immediately applies all the routes it knew about
before it shutdown.
The network config may have change in this time.
If it has, then your default route is via a route
you are blocked from talking on. So you can't get the current
network config, so your internet does not work.
Other options include
- don't use cached network state on boot
- find a better criteria than "online"
Add `--noexecstack` to the assembler flags so the resulting binary
will link with a non-executable stack.
Fixeszerotier/ZeroTierOne#1179
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
Stumbled upon this while debugging something else.
If we add search domain to our system configuration for
network DNS, then search domains work:
```
ping server1 ~
PING server1.my.domain (10.123.3.1): 56 data bytes
64 bytes from 10.123.3.1
```
It stopped working for ipv4 only networks in Monterey.
See #1696
We add some config like so to System Configuration
```
scutil
show State:/Network/Service/9bee8941b5xxxxxx/IPv4
<dictionary> {
Addresses : <array> {
0 : 10.2.1.36
}
InterfaceName : feth4823
Router : 10.2.1.36
ServerAddress : 127.0.0.1
}
```
The original way we applied default route, by forking
0.0.0.0/0 into 0/1 and 128/1 works, but if mac os has any networking
hiccups -if you change SSIDs or sleep/wake- macos erases the system default route.
And then all networking on the computer is broken.
to summarize the new way:
allowDefault=1
```
sudo route delete default 192.168.82.1
sudo route add default 10.2.0.2
sudo route add -ifscope en1 default 192.168.82.1
```
gives us this routing table
```
Destination Gateway RT_IFA Flags Refs Use Mtu Netif Expire rtt(ms) rttvar(ms)
default 10.2.0.2 10.2.0.18 UGScg 90 1 2800 feth4823
default 192.168.82.1 192.168.82.217 UGScIg
```
allowDefault=0
```
sudo route delete default
sudo route delete -ifscope en1 default
sudo route add default 192.168.82.1
```
Notice the I flag, for -ifscope, on the physical default route.
route change does not seem to work reliably.
* Exit if loading an invalid identity from disk
Previously, if an invalid identity was loaded from disk, ZeroTier would
generate a new identity & chug along and generate a brand new identity
as if nothing happened. When running in containers, this introduces the
possibility for key matter loss; especially when running in containers
where the identity files are mounted in the container read only. In
this case, ZT will continue chugging along with a brand new identity
with no possibility of recovering the private key.
ZeroTier should exit upon loading of invalid identity.public/identity.secret #2056
* add validation test for #2056
It doesn't work.
Not possible to fix with deficient network
stack and APIs.
ZeroTierOne-freebsd # zerotier-cli set 9bee8941b5xxxxxx allowDefault=1
400 set Allow Default does not work properly on FreeBSD. See #580
root@freebsd13-a:~/ZeroTierOne-freebsd # zerotier-cli get 9bee8941b5xxxxxx allowDefault
1
* Add a 2nd auth token for /metrics
Allows administrators to distribute a token that only has access to read
metrics and nothing else.
Also added support for using bearer auth tokens for both types of tokens
Separate endpoint for metrics #2041
* Update readme
* fix a couple of cases of writing the wrong token
getAuthURL() was not calling zeroidc::free_cstr(url);
the only place authAuthURL is called, the url can be retrieved
from the network config instead.
You could alternatively copy the string and call free_cstr in getAuthURL.
If that's better we can change the PR.
Since now there are no callers of getAuthURL I deleted it.
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* internal db metrics
* use shared mutexes for read/write locks
* remove this lock. only used for a metric
* more metrics
* remove exploratory metrics
place controller request benchmarks behind ifdef
The new prometheus histogram stuff needs it.
Access violation - no RTTI data!INVALID packet 636ebd9ee8cac6c0 from cafe9efeb9(2605:9880:200:1200:30:571:e34:51/9993) (unexpected exception in tryDecode())
* add new metrics for network config request queue size and sso expirations
* move sso expiration to its own thread in the controller
* fix potential undefined behavior when modifying a set
* Rename zt_packet_incoming -> zt_packet
Also consolidate zt_peer_packets into a single metric with tx and rx labels. Same for ztc_tcp_data and ztc_udp_data
* Further collapse tcp & udp into metric labels for zt_data
* Fix zt_data metric description
* zt_peer_packets description fix
* Consolidate incoming/outgoing network packets to a single metric
* zt_incoming_packet_error -> zt_packet_error
* Disable peer metrics for central controllers
Can change in the future if needed, but given the traffic our controllers serve, that's going to be a *lot* of data
* Disable peer metrics for controllers pt 2
* Adding peer metrics
still need to be wired up for use
* per peer packet metrics
* Fix crash from bad instantiation of histogram
* separate alive & dead path counts
* Add peer metric update block
* add peer latency values in doPingAndKeepalive
* prevent deadlock
* peer latency histogram actually works now
* cleanup
* capture counts of packets to specific peers
---------
Co-authored-by: Joseph Henry <joseph.henry@zerotier.com>
Switching `/t:ZeroTierOne:Rebuild` to just `/t:ZeroTierOne` allows the Windows build to use the rust cache. `/t:ZeroTierOne:Rebuild` cleared the cache before building.
refactored the old control plane code to use [cpp-httplib](https://github.com/yhirose/cpp-httplib) instead of a hand rolled HTTP server. Makes the control plane code much more legible. Also no longer randomly stops responding.
* allow specifying authtoken in central startup
* set allowManagedFrom
* move redis_mem_notification to the correct place
* add node checkins metric
* wire up min/max connection pool size metrics
* add header-only prometheus lib to ext
* rename folder
* Undo rename directory
* prometheus simpleapi included on mac & linux
* wip
* wire up some controller stats
* Get windows building with prometheus
* bsd build flags for prometheus
* Fix multiple network join from environment entrypoint.sh.release (#1961)
* _bond_m guards _bond, not _paths_m (#1965)
* Fix: warning: mutex '_aqm_m' is not held on every path through here [-Wthread-safety-analysis] (#1964)
* Serve prom metrics from /metrics endpoint
* Add prom metrics for Central controller specific things
* reorganize metric initialization
* testing out a labled gauge on Networks
* increment error counter on throw
* Consolidate metrics definitions
Put all metric definitions into node/Metrics.hpp. Accessed as needed
from there.
* Revert "testing out a labled gauge on Networks"
This reverts commit 499ed6d95e.
* still blows up but adding to the record for completeness right now
* Fix runtime issues with metrics
* Add metrics files to visual studio project
* Missed an "extern"
* add copyright headers to new files
* Add metrics for sent/received bytes (total)
* put /metrics endpoint behind auth
* sendto returns int on Win32
---------
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
* add note about forceTcpRelay
* Create a sample systemd unit for tcp proxy
* set gitattributes for rust & cargo so hashes dont conflict on Windows
* Revert "set gitattributes for rust & cargo so hashes dont conflict on Windows"
This reverts commit 032dc5c108.
* Turn off autocrlf for rust source
Doesn't appear to play nice well when it comes to git and vendored cargo package hashes
* Fix#1883 (#1886)
Still unknown as to why, but the call to `nc->GetProperties()` can fail
when setting a friendly name on the Windows virtual ethernet adapter.
Ensure that `ncp` is not null before continuing and accessing the device
GUID.
* Don't vendor packages for zeroidc (#1885)
* Added docker environment way to join networks (#1871)
* add StringUtils
* fix headers
use recommended headers and remove unused headers
* move extern "C"
only JNI functions need to be exported
* cleanup
* fix ANDROID-50: RESULT_ERROR_BAD_PARAMETER typo
* fix typo in log message
* fix typos in JNI method signatures
* fix typo
* fix ANDROID-51: fieldName is uninitialized
* fix ANDROID-35: memory leak
* fix missing DeleteLocalRef in loops
* update to use unique error codes
* add GETENV macro
* add LOG_TAG defines
* ANDROID-48: add ZT_jnicache.cpp
* ANDROID-48: use ZT_jnicache.cpp and remove ZT_jnilookup.cpp and ZT_jniarray.cpp
* add Event.fromInt
* add PeerRole.fromInt
* add ResultCode.fromInt
* fix ANDROID-36: issues with ResultCode
* add VirtualNetworkConfigOperation.fromInt
* fix ANDROID-40: VirtualNetworkConfigOperation out-of-sync with ZT_VirtualNetworkConfigOperation enum
* add VirtualNetworkStatus.fromInt
* fix ANDROID-37: VirtualNetworkStatus out-of-sync with ZT_VirtualNetworkStatus enum
* add VirtualNetworkType.fromInt
* make NodeStatus a plain data class
* fix ANDROID-52: synchronization bug with nodeMap
* Node init work: separate Node construction and init
* add Node.toString
* make PeerPhysicalPath a plain data class
* remove unused PeerPhysicalPath.fixed
* add array functions
* make Peer a plain data class
* make Version a plain data class
* fix ANDROID-42: copy/paste error
* fix ANDROID-49: VirtualNetworkConfig.equals is wrong
* reimplement VirtualNetworkConfig.equals
* reimplement VirtualNetworkConfig.compareTo
* add VirtualNetworkConfig.hashCode
* make VirtualNetworkConfig a plain data class
* remove unused VirtualNetworkConfig.enabled
* reimplement VirtualNetworkDNS.equals
* add VirtualNetworkDNS.hashCode
* make VirtualNetworkDNS a plain data class
* reimplement VirtualNetworkRoute.equals
* reimplement VirtualNetworkRoute.compareTo
* reimplement VirtualNetworkRoute.toString
* add VirtualNetworkRoute.hashCode
* make VirtualNetworkRoute a plain data class
* add isSocketAddressEmpty
* add addressPort
* add fromSocketAddressObject
* invert logic in a couple of places and return early
* newInetAddress and newInetSocketAddress work
allow newInetSocketAddress to return NULL if given empty address
* fix ANDROID-38: stack corruption in onSendPacketRequested
* use GETENV macro
* JniRef work
JniRef does not use callbacks struct, so remove
fix NewGlobalRef / DeleteGlobalRef mismatch
* use PRId64 macros
* switch statement work
* comments and logging
* Modifier 'public' is redundant for interface members
* NodeException can be made a checked Exception
* 'NodeException' does not define a 'serialVersionUID' field
* 'finalize()' should not be overridden
this is fine to do because ZeroTierOneService calls close() when it is done
* error handling, error reporting, asserts, logging
* simplify loadLibrary
* rename Node.networks -> Node.networkConfigs
* Windows file permissions fix (#1887)
* Allow macOS interfaces to use multiple IP addresses (#1879)
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Fix condition where full HELLOs might not be sent when necessary (#1877)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* 1.10.4 version bumps
* Add security policy to repo (#1889)
* [+] add e2k64 arch (#1890)
* temp fix for ANDROID-56: crash inside newNetworkConfig from too many args
* 1.10.4 release notes
* Windows 1.10.4 Advanced Installer bump
* Revert "temp fix for ANDROID-56: crash inside newNetworkConfig from too many args"
This reverts commit dd627cd7f4.
* actual fix for ANDROID-56: crash inside newNetworkConfig
cast all arguments to varargs functions as good style
* Fix addIp being called with applied ips (#1897)
This was getting called outside of the check for existing ips
Because of the added ifdef and a brace getting moved to the
wrong place.
```
if (! n.tap()->addIp(*ip)) {
fprintf(stderr, "ERROR: unable to add ip address %s" ZT_EOL_S, ip->toString(ipbuf));
}
WinFWHelper::newICMPRule(*ip, n.config().nwid);
```
* 1.10.5 (#1905)
* 1.10.5 bump
* 1.10.5 for Windows
* 1.10.5
* Prevent path-learning loops (#1914)
* Prevent path-learning loops
* Only allow new overwrite if not bonded
* fix binding temporary ipv6 addresses on macos (#1910)
The check code wasn't running.
I don't know why !defined(TARGET_OS_IOS) would exclude code on
desktop macOS. I did a quick search and changed it to defined(TARGET_OS_MAC).
Not 100% sure what the most correct solution there is.
You can verify the old and new versions with
`ifconfig | grep temporary`
plus
`zerotier-cli info -j` -> listeningOn
* 1.10.6 (#1929)
* 1.10.5 bump
* 1.10.6
* 1.10.6 AIP for Windows.
---------
Co-authored-by: travis laduke <travisladuke@gmail.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Joseph Henry <joseph-henry@users.noreply.github.com>
Co-authored-by: Roman Peshkichev <roman.peshkichev@gmail.com>
The check code wasn't running.
I don't know why !defined(TARGET_OS_IOS) would exclude code on
desktop macOS. I did a quick search and changed it to defined(TARGET_OS_MAC).
Not 100% sure what the most correct solution there is.
You can verify the old and new versions with
`ifconfig | grep temporary`
plus
`zerotier-cli info -j` -> listeningOn
This was getting called outside of the check for existing ips
Because of the added ifdef and a brace getting moved to the
wrong place.
```
if (! n.tap()->addIp(*ip)) {
fprintf(stderr, "ERROR: unable to add ip address %s" ZT_EOL_S, ip->toString(ipbuf));
}
WinFWHelper::newICMPRule(*ip, n.config().nwid);
```
* add note about forceTcpRelay
* Create a sample systemd unit for tcp proxy
* set gitattributes for rust & cargo so hashes dont conflict on Windows
* Revert "set gitattributes for rust & cargo so hashes dont conflict on Windows"
This reverts commit 032dc5c108.
* Turn off autocrlf for rust source
Doesn't appear to play nice well when it comes to git and vendored cargo package hashes
* Fix#1883 (#1886)
Still unknown as to why, but the call to `nc->GetProperties()` can fail
when setting a friendly name on the Windows virtual ethernet adapter.
Ensure that `ncp` is not null before continuing and accessing the device
GUID.
* Don't vendor packages for zeroidc (#1885)
* Added docker environment way to join networks (#1871)
* add StringUtils
* fix headers
use recommended headers and remove unused headers
* move extern "C"
only JNI functions need to be exported
* cleanup
* fix ANDROID-50: RESULT_ERROR_BAD_PARAMETER typo
* fix typo in log message
* fix typos in JNI method signatures
* fix typo
* fix ANDROID-51: fieldName is uninitialized
* fix ANDROID-35: memory leak
* fix missing DeleteLocalRef in loops
* update to use unique error codes
* add GETENV macro
* add LOG_TAG defines
* ANDROID-48: add ZT_jnicache.cpp
* ANDROID-48: use ZT_jnicache.cpp and remove ZT_jnilookup.cpp and ZT_jniarray.cpp
* add Event.fromInt
* add PeerRole.fromInt
* add ResultCode.fromInt
* fix ANDROID-36: issues with ResultCode
* add VirtualNetworkConfigOperation.fromInt
* fix ANDROID-40: VirtualNetworkConfigOperation out-of-sync with ZT_VirtualNetworkConfigOperation enum
* add VirtualNetworkStatus.fromInt
* fix ANDROID-37: VirtualNetworkStatus out-of-sync with ZT_VirtualNetworkStatus enum
* add VirtualNetworkType.fromInt
* make NodeStatus a plain data class
* fix ANDROID-52: synchronization bug with nodeMap
* Node init work: separate Node construction and init
* add Node.toString
* make PeerPhysicalPath a plain data class
* remove unused PeerPhysicalPath.fixed
* add array functions
* make Peer a plain data class
* make Version a plain data class
* fix ANDROID-42: copy/paste error
* fix ANDROID-49: VirtualNetworkConfig.equals is wrong
* reimplement VirtualNetworkConfig.equals
* reimplement VirtualNetworkConfig.compareTo
* add VirtualNetworkConfig.hashCode
* make VirtualNetworkConfig a plain data class
* remove unused VirtualNetworkConfig.enabled
* reimplement VirtualNetworkDNS.equals
* add VirtualNetworkDNS.hashCode
* make VirtualNetworkDNS a plain data class
* reimplement VirtualNetworkRoute.equals
* reimplement VirtualNetworkRoute.compareTo
* reimplement VirtualNetworkRoute.toString
* add VirtualNetworkRoute.hashCode
* make VirtualNetworkRoute a plain data class
* add isSocketAddressEmpty
* add addressPort
* add fromSocketAddressObject
* invert logic in a couple of places and return early
* newInetAddress and newInetSocketAddress work
allow newInetSocketAddress to return NULL if given empty address
* fix ANDROID-38: stack corruption in onSendPacketRequested
* use GETENV macro
* JniRef work
JniRef does not use callbacks struct, so remove
fix NewGlobalRef / DeleteGlobalRef mismatch
* use PRId64 macros
* switch statement work
* comments and logging
* Modifier 'public' is redundant for interface members
* NodeException can be made a checked Exception
* 'NodeException' does not define a 'serialVersionUID' field
* 'finalize()' should not be overridden
this is fine to do because ZeroTierOneService calls close() when it is done
* error handling, error reporting, asserts, logging
* simplify loadLibrary
* rename Node.networks -> Node.networkConfigs
* Windows file permissions fix (#1887)
* Allow macOS interfaces to use multiple IP addresses (#1879)
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* Fix condition where full HELLOs might not be sent when necessary (#1877)
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
* 1.10.4 version bumps
* Add security policy to repo (#1889)
* [+] add e2k64 arch (#1890)
* temp fix for ANDROID-56: crash inside newNetworkConfig from too many args
* 1.10.4 release notes
---------
Co-authored-by: travis laduke <travisladuke@gmail.com>
Co-authored-by: Grant Limberg <grant.limberg@zerotier.com>
Co-authored-by: Grant Limberg <glimberg@users.noreply.github.com>
Co-authored-by: Leonardo Amaral <leleobhz@users.noreply.github.com>
Co-authored-by: Brenton Bostick <bostick@gmail.com>
Co-authored-by: Sean OMeara <someara@users.noreply.github.com>
Co-authored-by: Joseph Henry <joseph-henry@users.noreply.github.com>
Co-authored-by: Roman Peshkichev <roman.peshkichev@gmail.com>
Still unknown as to why, but the call to `nc->GetProperties()` can fail
when setting a friendly name on the Windows virtual ethernet adapter.
Ensure that `ncp` is not null before continuing and accessing the device
GUID.
When adding Routes to zerotier's Managed Routes, the helper will
add a route rule to the device that does not have a via ip,
so that the address of the Destination segment cannot be routed
correctly within the container.
Here, based on the contents of the routes key in
`zerotier-cli -j listnetworks`,
by determining whether the via key has an ip address,
if it is not null, helper will no longer add route rules.
ARM Cryptography Extension is optional and not all ARM CPUs support it.
For example, the CPU in Raspberry Pi 4 does not support it.
Check for `__ARM_FEATURE_CRYPTO` before attempting to use the optional
extension.
`__ARM_FEATURE_CRYPTO` is defined by both clang and gcc when the target
has the cryptography extension.
Fixes#1854.
Client side:
* Fix compatibility with OneLogin
* Requested scopes vary by OIDC provider. Different providers have different
Controller side:
*Update Postgres queries to latest Central schema
* Added Central Controller support for the different providers
* Base OIDC provider details are still attached to an org. Client ID & group/email lists are now associated with individual networks.
Xcode warns about "Possible misuse of comma operator here".
Comma is a sequencing operator in C++ and original code does work, but
is highly non-idiomatic.
This patch implements a "TUNNELED" status indicator and "forceTcpRelay" setting for custom relays via local.conf.
For example:
{
"settings":
{
"tcpFallbackRelay": "6.79.53.215/443",
"forceTcpRelay":true
}
}
If you have a VM host like parallels, sometimes you get these link-local
default routes:
```
netstat -nrfinet | grep "default\|\/1"
0/1 10.2.0.12 UGScg feth4823
default 192.168.82.1 UGScg en1
0/1 192.168.82.1 UGScIg en1
default link#22 UCSIg bridge101 !
128.0/1 10.2.0.12 UGSc feth4823
128.0/1 192.168.82.1 UGScI en1
```
(the link#22 one)
The _getRTEs function inclused these routes in the list it makes as like:
device: bridge101, target: 0.0.0.0/0
If it happens to be first in the list, bridge101 gets
selected as the default route.
Then Full Tunnel Mode doesn't work.
The other routes in the list are like:
device: en1 target: 192.168.1.0/24 via: metric: 0 ifscope: 0
device: en1 target: 192.168.1.1/32 via: metric: 0 ifscope: 0
We only need the device name from this, so either one will work.
Through using ndk-build, -Wno-unused-command-line-argument is passed in
somewhere in the pipeline and hides this warning.
The warning can be turned on with:
APP_CPPFLAGS := -Wunused-command-line-argument ...
and then when building, you can see:
C/C++: clang++: warning: argument unused during compilation: '-mfloat-abi=softfp' [-Wunused-command-line-argument]
C/C++: clang++: warning: argument unused during compilation: '-mfpu=neon' [-Wunused-command-line-argument]
C/C++: clang++: warning: argument unused during compilation: '-maes' [-Wunused-command-line-argument]
These are unused because both floating-point and NEON are required in
all standard ARMv8 implementations. [1] [2]
[1] https://developer.arm.com/documentation/den0024/a/AArch64-Floating-point-and-NEON
[2] https://stackoverflow.com/a/29891469
Since NDKr15 (released 2017), unified headers are used by default [1]
Remove -isystem option that was passing bad values to command-line.
The actual value being passed to command-line was:
```
-isystem DK/sysroot/usr/include/RIPLE
```
because of using $NDK and $TRIPLE instead of $(NDK) and $(TRIPLE)
But regardless, $NDK and $TRIPLE were never actually defined values and were just
place-holders mentioned in [1]
[1] https://android.googlesource.com/platform/ndk/+/ndk-release-r16/docs/UnifiedHeaders.md
Surface Addresses are the addresses that
the roots report back to you.
This is helpful for trouble shooting.
If you're behind NAT, the source port is different
than what zerotier is bound to.
If the list of surface address ports is larger than the list of
bound addresses, you are probably behind symmetric NAT.
Anways this can be added to later with a more simple
"easy" or "hard" nat computed message somewhere.
* update entrypoint.sh
- propagate TERM/QUIT/INT signals
- add some basic logging
- check for unbound variables
- update "route helper"
- run as subshell, exit if zerotier-one is unavailable so pod can be restarted
- only call `zerotier-cli` once, avoids race conditions
- only add default routes if allowDefault is enabled for that network
- add some more error handling
- sleep after all networks are processed
* switch to polling ZT service at startup
Co-authored-by: Daniel Quinlan <dq@chaosengine.net>
redis plus plus has an annoying feature where it will open a new
connection for each tx or pipeline by default, rather than just fetching
an existing connection from the pool. Let's change that
Port used for PortMapping was not properly randomized causing multiple clients on the same lan to request the same UPnP port, and not all routers handle this gracefully.
Also fixes issue where the portmapper wasn't started at all if a secondary port wasn't specified, or if the tertiary port was manually specified.
When I first bring up the container, I want to know I'm approving the join request for the right node. I can get the node's ZT address by manually executing `zerotier-cli info` in the node (e.g. with `docker-compose exec zerotier zerotier-cli info`) but just having it in the logs to start with is very convenient.
- Resolve issue with join not being checked properly for success without
using external tools
- Resolve issue where initial boot was not being checked properly
- Now output errors when zerotier fails to start
closes#1581
cc @altano for inspiration for this patch
Signed-off-by: Erik Hollensbe <git@hollensbe.org>
Proactively seek, and distribute external surface addresses
This patch introduces a new "self-awareness" behavior which proactively queries peers for external surface addresses and distributes them via PUSH_DIRECT_PATHS. This has the effect of making ZT more responsive to interface changes.
Current behavior:
Previously, this type of information was only mediated via RENDEZVOUS and was only triggered when the client detected that it no longer had a single alive path to a peer. While PUSH_DIRECT_PATHS would correctly (and often) send local addresses, this was not the case for external addresses collected from response HELLOs. This would lead to situations where only one physical address would be distributed to peers. Additionally, if a new physical interface were to be made available to the client, the client would correctly bind to it but never seek information about its external mapping from a peer, and thus the new physical interface would remain unavailable for other peers to learn about until all paths on the previous interface have expired which can take a couple of minutes. In traditional usage of ZT this is not usually a problem, but it becomes a problem in the following scenarios:
Network interfaces go up and down while ZT is running (e.g. switching to LTE or WiFi from a wired connection)
Network interfaces are added or removed in multipath setups
Proposed behavior:
I propose that normal full HELLOs are sent not only on the first interface in use, but all interfaces. This causes planets to respond with a HELLO containing the surface address for each interface. We then collect each address using SelfAwareness::whoami() and distribute them via the normal PUSH_DIRECT_PATHS mechanism.
Rate gate ECHO per Path instead of per Peer
In multipath scenarios user traffic is used to judge the aliveness of a path. If the user traffic is too infrequent to establish aliveness for a given time window (say 500 ms), the bonding layer will send extra ECHOs at a maximum rate of failoverInterval / 3 (or ~ 166 ms) per path. This patch relaxes the rate-limiting of ECHOs significantly in order to prevent a non-multipath node from dropping ECHOs causing multipath nodes to erroneously judge paths to that node to be dead.
Details
This patch decreases the rate limiting from 1000 ms per peer by a factor of 6 to ~166 ms and rate limits ECHOs per Path instead of per Peer. This allows rate limiting to scale with the number of established paths to a peer.
As a result, if all 64 path slots are used a total of 64 x 6 = 384 ECHOs per second will be allowed in the most aggressive case where failoverInterval is set to 500 ms.
Add a method to "kick" the refresh thread and re-post the tokens in the case where the thread is somehow still running & controller pushes out an AUTH_REQUIRED. This situation happens in a corner case still under investigation where the controller pushes out many copies of the network config repeatedly
See issue #750.
zerotier doesn't currently set a metric on routes. Linux
takes this to mean "0", the highest priority.
Every comment in the issue is about routing between zerotier
and lan and how they conflict.
This quick change could fix this problem for most people.
The subnet route for the zerotier network, the one with no
via, is still 0 in this patch. Just the "via" routes get
higher metrics.
If for some reason, you needed your via routes to have
higher priority, you could use a prefix work-around:
192.168.1.0/25 via 10.147.17.1
192.168.1.128/25 via 10.147.17.1
consolidated everything into the single IDC struct. Should help keep from rotating the pkce token as often & causing issues with the login window flapping
and vice versa.
For issue #1104
With some printf debugging, I was seeing:
here, src fe80::3c7a:2dff:fe0c:21ed, target 10.147.20.0, matchingPrefixBits 0, mostMatchingPrefixBits 0
here, src fd8b:d512:4fd6:255:3c99:932f:2fda:6eff, target 10.147.20.0, matchingPrefixBits 0, mostMatchingPrefixBits 0
and (matchingPrefixBits >= mostMatchingPrefixBits) would be true
Then on mac, somewhere downstream from there, the default route would
get messed up:
default via 92:29:f1:6f:2f:76 dev en0
- Can now provide the following environment variables to populate
secrets (nice for kubernetes, other situations)
- ZEROTIER_API_SECRET: authtoken.secret
- ZEROTIER_IDENTITY_PUBLIC: identity.public
- ZEROTIER_IDENTITY_SECRET: identity.secret
- Joining networks by providing them as a part of docker's "command"
array should now work properly
Signed-off-by: Erik Hollensbe <linux@hollensbe.org>
On certain OSes (Linux & Apple) tun#, tap#, and of course zt# are blacklisted by default, this adds wg# to the list as WireGuard is a similar popular service with wg# being the default adapter name(s) by convention.
Unaligned access caused SIGBUS errors on ARMv6 and ARMv7 targets under FreeBSD.
This was also the cause of the repeating TAP devices. Each time the SIGBUS happened, the service would auto-restart itself, create a new TAP device, and then crash again.
The particular place causing the SIGBUS was:
https://github.com/zerotier/ZeroTierOne/blob/master/node/Utils.hpp#L695
Major new features are:
* **Multipath support** with modes modeled after the Linux kernel's bonding driver. This includes active-passive and active-active modes with fast failover and load balancing. See section 2.1.5 of the manual.
* **DNS configuration** push from network controllers to end nodes, with locally configurable permissions for whether or not push is allowed.
* **AES-GMAC-SIV** encryption mode, which is both somewhat more secure and significantly faster than the old Salsa20/12-Poly1305 mode on hardware that supports AES acceleration. This includes virtually all X86-64 chips and most ARM64. This mode is based on AES-SIV and has been audited by Trail of Bits to ensure that it is equivalent security-wise.
Known issues that are not yet fixed in this beta:
* Some Mac users have reported periods of 100% CPU in kernel_task and connection instability after leaving networks that have been joined for a period of time, or needing to kill ZeroTier and restart it to finish leaving a network. This doesn't appear to affect all users and we haven't diagnosed the root cause yet.
* The service sometimes hangs on shutdown requiring a kill -9. This also does not affect all systems or users.
* AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
* Some users have reported multicast/broadcast outages on networks lasting up to 30 seconds. Still investigating.
We're trying to fix all these issues before the 1.6.0 release. Stay tuned.
Just adding it to the repo, but it still needs to be dealt with during install.
Probably put it in $ZT_HOME and then symlink to the proper place for the distro?
searches $ZT_HOME/networks.d/ for network IDs
searches HISTORY for 16 digit numbers that look like network IDs.
If a redis cluster member fails over to the slave, we'll get an error from not specifying the key for the insert. Recover from that instead of crashing the controller
On system shutdown, zerotier is stopped after the network and gets
itself into a connection timeout loop. It hits the TimeoutStopUSec= and
is forcibly killed by SIGKILL. Order zerotier after network.target so it
can shutdown gracefully while the network is still up.
From systemd.special(7):
at shutdown, a unit that is ordered after network.target will be stopped
before the network — to whatever level it might be set up then — is shut
down. It is hence useful when writing service files that require network
access on shutdown, which should order themselves after this target, but
not pull it in
* ARM32 platform build and flag fixes
* Add a clarification line to LICENSE.txt
* Fix license message in CLI
* Windows service now looks for service command line arguments
* Fixed a bug that could cause excessive queued multicasts
Major Changes
* Mac version no longer requires a kernel extension, instead making use of the [feth interfaces](https://apple.stackexc
* Added support for concurrent multipath (multiple paths at once) with traffic weighting by link quality and faster rec
* Added under-the-hood support for QoS (not yet exposed) that will eventually be configurable via our rules engine.
Minor Changes and Bug Fixes
* Experimental DB driver for [LF](https://github.com/zerotier/lf) to store network controller data (LFDB.cpp / LFDB.hpp
* Modified credential push and direct path push timings and algorithms to somewhat reduce "chattiness" of the protocol
* Removed our beta/half-baked integration of Central with the Windows UI. We're going to do a whole new UI of some kind
* Fixed stack overflow issues on Linux versions using musl libc.
* Fixed some alignment problems reported on ARM and ARM64, but some reports we could not reproduce so please report any
* Fixed numerous other small issues and bugs such as ARM alignment issues causing crashes on some devices.
* Windows now sets the adapter name such that it is consistent in both the Windows UI and command line utilities.
Because zerotier-one depends on "one", which is a phony target,
making it relinks zerotier-one every time, which is probably
not the desired behavior.
In any case, zerotier-one, zerotier-cli, and zerotier-idtool are
real target, so they should have real dependencies and actions.
The "one" target now depends on these three files.
Behavior is mostly unchanged otherwise.
Made the smallest change I could figure out.
Had to remove -Werror because something in the postgres adapter woudn't
compile.
brew install postgresql rabbitmq-c
and seems to put the libs in the right place
This commit changes the Dockerfile ot use the multi-stage build and help
to get an automated build on Docker.
The idea of the multi-stage build is to use the already stable Debian
distribution channel to provide up-to-date versions of ZeroTier.
The benefit is that it would be possible to automate the image build,
either on Docker Hub, Travis or taking advantage of the [docker-library/official-images] infrastructure.
This changes follows the best-practices suggested by
[docker-library/official-images], such as using a High Availability GPG
keyserver, providing a default CMD, allowing "bash" on `docker run` and
others.
Given that both the builder `debian:stretch` and `alpine:latest` are
official messages and have [manifests], this means that this Dockerfile
is also multi-platform. This means that this same Dockerfile will
pick-up the correct Debian package according to the architecture of the
running system during build.
With this changes we could try to promote the image to be parte of
[docker-library/official-images], and take advantage of automated
publishing of multi-architecture images. Others would be able to use
`docker run zerotier` and download the latest version appropriate to
their system.
Related to #682
[docker-library/official-images]: https://github.com/docker-library/official-images
[manifests]: https://blog.docker.com/2017/09/docker-official-images-now-multi-platform/
1) Use existing standard libraries to convert to/from IPv4/IPv6 strings and binary representation.
2) Move null terminator assignment InetAddress::toIpString to top of function, this way if ANY errors occurs that don't write content to the buffer, we're not passing a potentially dangerous buffer around.
* Fixed a bug that caused exits to take a long time on Mac due to huge numbers of redundant attempts to delete managed routes.
* Fixed a socket limit problem on Windows that caused the ZeroTier service to run out of sockets, causing the UI and CLI to be unable to access the API.
* Fixed a threading bug in the ZeroTier Core, albeit one that never manifested on the regular ZeroTier One service/client.
* Fixed a bug that could cause the service to crash if an authorized local client accessed an invalid URL via the control API. (Not exploitable since you needed admin access anyway.)
Added LinuxNetLink to talk to the rtnetlink socket for adding interfaces, addresses routes. Not yet complete. Can currently monitor changes on the system.
when building with `ZT_DEBUG=1` this hint produces a warning:
> node/Packet.cpp:335:43: error: 'register' storage class specifier is deprecated and incompatible with C++17 [-Werror,-Wdeprecated-register]
See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n4340
The Hyperledger implementation (https://github.com/hyperledger/iroha-ed25519)
contains changes to the assembly code to allow PIC. This in turn fixes
compilation/linking of ZeroTier One when "full hardening" flags are used.
* Features and Core Improvements
* Path selection has been overhauled to improve path stability, simplify code, and prepare for multi-path and trunking in the next major release.
* This version introduces remote tracing for remote diagnostics. Network controllers can set a node (usually the controller itself) to receive remote tracing events from all members of the network or from select members. Events are only sent if they pertain to a given network for security reasons.
* Multicast replication can now be done by designated multicast replicators on a network (flagged as such at the controller) rather than by the sender. Most users won't want this, but it's useful for specialized use cases on hub-and-spoke networks and for low-power devices.
* Cryptographic performance improvements on several platforms.
* Multithreaded performance improvements throughout the code base, including the use of an inline lightweight spinlock for low-contention resources.
* Bugs fixed
* Disappearing routes on Mac (GitHub issue #600)
* Route flapping and path instability in some dual-stack V4/V6 networks
* Blacklist (in local.conf) doesn't work reliably (GitHub issue #656)
* Connection instabilities due to unsigned integer overflows in timing comparisons (use int64_t instead of uint64_t)
* Binaries don't run on some older or lower-end 32-bit ARM chips (build problem)
* ARM NEON crypto code crashes (build problem)
* Fixed some lock ordering issues revealed by "valgrind" tool
* The "zerotier-idtool" command could not be accessed from "zerotier-one" via command line switch
* Leaking sockets on some platforms when uPnP/NAT-PMP is enabled
* Fixed two very rare multithreading issues that were only observed on certain systems
* Platform-Specific Changes
* MacOS
* Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done driectly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
* Windows
* The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
* The Windows official packages are now signed with an EV certificate (with hardware key).
* The Windows UI can now log into ZeroTier Central and join networks via the Central API.
* The `zerotier-idtool` command should now work on Windows without ugly hacks.
* Upgraded the installer version.
* Made a few changes to hopefully fix sporadic "will not uninstall" problems, though we cannot duplicate these issues ourselves.
* Linux
* Device names are now generated deterministically based on network IDs for all newly joined networks.
* Android
* Multicast now works on Android in most cases! Android apps can send and receive multicast and subscribe to multicast group IPs. Note that in some cases the app must bind to the specific correct interface for this to work.
* IPv6 can be disabled in UI for cases where it causes problems.
* Path selection has been overhauled to improve path stability, simplify code, and prepare for multi-path and trunking in the next major release.
* This version introduces remote tracing for remote diagnostics. Network controllers can set a node (usually the controller itself) to receive remote tracing events from all members of the network or from select members. Events are only sent if they pertain to a given network for security reasons.
* Multicast replication can now be done by designated multicast replicators on a network (flagged as such at the controller) rather than by the sender. Most users won't want this, but it's useful for specialized use cases on hub-and-spoke networks and for low-power devices.
* Cryptographic performance improvements on several platforms.
* Multithreaded performance improvements throughout the code base, including the use of an inline lightweight spinlock for low-contention resources.
* Bugs fixed
* Disappearing routes on Mac (GitHub issue #600)
* Route flapping and path instability in some dual-stack V4/V6 networks
* Blacklist (in local.conf) doesn't work reliably (GitHub issue #656)
* Connection instabilities due to unsigned integer overflows in timing comparisons (use int64_t instead of uint64_t)
* Binaries don't run on some older or lower-end 32-bit ARM chips (build problem)
* ARM NEON crypto code crashes (build problem)
* Fixed some lock ordering issues revealed by "valgrind" tool
* The "zerotier-idtool" command could not be accessed from "zerotier-one" via command line switch
* Leaking sockets on some platforms when uPnP/NAT-PMP is enabled
* Fixed two very rare multithreading issues that were only observed on certain systems
* Platform-Specific Changes
* MacOS
* Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done driectly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
* Windows
* The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
* The Windows official packages are now signed with an EV certificate (with hardware key).
* The Windows UI can now log into ZeroTier Central and join networks via the Central API.
* The `zerotier-idtool` command should now work on Windows without ugly hacks.
* Upgraded the installer version.
* Made a few changes to hopefully fix sporadic "will not uninstall" problems, though we cannot duplicate these issues ourselves.
* Linux
* Device names are now generated deterministically based on network IDs for all newly joined networks.
* Android
* Multicast now works on Android in most cases! Android apps can send and receive multicast and subscribe to multicast group IPs. Note that in some cases the app must bind to the specific correct interface for this to work.
* IPv6 can be disabled in UI for cases where it causes problems.
Adds a "vault" section to local.conf. Example local.conf:
{
"config": {
"vault": {
"vaultURL": "https://some.vault.host:8200",
"vaultToken": "my-super-secret-vault-token",
"vaultPath": "secure/place/to/put/identity"
}
}
Additionally, the following environment variables can be set. Environment variables override local.conf:
VAULT_ADDR
VAULT_TOKEN
VAULT_PATH
Identities will be placed in the keys "public" and "secret" under the user specified path. If no path is specified, they will be placed in the token specific cubbyhole.
If identity.public and identity.secret exist on disk and vault is configured, they will be automatically added to Vault and removed from disk.
TODO:
* Decide behavior for if Vault cannot be reached.
* Add libcurl as a dependency in Linux & Mac builds
* Add libcurl as a requirement for linux packages
When running ArchlinuxARM with 64bit support on Raspberry Pi3, the default build fails due to the architecture look-up at the beginning of the build process. A simple addition of the Armv7l section above allows the build to continue and successfully run.
There were cases in the code where time calculations and comparisons were overflowing and causing connection instability. This will keep time calculations within expected ranges.
I've only tested locally, on a power8 box running Ubuntu 16.10, but everything (and earth) checks out.
Let me know if more testing infrastructure is needed.
* Managed routes are now only bifurcated for the default route. This is a change in behavior, though few people will probably notice. Bifurcating all managed routes was causing more trouble than it was worth for most users.
* Up to 2X crypto speedup on x86-64 (except Windows, which will take some porting) and 32-bit ARM platforms due to integration of fast assembly language implementations of Salsa20/12 from the [supercop](http://bench.cr.yp.to/supercop.html) code base. These were written by Daniel J. Bernstein and are in the public domain. My Macbook Pro (Core i5 2.8ghz) now does almost 1.5GiB/sec Salsa20/12 per core and a Raspberry Pi got a 2X boost. 64-bit ARM support and Windows support will take some work but should not be too hard.
* Refactored code that manages credentials to greatly reduce memory use in most cases. This may also result in a small performance improvement.
* Reworked and simplified path selection and priority logic to fix path instability and dead path persistence edge cases. There have been some sporadic reports of persistent path instabilities and dead paths hanging around that take minutes to resolve. These have proven difficult to reproduce in house, but hopefully this will fix them. In any case it seems to speed up path establishment in our tests and it makes the code simpler and more readable.
* Eliminated some unused cruft from the code around path management and in the peer class.
* Fixed an issue causing build problems on some MIPS architecture systems.
* Fixed Windows forgetting routes on sleep/wake or in some other circumstances. (GitHub issue #465)
If there is something you'd like to have added to ZeroTier, to go to https://discuss.zerotier.com/c/feature-requests/ instead. Issues there can be voted on and discussed in-depth.
about: Game issues are better served by forum posts
title: Please go to our Discuss or Reddit for game-related issues. Thanks!
labels: wontfix
assignees: ''
---
Are you having trouble connecting to a game on your virtual network after installing ZeroTier?
- [ ] Yes
- [ ] No
If you answered yes, then it is very likely that your question would be better answered on our [Community Forums](https://discuss.zerotier.com) or [Reddit](https://www.reddit.com/r/zerotier/) community; we monitor both regularly. We also have extensive documentation on our [Knowledge Base](https://zerotier.atlassian.net/wiki/spaces/SD/overview). Thank you!
# Authors and Third Party Code Licensing Information
## Primary Authors
## Primary Authors
* ZeroTier Core and ZeroTier One virtual networking service<br>
* ZeroTier Core and ZeroTier One virtual networking service<br>
Adam Ierymenko / adam.ierymenko@zerotier.com
Adam Ierymenko / adam.ierymenko@zerotier.com
Joseph Henry / joseph.henry@zerotier.com (QoS and multipath)
* Java JNI Interface to enable Android application development, and Android app itself (code for that is elsewhere)<br>
* Java JNI Interface to enable Android application development, and Android app itself (code for that is elsewhere)<br>
Grant Limberg / glimberg@gmail.com
Grant Limberg / glimberg@gmail.com
@ -25,7 +28,7 @@
## Third-Party Code
## Third-Party Code
ZeroTier includes the following third party code, either in ext/ or incorporated into the ZeroTier core.
ZeroTier includes the following third party code, either in ext/ or incorporated into the ZeroTier core. This third party code remains licensed under its original license and is not subject to ZeroTier's BSL license.
* LZ4 compression algorithm by Yann Collet
* LZ4 compression algorithm by Yann Collet
@ -45,13 +48,6 @@ ZeroTier includes the following third party code, either in ext/ or incorporated
* Home page: https://github.com/nlohmann/json
* Home page: https://github.com/nlohmann/json
* License grant: MIT
* License grant: MIT
* TunTapOSX by Mattias Nissler
* Files: ext/tap-mac/tuntap/*
* Home page: http://tuntaposx.sourceforge.net/
* License grant: BSD attribution no-endorsement
* ZeroTier Modifications: change interface name to zt#, increase max MTU, increase max devices
* tap-windows6 by the OpenVPN project
* tap-windows6 by the OpenVPN project
* Files: windows/TapDriver6/*
* Files: windows/TapDriver6/*
@ -71,3 +67,9 @@ ZeroTier includes the following third party code, either in ext/ or incorporated
* Files: ext/libnatpmp/* ext/miniupnpc/*
* Files: ext/libnatpmp/* ext/miniupnpc/*
* Home page: http://miniupnp.free.fr/
* Home page: http://miniupnp.free.fr/
* License grant: BSD attribution no-endorsement
* License grant: BSD attribution no-endorsement
* cpp-httplib by yhirose
* Files: ext/cpp-httplib/*
* Home page: https://github.com/yhirose/cpp-httplib
The final .AIP file can only be edited on Windows with [Advanced Installer Enterprise](http://www.advancedinstaller.com/). In addition to incrementing the version be sure that a new product code is generated. (The "upgrade code" GUID on the other hand must never change.)
The final .AIP file can only be edited on Windows with [Advanced Installer Enterprise](http://www.advancedinstaller.com/). In addition to incrementing the version be sure that a new product code is generated. (The "upgrade code" GUID on the other hand must never change.)
@ -29,36 +28,6 @@ Mac's easy. Just type:
You will need [Packages](http://s.sudre.free.fr/Software/Packages/about.html) and our release signing key in the keychain.
You will need [Packages](http://s.sudre.free.fr/Software/Packages/about.html) and our release signing key in the keychain.
## Linux
Mount the GPG key for *contact@zerotier.com* and then on an x86_64 box with a recent version of Docker and an Internet connection run:
make distclean
cd linux-build-farm
./build.sh
This will build i386 and x86_64 packages. Now ssh into our build Raspberry Pi and type `make debian` there to build the Raspbian armhf package. Copy it to `debian-jessie/` inside `linux-build-farm` so that it will be included in the repositories we generate. Now generate the YUM and APT repos:
rm -rf ~/.aptly*
rm -rf /tmp/zt-rpm-repo
./make-apt-repos.sh
./make-rpm-repos.sh
This will require the passphrase for *contact@zerotier.com*.
The contents of ~/.aptly/public must be published as `debian/` on `download.zerotier.com`. The contents of /tmp/zt-rpm-repo are published as `redhat/` on same.
## Windows
## Windows
First load the Visual Studio solution and rebuild the UI and ZeroTier One in both x64 and i386 `Release` mode. Then load [Advanced Installer Enterprise](http://www.advancedinstaller.com/), check that the version is correct, and build. The build will fail if any build artifacts are missing, and Windows must have our product singing key (from DigiCert) available to sign the resulting MSI file. The MSI must then be tested on at least a few different CLEAN Windows VMs to ensure that the installer is valid and properly signed.
First load the Visual Studio solution and rebuild the UI and ZeroTier One in both x64 and i386 `Release` mode. Then load [Advanced Installer Enterprise](http://www.advancedinstaller.com/), check that the version is correct, and build. The build will fail if any build artifacts are missing, and Windows must have our product singing key (from DigiCert) available to sign the resulting MSI file. The MSI must then be tested on at least a few different CLEAN Windows VMs to ensure that the installer is valid and properly signed.
*After the MSI is published to download.zerotier.com in the proper RELEASE/#.#.#/dist subfolder for its version* the Chocolatey package must be rebuilt and published. Open a command prompt, change to `ext/installfiles/windows/chocolatey`, and type `choco pack`. Then use `choco push` to push it to Chocolatey (API key required).
Note that this does not cover rebuilding the drivers or their containing MSI projects, as this is typically not necessary and they are shipped in binary form in the repository for convenience.
## iOS, Android
... no docs here yet since this is done entirely out of band with regular installs.
**NOTE:** _Most of this information pertains to the docker image only. For more information about ZeroTier, check out the repository_: [here](https://github.com/zerotier/ZeroTierOne) or the [commercial website](https://www.zerotier.com).
[ZeroTier](https://www.zerotier.com) is a smart programmable Ethernet switch for planet Earth. It allows all networked devices, VMs, containers, and applications to communicate as if they all reside in the same physical data center or cloud region.
This is accomplished by combining a cryptographically addressed and secure peer to peer network (termed VL1) with an Ethernet emulation layer somewhat similar to VXLAN (termed VL2). Our VL2 Ethernet virtualization layer includes advanced enterprise SDN features like fine grained access control rules for network micro-segmentation and security monitoring.
All ZeroTier traffic is encrypted end-to-end using secret keys that only you control. Most traffic flows peer to peer, though we offer free (but slow) relaying for users who cannot establish peer to peer connections.
The goals and design principles of ZeroTier are inspired by among other things the original [Google BeyondCorp](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43231.pdf) paper and the [Jericho Forum](https://en.wikipedia.org/wiki/Jericho_Forum) with its notion of "deperimeterization."
Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download/). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
ZeroTier is licensed under the [BSL version 1.1](https://mariadb.com/bsl11/). See [LICENSE.txt](https://github.com/zerotier/ZeroTierOne/blob/master/LICENSE.txt) and the [ZeroTier pricing page](https://www.zerotier.com/pricing) for details. ZeroTier is free to use internally in businesses and academic institutions and for non-commercial purposes. Certain types of commercial use such as building closed-source apps and devices based on ZeroTier or offering ZeroTier network controllers and network management as a SaaS service require a commercial license.
A small amount of third party code is also included in ZeroTier and is not subject to our BSL license. See [AUTHORS.md](https://github.com/zerotier/ZeroTierOne/blob/master/AUTHORS.md) for a list of third party code, where it is included, and the licenses that apply to it. All of the third party code in ZeroTier is liberally licensed (MIT, BSD, Apache, public domain, etc.).
## Building the docker image
Due to the network being a substrate for most applications and not an application unto itself, it makes sense that many people would want to build their own image based on our formula.
The image is based on `debian:buster`.
The `Dockerfile.release` file contains build instructions for building the described image in the rest of the README. The build is multi-arch and multi-release capable.
These build arguments power the build:
- `PACKAGE_BASEURL`: The base URL of the package repository to fetch from. (default: `https://download.zerotier.com/debian/buster/pool/main/z/zerotier-one/`)
- `ARCH`: The architecture of the package, in debian format. Must match your image arch. (default: `amd64`)
- `VERSION`: **REQUIRED** the version of ZeroTier to fetch.
The `entrypoint.sh` in the docker image is a little different; zerotier will be spawned in the background and the "main process" is actually just a sleeping shell script. This allows `zerotier-one` to gracefully terminate in some situations largely unique to docker.
The `zerotier/zerotier` image requires the `CAP_NET_ADMIN` capability and the `/dev/net/tun` device must be forwarded to it.
To join a network, simply supply it on the command-line; you can supply multiple networks.
Once joining all the networks you have provided, it will sleep until terminated. Note that in ZeroTier, joining a network does not necessarily mean you have an IP or can do anything, really. You will want to probe the control socket:
```
docker exec myzerotier zerotier-cli listnetworks
```
To ensure you have a network available before trying to listen on it. Without pre-configuring the identity, this usually means going to the central admin panel and clicking the checkmark against your zerotier identity.
### Environment Variables
You can control a few settings including the identity used and the authtoken used to interact with the control socket (which you can forward and access through `localhost:9993`).
- `ZEROTIER_JOIN_NETWORKS`: additional way to set networks to join.
- `ZEROTIER_API_SECRET`: replaces the `authtoken.secret` before booting and allows you to manage the control socket's authentication key.
- `ZEROTIER_IDENTITY_PUBLIC`: the `identity.public` file for zerotier-one. Use `zerotier-idtool` to generate one of these for you.
- `ZEROTIER_IDENTITY_SECRET`: the `identity.secret` file for zerotier-one. Use `zerotier-idtool` to generate one of these for you.
- `ZEROTIER_LOCAL_CONF`: Sets the the `local.conf` file content for zerotier-one
### Tips
- Forwarding port `<dockerip>:9993` to somewhere outside is probably a good idea for highly trafficked services.
- Forwarding `localhost:9993` to a control network where you can drive it remotely might be a good idea, just be sure to set your authtoken properly through environment variables.
- Pre-generating your identities could be much simpler to do via our [terraform plugin](https://github.com/zerotier/terraform-provider-zerotier)
ZeroTier is an enterprise Ethernet switch for planet Earth.
*This document is written for a software developer audience. For information on using ZeroTier, see the: [Website](https://www.zerotier.com), [Documentation Site](https://docs.zerotier.com), and [Discussion Forum](https://discuss.zerotier.com).*
It erases the LAN/WAN distinction and makes VPNs, tunnels, proxies, and other kludges arising from the inflexible nature of physical networks obsolete. Everything is encrypted end-to-end and traffic takes the most direct (peer to peer) path available.
ZeroTier is a smart programmable Ethernet switch for planet Earth. It allows all networked devices, VMs, containers, and applications to communicate as if they all reside in the same physical data center or cloud region.
Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download.shtml). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
This is accomplished by combining a cryptographically addressed and secure peer to peer network (termed VL1) with an Ethernet emulation layer somewhat similar to VXLAN (termed VL2). Our VL2 Ethernet virtualization layer includes advanced enterprise SDN features like fine grained access control rules for network micro-segmentation and security monitoring.
All ZeroTier traffic is encrypted end-to-end using secret keys that only you control. Most traffic flows peer to peer, though we offer free (but slow) relaying for users who cannot establish peer to peer connections.
The goals and design principles of ZeroTier are inspired by among other things the original [Google BeyondCorp](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/43231.pdf) paper and the [Jericho Forum](https://en.wikipedia.org/wiki/Jericho_Forum) with its notion of "deperimeterization."
Visit [ZeroTier's site](https://www.zerotier.com/) for more information and [pre-built binary packages](https://www.zerotier.com/download/). Apps for Android and iOS are available for free in the Google Play and Apple app stores.
ZeroTier is licensed under the [BSL version 1.1](https://mariadb.com/bsl11/). See [LICENSE.txt](LICENSE.txt) and the [ZeroTier pricing page](https://www.zerotier.com/pricing) for details. ZeroTier is free to use internally in businesses and academic institutions and for non-commercial purposes. Certain types of commercial use such as building closed-source apps and devices based on ZeroTier or offering ZeroTier network controllers and network management as a SaaS service require a commercial license.
A small amount of third party code is also included in ZeroTier and is not subject to our BSL license. See [AUTHORS.md](AUTHORS.md) for a list of third party code, where it is included, and the licenses that apply to it. All of the third party code in ZeroTier is liberally licensed (MIT, BSD, Apache, public domain, etc.).
### Getting Started
### Getting Started
ZeroTier's basic operation is easy to understand. Devices have 10-digit *ZeroTier addresses* like `89e92ceee5` and networks have 16-digit network IDs like `8056c2e21c000001`. All it takes for a device to join a network is its 16-digit ID, and all it takes for a network to authorize a device is its 10-digit address. Everything else is automatic.
Everything in the ZeroTier world is controlled by two types of identifier: 40-bit/10-digit *ZeroTier addresses* and 64-bit/16-digit *network IDs*. These identifiers are easily distinguished by their length. A ZeroTier address identifies a node or "device" (laptop, phone, server, VM, app, etc.) while a network ID identifies a virtual Ethernet network that can be joined by devices.
A "device" in our terminology is any "unit of compute" capable of talking to a network: desktops, laptops, phones, servers, VMs/VPSes, containers, and even user-space applications via our [SDK](https://github.com/zerotier/ZeroTierSDK).
ZeroTier addresses can be thought of as port numbers on an enormous planet-wide enterprise Ethernet smart switch supporting VLANs. Network IDs are VLAN IDs to which these ports may be assigned. A single port can be assigned to more than one VLAN.
For testing purposes we provide a public virtual network called *Earth* with network ID `8056c2e21c000001`. You can join it with:
A ZeroTier address looks like `8056c2e21c` and a network ID looks like `8056c2e21c000001`. Network IDs are composed of the ZeroTier address of that network's primary controller and an arbitrary 24-bit ID that identifies the network on this controller. Network controllers are roughly analogous to SDN controllers in SDN protocols like [OpenFlow](https://en.wikipedia.org/wiki/OpenFlow), though as with the analogy between VXLAN and VL2 this should not be read to imply that the protocols or design are the same. You can use our convenient and inexpensive SaaS hosted controllers at [my.zerotier.com](https://my.zerotier.com/) or [run your own controller](controller/) if you don't mind messing around with JSON configuration files or writing scripts to do so.
sudo zerotier-cli join 8056c2e21c000001
Now wait about 30 seconds and check your system with `ip addr list` or `ifconfig`. You'll see a new interface whose name starts with *zt* and it should quickly get an IPv4 and an IPv6 address. Once you see it get an IP, try pinging `earth.zerotier.net` at `29.209.112.93`. If you've joined Earth from more than one system, try pinging your other machine. If you don't want to belong to a giant Ethernet party line anymore, just type:
sudo zerotier-cli leave 8056c2e21c000001
The *zt* interface will disappear. You're no longer on the network.
To create networks of your own, you'll need a network controller. ZeroTier One (for desktops and servers) includes controller functionality in its default build that can be configured via its JSON API (see [README.md in controller/](controller/)). ZeroTier provides a hosted solution with a nice web UI and SaaS add-ons at [my.zerotier.com](https://my.zerotier.com/). Basic controller functionality is free for up to 100 devices.
### Project Layout
### Project Layout
The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
- `artwork/`: icons, logos, etc.
- `artwork/`: icons, logos, etc.
- `attic/`: old stuff and experimental code that we want to keep around for reference.
- `attic/`: old stuff and experimental code that we want to keep around for reference.
- `controller/`: the reference network controller implementation, which is built and included by default on desktop and server build targets.
- `controller/`: the reference network controller implementation, which is built and included by default on desktop and server build targets.
@ -35,91 +37,161 @@ To create networks of your own, you'll need a network controller. ZeroTier One (
- `ext/`: third party libraries, binaries that we ship for convenience on some platforms (Mac and Windows), and installation support files.
- `ext/`: third party libraries, binaries that we ship for convenience on some platforms (Mac and Windows), and installation support files.
- `include/`: include files for the ZeroTier core.
- `include/`: include files for the ZeroTier core.
- `java/`: a JNI wrapper used with our Android mobile app. (The whole Android app is not open source but may be made so in the future.)
- `java/`: a JNI wrapper used with our Android mobile app. (The whole Android app is not open source but may be made so in the future.)
- `macui/`: a Macintosh menu-bar app for controlling ZeroTier One, written in Objective C.
- `node/`: the ZeroTier virtual Ethernet switch core, which is designed to be entirely separate from the rest of the code and able to be built as a stand-alone OS-independent library. Note to developers: do not use C++11 features in here, since we want this to build on old embedded platforms that lack C++11 support. C++11 can be used elsewhere.
- `node/`: the ZeroTier virtual Ethernet switch core, which is designed to be entirely separate from the rest of the code and able to be built as a stand-alone OS-independent library. Note to developers: do not use C++11 features in here, since we want this to build on old embedded platforms that lack C++11 support. C++11 can be used elsewhere.
- `osdep/`: code to support and integrate with OSes, including platform-specific stuff only built for certain targets.
- `osdep/`: code to support and integrate with OSes, including platform-specific stuff only built for certain targets.
- `rule-compiler/`: JavaScript rules language compiler for defining network-level rules.
- `service/`: the ZeroTier One service, which wraps the ZeroTier core and provides VPN-like connectivity to virtual networks for desktops, laptops, servers, VMs, and containers.
- `service/`: the ZeroTier One service, which wraps the ZeroTier core and provides VPN-like connectivity to virtual networks for desktops, laptops, servers, VMs, and containers.
- `tcp-proxy/`: TCP proxy code run by ZeroTier, Inc. to provide TCP fallback (this will die soon!).
- `windows/`: Visual Studio solution files, Windows service code, and the Windows task bar app UI.
- `windows/`: Visual Studio solution files, Windows service code for ZeroTier One, and the Windows task bar app UI.
- `zeroidc/`: OIDC implementation used by ZeroTier service to log into SSO-enabled networks. (This part is written in Rust, and more Rust will be appearing in this repository in the future.)
The base path contains the ZeroTier One service main entry point (`one.cpp`), self test code, makefiles, etc.
### Contributing
Please do pull requests off of the `dev` branch.
Releases are done by merging `dev` into `main` and then tagging and doing builds.
### Build and Platform Notes
### Build and Platform Notes
To build on Mac and Linux just type `make`. On FreeBSD and OpenBSD `gmake` (GNU make) is required and can be installed from packages or ports. For Windows there is a Visual Studio solution in `windows/'.
To build on Mac and Linux just type `make`. On FreeBSD and OpenBSD `gmake` (GNU make) is required and can be installed from packages or ports. For Windows there is a Visual Studio solution in `windows/`.
- **Mac**
- **Mac**
- Xcode command line tools for OSX 10.7 or newer are required.
- Xcode command line tools for macOS 10.13 or newer are required.
- Tap device driver kext source is in `ext/tap-mac` and a signed pre-built binary can be found in `ext/bin/tap-mac`. You should not need to build it yourself. It's a fork of [tuntaposx](http://tuntaposx.sourceforge.net) with device names changed to `zt#`, support for a larger MTU, and tun functionality removed.
- Rust for x86_64 and ARM64 targets *if SSO is enabled in the build*.
- **Linux**
- **Linux**
- The minimum compiler versions required are GCC/G++ 4.9.3 or CLANG/CLANG++ 3.4.2.
- The minimum compiler versions required are GCC/G++ 8.x or CLANG/CLANG++ 5.x.
- Linux makefiles automatically detect and prefer clang/clang++ if present as it produces smaller and slightly faster binaries in most cases. You can override by supplying CC and CXX variables on the make command line.
- Linux makefiles automatically detect and prefer clang/clang++ if present as it produces smaller and slightly faster binaries in most cases. You can override by supplying CC and CXX variables on the make command line.
- CentOS 7 ships with a version of GCC/G++ that is too old, but a new enough version of CLANG can be found in the *epel* repositories. Type `yum install epel-release` and then `yum install clang` to build there.
- Rust for x86_64 and ARM64 targets *if SSO is enabled in the build*.
- **Windows**
- **Windows**
- Windows 7 or newer (and equivalent server versions) are supported. This *may* work on Vista but you're on your own there. Windows XP is not supported since it lacks many important network API functions.
- Visual Studio 2022 on Windows 10 or newer.
- We build with Visual Studio 2015. Older versions may not work with the solution file and project files we ship and may not have new enough C++11 support.
- Rust for x86_64 and ARM64 targets *if SSO is enabled in the build*.
- Pre-built signed Windows drivers are included in `ext/bin/tap-windows-ndis6`. The MSI files found there will install them on 32-bit and 64-bit systems. (These are included in our multi-architecture installer as chained MSIs.)
- Windows builds are more painful in general than other platforms and are for the adventurous.
- **FreeBSD**
- **FreeBSD**
- Tested most recently on FreeBSD-11. Older versions may work but we're not sure.
- GNU make is required. Type `gmake` to build.
- GCC/G++ 4.9 and gmake are required. These can be installed from packages or ports. Type `gmake` to build.
- `binutils` is required. Type `pkg install binutils` to install.
- Rust for x86_64 and ARM64 targets *if SSO is enabled in the build*.
- **OpenBSD**
- **OpenBSD**
- There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`). We're not sure if this can be increased.
- There is a limit of four network memberships on OpenBSD as there are only four tap devices (`/dev/tap0` through `/dev/tap3`).
- OpenBSD lacks `getifmaddrs` (or any equivalent method) to get interface multicast memberships. As a result multicast will only work on OpenBSD for ARP and NDP (IP/MAC lookup) and not for other purposes.
- GNU make is required. Type `gmake` to build.
- Only tested on OpenBSD 6.0. Older versions may not work.
- Rust for x86_64 and ARM64 targets *if SSO is enabled in the build*.
- GCC/G++ 4.9 and gmake are required and can be installed using `pkg_add` or from ports. They get installed in `/usr/local/bin` as `egcc` and `eg++` and our makefile is pre-configured to use them on OpenBSD.
Typing `make selftest` will build a *zerotier-selftest* binary which unit tests various internals and reports on a few aspects of the build environment. It's a good idea to try this on novel platforms or architectures.
Typing `make selftest` will build a *zerotier-selftest* binary which unit tests various internals and reports on a few aspects of the build environment. It's a good idea to try this on novel platforms or architectures.
### Running
### Running
Running *zerotier-one* with -h will show help.
Running *zerotier-one* with `-h` option will show help.
On Linux and BSD you can start the service with:
On Linux and BSD, if you built from source, you can start the service with:
sudo ./zerotier-one -d
sudo ./zerotier-one -d
On most distributions, macOS, and Windows, the installer will start the service and set it up to start on boot.
A home folder for your system will automatically be created.
A home folder for your system will automatically be created.
The service is controlled via the JSON API, which by default is available at 127.0.0.1 port 9993. We include a *zerotier-cli* command line utility to make API calls for standard things like joining and leaving networks. The *authtoken.secret* file in the home folder contains the secret token for accessing this API. See README.md in [service/](service/) for API documentation.
The service is controlled via the JSON API, which by default is available at `127.0.0.1:9993`. It also listens on `0.0.0.0:9993` which is only usable if `allowManagementFrom` is properly configured in `local.conf`. We include a *zerotier-cli* command line utility to make API calls for standard things like joining and leaving networks. The *authtoken.secret* file in the home folder contains the secret token for accessing this API. See [service/README.md](service/README.md) for API documentation.
Here's where home folders live (by default) on each OS:
Here's where home folders live (by default) on each OS:
* **Windows**: `\ProgramData\ZeroTier\One` (That's for Windows 7. The base 'shared app data' folder might be different on different Windows versions.)
* **Windows**: `\ProgramData\ZeroTier\One` (That's the default. The base 'shared app data' folder might be different if Windows is installed with a non-standard drive letter assignment or layout.)
Running ZeroTier One on a Mac is the same, but OSX requires a kernel extension. We ship a signed binary build of the ZeroTier tap device driver, which can be installed on Mac with:
### Basic Troubleshooting
sudo make install-mac-tap
This will create the home folder for Mac, place *tap.kext* there, and set its modes correctly to enable ZeroTier One to manage it with *kextload* and *kextunload*.
### Troubleshooting
For most users, it just works.
For most users, it just works.
If you are running a local system firewall, we recommend adding a rule permitting UDP port 9993 inbound and outbound. If you installed binaries for Windows this should be done automatically. Other platforms might require manual editing of local firewall rules depending on your configuration.
If you are running a local system firewall, we recommend adding a rules permitting zerotier. If you installed binaries for Windows this should be done automatically. Other platforms might require manual editing of local firewall rules depending on your configuration.
The Mac firewall can be found under "Security" in System Preferences. Linux has a variety of firewall configuration systems and tools. If you're using Ubuntu's *ufw*, you can do this:
See the [documentation site](https://docs.zerotier.com/zerotier/troubleshooting) for more information.
sudo ufw allow 9993/udp
The Mac firewall can be found under "Security" in System Preferences. Linux has a variety of firewall configuration systems and tools.
On CentOS check `/etc/sysconfig/iptables` for IPTables rules. For other distributions consult your distribution's documentation. You'll also have to check the UIs or documentation for commercial third party firewall applications like Little Snitch (Mac), McAfee Firewall Enterprise (Windows), etc. if you are running any of those. Some corporate environments might have centrally managed firewall software, so you might also have to contact IT.
On CentOS check `/etc/sysconfig/iptables` for IPTables rules. For other distributions consult your distribution's documentation. You'll also have to check the UIs or documentation for commercial third party firewall applications like Little Snitch (Mac), McAfee Firewall Enterprise (Windows), etc. if you are running any of those. Some corporate environments might have centrally managed firewall software, so you might also have to contact IT.
ZeroTier One peers will automatically locate each other and communicate directly over a local wired LAN *if UDP port 9993 inbound is open*. If that port is filtered, they won't be able to see each others' LAN announcement packets. If you're experiencing poor performance between devices on the same physical network, check their firewall settings. Without LAN auto-location peers must attempt "loopback" NAT traversal, which sometimes fails and in any case requires that every packet traverse your external router twice.
ZeroTier One peers will automatically locate each other and communicate directly over a local wired LAN *if UDP port 9993 inbound is open*. If that port is filtered, they won't be able to see each others' LAN announcement packets. If you're experiencing poor performance between devices on the same physical network, check their firewall settings. Without LAN auto-location peers must attempt "loopback" NAT traversal, which sometimes fails and in any case requires that every packet traverse your external router twice.
Users behind certain types of firewalls and "symmetric" NAT devices may not able able to connect to external peers directly at all. ZeroTier has limited support for port prediction and will *attempt* to traverse symmetric NATs, but this doesn't always work. If P2P connectivity fails you'll be bouncing UDP packets off our relay servers resulting in slower performance. Some NAT router(s) have a configurable NAT mode, and setting this to "full cone" will eliminate this problem. If you do this you may also see a magical improvement for things like VoIP phones, Skype, BitTorrent, WebRTC, certain games, etc., since all of these use NAT traversal techniques similar to ours.
Users behind certain types of firewalls and "symmetric" NAT devices may not be able to connect to external peers directly at all. ZeroTier has limited support for port prediction and will *attempt* to traverse symmetric NATs, but this doesn't always work. If P2P connectivity fails you'll be bouncing UDP packets off our relay servers resulting in slower performance. Some NAT router(s) have a configurable NAT mode, and setting this to "full cone" will eliminate this problem. If you do this you may also see a magical improvement for things like VoIP phones, Skype, BitTorrent, WebRTC, certain games, etc., since all of these use NAT traversal techniques similar to ours.
If you're interested, there's a [technical deep dive about NAT traversal on our blog](https://www.zerotier.com/blog/?p=226). A troubleshooting tool to help you diagnose NAT issues is planned for the future as are uPnP/IGD/NAT-PMP and IPv6 transport.
If a firewall between you and the Internet blocks ZeroTier's UDP traffic, you will fall back to last-resort TCP tunneling to rootservers over port 443 (https impersonation). This will work almost anywhere but is *very slow* compared to UDP or direct peer to peer connectivity.
If a firewall between you and the Internet blocks ZeroTier's UDP traffic, you will fall back to last-resort TCP tunneling to rootservers over port 443 (https impersonation). This will work almost anywhere but is *very slow* compared to UDP or direct peer to peer connectivity.
### Contributing
Additional help can be found in our [knowledge base](https://zerotier.atlassian.net/wiki/spaces/SD/overview).
Please make pull requests against the `dev` branch. The `master` branch is release, and `edge` is for unstable and work in progress changes and is not likely to work.
### Prometheus Metrics
### License
Prometheus Metrics are available at the `/metrics` API endpoint. This endpoint is protected by an API key stored in `metricstoken.secret` to prevent unwanted information leakage. Information that could be gleaned from the metrics include joined networks and peers your instance is talking to.
The ZeroTier source code is open source and is licensed under the GNU GPL v3 (not LGPL). If you'd like to embed it in a closed-source commercial product or appliance, please e-mail [contact@zerotier.com](mailto:contact@zerotier.com) to discuss commercial licensing. Otherwise it can be used for free.
Access control is via the ZeroTier control interface itself and `metricstoken.secret`. This can be sent as a bearer auth token, via the `X-ZT1-Auth` HTTP header field, or appended to the URL as `?auth=<token>`. You can see the current metrics via `cURL` with the following command:
To configure a scrape job in Prometheus on the machine ZeroTier is running on, add this to your Prometheus `scrape_config`:
- job_name: zerotier-one
honor_labels: true
scrape_interval: 15s
metrics_path: /metrics
static_configs:
- targets:
- 127.0.0.1:9993
labels:
group: zerotier-one
node_id: $YOUR_10_CHARACTER_NODE_ID
authorization:
credentials: $YOUR_METRICS_TOKEN_SECRET
If neither of these methods are desirable, it is probably possible to distribute metrics via [Prometheus Proxy](https://github.com/pambrose/prometheus-proxy) or some other tool. Note: We have not tested this internally, but will probably work with the correct configuration.
Metrics are also available on disk in ZeroTier's working directory:
Version 1.2.2 fixes a few bugs discovered after the 1.2.0 release. These are:
* Fix for missing entitlement on macOS Sequoia.
* Fix for a problem correctly parsing local.conf to enable low bandwidth mode.
* Increment versions of some dependent libraries.
* Other fixes.
# 2024-09-12 -- Version 1.14.1
* Multithreaded packet I/O support! Currently this is just for Linux and must
be enabled in local.conf. It will likely make the largest difference on small
multi-core devices where CPU is a bottleneck and high throughput is desired.
It may be enabled by default in the future but we want it to be thoroughly
tested. It's a little harder than it seems at first glance due to the need
to keep packets in sequence and balance load.
* Several multipath bug fixes.
* Updated the versions on a number of libraries related to OIDC support and HTTP.
* MacOS .app now shows the correct version in its Info.plist manifest.
* Sanitize MAC addresses in JSON format rules parser.
* Some basic information about the platform (OS, CPU architecture) is now reported
to network controllers when networks are joined so it can be displayed to
network admins and in the future used in policy checking and inventory operations.
# 2024-05-02 -- Version 1.14.0
* Linux I/O performance improvements under heavy load
* Improvements to multipath
* Fix for port rebinding "coma" bug after periods offline (some laptop users)
* Fixed a rules engine quirk/ambiguity (GitHub Issue #2200)
* Controller API enhancements: node names and other node meta-data
* Other bug fixes
# 2023-09-12 -- Version 1.12.2
* More improvements to macOS full tunnel mode.
* Faster recovery after changes to physical network settings.
# 2023-08-25 -- Version 1.12.1
* Minor release to fix a port binding issue in Linux.
* Update Debian dependencies.
* No changes for other platforms.
# 2023-08-23 -- Version 1.12.0
* Experimental Windows ARM64 support
* Fix numerous sleep/wake issues on macOS and other platforms
* Faster recovery after changes to physical network settings
* Prometheus compatible metrics support!
* Fix full tunnel mode on recent macOS versions
* Numerous macOS DNS fixes
* 10-30% speed improvement on Linux
# 2023-03-23 -- Version 1.10.6
* Prevent binding temporary ipv6 addresses on macos (#1910)
* Prevent path-learning loops (#1914)
* Prevent infinite loop of UAC prompts in tray app
# 2023-03-10 -- Version 1.10.5
* Fix for high CPU usage bug on Windows
# 2023-03-07 -- Version 1.10.4
* SECURITY FIX (Windows): this version fixes a file permission problem on
Windows that could allow non-privileged users on a Windows system to read
privileged files in the ZeroTier service's working directory. This could
allow an unprivileged local Windows user to administrate the local ZeroTier
instance without appropriate local permissions. This issue is not remotely
exploitable unless a remote user can read arbitrary local files, and does
not impact other operating systems.
* Fix a bug in the handling of multiple IP address assignments to virtual
interfaces on macOS.
# 2023-02-15 -- Version 1.10.3
* Fix for duplicate paths in client. Could cause connectivity issues. Affects all platforms.
* Fix for Ethernet Tap MTU setting, would not properly apply on Linux.
* Fix default route bugs (macOS.)
* Enable Ping automatically for ZeroTier Adapters (Windows.)
* SSO updates and minor bugfixes.
* Add low-bandwidth mode.
* Add forceTcpRelay mode (optionally enabled.)
* Fix bug that prevented setting of custom TCP relay address.
* Build script improvements and bug fixes.
# 2022-11-01 -- Version 1.10.2
* Fix another SSO "stuck client" issue in zeroidc.
* Expose root-reported external IP/port information via the local JSON API for better diagnostics.
* Multipath: CLI output improvement for inspecting bonds
* Multipath: balance-aware mode
* Multipath: Custom policies
* Multipath: Link quality measurement improvements
Note that releases are coming few and far between because most of our dev effort is going into version 2.
# 2022-06-27 -- Version 1.10.1
* Fix an issue that could cause SSO clients to get "stuck" on stale auth URLs.
* A few other SSO related bug fixes.
# 2022-06-07 -- Version 1.10.0
* Fix formatting problem in `zerotier-cli` when using SSO networks.
* Fix a few other minor bugs in SSO signin to prepare for general availability.
* Remove requirement for webview in desktop UI and instead just make everything available via the tray pulldown/menu. Use [libui-ng](https://github.com/libui-ng/libui-ng) for minor prompt dialogs. Saves space and eliminates installation headaches on Windows.
* Fix SSO "spam" bug in desktop UI.
* Use system default browser for SSO login so all your plugins, MFA devices, password managers, etc. will work as you have them configured.
* Minor fix for bonding/multipath.
# 2022-05-10 -- Version 1.8.10
* Fixed a bug preventing SSO sign-on on Windows.
# 2022-04-25 -- Version 1.8.9
* Fixed a long-standing and strange bug that was causing sporadic "phantom" packet authentication failures. Not a security problem but could be behind sporadic reports of link failures under some conditions.
* Fixed a memory leak in SSO/OIDC support.
* Fixed SSO/OIDC display error on CLI.
* Fixed a bug causing nodes to sometimes fail to push certs to each other (primarily affects SSO/OIDC use cases).
* Fixed a deadlock bug on leaving SSO/OIDC managed networks.
* Added some new Linux distributions to the build subsystem.
# 2022-04-11 -- Version 1.8.8
* Fix a local privilege escalation bug in the Windows installer.
* Dependency fix for some Ubuntu versions.
* No changes for other platforms. Windows upgrade recommended, everyone else optional.
# 2022-03-30 -- Version 1.8.7
* Fix for dependency installations in Windows MSI package.
* Fix for desktop UI setup when run by a non-super-user.
* Bug fix in local OIDC / SSO support for auth0 and other providers.
* Other minor fixes for e.g. old Linux distributions.
# 2022-03-04 -- Version 1.8.6
* Fixed an issue that could cause the UI to be non-responsive if not joined to any networks.
* Fix dependency issues in Debian and RedHat packages for some distributions (Fedora, Mint).
* Bumped the peer cache serialization version to prevent "coma" issues on upgrade due to changes in path logic behaving badly with old values.
# 2022-02-22 -- Version 1.8.5
* Plumbing under the hood for endpoint device SSO support.
* Fix in LinuxEthernetTap to tap device support on very old (2.6) Linux kernels.
* Fix an issue that could cause self-hosted roots ("moons") to fail to assist peers in making direct links. (GitHub issue #1512)
* Merge a series of changes by Joseph Henry (of ZeroTier) that should fix some edge cases where ZeroTier would "forget" valid paths.
* Minor multipath improvements for automatic path negotiation.
# 2021-11-30 -- Version 1.8.4
* Fixed an ugly font problem on some older macOS versions.
* Fixed a bug that could cause the desktop tray app control panel to stop opening after a while on Windows.
* Fixed a possible double "release" in macOS tray app code that crashed on older macOS versions.
* Fixed installation on 32-bit Windows 10.
* Fixed a build flags issue that could cause ZeroTier to crash on older ARM32 CPUs.
# 2021-11-15 -- Version 1.8.3
* Remove problematic spinlock, which was only used on x86_64 anyway. Just use pthread always.
* Fix fd leak on MacOS that caused non-responsiveness after some time.
* Fix Debian install scripts to set /usr/sbin/nologin as shell on service user.
* Fix regression that could prevent managed routes from being deleted.
* DesktopUI: Remove NSDate:now() call, now works on MacOS 10.13 or newer!
# 2021-11-08 -- Version 1.8.2
* Fix multicast on linux.
* Fix a bug that could cause the tap adapter to have the wrong MAC on Linux.
* Update build flags to possibly support MacOS older than 10.14, but more work needs to be done. It may not work yet.
* Fix path variable setting on Windows.
# 2021-10-28 -- Version 1.8.1
* Fix numerous UI issues from 1.8.0 (never fully released).
* Remove support for REALLY ancient 1.1.6 or earlier network controllers.
* MacOS IPv6 no longer binds to temporary addresses as these can cause interruptions if they expire.
* Added additional hardening against address impersonation on networks (also in 1.6.6).
* Fix an issue that could cause clobbering of MacOS IP route settings on restart.
* NOTE: Windows 7 is no longer supported! Windows 7 users will have to use version 1.6.5 or earlier.
# 2021-09-15 -- Version 1.8.0 (preview release only)
* A *completely* rewritten desktop UI for Mac and Windows!
* Implement a workaround for one potential source of a "coma" bug, which can occur if buggy NATs/routers stop allowing the service to communicate on a given port. ZeroTier now reassigns a new secondary port if it's offline for a while unless a secondary port is manually specified in local.conf. Working around crummy buggy routers is an ongoing effort.
* Fix for MacOS MTU capping issue on feth devices
* Fix for mistakenly using v6 source addresses for v4 routes on some platforms
* Stop binding to temporary IPv6 addresses
* Set MAC address before bringing up Linux TAP link
* Check if DNS servers need to be applied on macOS
* Upgrade json.hpp dependency to version 3.10.2
# 2021-09-21 -- Version 1.6.6
* Backport COM hash check mitigation against network member impersonation.
# 2021-04-13 -- Version 1.6.5
* Fix a bug in potential network path filtering that could in some circumstances lead to "software laser" effects.
* Fix a printf overflow in zerotier-cli (not exploitable or a security risk)
* Windows now looks up the name of ZeroTier devices instead of relying on them having "ZeroTier" in them.
# 2021-02-15 -- Version 1.6.4
* The groundhog saw his shadow, which meant that the "connection coma" bug still wasn't gone. We think we found it this time.
# 2021-02-02 -- Version 1.6.3
* Likely fix for GitHub issue #1334, an issue that could cause ZeroTier to
go into a "coma" on some networks.
* Also groundhog day
# 2020-11-30 -- Version 1.6.2
* Fix an ARM hardware AES crypto issue (not an exploitable vulnerability).
* Fix a Linux network leave hang due to a mutex deadlock.
# 2020-11-24 -- Version 1.6.1
This release fixes some minor bugs and other issues in 1.6.0.
* Fixed a bug that caused IP addresses in the 203.0.0.0/8 block to be miscategorized as not being in global scope.
* Changed Linux builds to (hopefully) fix LXC and SELinux issues.
* Fixed unaligned memory access that caused crash on FreeBSD systems on the ARM architecture.
* Merged CLI options for controlling bonded devices into the beta multipath code.
* Updated Windows driver with Microsoft cross-signing to fix issues on some Windows systems.
# 2020-11-19 -- Version 1.6.0
Version 1.6.0 is a major release that incorporates back-ported features from the 2.0 branch, which is still under development. It also fixes a number of issues.
New features and improvements (including those listed under 1.5.0):
* **Apple Silicon** (MacOS ARM64) native support via universal binary. ZeroTier now requires the very latest Xcode to build.
* **Linux performance improvements** for up to 25% faster tun/tap I/O performance on multi-core systems.
* **Multipath support** with modes modeled after the Linux kernel's bonding driver. This includes active-passive and active-active modes with fast failover and load balancing. See section 2.1.5 of the manual.
* **DNS configuration** push from network controllers to end nodes, with locally configurable permissions for whether or not push is allowed.
* **AES-GMAC-SIV** encryption mode, which is both somewhat more secure and significantly faster than the old Salsa20/12-Poly1305 mode on hardware that supports AES acceleration. This includes virtually all X86-64 chips and most ARM64. This mode is based on AES-SIV and has been audited by Trail of Bits to ensure that it is equivalent security-wise.
Bug fixes:
* **Managed route assignment fixes** to eliminate missing routes on Linux and what we believe to be the source of sporadic high CPU usage on MacOS.
* **Hang on shutdown** issues should be fixed.
* **Sporadic multicast outages** should be fixed.
Known remaining issues:
* AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
# 2020-10-05 -- Version 1.5.0 (actually 1.6.0-beta1)
Version 1.6.0 (1.5.0 is a beta!) is a significant release that incorporates a number of back-ported fixes and features from the ZeroTier 2.0 tree.
Major new features are:
* **Multipath support** with modes modeled after the Linux kernel's bonding driver. This includes active-passive and active-active modes with fast failover and load balancing. See section 2.1.5 of the manual.
* **DNS configuration** push from network controllers to end nodes, with locally configurable permissions for whether or not push is allowed.
* **AES-GMAC-SIV** encryption mode, which is both somewhat more secure and significantly faster than the old Salsa20/12-Poly1305 mode on hardware that supports AES acceleration. This includes virtually all X86-64 chips and most ARM64. This mode is based on AES-SIV and has been audited by Trail of Bits to ensure that it is equivalent security-wise.
Known issues that are not yet fixed in this beta:
* Some Mac users have reported periods of 100% CPU in kernel_task and connection instability after leaving networks that have been joined for a period of time, or needing to kill ZeroTier and restart it to finish leaving a network. This doesn't appear to affect all users and we haven't diagnosed the root cause yet.
* The service sometimes hangs on shutdown requiring a kill -9. This also does not affect all systems or users.
* AES hardware acceleration is not yet supported on 32-bit ARM, PowerPC (32 or 64), or MIPS (32 or 64) systems. Currently supported are X86-64 and ARM64/AARCH64 with crypto extensions.
* Some users have reported multicast/broadcast outages on networks lasting up to 30 seconds. Still investigating.
We're trying to fix all these issues before the 1.6.0 release. Stay tuned.
# 2019-08-30 -- Version 1.4.6
* Update default root list to latest
* ARM32 platform build and flag fixes
* Add a clarification line to LICENSE.txt
* Fix license message in CLI
* Windows service now looks for service command line arguments
* Fixed a bug that could cause excessive queued multicasts
# 2019-08-23 -- Version 1.4.4
* Change license from GPL3 to BSL 1.1, see LICENSE.txt
* Fix an issue with the "ipauth" rule and auto-generated unforgeable IPv6 addresses
* Fix socket/bind errors setting IPs and routes on Linux
# 2019-08-12 -- Version 1.4.2
* Fix high CPU use bug on some platforms
* Fix issues with PostgreSQL controller DB (only affects Central)
* Restore backward compatibility with MacOS versions prior to 10.13
# 2019-07-29 -- Version 1.4.0
### Major Changes
* Mac version no longer requires a kernel extension, instead making use of the [feth interfaces](https://apple.stackexchange.com/questions/337715/fake-ethernet-interfaces-feth-if-fake-anyone-ever-seen-this).
* Added support for concurrent multipath (multiple paths at once) with traffic weighting by link quality and faster recovery from lost links.
* Added under-the-hood support for QoS (not yet exposed) that will eventually be configurable via our rules engine.
### Minor Changes and Bug Fixes
* Experimental controller DB driver for [LF](https://github.com/zerotier/lf) to store network controller data (LFDB.cpp / LFDB.hpp).
* Modified credential push and direct path push timings and algorithms to somewhat reduce "chattiness" of the protocol when idle. More radical background overhead reductions will have to wait for the 2.x line.
* Removed our beta/half-baked integration of Central with the Windows UI. We're going to do a whole new UI of some kind in the future at least for Windows and Mac.
* Fixed stack overflow issues on Linux versions using musl libc.
* Fixed some alignment problems reported on ARM and ARM64, but some reports we could not reproduce so please report any issues with exact chip, OS/distro, and ZeroTier version in use.
* Fixed numerous other small issues and bugs such as ARM alignment issues causing crashes on some devices.
* Windows now sets the adapter name such that it is consistent in both the Windows UI and command line utilities.
# 2018-07-27 -- Version 1.2.12
* Fixed a bug that caused exits to take a long time on Mac due to huge numbers of redundant attempts to delete managed routes.
* Fixed a socket limit problem on Windows that caused the ZeroTier service to run out of sockets, causing the UI and CLI to be unable to access the API.
* Fixed a threading bug in the ZeroTier Core, albeit one that never manifested on the regular ZeroTier One service/client.
* Fixed a bug that could cause the service to crash if an authorized local client accessed an invalid URL via the control API. (Not exploitable since you needed admin access anyway.)
# 2018-05-08 -- Version 1.2.10
* Fix bug loading `moons.d/` files for federated root operation.
* Fix compile problem with ZT_DEBUG on some versions of `clang`
* Fix slow network startup bug related to loading of `networks.d/` cache files
# 2018-04-27 -- Version 1.2.8
* Linux version once again builds with PIE (position independent executable) flags
* Fixed bug in zerotier-idtool file sign and verify
* Fixed minor OSX app typo
* Merged alpha NetBSD support (mostly untested, so YMMV)
* Merged several minor typo and one-liner bug fixes
# 2018-04-17 -- Version 1.2.6
* Features and Core Improvements
* Path selection has been overhauled to improve path stability, simplify code, and prepare for multi-path and trunking in the next major release.
* This version introduces remote tracing for remote diagnostics. Network controllers can set a node (usually the controller itself) to receive remote tracing events from all members of the network or from select members. Events are only sent if they pertain to a given network for security reasons.
* Multicast replication can now be done by designated multicast replicators on a network (flagged as such at the controller) rather than by the sender. Most users won't want this, but it's useful for specialized use cases on hub-and-spoke networks and for low-power devices.
* Cryptographic performance improvements on several platforms.
* Multithreaded performance improvements throughout the code base, including the use of an inline lightweight spinlock for low-contention resources.
* Bugs fixed
* Disappearing routes on Mac (GitHub issue #600)
* Route flapping and path instability in some dual-stack V4/V6 networks
* Blacklist (in local.conf) doesn't work reliably (GitHub issue #656)
* Connection instabilities due to unsigned integer overflows in timing comparisons (use int64_t instead of uint64_t)
* Binaries don't run on some older or lower-end 32-bit ARM chips (build problem)
* ARM NEON crypto code crashes (build problem)
* Fixed some lock ordering issues revealed by "valgrind" tool
* The "zerotier-idtool" command could not be accessed from "zerotier-one" via command line switch
* Leaking sockets on some platforms when uPnP/NAT-PMP is enabled
* Fixed two very rare multithreading issues that were only observed on certain systems
* Platform-Specific Changes
* MacOS
* Installer now loads the kernel extension right away so that High Sierra users will see the prompt to authorize it. This is done in the "Security & Privacy" preference pane and must be done directly on the console (not via remote desktop). On High Sierra and newer kexts must be authorized at the console via security settings system preferences pane.
* Windows
* The Windows installer should now install the driver without requiring a special prompt in most cases. This should make it easier for our packages to be accepted into and updated in the Chocolatey repository and should make it easier to perform remote installs across groups of machines using IT management and provisioning tools.
* The Windows official packages are now signed with an EV certificate (with hardware key).
* The Windows UI can now log into ZeroTier Central and join networks via the Central API.
* The `zerotier-idtool` command should now work on Windows without ugly hacks.
* Upgraded the installer version.
* Made a few changes to hopefully fix sporadic "will not uninstall" problems, though we cannot duplicate these issues ourselves.
* Linux
* Device names are now generated deterministically based on network IDs for all newly joined networks.
* Android
* Multicast now works on Android in most cases! Android apps can send and receive multicast and subscribe to multicast group IPs. Note that in some cases the app must bind to the specific correct interface for this to work.
* IPv6 can be disabled in UI for cases where it causes problems.
# 2017-04-20 -- Version 1.2.4
* Managed routes are now only bifurcated for the default route. This is a change in behavior, though few people will probably notice. Bifurcating all managed routes was causing more trouble than it was worth for most users.
* Up to 2X crypto speedup on x86-64 (except Windows, which will take some porting) and 32-bit ARM platforms due to integration of fast assembly language implementations of Salsa20/12 from the [supercop](http://bench.cr.yp.to/supercop.html) code base. These were written by Daniel J. Bernstein and are in the public domain. My MacBook Pro (Core i5 2.8ghz) now does almost 1.5GiB/sec Salsa20/12 per core and a Raspberry Pi got a 2X boost. 64-bit ARM support and Windows support will take some work but should not be too hard.
* Refactored code that manages credentials to greatly reduce memory use in most cases. This may also result in a small performance improvement.
* Reworked and simplified path selection and priority logic to fix path instability and dead path persistence edge cases. There have been some sporadic reports of persistent path instabilities and dead paths hanging around that take minutes to resolve. These have proven difficult to reproduce in house, but hopefully this will fix them. In any case it seems to speed up path establishment in our tests and it makes the code simpler and more readable.
* Eliminated some unused cruft from the code around path management and in the peer class.
* Fixed an issue causing build problems on some MIPS architecture systems.
* Fixed Windows forgetting routes on sleep/wake or in some other circumstances. (GitHub issue #465)
# 2017-03-17 -- Version 1.2.2
* A bug causing unreliable multicast propagation (GitHub issue #461).
* A bug causing unreliable multicast propagation (GitHub issue #461).
* A crash in ARM binaries due to a build chain and flags problem.
* A crash in ARM binaries due to a build chain and flags problem.
* A bug in the network controller preventing members from being listed (GitHub issue #460).
* A bug in the network controller preventing members from being listed (GitHub issue #460).
------
# 2017-03-14 -- Version 1.2.0
# 2017-03-14 -- Version 1.2.0
Version 1.2.0 is a major milestone release representing almost nine months of work. It includes our rules engine for distributed network packet filtering and security monitoring, federated roots, and many other architectural and UI improvements and bug fixes.
Version 1.2.0 is a major milestone release representing almost nine months of work. It includes our rules engine for distributed network packet filtering and security monitoring, federated roots, and many other architectural and UI improvements and bug fixes.
@ -23,7 +397,7 @@ The largest new feature in 1.2.0, and the product of many months of work, is our
Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
Rules allow you to filter packets on your network and vector traffic to security observers. Security observation can be performed in-band using REDIRECT or out of band using TEE.
Tags and capabilites provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
Tags and capabilities provide advanced methods for implementing fine grained permission structures and micro-segmentation schemes without bloating the size and complexity of your rules table.
See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
See the [rules engine announcement blog post](https://www.zerotier.com/blog/?p=927) for an in-depth discussion of theory and implementation. The [manual](https://www.zerotier.com/manual.shtml) contains detailed information on rule, tag, and capability use, and the `rule-compiler/` subfolder of the ZeroTier source tree contains a JavaScript function to compile rules in our human-readable rule definition language into rules suitable for import into a network controller. (ZeroTier Central uses this same script to compile rules on [my.zerotier.com](https://my.zerotier.com/).)
@ -106,7 +480,7 @@ A special kind of public network called an ad-hoc network may be accessed by joi
| Start of port range (hex)
| Start of port range (hex)
Reserved ZeroTier address prefix indicating a controller-less network
Reserved ZeroTier address prefix indicating a controller-less network
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to desintation ports within the encoded range.
Ad-hoc networks are public (no access control) networks that have no network controller. Instead their configuration and other credentials are generated locally. Ad-hoc networks permit only IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using 6plane format NDP-emulated IPv6 addresses. In addition an ad-hoc network ID encodes an IP port range. UDP packets and TCP SYN (connection open) packets are only allowed to destination ports within the encoded range.
For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
For example `ff00160016000000` is an ad-hoc network allowing only SSH, while `ff0000ffff000000` is an ad-hoc network allowing any UDP or TCP port.
@ -121,7 +495,7 @@ If you have data in an old SQLite3 controller we've included a NodeJS script in
## Major Bug Fixes in 1.2.0
## Major Bug Fixes in 1.2.0
* **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
* **The Windows HyperV 100% CPU bug is FINALLY DEAD**: This long-running problem turns out to have been an issue with Windows itself, but one we were triggering by placing invalid data into the Windows registry. Microsoft is aware of the issue but we've also fixed the triggering problem on our side. ZeroTier should now co-exist quite well with HyperV and should now be able to be bridged with a HyperV virtual switch.
* **Segmenation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Segmentation faults on musl-libc based Linux systems**: Alpine Linux and some embedded Linux systems that use musl libc (a minimal libc) experienced segmentation faults. These were due to a smaller default stack size. A work-around that sets the stack size for new threads has been added.
* **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
* **Windows firewall blocks local JSON API**: On some Windows systems the firewall likes to block 127.0.0.1:9993 for mysterious reasons. This is now fixed in the installer via the addition of another firewall exemption rule.
* **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
* **UI crash on embedded Windows due to missing fonts**: The MSI installer now ships fonts and will install them if they are not present, so this should be fixed.
// Start at utun8 to leave lower utuns unused since other stuff might
// want them -- OpenVPN, cjdns, etc. I'm not sure if those are smart
// enough to scan upward like this.
for(utunNo=8;utunNo<=256;++utunNo) {
if ((_fd = _make_utun(utunNo)) > 0)
break;
}
}
if (_fd <= 0)
throw std::runtime_error("unable to find/load ZeroTier tap driver OR use built-in utun driver in OSX; permission or system problem or too many open devices?");
// designed telephone using water microphone in 1876
"gray",
// Tivadar Puskás invented the telephone switchboard exchange in 1876.
"puskas",
// Thomas Edison, invented the carbon microphone which produced a strong telephone signal.
"edison",
// 1950s, Paul Baran developed the concept Distributed Adaptive Message Block Switching
"baran",
// Donald Davies coined the phrase 'packet switching network'
"davies",
// Robert Licklider helped get ARPANET funded
"licklider",
// Robert Taylor, ARPANET pioneer
"taylor",
// Lawrence Roberts, ARPANET
"roberts",
// Vint Cerf, TCP
"cerf",
// Bob Kahn, TCP
"kahn",
// David P Reed, UDP
"reed",
// Community Memory was created by Efrem Lipkin, Mark Szpakowski, and Lee Felsenstein, acting as The Community Memory Project within the Resource One computer center at Project One in San Francisco.
"lipkin",
"szpakowski",
"felsenstein",
// The first public dial-up BBS was developed by Ward Christensen and Randy Suess.
"christensen",
"suess",
// Joybubbles (May 25, 1949 – August 8, 2007), born Josef Carl Engressia, Jr. in Richmond, Virginia, USA, was an early phone phreak.
"engressia",
"joybubbles",
// John Thomas Draper (born 1943), also known as Captain Crunch, Crunch or Crunchman (after Cap'n Crunch breakfast cereal mascot), is an American computer programmer and former phone phreak
"draper",
// Dennis C. Hayes, founder of Hayes Microcomputer Products
// "The Modem of Dennis Hayes and Dale Heatherington."
"hayes",
"heatherington",
// "Ethernet was developed at Xerox PARC between 1973 and 1974.[7][8] It was inspired by ALOHAnet, which Robert Metcalfe had studied as part of his PhD dissertation."
"metcalfe",
// William Bradford Shockley Jr. (February 13, 1910 – August 12, 1989) was an American physicist and inventor. Shockley was the manager of a research group that included John Bardeen and Walter Brattain. The three scientists invented the point contact transistor in 1947
"shockley",
"bardeen",
"brattain",
// "Randall Erck invented the modern modem as we know it today. There were devices similar to modems used by the military, but they were designed more for the purpose of sending encripted nuclear launch codes to various bases around the world."
"erck",
// Leonard Kleinrock, packet switching network pioneer
"kleinrock",
// Tim Berners-Lee, WWW
"berners_lee",
// Steve Wozniak, early phone phreak
"wozniak",
// James Fields Smathers of Kansas City invented what is considered the first practical power-operated typewriter in 1914.
"smathers",
// The teleprinter evolved through a series of inventions by a number of engineers, including Royal Earl House, David Edward Hughes, Emile Baudot, Donald Murray, Charles L. Krum, Edward Kleinschmidt and Frederick G. Creed.
"house",
"hughes",
"baudot",
"murray",
"krum",
"kleinschmidt",
"creed",
// Ron Rosenbaum, author of "Secrets of the Little Blue Box" which mainstreamed phone phreaking
"rosenbaum",
// Bram Cohen. Bram Cohen (born October 12, 1975) is an American computer programmer, best known as the author of the peer-to-peer (P2P) BitTorrent protocol,
"cohen",
// Jarkko Oikarinen (born 16 August 1967, in Kuusamo, Finland) is the inventor of the first Internet chat network, called Internet Relay Chat (IRC), where he is known as WiZ.
"oikarinen",
// "What you probably didn't know is that the author of Trumpet Winsock — Peter Tattam from Tasmania, Australia — didn't see much money for his efforts."
"tattam",
// Satoshi Nakamoto
"nakamoto",
// Philo Farnsworth, inventor of the first practical TV tube
"farnsworth",
// Scottish inventor John Logie Baird employed the Nipkow disk in his prototype video systems. On 25 March 1925, Baird gave the first public demonstration of televised silhouette images in motion, at Selfridge's Department Store in London.
"baird",
// Beginning in 1836, the American artist Samuel F. B. Morse, the American physicist Joseph Henry, and Alfred Vail developed an electrical telegraph system.
*This is really internal use code. You're free to test it out but expect to do some editing/tweaking to make it work. We used this to run some massive scale tests of our new geo-cluster-based root server infrastructure prior to taking it live.*
Before using this code you will want to edit agent.js to change SERVER_HOST to the IP address of where you will run server.js. This should typically be an open Internet IP, since this makes reporting not dependent upon the thing being tested. Also note that this thing does no security of any kind. It's designed for one-off tests run over a short period of time, not to be anything that runs permanently. You will also want to edit the Dockerfile if you want to build containers and change the network ID to the network you want to run tests over.
This code can be deployed across a large number of VMs or containers to test and benchmark HTTP traffic within a virtual network at scale. The agent acts as a server and can query other agents, while the server collects agent data and tells agents about each other. It's designed to use RFC4193-based ZeroTier IPv6 addresses within the cluster, which allows the easy provisioning of a large cluster without IP conflicts.
The Dockerfile builds an image that launches the agent. The image must be "docker run" with "--device=/dev/net/tun --privileged" to permit it to open a tun/tap device within the container. (Unfortunately CAP_NET_ADMIN may not work due to a bug in Docker and/or Linux.) You can run a bunch with a command like:
for ((n=0;n<10;n++));dodockerrun--device=/dev/net/tun--privileged-dzerotier/http-test;done