Rusty stuff and a README for AES-GMAC-SIV.

aes-gmac-siv/README.md

@@ -0,0 +1,111 @@
+AES-GMAC-SIV
+======
+
+Rust implementation for ZeroTier 2.0.
+
+*WARNING: while this construction has been subjected to peer review, this code has not (yet!). Use at your own risk.*
+
+## Introduction
+
+AES-GMAC-SIV is a "synthetic IV" (SIV) cipher construction implemented using only FIPS140 and NIST accepted cryptographic building blocks: AES-ECB (single block), AES-CTR, and GMAC (the MAC part of GCM, which can also be used separately).
+
+AES-GMAC-SIV is almost identical to [AES-GCM-SIV](https://en.wikipedia.org/wiki/AES-GCM-SIV), but that mode uses a non-standard MAC called POLYVAL in place of GMAC. POLYVAL is basically little-endian GMAC but the fact that it is not standard GMAC means it's not found in most cryptographic libraries and is not approved by FIPS140 and many other sets of compliance guidelines.
+
+## Why SIV? Why not just GCM?
+
+Stream ciphers like AES-CTR, ChaCha20, and others require a number called an initialization vector (IV) for each use. The IV is sometimes called a nonce, or *number used once*, because using the same value for different messages with the same key is a major no-no.
+
+Repeating an IV/nonce with the same key allows both messages to be decrypted. This is because XOR, which is used to apply the stream cipher's pseudorandom bits as a one time pad, is commutative. Repeating a nonce can in some cases also allow an attacker to attack the MAC (e.g. GMAC or Poly1305) and forge messages that will appear valid. In many systems compromising the MAC is more serious than compromising encryption for a few messages.
+
+SIV modes provide strong protection against IV reuse by generating a *synthetic IV* from the plaintext. This means that two different plaintexts will almost certainly perform encryption using different IVs even if the input IV is duplicated.
+
+With SIV a duplicate IV has no effect at all except in the case where the same IV is used to encrypt the same message twice. In this case the encrypted messages would also be identical, revealing that a duplicate was sent, but because both the IV and message are the same this would not compromise security like IV reuse does in standard modes.
+
+SIV modes could be used with no external IV at all without compromising plaintext secrecy, but this is not recommended since it would leak message duplication. An external IV is supplied in this and other common SIV constructions to avoid this, since it causes duplicate messages to result in entirely different ciphertexts.
+
+We recommend treating AES-GMAC-SIV (and other SIV modes) as if they were normal stream ciphers and endeavoring to make the IV unique as those would require.
+
+SIV modes might seem like paranoia, but accidental IV reuse is easier than you might think. Here's a few scenarios where it might happen:
+
+ * Embedded devices that initialize PRNGs from deterministic sources.
+ * Forgetting to use atomics or a mutex to synchronize an IV counter variable in multithreaded code. 
+ * Concurrent use of a non-thread-safe random number generator.
+ * Multiple dependencies in a project initializing or using a random source from a common shared library that uses static state.
+ * Live cloning of virtual machines or application state, resulting in two clones with identical random number generator states.
+ * Memory errors or other bugs that break IV generation. If these can be triggered remotely, this could be an attack vector.
+ * Time changes if the clock is used as an input to generate the IV.
+
+... and so on. "Sudden death" on IV re-use is a foot-gun that's worth removing.
+
+## AES-GMAC-SIV construction
+
+![AES-GMAC-SIV block diagram](AES-GMAC-SIV.png)
+
+Initialization parameters:
+
+ 1. K0, an AES key used to initialize AES-GMAC.
+ 2. K1, a second (and different) AES key used to initialize AES-ECB and AES-CTR.
+
+Per-message parameters:
+
+ 1. A per-message unique 64-bit IV (can be a counter or random).
+ 2. Optional additional associated data (AAD) to authenticate but not encrypt.
+ 3. A plaintext message to encrypt.
+
+Encryption steps:
+
+ 1. Pad 64-bit IV to 96 bits and reset GMAC for next message.
+ 2. Feed AAD (if any) into GMAC.
+ 3. Pad AAD length to a multiple of 16 by feeding zeroes into GMAC.
+ 4. Feed plaintext into GMAC to compute final MAC.
+ 5. XOR lower 64 bits and higher 64 bits of GMAC tag.
+ 6. Concatenate IV and 64-bit shortened tag to form a 128-bit block.
+ 7. AES-ECB encrypt this IV+tag to yield message tag and AES-CTR IV.
+ 8. Clear bit 31 (from the right) in the tag and use this to initialize AES-CTR. Bit 31 is cleared so AES-CTR implementations that use a 32-bit counter will not overflow for messages less than 2^31 bytes in length.
+ 9. Encrypt plaintext with AES-CTR.
+
+The message tag is the 128-bit encrypted block from step 7 before bit 31 is cleared. Only this 128-bit tag needs to be sent with the message. The IV supplied for encryption should not be sent, as it's obtained by decrypting the tag.
+
+Decryption steps:
+
+ 1. Initialize AES-CTR with the tag (with bit 31 cleared as in step 8 above).
+ 2. Decrypt ciphertext with AES-CTR.
+ 3. AES-ECB *decrypt* 128-bit tag to yield original IV and 64-bit shortened GMAC tag.
+ 4. Initialize GMAC as in encryption step 1.
+ 5. Feed AAD into GMAC (if any).
+ 6. Zero-pad AAD to a multiple of 16 as in encryption.
+ 7. Feed *decrypted plaintext* into GMAC.
+ 8. Generate GMAC tag, XOR least and most significant 64 bits, and check MAC. Discard packet if these do not match.
+
+Note that while MAC comparison is only 64 bits, this MAC is concealed within an encrypted block that mixes its bits with the IV. This helps prevent an attacker from directly attacking GMAC or attempting to exploit an oracle. An attacker could only detect a 64-bit MAC collision if the IV was also duplicated, which is another reason to supply unique IVs.
+
+## Performance
+
+Performance is very close to AES-GCM on a given platform. It's very slightly slower because encryption requires two passes, but for short messages the second pass will operate on data already in the CPU's L0 cache which minimizes the actual overhead.
+
+You can run tests with `cargo test -- --nocapture` and see encrypt and decrypt performance. Here's some single core benchmarks:
+
+ * AMD Threadripper 2990WX: **2055.60 MiB/sec**, decrypt **2050.09 MiB/sec**.
+ * M1 Mac Mini (2021): encrypt **4690.89 MiB/sec**, decrypt **4977.39 MiB/sec**.
+
+Since this uses AES it will be much slower on systems that don't have AES hardware extensions. On the flip side it's extremely fast on systems that do, and also more energy efficient than ARX-type ciphers that slam the ALU to achieve high performance.
+
+## Cryptanalysis
+
+[Trail of Bits](https://www.trailofbits.com) analyzed AES-GMAC-SIV as part of their [design analysis of ZeroTier 2.0](https://github.com/trailofbits/publications/blob/master/reviews/ZeroTierProtocol.pdf) and concluded that its security is equivalent to the original AES-SIV and AES-GCM-SIV constructions.
+
+The algorithms on which this is built, namely AES, AES-CTR, and GMAC, are well known and standard.
+
+## FIPS-140 Compliance
+
+AES-ECB, AES-CTR, and GMAC are all algorithms allowed by FIPS-140. For FIPS purposes AES-GMAC-SIV would be described as AES-CTR with GMAC authentication. Since this is built out of compliant components, a standard FIPS-certified cryptographic library could be used.
+
+## Dependencies
+
+This is implemented using the [libgcrypt](https://github.com/gpg/libgcrypt) library (via Rust bindings) on Linux, BSD, and Windows, and built-in CommonCrypto libraries on MacOS and iOS. CommonCrypto was used on Mac because libgcrypt has issues on the ARM64 platform with Apple's clang.
+
+## License
+
+This Rust implementation of AES-GMAC-SIV is released under the BSD 2-clause license.
+
+(c) 2021 ZeroTier, Inc.

aes-gmac-siv/src/impl_gcrypt.rs

@@ -16,10 +16,13 @@ impl AesGmacSiv {
     /// Create a new keyed instance of AES-GMAC-SIV
     /// The key may be of size 16, 24, or 32 bytes (128, 192, or 256 bits). Any other size will panic.
     #[inline(always)]
-    pub fn new(key: &[u8]) -> Self {
+    pub fn new(k0: &[u8], k1: &[u8]) -> Self {
-        if key.len() != 32 && key.len() != 24 && key.len() != 16 {
+        if k0.len() != 32 && k0.len() != 24 && k0.len() != 16 {
             panic!("AES supports 128, 192, or 256 bits keys");
         }
+        if k1.len() != k0.len() {
+            panic!("k0 and k1 must be of the same size");
+        }
         let mut c = AesGmacSiv {
             tag: [0_u8; 16],
             tmp: [0_u8; 16],
@@ -27,9 +30,9 @@ impl AesGmacSiv {
             ecb: gcrypt::cipher::Cipher::new(gcrypt::cipher::Algorithm::Aes, gcrypt::cipher::Mode::Ecb).unwrap(),
             gmac: gcrypt::mac::Mac::new(gcrypt::mac::Algorithm::GmacAes).unwrap(),
         };
-        c.ctr.set_key(key).expect("AES-CTR init failed");
+        c.ctr.set_key(k1).expect("AES-CTR init failed");
-        c.ecb.set_key(key).expect("AES-ECB init failed");
+        c.ecb.set_key(k1).expect("AES-ECB init failed");
-        c.gmac.set_key(key).expect("AES-GMAC init failed");
+        c.gmac.set_key(k0).expect("AES-GMAC init failed");
         c
     }
 

aes-gmac-siv/src/impl_macos.rs

@@ -61,11 +61,15 @@ impl Drop for AesGmacSiv {
 impl AesGmacSiv {
     /// Create a new keyed instance of AES-GMAC-SIV
     /// The key may be of size 16, 24, or 32 bytes (128, 192, or 256 bits). Any other size will panic.
+    /// Two keys are required: one for GMAC and one for AES-CTR.
     #[inline(always)]
-    pub fn new(key: &[u8]) -> Self {
+    pub fn new(k0: &[u8], k1: &[u8]) -> Self {
-        if key.len() != 32 && key.len() != 24 && key.len() != 16 {
+        if k0.len() != 32 && k0.len() != 24 && k0.len() != 16 {
             panic!("AES supports 128, 192, or 256 bits keys");
         }
+        if k1.len() != k0.len() {
+            panic!("k0 and k1 must be of the same size");
+        }
         let mut c: AesGmacSiv = AesGmacSiv {
             tag: [0_u8; 16],
             tmp: [0_u8; 16],
@@ -75,19 +79,19 @@ impl AesGmacSiv {
             gmac: null_mut(),
         };
         unsafe {
-            let result = CCCryptorCreateWithMode(kCCEncrypt, kCCModeCTR, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), key.as_ptr().cast(), key.len(), null(), 0, 0, 0, &mut c.ctr);
+            let result = CCCryptorCreateWithMode(kCCEncrypt, kCCModeCTR, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), k1.as_ptr().cast(), k1.len(), null(), 0, 0, 0, &mut c.ctr);
             if result != 0 {
                 panic!("CCCryptorCreateWithMode for CTR mode returned {}", result);
             }
-            let result = CCCryptorCreateWithMode(kCCEncrypt, kCCModeECB, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), key.as_ptr().cast(), key.len(), null(), 0, 0, kCCOptionECBMode, &mut c.ecb_enc);
+            let result = CCCryptorCreateWithMode(kCCEncrypt, kCCModeECB, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), k1.as_ptr().cast(), k1.len(), null(), 0, 0, kCCOptionECBMode, &mut c.ecb_enc);
             if result != 0 {
                 panic!("CCCryptorCreateWithMode for ECB encrypt mode returned {}", result);
             }
-            let result = CCCryptorCreateWithMode(kCCDecrypt, kCCModeECB, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), key.as_ptr().cast(), key.len(), null(), 0, 0, kCCOptionECBMode, &mut c.ecb_dec);
+            let result = CCCryptorCreateWithMode(kCCDecrypt, kCCModeECB, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), k1.as_ptr().cast(), k1.len(), null(), 0, 0, kCCOptionECBMode, &mut c.ecb_dec);
             if result != 0 {
                 panic!("CCCryptorCreateWithMode for ECB decrypt mode returned {}", result);
             }
-            let result = CCCryptorCreateWithMode(kCCEncrypt, kCCModeGCM, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), key.as_ptr().cast(), key.len(), null(), 0, 0, 0, &mut c.gmac);
+            let result = CCCryptorCreateWithMode(kCCEncrypt, kCCModeGCM, kCCAlgorithmAES, 0, crate::ZEROES.as_ptr().cast(), k0.as_ptr().cast(), k0.len(), null(), 0, 0, 0, &mut c.gmac);
             if result != 0 {
                 panic!("CCCryptorCreateWithMode for GCM (GMAC) mode returned {}", result);
             }

aes-gmac-siv/src/lib.rs

@@ -26,7 +26,8 @@ mod tests {
 
     #[test]
     fn encrypt_decrypt() {
-        let aes_key: [u8; 32] = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32];
+        let aes_key_0: [u8; 32] = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32];
+        let aes_key_1: [u8; 32] = [2,3,4,5,6,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32];
         let iv: [u8; 8] = [0,1,2,3,4,5,6,7];
 
         let mut buf = [0_u8; 12345];
@@ -34,7 +35,7 @@ mod tests {
             buf[i] = i as u8;
         }
 
-        let mut c = AesGmacSiv::new(&aes_key);
+        let mut c = AesGmacSiv::new(&aes_key_0, &aes_key_1);
 
         for _ in 0..256 {
             c.reset();
@@ -45,8 +46,8 @@ mod tests {
             let tag = c.encrypt_second_pass_finish().clone();
             let sha = sha2::Sha384::digest(&buf).to_vec();
             let sha = to_hex(sha.as_slice());
-            if sha != "f455fa8a1a6badaeccdefe573a10d5d79eb7f4009b84dff3d37f9f1e95ee2b0ba6149737c0701d5ef75f58f793174d3d" {
+            if sha != "b5c0997f5a0e31748dfd3a22baa4bf257b1ebf2c2ecb78a68360fa5a5f0763aa5b8fde8be502255c82d937d8adba11eb" {
-                panic!("encrypt result hash check failed!");
+                panic!("encrypt result hash check failed! {}", sha);
             }
             //println!("Encrypt OK, tag: {}, hash: {}", to_hex(&tag), sha);
 

network-hypervisor/Cargo.lock

@@ -339,6 +339,12 @@ version = "0.2.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "8ccb82d61f80a663efe1f787a51b16b5a51e3314d6ac365b08639f52387b33f3"
 
+[[package]]
+name = "urlencoding"
+version = "2.1.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68b90931029ab9b034b300b797048cf23723400aa757e8a2bfb9d748102f9821"
+
 [[package]]
 name = "version_check"
 version = "0.9.3"
@@ -424,6 +430,7 @@ dependencies = [
  "gcrypt",
  "libc",
  "rand_core",
+ "urlencoding",
  "winapi",
  "x25519-dalek",
 ]

network-hypervisor/Cargo.toml

@@ -21,6 +21,7 @@ x25519-dalek = "^1"
 ed25519-dalek = "^1"
 gcrypt = "^0"
 base64 = "^0"
+urlencoding = "^2"
 
 [target."cfg(not(windows))".dependencies]
 libc = "^0"

network-hypervisor/src/vl1/buffer.rs

@@ -1,6 +1,7 @@
-use std::mem::{size_of, MaybeUninit};
+use std::mem::{size_of, MaybeUninit, transmute};
 use std::marker::PhantomData;
 use std::io::Write;
+use std::hash::{Hash, Hasher};
 
 const OVERFLOW_ERR_MSG: &'static str = "overflow";
 
@@ -18,8 +19,10 @@ unsafe impl RawObject for NoHeader {}
 ///
 /// This also supports a generic header that must be a RawObject and will always be
 /// placed at the beginning of the buffer. When you construct or clear() a buffer
-/// space will be maintained for the header. Use NoHeader if you don't want a header.
+/// space will be maintained for the header by setting the buffer's size to the
-#[derive(Clone)]
+/// header size. The header must have a size less than or equal to L. Use NoHeader
+/// if you don't want a header.
+#[derive(Clone, PartialEq, Eq)]
 pub struct Buffer<H: RawObject, const L: usize>(usize, [u8; L], PhantomData<H>);
 
 unsafe impl<H: RawObject, const L: usize> RawObject for Buffer<H, L> {}
@@ -27,6 +30,7 @@ unsafe impl<H: RawObject, const L: usize> RawObject for Buffer<H, L> {}
 impl<H: RawObject, const L: usize> Default for Buffer<H, L> {
     #[inline(always)]
     fn default() -> Self {
+        assert!(size_of::<H>() <= L);
         Buffer(size_of::<H>(), [0_u8; L], PhantomData::default())
     }
 }
@@ -34,9 +38,18 @@ impl<H: RawObject, const L: usize> Default for Buffer<H, L> {
 impl<H: RawObject, const L: usize> Buffer<H, L> {
     #[inline(always)]
     pub fn new() -> Self {
+        assert!(size_of::<H>() <= L);
         Self::default()
     }
 
+    /// Change the header "personality" of this buffer.
+    /// Note that the new buffer must be of the same size and both the old and
+    /// new headers must be RawObjects with size less than or equal to this size.
+    pub fn change_header_type<NH: RawObject>(self) -> Buffer<NH, L> {
+        assert!(size_of::<NH>() <= L);
+        unsafe { transmute(self) }
+    }
+
     /// Create a buffer that contains a copy of a slice.
     /// If the slice is larger than the maximum size L of the buffer, only the first L bytes
     /// are copied and the rest is ignored.
@@ -58,6 +71,12 @@ impl<H: RawObject, const L: usize> Buffer<H, L> {
         &self.1[0..self.0]
     }
 
+    /// Get a slice containing the entire buffer in raw form including the header.
+    #[inline(always)]
+    pub fn as_bytes_mut(&mut self) -> &mut [u8] {
+        &mut self.1[0..self.0]
+    }
+
     /// Erase contents and reset size to the size of the header.
     #[inline(always)]
     pub fn clear(&mut self) {
@@ -339,6 +358,42 @@ impl<H: RawObject, const L: usize> Write for Buffer<H, L> {
     }
 }
 
+impl<H: RawObject, const L: usize> AsRef<[u8]> for Buffer<H, L> {
+    #[inline(always)]
+    fn as_ref(&self) -> &[u8] {
+        self.as_bytes()
+    }
+}
+
+impl<H: RawObject, const L: usize> AsMut<[u8]> for Buffer<H, L> {
+    #[inline(always)]
+    fn as_mut(&mut self) -> &mut [u8] {
+        self.as_bytes_mut()
+    }
+}
+
+impl<H: RawObject, const L: usize> AsRef<H> for Buffer<H, L> {
+    #[inline(always)]
+    fn as_ref(&self) -> &H {
+        self.header()
+    }
+}
+
+impl<H: RawObject, const L: usize> AsMut<H> for Buffer<H, L> {
+    #[inline(always)]
+    fn as_mut(&mut self) -> &mut H {
+        self.header_mut()
+    }
+}
+
+impl<H: RawObject, const L: usize> Hash for Buffer<H, L> {
+    #[inline(always)]
+    fn hash<H: Hasher>(&self, state: &mut H) {
+        state.write_u32(self.0 as u32);
+        state.write(&self.1[0..self.0]);
+    }
+}
+
 #[cfg(test)]
 mod tests {
     use std::mem::size_of;

network-hypervisor/src/vl1/endpoint.rs

@@ -0,0 +1,160 @@
+use crate::vl1::{Address, MAC};
+use crate::vl1::inetaddress::InetAddress;
+use crate::vl1::buffer::{RawObject, Buffer};
+
+#[repr(u8)]
+pub enum Type {
+    Nil = 0,
+    ZeroTier = 1,
+    Ethernet = 2,
+    WifiDirect = 3,
+    Bluetooth = 4,
+    Ip = 5,
+    IpUdp = 6,
+    IpTcp = 7,
+    Http = 8,
+    WebRTC = 9,
+}
+
+#[derive(Clone, PartialEq, Eq)]
+pub enum Endpoint {
+    Nil,
+    ZeroTier(Address),
+    Ethernet(MAC),
+    WifiDirect(MAC),
+    Bluetooth(MAC),
+    Ip(InetAddress),
+    IpUdp(InetAddress),
+    IpTcp(InetAddress),
+    Http(String),
+    WebRTC(String),
+}
+
+impl Default for Endpoint {
+    #[inline(always)]
+    fn default() -> Endpoint {
+        Endpoint::Nil
+    }
+}
+
+impl Endpoint {
+    pub fn ep_type(&self) -> Type {
+        match self {
+            Endpoint::Nil => Type::Nil,
+            Endpoint::ZeroTier(_) => Type::ZeroTier,
+            Endpoint::Ethernet(_) => Type::Ethernet,
+            Endpoint::WifiDirect(_) => Type::WifiDirect,
+            Endpoint::Bluetooth(_) => Type::Bluetooth,
+            Endpoint::Ip(_) => Type::Ip,
+            Endpoint::IpUdp(_) => Type::IpUdp,
+            Endpoint::IpTcp(_) => Type::IpTcp,
+            Endpoint::Http(_) => Type::Http,
+            Endpoint::WebRTC(_) => Type::WebRTC,
+        }
+    }
+
+    #[inline(always)]
+    pub fn ip(&self) -> Option<(&InetAddress, Type)> {
+        match self {
+            Endpoint::Ip(ip) => Some((&ip, Type::Ip)),
+            Endpoint::IpUdp(ip) => Some((&ip, Type::IpUdp)),
+            Endpoint::IpTcp(ip) => Some((&ip, Type::IpTcp)),
+            _ => None
+        }
+    }
+
+    pub fn marshal<BH: RawObject, const BL: usize>(&self, buf: &mut Buffer<BH, BL>) -> std::io::Result<()> {
+        match self {
+            Endpoint::Nil => {
+                buf.append_u8(Type::Nil as u8)
+            }
+            Endpoint::ZeroTier(a) => {
+                buf.append_u8(16 + (Type::ZeroTier as u8))?;
+                buf.append_bytes_fixed(&a.to_bytes())
+            }
+            Endpoint::Ethernet(m) => {
+                buf.append_u8(16 + (Type::Ethernet as u8))?;
+                buf.append_bytes_fixed(&m.to_bytes())
+            }
+            Endpoint::WifiDirect(m) => {
+                buf.append_u8(16 + (Type::WifiDirect as u8))?;
+                buf.append_bytes_fixed(&m.to_bytes())
+            }
+            Endpoint::Bluetooth(m) => {
+                buf.append_u8(16 + (Type::Bluetooth as u8))?;
+                buf.append_bytes_fixed(&m.to_bytes())
+            }
+            Endpoint::Ip(ip) => {
+                buf.append_u8(16 + (Type::Ip as u8))?;
+                ip.marshal(buf)
+            }
+            Endpoint::IpUdp(ip) => {
+                ip.marshal(buf)
+            }
+            Endpoint::IpTcp(ip) => {
+                buf.append_u8(16 + (Type::IpTcp as u8))?;
+                ip.marshal(buf)
+            }
+            Endpoint::Http(url) => {
+                buf.append_u8(16 + (Type::Http as u8))?;
+                let b = url.as_bytes();
+                buf.append_u16(b.len() as u16)?;
+                buf.append_bytes(b)
+            }
+            Endpoint::WebRTC(offer) => {
+                buf.append_u8(16 + (Type::WebRTC as u8))?;
+                let b = offer.as_bytes();
+                buf.append_u16(b.len() as u16)?;
+                buf.append_bytes(b)
+            }
+        }
+    }
+
+    pub fn unmarshal<BH: RawObject, const BL: usize>(buf: &Buffer<BH, BL>, cursor: &mut usize) -> std::io::Result<Endpoint> {
+        let type_byte = buf.get_u8(cursor)?;
+        if type_byte < 16 {
+            let ip = InetAddress::unmarshal(buf, cursor)?;
+            if ip.is_nil() {
+                Ok(Endpoint::Nil)
+            } else {
+                Ok(Endpoint::IpUdp(ip))
+            }
+        } else {
+            match (type_byte - 16) as Type {
+                Type::Nil => Ok(Endpoint::Nil),
+                Type::ZeroTier => Ok(Endpoint::ZeroTier(Address::from(buf.get_bytes_fixed(cursor)?))),
+                Type::Ethernet => Ok(Endpoint::Ethernet(MAC::from(buf.get_bytes_fixed(cursor)?))),
+                Type::WifiDirect => Ok(Endpoint::WifiDirect(MAC::from(buf.get_bytes_fixed(cursor)?))),
+                Type::Bluetooth => Ok(Endpoint::Bluetooth(MAC::from(buf.get_bytes_fixed(cursor)?))),
+                Type::Ip => Ok(Endpoint::Ip(InetAddress::unmarshal(buf, cursor)?)),
+                Type::IpUdp => Ok(Endpoint::IpUdp(InetAddress::unmarshal(buf, cursor)?)),
+                Type::IpTcp => Ok(Endpoint::IpTcp(InetAddress::unmarshal(buf, cursor)?)),
+                Type::Http => {
+                    let l = buf.get_u16(cursor)?;
+                    Ok(Endpoint::Http(String::from_utf8_lossy(buf.get_bytes(l as usize, cursor)?).to_string()))
+                }
+                Type::WebRTC => {
+                    let l = buf.get_u16(cursor)?;
+                    Ok(Endpoint::WebRTC(String::from_utf8_lossy(buf.get_bytes(l as usize, cursor)?).to_string()))
+                }
+            }
+        }
+    }
+}
+
+impl ToString for Endpoint {
+    fn to_string(&self) -> String {
+        match self {
+            Endpoint::Nil => format!("nil"),
+            Endpoint::ZeroTier(a) => format!("zt:{}", a.to_string()),
+            Endpoint::Ethernet(m) => format!("eth:{}", m.to_string()),
+            Endpoint::WifiDirect(m) => format!("wifid:{}", m.to_string()),
+            Endpoint::Bluetooth(m) => format!("bt:{}", m.to_string()),
+            Endpoint::Ip(ip) => format!("ip:{}", ip.to_ip_string()),
+            Endpoint::IpUdp(ip) => format!("udp:{}", ip.to_string()),
+            Endpoint::IpTcp(ip) => format!("tcp:{}", ip.to_string()),
+            Endpoint::Http(url) => url,
+            Endpoint::WebRTC(offer) => format!("webrtc:offer:{}", urlencoding::encode(offer.as_str())),
+        }
+    }
+}

network-hypervisor/src/vl1/inetaddress.rs

@@ -1,4 +1,4 @@
-use std::mem::{zeroed, size_of};
+use std::mem::{zeroed, size_of, MaybeUninit};
 use std::ptr::{write_bytes, copy_nonoverlapping, null};
 use std::str::FromStr;
 use std::cmp::Ordering;
@@ -10,6 +10,7 @@ use crate::util::equal_bytes;
 
 #[cfg(windows)]
 use winapi::um::winsock2 as winsock2;
+use crate::vl1::buffer::{RawObject, Buffer};
 
 #[allow(non_camel_case_types)]
 #[cfg(not(windows))]
@@ -23,6 +24,10 @@ type sockaddr_in = libc::sockaddr_in;
 #[cfg(not(windows))]
 type sockaddr_in6 = libc::sockaddr_in6;
 
+#[allow(non_camel_case_types)]
+#[cfg(not(windows))]
+type sockaddr_storage = libc::sockaddr_storage;
+
 #[allow(non_camel_case_types)]
 #[cfg(not(windows))]
 type in6_addr = libc::in6_addr;
@@ -44,7 +49,7 @@ pub enum IpScope {
     Global = 4,
     LinkLocal = 5,
     Shared = 6,
-    Private = 7
+    Private = 7,
 }
 
 /// An IPv4 or IPv6 socket address that directly encapsulates C sockaddr types.
@@ -54,7 +59,19 @@ pub enum IpScope {
 pub union InetAddress {
     sa: sockaddr,
     sin: sockaddr_in,
-    sin6: sockaddr_in6
+    sin6: sockaddr_in6,
+    ss: sockaddr_storage, // some external code may expect the struct to be this full length
+}
+
+impl Clone for InetAddress {
+    #[inline(always)]
+    fn clone(&self) -> Self {
+        unsafe {
+            let mut c = MaybeUninit::<InetAddress>::uninit().assume_init();
+            copy_nonoverlapping((self as *const Self).cast::<u8>(), (&mut c as *mut Self).cast::<u8>(), size_of::<Self>());
+            c
+        }
+    }
 }
 
 impl Default for InetAddress {
@@ -71,6 +88,17 @@ impl InetAddress {
         unsafe { zeroed() }
     }
 
+    /// Construct from IP and port.
+    /// If the IP is not either 4 or 16 bytes in length, a nil/0 InetAddress is returned.
+    #[inline(always)]
+    pub fn from_ip_port(ip: &[u8], port: u16) -> InetAddress {
+        unsafe {
+            let mut c = MaybeUninit::<InetAddress>::uninit().assume_init();
+            c.set(ip, port);
+            c
+        }
+    }
+
     /// Zero the contents of this InetAddress.
     #[inline(always)]
     pub fn zero(&mut self) {
@@ -129,6 +157,12 @@ impl InetAddress {
         unsafe { self.sa.sa_family as u8 == AF_INET6 }
     }
 
+    /// Get the address family of this InetAddress: AF_INET, AF_INET6, or 0 if nil.
+    #[inline(always)]
+    pub fn family(&self) -> u8 {
+        unsafe { self.sa.sa_family }
+    }
+
     /// Get a pointer to the C "sockaddr" structure and the size of the returned structure in bytes.
     /// This is useful for interacting with C-level socket APIs. If this is a nil InetAddress this
     /// returns a null pointer and 0 for the size.
@@ -221,42 +255,42 @@ impl InetAddress {
                             } else {
                                 IpScope::Global
                             }
-                        },
+                        }
                         0xa9 => {
                             if (ip & 0xffff0000) == 0xa9fe0000 { // 169.254.0.0/16
                                 IpScope::LinkLocal
                             } else {
                                 IpScope::Global
                             }
-                        },
+                        }
                         0xac => {
                             if (ip & 0xfff00000) == 0xac100000 { // 172.16.0.0/12
                                 IpScope::Private
                             } else {
                                 IpScope::Global
                             }
-                        },
+                        }
                         0xc0 => {
                             if (ip & 0xffff0000) == 0xc0a80000 || (ip & 0xffffff00) == 0xc0000200 { // 192.168.0.0/16 and 192.0.2.0/24
                                 IpScope::Private
                             } else {
                                 IpScope::Global
                             }
-                        },
+                        }
                         0xc6 => {
                             if (ip & 0xfffe0000) == 0xc6120000 || (ip & 0xffffff00) == 0xc6336400 { // 198.18.0.0/15 and 198.51.100.0/24
                                 IpScope::Private
                             } else {
                                 IpScope::Global
                             }
-                        },
+                        }
                         0xcb => {
                             if (ip & 0xffffff00) == 0xcb007100 { // 203.0.113.0/24
                                 IpScope::Private
                             } else {
                                 IpScope::Global
                             }
-                        },
+                        }
                         _ => {
                             if [
                                 0x06_u8, // 6.0.0.0/8 (US Army)
@@ -281,7 +315,7 @@ impl InetAddress {
                             }
                         }
                     }
-                },
+                }
                 AF_INET6 => {
                     let ip = &*(&(self.sin6.sin6_addr) as *const in6_addr).cast::<[u8; 16]>();
                     if (ip[0] & 0xf0) == 0xf0 {
@@ -293,11 +327,11 @@ impl InetAddress {
                             while ip[k] == 0 && k < 15 {
                                 k += 1;
                             }
-                            if k == 15 && ip[15] == 0x01 {
+                            return if k == 15 && ip[15] == 0x01 {
-                                return IpScope::Loopback; // fe80::1/128
+                                IpScope::Loopback // fe80::1/128
                             } else {
-                                return IpScope::LinkLocal; // fe80::/10
+                                IpScope::LinkLocal // fe80::/10
-                            }
+                            };
                         }
                         if (ip[0] & 0xfe) == 0xfc {
                             return IpScope::Private; // fc00::/7
@@ -315,7 +349,7 @@ impl InetAddress {
                         }
                     }
                     IpScope::Global
-                },
+                }
                 _ => IpScope::None
             }
         }
@@ -326,37 +360,77 @@ impl InetAddress {
         unsafe {
             match self.sa.sa_family as u8 {
                 AF_INET => {
-                    let ip = self.sin.sin_addr.s_addr as u32;
+                    let ip = &*(&self.sin.sin_addr.s_addr as *const u32).cast::<[u8; 4]>();
-                    format!("{}.{}.{}.{}", (ip >> 24) & 0xff, (ip >> 16) & 0xff, (ip >> 8) & 0xff, ip & 0xff)
+                    format!("{}.{}.{}.{}", ip[0], ip[1], ip[2], ip[3])
-                },
+                }
                 AF_INET6 => {
                     Ipv6Addr::from(*(&(self.sin6.sin6_addr) as *const in6_addr).cast::<[u8; 16]>()).to_string()
-                },
+                }
                 _ => {
                     String::from("(null)")
                 }
             }
         }
     }
+
+    pub fn marshal<BH: RawObject, const BL: usize>(&self, buf: &mut Buffer<BH, BL>) -> std::io::Result<()> {
+        unsafe {
+            match self.sa.sa_family as u8 {
+                AF_INET => {
+                    buf.append_and_init_bytes_fixed(|b: &mut [u8; 7]| {
+                        b[0] = 4;
+                        copy_nonoverlapping((&self.sin.sin_addr.s_addr as *const u32).cast::<u8>(), b.as_mut_ptr().offset(1), 4);
+                        b[5] = *(&self.sin.sin_port as *const u16).cast::<u8>();
+                        b[6] = *(&self.sin.sin_port as *const u16).cast::<u8>().offset(1);
+                    })
+                }
+                AF_INET6 => {
+                    buf.append_and_init_bytes_fixed(|b: &mut [u8; 19]| {
+                        b[0] = 6;
+                        copy_nonoverlapping((&(self.sin6.sin6_addr) as *const in6_addr).cast::<u8>(), b.as_mut_ptr().offset(1), 16);
+                        b[17] = *(&self.sin6.sin6_port as *const u16).cast::<u8>();
+                        b[18] = *(&self.sin6.sin6_port as *const u16).cast::<u8>().offset(1);
+                    })
+                }
+                _ => {
+                    buf.append_u8(0)
+                }
+            }
+        }
+    }
+
+    pub fn unmarshal<BH: RawObject, const BL: usize>(buf: &Buffer<BH, BL>, cursor: &mut usize) -> std::io::Result<InetAddress> {
+        match buf.get_u8(cursor)? {
+            4 => {
+                let b: &[u8; 6] = buf.get_bytes_fixed(cursor)?;
+                Ok(InetAddress::from_ip_port(&b[0..4], crate::util::integer_load_be_u16(&b[4..6])))
+            }
+            6 => {
+                let b: &[u8; 18] = buf.get_bytes_fixed(cursor)?;
+                Ok(InetAddress::from_ip_port(&b[0..16], crate::util::integer_load_be_u16(&b[16..18])))
+            }
+            _ => Ok(InetAddress::new())
+        }
+    }
 }
 
 impl ToString for InetAddress {
     fn to_string(&self) -> String {
-        let mut s = self.to_ip_string();
         unsafe {
+            let mut s = self.to_ip_string();
             match self.sa.sa_family as u8 {
                 AF_INET => {
                     s.push('/');
                     s.push_str(u16::from_be(self.sin.sin_port as u16).to_string().as_str())
-                },
+                }
                 AF_INET6 => {
                     s.push('/');
                     s.push_str(u16::from_be(self.sin6.sin6_port as u16).to_string().as_str())
-                },
+                }
                 _ => {}
             }
+            s
         }
-        s
     }
 }
 
@@ -369,7 +443,11 @@ impl FromStr for InetAddress {
             (s, 0)
         }, |pos| {
             let ss = s.split_at(pos);
-            (ss.0, u16::from_str_radix(ss.1, 10).unwrap_or(0).to_be())
+            let mut port_str = ss.1;
+            if port_str.starts_with('/') {
+                port_str = &port_str[1..];
+            }
+            (ss.0, u16::from_str_radix(port_str, 10).unwrap_or(0).to_be())
         });
         IpAddr::from_str(ip_str).map_or_else(|_| Err(InvalidFormatError), |ip| {
             unsafe {
@@ -378,7 +456,7 @@ impl FromStr for InetAddress {
                         addr.sin.sin_family = AF_INET.into();
                         addr.sin.sin_port = port.into();
                         copy_nonoverlapping(v4.octets().as_ptr(), (&mut (addr.sin.sin_addr.s_addr) as *mut u32).cast(), 4);
-                    },
+                    }
                     IpAddr::V6(v6) => {
                         addr.sin6.sin6_family = AF_INET6.into();
                         addr.sin6.sin6_port = port.into();
@@ -396,14 +474,14 @@ impl PartialEq for InetAddress {
         unsafe {
             if self.sa.sa_family == other.sa.sa_family {
                 match self.sa.sa_family as u8 {
-                    AF_INET => { self.sin.sin_port == other.sin.sin_port && self.sin.sin_addr.s_addr == other.sin.sin_addr.s_addr },
+                    AF_INET => { self.sin.sin_port == other.sin.sin_port && self.sin.sin_addr.s_addr == other.sin.sin_addr.s_addr }
                     AF_INET6 => {
                         if self.sin6.sin6_port == other.sin6.sin6_port {
                             equal_bytes((&(self.sin6.sin6_addr) as *const in6_addr).cast(), (&(other.sin6.sin6_addr) as *const in6_addr).cast(), 16)
                         } else {
                             false
                         }
-                    },
+                    }
                     _ => true
                 }
             } else {
@@ -433,7 +511,7 @@ impl Ord for InetAddress {
                         } else {
                             u16::from_be(self.sin.sin_port as u16).cmp(&u16::from_be(other.sin.sin_port as u16))
                         }
-                    },
+                    }
                     AF_INET6 => {
                         if self.sin6.sin6_port == other.sin6.sin6_port {
                             let a = &*(&(self.sin6.sin6_addr) as *const in6_addr).cast::<[u8; 16]>();
@@ -442,7 +520,7 @@ impl Ord for InetAddress {
                         } else {
                             u16::from_be(self.sin6.sin6_port as u16).cmp(&u16::from_be(other.sin6.sin6_port as u16))
                         }
-                    },
+                    }
                     _ => {
                         Ordering::Equal
                     }
@@ -464,11 +542,11 @@ impl Hash for InetAddress {
                 AF_INET => {
                     state.write_u16(self.sin.sin_port as u16);
                     state.write_u32(self.sin.sin_addr.s_addr as u32);
-                },
+                }
                 AF_INET6 => {
                     state.write_u16(self.sin6.sin6_port as u16);
                     state.write(&*(&(self.sin6.sin6_addr) as *const in6_addr).cast::<[u8; 16]>());
-                },
+                }
                 _ => {}
             }
         }
@@ -477,19 +555,38 @@ impl Hash for InetAddress {
 
 #[cfg(test)]
 mod tests {
+    use std::str::FromStr;
     use crate::vl1::inetaddress::InetAddress;
 
     #[test]
-    fn struct_layout() {
+    fn layout() {
-        let mut tmp = InetAddress::new();
         unsafe {
-            tmp.sa.sa_family = 0xf7;
+            // Make sure union is laid out such that all the sockaddr structures overlap.
-            if tmp.sin.sin_family != 0xf7 {
+            let mut tmp = InetAddress::new();
+            tmp.sa.sa_family = 0xab;
+            if tmp.sin.sin_family != 0xab {
                 panic!("sin_family misaligned in union");
             }
-            if tmp.sin6.sin6_family != 0xf7 {
+            if tmp.sin6.sin6_family != 0xab {
                 panic!("sin6_family misaligned in union");
             }
+            if tmp.ss.ss_family != 0xab {
+                panic!("ss_family misaligned in union");
+            }
         }
     }
+
+    #[test]
+    fn ipv6_string() {
+        let ip = InetAddress::from_str("2603:6010:6e00:1118:d92a:ab88:4dfb:670a/1234").unwrap();
+        assert_eq!("2603:6010:6e00:1118:d92a:ab88:4dfb:670a/1234", ip.to_string());
+        let ip = InetAddress::from_str("fd80::1/1234").unwrap();
+        assert_eq!("fd80::1/1234", ip.to_string());
+    }
+
+    #[test]
+    fn ipv4_string() {
+        let ip = InetAddress::from_str("1.2.3.4/1234").unwrap();
+        assert_eq!("1.2.3.4/1234", ip.to_string());
+    }
 }

network-hypervisor/src/vl1/mod.rs

@@ -1,9 +1,15 @@
-pub mod protocol;
+pub(crate) mod protocol;
-pub mod packet;
+pub(crate) mod packet;
-pub mod buffer;
+pub(crate) mod buffer;
-pub mod node;
 pub mod identity;
 pub mod inetaddress;
+pub mod endpoint;
+pub(crate) mod node;
 
 mod address;
+mod mac;
+
 pub use address::Address;
+pub use mac::MAC;
+pub use identity::Identity;
+pub use endpoint::Endpoint;

network-hypervisor/src/vl1/packet.rs

@@ -1,9 +1,11 @@
-use crate::vl1::buffer::{Buffer, RawObject};
 use crate::vl1::protocol::{HEADER_FLAGS_FIELD_MASK_CIPHER, HEADER_FLAGS_FIELD_MASK_HOPS, HEADER_FLAG_FRAGMENTED};
 use std::ops::Not;
 
 type PacketID = u64;
 
+/// A packet buffer, but with no header "personality."
+pub type Buffer = crate::vl1::buffer::Buffer<crate::vl1::buffer::NoHeader, { crate::vl1::protocol::PACKET_SIZE_MAX }>;
+
 #[derive(Clone)]
 #[repr(packed)]
 pub struct Header {
@@ -14,7 +16,7 @@ pub struct Header {
     pub message_auth: [u8; 8],
 }
 
-unsafe impl RawObject for Header {}
+unsafe impl crate::vl1::buffer::RawObject for Header {}
 
 impl Header {
     /// Get this packet's ID as a u64.
@@ -51,7 +53,7 @@ impl Header {
     }
 }
 
-pub type Packet = Buffer<Header, { crate::vl1::protocol::PACKET_SIZE_MAX }>;
+pub type Packet = crate::vl1::buffer::Buffer<Header, { crate::vl1::protocol::PACKET_SIZE_MAX }>;
 
 #[derive(Clone)]
 #[repr(packed)]
@@ -63,7 +65,7 @@ pub struct FragmentHeader {
     pub hops: u8,
 }
 
-unsafe impl RawObject for FragmentHeader {}
+unsafe impl crate::vl1::buffer::RawObject for FragmentHeader {}
 
 impl FragmentHeader {
     #[inline(always)]
@@ -82,7 +84,7 @@ impl FragmentHeader {
     }
 }
 
-pub type Fragment = Buffer<FragmentHeader, { crate::vl1::protocol::PACKET_SIZE_MAX }>;
+pub type Fragment = crate::vl1::buffer::Buffer<FragmentHeader, { crate::vl1::protocol::PACKET_SIZE_MAX }>;
 
 #[cfg(test)]
 mod tests {

network-hypervisor/src/vl2/mod.rs

@@ -1,3 +0,0 @@
-mod mac;
-
-pub use mac::MAC;