Routing and Addresses
How anchors choose a route to each peer, and how every anchor gets its addresses without any configuration.
Source Routing
Anchors use source routing: the sending anchor chooses the whole route to a peer, hop by hop. It works the route out from what it already knows about the realm, so there is no routing protocol to run and no routing table to maintain on the relays.
- A route is either direct, or relayed through up to five relays.
- A route never loops and never passes through the same hop twice.
- Relays keep no routing state. Each relay only knows the hop before it and the hop after it.
- Only anchors that offer relaying for your realm can be a hop on your routes.
The sender works out the route itself, from what gossip has told it. A direct connection is one of the routes it compares, and traffic leaves it only for a route that stays faster over time. The onion header of a relayed route is the same size whatever the route, so no relay can tell how far it is from either end.
Route Cost
When several routes lead to a peer, the anchor sends traffic on the cheapest one. The cost of a route is the sum of the round-trip times of its hops, so a direct connection and a relayed route are compared in the same way.
Round-trip times come from the traffic you already send. Anchors never send probes just to measure a route.
Here the relayed route costs less than the direct one, so traffic moves onto it once it stays cheaper over time. The direct connection stays open.
- Traffic moves to another route only when that route stays faster over time. A single slow sample never moves it, so routes stay stable.
- A route that fails is taken out of use for 30 seconds, then tried again.
- When you first reach a peer, the anchor tries a direct connection. If it is not up within 250 ms, the anchor also builds a relayed route, and uses whichever connects first. A direct connection that comes up later takes the traffic over.
Relays
A relayed route is also called a circuit. Traffic on a circuit stays encrypted end to end: a relay cannot read or change it, and cannot pretend to be either end.
| Limit | Value |
|---|---|
| Relays on one route | 5 |
| Circuits on one relay | 16,384 |
| Circuit lifetime | 10 minutes. A new circuit replaces it before it expires, without interrupting traffic |
| Bandwidth | Not limited. Relays forward at line rate |
Overlay Addresses
Anchors never allocate or negotiate addresses. Every member derives each other member's addresses from its AnchorID, the identifier of its public key. Because every member derives the same addresses, two anchors can reach each other before they have exchanged a single packet.
| Address | How it is formed |
|---|---|
| MAC address | From a hash of the AnchorID |
| IPv6 address | The realm's /48 prefix, followed by 80 bits from a hash of the AnchorID |
| IPv4 address | Chosen by you. See IPv4 Addresses |
An anchor drops any frame that uses another member's MAC address, so no member can send traffic in another member's name.
IPv4 Addresses
The overlay itself is IPv6, but you can give any anchor an IPv4 address and use it as you would on a LAN. The anchor carries IPv4 traffic inside IPv6 and converts it back at the other end. In route listings, IPv4 networks appear with the IPv6 prefix they travel as, under 64:ff9b:1::/96.
To give the anchor on a conflux machine an IPv4 address, pass it to conflux up:
$ sudo conflux up --ipv4 10.128.0.5/24
- Choose any unicast address and prefix length. Two anchors can use the same IPv4 address without a conflict.
- An anchor with no IPv4 address can still reach IPv4 addresses, but cannot be reached at one.
- The receiving host sees each sender at an address from
198.18.0.0/15, one per sender, rather than at the sender's own IPv4 address. - IPv4 broadcast and multicast traffic, such as mDNS and DHCP, stays on the local host.
For subnet routing, exit nodes and the other per-machine settings, see Subnets, Exits and Uplink.
Ethernet Switching
Each anchor works as a small Ethernet switch between your host and the overlay, so ordinary IP traffic crosses the overlay unchanged. The anchor already knows every member's MAC address from the realm's membership. It never floods unknown traffic, and ARP and neighbour discovery never go out on the wire.
- A frame for an address that no member holds is dropped, never flooded.
- Broadcast and multicast frames reach every peer the anchor can reach, up to 256 frames a second from each sender.
Inspecting Routes
On a conflux machine, run anchor commands through sudo conflux. To see this machine's own addresses, check the end of sudo conflux status:
anchor anchorxru3k54nd3… underlay 192.0.2.7:44700 overlay fd5f:23c6:acf0:883b:d9c6:6497:fe97:46af/48 hardware 92:c6:9a:e1:36:f6
To see the route that traffic to a peer takes now, run sudo conflux route -to ID. It prints direct, or the relays in order: relayed -> hop -> hop.
To list IPv4 routes, run sudo conflux routes:
$ sudo conflux routes NETWORK VIA STATE WHY 10.128.0.2/32 (64:ff9b:1::a80:2/128) x7xcjhvdvt accepted - 10.128.0.16/32 (64:ff9b:1::a80:10/128) - served -
If a peer is known but has no route, run sudo conflux peers -long and check whether any relay offers relaying for it. A peer with no relays listed is the usual cause.
| Metric or event | What it tells you |
|---|---|
anchor_frames_dropped_total | Dropped frames, by reason. unknown-destination means this anchor does not know the whole realm yet. wrong-source means a forged or conflicting address |
anchor_relay_circuits | Circuits this anchor carries for others, out of 16,384. Watch it before a relay fills up |
anchor_relay_refused_total | Circuits this anchor refused, by reason |
anchor_path_cost_seconds | The cost of the relayed routes this anchor chose |
relay-abuse | A relay refused a circuit because it was full |
address-conflict | Two anchors derived the same MAC address. Give one of them a new identity |