Deploying VeilNet with Docker

Prerequisites

• Docker and Docker Compose installed
• VeilNet registration token
• Access to VeilNet Guardian service

Setup VeilNet Conflux with Docker

This guide walks you through setting up VeilNet Conflux using Docker to enable access to container networks across multiple hosts.

Step 1: Create Docker Compose Configuration

Create a docker-compose.yml file with the following configuration:

services:
  veilnet-conflux-1:
    container_name: veilnet-conflux-1
    restart: unless-stopped
    env_file:
      - .env
    image: veilnet/conflux:beta
    pull_policy: always
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    network_mode: host

Step 2: Create Environment File

Create a .env file in the same directory as your docker-compose.yml with the following variables:

VEILNET_REGISTRATION_TOKEN=<YOUR_REGISTRATION_TOKEN>
VEILNET_GUARDIAN=<YOUR_GUARDIAN_URL>
VEILNET_PORTAL=true
VEILNET_CONFLUX_TAG=<YOUR_CONFLUX_TAG>

Replace the placeholders:

• <YOUR_REGISTRATION_TOKEN>: Your VeilNet registration token (obtained from the VeilNet portal)
• <YOUR_GUARDIAN_URL>: The URL of your VeilNet Guardian service (e.g., https://guardian.veilnet.app)
• <YOUR_CONFLUX_TAG>: A tag to identify this Conflux instance (e.g., dev-server, production-1)

Step 3: Deploy the Container

Start the VeilNet Conflux container:

docker-compose up -d

This will:

• Pull the veilnet/conflux:beta image if not already present
• Start the container with NET_ADMIN capability and TUN device access
• Automatically restart the container if it stops

Step 4: Verify Deployment

Check that the container is running:

docker ps | grep veilnet-conflux

View the container logs to verify it's connecting successfully:

docker logs veilnet-conflux-1 -f

You should see logs indicating successful registration and connection to the VeilNet network.

Accessing Container Networks

Once the VeilNet Conflux container is running, you can access containers by their container network IP addresses, even when they're running on different Docker hosts. This works because VeilNet routes traffic to container networks when the host is connected to the VeilNet overlay network.

Accessing Containers by Container Network IP

When your Docker host is connected to VeilNet, you can access containers using their container network IP addresses from:

• Other containers on the same host
• Containers on other Docker hosts connected to VeilNet
• Any device connected to the VeilNet network

For example, if a container has IP 172.17.0.2 in its Docker network, you can access it directly using that IP from any other device on the VeilNet network, regardless of which physical host the container is running on.

Creating a Service Mesh with Network Namespace Sharing

The recommended approach for creating a service mesh with VeilNet is to have your application containers share the network namespace with the VeilNet Conflux container. This approach is recommended because:

• No IP conflicts: Services are accessed via the host's VeilNet IP address, eliminating the need to manage different CIDR ranges across hosts
• Containers can directly use the VeilNet TUN device for optimal network performance
• Simplified configuration without port mapping complexity

Benefits of Network Namespace Sharing

When containers share the network namespace with veilnet-conflux:

• No IP conflicts: Services are accessed via the host's VeilNet IP address (e.g., 10.128.0.5:3000), not container IPs. This eliminates the need for different CIDR ranges across hosts
• Containers can directly access the VeilNet TUN device
• All containers use the host's network stack, eliminating port mapping complexity
• Services can communicate using localhost when on the same host
• Better network performance with direct access to the overlay network
• Simplified configuration - no need for custom Docker networks or port mappings

Step 1: Deploy VeilNet Conflux

First, deploy VeilNet Conflux on each host using the configuration from Step 1 above.

Step 2: Deploy Services with Network Namespace Sharing

Deploy your services using network_mode: "container:veilnet-conflux" to share the network namespace:

# docker-compose.yml on Host 1
services:
  veilnet-conflux:
    container_name: veilnet-conflux
    restart: unless-stopped
    env_file:
      - .env
    image: veilnet/conflux:beta
    pull_policy: always
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    network_mode: host

  web-app:
    image: your-web-app:latest
    network_mode: "container:veilnet-conflux"
    depends_on:
      - veilnet-conflux

  api-service:
    image: your-api:latest
    network_mode: "container:veilnet-conflux"
    depends_on:
      - veilnet-conflux
    environment:
      - WEB_APP_URL=http://localhost:3000

# docker-compose.yml on Host 2
services:
  veilnet-conflux:
    container_name: veilnet-conflux
    restart: unless-stopped
    env_file:
      - .env
    image: veilnet/conflux:beta
    pull_policy: always
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun
    network_mode: host

  database:
    image: postgres:15-alpine
    network_mode: "container:veilnet-conflux"
    depends_on:
      - veilnet-conflux
    environment:
      - POSTGRES_DB=mydb
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

Step 3: Access Services Across Hosts

Once all hosts are connected to VeilNet, services can communicate using the host's VeilNet IP address:

• Services on Host 1 can be accessed from Host 2 using http://<host1-veilnet-ip>:<port>
• Services on Host 2 can be accessed from Host 1 using http://<host2-veilnet-ip>:<port>
• All services are accessible from any device connected to the VeilNet network

Example:

• Host 1 has VeilNet IP 10.128.0.5 and runs a web app on port 3000
• Host 2 has VeilNet IP 10.128.0.6 and runs a database on port 5432
• From Host 1, the web app can connect to the database using 10.128.0.6:5432
• From any VeilNet-connected device, access the web app using 10.128.0.5:3000

Alternative: Service Mesh with Custom Docker Networks

If you prefer to use custom Docker networks instead of network namespace sharing, you can create a service mesh by configuring each host's Docker networks with different CIDR ranges. This allows you to organize services logically while maintaining network isolation.

Important: When using custom Docker networks and accessing containers by their container IP addresses, each host MUST use different CIDR ranges to avoid IP conflicts. If two hosts use the same subnet (e.g., both use 172.17.0.0/16), containers on different hosts could have the same IP address (e.g., both could have 172.17.0.2), causing routing conflicts. This is why network namespace sharing (accessing via VeilNet IP) is recommended - it avoids this IP conflict issue entirely.

Step 1: Create Custom Docker Networks with Specific CIDRs

On each Docker host, create custom networks with unique, non-overlapping CIDR ranges:

Host 1:

docker network create --subnet=172.17.0.0/16 frontend-network
docker network create --subnet=172.18.0.0/16 backend-network

Host 2:

docker network create --subnet=172.19.0.0/16 frontend-network
docker network create --subnet=172.20.0.0/16 backend-network

Host 3:

docker network create --subnet=172.21.0.0/16 frontend-network
docker network create --subnet=172.22.0.0/16 backend-network

Note: Each host uses different CIDR ranges to ensure no IP conflicts when accessing containers by their container network IP addresses.

Step 2: Deploy Services on Different Hosts

Deploy your services using these custom networks:

# docker-compose.yml on Host 1
services:
  web-app:
    image: your-web-app:latest
    networks:
      - frontend-network

networks:
  frontend-network:
    external: true

# docker-compose.yml on Host 2
services:
  api-service:
    image: your-api:latest
    networks:
      - backend-network

networks:
  backend-network:
    external: true

Step 3: Access Services Across Hosts

Once all hosts are connected to VeilNet, services can communicate using their container network IPs:

• A container on Host 1 with IP 172.17.0.5 can be accessed from Host 2 using 172.17.0.5
• A container on Host 2 with IP 172.20.0.3 can be accessed from Host 1 using 172.20.0.3
• All containers can communicate as if they were on the same physical network

Connecting Multiple Docker Hosts

To connect multiple Docker hosts:

1. Deploy VeilNet Conflux on each host using the same Docker Compose configuration
2. Use different VEILNET_CONFLUX_TAG values to identify each instance
3. Ensure all instances use the same VEILNET_GUARDIAN URL and are registered to the same VeilNet realm
4. If using custom Docker networks, configure them with non-overlapping CIDR ranges on each host (required to avoid IP conflicts)

With Network Namespace Sharing (Recommended):

• Each host will be able to communicate with services on other hosts through the VeilNet overlay network using the host's VeilNet IP address
• Services on the same host can communicate using localhost
• All containers sharing the network namespace can use the VeilNet TUN device directly
• No IP conflicts: Services are accessed via host VeilNet IPs (e.g., 10.128.0.5:3000), eliminating the need for different CIDR ranges per host

With Custom Docker Networks:

• Each host will be able to communicate with containers on other hosts using their container network IP addresses
• Services maintain network isolation through separate Docker networks
• Critical: Each host MUST use different CIDR ranges to prevent IP conflicts when accessing containers by their container IP addresses. This is why network namespace sharing is recommended - it avoids this complexity

Updating VeilNet Conflux

To update to a newer version of VeilNet Conflux:

1. Pull the latest image:

docker-compose pull

2. Restart the container:

docker-compose up -d

The container will automatically use the updated image and reconnect to the VeilNet network.

Stopping and Removing

To stop the VeilNet Conflux container:

docker-compose down

To remove the container and its configuration:

docker-compose down -v

Note: This will not remove the .env file, so your configuration will be preserved for future deployments.

FAQ

Why is NET_ADMIN capability required?

The NET_ADMIN capability is required because VeilNet Conflux needs to create and manage network interfaces (TUN devices) on the host system. This capability, along with access to /dev/net/tun, allows it to establish the overlay network connection necessary for routing traffic without requiring full privileged mode.

Why use host network mode?

Host network mode (network_mode: host) is used to give the container direct access to the host's network stack. This is necessary for VeilNet to properly route traffic and create the network interfaces required for the overlay network.

Can I run multiple Conflux instances on the same host?

Yes, you can run multiple Conflux instances on the same host by:

• Using different container names
• Using different VEILNET_CONFLUX_TAG values
• Ensuring each instance has a unique registration token or is configured for different realms

How do I access containers on remote Docker hosts?

There are two approaches:

With Network Namespace Sharing (Recommended):

• Access services using the host's VeilNet IP address and the service's port
• For example, if Host 1 has VeilNet IP 10.128.0.5 and runs a service on port 3000, access it using http://10.128.0.5:3000
• Services on the same host can communicate using localhost
• All containers share the network namespace with veilnet-conflux and can use the TUN device directly

With Custom Docker Networks:

• Access containers using their container network IP addresses (not just VeilNet IPs)
• For example, if a container has IP 172.17.0.2 in its Docker network, you can access it directly using that IP from any other device on the VeilNet network, regardless of which physical host the container is running on
• The overlay network routes traffic to container networks automatically, allowing containers on different physical hosts to communicate as if they were on the same local network
• Important: Each host must use different CIDR ranges. If Host 1 uses 172.17.0.0/16 and Host 2 also uses 172.17.0.0/16, containers on both hosts could have the same IP (e.g., 172.17.0.2), causing routing conflicts

Should I use network namespace sharing or custom Docker networks?

Use network namespace sharing when (Recommended):

• You want to avoid IP conflicts - accessing services via VeilNet IP (e.g., 10.128.0.5:3000) eliminates the need for different CIDR ranges per host
• You want optimal network performance with direct TUN device access
• You prefer simpler configuration without port mappings
• Services on the same host need to communicate via localhost
• You want all containers to benefit from VeilNet's network capabilities

Use custom Docker networks when:

• You need strict network isolation between services
• You want to organize services into logical network segments
• You're migrating from an existing Docker network setup
• You need to maintain compatibility with existing network configurations
• Important: You must ensure each host uses different, non-overlapping CIDR ranges to avoid IP conflicts when accessing containers by their container IP addresses. This is the main reason network namespace sharing is recommended - it avoids this IP conflict issue by using VeilNet IPs instead

What happens if the container stops?

With restart: unless-stopped, Docker will automatically restart the container if it stops unexpectedly. This ensures your VeilNet connection remains available even after system reboots or container crashes.