Previous Page

nihilist@mainpc - 2024-04-07

Internet Failover (dual wan pfsense setup)

Threat Model:

Your ISP connection comes with a closed-source router. What makes you think that your ISP isn't giving access to it to an adversary so that he may be able to spy on your home network ? How do you protect against that?

That same adversary suspects that you are running a hidden service from home. That adversary makes your ISP shut down your internet connection to check if you are actually running it or not. How do you ensure your hidden service keeps running ?

In this tutorial we're going to setup a pfsense VM inside of virt-manager to make sure that our .onion Hidden service is hidden behind an open-source router, rather than a closed-source one. as detailed below:

We're going to also make sure that we protect the hidden service from controlled internet downtimes, with a failover internet connection to a mobile hotspot.

Initial Setup

First you're going to need a Libvirtd QEMU hypervisor on your home server, check this tutorial to know how to set it up.

So here we create the pfsense VM as shown in this tutorial, and we make sure to adjust it to have the following network configuration:

So for the main network interface we setup the network interface as a direct attachment to the host network interface enp8s0 (as a macvtap device in virt-manager):

As detailed in the previous tutorial, for the LAN network we setup an isolated network and use it like so:

Then from inside pfsense we can set them both like so:

Then we setup the second WAN, which is our mobile USB tethering hotspot. First just connect the mobile phone to the homeserver via USB:

Once plugged in, you can check if the homeserver detects it via the lsusb command, and if it does, just add the USB host device to the VM directly like so:

However that's not enough as when you enable USB tethering the USB device ID changes, so we enable USB tethering like so (ex: in Graphene OS you go to: Settings > Network and Internet > Hotspot & Tethering > Toggle USB Tethering ON) before adding it in the pfsense VM:

Now that the device is added, enable USB tethering from your phone , then let's make sure that it is properly configured as a second WAN interface in pfsense:

Here you see the pfsense VM detecting the usb device from console, however to make the setup simpler we'll set it up from the pfsense dashboard, from the VM inside the LAN network:

So after clicking "add" we have now the OPT3 interface that we can configure:

We rename it to WAN-Mobile, set it to DHCP (as it is the mobile phone that gives the DHCP lease to that interface), and hit save:

Here you can also see that pfsense detects that interface as a gateway in the routing section:

Now that's done, we need to setup the failover by first having both gateways into the same gateway group:

Now here we have a gateway group, we have set our main WAN interface (WANGW, the ethernet connection) to be tier 1 as in first priority, and we have set our secondary WAN interface (WANMOBILE) to be Tier 2 as in second priority. The trigger level to switch between the 2 is going to be Packet Loss. Meaning if the ethernet connection goes down, the internet connection will resume through the mobile USB tethering hotspot:

Now we hit save and apply, then we need to edit the LAN firewall rule because otherwise it won't accept any traffic to be routed to the other gateway:

Now with this, the lan subnet will automatically route traffic through either gateway as dictated by pfsense. which is what we want. Now hit save and apply:

And now we can see it in action when we unplug the ethernet cable like so:

As you can see here, the traffic first goes through the default WAN interface, and after i unplug the ethernet cable, the same traffic starts to go through the other WAN interface via the mobile connection. Which concludes today's tutorial.