Need help setting up a DHCP server in my homelab

I’m building a small homelab and want a dedicated DHCP server instead of relying on my router, but I’m not sure how to set it up correctly for multiple VLANs and devices. I’m worried about messing up my existing network, IP ranges, and DNS settings. Can someone walk me through the best practices, recommended software, and basic configuration steps for a stable, secure DHCP setup in a homelab environment?

Short version: put DHCP on a server, route with VLAN interfaces, use DHCP relay on the switch or router, then turn off DHCP on the ISP router.

Here is a simple, safe layout for a homelab.

Example setup
Router or L3 switch: 192.168.10.1/24 VLAN 10, 192.168.20.1/24 VLAN 20
DHCP server: Linux or Windows VM, on a trunk port, with subinterfaces or one IP in a “services” VLAN plus relay.

Step 1, pick your VLANs and subnets
VLAN 10: LAN, 192.168.10.0/24, gateway 192.168.10.1
VLAN 20: LAB, 192.168.20.0/24, gateway 192.168.20.1
Optionally a management VLAN like 192.168.30.0/24.

Step 2, routing device
You need something to do inter‑VLAN routing and DHCP relay.
Examples.
• pfSense, OPNsense, VyOS, RouterOS
• L3 switch (Cisco, HP, etc)

Create VLAN interfaces on that box.
Assign IPs as above.
Enable DHCP relay for each VLAN and point it to your DHCP server IP.

Step 3, DHCP server (Linux example, isc‑dhcp‑server)

/etc/dhcp/dhcpd.conf:

authoritative;

option domain-name ‘home.lab’;
option domain-name-servers 192.168.10.2, 1.1.1.1;

subnet 192.168.10.0 netmask 255.255.255.0 {
range 192.168.10.50 192.168.10.200;
option routers 192.168.10.1;
option broadcast-address 192.168.10.255;
}

subnet 192.168.20.0 netmask 255.255.255.0 {
range 192.168.20.50 192.168.20.200;
option routers 192.168.20.1;
option broadcast-address 192.168.20.255;
}

If the DHCP server sits in VLAN 10 only, it has a single IP like 192.168.10.2 and you rely on relays from other VLANs.
If you prefer subinterfaces:

On Linux with a trunk:

ip link add link eth0 name eth0.10 type vlan id 10
ip addr add 192.168.10.2/24 dev eth0.10
ip link set eth0.10 up

ip link add link eth0 name eth0.20 type vlan id 20
ip addr add 192.168.20.2/24 dev eth0.20
ip link set eth0.20 up

Then bind dhcpd to those interfaces in /etc/default/isc-dhcp-server:

INTERFACESv4=‘eth0.10 eth0.20’

Step 4, adjust your ISP router
Put it in bridge mode if possible.
Let your own router do PPPoE or DHCP on WAN.
Disable DHCP on the ISP router or isolate it to a different network, so no double DHCP.

Step 5, switch config idea (Cisco‑style syntax)

vlan 10
name LAN

vlan 20
name LAB

interface Gig1/0/1 ← trunk to router or pfSense
switchport mode trunk
switchport trunk allowed vlan 10,20

interface Gig1/0/2 ← access port for LAN
switchport mode access
switchport access vlan 10

Repeat for other ports.

On L3 switch with relay:

interface vlan 10
ip address 192.168.10.1 255.255.255.0
ip helper-address 192.168.10.2

interface vlan 20
ip address 192.168.20.1 255.255.255.0
ip helper-address 192.168.10.2

Step 6, testing without breaking your live stuff

  1. Create a new VLAN and subnet that no one uses yet.
  2. Plug one test device into an access port for that VLAN.
  3. Confirm it gets an IP from your DHCP server.
  4. Check it reaches the gateway, then the internet.
  5. After it looks solid, move your main LAN to the new system.

Common pitfalls
• Two DHCP servers on same VLAN. You get random leases. Double check the ISP router DHCP is off before migrating.
• Wrong IP helper on the router. DHCP requests never reach the server.
• Firewall rules on pfSense or Linux that block DHCP (UDP 67 and 68).
• Trunk or VLAN mismatch between switch, router, and the VMware or Proxmox host.

If you share the gear you use, people can post exact config snippets.

You’re right to be cautious, but you can make this almost zero‑risk if you build it alongside your current setup and only flip things over when you’re happy.

@cacadordeestrelas already gave you the “proper” VLAN + relay model. I’ll go a bit sideways and show another safe pattern and some guardrails so you don’t nuke the house Wi‑Fi.


1. Start with a lab VLAN that cannot hurt anything

Before touching your main LAN:

  1. Pick a totally unused subnet
    Example:

    • VLAN 50: TEST
    • 192.168.50.0/24
    • Gateway (future): 192.168.50.1
  2. On your switch, create VLAN 50 and one access port in it.
    Plug a spare laptop/desktop there. This is your “sacrifice” device.

  3. Do not disable DHCP on your ISP router yet. Your regular LAN still lives on the old network.

This way, anything you break only affects VLAN 50.


2. Decide where your “real router” lives

Two basic patterns:

A. Dedicated firewall/router VM or box (pfSense / OPNsense / VyOS, etc.)
This becomes your default gateway for all VLANs.
Your ISP router becomes just a modem (bridge mode).

B. L3 switch does routing, firewall/router just does WAN
More “enterprisy” but honestly overkill for a small homelab unless you already have a good L3 switch.

For a first homelab, I’d actually disagree slightly with the idea of going too hard on L3 switching: one firewall/router VM with VLAN interfaces is easier to visualize and troubleshoot.

Example with pfSense as the main router:

  • pfSense VLAN 10: 192.168.10.1
  • pfSense VLAN 20: 192.168.20.1
  • pfSense VLAN 50 (test): 192.168.50.1

Let pfSense do the inter‑VLAN routing, not your switch.


3. Keep DHCP simple at first

Instead of immediately doing a separate Linux DHCP server plus relays, a low‑stress path:

  1. Enable DHCP on pfSense/OPNsense for each VLAN.
  2. Confirm the whole VLAN + firewall + DHCP flow works.
  3. Only after that, move DHCP to a dedicated VM if you still want that separation.

Why: your first tricky bit is actually VLAN and routing, not DHCP itself. Minimizing moving parts while you prove the design saves a lot of “why the hell is this broken” time.

If you really want the standalone DHCP from day one, fine, but:

  • Put the DHCP VM in one “services” VLAN or LAN (for example VLAN 10)
  • Use DHCP relay on the router/firewall for the other VLANs, like @cacadordeestrelas described

4. Bullet‑proof rules to avoid breaking the existing network

This is the part you’re worrying about, so here’s the checklist I use at home:

  1. Never have two DHCP servers on the same broadcast domain.

    • Before you move your main LAN to the new DHCP, either
      • Disable DHCP on the ISP router, or
      • Put the ISP router’s LAN on a different subnet / network entirely.
  2. Only test on the test VLAN until it works end‑to‑end:

    • Device on VLAN 50 gets IP from your new DHCP
    • Device can ping VLAN 50 gateway
    • Device can reach the internet
  3. Do not renumber your current LAN yet.
    You can keep 192.168.1.0/24 on the ISP router and build 192.168.10.0/24, 20.0/24, 50.0/24 on your new gear.
    When you’re ready, you migrate devices over one by one by moving switch ports to the new VLANs or reconnecting WiFi APs.

  4. Document the cutover step in advance.
    Literally write a 5‑line “if this fails, do X” plan so you can revert DHCP back to the ISP router in 2 minutes if you panic.


5. Concrete “first working test” recipe

Very short version:

  1. Create VLAN 50 on switch and on pfSense.
  2. pfSense: assign VLAN 50 interface, give it 192.168.50.1/24.
  3. pfSense: enable DHCP for VLAN 50, range 192.168.50.50–192.168.50.200.
  4. Plug test laptop into VLAN 50 access port on switch.
  5. Laptop should get an IP in 192.168.50.x, gateway 192.168.50.1, DNS from pfSense or upstream.
  6. If that works, then introduce your dedicated DHCP server and relays.

Once that’s good, copy‑paste the pattern for VLAN 10 and VLAN 20, verify both, and only then change where your “real” devices plug in.


6. When you finally move DHCP off the router

Later, when you’re comfortable:

  • Stand up a Linux DHCP VM (isc‑dhcp, kea, dnsmasq, whatever you like) in, say, VLAN 10 with static IP 192.168.10.2.
  • On pfSense, turn off its own DHCP for each VLAN and add DHCP relay for each interface pointing to 192.168.10.2.
  • Double‑check firewall rules allow DHCP traffic (UDP 67/68) between router and DHCP VM.

Do it one VLAN at a time so if something’s off, only that VLAN is affected and you know exactly where to look.


If you post your exact gear (switch model, firewall choice, hypervisor) and current subnet, people can throw you configs specific to your hardware so you’re basically just copy/pasting instead of guessing.