Ethernet, ARP & Switching

The Local Neighborhood#

IP routing gets all the attention, but a packet can only cross the internet because it first crosses one hop of your local network. That hop is Ethernet (or Wi-Fi, which behaves the same way from the host's point of view), and it works entirely by MAC addresses, not IP addresses. Many "network outages" that look like application failures turn out to be problems at this layer.

MAC Addresses#

A Media Access Control (MAC) address is a 48-bit identifier for a network interface, written as six hex octets (3c:22:fb:1a:2b:9c, or 3C-22-FB-1A-2B-9C on Windows). The first three octets are normally the OUI (Organizationally Unique Identifier) assigned to the manufacturer; the rest is assigned by that manufacturer.

Two bits in the first octet carry meaning:

• Bit 0 (I/G): if set, the address is multicast. ff:ff:ff:ff:ff:ff is the broadcast address; 01:00:5e:... is IPv4 multicast; 33:33:... is IPv6 multicast.
• Bit 1 (U/L): if set, the address is locally administered, meaning software chose it rather than the factory. Phones that randomize their Wi-Fi MAC per network, virtual machines and containers all use locally administered addresses.

def describe_mac(mac: str) -> dict:
    octets = bytes.fromhex(mac.replace(":", "").replace("-", ""))
    first = octets[0]
    return {
        "oui": octets[:3].hex(":"),
        "multicast": bool(first & 0b01),             # I/G bit
        "locally_administered": bool(first & 0b10),  # U/L bit
        "broadcast": octets == b"\xff" * 6,
    }

for m in ["3c:22:fb:1a:2b:9c", "da:a1:19:00:00:01", "01:00:5e:00:00:fb"]:
    print(m, describe_mac(m))

3c:22:fb:1a:2b:9c {'oui': '3c:22:fb', 'multicast': False, 'locally_administered': False, 'broadcast': False}
da:a1:19:00:00:01 {'oui': 'da:a1:19', 'multicast': False, 'locally_administered': True, 'broadcast': False}
01:00:5e:00:00:fb {'oui': '01:00:5e', 'multicast': True, 'locally_administered': False, 'broadcast': False}

The practical consequence: a vendor lookup on a randomized MAC tells you nothing, and MAC-based allowlists on Wi-Fi are weak because any host can set its own MAC. MAC addresses only mean something inside one Layer 2 broadcast domain; routers rewrite the Ethernet header at every hop.

The Ethernet Frame#

A rising "CRC errors" or "FCS errors" counter on a switch port (ethtool -S eth0 on Linux, show interface on most switches) points to a bad cable, dirty fiber or a duplex mismatch, not to software.

ARP: Bridging IP and MAC#

Your computer knows the destination IP, but Ethernet frames need a destination MAC. The Address Resolution Protocol (ARP) bridges the two for IPv4:

1. The host checks whether the destination is on its own subnet (using the subnet mask). If not, the next hop is the default gateway, and it resolves the gateway's MAC instead.
2. It broadcasts an ARP request to ff:ff:ff:ff:ff:ff: "Who has 192.168.1.1? Tell 192.168.1.20."
3. The owner of that IP replies with a unicast frame containing its MAC.
4. The host caches the mapping for a short time and sends the real frame.

Inspect the cache:

ip neigh show            # Linux
arp -a                   # macOS, Windows, older Linux

192.168.1.1 dev wlan0 lladdr 3c:22:fb:1a:2b:9c REACHABLE
192.168.1.37 dev wlan0 lladdr da:a1:19:00:00:01 STALE
192.168.1.99 dev wlan0  FAILED

REACHABLE was recently confirmed, STALE will be re-verified on next use, and FAILED means nobody answered: the host is off, on another VLAN, or firewalled at Layer 2. If your gateway shows FAILED, nothing beyond the LAN will work, no matter how correct your routes and DNS are.

Watch ARP on the wire with sudo tcpdump -n -e arp. You will also see gratuitous ARP: a host announcing its own IP-to-MAC mapping unprompted, used when an interface comes up or a virtual IP fails over between servers so neighbours update their caches at once.

IPv6 does not use ARP. It uses Neighbor Discovery (NDP), which does the same job with ICMPv6 multicast messages; ip -6 neigh shows that cache.

Duplicate IPs and ARP Debugging#

Two hosts configured with the same IP cause an intermittent failure that is maddening to debug: connections work, then break, as neighbours' caches flip between the two MACs. Check before assigning a static address:

sudo arping -D -I eth0 -c 2 192.168.1.50   # iputils arping: duplicate address detection

With the iputils version of arping, an exit status of 0 means no one answered and the address is free; a reply means it is already in use. Seeing the same IP under two different MACs in tcpdump -e arp output is the definitive evidence.

ARP Spoofing and Its Defences#

ARP has no authentication. Any host on the segment can send replies claiming to be the gateway, and victims will update their caches and send traffic to the attacker, who forwards it on: a classic man-in-the-middle on local networks. Tools such as Ettercap and Bettercap automate it, and it is worth reproducing in an isolated lab (two VMs and a virtual switch) to see the cache change.

Signs and defences:

• Detection: the gateway's MAC in ip neigh changes, or two IPs share one MAC. Tools such as arpwatch log these changes.
• Dynamic ARP Inspection (DAI) on managed switches drops ARP replies that do not match the DHCP snooping binding table.
• DHCP snooping only allows DHCP server replies on trusted ports, blocking rogue DHCP servers that hand out a fake gateway.
• Encryption above Layer 2 (TLS, SSH, a VPN) means an intercepting host sees ciphertext. This is the real fix for untrusted networks; see Network Security.
• Static ARP entries for critical gateways work on small networks but do not scale.

Switches vs. Hubs#

A hub repeats every incoming frame out every port, so all devices share one collision domain and one device's traffic is visible to everyone. A switch learns which MAC lives on which port and forwards frames only where they need to go. Its forwarding table (the MAC address table, or CAM table) is built by watching the source MAC of every frame.

A switch in a dozen lines of Python makes the logic concrete:

class LearningSwitch:
    def __init__(self, ports):
        self.ports = ports
        self.table = {}                       # MAC -> port

    def receive(self, in_port, src, dst):
        self.table[src] = in_port             # learn from the SOURCE address
        if dst in self.table and not describe_mac(dst)["multicast"]:
            return [self.table[dst]]          # known unicast: one port
        return [p for p in self.ports if p != in_port]   # unknown or broadcast: flood

sw = LearningSwitch(ports=[1, 2, 3, 4])
print(sw.receive(1, "aa:aa:aa:aa:aa:01", "ff:ff:ff:ff:ff:ff"))  # ARP request
print(sw.receive(3, "aa:aa:aa:aa:aa:03", "aa:aa:aa:aa:aa:01"))  # ARP reply
print(sw.receive(1, "aa:aa:aa:aa:aa:01", "aa:aa:aa:aa:aa:03"))  # data

[2, 3, 4]
[1]
[3]

The ARP request is flooded, the reply goes straight back to port 1 because the switch learned that MAC from the request, and from then on the conversation uses exactly one port each way. Real switches also age entries out (commonly after a few minutes) so moved devices are relearned. On a Linux bridge, bridge fdb show prints this table.

This also explains MAC flooding: filling the table with thousands of fake source MACs so the switch floods everything like a hub. Port security features that cap MACs per port are the defence.

VLANs#

A single physical switch can host many isolated logical networks using VLANs (IEEE 802.1Q). An access port belongs to one VLAN and sends untagged frames to the device. A trunk port carries many VLANs between switches or to a hypervisor, inserting a 4-byte tag with a 12-bit VLAN ID (so IDs 1-4094 are usable). Frames on different VLANs never mix at Layer 2; traffic between them must pass through a router or firewall, which is precisely where you enforce policy between guest Wi-Fi, corporate machines and IoT devices.

On Linux you can create a tagged sub-interface to test a trunk:

sudo ip link add link eth0 name eth0.20 type vlan id 20
sudo ip link set eth0.20 up
ip -d link show eth0.20        # shows "vlan protocol 802.1Q id 20"

A common mistake is a native-VLAN mismatch: the two ends of a trunk disagree about which VLAN travels untagged, so traffic silently lands in the wrong network.

Broadcast Domains and Loops#

ARP and DHCP rely on broadcasts, so an Ethernet network is a broadcast domain, bounded by routers. Ethernet frames have no TTL, so a physical loop between switches creates a broadcast storm: frames circulate forever, MAC tables thrash, and the network collapses within seconds. The Spanning Tree Protocol (and its faster successor, Rapid STP) prevents this by blocking redundant links, keeping exactly one active path and unblocking a backup when the primary fails. On switch ports facing end devices, "PortFast" plus "BPDU guard" (vendor names vary) let hosts come up quickly and shut the port if someone plugs a switch in and forms a loop.

The telltale signs of a loop are link lights blinking in unison, CPU spikes on every switch, and tcpdump showing the same broadcast frame repeated endlessly.

Why This Matters Above Layer 2#

A flapping cable, a duplicate IP, an ARP storm or a misconfigured VLAN presents as an intermittent outage that looks like an application bug. Knowing that the local hop is switch-and-MAC territory also explains why data centres use routed spine-leaf designs with overlays such as VXLAN (see Virtual Networks & Overlays) instead of giant flat Layer 2 networks, and why cloud providers hide all of it behind a virtual NIC that answers ARP for you.

Each hop across the internet repeats the same pattern: compare the destination with the local subnet, pick the next hop, resolve its MAC, frame the packet, send. ARP and switching are the mechanism that makes routing possible.

Practice#

On your own LAN, run sudo tcpdump -n -e arp in one terminal, then delete the gateway entry (sudo ip neigh del 192.168.1.1 dev eth0, adjusting the address and interface) and ping the gateway. Identify the broadcast request, the unicast reply, and the new REACHABLE entry in ip neigh.