Since when does ping have state?
I have a small Python service pinging 8.8.8.8 once a second to detect internet outages:
for host in get_config().ping.internet_hosts:
result = ping3.ping(host, timeout=get_config().ping.timeout)
if result is not None:
logger.debug("ping internet OK host=%s", host)
return NetworkCheckResult.UP
I disabled internet access on the router to test this but the pings kept succeeding. What the heck?!
- Running
ping 8.8.8.8from the CLI on the same host failed correctly. - Restarting the Python process made it fail correctly.
- Leaving Python running meant
pingcontinued to report success
Firewall state table
On the OPNsense box:
root@OPNsense:~ # pfctl -ss | grep 8.8.8.8
all icmp 8.8.8.8:8 <- 10.20.86.158:55172 0:0
all icmp 167.179.158.61:19000 (10.20.86.158:55172) -> 8.8.8.8:8 0:0
root@OPNsense:~ # pfctl -F state
2217 states cleared
The moment I flushed the states, Python started reporting the outage. So the packets weren’t being evaluated against the firewall rules at all - they were matching an existing state entry created back when the internet still worked, and states are checked before rules.
That explains why blocking had no effect. It doesn’t yet explain why the CLI behaved differently.
Python ping3 stable ICMP identifiers
pf keys an ICMP state on the identifier field in the echo request. Here’s how ping3 picks that value:
thread_id = threading.get_native_id()
process_id = os.getpid()
icmp_id = zlib.crc32(f"{process_id}{thread_id}".encode()) & 0xffff
It’s recomputed on every call, but the inputs never change while the process is alive. So every ping from my long-running service carried the identical identifier.
To pf, that isn’t a thousand separate probes. It’s one continuous flow. The state matched, so the packet skipped rule evaluation - and because traffic kept arriving every second, the state’s idle timer kept resetting and it never aged out.
The CLI behaves differently for a boring reason: each ping invocation is a new process with a new PID, so it gets a new identifier, matches no existing state, falls through to the actual rules, and gets blocked. Restarting Python works for exactly the same reason.
This identifier scheme isn’t a bug. It exists specifically so that pings from different threads and processes don’t collide with each other. It just happens to interact badly with stateful firewalls when you’re probing on a loop.
The fix
ping3.ping() doesn’t expose an id parameter:
(dest_addr, timeout=4, unit='s', src_addr='', ttl=None, seq=0, size=56, interface='', version=None)
But the functions it calls internally are public and do accept icmp_id. This is the code I ended up using, which fixes the problem:
import socket
import random
import ping3
def ping_random_id(dest_addr, timeout=4, size=56):
icmp_id = random.randint(0, 0xffff)
with socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_ICMP) as sock:
try:
sock.settimeout(timeout)
ping3.send_one_ping(sock=sock, dest_addr=dest_addr, icmp_id=icmp_id, seq=0, size=size)
delay = ping3.receive_one_ping(sock=sock, icmp_id=icmp_id, seq=0, timeout=timeout)
except Exception as e:
logger.debug("caught and ignored ping error: %s", e)
# typically means timeout
return None
return delay # seconds
Won’t a new ID every second flood the state table?
ICMP states expire on a short idle timer — the FreeBSD pf defaults are icmp.first 20s and icmp.error 10s. If an identifier is never seen again, its state just ages out. So pinging once a second with a fresh id gives you roughly 10–20 concurrent states per monitored host at any instant, constantly being replaced. Across a handful of hosts that’s under a hundred entries, in a table typically sized for hundreds of thousands.
If you want to confirm it for yourself, watch pfctl -ss | grep icmp | wc -l for a few minutes after the change. It should hover, not climb.
Claude’s final thoughts
A monitoring probe that looks identical to itself on every run is indistinguishable from an established, already-permitted flow. That’s a nice property for real traffic and a terrible one for a health check.
Yes, most of this article was written by Claude. He also did most of the research and troubleshooting. AI slop? You tell me. I wanted to publish my notes in case this comes up again.