{"id":70,"date":"2026-08-29T10:52:17","date_gmt":"2026-08-29T10:52:17","guid":{"rendered":"https:\/\/networkguard360.com\/?p=70"},"modified":"2026-08-29T10:52:17","modified_gmt":"2026-08-29T10:52:17","slug":"f5-vip-reachable-application-down","status":"publish","type":"post","link":"https:\/\/networkguard360.com\/?p=70","title":{"rendered":"Application Down Behind F5: VIP Reachable, Pool Healthy – So Where Did the Traffic Stop?"},"content":{"rendered":"

Category:<\/strong> F5
\nTechnologies:<\/strong> F5 BIG-IP LTM, FortiGate, TCP, SNAT
\nIncident type:<\/strong> Application connectivity
\nDifficulty:<\/strong> Intermediate<\/p>\n

\n

Sanitization notice:<\/strong> The IP addresses are
\nrepresentative private values. Customer names, hostnames, credentials,
\nand sensitive topology details have been removed or changed.<\/p>\n<\/blockquote>\n

Incident summary<\/h2>\n

The application team reported that their service was down. Connection
\ntests to TCP ports 20055<\/strong>, 30055<\/strong>, and
\n40055<\/strong> failed through the F5 VIP
\n10.200.252.190<\/code>.<\/p>\n

FortiGate logs showed a server reset (RST)<\/strong>, which
\ninitially suggested a backend issue. The server team found something
\ndifferent: all three ports were listening, but no connection attempt
\nreached the servers and no session entered the ESTABLISHED<\/code>
\nstate.<\/p>\n

The pool was healthy and the VIP was receiving traffic. The
\ninvestigation therefore focused on one question: where did the
\nflow stop between the F5 and the backend servers?<\/strong><\/p>\n

Environment<\/h2>\n\n\n\n\n\n\n\n\n\n
Component<\/th>\nDetails<\/th>\n<\/tr>\n<\/thead>\n
F5 BIG-IP LTM<\/td>\nVIP 10.200.252.190<\/code><\/td>\n<\/tr>\n
F5 internal addresses<\/td>\n172.22.72.251-254<\/code><\/td>\n<\/tr>\n
Backend servers<\/td>\n192.168.10.20-23<\/code><\/td>\n<\/tr>\n
Application ports<\/td>\nTCP 20055<\/code>, 30055<\/code>, 40055<\/code><\/td>\n<\/tr>\n
Firewall<\/td>\nFortiGate<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Traffic path<\/h2>\n
Client -> FortiGate -> F5 VIP -> F5 server-side leg -> FortiGate -> Backend server<\/code><\/pre>\n
This path contains two separate flows:<\/p>\n
    \n
  1. Client to F5 VIP<\/li>\n
  2. F5 to backend server<\/li>\n<\/ol>\n
    Proving that the first flow works does not prove that the second flow
    \nis permitted.<\/p>\n
    Investigation<\/h2>\n
    1. Check the VIP and
    \nsource translation<\/h3>\n
    The virtual server, pool, and SNAT behavior were reviewed. The server
    \nteam then checked for connection attempts from the expected F5-side
    \nsource addresses.<\/p>\n
    No traffic was visible on the backend servers.<\/p>\n
    Finding:<\/strong> The servers were not rejecting the
    \nconnection; they were not receiving it.<\/p>\n
    2. Confirm that traffic
    \nreaches the F5<\/h3>\n
    A packet capture was taken on the F5 for the VIP and affected
    \nports:<\/p>\n
    \n<\/a>tcpdump<\/span> -nni<\/span> 0.0:nnn <\/span><\/span>\n<\/a>  'host 10.200.252.190 and (port 20055 or port 30055 or port 40055)'<\/span><\/span><\/code><\/pre>\n<\/div>\n
    The capture showed client packets arriving at the F5 correctly.<\/p>\n
    Finding:<\/strong> Client routing and firewall policy up to
    \nthe VIP were working.<\/p>\n
    3. Trace the server-side flow<\/h3>\n
    The next capture focused on the backend network:<\/p>\n
    \n<\/a>tcpdump<\/span> -nni<\/span> 0.0:nnn <\/span><\/span>\n<\/a>  'net 192.168.10.0\/24 and (port 20055 or port 30055 or port 40055)'<\/span><\/span><\/code><\/pre>\n<\/div>\n
    The expected connection did not reach the backend servers. The
    \nfailure was on the internal leg after the F5 accepted the client
    \nrequest.<\/p>\n
    4. Eliminate common F5 causes<\/h3>\n\n\n\n\n\n\n\n\n\n\n
    Check<\/th>\nResult<\/th>\n<\/tr>\n<\/thead>\n
    Route to backend network<\/td>\nCorrect<\/td>\n<\/tr>\n
    Internal VLAN and trunk<\/td>\nCorrect<\/td>\n<\/tr>\n
    Self IP and Floating IP<\/td>\nCorrect<\/td>\n<\/tr>\n
    Pool and pool members<\/td>\nCorrect<\/td>\n<\/tr>\n
    Backup member\/port settings<\/td>\nCorrect<\/td>\n<\/tr>\n
    Pool member state<\/td>\nUp<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
    Nothing on the F5 explained why the backend servers saw no SYN
    \npackets.<\/p>\n
    5. Do not rely only on pool
    \nhealth<\/h3>\n
    The pool showed UP<\/strong>, but the monitor did not prove
    \nthat production traffic on all three application ports was allowed end
    \nto end.<\/p>\n
    Monitor traffic can use a different source address, port, policy, or
    \npath. A successful monitor proves only that the configured monitor
    \nsucceeded.<\/p>\n
    6.
    \nRecheck the firewall using the actual server-side flow<\/h3>\n
    The FortiGate logs and policies were reviewed again with the correct
    \nvalues:<\/p>\n
      \n
    • Source:<\/strong> F5 internal addresses
      \n172.22.72.251-254<\/code><\/li>\n
    • Destination:<\/strong> Backend servers
      \n192.168.10.20-23<\/code><\/li>\n
    • Ports:<\/strong> TCP 20055<\/code>, 30055<\/code>,
      \n40055<\/code><\/li>\n<\/ul>\n
      This exposed the missing part of the design.<\/p>\n
      Root cause<\/h2>\n
      The FortiGate allowed client traffic to reach the F5 VIP, but there
      \nwas no policy permitting the separate connection from the F5 internal
      \naddresses to the backend servers on the required application ports.<\/p>\n
      The firewall terminated the downstream flow. This produced the reset
      \nseen in the logs, even though the backend servers never received the
      \nconnection.<\/p>\n
      Resolution<\/h2>\n
      A FortiGate policy was added with the following scope:<\/p>\n\n\n\n\n\n\n\n\n\n
      Policy field<\/th>\nValue<\/th>\n<\/tr>\n<\/thead>\n
      Source<\/td>\nF5 internal addresses 172.22.72.251-254<\/code><\/td>\n<\/tr>\n
      Destination<\/td>\nBackend servers 192.168.10.20-23<\/code><\/td>\n<\/tr>\n
      Service<\/td>\nTCP 20055<\/code>, 30055<\/code>, 40055<\/code><\/td>\n<\/tr>\n
      Action<\/td>\nAccept<\/td>\n<\/tr>\n
      Logging<\/td>\nEnabled<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
      The rule was limited to the required sources, destinations, and ports
      \nrather than using broad ANY<\/code> objects.<\/p>\n
      Verification<\/h2>\n
      After the policy was applied:<\/p>\n
        \n
      • TCP connection tests succeeded on all three ports.<\/li>\n
      • F5 captures showed traffic on both client-side and server-side
        \nlegs.<\/li>\n
      • FortiGate logs showed the new policy accepting the sessions.<\/li>\n
      • Backend servers received the connections and established
        \nsessions.<\/li>\n
      • The application team confirmed that the service was working
        \nnormally.<\/li>\n<\/ul>\n
        What made this incident
        \nmisleading?<\/h2>\n\n\n\n\n\n\n\n\n
        Observation<\/th>\nIncorrect assumption<\/th>\nCorrect interpretation<\/th>\n<\/tr>\n<\/thead>\n
        FortiGate reported a server reset<\/td>\nThe backend server sent the RST<\/td>\nAn intermediate device may terminate the flow<\/td>\n<\/tr>\n
        Pool members were UP<\/td>\nApplication traffic must be allowed<\/td>\nOnly the configured monitor was successful<\/td>\n<\/tr>\n
        VIP received client packets<\/td>\nThe complete path was working<\/td>\nOnly the client-to-VIP leg was proven<\/td>\n<\/tr>\n
        Server ports were listening<\/td>\nThe server should establish a session<\/td>\nThe SYN still had to reach the host<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
        Key lessons<\/h2>\n
        Trace each segment
        \nseparately<\/h3>\n
        Client -> firewall -> VIP -> F5 server side -> firewall -> backend<\/code><\/pre>\n
        At each step, confirm both packet arrival and departure.<\/p>\n
        Verify the source
        \naddress seen by the backend<\/h3>\n
        Depending on the F5 design, the server may see a SNAT, automap, Self
        \nIP-related, or other translated source. Firewall rules must match the
        \nactual server-side source.<\/p>\n
        A healthy pool is not
        \nan end-to-end test<\/h3>\n
        Monitor success does not validate every production source,
        \ndestination port, firewall policy, or return path.<\/p>\n
        Confirm who generated the
        \nreset<\/h3>\n
        Do not rely only on the session-end label in a firewall log.
        \nCorrelate captures and logs to locate the packet carrying the RST.<\/p>\n
        Prevention checklist<\/h2>\n
          \n