GlobalProtect Complete Failure — 'Entire Remote Workforce Disconnected'

The Call Comes In

"Every remote employee is calling — nobody can connect to VPN"
"GlobalProtect shows connected but no internal resources are reachable"
"Portal loads fine but the gateway fails with authentication error"
"Users stuck at Connecting... for minutes before it times out"
"Only our Europe region is affected — US users are fine"
"Started immediately after we renewed the SSL certificate last night"
"Some users connect but after 10 minutes they drop and cannot reconnect"

Scale of This Outage

A GlobalProtect outage in 2025 means the entire remote workforce cannot work. For companies that went fully remote or hybrid, this is equivalent to the office building losing power. Every minute of outage is quantifiable business loss. TAC treats this as the highest severity because there is no manual workaround — users cannot just walk in.

GlobalProtect Architecture — Know This Before Debugging

GlobalProtect has two distinct components on the firewall side. They are independent and can fail independently. Confusing them wastes time.

Component	What It Does	Port	Where It Runs
Portal	Delivers configuration to GP clients — which gateways to connect to, authentication methods, agent settings	443 (HTTPS)	Any interface — usually the external interface
Gateway	Terminates the actual VPN tunnel, authenticates users, enforces HIP policy, routes traffic to internal networks	443 (HTTPS) + IPSec 4501 (UDP)	External interface(s)

First Question in Every GP Outage

Is the portal reachable? Is the gateway reachable? Are both failing or just one? This single distinction eliminates half the possible root causes immediately. If only the gateway fails but portal works, the issue is in gateway authentication, certificates, or routing. If both fail, the issue is at the network layer — firewall interface unreachable, DNS failure, or a firewall service crash.

What Most Engineers Try First (And Why It Fails)

Reinstall GlobalProtect client — client is fine, server-side configuration or certificate is the issue
Reboot the user laptop — same result, problem is on the firewall or gateway side
Restart the GlobalProtect gateway service — may temporarily fix a crashed service but does not address root cause
Check the security policy — policy is not typically the cause of a sudden complete outage
Blame the ISP — ISP delivering traffic correctly, issue is in authentication or certificate chain

The Actual TAC Debug Sequence

Step 1 — Isolate Portal vs Gateway

pan-os-cli

# Test portal reachability from the firewall itself
# (Replace with your portal FQDN or IP)
test connection protocol https host portal.company.com port 443

# Check portal configuration
show global-protect portal config

# Check portal status
show global-protect portal statistics

# Test gateway reachability
test connection protocol https host gateway.company.com port 443

# Check all gateway configurations
show global-protect gateway config

# Check gateway statistics — active tunnels, authentication failures
show global-protect gateway statistics

Step 2 — Check Gateway Authentication Log

pan-os-cli

# Check authentication logs for failure reason
show log authentication direction equal forward   | match "globalprotect|Authentication failed"

# Common authentication failure messages:
# "Authentication failed: invalid credentials" — LDAP/RADIUS issue
# "Authentication failed: user not found" — directory sync issue
# "Authentication failed: MFA timeout" — MFA server unreachable
# "Authentication failed: certificate error" — client cert or server cert issue

# Check if RADIUS or LDAP server is reachable
test authentication authentication-profile <profile-name>   username <test-user> password

Step 3 — Certificate Validation (Most Common After Cert Renewal)

pan-os-cli

# When a certificate renewal breaks GlobalProtect:
# The new certificate may have different properties than the old one
# OR the certificate chain is incomplete
# OR the private key did not import correctly

# Check the certificate currently bound to the gateway
show config running | match "ssl-tls-service-profile"

# Check certificate expiry and chain
show certificate <certificate-name>

# Test the certificate from an external perspective
# (Run from a machine outside, or check from client logs)

# Verify certificate chain completeness
# A certificate without its intermediate CA will cause failures in strict clients
# Check: leaf cert → intermediate CA → root CA must be present on firewall

# Check if private key matches certificate
show system state | match "certificate"

The Certificate Renewal Trap

This is one of the most common GP outage patterns. A new SSL certificate is installed, but the intermediate CA certificate is missing from the certificate chain uploaded to the firewall. The firewall presents an incomplete chain to GP clients. Strict clients (corporate managed devices with enforced chain validation) reject it. Fix: always upload the full certificate chain including all intermediate CAs, not just the leaf certificate. Verify with an SSL chain checker before applying to production.

Step 4 — Check HIP (Host Information Profile) Policy

pan-os-cli

# HIP mismatches silently deny tunnel establishment
# Users connect but are placed in a limited-access network segment
# or tunnel is established but all traffic is denied by HIP-based security policy

# Check HIP profiles configured
show global-protect gateway hip-profile

# Check HIP reports for specific users
show global-protect gateway current-user

# Check if HIP policy is enforcing and what it checks
show config running | match "hip-profile|host-info"

# Common HIP failures after a software update:
# - OS version check fails (Windows updated past the allowed version)
# - Antivirus definition date check fails
# - Disk encryption check fails on new device enrollment

# Temporarily disable HIP enforcement to confirm it is the cause:
# (Do this in a test rule — never in production without change management)

Step 5 — Review GP Client Logs

When the server-side CLI gives incomplete information, the client-side logs contain the exact TLS handshake failure messages and authentication exchange details. These are the most specific.

Platform	Log Location
Windows	C:\Program Files\Palo Alto Networks\GlobalProtect\PanGPS.log (service log)
Windows	C:\Program Files\Palo Alto Networks\GlobalProtect\PanGPA.log (agent/UI log)
macOS	/Library/Application Support/PaloAltoNetworks/GlobalProtect/PanGPS.log
macOS	~/Library/Logs/PaloAltoNetworks/GlobalProtect/PanGPA.log
iOS/Android	Export logs from GP app settings → Send logs

pan-os-cli

# Key strings to search in PanGPS.log:
# "SSL_connect failed" — TLS handshake to gateway failed
# "certificate verify failed" — client cannot validate gateway cert
# "Authentication failed" — credential or MFA issue
# "gateway selection" — gateway selection algorithm issues
# "HIP check failed" — HIP compliance blocking tunnel
# "route conflict" — local route on client device overlapping with VPN routes

Root Cause Patterns

Pattern 1: Expired Certificate (Most Common)

→

Certificate on the portal or gateway expired. GP clients show a certificate error. Fix: renew and install certificate immediately. Always set calendar reminders 60 and 30 days before GP certificate expiry — this should never be a surprise outage.

Pattern 2: Split Tunnel Misconfiguration

VPN shows as connected but users cannot reach internal resources. This is almost always a split tunnel routing issue. The GP tunnel is up, but the route for the internal subnet is either not included in the split tunnel include list, or incorrectly configured on the gateway.

pan-os-cli

# Check split tunnel configuration on the gateway
show config running | match "split-tunnel|include-list|exclude-list"

# Check what routes are being pushed to connected clients
show global-protect gateway current-user user <username>

# Verify internal routes are in the include list
# If using include-list: only listed routes go through VPN
# If using exclude-list: everything goes through VPN except listed routes

# Common mistake: 10.0.0.0/8 in exclude list when internal servers are in 10.x range
# This pushes internal traffic to the internet directly, bypassing VPN

Pattern 3: DNS Resolution Failure Inside Tunnel

VPN is connected, routes are correct, but applications cannot resolve internal hostnames. Users can ping internal IPs but not internal FQDNs.

pan-os-cli

# Check DNS server pushed to GP clients
show config running | match "dns-server"

# Verify DNS split tunneling configuration
# Internal FQDNs should resolve via internal DNS (pushed through tunnel)
# External FQDNs can resolve via external DNS

# Common issue: GP pushes internal DNS IP but the internal DNS IP
# is not in the split tunnel include routes
# Traffic to the DNS server IP goes to the internet, times out

Pattern 4: Gateway Selection Algorithm Sending Users to Wrong Gateway

Users in one region connect to a geographically distant gateway because the preferred local gateway is down or not responding to health checks. Performance suffers severely, and some applications time out.

→

Fix: Check gateway health check configuration. If a gateway is failing health checks, identify and fix that gateway. Do not permanently disable the priority gateway — that removes geographic redundancy.

Validation After Fix

pan-os-cli

# Confirm gateway is accepting connections after fix
show global-protect gateway statistics
# Look for: active-tunnels increasing, no authentication failures

# Confirm specific user is connected and getting correct routes
show global-protect gateway current-user user <test-username>

# Confirm tunnel is routing traffic correctly
# Have the connected user run: ping <internal-server-ip>
# Then check session table on firewall for their traffic
show session all filter source <user-tunnel-ip>

# Run for 10 minutes — watch for drops or disconnections
# Certificate and auth issues typically manifest within first 2 minutes
# HIP issues may take up to the HIP check interval (often 60 seconds)

Previous Module Next Module