Web Proxies
Although proxy servers support many protocols,
the most common outbound filtering system is a web proxy server,
which can inspect and manipulate HTTP and HTTPS connections.
Simply put, web proxy servers accept and forward web traffic
on behalf of a client, for example, a web browser.
This is often done in a Network Address Translation environment,
in which the internal private source IP addresses
are translated into Internet-routable addresses.
If a user on an internal network requests an external web-based resource,
and the network enforces the use of a proxy,
the request will be sent to the proxy server,
which will terminate the connection
and initiate a new one to the outside world.
In this figure,
the Workstation client sends a web request to www.example.com,
but because of the proxy configuration,
the request is actually sent to the proxy server,
with a destination address of 10.0.0.254 first.
The proxy server will then NAT the connection,
setting the source IP to its own public IP,
and setting the destination IP to the real IP of the server
hosting www.example.com.
The proxy passes on the GET request for index.html
to www.example.com
and may also read, insert, delete, or modify HTTP headers
such as the User-Agent.
By acting as a Man-In-The-Middle,
a web proxy is an excellent single-unit defensive tool
that can perform URL and IP filtering
and HTTPS inspection.
For example, it could block traffic based on fields such as the User-Agent
to disallow certain browsers.
It can also actively modify data within a connection including the HTTP headers.
Similar to a DNS filter, a web filter can inspect and manipulate full URLs
and is database-driven, filtering by blocklists or categories.
If a URL is disallowed, the proxy will often return a block page.
Even if the traffic is allowed, the request details,
like common HTTP headers as well as the request method and resource path
will almost certainly be logged.
If the company uses a central log server with a Security Information
and Event Management system,
the proxy logs might be subject to a second review
and if something is suspicious, an alert might be generated.
Since this could jeopardize our penetration test, we must tread carefully
and employ a variety of bypass, obfuscation,
and normalization techniques on our web-based traffic.
In the next section, we’ll explore a few of these techniques.