thesis

HTTP Proxy Server
A Dissertation submitted to
Rajiv Gandhi University of Knowledge Technologies
in partial fulﬁllment of the degree of
Bachelor of Technology
in
Computer Sciences and Engineering
By
U.Venkata Jaswanth, R092480
K.Hari Krishna, R092848
M.Aneel Kumar, R092514
Department of Computer Sciences
Rajiv Gandhi University of Knowledge Technologies
RK Valley, Kadapa - 516330
Andhra Pradesh, India

CERTIFICATE
This is to certify that the dissertation entitled “HTTP Proxy Server” submitted by
U.Venkata Jaswanth (R092480), K.Hari Krishna (R092848), M.Aneel Kumar
(R092514), in partial fulﬁllment of the requirements for the award of Bachelor of Tech-
nology in Computer Science is a bonaﬁde work carried out by him under my supervision
and guidance.
The dissertation has not been submitted previously in part or in full to this or any other
University or Institution for the award of any degree or diploma.
Md.Abdul Aziz,
Department Coordinator,
Department of CSE,
RGUKT, R.K.Valley.
Siva Rama Sastry Gumma,
Project Supervisor,
Department of CSE,
RGUKT, R.K.Valley.

DECLARATION
We U.Venkata Jaswanth, K.Hari Krishna, M.Aneel Kumar hereby declare that this Dis-
sertation entitled “HTTP Proxy Server” submitted by us under the guidance and su-
pervision of Siva Rama Sastry Gumma is a bonaﬁde work. We also declare that it has
not been submitted previously in part or in full to this University or other University or
Institution for the award of any degree or diploma.
Date:
Place:
(U.Venkata Jaswanth)
Reg. No. R092480
(K.Hari Krishna)
Reg. No. R092848
(M.Aneel Kumar)
Reg. No. R092514

To,
RGUKT and The Open Source Community

Acknowledgments
We would like to express our sincere gratitude to Siva Rama Sastry Gumma, our
project supervisor, for valuable suggestions and keen interest through out the progress of
our course of project.
With Sincere Regards,
U.Venkata Jaswanth,
K.Hari Krishna,
M.Aneel Kumar.
iv

Abstract
A proxy server is a vital component of a computer network. It has different uses like
firewall for security, content filter for regulating access to the Internet, user authentication
for Internet access, user quota management to limit maximum data consumed by a user,
web accelerator for increasing the speed and reducing the consumption of bandwidth by
caching web content. They are also used to hide the identity of users on the Internet.
Simply, a proxy server acts an intermediary between a client and a server. Our project
is to create a proxy server with enhanced security and flexible access control mechanisms.
This software is fully independent of the underlying operating system. This has been
optimized in every aspect to increase performance on crowded networks.
v

Contents
Acknowledgments iv
Abstract v
Abbreviations viii
1 Introduction 1
1.1 Introduction to Proxy Servers . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 User authenticated Internet access . . . . . . . . . . . . . . . . . . . 1
1.1.2 User quota management . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.3 Caching of resources . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.4 Access control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1.5 Logging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Organization of Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Literature Survey 4
2.1 Review of existing solutions . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.1.1 SQUID cache proxy server . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 How HTTP helps in implementing proxy features . . . . . . . . . . . . . . 5
2.2.1 User authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2.2 User quota management . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.3 Caching of resources . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.2.4 Access control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
vi

3 Phoenix proxy server 8
3.1 Problem Solving Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1.1 Implementation of authentication . . . . . . . . . . . . . . . . . . . 8
3.1.2 Implementation of access control . . . . . . . . . . . . . . . . . . . 9
3.1.3 Implementation of caching . . . . . . . . . . . . . . . . . . . . . . . 9
3.1.4 Implementation of quota management . . . . . . . . . . . . . . . . 9
3.1.5 Implementation of Logging . . . . . . . . . . . . . . . . . . . . . . . 10
3.2 Configuration of the Phoenix proxy server . . . . . . . . . . . . . . . . . . 10
3.2.1 Master configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.2.2 Authentication configuration . . . . . . . . . . . . . . . . . . . . . . 12
3.2.3 Quota configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 12
3.2.4 Access control configuration . . . . . . . . . . . . . . . . . . . . . . 12
3.2.5 Log configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2.6 Merits of using filesystem . . . . . . . . . . . . . . . . . . . . . . . 13
4 Phoenix proxy client 14
4.1 Problem to be solved . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.2 Working of the proxy client . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.3 Interface of the proxy client . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.4 Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5 Web interface for proxy management 16
5.1 Intefaces for users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.2 IPC between web interface and the proxy server . . . . . . . . . . . . . . . 19
5.3 IPC message formats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.3.1 Changes in credentials . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.3.2 Changes in access control lists . . . . . . . . . . . . . . . . . . . . . 20
5.3.3 Changes in user quota limits . . . . . . . . . . . . . . . . . . . . . . 20
6 Conclusion and Future Work 21
Bibliography 22
vii

Abbreviations
HTTP Hyper Text Transfer Protocol
HTTPS Hyper Text Transfer Protocol Secure
FTP File Transfer Protocol
TCP Transmission Control Protocol
LRU Least Recently Used
IPC Inter Process Communication
MB Mega Bytes
KB Kilo Bytes
viii

Chapter 1
Introduction
1.1 Introduction to Proxy Servers
A proxy server is mainly used to regulate Internet access. It is primarily used in large
corporations, universities etc.,
Prime uses of proxy servers are :
• User authenticated Internet access
• User quota management
• Caching of resources
• Access control
• Logging
1.1.1 User authenticated Internet access
Proxy servers can be conﬁgured to allow authorized users only. This makes the resources
accessible to authorized users only. This feature paves way for other features which require
user identiﬁcation.
1

1.1.2 User quota management
Using user identification, we can track data usage by each individual user. We can also
pose a limit on the data used by all users for a day. Thus, we can eliminate monopoly and
provide uniform service to all users.
1.1.3 Caching of resources
Proxy server can cache frequently used resources. This single feature is very important in
large networks. Usually users mostly visit same websites. All access have some temporal
relation, which we can exploit to provide enhanced performance. Caching frequently used
objects, can eliminate re-retrieval of them from servers, which is a very costly operation.
This usually saves lot of bandwidth on large networks, which is also economical.
1.1.4 Access control
Access control provides a way to regulate Internet access. We can actually control which
websites are allowed for the users inside the network. Using this, we can block unwanted
websites inside our network. Basically, this is done in two ways. In whitelisting, we actually
maintain a list of allowed websites. Everything else is blocked. In blacklisting, we maintain
a list of blocked websites. Everything else is allowed. As the Internet hosts billions of
websites, the choice heavily depends on the purpose.
1.1.5 Logging
Logging is a very useful feature. This can be used to provide detailed statistics of web
access by all users. It is mainly used to trace activities of users and detect suspicious
activities.
1.2 Motivation
Existing proxy server solutions are clumsy to configure or deploy. Generally, administrators
need simple features. Bundling simple features with our software greatly simplifies the code
2

base and also increases performance of the system. All features should be easily conﬁgurable
by an average user. Our solution, provides required features in an eﬃcient way.
1.3 Organization of Thesis
The rest of this thesis is organized as follows: Chapter 2 gives literature survey about how
HTTP works and how the features of proxy server are implemented. Chapter 3 gives in
depth explanation of our solution. Chapter 4 explains the proxy client. Chapter 5 explains
the web interface.
3

Chapter 2
Literature Survey
2.1 Review of existing solutions
2.1.1 SQUID cache proxy server
SQUID cache proxy server is an open source software that is widely used. It runs on both
Windows and Linux. Support for Windows is not active. It is completely written in C++.
It supports access control, cache management, logging, user quota management. It can
also be used to create hierarchies of proxy servers, commonly known as proxy chaining.
It uses files to store all configuration related details. It is controlled by directly editing
the configuration files. It is also very efficient. It can also be configured as a reverse proxy.
It acts as a front end for the server and serves the requests of clients with the cache of
previous requests. Thus, it helps to decrease load on servers.
Figure 2.1: SQUID proxy logo
4

2.2 How HTTP helps in implementing proxy features
2.2.1 User authentication
By default, browsers do not use any authentication schemes while sending HTTP requests
to the proxy server. The browser has to be challenged by a 407 Proxy Authentication
required response to tell it that the proxy needs authentication. Browsers have a built-in
mechanism to prompt for user name and password when they get challenged by the proxy
server. When the user enters the required credentials, browser stores it in memory and
adds Proxy-Authorization header to every HTTP request it sends to that proxy server.
Request:
GET http://www.rgukt.in HTTP/1.1
Response:
HTTP/1.1 407 Proxy authentication required
Proxy-Authenticate: Basic realm="Phoenix Proxy Server"
The Proxy-Authenticate header informs the client, the authentication scheme the
server is willing to use and also a message to display in the browser prompt. The browser
then sends the credentials in the required scheme to the server.
Request:
GET http://www.rgukt.in HTTP/1.1
Proxy-Authorization: Basic QskfMk4AKC==
The Basic authentication scheme encodes the username:password combination in
Base64 format.
5

2.2.2 User quota management
HTTP adds Content-Length header to every request and response for which it knows
the size in advance. Else it uses chunked-encoding transfer and sends response in blocks
with block size preceeding each block. Size is represented in bytes. We can use it to track
data consumption.
Request:
GET /logo.png HTTP/1.1
Host: www.rgukt.in
Response:
HTTP/1.1 200 OK
Content-Length: 12445
---Data ----
2.2.3 Caching of resources
HTTP adds Etag header for static web resources like images, text, html, js, css files. It
doesn’t add it for files that generate dynamic output like php, jsp etc.,
Request:
Host: www.rgukt.in
Response:
HTTP/1.1 200 OK
Etag: "ahn3rkjd4em9i4s"
Content-Length: 12445
The Etag value uniquely identifies a resource in a web server. Instead of retrieving the
6

full resource, we can simply ask the web server whether the resource has been modified or
not. If it is not modified, then we can serve a local copy instead of downloading it again.
The validation mechanism is done as follows.
Request:
If-None-Match: "ahn3rkjd4em9i4s"
Host: www.rgukt.in
Response:
HTTP/1.1 304 Not Modified
If the specified resource is not modified then the server sends a 304 Not Modified
response. We can serve the local copy. If it is modified the server sends the complete
resource.
2.2.4 Access control
A web browser sends full HTTP resource url in the initial line of the request itself. It is
not HTTP/1.1 compliant. We can use that resource url to determine whether the request
can be allowed or not.
Request:
GET http://en.wikipedia.org/ HTTP/1.1
Host: en.wikipedia.org
7

Chapter 3
Phoenix proxy server
3.1 Problem Solving Approach
Efficiency is very important in real-time systems. To gain efficiency we should remove
unnecessary features. This greatly enhances security of the system. Software engineering
principles say that 90 percent of code is only used 10 percent of the time. So, we
must concentrate on those 10 percent of lines that are used 90 percent of time. We need
to focus on those lines of code that run for every request.
They are authentication, access control, caching, quota management.
3.1.1 Implementation of authentication
Authentication maintains a cache of Proxy-Authorization header values to make very
fast decisions. When a new authorization header is found, it decodes it according to the
authentication scheme. If it is valid, then it is added to the cache to prevent repeating this
costly operation again.
If the user is using a web browser and directly connects to the proxy server, then it
sends Proxy-Authorization: Basic LDnfsldjfDFSf== header. Phoenix decodes the
Base64 encoded string and verifies the user:password combination. If the user uses the
client software to connect to the proxy server, then it sends Proxy-Authorization: Cus-
tom lSdfklDJFsdjfl== which is the encoded form of user:SHA1HashedPassword to
the server.
8

3.1.2 Implementation of access control
Access control in Phoenix proxy server is done using domain names of websites only. This
is because, whenever we access a website, it usually loads several other files from the same
domain or other servers. Moreover, we do not have any control over the contents and links
in the website. So, it is best if we do not interfere with them.
So, domain based access control is neutral to changes in the website’s structure. Phoenix
supports regular expression * in domain names.
E.g. www.google.com, *.rgukt.in etc.,
Phoenix supports role based access control. This is a very essential feature in large
organizations. Every user is assigned a specific role. A user can only access domains
allowed for his role only.
Access control algorithm is implemented using a Trie data structure. It builds a Trie
using reversed domain names. So, it essentially checks suffixes in domain names. This is
done to allow use of the * regular expression. It is faster than a hash table. In worst case it
runs in O(length of domain name) time. In average case it runs in O(half the length
of domain name) time. In best case it runs in O(1) time.
3.1.3 Implementation of caching
Caching is implemented using a hash table. All HTTP responses which are eligible for
caching are saved in the hash table with the resource as its key. Every cache item is
validated according to its criteria before serving it. Cache replacement is done if the size of
all cached items reach a certain limit. Phoenix uses LRU cache replacement algorithm. It is
chosen because, many web requests have a temporal relation. For example, in the morning
many users prefer visiting news paper sites than entertainment sites. We can exploit this
relation and improve performance of the network and at the same time reduce bandwidth
consumption.
3.1.4 Implementation of quota management
Quota management is done by tracking the data usage of each user. Each user has an
individual limit. Whenever the user hits his quota limit, Phoenix sends a quota exceeded
9

error page. Quota resets after every 24 hours, at the reset point set by the administrator.
3.1.5 Implementation of Logging
Phoenix logs every request into log files. It logs the timestamp, user name, client ip address,
resource retrieved, cache status, uploaded data, downloaded data.
3.2 Configuration of the Phoenix proxy server
3.2.1 Master configuration
Phoenix has built-in default configuration settings. If we start the server without passing
any command line options, it falls back to the default mode. If the server is passed its
home folder as the command line option, then it loads the configuration file in its home
folder.
The master configuration file contains settings as key and values. All comments start
with #. Every aspect of the proxy server can be controlled using this master configuration.
Settings in the master configuration:
# configuration file for the Phoenix Proxy Server
# all times are represented in hh:mm:ss
# all data sizes are represented as KB, MB, GB, TB etc.,
# port for proxy server
proxy server port 3128
# port for IPC server
ipc server port 9090
# enable IPC server
enable ipc server true
# number of roles
10

number of roles 2
# maximum number of concurrent threads
max concurrent threads 4
# log file path
log file /home/user/.phoenix/logs/phoenix.log
# credentials file path
credentials file /home/user/.phoenix/credentials
# quota file path
quota file /home/user/.phoenix/quotas/quota
# quota reset point
quota reset point 00:00:00
# maximum server cache size
max cache size 100MB
# maximum amount of file to cache
max cache item size 10MB
# quota dump interval
quota dump interval 00:05:00
# stream buffer size
stream buffer size 64KB
# maximum single log file size. After reaching this limit another file will
be created
11

max log file rotate size 40MB
3.2.2 Authentication configuration
All credentials are loaded from the credentials file. Every line in the credentials file contains
a user name, role of the user and his hashed password. Role is determined by a number
between 0 and K, where K denotes the maximum number of roles.
File: credentials
user1 0 idfndf92405eb6219f14f3fa195seehdbs4s5ebsb
user2 1 sfd1769240sdfe219f14f3as195sbdsbd4e5db2ds
3.2.3 Quota configuration
Quota details of all users is loaded from the quotas file. Every line contains user name and
his allotted quota for the day.
Quota management is completely done in memory. Data usage of each user is saved
in a hash table. This list is periodically synced to the file system to recover from system
failures.
File: quotaLimits
user1 200MB
user2 100MB
3.2.4 Access control configuration
Access control details are loaded from different files, each for a role. Each file has the
name acl [0-K], where K denotes the maximum number of roles. Each file contains a list
of domains allowed.
12

File: acl 0
∗.google.com
∗.rgukt.in
slashdot.org
3.2.5 Log configuration
Log is completely written to files. A maximum file size limit can be set by the user. If a
log file reaches the maximum limit, it is rotated. It means, it is renamed as an old file and
a new file is used afterwards.
File: phoenix.log:
2015-05-03 22:04:08.192,user1,127.0.0.1,http://localhost/xampp/navi.php,0,384,2660
2015-05-03 22:04:08.195,user2,127.0.0.1,http://localhost/xampp/start.php,0,385,1776
2015-05-03 22:04:49.191,user1,127.0.0.1,http://localhost/xampp/favicon.ico,0,403,1343
3.2.6 Merits of using filesystem
Phoenix does everything using the underlying filesystem only. It is shielded from actual
platform’s filesystem by the Java Runtime. All HTTP requests take few hundred millisec-
onds of time to complete. If we access a database for every request, then the overhead
of accessing database crosses more than 40 percent of the total HTTP transaction time.
Because, every database access is again a request sent through the busy network. So, di-
rect database access is abandoned. Using a filesystem is several orders of magnitude faster.
Because, every file write is buffered in memory itself. Maintaining configuration in files
decouples our proxy server from the database server. Proxy server can be configured and
controlled even if the database server is offline.
13

Chapter 4
Phoenix proxy client
4.1 Problem to be solved
Normally, a user uses a single proxy server. Web browsers have a setting to use a proxy
server. They also provide a user friendly interface for entering proxy credentials. But,
several third party softwares and command line programs do not provide such a friendly
interface. So, we created a proxy client to take care of these issues.
4.2 Working of the proxy client
Proxy client works in the middle between a proxy server and the program willing to use
the server. It is a GUI program. As a bare minimum proxy client needs the proxy server’s
address and port number. When the users presses the start button, it starts listening on a
local port. All the user needs to do is set this local application as the proxy.
It also takes user name and password of the user. It then adds these credentials to
every HTTP request sent using it. Thus it shields all other programs using it from cre-
dentials problem. By default, proxy client uses custom authentication scheme. It sends
user:SHA1password combination in Proxy-Authorization header.
14

4.3 Interface of the proxy client
Figure 4.1: Client settings window Figure 4.2: Server settings window
4.4 Advantages
It is very useful for people who use many browsers and also diﬀerent third party softwares.
Password should only be entered once. They can use the proxy server till they remove
password from the proxy client.
15

Chapter 5
Web interface for proxy management
Phoenix has a web interface for managing access control lists, users, roles, quotas. It also
shows log of each user, when the user logs in. Admin has all privileges to add users, modify
their passwords, quota limits, roles, access control lists etc., Each user can only change
their password only.
Web interface has been completely written in Python using the Django framework.
It has lot of advantages over traditional methods.
5.1 Interfaces for users
Figure 5.1: Login page
16

Figure 5.2: Users addition
Figure 5.3: Websites list
17

Figure 5.4: Role addition
Figure 5.5: Roles list
18

Figure 5.6: Log
5.2 IPC between web interface and the proxy server
IPC is implemented using sockets. Proxy server listens on a local port for IPC messages.
The local port can be configured through the master configuration file. Realtime changes
can only occur in credentials, quota limits, access control lists. These changes are sent as
commands through the socket.
5.3 IPC message formats
5.3.1 Changes in credentials
Addition of new user:
TOKEN A:CRD:user:role:password
Removing existing user:
TOKEN R:CRD:user
Updating user password:
TOKEN U:CRD:user:role:newPassword
19

5.3.2 Changes in access control lists
Addition of a domain for a role:
TOKEN A:ACL:role:domain
Removing domain for a role:
TOKEN R:ACL:role:domain
5.3.3 Changes in user quota limits
Addition of a quota for a user:
TOKEN A:QTA:user:quotaLimit
Removal of quota for a user:
TOKEN R:QTA:user
20

Chapter 6
Conclusion and Future Work
Phoenix proxy server contains all basic and required features. Currently it supports
HTTP/1.1 protocol only. It has to be slightly modified to support HTTP/1.0 also.
Support for HTTPS is currently not included. It is trivial, a TCP tunneling algorithm
has to be implemented. FTP protocol should also be supported. Proxy client can be im-
proved to provide encryption. An efficient and lightweight encryption algorithm has to be
implemented. IPC mechanism has to be improvised to provide security between the web
interface and the proxy server. A configuration file creator software has to be implemented.
All configuration files have to be encrypted. A unique user has to be created on the oper-
ating system dedicated to running proxy software and also owning all proxy configuration
and log files.
21

Bibliography
[1] About proxy server, https://en.wikipedia.org/Proxy server
[2] Introduction to HTTP, http://www.jmarshall.com/easy/http
[3] HTTP RFC 2616, http://tools.ietf.org/html/rfc2616
[4] About SQUID proxy, http://www.squid-cache.org
[5] About SQUID proxy, https://en.wikipedia.org/Squid (software)
[6] HTTP caching, http://devcenter.heroku.com/articles/increasing-application-
performance-with-http-cache-headers
[7] HTTP caching, http://www.mnot.net/cache docs
[8] HTTP caching, http://www.peej.co.uk/articles/http-caching.html
22

thesis

More Related Content

What's hot (15)

Viewers also liked (17)

Similar to thesis (20)

thesis