2006/07/12
ICPADS 2006@Minneapolis
1
Secure and High-performance Web Server System for Shared Hosting Service
Daisuke Hara and Yasuichi Nakayama
The University of Electro-Communications, Tokyo, Japan
2006/07/12
2
Outline
Introduction Background Problems of large-scale hosting service and web server Proposal - Hi-sap Design Implementation Evaluation Conclusions2006/07/12
3
Introduction
Problem of existing web servers Server embedded interpreters cannot be used safely in large-scale environments like a shared hosting service. Proposal - Hi-sap Web objects that are stored in a server are divided into partitions*. Server processes run under the privilege of different users in every partition. Achievement Hi-sap solves the problem. It achieves high performance & scalability.(*) 绐跺0artition绐躲0 is a unit of division of web objects.
(e.g. site, content, QUERY_STRING)
2006/07/12
4
Background
More people are creating their own websites as the Internet grows in popularity. weblog, wiki, CMS Shared hosting services are widely used. Many customers share a server. 100s - 1000s sites/server low price & flexible custom CGI, etc.2006/07/12
5
Server embedded interpreters
e.g. PHP, mod_ruby, mod_perl Because they have server processes including interpreters of language processors, they can improve performance in processing dynamic content like weblogs and wikis.2006/07/12
6
Problem of existing web servers
A绐跺0 website
B绐跺0 website
C绐跺0 website
Server
Internal users can steal & delete authentication content without
authentication (cp, rm commands or malicious CGI scripts).
browser
authentication
auth content
auth content
steal & delete
ID & Pass
It is required to grant read permission to an other.
(rw-r--r--)
2006/07/12
7
Problem of existing web servers (cont.)
Existing solution: POSIX ACL & suEXEC CGI scripts run under the privilege of the site owner by using suEXEC. Permissions of public access files are granted only to the dedicated user* by using POSIX ACL. It is not required to grant read permission to an other.(*) 绐跺江edicated user绐躲0is user account that runs server processes.
e.g. www, apache, www-data
2006/07/12
8
Problem of existing web servers (cont.)
Even if POSIX ACL & suEXEC is used, the problem occurrs when server embedded interpreters are used. Dynamic content that use server embedded interpreters (e.g. PHP, mod_ruby, mod_perl) also run under the privilege of a dedicated user. Malicious PHP scripts can steal & delete authentication content.2006/07/12
9
Harache ([13][14])
Predecessor of Hi-sap Server processes run under the privilege of the site owner.root
root
root
绔00
绔0健
绔0剑
browser
GET /~userA/
A browser sends request to the user A's website. The privilege of the server process is changed to user A. The server process processes the request. It returns a response to the browser.Harache
Server Process
userA
绔0舰
2006/07/12
10
Harache (cont.)
Server embedded interpreters can be used safely. File permissions to a dedicated user are not necessary. It is required to grant permissions only to the site owner. But, it cannot fully use the increased speed of server embedded interpreters. Server processes terminate after each session. (= CGI)Hi-sap solves Harache绐跺0 performance problem.
2006/07/12
11
Goal
Realization of secure, high-performance, and scalable web server system, Hi-sap Secure: Scripts of a partition cannot access other partitions. High performance: Dynamic content can be processed at high speed by fully using the increased speed of server embedded interpreters. Scalable: A number of partitions can be housed in a server.2006/07/12
12
Design
Security Server processes run under the privilege of different users in every partition. (= Harache) The system brings access control into operation with a secure OS. Performance The system pools server processes that run under the privilege of the different users. (!= Harache) Scalability The system controls the creation and termination of server processes.Content Access Scheduler
2006/07/12
13
Content Access Scheduler
Web-server level scheduler [aim] It enhances the scalability of the number of partitions in a server. [method] It controls the creation and termination of server processes.By using the suitable scheduler for the purpose, it achieves high-scalability.
2006/07/12
14
Implementation
OS: Linux OS with SELinux dispatcher reverse proxy server Apache 2.0.55 + mod_hisap workers Each worker runs under the privilege of a different user and processes requests for a specific dedicated partition. Apache 2.0.55 x 1000 Any web server software can be used. hisapd Content Access Scheduler2006/07/12
15
Overview of request processing
B
workers
绐讹溅
GET / HTTP/1.1
Host: www.C.net
terminating worker A
www
www
A
A
A
B
B
C
C
C
C
hisapd
asking to activate worker C
root
root
worker A has no requests
HTTP
UNIX Domain socket
sending the response
process the request
reverse proxy
activating worker C
confirming if worker C is active
dispatcher
OK
Browser
Server
heavy load
2006/07/12
16
Scheduling algorithm
We developed Content Access Scheduler to avoid thrashing. Thrashing decreases the performance of web servers dramatically. Algorithm of worker activation hisapd dynamically activates workers after requests from the dispatcher. Algorithm of worker termination When thrashing seems to occur, hisapd terminates workers that have not been requested recently.2006/07/12
17
Scheduling algorithm (cont.)
Conditions for which hisapd judges that thrashing seems to occur A swap-in occurs. A swap-out occurs. Memory use is 99% or more. Conditions for which hisapd chooses workers to terminate The worker is active. The worker is not recorded in the most recent 10,000 requests.2006/07/12
18
Evaluation
Experimental environmentsDELL PowerConnect 2724
1000 BASE-T x 24
Switching Hub
Network
Broadcom BCM5704C
1 Gbps
NIC
Fedora Core 4
(kernel 2.6.14)
OS
4 GB (swap 8 GB)
Memory
AMD Opteron 240EE
1.4 GHz x 2
CPU
Server
Intel PRO/1000XT PWLA8490XT 1 Gbps
NIC
Fedora Core 4
(kernel 2.6.14)
OS
256 MB (swap 512 MB)
Memory
Intel Pentium III Xeon
500 MHz x 4
CPU
Client
Gigabit Ethernet
Gigabit Ethernet
2006/07/12
19
Evaluation (conf.)
Basic performance evaluation We evaluated the basic performance in processing dynamic content. Scalability evaluation We evaluated the scalability of the number of partitions in a server in processing dynamic content. Target content We sent requests to a PHP script that calls phpinfo(). The script displays the system information of the PHP language processor. (40 KB per request)2006/07/12
20
Basic performance evaluation
Aim to determine useful performance of our system Systems for comparison Apache One-to-one It uses networks with a reverse proxy, and has a dispatcher and many workers that are dedicated to process requests for each partition. Although it is similar to our system, mod_hisap and hisapd are not installed. Apache with suEXEC Benchmark httperf benchmark ver. 0.82006/07/12
21
Basic performance evaluation (cont.)
The system loses an avg. of 28.0% of the throughput relative to Apache. The overhead of the system is because of a reverse proxy. However, the system has high throughput relative to suEXEC. The system loses an avg. of 1.0% of the throughput relative to One-to-one. The overhead of mod_hisap & hisapd is very low.2006/07/12
22
Scalability evaluation
Aim to determine the effectiveness of Content Access Scheduler Comparison system One-to-one mod_hisap and hisapd (Content Access Scheduler) are not installed. Benchmark Apache benchmark ver. 2.0.41-dev2006/07/12
23
Scalability evaluation (cont.)
Our system绐跺0 scalability is high. The throughput decrement due to an increase in the number of partitions was low. For One-to-one, the OS crashed due to a memory shortage when the number of partitions was 600.2006/07/12
24
Scalability evaluation (cont.)
The swap use of One-to-one dramatically increases due to an increase in the number of partitions. This is the reason of the OS crash. Our system does not use swap space as much because of Content Access Scheduler.2006/07/12
25
Comparison of approaches
good
poor
good
good
One-to-one
good
good
poor
good
Harache
good
poor / very poor
excellent
excellent
good
good
excellent
very poor
Apache
good
good
very poor
good
suEXEC & POSIX ACL
Generality
Scalability
Basic Performance
Security in a Server
good
good
good
excellent
Hi-sap
good
Apache perchild MPM
very poor
good
excellent
good
PHP safe mode
2006/07/12
26
Conclusions
Proposal: Hi-sap Secure and high-performance web server system Implementation: On a Linux OS with SELinux. Achievement: High performance High scalability2006/07/12
27
Future Work
Creating various Content Access Schedulers for wiki for weblog for CMS, etc. Evaluating these schedulers2006/07/12
28
Thank you.
Any questions/comments?
download Secure and High-performance Web Server System for Shared Hosting ...
