Apache 2.x configuration files, terms and utilities

Apache log files configuration and content

Access restriction methods and files

mod_perl and PHP configuration

Client user authentication files and utilities

Configuration of maximum requests, minimum and maximim servers and clients

Resources: LinuxRef06; Coar00; Poet99; Wilson00; Engelschall00; PerlRef01; Krause01; the man pages for the various commands.

Apache can be built from source, but in most cases it is already part of a Linux distribution. The next step is to edit the configuration files for the server: by default, depending on your distribution, these files are located in /etc/apache/config or /etc/httpd/config. The main configuration file is called httpd.conf. This can be, again depending on your distribution, one big file with configuration of all modules, or a small generic file with references to other configuration files. Some configuration options can be set on a per-directory basis by placing a file called .htaccess in the directory. Options set in a .htaccess file are valid for the directory containg the file and all its subdirectories.

First, edit httpd.conf. This sets up the general attributes of the server: the port number, the user it runs as, etc. The default configuration file is meant to be self explanatory. The configuration option AllowOverride can be set in a Directory context and specifies which values can be overridden by .htaccess files.

Apache, like many other successful open source projects, has a modular source code architecture. This means that, to add or modify functionality, you do not need to know the whole code base. You can custom build the server with only the modules you need and include your own modules.

Extending Apache can be done in C or in a variety of other languages using the appropriate modules. These modules expose Apache's internal functionality to various programming languages such as Perl or Tcl. There are many modules available, too many to list in this book. If you have any questions about the development of an Apache module, you should join the Apache-modules mailing list at http://modules.apache.org. Remember to do your homework first: research past messages and check all the documentation on the Apache site. Chances are that someone has already written a module that solves the problem you are experiencing.

The modular structure of Apache's source code should not be confused with the functionality of run-time loading of Apache modules. Run-time modules are loaded after the core functionality of Apache has started and are a relatively new feature. In older versions, to use the functionality of a module, it needed to be compiled in during the build phase. Current implementations of Apache are capable of loading modules run-time, see DSO.

Most modern Unix derivatives include a mechanism called dynamic linking/loading of Dynamic Shared Objects (DSO). This provides a way to build a piece of program code in a special format for loading at run-time into the address space of an executable program. This loading can usually be done in two ways: either automatically by a system program called ld.so when an executable program is started, or manually, from within the executing program via a system interface to the Unix loader through the system calls dlopen()/dlsym().

In the latter method the DSO's are usually called shared objects or DSO files and can be named with an arbitrary extension (although the canonical name is foo.so). These files are usually installed in a program-specific directory. The executable program manually loads the DSO at run-time into its address space via dlopen().

The fact that Apache already uses a module concept to extend its functionality, and internally uses a dispatch-list-based approach to link external modules into the Apache core functionality has predestined Apache for using DSO's. Starting with version 1.3 Apache began using the DSO mechanism to extend its functionality at run-time. Since then the configuration system supports two optional features for taking advantage of the modular DSO approach: compilation of the Apache core program into a DSO library for shared usage and compilation of the Apache modules into DSO files for explicit loading at run-time.

Apache has been modified to support Secure Socket Layers for Secure On-Line communication. The Secure Sockets Layer protocol (SSL) is a protocol layer which may be placed between a reliable connection-oriented network layer protocol (e.g., TCP/IP) and the application protocol layer (e.g., HTTP). SSL provides secure communication between client and server by allowing mutual authentication, the use of digital signatures for integrity and encryption for privacy. There are a currently two versions of SSL still in use: versions 2 and 3. Additionally, there is the successor to SSL, TLS (version 1, which is based on SSL), designed by the IETF.

Apache is a generic web-server and has been designed to be correct first and fast second. (Even so, its performance is quite satisfactory.) Most sites have less than 10Mbits of outgoing bandwidth, which Apache can fill using only a low end Pentium-based computer. In practice, sites with more bandwidth require more than one machine to fill the bandwidth due to other constraints (such as CGI or database transaction overhead). For these reasons, the development focus has stayed on correctness and configurability.

The single biggest hardware issue affecting web-server performance is RAM. A webserver should never ever have to swap. It increases the latency of each request beyond a point that users consider “fast enough”. This causes users to hit stop and reload, further increasing the load. You can, and should, control the MaxClients setting so that your server does not spawn so many children that it starts swapping.

An Open Source system that can be used to periodically load-test pages of web-servers is Cricket. Cricket can be easily set up to record page-load times, and it has a web-based grapher that will generate charts to display the data in several formats. It is based on RRDtool, whose ancestor is MRTG (short for “Multi-Router Traffic Grapher”). RRDtool (Round Robin Data Tool) is a package that collects data in “round robin” databases; each data file is fixed in size so that running Cricket does not slowly fill up your disks. The database tables are sized when created and do not grow larger over time. As the data ages, it is averaged.

The access_log gives a generic overview of the access to your web-server. The format of the access log is highly configurable. The format is specified using a format string that looks much like a C-style printf format string. A typical configuration for the access log might look like the following.

LogFormat "%h %l %u %t \"%r\" %>s %b" common
CustomLog logs/access_log common
      

This defines the nickname common and associates it with a particular log format string. The above configuration will write log entries in a format known as the Common Log Format (CLF). This standard format can be produced by many different web servers and read by many log analysis programs. The log file entries produced in CLF will look something like this:

127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
      

and contain the following fields:

Apache is aware of two methods of access control:

In Web terms in general, and Apache terms in particular, discretionary controls are based on usernames and passwords, and mandatory controls are based on things such as the IP address of the requesting client.

Apache uses modules to authenticate and authorise users. Generally, modules let you store the valid credential information in one format or another. The mod_auth module, for instance, looks in normal text files for the username and password info, and mod_auth_dbm looks in a DBM database for it.

Below is a list of the security-related modules that are included as part of the standard Apache distribution.

The security modules are passed the information which authentication databases to use via directives in the Apache configuration files, such as AuthUserFile or AuthDBMGroupFile. An alternate approach is the use of .htaccess files, which can be placed in the directories to be protected.

The first approach.  The resource being protected is determined by the placement of the directives in the configuration files; in this example:

<Directory /home/johnson/public_html>
<Files foo.bar>
AuthName "Foo for Thought"
            AuthType Basic
            AuthUserFile /home/johnson/foo.htpasswd
            Require valid-user
</Files>
</Directory>
</screen>
      

The resource being protected is “any file named foo.bar” in the /home/johnson/public_html directory or any subdirectory thereof. Likewise, the identification of which users are authorized to access foo.bar is stated by the directives -- in this case, any user with valid credentials in the /home/johnson/foo.htpasswd file can access the file.

The other approach.  Create an .htaccess file. The server behaviour can be configured on a directory-by-directory basis by using .htaccess files in directories accessed by the server (provided you configured this in the httpd.conf file using the AllowOverride option).

The authentication part of an .htaccess file contains 2 sections:

The first section of .htaccess can contain lines to determine which authentication type to use. It can also contain the name of the password file to be used, or the group file to be used, e.g.:

AuthUserFile {path to passwd file}
AuthGroupFile {path to group file}
AuthName {title for dialog box}
AuthType Basic
     

The second section of .htaccess contains a section that defines the access rights for the current directory to ensure that only user {username} can access the current directory:

<Limit GET>
  require user {username} 
</Limit>
     

The Limit section can contain other directives, that allow access from only certain IP addresses or only for users who are part of a certain set of users, a group.

As an example of the usage of both access control types (DAC and MAC), the following would permit any client on the local network (IP addresses 10.*.*.*) to access the foo.html page without hindrance, but require a username and password for anyone else:

<Files foo.html>
Order Deny,Allow
        Deny from All
        Allow from 10.0.0.0/255.0.0.0
        AuthName "Insiders Only"
        AuthType Basic
        AuthUserFile /usr/local/web/apache/.htpasswd-foo
        Require valid-user
        Satisfy Any
</Files>
   

The mod_auth module uses plain text user files. Its entries are of the form “username:password”; additional fields may follow the password, separated from it by a colon, but these are ignored. The password field should be encrypted. To create and update the flat-files used to store usernames and password for basic authentication of HTTP users, you can use the command htpasswd. This program can only be used when the usernames are stored in a flat-file. htpasswd encrypts passwords using either a version of MD5 modified for Apache, or the system's crypt() routine. It is permissible to have some user records using MD5-encrypted passwords, while others in the same file may have passwords encrypted with crypt(). To use a DBM database (as used by mod_auth_db) you may use dbmmanage. For other types of user files/databases, please consult the documentation that comes with the chosen module.

It is possible to deny or allow access to a group of users. This is done by creating a group file. It contains a list of groups and members. The group names are completely arbitrary. The usernames should occur in the password file. The format looks like this:

{group1}:{username1} {username2} etc... 
{group2}:{username1} {username3} etc... 
     

In this example, group1 and group2 are different group names and username1, username2, and username3 are usernames from the password file. Be sure to put each group entry on its own line.

The last step is to make sure that the read permissions for the group file have been set for everyone (i.e., owner, group and other). Referral to this file is done by insertion of AuthGroupFile directives into either the master configuration file or into the .htaccess file.

AuthGroupFile /etc/httpd/groups/groupfile
<Limit GET>
  require group mygroup
</Limit>
     

mod_perl is another module for Apache. You can configure it either at compile-time, or as a DSO. With mod_perl it is possible to write Apache modules entirely in Perl, letting you easily do things that are more difficult or impossible in CGI programs. In addition, the persistent Perl interpreter embedded in the module saves the overhead of starting an external interpreter. Another important feature is code-caching: modules and scripts are loaded and compiled only once, and for the rest of the server's life they are served from the cache. Thus, the server spends its time only running already loaded and compiled code, which is very fast.

You have full access to the inner workings of the web server and can intervene at any stage of request-processing. This allows for customized processing of (to name just a few of the phases) URI->filename translation, authentication, response generation and logging. There is very little run-time overhead.

The standard Common Gateway Interface (CGI) within Apache can be replaced entirely with Perl code that handles the response generation phase of request processing. mod_perl includes two general purpose modules for this purpose: Apache::Registry, which can transparently run existing perl CGI scripts and Apache::PerlRun, which does a similar job, but allows you to run “dirtier” (to some extent) scripts.

You can configure your httpd server and handlers in Perl (using PerlSetVar, and <Perl> sections). You can also define your own configuration directives.

There are many ways to install mod_perl, e.g. as a DSO, either using APXS or not, from source or from RPM's. Most of the possible scenarios can be found in the Mod_perl Guide PerlRef01. As an example we describe a scenario for building Apache and mod_perl from source code.

You should have the Apache source code, the source code for mod_perl and have unpacked these in the same directory ^[9]. You'll need a recent version of perl installed on your system. To build the module, in most cases, these commands will suffice:

$ cd ${the-name-of-the-directory-with-the-sources-for-the-module}
$ perl Makefile.PL APACHE_SRC=../apache_x.x.x/src \
                DO_HTTPD=1 USE_APACI=1 EVERYTHING=1
$ make && make test && make install
     

After building the module, you should also build the Apache server. This can be done using the commands:

$ cd ${the-name-of-the-directory-with-the-sources-for-Apache}
$ make install
       

All that's left then is to add a few configuration lines to httpd.conf (the Apache configuration file) and start the server. Which lines you should add depends on the specific type of installation, but usually a few LoadModule and AddModule lines suffice.

As an example, these are the lines you would need to add to use mod_perl as a DSO:

LoadModule perl_module modules/libperl.so
AddModule mod_perl.c
PerlModule Apache::Registry 

Alias /perl/ /home/httpd/perl/ 
<Location /perl>
  SetHandler perl-script 
  PerlHandler Apache::Registry 
  Options +ExecCGI
  PerlSendHeader On 
</Location>

The first two lines will add the mod_perl module when Apache boots. On boot, the PerlModule directive ensures that the named Perl module is read in too. This usually is a Perl package file ending in .pm. The Alias keyword reroutes requests for URIs in the form http://www.host.com/perl/file.pl to the directory /home/httpd/perl. Next, we define settings for that location. By setting the SetHandler, all requests for a Perl file in the directory /home/httpd/perl now will be redirected to the perl-script handler, which is part of the Apache::Registry module. The next line simply allows execution of CGI scripts in the specified location. Any URI of the form http://www.host.com/perl/file.pl now will be compiled once and cached in memory. The memory image will be refreshed by recompiling the Perl routine whenever its source is updated on disk.

PHP is a server-side, cross-platform, HTML embedded scripting language. PHP started as a quick Perl hack written by Rasmus Lerdorf in late 1994. Over the next two to three years, it evolved into what we today know as PHP/FI 2.0. PHP/FI started to get a lot of users, but things didn't start flying until Zeev Suraski and Andi Gutmans suddenly came along with a new parser in the summer of 1997, leading to PHP 3.0. PHP 3.0 defined the syntax and semantics used in both versions 3 and 4.

PHP can be called from the CGI interface, but most common approach is to configure PHP in the Apache web server as a (dynamic DSO) module. To do this, you can either use pre-built modules extracted from RPM's or roll your own from the source code^[10]. You need to configure the make-process first. To tell configure to build the module as a DSO, you need to tell it to use APXS:

./configure -with-apxs
     

.. or, in case you want to specify the location for the apxs binary:

./configure -with-apxs={path-to-apxs}/apxs
     

Next, you can compile PHP by running the make command. Once all the source files are successfully compiled, install PHP by using the make install command.

Before Apache can use PHP, it has to know about the PHP module and when to use it. The apxs program took care of telling Apache about the PHP module, so all that is left to do is tell Apache about .php files. File types are controlled in the httpd.conf file, and it usually includes lines about PHP that are commented out. You want to search for these lines and uncomment them:

Addtype application/x-httpd-php .php 
     

.. and restart Apache by issuing the apachectl restart command.

To test whether it actually works, create the following page:

<HTML>
<HEAD><TITLE>PHP Test </TITLE></HEAD>
<BODY>
<?phpinfo( ) ?>
</BODY>
</HTML>
      

Notice that PHP commands are contained by <? and ?> tags. Save the file as test.php in Apache's htdocs directory and aim your browser at http://localhost/test.php. A page should appear with the PHP logo and additional information about your PHP configuration.

The httpd.conf file contains a number of sections that allow you to configure the behavior of the Apache server. A number of keywords/sections are listed below.