Apache HTTP Server log files provides insight into the activity and performance of the server. Apache HTTP Server provides a variety of mechanisms for logging everything that happens on our server. Be it from the initial request, through the URL mapping process, to the final resolution of the connection, including any errors that may have occurred in the process.
In addition to this, third-party modules may provide logging capabilities, or inject entries into the existing log files, and applications such as CGI programs, or PHP scripts, or other handlers, may send messages to the server error log.
By default, Apache server writes all logs to the apache2/logs/ directory.
http://httpd.apache.org/docs/2.4/logs.html
NOTE:
• Anyone who can write to the directory where Apache HTTP Server is writing a log file can almost certainly gain access to the UID that the server is started as. Do NOT give people write access to the directory the logs are stored in without being aware of the consequences.
• Log files may contain information supplied directly by the client, without escaping. Therefore, it is possible for malicious clients to insert control-characters in the log files, so care must be taken in dealing with raw logs.
There are five types of log files.
1) Error logs
2) Access logs
3) Per module logs
4) Piped logs
5) Other logs
1) Error Logs
Provided by the core module.
Error Log is the place where Apache HTTP Server will send diagnostic information and record any errors that it encounters in processing requests. Error log will also contain debugging output from CGI scripts. Any information written to STDERR by a CGI script will be copied directly to the error log. The default filename for Error log is error_log. If there are VirtualHosts, filenames will follow the syntax SERVERNAME-error_log. For example, localhost-error_log.
Errorlog can be managed by the following three directives provided by core module.
ErrorLog SYNTAX: ErrorLog file-path|syslog[:facility] CONTEXT: server config, virtual host ENABLED by default. Default value is logs/error_log. Defines the name and location where the server error log, which is the most important file. ErrorLogFormat SYNTAX: ErrorLogFormat [connection|request] format CONTEXT: server config, virtual host Defines the format specification for error log entries. LogLevel SYNTAX: LogLevel [module:]level [module:level] ... CONTEXT: server config, virtual host, directory ENABLED by default. Default value is warn. Controls the verbosity of error logs using pre-defined levels.
http://httpd.apache.org/docs/2.4/mod/core.html
2) Access Logs
Provided by the modules mod_log_config.so and mod_setenvif.so.
NOTE:
• mod_log_config.so is compiled and enabled by default on all installations unless we explicitly disable it using --disable-log-config option.
• mod_setenvif.so is compiled and enabled by default on all installations unless we explicitly disable it using --disable-setenvif option.
Access Log records all requests processed by the server. The default filename for Access log is access_log. If there are VirtualHosts, the filenames will follow a syntax SERVERNAME-access_log. For example, localhost-access_log. Access Log can be managed by the directives provided by following modules.
mod_log_config.so
Logs are written in a customizable format, and may be written directly to a file, or to an external program. Conditional logging is provided so that individual requests may be included or excluded from the logs based on characteristics of the request. This module provides the following four directives.
BufferedLogs SYNTAX: BufferedLogs On|Off CONTEXT: server config DISABLED by default. Buffer log entries in memory before writing to disk, rather than writing them after each request. This may result in more efficient disk access and hence higher performance. WARNING: This directive might cause loss of logging data if system crashes. CustomLog SYNTAX: CustomLog file|pipe format|nickname [env=[!]environment-variable| expr=expression] CONTEXT: server config, virtual host Sets filename and format of log file. Logging can optionally be made conditional on request characteristics using environment variables. LogFormat SYNTAX: LogFormat format|nickname [nickname] CONTEXT: server config, virtual host ENABLED by default. Default value is "%h %l %u %t "%r" %>s %b" Describes a format for use in a log file. TransferLog SYNTAX: TransferLog file|pipe CONTEXT: server config, virtual host Specifies the location of a log file. This directive has the same arguments and effect as the CustomLog directive, with the exception that it does not allow the log format to be specified explicitly or for conditional logging of requests. Instead, the log format is determined by the most recently specified LogFormat directive which does not define a nickname. Common Log Format is used if no other format has been specified.
http://httpd.apache.org/docs/2.4/mod/mod_log_config.html
mod_setenvif.so
Allows to set internal environment variables according to different aspects of the request match regular expressions we specify. This module provides the following five directives.
BrowserMatch SYNTAX: BrowserMatch regex [!]env-variable[=value] [[!]env-variable[=value]] ... CONTEXT: server config, virtual host, directory, .htaccess This is a special cases of the SetEnvIf directive that sets environment variables conditional on the User-Agent HTTP request header. BrowserMatchNoCase SYNTAX: BrowserMatchNoCase regex [!]env-variable[=value] [[!]env-variable[=value]] ... CONTEXT: server config, virtual host, directory, .htaccess This is a special cases of the SetEnvIfNoCase directive that sets environment variables conditional on the User-Agent HTTP request header. SetEnvIf SYNTAX: SetEnvIf attribute regex [!]env-variable[=value] [[!]env-variable[=value]] ... CONTEXT: server config, virtual host, directory, .htaccess Defines environment variables based on attributes of the request. SetEnvIfExpr SYNTAX: SetEnvIfExpr expr [!]env-variable[=value] [[!]env-variable[=value]] ... CONTEXT: server config, virtual host, directory, .htaccess Defines environment variables based on an <If> ap_expr. These expressions will be evaluated and applied env-variable at runtime. SetEnvIfNoCase SYNTAX: SetEnvIfNoCase attribute regex [!]env-variable[=value] [[!]env-variable[=value]] ... CONTEXT: server config, virtual host, directory, .htaccess Defines environment variables based on an <If> ap_expr. These expressions will be evaluated without respect to case and applied env-variable at runtime.
http://httpd.apache.org/docs/2.4/mod/mod_setenvif.html
3) Per module logs
Provided by the core module.
This feature uses the LogLevel directive provided by core module. LogLevel directive allows us to specify a log severity level on a per-module basis. For example, if we have to get the details only related to rewrite module, we can use the below syntax.
syntax: LogLevel info rewrite:trace5
This will set the main LogLevel to info, But turns up the logging for module mod_rewrite.so upto trace5.
http://httpd.apache.org/docs/2.4/mod/core.html
4) Piped logs
Apache HTTP Server is capable of writing error and access log files through a pipe to another process, rather than directly to a file. This capability dramatically increases the flexibility of logging, without adding code to the main server. In order to write logs to a pipe, simply replace the filename, with the pipe character ” | “, followed by the name of the executable which should accept log entries on its standard input. The server will start the piped-log process when the server starts, and will restart it if it crashes while the server is running. So this technique is also called reliable piped logging. This feature is applicable to both Error logs and Access logs.
For example, if we have to rotate the logs every 24 hours, we can use the following syntax.
syntax: CustomLog “|/usr/local/apache/bin/rotatelogs /var/log/access_log 86400″ common
WARNING: Piped log processes are spawned by the parent Apache HTTP Server process, and inherit the UID of that process. Thus piped log programs run as Apache HTTP User.
5) Other logs
Other types of log files are provided by the following modules.
mod_cgi.so & mod_cgid.so
NOTE:
• mod_cgi.so is compiled only if we explicitly enable it using --enable-cgi or --enable-mods-shared=reallyall option.
• mod_cgid.so is compiled and enabled by default on all installations unless we explicitly disable it using --disable-cgid option.
• We have to manually enable them from the configuration file.
Both the modules provide the feature for execution of CGI scripts. mod_cgi.so is used with single-threaded MPM, while mod_cgid.so is used with multi-threaded MPM. They provide three common directives with mod_cgid.so providing an additional directive ScriptSock.
ScriptLog SYNTAX: ScriptLog file-path CONTEXT: server config, virtual host DISABLED by default. This directive sets the CGI script error logfile. If no ScriptLog is given, no error log is created. NOTE: Script logging is meant to be a debugging feature when writing CGI scripts, and is not meant to be activated continuously on running servers. It is not optimized for speed or efficiency, and may have security problems if used in a manner other than that for which it was designed. ScriptLogBuffer SYNTAX: ScriptLogBuffer bytes CONTEXT: server config, virtual host Default value is '1024'. Defines the maximum amount of PUT or POST requests that will be recorded in the scriptlog. ScriptLogLength SYNTAX: ScriptLogLength bytes CONTEXT: server config, virtual host Default value is 'ScriptLogLength 10385760'. Defines the size limit of CGI script logfile. If the file exceeds this size, no more information will be written to it. ScriptSock SYNTAX: ScriptSock file-path CONTEXT: server config Default value is cgisock. Defines the filename prefix of the socket to use for communication with the cgi daemon.
http://httpd.apache.org/docs/2.4/mod/mod_cgi.html
http://httpd.apache.org/docs/2.4/mod/mod_cgid.html
mod_dumpio.so
NOTE:
• mod_dumpio.so is compiled by default on all installations unless we explicitly disable it using --disable-dumpio option.
• We have to manually enable this module from the configuration file.
• To enable functionality, we need to be configure mod_dumpio.so for Per module logging with a LogLevel of trace7.
This module allows for the logging of all input received by Apache HTTP Server and/or all output sent by Apache HTTP Server to be logged (dumped) to the Error Log file. This can produce extreme volumes of data, and should only be used when debugging problems. The data logging is done right after SSL decoding (for input) and right before SSL encoding (for output).
This module provides two directives.
DumpIOInput SYNTAX: DumpIOInput On|Off CONTEXT: server config Default value is Off. Enables dumping of all input. DumpIOOutput SYNTAX: DumpIOOutput On|Off CONTEXT: server config Default value is Off. Enable dumping of all output.
http://httpd.apache.org/docs/2.4/mod/mod_dumpio.html
mod_log_forensic.so
NOTE:
• mod_log_forensic.so is compiled only if we explicitly enable it using --enable-log-forensic or --enable-mods-shared=all | reallyall option.
• We have to manually enable this module from the configuration file.
This module provides Forensic Logging of the requests made to the server. Logging is done before and after processing a request, so the forensic log contains two log lines for each request. Each log entry is assigned a unique ID which can be associated with the request using the normal CustomLog directive. Also this module creates a token that can be added to the Transfer log using a format string.
The forensic logging is very strict, which means:
• The format is fixed. We cannot modify the logging format at runtime.
• If it cannot write its data, the child process exits immediately and may dump core (depending on our CoreDumpDirectory configuration).
This module provides the following directive.
ForensicLog SYNTAX: ForensicLog filename|pipe CONTEXT: server config, virtual host Defines the filename of the forensic log.
http://httpd.apache.org/docs/2.4/mod/mod_log_forensic.html#forensiclog
mod_log_debug.so
NOTE:
• mod_log_debug.so is compiled by default on all installations unless we explicitly disable it using --disable-log-debug option.
• We have to manually enable this module from the configuration file.
This module provides additional configuration for debug logging. It provides the following directive.
LogMessage SYNTAX: LogMessage message [hook=hook] [expr=expression] CONTEXT: directory Defines the user-defined message to log. The messages are logged at loglevel info.
http://httpd.apache.org/docs/2.4/mod/mod_log_debug.html#logmessage
mod_logio.so
NOTE:
• mod_logio.so is compiled by default on all installations unless we explicitly disable it using --disable-logio option.
• We have to manually enable this module from the configuration file.
• This module requires the module mod_log_config.so
This module provides the logging of input and output number of bytes received/sent per request. The numbers reflect the actual bytes as received on the network, which then takes into account the headers and bodies of requests and responses. The counting is done before SSL/TLS on input and after SSL/TLS on output, so the numbers will correctly reflect any changes made by encryption.
This module provides no directives. But it introduces three new format strings.
• %I – Bytes received, including request and headers, cannot be zero.
• %O – Bytes sent, including headers, cannot be zero.
• %S – Bytes transferred (received and sent), including request and headers, cannot be zero. This is the combination of %I and %O.
http://httpd.apache.org/docs/2.4/mod/mod_logio.html
mod_unique_id.so
NOTE:
• mod_unique_id.so is compiled by default on all installations unless we explicitly disable it using --disable-unique-id option.
• We have to manually enable this module from the configuration file.
This module provides a magic token for each request which is guaranteed to be unique across “all” requests under very specific conditions. The unique identifier is even unique across multiple machines in a properly configured cluster of machines. The environment variable UNIQUE_ID is set to the identifier for each request. This module does not provide any directives. But it provides a format string %L. By putting %L token in both ErrorLogFormat(Error Logs) and LogFormat(Access Logs) directives, a log entry ID(unique request ID) will be produced, with which we can correlate both logs.
http://httpd.apache.org/docs/2.4/mod/mod_unique_id.html
http://httpd.apache.org/docs/2.4/logs.html#errorlog
mod_usertrack.so
NOTE:
• mod_usertrack.so is compiled only if we explicitly enable it using --enable-usertrack or --enable-mods-shared=all | reallyall option.
• We have to manually enable this module from the configuration file.
This module allows tracking of a user through a website via browser cookies. Thus it provides Clickstream logging of user activity on a site. This module provides five directives.
CookieDomain SYNTAX: CookieDomain domain CONTEXT: server config, virtual host, directory, .htaccess Controls the setting of the domain to which the tracking cookie applies. CookieExpires SYNTAX: CookieExpires expiry-period CONTEXT: server config, virtual host, directory, .htaccess Sets an expiry time on the cookie generated by the usertrack module. CookieName SYNTAX: CookieName token CONTEXT: server config, virtual host, directory, .htaccess Default name is 'Apache'. Allows us to change the name of the cookie this module uses for its tracking purposes. CookieStyle SYNTAX: CookieStyle Netscape|Cookie|Cookie2|RFC2109|RFC2965 CONTEXT: server config, virtual host, directory, .htaccess Default value is 'Netscape'. Controls the format of the cookie header field. CookieTracking SYNTAX: CookieTracking on|off CONTEXT: server config, virtual host, directory, .htaccess Default value is 'off'. Enables tracking cookie.
http://httpd.apache.org/docs/2.4/mod/mod_usertrack.html
You may go back to the following section.
Installing Apache HTTP Server
