Developer Documentation for The W3C MarkUp Validation Service

MarkUp Validation Service

The W3C MarkUp Validation Service consists of an SGML Parser, an SGML catalog, a CGI program and it's configuration files. In addition it relies on a moderately large set of Perl modules for it's operation.

This document tries to draw a road map of the prerequisites and what the different parts of the system do. It is intended for system administrators and people interested in helping developing the validator. This is not end user documentation. See the User Manual for usage instructions.

Prerequisites

Apart from a properly configured web server, the Validator needs a SGML parser -- that does all the hard work -- and several Perl modules used by the "check" CGI script.

The SGML parser we're currently using is OpenSP 1.5, which can be found on the OpenJade home page.

The canonical list of Perl modules we use can be found in the source for the "check" CGI script. There is a bunch of lines that of the form "use Foo::Bar" where each "Foo::Bar" represents a module. Most modules can be found on CPAN (minimum versions in parenthesis after the name). The following list was complete when CVS spit out: $Date: 2002/11/30 16:45:33 $ . :-)

CGI (2.81): The all-singing, all-dancing, everything-and-the-kitchen-sink, Perl CGI library. This takes care of all those niggly little bits of CGI for us and make options parsing and file upload a breeze.
CGI::Carp: CGI-aware warn()/die()
File::Spec: Portable filespecs.
HTML::Parser (3.25): Minimal HTML Parser used for preparse and finding metadata.
LWP::UserAgent (1.90): Gisle Aas' most excellent WWW library for Perl. This is where our support for downloading pages off the net comes from.
Set::IntSpan: Efficient Set operations.
Text::Iconv: Perl-native interface to the (g)libc iconv(3) library. Handles charset conversion issues.
Text::Wrap: Wrap text to a sane width. Needed for source output in results.
URI::Escape: Module to handle escaping special characters in URIs.

Configuration Files

The validator uses a number of configuration files -- most of which are really mapping tables of some form -- to avoid having to check in a new version of the code every time a new version of HTML comes out. All configuration files can be found in $CVSROOT/validator/htdocs/config/.

To really understand what each does you should read the source, but here is a short description to get you started.

eref.cfg: Contains the mappings from element names to an URI fragment (relative to a configurable URI) for their definitions. Used in output when the "Show Source Input" option is enabled.
fpis.cfg: Maps FPIs to plain text version strings.
frag.cfg: Maps error messages to an URI fragment identifier where an explanation of that error can be found.
type.cfg: Maps MIME/HTTP Content-Types to an internal "document type" which is used for treating HTML, XML, and XHTML in different ways.
check.cfg: Main configuration file. Gives various parameters (such as the address of the maintainer and the URL for the "Home Page") and the locations of the other configuration files and mapping tables.

TODO

The TODO list for the Validator is online at <http://validator.w3.org/todo.html>. This is probably the best place to start.

However this list is by no means comprehensive. Feel free to suggest other features that should be on this list or send patches for your favourite feature.

Keep in mind that features should be of general utility and that the point if the validator is that it does an objective validation instead of just what some random developer happens to think is a Good Idea®. While extra features are nice, they shouldn't dilute the value of the validator as an objective check.