DLXS Directory Conventions

  1. Rationale
  2. Conventions for Content and Middleware
  3. Conventions for Software Packages
  4. "Data Flow" Describing the Preparation and Release of a Collection

Rationale

Multiply the amount of data you have by the number of people dealing with it and you get a metric of complexity indicating how badly you need conventions. Having some basic directory conventions solves some general problems:

With respect to content and middleware, a good convention:

From a system administration point of view, a good convention:

Conventions for Content and Middleware

Content should be organized by DLPS identifier or, if that's unavailable, collection identifier. Middleware and everything else should be organized by collection identifier.

The basic scheme is as follows:

<prefix>/bin/c/class

or

<prefix>/bin/c/collection

 loading/munging/processing scripts
  • optionally by class or collection, if appropriate
  • typically on the staging server, not production servers

<prefix>/cgi/c/class

or

<prefix>/cgi/c/collection

 CGI middleware
  • by class or collection
<prefix>/img/c/collection continuous tone image data
  • non-web-ready (archival) images, to be delivered through middleware; i.e., like image services
  • web-ready (GIF, JPEG, PNG) images which are licensed content but to be delivered directly; to make this work, there is typically a symbolic link from %lt;prefix>/web; see below
<prefix>/idx/c/collection search indexes and index-related files

<prefix>/lib/[c/class]

or

<prefix>/lib/[c/collection]

 CGI middleware library modules
  • optionally by class or collection, if appropriate

<prefix>/misc/[c/class/]

or

<prefix>/misc/[c/collection/]

 additional files for DLXS distribution that don't belong elsewhere: e.g., DTDs, character entity sets, catalogs, etc.
  • optionally by class or collection, if appropriate

<prefix>/misc/db

 text-based databases: e.g., DLXS sessions database in CSV format

<prefix>/misc/[c/class/]maps

or

<prefix>/misc/[c/collection/]maps

 region map files
  • optionally by class or collection, if appropriate
<prefix>/obj/c/collection production content organized by collection identifier
  • .raw for "raw" (i.e., unprocessed OCR) text
  • .sgm for "cooked" (i.e., proofed and encoded) text
  • .tif for page images, etc.
<prefix>/obj/d/l/p/dlpsid.vvvv.iii production content organized by DLPS identifier
  • same file naming conventions as above
<prefix>/prep/c/collection pre-production (munge-in-process) content organized by collection identifier
  • same file naming conventions as above
  • typically on the staging server, not production servers
<prefix>/prep/d/l/p/dlpsid.vvvv.iii pre-production (munge-in-process) content organized by DLPS identifier
  • same file naming conventions as above
  • typically on the staging server, not production servers
<prefix>/web/c/collection static HTML pages

<prefix>/web/c/class/graphics

or

<prefix>/web/c/collection/graphics

 user interface glitter: banners, icons, logos, etc.
  • optionally by class or collection, if appropriate

<prefix>/web/c/class/images

or

<prefix>/web/c/collection/images

web-ready image content served directly, usually figures that appear in text collections

if image content is licensed, files are stored in <prefix>/img (see above) and this is just a symbolic link

  • optionally by class or collection, if appropriate

<prefix> varies from installation to installation. All DLPS servers use at least the prefix /l1; most have additional file systems named /l2, /l3, etc.

Note that although classes and parts of the organization can also exist at this level (for example, h/hti for the Humanities Text Initiative or i/is for Image Services), the individual collections would exist within their own spaces, i.e., <prefix>/.../m/musart.

Personal directories can also exist at this level, i.e., <prefix>/cgi/c/csnavely, for areas to do testing in.

The organization of files below the levels described here is outside the scope of this convention. We don't feel a need to prescribe a structure for inside /l1/web/b/bas, for example. Of course, consistency between like things is good practice.

Conventions for Software Packages

Basic Convention

The DLPS institutional convention is that software should be installed as

/l/local/package-version

where

package is the proper name of the software package, preferably lowercase (e.g. "apache"), and
version is the full version number (e.g. "1.3.9"), if available.

Likewise, source code can be found in

/l/local/src/package-version

or compressed up in a tarball named

/l/local/src/package-version.tar.gz

Special Cases

For cases where a version-independent path is needed or wanted, a symbolic link should be created:

/l/local/package -> /l/local/package-version

For commonly-used utilities that are conventionally installed in places like /usr/local/bin, a symbolic link should be created in /l/local/bin:

/l/local/bin/command -> /l/local/package/[subpath/]command

if present, or

/l/local/bin/command -> /l/local/package-version/[subpath/]command

Data Flow Describing the Preparation and Release of a Collection

Looking at the conventions from a "process" point of view, the following diagram illustrates how we use the directory conventions in the workflow of preparing and releasing a collection.
data flow diagram