Conventions and Guidelines for Parrot Source Code

Version

$Revision: 36833 $

Abstract

This document describes the various rules, guidelines and advice for those wishing to contribute to the source code of Parrot, in such areas as code structure, naming conventions, comments etc.

Synopsis

Not applicable.

Description

One of the criticisms of Perl 5 is that its source code is impenetrable to newcomers, due to such things as inconsistent or obscure variable naming conventions, lack of comments in the source code, and so on. We don't intend to make the same mistake when writing Parrot. Hence this document.

We define three classes of conventions:

"must"
Items labelled must are mandatory; and code will not be accepted (apart from in exceptional circumstances) unless it obeys them.
"should"
Items labelled should are strong guidelines that should normally be followed unless there is a sensible reason to do otherwise.
"may"
Items labelled may are tentative suggestions to be used at your discretion.

Note that since Parrot is substantially implemented in C, these rules apply to C language source code unless otherwise specified.

Implementation

Language Standards and Portability

Code Formatting

The following must apply:

The following should apply:

Code Structure

The following must apply:

The following should apply

Smart Editor Style Support

All developers using Emacs must ensure that their Emacs instances load the elisp source file editor/parrot.el before opening Parrot source files. See "README.pod" in editor for instructions.

All source files must end with an editor instruction coda:

{{ XXX - Proper formatting and syntax coloring of C code under Emacs requires that Emacs know about typedefs. We should provide a simple script to update a list of typedefs, and parrot.el should read it or contain it. }}

Portability

Parrot runs on many, many platforms, and will no doubt be ported to ever more bizarre and obscure ones over time. You should never assume an operating system, processor architecture, endian-ness, size of standard type, or anything else that varies from system to system.

Since most of Parrot's development uses GNU C, you might accidentally depend on a GNU feature without noticing. To avoid this, know what features of gcc are GNU extensions, and use them only when they're protected by #ifdefs.

Defensive Programming

Use Parrot data structures instead of C strings and arrays

C arrays, including strings, are very sharp tools without safety guards, and Parrot is a large program maintained by many people. Therefore:

Don't use a char * when a Parrot STRING would suffice. Don't use a C array when a Parrot array PMC would suffice. If you do use a char * or C array, check and recheck your code for even the slightest possibility of buffer overflow or memory leak.

Note that efficiency of some low-level operations may be a reason to break this rule. Be prepared to justify your choices to a jury of your peers.

Pass only unsigned char to isxxx() and toxxx()

Pass only values in the range of unsigned char (and the special value -1, a.k.a. EOF) to the isxxx() and toxxx() library functions. Passing signed characters to these functions is a very common error and leads to incorrect behavior at best and crashes at worst. And under most of the compilers Parrot targets, char is signed.

The const keyword on arguments

Use the const keyword as often as possible on pointers. It lets the compiler know when you intend to modify the contents of something. For example, take this definition:

    int strlen(const char *p);

The const qualifier tells the compiler that the argument will not be modified. The compiler can then tell you that this is an uninitialized variable:

    char *p;
    int n = strlen(p);

Without the const, the compiler has to assume that strlen() is actually initializing the contents of p.

The const keyword on variables

If you're declaring a temporary pointer, declare it const, with the const to the right of the *, to indicate that the pointer should not be modified.

    Wango * const w = get_current_wango();
    w->min  = 0;
    w->max  = 14;
    w->name = "Ted";

This prevents you from modifying w inadvertantly.

    new_wango = w++; /* Error */

If you're not going to modify the target of the pointer, put a const to the left of the type, as in:

    const Wango * const w = get_current_wango();
    if (n < wango->min || n > wango->max) {
        /* do something */
    }

Localizing variables

Declare variables in the innermost scope possible.

    if (foo) {
        int i;
        for (i = 0; i < n; i++)
            do_something(i);
    }

Don't reuse unrelated variables. Localize as much as possible, even if the variables happen to have the same names.

    if (foo) {
        int i;
        for (i = 0; i < n; i++)
            do_something(i);
    }
    else {
        int i;
        for (i = 14; i > 0; i--)
            do_something_else(i * i);
    }

You could hoist the int i; outside the test, but then you'd have an i that's visible after it's used, which is confusing at best.

Subversion Properties

svn:ignore

Sometimes new files will be created in the configuration and build process of Parrot. These files should not show up when checking the distribution with

    svn status

or

    perl tools/dev/manicheck.pl

The list of these ignore files can be set up with:

    svn propedit svn:ignore <PATH>

In order to keep the two different checks synchronized, the MANIFEST and MANIFEST.SKIP files should be regenerated with:

    perl tools/dev/mk_manifest_and_skip.pl

and the files then committed to the Parrot svn repository.

svn:mime-type

The svn:mime-type property must be set to text/plain for all test files, and may be set to text/plain for other source code files in the repository. Using auto-props, Subversion can automatically set this property for you on test files. To enable this option, add the following to your ~/.subversion/config:

    [miscellany]
    enable-auto-props = yes
    [auto-props]
    *.t = svn:mime-type=text/plain

The t/distro/file_metadata.t test checks that the files needing this property have it set.

svn:keywords

The svn:keywords property should be set to:

    Author Date Id Revision

on each file with a mime-type of text/plain. Do this with the command:

    svn propset svn:keywords "Author Date Id Revision" <filename>

The t/distro/file_metadata.t test checks that the files needing this property have it set.

svn:eol-style

The svn:eol-style property makes sure that whenever a file is checked out of subversion it has the correct end-of-line characters appropriate for the given platform. Therefore, most files should have their svn:eol-style property set to native. However, this is not true for all files. Some input files to tests (such as the *.input and *.output files for PIR tests) need to have LF as their svn:eol-style property. The current list of such files is described in t/distro/file_metadata.t.

Set the svn:eol-style property to native with the command:

    svn propset svn:eol-style "native" <filename>

Set the svn:eol-style property to LF with the command:

    svn propset svn:eol-style "LF" <filename>

The t/distro/file_metadata.t test checks that the files needing this property have it set.

Naming Conventions

Filenames
Filenames must be assumed to be case-insensitive, in the sense that that you may not have two different files called Foo and foo. Normal source-code filenames should be all lower-case; filenames with upper-case letters in them are reserved for notice-me-first files such as README, and for files which need some sort of pre-processing applied to them or which do the preprocessing - e.g. a script foo.SH might read foo.TEMPLATE and output foo.c.The characters making up filenames must be chosen from the ASCII set A-Z,a-z,0-9 plus .-_An underscore should be used to separate words rather than a hyphen (-). A file should not normally have more than a single '.' in it, and this should be used to denote a suffix of some description. The filename must still be unique if the main part is truncated to 8 characters and any suffix truncated to 3 characters. Ideally, filenames should restricted to 8.3 in the first place, but this is not essential.Each subsystem foo should supply the following files. This arrangement is based on the assumption that each subsystem will -- as far as is practical -- present an opaque interface to all other subsystems within the core, as well as to extensions and embeddings.
foo.h
This contains all the declarations needed for external users of that API (and nothing more), i.e. it defines the API. It is permissible for the API to include different or extra functionality when used by other parts of the core, compared with its use in extensions and embeddings. In this case, the extra stuff within the file is enabled by testing for the macro PARROT_IN_CORE.
foo_private.h
This contains declarations used internally by that subsystem, and which must only be included within source files associated the subsystem. This file defines the macro PARROT_IN_FOO so that code knows when it is being used within that subsystem. The file will also contain all the 'convenience' macros used to define shorter working names for functions without the perl prefix (see below).
foo_globals.h
This file contains the declaration of a single structure containing the private global variables used by the subsystem (see the section on globals below for more details).
foo_bar.[ch] etc.
All other source files associated with the subsystem will have the prefix foo_.
Names of code entities
Code entities such as variables, functions, macros etc. (apart from strictly local ones) should all follow these general guidelines.
  • Multiple words or components should be separated with underscores rather than using tricks such as capitalization, e.g. new_foo_bar rather than NewFooBar or (gasp) newfoobar.
  • The names of entities should err on the side of verbosity, e.g. create_foo_from_bar() in preference to ct_foo_bar(). Avoid cryptic abbreviations wherever possible.
  • All entities should be prefixed with the name of the subsystem in which they appear, e.g. pmc_foo(), struct io_bar.
  • Functions with external visibility should be of the form Parrot_foo, and should only use typedefs with external visibility (or types defined in C89). Generally these functions should not be used inside the core, but this is not a hard and fast rule.
  • Variables and structure names should be all lower-case, e.g. pmc_foo.
  • Structure elements should be all lower-case, and the first component of the name should incorporate the structure's name or an abbreviation of it.
  • Typedef names should be lower-case except for the first letter, e.g. Foo_bar. The exception to this is when the first component is a short abbreviation, in which case the whole first component may be made uppercase for readability purposes, e.g. IO_foo rather than Io_foo. Structures should generally be typedefed.
  • Macros should have their first component uppercase, and the majority of the remaining components should be likewise. Where there is a family of macros, the variable part can be indicated in lowercase, e.g. PMC_foo_FLAG, PMC_bar_FLAG, ....
  • A macro which defines a flag bit should be suffixed with _FLAG, e.g. PMC_readonly_FLAG (although you probably want to use an enum instead.)
  • A macro which tests a flag bit should be suffixed with _TEST, e.g. if (PMC_readonly_TEST(foo)) ...
  • A macro which sets a flag bit should be suffixed with _SET, e.g. PMC_readonly_SET(foo);
  • A macro which clears a flag bit should be suffixed with _CLEAR, e.g. PMC_readonly_CLEAR(foo);
  • A macro defining a mask of flag bits should be suffixed with _MASK, e.g. foo &= ~PMC_STATUS_MASK (but see notes on extensibility below).
  • Macros can be defined to cover common flag combinations, in which case they should have _SETALL, _CLEARALL, _TESTALL or _TESTANY suffixes as appropriate, to indicate aggregate bits, e.g. PMC_valid_CLEARALL(foo).
  • A macro defining an auto-configuration value should be prefixed with HAS_, e.g. HAS_BROKEN_FLOCK, HAS_EBCDIC.
  • A macro indicating the compilation 'location' should be prefixed with IN_, e.g. PARROT_IN_CORE, PARROT_IN_PMC, PARROT_IN_X2P. Individual include file visitations should be marked with PARROT_IN_FOO_H for file foo.h
  • A macro indicating major compilation switches should be prefixed with USE_, e.g. PARROT_USE_STDIO, USE_MULTIPLICITY.
  • A macro that may declare stuff and thus needs to be at the start of a block should be prefixed with DECL_, e.g. DECL_SAVE_STACK. Note that macros which implicitly declare and then use variables are strongly discouraged, unless it is essential for portability or extensibility. The following are in decreasing preference style-wise, but increasing preference extensibility-wise.
  •     { Stack sp = GETSTACK;  x = POPSTACK(sp) ... /* sp is an auto variable */
        { DECL_STACK(sp);  x = POPSTACK(sp); ... /* sp may or may not be auto */
        { DECL_STACK; x = POPSTACK; ... /* anybody's guess */
Global Variables
Global variables must never be accessed directly outside the subsystem in which they are used. Some other method, such as accessor functions, must be provided by that subsystem's API. (For efficiency the 'accessor functions' may occasionally actually be macros, but then the rule still applies in spirit at least).All global variables needed for the internal use of a particular subsystem should all be declared within a single struct called foo_globals for subsystem foo. This structure's declaration is placed in the file foo_globals.h. Then somewhere a single compound structure will be declared which has as members the individual structures from each subsystem. Instances of this structure are then defined as a one-off global variable, or as per-thread instances, or whatever is required.[Actually, three separate structures may be required, for global, per-interpreter and per-thread variables.]Within an individual subsystem, macros are defined for each global variable of the form GLOBAL_foo (the name being deliberately clunky). So we might for example have the following macros:
    /* perl_core.h or similar */

    #ifdef HAS_THREADS
    #  define GLOBALS_BASE (aTHX_->globals)
    #else
    #  define GLOBALS_BASE (Parrot_globals)
    #endif

    /* pmc_private.h */

    #define GLOBAL_foo   GLOBALS_BASE.pmc.foo
    #define GLOBAL_bar   GLOBALS_BASE.pmc.bar
    ... etc ...

Code Comments

The importance of good code documentation cannot be stressed enough. To make your code understandable by others (and indeed by yourself when you come to make changes a year later), the following conventions apply to all source files.

Developer files
Each source file (e.g. a foo.c, foo.h pair), should contain inline POD documentation containing information on the implementation decisions associated with the source file. (Note that this is in contrast to PDDs, which describe design decisions). In addition, more discussive documentation can be placed in *.pod files in the docs/dev directory. This is the place for mini-essays on how to avoid overflows in unsigned arithmetic, or on the pros and cons of differing hash algorithms, and why the current one was chosen, and how it works.In principle, someone coming to a particular source file for the first time should be able to read the inline documentation file and gain an immediate overview of what the source file is for, the algorithms it implements, etc.The POD documentation should follow the standard POD layout:
Copyright
The Parrot copyright statement.
SVN
A SVN id string.
NAME
src/foo.c - Foo
SYNOPSIS
When appropriate, some simple examples of usage.
DESCRIPTION
A description of the contents of the file, how the implementation works, data structures and algorithms, and anything that may be of interest to your successors, e.g. benchmarks of differing hash algorithms, essays on how to do integer arithmetic.
SEE ALSO
Links to pages and books that may contain useful information relevant to the stuff going on in the code -- e.g. the book you stole the hash function from.
Don't include author information in individual files. Author information can be added to the CREDITS file. (Languages are an exception to this rule, and may follow whatever convention they choose.)
Per-section comments
If there is a collection of functions, structures or whatever which are grouped together and have a common theme or purpose, there should be a general comment at the start of the section briefly explaining their overall purpose. (Detailed essays should be left to the developer file). If there is really only one section, then the top-of-file comment already satisfies this requirement.
Per-entity comments
Every non-local named entity, be it a function, variable, structure, macro or whatever, must have an accompanying comment explaining its purpose. This comment must be in the special format described below, in order to allow automatic extraction by tools - for example, to generate per API man pages, perldoc -f style utilities and so on.Often the comment need only be a single line explaining its purpose, but sometimes more explanation may be needed. For example, "return an Integer Foo to its allocation pool" may be enough to demystify the function del_I_foo()Each comment should be of the form
    /*

    =item C<function(arguments)>

    Description.

    =cut

    */
This inline POD documentation is parsed to HTML by running:
    % perl tools/docs/write_docs.pl -s
Optimizations
Whenever code has deliberately been written in an odd way for performance reasons, you should point this out - if nothing else, to avoid some poor schmuck trying subsequently to replace it with something 'cleaner'.
    /* The loop is partially unrolled here as it makes it a lot faster.
     * See the file in docs/dev for the full details
     */
General comments
While there is no need to go mad commenting every line of code, it is immensely helpful to provide a "running commentary" every 10 lines or so if nothing else, this makes it easy to quickly locate a specific chunk of code. Such comments are particularly useful at the top of each major branch, e.g.
    if (FOO_bar_BAZ(**p+*q) <= (r-s[FOZ & FAZ_MASK]) || FLOP_2(z99)) {
        /* we're in foo mode: clean up lexicals */
        ... (20 lines of gibberish) ...
    }
    else if (...) {
        /* we're in bar mode: clean up globals */
        ... (20 more lines of gibberish) ...
    }
    else {
        /* we're in baz mode: self-destruct */
        ....
    }

Extensibility

Over the lifetime of Parrot, the source code will undergo many major changes never envisaged by its original authors. To this end, your code should balance out the assumptions that make things possible, fast or small, with the assumptions that make it difficult to change things in future. This is especially important for parts of the code which are exposed through APIs -- the requirements of source or binary compatibility for such things as extensions can make it very hard to change things later on.

For example, if you define suitable macros to set/test flags in a struct, then you can later add a second word of flags to the struct without breaking source compatibility. (Although you might still break binary compatibility if you're not careful.) Of the following two methods of setting a common combination of flags, the second doesn't assume that all the flags are contained within a single field:

    foo->flags |= (FOO_int_FLAG | FOO_num_FLAG | FOO_str_FLAG);
    FOO_valid_value_SETALL(foo);

Similarly, avoid using a char* (or {char*,length}) if it is feasible to later use a PMC * at the same point: c.f. UTF-8 hash keys in Perl 5.

Of course, private code hidden behind an API can play more fast and loose than code which gets exposed.

Performance

We want Parrot to be fast. Very fast. But we also want it to be portable and extensible. Based on the 90/10 principle, (or 80/20, or 95/5, depending on who you speak to), most performance is gained or lost in a few small but critical areas of code. Concentrate your optimization efforts there.

Note that the most overwhelmingly important factor in performance is in choosing the correct algorithms and data structures in the first place. Any subsequent tweaking of code is secondary to this. Also, any tweaking that is done should as far as possible be platform independent, or at least likely to cause speed-ups in a wide variety of environments, and do no harm elsewhere.

If you do put an optimization in, time it on as many architectures as you can, and be suspicious of it if it slows down on any of them! Perhaps it will be slow on other architectures too (current and future). Perhaps it wasn't so clever after all? If the optimization is platform specific, you should probably put it in a platform-specific function in a platform-specific file, rather than cluttering the main source with zillions of #ifdefs.

And remember to document it.

Exemptions

Not all files can strictly fall under these guidelines as they are automatically generated by other tools, or are external files included in the Parrot repository for convenience. Such files include the C header and source files automatically generated by (f)lex and yacc/bison, and some of the Perl modules under the lib/ directory.

To exempt a file (or directory of files) from checking by the coding standards tests, one must edit the appropriate exemption list within lib/Parrot/Distribution.pm (in either of the methods is_c_exemption() or is_perl_exemption()). One can use wildcards in the list to exempt, for example, all files under a given directory.

References

none