Documenting Python APIs¶

We document Python code in three ways:

By writing docstrings for all public python objects (modules, classes, methods, functions and constants).

These docstrings are exposed to users in a variety of contexts, from developers reading the code, to interactive Python users introspecting an object with help(), Jupyter notebook users typing object?, and finally to readers of this user guide.

Docstrings are the public specification of our Python API.
By commenting our code internally with hash marks (#).

These comments are meant to be read only by developers reading and editing the source code.
By allowing Python objects to be introspected interactively with the __str__ and __repr__ magic methods.

This page focuses on public code documentation through docstrings, while the latter two are discussed in our DM Python Style Guide.

Treat the guidelines on this page as an extension of the DM Python Style Guide.

Note

Changes to this document must be approved by the System Architect (RFC-24). To request changes to these standards, please file an RFC.

Basic Format of Docstrings¶

Python docstrings are special strings that form the __doc__ attributes attached to modules, classes, methods and functions. Docstrings are specified by PEP 257.

Docstrings MUST be delimited by triple double quotes¶

Docstrings must be delimited by triple double quotes: """. This allows docstrings to span multiple lines. You may use u""" for unicode but it is usually preferable to stick to ASCII.

For consistency, do not use triple single quotes: '''.

Docstrings SHOULD begin with `"""` and terminate with `"""` on its own line¶

The docstring’s summary sentence occurs on the same line as the opening """.

The terminating """ should be on its own line, even for ‘one-line’ docstrings (this is a minor departure from PEP 257). For example, a one-line docstring:

"""Sum numbers in an array.
"""

(Note: one-line docstrings are rarely used for public APIs, see Common Structure of Docstrings.)

An example of a multi-paragraph docstring:

"""Sum numbers in an array.

Parameters
----------
values : iterable
   Python iterable whose values are summed.

Returns
-------
sum : `float`
   Sum of `values`.
"""

Docstrings of methods and functions SHOULD NOT be preceded or followed by a blank line¶

Inside a function or method, there should be no blank lines surrounding the docstring.

def sum(values):
    """Sum numbers in an array.

    Parameters
    ----------
    values : iterable
       Python iterable whose values are summed.

    Returns
    -------
    sum : `float`
       Sum of `values`.
    """
    pass

Docstrings of classes SHOULD be followed, but not preceded, by a blank line¶

Like method and function docstrings, the docstring should immediately follow the class definition, without a blank space. However, there should be a single blank line before following code such as class variables or the __init__ method.

class Point(object):
    """Point in a 2D cartesian space.

    Parameters
    ----------
    x, y : `float`
       Coordinate of the point.
    """

    def __init__(x, y):
        self.x = x
        self.y = y

Docstring content MUST be indented with the code’s scope¶

For example:

def sum(values):
    """Sum numbers in an array.

    Parameters
    ----------
    values : iterable
       Python iterable whose values are summed.
    """
    pass

Not:

def sum(values):
    """Sum numbers in an array.

Parameters
----------
values : iterable
   Python iterable whose values are summed.
"""
    pass

Docstring Placement¶

Modules¶

Module-level docstrings must be placed as close to the top of the Python file as possible: below any #!/usr/bin/env python and license statements, but above imports. See also: Standard code order SHOULD be followed.

Module docstrings should not be indented. For example:

#
# LSST Data Management System
# See COPYRIGHT file at the top of the source tree.
#
# [...]
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#
"""Summary of MyModule.

Extended discussion of my module.
"""

import lsst.afw.table as afw_table
# [...]

Classes, Methods, and Functions¶

Class/method/function docstrings must be placed directly below the declaration, and indented according to the code scope.

class MyClass(object):
    """Summary of MyClass.

    Additional discussion.
    """

    def __init__(self):
        pass

    def method(self):
        """Summary of method.

        Extended Discussion of my method.
        """
        pass


def my_function():
    """Summary of my_function.

    Extended discussion of my_function.
    """
    pass

Note that the class docstring takes the place of a docstring for the __init__ method; __init__ has no docstring.

ReStructuredText in Docstrings¶

We use reStructuredText to mark up and give semantic meaning to text in docstrings. ReStructuredText is lightweight enough to read in raw form, such as command line terminal printouts, but is also parsed and rendered with our Sphinx-based documentation build system. All of the style guidance for using reStructuredText from our ReStructuredText Style Guide applies in docstrings with a few exceptions defined here.

No space between headers and paragraphs¶

For docstrings, the Numpydoc standard is to omit any space between a header and the following paragraph.

For example

"""A summary

A Headline
----------
A paragraph
"""

This deviation from the normal style guide is in keeping with Python community idioms and to save vertical space in terminal help printouts.

Top level headers are defined with ‘-‘¶

In docstrings, the top level header is marked up with a -, the third level listed in our ReStructuredText Style Guide. The header hierarchy is thus:

Sections -,
Subsections ^,
Subsubsections ".

This deviation from our reST Style Guide is in keeping with NumPy community idioms, and required by our Sphinx tooling.

Common Structure of Docstrings¶

We organize Python docstrings into sections that appear in a common order. This format follows the Numpydoc format (used by NumPy, SciPy, and Astropy, among other scientific Python packages) rather than the format described in PEP 287. The sections and their relative order is:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Parameters (if applicable; for classes, methods, and functions)
Methods (if applicable; for classes)
Attributes (if applicable; for classes)
Returns or Yields (if applicable; for functions, methods, and generators)
Other Parameters (if applicable; for classes, methods, and functions)
Raises (if applicable)
See Also (optional)
Notes (optional)
References (optional)
Examples (optional)

For summaries of how these docstring sections are composed in specific contexts, see:

Documenting Modules
Documenting Classes
Documenting Methods and Functions
Documenting Constants, Class Properties, and Attributes

Short Summary¶

A one-line summary that does not use variable names or the function’s name:

def add(a, b):
    """Sum two numbers.
    """
    return a + b

For functions and methods, the summary should be written in the imperative voice (i.e., as a command that the API consumer is giving).

Deprecation Warning¶

A section (where applicable) to warn users that the object is deprecated. Section contents should include:

In what stack version the object was deprecated, and when it will be removed.
Reason for deprecation if this is useful information (e.g., object is superseded, duplicates functionality found elsewhere, etc.).
New recommended way of obtaining the same functionality.

This section should use the note Sphinx directive instead of an underlined section header.

.. note:: Deprecated in 11_0
          `ndobj_old` will be removed in 12_0, it is replaced by
          `ndobj_new` because the latter works also with array subclasses.

Extended Summary¶

A few sentences giving an extended description. This section should be used to clarify functionality, not to discuss implementation detail or background theory, which should rather be explored in the ‘Notes’ section below. You may refer to the parameters and the function name, but parameter descriptions still belong in the ‘Parameters’ section.

Parameters¶

For functions, methods and classes.

‘Parameters’ is a description of a function or method’s arguments and their respective types.

Parameters
----------
x : type
    Description of parameter `x`.

Notice that the description is indented by four spaces from the prior {name} : {type} line of each argument. If a description spans more than one line, the continuation lines must be indented to the same level.

Arguments should be listed in the same order as they appear in the function or method signature.

Parameter Types¶

Be as precise as possible when describing types for parameters. The type description is free-form text, making it possible to list several supported types or indicate nuances. Complex and lengthy descriptions can be moved to the description field.

Parameters
----------
filename : `str`
    Description of `filename`.
copy : `bool`
    Description of `copy`.
dtype : data-type
    Description of `dtype`.
iterable : iterable object
    Description of `iterable`.
shape : `int` or `tuple` of int
    Description of `shape`.
files : `list` of `str`
    Description of `files`.

Note that concrete types are wrapped in backticks, which is the default role in reStructuredText. When possible, Sphinx will make a link to the API reference for that object using intersphinx. (In docstrings, :py:obj: is the default role.)

For instances of classes, provide the full namespace to the class, such as `lsst.afw.table.ExposureTable`.

When a parameter can only assume one of a fixed set of values, those values can be listed in braces:

order : {'C', 'F', 'A'}
    Description of `order`.

Optional Parameters¶

For keyword arguments, add ‘optional’ to the type specification:

x : `int`, optional

Optional keyword parameters have default values, which are automatically documented as part of the function or method’s signature. Default values can also be detailed in the description:

Parameters
----------
x : `int`, optional
    Description of parameter `x` (the default is -1, which implies summation
    over all axes).

Shorthand¶

When two or more consecutive input parameters have exactly the same type, shape and description, they can be combined:

x1, x2 : array-like
    Input arrays, description of `x1`, `x2`.

Methods¶

For classes.

If a class has a very large number of methods, which are hard to discover, an additional ‘Methods’ section can be provided to list them:

Methods
-------
read(filename)
   Read a table from a file
sort(column, order='ascending')
   Sort by `column`

Do not list private methods in the ‘Methods’ section. If it is necessary to explain a private method (use with care!), it can be mentioned in the Extended Summary or Notes sections.

Do not list self as the first parameter of a method.

Attributes¶

For classes.

An ‘Attributes’ section, located below the ‘Parameters’ section, may be used to describe class variables:

Attributes
----------
x : `float`
    The X coordinate.
y : `float`
    The Y coordinate.

Attributes that are properties and have their own docstrings can be simply listed by name:

Attributes
----------
real
imag
x : `float`
    The X coordinate
y : `float`
    The Y coordinate

Returns¶

For functions and methods.

‘Returns’ is an explanation of the returned values and their types, in the same format as ‘Parameters’.

If a sequence of values is returned, each value may be separately listed, in order:

Returns
-------
x : `int`
    Description of x.
y : `int`
    Description of y.

If a return type is dict, ensure that the key-value pairs are documented in the description.

Yields¶

For generators.

‘Yields’ is used identically to ‘Returns’, but for generators.

Other Parameters¶

For classes, methods and functions.

‘Other Parameters’ is an optional section used to describe infrequently used parameters. It should only be used if a function has a large number of keyword parameters, to prevent cluttering the Parameters section.

Raises¶

For classes, methods and functions.

‘Raises’ is an optional section detailing which errors get raised and under what conditions:

Raises
------
`IOError`
    If the file could not be read.

This section should be used judiciously—only for errors that are non-obvious or have a large chance of getting raised.

Notes¶

Notes is an optional section that provides additional information about the code, possibly including a discussion of the algorithm. This section may include mathematical equations, written in LaTeX format:

The FFT is a fast implementation of the discrete Fourier transform:

.. math:: X(e^{j\omega } ) = x(n)e^{ - j\omega n}

Longer equations can also be typeset underneath the math directive:

The discrete-time Fourier time-convolution property states that

.. math::

   x(n) * y(n) \Leftrightarrow X(e^{j\omega } )Y(e^{j\omega } )\\
   another equation here

Math can also be used inline:

The value of :math:`\omega` is larger than 5.

Variable names are displayed in typewriter font, obtained by using \mathtt{var}:

We square the input parameter `alpha` to obtain
:math:`\mathtt{alpha}^2`.

See Math for more details on math typesetting in reStructuredText.

Note that LaTeX is not particularly easy to read, so use equations sparingly.

Images are allowed, but should not be central to the explanation; users viewing the docstring as text must be able to comprehend its meaning without resorting to an image viewer. These additional illustrations are included using:

.. image:: filename

where filename is a path relative to the reference guide source directory.

References¶

References cited in the ‘Notes’ section may be listed here, e.g. if you cited the article below using the text [1]_, include it as in the list as follows:

.. [1] O. McNoleg, "The integration of GIS, remote sensing,
   expert systems and adaptive co-kriging for environmental habitat
   modelling of the Highland Haggis using object-oriented, fuzzy-logic
   and neural-network techniques," Computers & Geosciences, vol. 22,
   pp. 585-588, 1996.

Note that Web pages should be referenced with regular inline links.

References are meant to augment the docstring, but should not be required to understand it. References are numbered, starting from one, in the order in which they are cited.

We may support bibtex-based references instead instead of explicitly writing bibliographies in docstrings.

Examples¶

‘Examples’ is an optional section for examples, using the doctest format. These examples do not replace unit tests, but are intended to be tested to ensure documentation and code are consistent. While optional, this section is very strongly encouraged.

When multiple examples are provided, they should be separated by blank lines. Comments explaining the examples should have blank lines both above and below them:

>>> np.add(1, 2)
3

Comment explaining the second example

>>> np.add([1, 2], [3, 4])
array([4, 6])

For tests with a result that is random or platform-dependent, mark the output as such:

>>> np.random.rand(2)
array([ 0.35773152,  0.38568979])  #random

It is not necessary to use the doctest markup <BLANKLINE> to indicate empty lines in the output.

Documenting Modules¶

Module docstrings are placed after the boilerplate and before any imports or other code. Module docstrings contain the following sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
See Also (optional)

Documenting Classes¶

Class docstrings are placed directly after the class definition, and serve to document both the class as a whole, and the arguments passed to the __init__ constructor. Class docstrings contain the following sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Parameters (if applicable)
Methods (if applicable)
Attributes (if applicable)
Other Parameters (if applicable)
Raises (if applicable)
See Also (optional)
Notes (optional)
References (optional)
Examples (optional)

Note that the Methods section is only used if the method list is extremely long. In general, trust that the tables of contents in the user guide pages will provide useful summaries of a class’s methods.

class SkyCoordinate(object):
    """Coordinate on the sky as Right Ascension and Declination.

    Parameters
    ----------
    ra : float
       Right ascension (degrees).
    dec : float
       Declination (degrees).
    frame : {'icrs', 'fk5'}, optional
       Coordinate frame.

    Raises
    ------
    `ValueError` : Input angles are outside range.

    See also
    --------
    `GalacticCoordinate`

    Examples
    --------
    To define the coordinate of the M31 galaxy,

    >>> m31_coord = SkyCoordinate(10.683333333, 41.269166667)
    SkyCoordinate(10.683333333, 41.269166667, frame='icrs')
    """

    def __init__(self, ra, dec, frame='icrs'):
        pass

Documenting Methods and Functions¶

Method and function docstrings contain the following sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Parameters (if applicable)
Returns or Yields (if applicable)
Other Parameters (if applicable)
Raises (if applicable)
See Also (optional)
Notes (optional)
References (optional)
Examples (optional)

A minimal example:

def log(message, level):
    """Submit a message to the log.

    Parameters
    ----------
    message : `str`
       Log message.
    level : `str`
       Priority level of the log message.
    """

Documenting Constants, Class Properties, and Attributes¶

Constants in modules, and properties and attributes in classes are all similar in that their values are accessed with arguments. At minimum, constants/properties/attributes should have a summary line, but can also have a more complete structure with sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Notes (optional)
References (optional)
Examples (optional)

In the short summary, a description of the type should be included:

NAME = 'LSST'
"""Name of the project (str)"""

Note that class attributes can alternatively be documented in an Attributes section of the class’s docstring. This is particularly useful when the attribute is not set in the class scope, but rather in a method such as __init__.

class Answer(object):
    """Container for the answer.

    Attributes
    ----------
    answer : obj
       The answer.
    source
    """

    def __init__(self):
        self.contents = 42

    @property
    def source(self):
        """Purveyor of the answer."""
        return 'Deep Thought'

Acknowledgements¶

These docstring guidelines are derived/adapted from the NumPy and Astropy documentation.