Note

Changes to this document must be approved by the System Architect (RFC-24). To request changes to these standards, please file an RFC.

Documenting Python APIs with Docstrings¶

We use Python docstrings to create reference documentation for our Python APIs. Docstrings are read by developers, interactive Python users, and readers of our online documentation. This page describes how to write these docstrings in Numpydoc, DM’s standard format:

Basic Format of Docstrings.
ReStructuredText in Docstrings.
Numpydoc Sections in Docstrings.
Documenting Modules.
Documenting Classes.
Documenting Methods and Functions.
Documenting Constants and Class Attributes.

Treat the guidelines on this page as an extension of the DM Python Style Guide.

Basic Format of Docstrings¶

Python docstrings form the __doc__ attributes attached to modules, classes, methods and functions. See PEP 257 for background.

Docstrings MUST be delimited by triple double quotes¶

Docstrings must be delimited by triple double quotes: """. This allows docstrings to span multiple lines. You may use u""" for unicode, but it’s usually preferable to stick to ASCII.

For consistency, do not use triple single quotes: '''.

Docstrings SHOULD begin with `"""` and terminate with `"""` on its own line¶

The docstring’s summary sentence occurs on the same line as the opening """.

The terminating """ should be on its own line, even for ‘one-line’ docstrings (this is a minor departure from PEP 257). For example, a one-line docstring:

"""Sum numbers in an array.
"""

(Note: one-line docstrings are rarely used for public APIs, see Numpydoc Sections in Docstrings.)

An example of a multi-paragraph docstring:

"""Sum numbers in an array.

Parameters
----------
values : iterable
   Python iterable whose values are summed.

Returns
-------
sum : `float`
   Sum of ``values``.
"""

Docstrings of methods and functions SHOULD NOT be preceded or followed by a blank line¶

Inside a function or method, there should be no blank lines surrounding the docstring:

def sum(values):
    """Sum numbers in an array.

    Parameters
    ----------
    values : iterable
       Python iterable whose values are summed.

    Returns
    -------
    sum : `float`
       Sum of ``values``.
    """
    pass

Docstrings of classes SHOULD be followed, but not preceded, by a blank line¶

Like method and function docstrings, the docstring should immediately follow the class definition, without a blank space. However, there should be a single blank line before following code such as class variables or the __init__ method:

class Point(object):
    """Point in a 2D cartesian space.

    Parameters
    ----------
    x, y : `float`
       Coordinate of the point.
    """

    def __init__(x, y):
        self.x = x
        self.y = y

Docstring content MUST be indented with the code’s scope¶

For example:

def sum(values):
    """Sum numbers in an array.

    Parameters
    ----------
    values : iterable
       Python iterable whose values are summed.
    """
    pass

Not:

def sum(values):
    """Sum numbers in an array.

Parameters
----------
values : iterable
   Python iterable whose values are summed.
"""
    pass

ReStructuredText in Docstrings¶

We use reStructuredText to mark up and give semantic meaning to text in docstrings. ReStructuredText is lightweight enough to read in raw form, such as command line terminal printouts, but is also parsed and rendered with our Sphinx-based documentation build system. All of the style guidance for using reStructuredText from our ReStructuredText Style Guide applies in docstrings with a few exceptions defined here.

No space between headers and paragraphs¶

For docstrings, the Numpydoc standard is to omit any space between a header and the following paragraph.

For example

"""A summary

Notes
-----
The content of the notes section directly follows the header, with no blank line.
"""

This deviation from the normal style guide is in keeping with Python community idioms and to save vertical space in terminal help printouts.

Sections are restricted to the Numpydoc section set¶

Sections must be from the set of standard Numpydoc sections (see Numpydoc Sections in Docstrings). You cannot introduce new section headers, or use the full reStructuredText subsection hierarchy, since these subsections won’t be parsed by the documentation toolchain.

Always use the dash (-) to underline sections. For example:

def myFunction(a):
    """Do something.

    Parameters
    ----------
    [...]

    Returns
    -------
    [...]

    Notes
    -----
    [...]
    """

Simulate subsections with bold text¶

Conventional reStructuredText subsections are not allowed in docstrings, given the previous guideline. However, you may structure long sections with bold text that simulates subsection headers. This technique is useful for the Notes and Examples Numpydoc sections. For example:

def myFunction(a):
    """Do something.

    [...]

    Examples
    --------
    **Example 1**

    [...]

    **Example 2**

    [...]
    """

Line Lengths¶

Hard-wrap text in docstrings to match the line length allowed by the coding standard.

Note

In the future we may require shorter line lengths specifically for docstrings. See RFC-107.

Marking Up Parameter Names¶

The default reStructuredText role in docstrings is :py:obj:. Sphinx automatically generates links when the API names are marked up in single backticks. For example: `str` or `lsst.pipe.base.Struct`.

You cannot use this role to mark up parameters, however. Instead, use the code literal role (double backticks) to mark parameters and return variables in monospace type. For example, the description for format references the should_plot parameter:

Parameters
----------
should_plot : `bool`
    Plot the fit if `True`.
plot_format : `str`, optional
    Format of the plot when ``should_plot`` is `True`.

Numpydoc Sections in Docstrings¶

We organize Python docstrings into sections that appear in a common order. This format follows the Numpydoc standard (used by NumPy, SciPy, and Astropy, among other scientific Python packages) rather than the format described in PEP 287. These are the sections and their relative order:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Parameters (if applicable; for classes, methods, and functions)
Returns or Yields (if applicable; for functions, methods, and generators)
Other Parameters (if applicable; for classes, methods, and functions)
Raises (if applicable)
See Also (optional)
Notes (optional)
References (optional)
Examples (optional)

For summaries of how these docstring sections are composed in specific contexts, see:

Documenting Modules
Documenting Classes
Documenting Methods and Functions
Documenting Constants and Class Attributes
Documenting Class Properties

Short Summary¶

A one-line summary that does not use variable names or the function’s name:

def add(a, b):
    """Sum two numbers.
    """
    return a + b

For functions and methods, write in the imperative voice. That is, the summary is treated a command that the API consumer can give. Some examples:

Get metadata for all tasks.
Make a `lsst.pex.config.ConfigurableField` for this task.
Create a `Measurement` instance from a parsed YAML or JSON document.

Deprecation Warning¶

A section (where applicable) to warn users that the object is deprecated. Section contents should include:

In what stack version the object was deprecated, and when it will be removed.
Reason for deprecation if this is useful information (for example, the object is superseded, or duplicates functionality found elsewhere).
New recommended way of obtaining the same functionality.

This section should use the note Sphinx directive instead of an underlined section header.

.. note:: Deprecated in 11_0
          `ndobj_old` will be removed in 12_0, it is replaced by
          `ndobj_new` because the latter works also with array subclasses.

Extended Summary¶

A few sentences giving an extended description. This section should be used to clarify functionality, not to discuss implementation detail or background theory, which should rather be explored in the ‘Notes’ section below. You may refer to the parameters and the function name, but parameter descriptions still belong in the ‘Parameters’ section.

Parameters¶

For functions, methods and classes.

‘Parameters’ is a description of a function or method’s arguments and their respective types. Parameters should be listed in the same order as they appear in the function or method signature.

For example:

def calcDistance(x, y, x0=0., y0=0.):
    """Calculate the distance between two points.

    Parameters
    ----------
    x : `float`
        X-axis coordinate.
    y : `float`
        Y-axis coordinate.
    x0 : `float`, optional
        X-axis coordinate for the second point (the origin, by default).
    y0 : `float`, optional
        Y-axis coordinate for the second point (the origin, by default).

    [...]
    """

Each parameter is declared with a line formatted as {name} : {type} that is justified to the docstring. A single space is required before and after the colon (:). The name corresponds to the variable name in the function or method’s arguments. The type is described below (Describing Parameter Types). The description is indented by four spaces relative to the docstring and appears without a preceding blank line.

Normally parameters are documented consecutively, without blank lines between (see the earlier example). However, if the descriptions of an individual parameter span multiple paragraphs, or include lists, then you must separate each parameter with a blank line. For example:

Parameters
----------
output_path : `str`
    Filepath where the plot will be saved.

plot_settings : `dict`, optional
    Settings for the plot that may include these fields:

    - ``'dpi'``: resolution of the plot in dots per inch (`int`).
    - ``'rasterize'``: if `True`, then rasterize the plot. `False` by default.

Describing Parameter Types¶

Be as precise as possible when describing parameter types. The type description is free-form text, making it possible to list several supported types or indicate nuances. Complex and lengthy type descriptions can be partially moved to the parameter’s description field. The following sections will help you deal with the different kinds of types commonly seen.

Concrete types¶

Wrap concrete types in backticks (in docstrings, single backticks are equivalent to :py:obj:) to make a link to either an internal API or an external API that is supported by intersphinx. This works for both built-in types and most importable objects:

Parameters
----------
filename : `str`
    [...]
n : `int`
    [...]
verbose : `bool`
    [...]
items : `list` or `tuple`
    [...]
magnitudes : `numpy.ndarray`
    [...]
struct : `lsst.pipe.base.Struct`
    [...]

In general, provide the full namespace to the object, such as `lsst.pipe.base.Struct`. It may be possible to reference objects in the same namespace as the current module without any namespace prefix. Always check the compiled documentation site to ensure the link worked.

Choices¶

When a parameter can only assume one of a fixed set of values, those choices can be listed in braces:

order : {'C', 'F', 'A'}
    [...]

Sequence types¶

When a type is a sequence container (like a list or tuple), you can describe the type of the contents. For example:

mags : `list` of `float`
    Sequence of magnitudes.

Dictionary types¶

For dictionaries it is usually best to document the keys and their values in the parameter’s description:

settings : `dict`
    Settings dictionary with fields:

    - ``color``: Hex colour code (`str`).
    - ``size``: Point area in pixels (`float`).

Array types¶

For Numpy arrays, try to include the dimensionality:

coords : `numpy.ndarray`, (N, 2)
    [...]
flags : `numpy.ndarray`, (N,)
    [...]
image : `numpy.ndarray`, (Ny, Nx)
    [...]

Choose conventional variables or labels to describe dimensions, like N for the number of sources or Nx, Ny for rectangular dimensions.

Callable types¶

For callback functions, describe the type as callable:

likelihood : callable
    Likelihood function that takes two positional arguments:

    - ``x``: current parameter (`float`).
    - ``extra_args``: additional arguments (`dict`).

Optional Parameters¶

For keyword arguments with useful defaults, add optional to the type specification:

x : `int`, optional

Optional keyword parameters have default values, which are automatically documented as part of the function or method’s signature. You can also explain defaults in the description:

x : `int`, optional
    Description of parameter ``x`` (the default is -1, which implies summation
    over all axes).

Shorthand¶

When two or more consecutive input parameters have exactly the same type, shape and description, they can be combined:

x1, x2 : array-like
    Input arrays, description of `x1`, `x2`.

Returns¶

For functions and methods.

‘Returns’ is an explanation of the returned values and their types, in the same format as ‘Parameters’.

If a sequence of values is returned, each value may be separately listed, in order:

def getCoord(self):
    """Get the point's pixel coordinate.

    Returns
    -------
    x : `int`
        X-axis pixel coordinate.
    y : `int`
        Y-axis pixel coordinate.
    """
    return self._x, self._y

If a return type is dict, ensure that the key-value pairs are documented in the description:

def getCoord(self):
    """Get the point's pixel coordinate.

    Returns
    -------
    pixelCoord : `dict`
       Pixel coordinates with fields:

       - ``x``: x-axis coordinate (`int`).
       - ``y``: y-axis coordinate (`int`).
     """
     return {'x': self._x, 'y': self._y}

Note that the names of the returned variables do not necessarily correspond to the names of variables. In the previous examples, the variables x, y, and pixelCoord never existed in the method scope. Simply choose a variable-like name that is clear. Order is important.

If a returned variable is named in the method or function scope, you will usually want to use that name for clarity. For example:

def getDistance(self, x, y):
    """Compute the distance of the point to an (x, y) coordinate.

    [...]

    Returns
    -------
    distance : `float`
        Distance, in units of pixels.
    """
    distance = np.hypot(self._x - x, self._y - y)
    return distance

Yields¶

For generators.

‘Yields’ is used identically to ‘Returns’, but for generators. For example:

def items(self):
    """Iterate over items in the container.

    Yields
    ------
    key : `str`
        Item key.
    value : obj
        Item value.
    """
    for key, value in self._data.items():
        yield key, value

Other Parameters¶

For classes, methods and functions.

‘Other Parameters’ is an optional section used to describe infrequently used parameters. It should only be used if a function has a large number of keyword parameters, to prevent cluttering the Parameters section. In practice, this section is seldom used.

Raises¶

For classes, methods and functions.

‘Raises’ is an optional section for describing the exceptions that can be raised. You usually cannot document all possible exceptions that might get raised by the entire call stack. Instead, focus on:

Exceptions that are commonly raised.
Exceptions that are unique (custom exceptions, in particular).
Exceptions that are important to using an API.

The ‘Raises’ section looks like this:

Raises
------
IOError
    Raised if the input file cannot be read.
TypeError
    Raised if parameter ``example`` is an invalid type.

Don’t wrap each exception’s name with backticks, as we do when describing types in Parameters and Returns). No namespace prefix is needed when referring to exceptions in the same module as the API. Providing the full namespace is often a good idea, though.

The description text is indented by four spaces from the docstring’s left justification. Like the description fields for Parameters and Returns, the description can consist of multiple paragraphs and lists.

Stylistically, write the first sentence of each description in the form:

Raised if [insert circumstance].

Notes¶

Notes is an optional section that provides additional information about the code, possibly including a discussion of the algorithm. Most reStructuredText formatting is allowed in the Notes section, including:

When using images, remember that many developers and users will be reading the docstring in its raw source form. Images should add information, but the docstring should still be useful and complete without them.

See also ReStructuredText in Docstrings for restrictions.

References¶

References cited in the ‘Notes’ section are listed here. For example, if you cited an article using the syntax [1]_, include its reference as follows:

References
----------
.. [1] O. McNoleg, "The integration of GIS, remote sensing,
   expert systems and adaptive co-kriging for environmental habitat
   modelling of the Highland Haggis using object-oriented, fuzzy-logic
   and neural-network techniques," Computers & Geosciences, vol. 22,
   pp. 585-588, 1996.

Web pages should be referenced with regular inline links.

References are meant to augment the docstring, but should not be required to understand it. References are numbered, starting from one, in the order in which they are cited.

Note

In the future we may support bibtex-based references instead instead of explicitly writing bibliographies in docstrings.

Examples¶

‘Examples’ is an optional section for usage examples written in the doctest format. These examples do not replace unit tests, but are intended to be tested to ensure documentation and code are consistent. While optional, this section is useful for users and developers alike.

When multiple examples are provided, they should be separated by blank lines. Comments explaining the examples should have blank lines both above and below them:

>>> np.add(1, 2)
3

Comment explaining the second example

>>> np.add([1, 2], [3, 4])
array([4, 6])

For tests with a result that is random or platform-dependent, mark the output as such:

>>> np.random.rand(2)
array([ 0.35773152,  0.38568979])  #random

It is not necessary to use the doctest markup <BLANKLINE> to indicate empty lines in the output.

For more information on doctest, see:

The official doctest documentation.
doctest — Testing Through Documentation from Python Module of the Week.

Documenting Modules¶

Sections in Module Docstrings¶

Module docstrings contain the following sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
See Also (optional)

Note

Module docstrings aren’t featured heavily in the documentation we generate and publish with Sphinx. Avoid putting important end-user documentation in module docstrings. Instead, write introductory and overview documentation in the module’s user guide (the doc/ directories of Stack packages).

Module docstrings can still be useful for developer-oriented notes, though.

Placement of Module Docstrings¶

Module-level docstrings must be placed as close to the top of the Python file as possible: below any #!/usr/bin/env python and license statements, but above imports. See also: Standard code order SHOULD be followed.

Module docstrings should not be indented. For example:

#
# LSST Data Management System
# See COPYRIGHT file at the top of the source tree.
#
# [...]
#
# You should have received a copy of the LSST License Statement and
# the GNU General Public License along with this program. If not,
# see <http://www.lsstcorp.org/LegalNotices/>.
#
"""Summary of MyModule.

Extended discussion of my module.
"""

import lsst.afw.table as afw_table
# [...]

Documenting Classes¶

Class docstrings are placed directly after the class definition, and serve to document both the class as a whole and the arguments passed to the __init__ constructor.

Sections in Class Docstrings¶

Class docstrings contain the following sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Parameters (if applicable)
Other Parameters (if applicable)
Raises (if applicable)
See Also (optional)
Notes (optional)
References (optional)
Examples (optional)

Placement of Class Docstrings¶

Class docstrings must be placed directly below the declaration, and indented according to the code scope:

class MyClass(object):
    """Summary of MyClass.

    [...]
    """

    def __init__(self):
        pass

Examples of Class Docstrings¶

Here’s an example of a class:

class SkyCoordinate(object):
    """Equatorial coordinate on the sky as Right Ascension and Declination.

    Parameters
    ----------
    ra : `float`
       Right ascension (degrees).
    dec : `float`
       Declination (degrees).
    frame : {'icrs', 'fk5'}, optional
       Coordinate frame.

    Raises
    ------
    ValueError
        Raised when input angles are outside range.

    See also
    --------
    lsst.example.GalacticCoordinate

    Examples
    --------
    To define the coordinate of the M31 galaxy:

    >>> m31_coord = SkyCoordinate(10.683333333, 41.269166667)
    SkyCoordinate(10.683333333, 41.269166667, frame='icrs')
    """

    def __init__(self, ra, dec, frame='icrs'):
        pass

Documenting Methods and Functions¶

Sections in Method and Function Docstring Sections¶

Method and function docstrings contain the following sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Parameters (if applicable)
Returns or Yields (if applicable)
Other Parameters (if applicable)
Raises (if applicable)
See Also (optional)
Notes (optional)
References (optional)
Examples (optional)

Placement of Module and Function Docstrings¶

Class, method, and function docstrings must be placed directly below the declaration, and indented according to the code scope:

class MyClass(object):
    """Summary of MyClass.

    [...]
    """

    def __init__(self):
        pass

    def method(self):
        """Summary of method.

        Extended Discussion of my method.
        """
        pass


def my_function():
    """Summary of my_function.

    Extended discussion of my_function.
    """
    pass

Again, the class docstring takes the place of a docstring for the __init__ method. __init__ methods don’t have docstrings.

Examples of Method and Function Docstrings¶

Here’s an example function:

def check_unit(self, quantity):
    """Check that a `~astropy.units.Quantity` has equivalent units to
    this metric.

    Parameters
    ----------
    quantity : `astropy.units.Quantity`
        Quantity to be tested.

    Returns
    -------
    is_equivalent : `bool`
        `True` if the units are equivalent, meaning that the quantity
        can be presented in the units of this metric. `False` if not.

    See also
    --------
    astropy.units.is_equivalent

    Examples
    --------
    Check that a quantity in arcseconds is compatible with a metric defined in arcminutes:

    >>> import astropy.units as u
    >>> from lsst.verify import Metric
    >>> metric = Metric('example.test', 'Example', u.arcminute)
    >>> metric.check_units(1.*u.arcsecond)
    True

    But mags are not a compatible unit:

    >>> metric.check_units(21.*u.mag)
    False
    """
    if not quantity.unit.is_equivalent(self.unit):
        return False
    else:
        return True

Documenting Constants and Class Attributes¶

Sections in Constant and Class Attribute Docstrings¶

Constants in modules and attributes in classes are all documented similarly. At a minimum, they should have a summary line that includes the type. They can also have a more complete structure with these sections:

Short Summary
Deprecation Warning (if applicable)
Extended Summary (optional)
Notes (optional)
References (optional)
Examples (optional)

Placement of Constant and Class Attribute Docstrings¶

Docstrings for module-level variables and class attributes appear directly below their first declaration. For example:

MAX_ITER = 10
"""Maximum number of iterations (`int`).
"""


class MyClass(object):
    """[...]
    """

    x = None
    """Description of x attribute.
    """

Examples of Constant and Class Attribute Docstrings¶

Include the attribute or constant’s type in parentheses at the end of the summary line:

NAME = 'LSST'
"""Name of the project (`str`)."""

Multi-section docstrings keep the type information in the summary line. For example:

PA1_DESIGN = 5. * u.mmag
"""PA1 design specification (`astropy.units.Quantity`).

Notes
-----
The PA1 metric [1]_ is defined so that the rms of the unresolved source
magnitude distribution around the mean value (repeatability) will not
exceed PA1 millimag (median distribution for a large number of sources).

References
----------
.. [1] Z. Ivezic and the LSST Science Collaboration. 2011, LSST Science
   Requirements Document, LPM-17, URL https://ls.st/LPM-17
"""

In many classes, public attributes are set in the __init__ method. The best way to document these public attributes is by declaring the attribute at the class level and including a docstring with that declaration:

class Metric(object):
    """Verification metric.

    [...]
    """

    name = None
    """Name of the metric (`str`)."""

    unit = None
    """Units of the metric (`astropy.units.Unit`)."""

    def __init__(name, unit):
        self.name = name
        self.unit = unit

Private attributes (prefixed by underscores: self._myAttribute) do not need to be documented with docstrings.

Documenting Class Properties¶

Properties are documented like class attributes rather than methods. After all, properties are designed to appear to the user like simple attributes.

For example:

class Measurement(object):

    # ...

    @property
    def quantity(self):
        """The measurement quantity (`astropy.units.Quantity`).
        """
        # ...

    @quantity.setter
    def quantity(self, q):
        # ...

    @property
    def unit(self):
        """Units of the measurement (`astropy.units.Unit`, read-only).
        """
        # ...

Note:

Do not use the Returns section in the property’s docstring. Instead, include type information in the summary, as is done for class attributes.
Only document the property’s “getter” method, not the “setter” (if present).
If a property does not have a “setter” method, include the words read-only after the type information.

Acknowledgements¶

These docstring guidelines are derived/adapted from the NumPy and Astropy documentation.

Documenting Python APIs with Docstrings¶

Basic Format of Docstrings¶

Docstrings MUST be delimited by triple double quotes¶

Docstrings SHOULD begin with """ and terminate with """ on its own line¶

Docstrings of methods and functions SHOULD NOT be preceded or followed by a blank line¶

Docstrings of classes SHOULD be followed, but not preceded, by a blank line¶

Docstring content MUST be indented with the code’s scope¶

ReStructuredText in Docstrings¶

No space between headers and paragraphs¶

Sections are restricted to the Numpydoc section set¶

Simulate subsections with bold text¶

Line Lengths¶

Marking Up Parameter Names¶

Numpydoc Sections in Docstrings¶

Short Summary¶

Deprecation Warning¶

Extended Summary¶

Parameters¶

Describing Parameter Types¶

Concrete types¶

Choices¶

Sequence types¶

Dictionary types¶

Array types¶

Callable types¶

Optional Parameters¶

Shorthand¶

Returns¶

Yields¶

Other Parameters¶

Raises¶

See Also¶

Notes¶

References¶

Examples¶

Documenting Modules¶

Sections in Module Docstrings¶

Placement of Module Docstrings¶

Documenting Classes¶

Sections in Class Docstrings¶

Placement of Class Docstrings¶

Examples of Class Docstrings¶

Documenting Methods and Functions¶

Sections in Method and Function Docstring Sections¶

Placement of Module and Function Docstrings¶

Examples of Method and Function Docstrings¶

Documenting Constants and Class Attributes¶

Sections in Constant and Class Attribute Docstrings¶

Placement of Constant and Class Attribute Docstrings¶

Examples of Constant and Class Attribute Docstrings¶

Documenting Class Properties¶

Acknowledgements¶

Docstrings SHOULD begin with `"""` and terminate with `"""` on its own line¶