*******************************
Advanced changelog layout setup
*******************************

This page attempts do demonstrate one way of setting up a docs site for
use with |project| and includes a few opinionated integration solutions.
But of course, this layout may not fit everybody's needs.
When configuring your project, try to figure out what works for you —
you don't have to follow everything laid out here blindly.

Project structure
^^^^^^^^^^^^^^^^^

The author likes the following project directory layout:

::

    {{ project_root }}/
    │
    ├─ docs/
    │  ├─ changelog.d/
    │  │  │─ .gitignore
    │  │  │─ .towncrier-template.rst.j2
    │  │  │─ {{ issue_number }}.{{ changelog_fragment_type }}.rst
    │  │  │─ ...
    │  │  └─ README.rst
    │  ├─ changelog.rst
    │  ├─ conf.py
    │  ├─ index.rst
    │  ├─ requirements.in
    │  └─ requirements.txt
    ├─ src/
    │  └─ {{ python_importable_name }}/
    │     └─ ...
    ├─ .readthedocs.yml
    ├─ CHANGELOG.rst
    ├─ pyproject.toml
    ├─ README.rst
    ├─ tox.ini
    └─ ...

This is an ``src``-layout project with a Python package located at
``src/{{ python_importable_name }}/`` but we won't touch this topic.
There are several automation, configuration, documentation and metadata
files in the project root that will be described later on this page.
Finally, a :doc:`Sphinx <sphinx:contents>`-based site is located under
the ``docs/``.

The rest of this page will describe what to have in each of those files.


``docs/changelog.d/``
^^^^^^^^^^^^^^^^^^^^^

Let's start with the ``docs/changelog.d/``. This is a folder where the
end-users are supposed to add their changelog fragments for Towncrier to
consume.

``docs/changelog.d/.gitignore``
-------------------------------

First, let's make sure Git only tracks files that we want there by adding
a ``.gitignore`` file in this folder. First thing, it adds everything to
"ignore" but then allows ``.gitignore``, ``.gitignore``, ``README.rst``
and any RST documents matching Towncrier change note fragment naming
convention.

.. code-block::

    *

    !.gitignore
    !.towncrier-template.rst.j2
    !*.*.rst
    !README.rst


``docs/changelog.d/.towncrier-template.rst.j2``
-----------------------------------------------

Then, there's ``.towncrier-template.rst.j2``. It's a changelog template,
for Towncrier to use. It can be copied from
https://github.com/twisted/towncrier/tree/master/src/towncrier/templates.
This name is set in ``pyproject.toml`` in the project root.

``docs/changelog.d/{{ issue_number }}.{{ changelog_fragment_type }}.rst``
-------------------------------------------------------------------------

These are changelog fragments in RST format. They are absorbed by
Towncrier during the release and before that, these files will be used
in the preview generated by |project|.

``docs/changelog.d/README.rst``
-------------------------------

This ``README.rst`` file would normally contain — a guide for the
contributors on how to write change notes. For example, setuptools has a
useful write-up on :ref:`authoring changelog fragments
<setuptools:adding change notes with your prs>`. It is useful to have it
in this place so that it shows up on GitHub when the users navigate to
the folder with the fragments via the web UI.


``docs/``
^^^^^^^^^^^^^^^^^^^^^

``docs/changelog.rst``
----------------------

This is a Sphinx page that contains both the future version changelog
preview via ``.. towncrier-draft-entries::`` directive and the changelog
for all already released versions that is managed by Towncrier in a
separate RST document ``CHANGELOG.rst`` in the project root.

.. code-block:: rst

    *********
    Changelog
    *********

    Versions follow `Semantic Versioning`_ (``<major>.<minor>.<patch>``).
    Backward incompatible (breaking) changes will only be introduced in major
    versions with advance notice in the **Deprecations** section of releases.

    .. _Semantic Versioning: https://semver.org/

    .. towncrier-draft-entries:: |release| [UNRELEASED DRAFT] as on |today|

    .. include:: ../CHANGELOG.rst


``docs/conf.py``
----------------

The Sphinx configuration demonstrates how to keep the version
information known to Sphinx in sync with the Git tag based metadata.
Note the exclusion of ``docs/changelog.d/`` and the settings prefixed
with ``towncrier_draft_``.

.. code-block:: python

    """Configuration for the Sphinx documentation generator."""

    from functools import partial
    from pathlib import Path

    from setuptools_scm import get_version


    # -- Path setup --------------------------------------------------------------

    PROJECT_ROOT_DIR = Path(__file__).parents[1].resolve()
    get_scm_version = partial(get_version, root=PROJECT_ROOT_DIR)


    # -- Project information -----------------------------------------------------

    github_url = 'https://github.com'
    github_repo_org = 'your-org'
    github_repo_name = 'your-project'
    github_repo_slug = f'{github_repo_org}/{github_repo_name}'
    github_repo_url = f'{github_url}/{github_repo_slug}'
    github_sponsors_url = f'{github_url}/sponsors'

    project = github_repo_name
    author = f'{project} Contributors'
    copyright = f'2021, {author}'

    # The short X.Y version
    version = '.'.join(
        get_scm_version(
            local_scheme='no-local-version',
        ).split('.')[:3],
    )

    # The full version, including alpha/beta/rc tags
    release = get_scm_version()

    rst_epilog = f"""
    .. |project| replace:: {project}
    """


    # -- General configuration ---------------------------------------------------

    extensions = [
        # Built-in extensions:
        'sphinx.ext.extlinks',
        'sphinx.ext.intersphinx',

        # Third-party extensions:
        'sphinxcontrib.towncrier',  # provides `towncrier-draft-entries` directive
    ]

    exclude_patterns = [
        '_build', 'Thumbs.db', '.DS_Store',  # <- Defaults
        'changelog.d/**',  # Towncrier-managed change notes
    ]


    # -- Options for HTML output -------------------------------------------------

    html_theme = 'furo'


    # -- Extension configuration -------------------------------------------------

    # -- Options for intersphinx extension ---------------------------------------

    intersphinx_mapping = {
        'python': ('https://docs.python.org/3', None),
        'rtd': ('https://docs.rtfd.io/en/stable', None),
        'sphinx': ('https://www.sphinx-doc.org/en/master', None),
    }

    # -- Options for extlinks extension ------------------------------------------

    extlinks = {
        'issue': (f'{github_repo_url}/issues/%s', '#'),
        'pr': (f'{github_repo_url}/pull/%s', 'PR #'),
        'commit': (f'{github_repo_url}/commit/%s', ''),
        'gh': (f'{github_url}/%s', 'GitHub: '),
        'user': (f'{github_sponsors_url}/%s', '@'),
    }

    # -- Options for towncrier_draft extension -----------------------------------

    towncrier_draft_autoversion_mode = 'draft'  # or: 'sphinx-version', 'sphinx-release'
    towncrier_draft_include_empty = True
    towncrier_draft_working_directory = PROJECT_ROOT_DIR
    # Not yet supported: towncrier_draft_config_path = 'pyproject.toml'  # relative to cwd

    # -- Strict mode -------------------------------------------------------------

    default_role = 'any'

    nitpicky = True


``docs/index.rst``
------------------

The root document includes most of the README excluding one badge and
its title. It allows to flexibly control what information goes to the
PyPI and GitHub repo pages and what appears in the docs.

This document must contain a ``.. toctree::`` directive that has a
pointer to the ``changelog`` document in the list.

.. code-block:: rst

    Welcome to |project|'s documentation!
    =====================================

    .. include:: ../README.rst
       :end-before: DO-NOT-REMOVE-docs-badges-END

    .. include:: ../README.rst
       :start-after: DO-NOT-REMOVE-docs-intro-START

    .. toctree::
       :maxdepth: 2
       :caption: Contents:

       changelog


``docs/requirements.in``
------------------------

``requirements.in`` is a standard ``requirements.txt``-type file that
only lists dependencies that are directly used by the :doc:`Sphinx
static docs site generator <sphinx:contents>`. It may optionally contain
the minimum necessary versions of those.

.. code-block:: text

    furo
    setuptools-scm
    Sphinx
    sphinxcontrib-towncrier

``docs/requirements.txt``
-------------------------

But stating just the direct dependencies without strict version
restrictions is not enough for reproducible builds. Since it is
important to keep the docs build predictable over time, we use
`pip-tools`_ to generate a ``constraints.txt``-type pip-compatible
lockfile with pinned version constraints for the whole transitive
dependency tree. This file is ``requirements.txt`` and using it will
ensure that the virtualenv for building the docs always has the same
software with the same versions in it.

.. tip::

    As a bonus, having a ``.in`` + ``.txt`` pair of files is natively
    supported by GitHub Dependabot.

.. _pip-tools: https://github.com/jazzband/pip-tools


``.readthedocs.yml``
^^^^^^^^^^^^^^^^^^^^

To set up Read the Docs, add a ``.readthedocs.yml`` file in the project
root. The following configuration makes sure that the lockfile is used
to provision the build env. It also configures how Sphinx should behave
like failing the build on any warnings and having nice URLs.

.. code-block:: yaml

    ---
    version: 2

    formats: all

    sphinx:
      builder: dirhtml
      configuration: docs/conf.py
      fail_on_warning: true

    build:
      image: latest

    python:
      version: 3.8
      install:
      - requirements: docs/requirements.txt
    ...

.. note::

    When you have a Read the Docs YAML config in your repository, none
    of the :ref:`settings supported by it <rtd:config-file/v2:supported
    settings>` are derived from the web UI.

.. tip::

    Having :doc:`Read the Docs <rtd:index>` plugged into your project it
    is also possible to :doc:`enable pull-request builds
    <rtd:pull-requests>`.

``CHANGELOG.rst``
^^^^^^^^^^^^^^^^^

This file in the project root contains the compiled changelog with the
notes from the released project versions. It is managed by Towncrier and
should not be edited by you manually.

.. code-block:: rst

    .. towncrier release notes start


``pyproject.toml``
^^^^^^^^^^^^^^^^^^

``pyproject.toml`` in the root contains the setup for Towncrier itself
under the ``[tool.towncrier]`` section. It binds it all together
pointing at the directory for the change notes, the target changelog
document and the template to use when generating it. It also lists the
categories for the change fragments.

.. code-block:: toml

    [tool.towncrier]
      directory = "docs/changelog.d/"
      filename = "CHANGELOG.rst"
      issue_format = ":issue:`{issue}`"
      package_dir = "src"
      template = "docs/changelog.d/.towncrier-template.rst.j2"
      title_format = "v{version} ({project_date})"
      underlines = ["=", "^", "-", "~"]

      [[tool.towncrier.section]]
        path = ""

      [[tool.towncrier.type]]
        directory = "bugfix"
        name = "Bugfixes"
        showcontent = true

      [[tool.towncrier.type]]
        directory = "feature"
        name = "Features"
        showcontent = true

      [[tool.towncrier.type]]
        directory = "deprecation"
        name = "Deprecations (removal in next major release)"
        showcontent = true

      [[tool.towncrier.type]]
        directory = "breaking"
        name = "Backward incompatible changes"
        showcontent = true

      [[tool.towncrier.type]]
        directory = "doc"
        name = "Documentation"
        showcontent = true

      [[tool.towncrier.type]]
        directory = "misc"
        name = "Miscellaneous"
        showcontent = true


``README.rst``
^^^^^^^^^^^^^^

The README document is an important bit of your project. It shows up on
GitHub and is normally shown on PyPI. Besides that, it's possible to
include its fragments into the docs front page.

The example below shows how to use comment markers to include a part of
the badges into a Sphinx document also embedding some prose from the
README. Scroll up and see how it's being embedded into
``docs/index.rst``.

.. code-block:: rst

    .. image:: https://img.shields.io/pypi/v/your-project.svg?logo=Python&logoColor=white
       :target: https://pypi.org/project/your-project
       :alt: your-project @ PyPI

    .. image:: https://github.com/your-org/your-project/actions/workflows/tox-tests.yaml/badge.svg?event=push
       :target: https://github.com/your-org/your-project/actions/workflows/tox-tests.yaml
       :alt: GitHub Actions CI/CD build status

    .. DO-NOT-REMOVE-docs-badges-END

    .. image:: https://img.shields.io/readthedocs/your-project/latest.svg?logo=Read%20The%20Docs&logoColor=white
       :target: https://your-project.rtfd.io/en/latest/?badge=latest
       :alt: Documentation Status @ RTD

    your-project
    ============

    .. DO-NOT-REMOVE-docs-intro-START

    A project with Sphinx-managed documentation and description sourced
    from this README.


``tox.ini``
^^^^^^^^^^^

This is an example of setting up a tox-based Sphinx invocation

.. code-block:: ini

    [tox]
    envlist = python
    isolated_build = true
    minversion = 3.21.0


    [testenv:docs]
    basepython = python3
    deps =
      -r{toxinidir}{/}docs{/}requirements.txt
    description = Build The Docs
    commands =
      # Retrieve possibly missing commits:
      -git fetch --unshallow
      -git fetch --tags

      # Build the html docs with Sphinx:
      {envpython} -m sphinx \
        -j auto \
        -b html \
        {tty:--color} \
        -a \
        -n \
        -W --keep-going \
        -d "{temp_dir}{/}.doctrees" \
        {posargs:} \
        . \
        "{envdir}{/}docs_out"

      # Print out the output docs dir and a way to serve html:
      -{envpython} -c\
      'import pathlib;\
      docs_dir = pathlib.Path(r"{envdir}") / "docs_out";\
      index_file = docs_dir / "index.html";\
      print("\n" + "=" * 120 +\
      f"\n\nDocumentation available under:\n\n\
      \tfile://\{index_file\}\n\nTo serve docs, use\n\n\
      \t$ python3 -m http.server --directory \
      \N\{QUOTATION MARK\}\{docs_dir\}\N\{QUOTATION MARK\} 0\n\n" +\
      "=" * 120)'
    changedir = {toxinidir}{/}docs
    isolated_build = true
    passenv =
      SSH_AUTH_SOCK
    skip_install = true
    whitelist_externals =
      git

With this setup, run ``tox -e docs`` to build the site locally. Integrate
the same command in your CI.