The PyData Sphinx Theme¶
This is a simple, Bootstrap-based Sphinx theme from the PyData community. This site is a guide for using the theme, and a demo for how it looks with various elements.
Other sites that are using this theme:
Pandas: https://pandas.pydata.org/docs/
NumPy: https://numpy.org/devdocs/
JupyterHub and Binder: https://docs.mybinder.org/, http://z2jh.jupyter.org/en/latest/, https://repo2docker.readthedocs.io/en/latest/, https://jupyterhub-team-compass.readthedocs.io/en/latest/
Jupyter Book beta version uses an extension of this theme: https://beta.jupyterbook.org
Fairlearn: https://fairlearn.github.io/quickstart.html
User Guide¶
Installation¶
The theme is available on PyPI and conda-forge, and can thus be installed with:
$ pip install pydata-sphinx-theme
$ conda install pydata-sphinx-theme --channel conda-forge
Then, in the conf.py
of your sphinx docs, you update the html_theme
configuration option:
html_theme = "pydata_sphinx_theme"
If you want to track the development version of the theme, you can install it from the git repo:
$ pip install git+https://github.com/pydata/pydata-sphinx-theme.git@master
or in a conda environment yml file, you can add:
- pip:
- git+https://github.com/pydata/pydata-sphinx-theme.git@master
Configuration¶
There are a number of options for configuring your site’s look and feel.
All configuration options are passed with the html_theme_options
variable
in your conf.py
file. This is a dictionary with key: val
pairs that
you can configure in various ways. This page describes the options available to you.
Configure project logo¶
To add a logo that’s placed at the left of your nav bar, put a logo file under your doc path’s _static folder, and use the following configuration:
html_logo = "_static/logo.png"
The logo links to master_doc
(usually the first page of your documentation) by default.
If you’d like it to link to another page or use an external link instead, use the following configuration:
html_theme_options = {
"logo_link": "<other page or external link>"
}
Configure icon links¶
If you’d like icon links to show up to the right of your main navigation bar, use the following configuration:
html_theme_options = {
...
"icon_links": [
{
"name": "GitHub",
"url": "https://github.com/<your-org>/<your-repo>",
"icon": "fab fa-github-square",
},
{
"name": "GitLab",
"url": "https://gitlab.com/<your-org>/<your-repo>",
"icon": "fab fa-gitlab",
},
{
"name": "Twitter",
"url": "https://twitter.com/<your-handle>",
"icon": "fab fa-twitter-square",
},
],
...
}
The value of icon
can be any full
FontAwesome 5 Free icon.
In addition to the main icon class, e.g. fa-cat
, the “style” class must
also be provided e.g. fab for branding, or fas for solid.
Hint
To get custom colors like “Twitter blue”, use the following in your CSS,
e.g. custom.css
:
i.fa-twitter-square:before {
color: #55acee;
}
This has already been added for the brands that have shortcuts.
The below are shortcuts for commonly-used services, but may be removed in a future
release in favor of icon_links
:
html_theme_options = {
...
"github_url": "https://github.com/<your-org>/<your-repo>",
"gitlab_url": "https://gitlab.com/<your-org>/<your-repo>",
"twitter_url": "https://twitter.com/<your-handle>",
...
}
Additionally, the screen-reader accessible label for this menu can be configured:
html_theme_options = {
...
"icon_links_label": "Quick Links",
...
}
Configure the sidebar¶
pydata_sphinx_theme
provides two new sidebar items by default:
sidebar-nav-bs.html
- a bootstrap-friendly navigation sectionsidebar-search-bs.html
- a bootstrap-friendly search bar
By default, this theme’s sidebar has these two elements in it. If you’d like to override this behavior and control the sidebar on a per-page basis, use the Sphinx html-sidebars configuration value.
Hiding the previous and next buttons¶
By default, each page of your site will have “previous” and “next” buttons at the bottom. You can hide these buttons with the following configuration:
html_theme_options = {
"show_prev_next": False
}
Add an Edit this Page button¶
You can add a button to each page that will allow users to edit the page text
directly and submit a pull request to update the documentation. To include this
button in the right sidebar of each page, add the following configuration to
your conf.py
file:
html_context = {
"github_user": "<your-github-org>",
"github_repo": "<your-github-repo>",
"github_version": "<your-branch>",
"doc_path": "<path-from-root-to-your-docs>",
}
You should also enable the edit option in your ‘html_theme_options’:
html_theme_options = {
"use_edit_page_button": True,
}
Optionally, if you have a self-hosted Github Enterprise instance, you can configure a custom url. This option defaults to ‘https://github.com’, and you do not need to specify it if you wish to use the default.
html_context = {
"github_url": "<your-github-url>",
}
Configure the search bar position¶
To modify the position of the search bar, change the following variable in
your configuration file conf.py
. Possible options are ‘navbar’ and ‘sidebar’.
By default the search bar is positioned in the sidebar since this is more suitable for large navigation bars.
html_theme_options = {
"search_bar_position": "navbar"
}
Configure the search bar text¶
To modify the text that is in the search bar before people click on it, add the
following configuration to your conf.py
file:
html_theme_options = {
"search_bar_text": "Your text here..."
}
Google Analytics¶
If the google_analytics_id
config option is specified (like UA-XXXXXXX
),
Google Analytics’ javascript is included in the html pages.
html_theme_options = {
"google_analytics_id": "UA-XXXXXXX",
}
Changing pages with keyboard presses¶
By default, pydata-sphinx-theme
allows users to move to the previous/next
page using the left/right arrow keys on a keyboard. To disable this behavior,
use the following configuration:
html_theme_options = {
"navigation_with_keys": False
}
Show more levels of the in-page TOC by default¶
Normally only the 2nd-level headers of a page are show in the right table of contents, and deeper levels are only shown when they are part of an active section (when it is scrolled on screen).
You can show deeper levels by default by using the following configuration, indicating how many levels should be displayed:
html_theme_options = {
"show_toc_level": 2
}
All headings up to and including the level specified will now be shown regardless of what is displayed on the page.
Remove the sidebar from some pages¶
If you’d like the left sidebar to be removed from a page, you can use the
following configuration in conf.py
:
html_sidebars = {
"pagename": []
}
This works for glob-style patterns as well. For example:
html_sidebars = {
"folder/*": []
}
If you’d like to remove the left sidebar from all pages of your documentation, use this pattern:
html_sidebars = {
"**": []
}
For information about configuring the sidebar’s contents, see Configure the sidebar.
Customizing the theme¶
In addition to the configuration options detailed at Configuration, it is also possible to customize the HTML layout and CSS style of the theme.
Custom CSS Stylesheets¶
You may customize the theme’s CSS by creating a custom stylesheet that Sphinx uses to build your site. Any rules in this style-sheet will over-ride the default theme rules.
To add a custom stylesheet, follow these steps:
Create a CSS stylesheet in
_static/css/custom.css
, and add the CSS rules you wish.Attach the stylesheet to your Sphinx build. Add the following to
conf.py
html_static_path = ['_static'] html_css_files = [ 'css/custom.css', ]
When you build your documentation, this stylesheet should now be activated.
CSS Theme variables¶
This theme defines several CSS variables that can be used to quickly control behavior across your documentation.
These are based on top of the basic Bootstrap CSS variables extended with some theme specific variables. An overview of all variables and every default is defined in the pydata default CSS variables file.
In order to change a variable, follow these steps:
Add a custom CSS stylesheet. This is where we’ll configure the variables.
Underneath a
:root
section, add the variables you wish to update. For example, to update the base font size, you might add this tocustom.css
::root { --pst-font-size-base: 17px; }
For a complete list of the theme variables that you may override, see the theme variables defaults CSS file.
Important
Note that these are CSS variables and not SASS variables. The theme is defined with CSS variables, not SASS variables!
Replacing/Removing Fonts¶
The theme includes the FontAwesome 5 Free
icon font (the .fa, .far, .fas
styles, which are used for
icon links and admonitions).
This is the only vendored font, and otherwise the theme by default relies on
available system fonts for normal body text and headers.
Attention
Previously-included fonts like Lato have been removed, preferring the most common default system fonts of the reader’s computer. This provides both better performance, and better script/glyph coverage than custom fonts, and is recommended in most cases.
The default body and header fonts can be changed as follows:
Using Custom CSS Stylesheets, you can specify which fonts to use for body, header and monospace text. For example, the following can be added to a custom css file:
:root { --pst-font-family-base: Verdana, var(--pst-font-family-base-system); --pst-font-family-heading: Cambria, Georgia, Times, var(--pst-font-family-base-system); --pst-font-family-monospace: Courier, var(--pst-font-family-monospace-system); }
The
-system
variables are available to use as fallback to the default fonts.If the font you want to specify in the section above is not generally available by default, you will additionally need to ensure the font is loaded. For example, you could download and vendor the font in the
_static
directory of your Sphinx site, and then update the base template to load the font resources:Configure the template_path in your
conf.py
Create a custom
layout.html
Jinja2 template which overloads thefonts
block (example for loading the Lato font that is included in the_static/vendor
directory):{% extends "pydata_sphinx_theme/layout.html" %} {% block fonts %} <!-- add `style` or `link` tags with your CSS `@font-face` declarations here --> <!-- ... and optionally preload the `woff2` for snappier page loads --> <link rel="stylesheet" href="{{ pathto('_static/vendor/lato_latin-ext/1.44.1/index.css', 1) }}"> {% endblock %}
To reduce the Flash of Unstyled Content, you may wish to explore various options for preloading content, specifically the binary font files. This ensure the files will be loaded before waiting for the CSS to be parsed, but should be used with care.
Contributing¶
The documentation for this theme (what you are looking at now) also serves as a demo site for the theme.
Hint
The top-level Demo site section includes more pages with typical Sphinx content and structural elements.
Installing Python dependencies¶
To run the demo site, first install the Python dependencies, for example with pip
or conda
:
# with pip
python -m pip install -r docs/requirements.txt
# or with conda
conda install -c conda-forge --file docs/requirements.txt
Installing this theme¶
Next, install this theme itself, a python package. When developing, it is recommended to install in “development” or “editable” mode, allowing changes in the repo to be directly tested with this documentation suite.
To install the package, from the root of this repo, run:
python -m pip install --editable .
Building the demo site¶
For a traditional Sphinx build of the demo site, navigate to the docs/
directory,
and run:
make html
Sphinx will build the HTML version of the site in the docs/_build/html
directory.
Note
If you wish to customize the CSS or JS beyond what is available in the Configuration and Customizing the theme sections of the user guide, extra steps are required. The next section covers the full workflow, from changing the source files, to seeing the updated site.
Developing the theme CSS and JS¶
The CSS and JS for this theme are built for the browser from src/*
with
webpack. The main entrypoints are:
CSS:
src/scss/index.scss
JS:
src/js/index.js
provides add-on Bootstrap features, as well as some custom navigation behavior
webpack:
webpack.common.js
captures the techniques for transforming the JS and CSS source files in
src/
into the production assets inpydata_sphinx_theme/static/
These entrypoints, and all files they reference, are bundled into
pydata_sphinx_theme/static/{css,js}/index.<hash>.{css,js}
.
The <hash>
ensures the correct asset versions are served when viewers return to your
site after upgrading the theme, and is reproducibly derived from src/**/*
,
webpack.{common,prod}.js
, and the dependencies
and devDependencies
in package.json
/yarn.lock
.
Web fonts, and their supporting CSS, are copied into
pydata_sphinx_theme/static/vendor/<font name>/<font version>/
. Including
the <font version>
also ensures the correct assets are served when upgrading.
The links to these unique file names are captured as Jinja2 macros in
pydata_sphinx_theme/static/webpack-macros.html
.
Finally, all of these files are committed to the repo, in-place, along with the
rest of the code. This allows use of the theme directly from a git
checkout,
without any of the finicky web development dependencies, or even a nodejs
runtime.
Hint
Theme development was inspired by the ReadTheDocs Sphinx theme.
Steps to develop the theme¶
Install
yarn
Install theme dependencies
Run development server
Build production assets
Install the testing infrastructure
Attention
In order to commit changes to the theme, ensure you run
yarn build:production
so all built assets will be bundled, copied, or
generated into pydata_sphinx_theme/static/
.
Installing yarn
¶
Yarn is a package manager for JS and CSS dependencies.
Yarn itself can be installed with a number of
package managers, including
conda
:
conda install -c conda-forge yarn
Installing JS dependencies¶
To install theme-related dependencies
and devDependencies
from package.json
,
from the root of this repo, run:
yarn
After adding/updating dependencies with yarn add
, or manually changing package.json
and re-running yarn
, the yarn.lock
and package.json
files will likely change.
Important
If changed, commit package.json
and yarn.lock
together to ensure
reproducible builds.
Running the development server¶
To preview the frontend assets, from the root of this repo, run:
yarn build:dev
This launches a development server at http://localhost:1919. When working on the theme, saving changes to any of:
src/js/index.js
src/scss/index.scss
docs/**/*.rst
docs/**/*.py
…causes the development server to reload:
bundle/copy the CSS, JS, and vendored fonts
regenerate the Jinja2 macros
re-run Sphinx
Building the production assets¶
To build the new theme assets into the python package, from the root of this repo, run:
yarn build:production
Install the test infrastructure¶
This theme uses pytest
for its testing, with a lightweight fixture defined
in the test_build.py
script that makes it easy to run a Sphinx build using
this theme and inspect the results.
In addition, we use pytest-regressions
to ensure that the HTML generated by the theme is what we’d expect. This module
provides a file_regression
fixture that will check the contents of an object
against a reference file on disk. If the structure of the two differs, then the
test will fail. If we expect the structure to differ, then delete the file on
disk and run the test. A new file will be created, and subsequent tests will pass.
Install the testing dependencies with:
pip install pytest pytest-regressions
Then run the tests by calling pytest
from the repository root.
Changing fonts¶
Three “styles” of the FontAwesome 5 Free icon font are used for icon links and admonitions, and is the only vendored font. Further font choices are described in the Customizing the theme section of the user guide, and require some knowledge of HTML and CSS.
Attention
Previously-included fonts like Lato have been removed, preferring the most common default system fonts of the reader’s computer. This provides both better performance, and better script/glyph coverage than custom fonts, and is recommended in most cases.
The remaining vendored font selection is:
managed as a dependency in
package.json
allowing the version to be managed centrally
copied directly into the site statics, including licenses
allowing the chosen font to be replaced (or removed entirely) with minimal templating changes: practically, changing the icon font is difficult at this point.
partially preloaded
reducing flicker and re-layout artifacts of early icon renders
mostly managed in
webpack.common.js
allowing upgrades to be handled in a relatively sane, manageable way, to ensure the most recent icons
Upgrading a font¶
If only the version of the existing font must change, for example to enable new icons, run:
yarn add <font name>@<version>
yarn build:production
It may also be necessary to clear out old font versions from
pydata_sphinx_theme/static/vendor/
before committing.
Changing a font¶
If the above doesn’t work, for example if file names for an existing font change,
or a new font variant altogether is being added, hand-editing of webpack.common.js
is required. The steps are roughly:
install the new font, as above, with
yarn add
in
webpack.common.js
:add the new font to
vendorVersions
andvendorPaths
add new
link
tags to the appropriate macro inmacroTemplate
add the new font files (including the license) to
CopyPlugin
remove references to the font being replaced/removed, if applicable
restart the development server, if running
rebuild the production assets, as above, with
yarn build:production
potentially remove the font being replaced from
package.json
and re-runyarn
commit all of the changed files
Contributing changes¶
We follow a typical GitHub workflow of:
create a personal fork of this repo
create a branch
open a pull request
fix findings of various linters and checks
work through code review
For each pull request, the demo site is built and deployed to make it easier to review the changes in the PR. To access this, click on “Details” of the “build_docs artifact” job of Circle CI:

Ensuring correct commits¶
To ensure all source files have been correctly built, a pre-commit hook is available.
To set this up, first install the pre-commit
package:
# with pip
pip install pre-commit
# or with conda
conda install -c conda-forge pre-commit
Then, from the root of this repo, run:
pre-commit install
Now all of the checks will be run each time you commit changes.
Note that if needed, you can skip these checks with:
git commit --no-verify
Make a release¶
This theme uses GitHub tags and releases to automatically push new releases to PyPI. For information on this process, see the release checklist.
Demo site¶
This is a simple demonstration site to show off a few visual and structural elements of the theme. Click the sections on the left sidebar to see how various elements look on this theme.
Structural Elements¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec lorem neque, interdum in ipsum nec, finibus dictum velit. Ut eu efficitur arcu, id aliquam erat. In sit amet diam gravida, imperdiet tellus eu, gravida nisl. Praesent aliquet odio eget libero elementum, quis rhoncus tellus tincidunt. Suspendisse quis volutpat ipsum. Sed lobortis scelerisque tristique. Aenean condimentum risus tellus, quis accumsan ipsum laoreet ut. Integer porttitor maximus suscipit. Mauris in posuere sapien. Aliquam accumsan feugiat ligula, nec fringilla libero commodo sed. Proin et erat pharetra.
Etiam turpis ante, luctus sed velit tristique, finibus volutpat dui. Nam sagittis vel ante nec malesuada. Praesent dignissim mi nec ornare elementum. Nunc eu augue vel sem dignissim cursus sed et nulla. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Pellentesque dictum dui sem, non placerat tortor rhoncus in. Sed placerat nulla at rhoncus iaculis.
Document Section¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed condimentum nulla vel neque venenatis, nec placerat lorem placerat. Cras purus eros, gravida vitae tincidunt id, vehicula nec nulla. Fusce aliquet auctor cursus. Phasellus ex neque, vestibulum non est vitae, viverra fringilla tortor. Donec vestibulum convallis justo, a faucibus lorem vulputate vel. Aliquam cursus odio eu felis sodales aliquet. Aliquam erat volutpat. Maecenas eget dictum mauris. Suspendisse arcu eros, condimentum eget risus sed, luctus efficitur arcu. Cras ut dictum mi. Nulla congue interdum lorem, semper semper enim commodo nec.
Document Subsection¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam efficitur in eros et blandit. Nunc maximus, nisl at auctor vestibulum, justo ex sollicitudin ligula, id faucibus urna orci tristique nisl. Duis auctor rutrum orci, in ornare lacus condimentum quis. Quisque arcu velit, facilisis quis interdum ac, hendrerit auctor mauris. Curabitur urna nibh, porttitor at ante sit amet, vestibulum interdum dolor. Duis dictum elit orci, tincidunt imperdiet sem pellentesque et. In vehicula pellentesque varius. Phasellus a turpis sollicitudin, bibendum massa et, imperdiet neque. Integer quis sapien in magna rutrum bibendum. Integer cursus ex sed magna vehicula finibus. Proin tempus orci quis dolor tempus, nec condimentum odio vestibulum. Etiam efficitur sollicitudin libero, tincidunt volutpat ligula interdum sed.
Document Subsubsection¶
Donec non rutrum lorem. Aenean sagittis metus at pharetra fringilla. Nunc sapien dolor, cursus sed nisi at, pretium tristique lectus. Sed pellentesque leo lectus, et convallis ipsum euismod a. Integer at leo vitae felis pretium aliquam fringilla quis odio. Sed pharetra enim accumsan feugiat pretium. Maecenas at pharetra tortor. Morbi semper eget mi vel finibus. Cras rutrum nulla eros, id feugiat arcu pellentesque ut. Sed finibus tortor ac nisi ultrices viverra. Duis feugiat malesuada sapien, at commodo ante porttitor ac. Curabitur posuere mauris mi, vel ornare orci scelerisque sit amet. Suspendisse nec fringilla dui.
Pellentesque nec est in odio ultrices elementum. Vestibulum et hendrerit sapien, quis vulputate turpis. Suspendisse potenti. Curabitur tristique sit amet lectus non viverra. Phasellus rutrum dapibus turpis sed imperdiet. Mauris maximus viverra ante. Donec eu egestas mauris. Morbi vulputate tincidunt euismod. Integer vel porttitor neque. Donec at lacus suscipit, lacinia lectus vel, sagittis lectus.
Structural Elements 2¶
Etiam turpis ante, luctus sed velit tristique, finibus volutpat dui. Nam sagittis vel ante nec malesuada. Praesent dignissim mi nec ornare elementum. Nunc eu augue vel sem dignissim cursus sed et nulla. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Pellentesque dictum dui sem, non placerat tortor rhoncus in. Sed placerat nulla at rhoncus iaculis.
Document Section¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed condimentum nulla vel neque venenatis, nec placerat lorem placerat. Cras purus eros, gravida vitae tincidunt id, vehicula nec nulla. Fusce aliquet auctor cursus. Phasellus ex neque, vestibulum non est vitae, viverra fringilla tortor. Donec vestibulum convallis justo, a faucibus lorem vulputate vel. Aliquam cursus odio eu felis sodales aliquet. Aliquam erat volutpat. Maecenas eget dictum mauris. Suspendisse arcu eros, condimentum eget risus sed, luctus efficitur arcu. Cras ut dictum mi. Nulla congue interdum lorem, semper semper enim commodo nec.
Document Subsection¶

This is a caption for a figure. Text should wrap around the caption.¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam efficitur in eros et blandit. Nunc maximus, nisl at auctor vestibulum, justo ex sollicitudin ligula, id faucibus urna orci tristique nisl. Duis auctor rutrum orci, in ornare lacus condimentum quis. Quisque arcu velit, facilisis quis interdum ac, hendrerit auctor mauris. Curabitur urna nibh, porttitor at ante sit amet, vestibulum interdum dolor. Duis dictum elit orci, tincidunt imperdiet sem pellentesque et. In vehicula pellentesque varius. Phasellus a turpis sollicitudin, bibendum massa et, imperdiet neque. Integer quis sapien in magna rutrum bibendum. Integer cursus ex sed magna vehicula finibus. Proin tempus orci quis dolor tempus, nec condimentum odio vestibulum. Etiam efficitur sollicitudin libero, tincidunt volutpat ligula interdum sed. Praesent congue sagittis nisl et suscipit. Vivamus sagittis risus et egestas commodo.Cras venenatis arcu in pharetra interdum. Donec quis metus porttitor tellus cursus lobortis. Quisque et orci magna. Fusce rhoncus mi mi, at vehicula massa rhoncus quis. Mauris augue leo, pretium eget molestie vitae, efficitur nec nulla. In hac habitasse platea dictumst. Sed sit amet imperdiet purus.
Paragraph Level Markup¶
Inline Markup¶
Paragraphs contain text and may contain inline markup: emphasis, strong emphasis, inline literals
,
standalone hyperlinks (http://www.python.org), external hyperlinks (Python 4), internal cross-references (example),
external hyperlinks with embedded URIs (Python web site), footnote references
(manually numbered 1, anonymous auto-numbered 3, labeled auto-numbered 2, or symbolic *),
citation references (12), substitution references (), and inline hyperlink targets
(see Targets below for a reference back to here). Character-level inline markup is also possible
(although exceedingly ugly!) in re
Structured
Text. Problems are indicated by problematic
text (generated by processing errors; this one is intentional).
Also with sphinx.ext.autodoc
, which I use in the demo, I can link to test_py_module.test.Foo
.
It will link you right to my code documentation for it.
The default role for interpreted text is Title Reference. Here are some explicit interpreted text roles:
a PEP reference (PEP 287); an RFC reference (RFC 2822); a subscript; a superscript;
and explicit roles for standard inline markup
.
GUI labels are a useful way to indicate that Some action is to be taken by the user.
The GUI label should not run over line-height
so as not to interfere with text from adjacent lines.
Key-bindings indicate that the read is to press a button on the keyboard or mouse,
for example MMB and Shift-MMB. Another useful markup to indicate a user action
is to use menuselection
this can be used to show short and long menus in software.
For example, and menuselection
can be seen here that breaks is too long to fit on this line.
.
Let’s test wrapping and whitespace significance in inline literals:
This is an example of --inline-literal --text, --including some--
strangely--hyphenated-words. Adjust-the-width-of-your-browser-window
to see how the text is wrapped. -- ---- -------- Now note the
spacing between the words of this sentence (words
should be grouped in pairs).
If the --pep-references
option was supplied, there should be a live link to PEP 258 here.
Math¶
This is a test. Here is an equation: \(X_{0:5} = (X_0, X_1, X_2, X_3, X_4)\). Here is another:
You can add a link to equations like the one above (1) by using :eq:
.
Meta¶
Blocks¶
Literal Blocks¶
Literal blocks are indicated with a double-colon (“::”) at the end of
the preceding paragraph (over there -->
). They can be indented:
if literal_block:
text = 'is left as-is'
spaces_and_linebreaks = 'are preserved'
markup_processing = None
Or they can be quoted without indentation:
>> Great idea!
>
> Why didn't I think of that?
Line Blocks¶
Take it away, Eric the Orchestra Leader!
A one, two, a one two three fourHalf a bee, philosophically,must, ipso facto, half not be.But half the bee has got to be,vis a vis its entity. D’you see?But can a bee be said to beor not to be an entire bee,when half the bee is not a bee,due to some ancient injury?Singing…
Block Quotes¶
Block quotes consist of indented body elements:
My theory by A. Elk. Brackets Miss, brackets. This theory goes as follows and begins now. All brontosauruses are thin at one end, much much thicker in the middle and then thin again at the far end. That is my theory, it is mine, and belongs to me and I own it, and what it is too.
—Anne Elk (Miss)
Doctest Blocks¶
>>> print 'Python-specific usage examples; begun with ">>>"'
Python-specific usage examples; begun with ">>>"
>>> print '(cut and pasted from interactive Python sessions)'
(cut and pasted from interactive Python sessions)
Code Blocks¶
# parsed-literal test curl -O http://someurl/release-.tar-gz
{
"windows": [
{
"panes": [
{
"shell_command": [
"echo 'did you know'",
"echo 'you can inline'"
]
},
{
"shell_command": "echo 'single commands'"
},
"echo 'for panes'"
],
"window_name": "long form"
}
],
"session_name": "shorthands"
}
Emphasized lines with line numbers¶
1 2 3 4 5 | def some_function():
interesting = False
print 'This line is highlighted.'
print 'This one is not...'
print '...but this one is.'
|
Sidebar¶
The first hexagram is made up of six unbroken lines. These unbroken lines stand for the primal power, which is light-giving, active, strong, and of the spirit. The hexagram is consistently strong in character, and since it is without weakness, its essence is power or energy. Its image is heaven. Its energy is represented as unrestricted by any fixed conditions in space and is therefore conceived of as motion. Time is regarded as the basis of this motion. Thus the hexagram includes also the power of time and the power of persisting in time, that is, duration.
The power represented by the hexagram is to be interpreted in a dual sense in terms of its action on the universe and of its action on the world of men. In relation to the universe, the hexagram expresses the strong, creative action of the Deity. In relation to the human world, it denotes the creative action of the holy man or sage, of the ruler or leader of men, who through his power awakens and develops their higher nature.
Code with Sidebar¶
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # -*- coding: utf-8 -*-
"""Test Module for sphinx_rtd_theme."""
class Foo:
"""Docstring for class Foo.
This text tests for the formatting of docstrings generated from output
``sphinx.ext.autodoc``. Which contain reST, but sphinx nests it in the
``<dl>``, and ``<dt>`` tags. Also, ``<tt>`` is used for class, method names
and etc, but those will *always* have the ``.descname`` or
``.descclassname`` class.
Normal ``<tt>`` (like the <tt> I just wrote here) needs to be shown with
the same style as anything else with ````this type of markup````.
It's common for programmers to give a code example inside of their
docstring::
from test_py_module import Foo
myclass = Foo()
myclass.dothismethod('with this argument')
myclass.flush()
print(myclass)
Here is a link to :py:meth:`capitalize`.
Here is a link to :py:meth:`__init__`.
"""
#: Doc comment for class attribute Foo.bar.
#: It can have multiple lines.
bar = 1
flox = 1.5 #: Doc comment for Foo.flox. One line only.
|
References¶
Footnotes¶
- 1(1,2)
A footnote contains body elements, consistently indented by at least 3 spaces.
This is the footnote’s second paragraph.
- 2(1,2)
Footnotes may be numbered, either manually (as in 1) or automatically using a “#”-prefixed label. This footnote has a label so it can be referred to from multiple places, both as a footnote reference (2) and as a hyperlink reference (label).
- 3
This footnote is numbered automatically and anonymously using a label of “#” only.
- *
Footnotes may also use symbols, specified with a “*” label. Here’s a reference to the next footnote: †.
- †
This footnote shows the next symbol in the sequence.
Citations¶
- 12(1,2)
This citation has some
code blocks
in it, maybe some bold and italics too. Heck, lets put a link to a meta citation 13 too.- 13
This citation will have two backlinks.
Here’s a reference to the above, 12.
Here is another type of citation: citation
Targets¶
This paragraph is pointed to by the explicit “example” target. A reference can be found under Inline Markup, above. Inline hyperlink targets are also possible.
Section headers are implicit targets, referred to by name. See Targets, which is a subsection of Body Elements.
Explicit external targets are interpolated into references such as “Python 4”.
Targets may be indirect and anonymous. Thus this phrase may also refer to the Targets section.
Directives¶
Contents¶
These are just a sample of the many reStructuredText Directives. For others, please see: http://docutils.sourceforge.net/docs/ref/rst/directives.html.
Centered text¶
You can create a statement with centered text with .. centered::
This is centered text!
Images & Figures¶
Images¶
An image directive (also clickable – a hyperlink reference):

A larger image, with no options:

With a fixed with of 600px and centered:

And using the img-fluid class:

Figures¶

A figure is an image with a caption and/or a legend:¶
re |
Revised, revisited, based on ‘re’ module. |
Structured |
Structure-enhanced text, structuredtext. |
Text |
Well it is, isn’t it? |
This paragraph is also part of the legend.
A figure directive with center alignment

This caption should be centered.¶
Admonitions¶
Attention
Directives at large.
Caution
Don’t take any wooden nickels.
Danger
Mad scientist at work!
Error
Does not compute.
Hint
It’s bigger than a bread box.
Important
Wash behind your ears.
Clean up your room.
Including the closet.
The bathroom too.
Take the trash out of the bathroom.
Clean the sink.
Call your mother.
Back up your data.
Note
This is a note. Equations within a note: \(G_{\mu\nu} = 8 \pi G (T_{\mu\nu} + \rho_\Lambda g_{\mu\nu})\).
Note
Lists in admonitions.
Here’s a list item
And another list item
They should be in-line with the text around them!
Tip
15% if the service is good.
Example |
---|
Thing1 |
Thing2 |
Thing3 |
Warning
Strong prose may provoke extreme mental exertion. Reader discretion is strongly advised.
And, by the way…
You can make up your own admonition too.
If you add a class flag, it will be styled
For example, this admonition block uses the following code:
.. admonition:: If you add a class flag, it will be styled
:class: warning
Here are a list of classes you can try:
note
important
tip
attention
caution
warning
danger
error
hint
Deprecated¶
Deprecated since version 0.1.1: Something is deprecated, use something else instead.
Target Footnotes¶
Replacement Text¶
I recommend you try Python, the best language around 4.
Compound Paragraph¶
This paragraph contains a literal block:
Connecting... OK
Transmitting data... OK
Disconnecting... OK
and thus consists of a simple paragraph, a literal block, and another simple paragraph. Nonetheless it is semantically one paragraph.
This construct is called a compound paragraph and can be produced with the “compound” directive.
Download Links¶
HTML¶
The HTML below shouldn’t display, but it uses RequireJS to make sure that all works as expected. If the widgets don’t show up, RequireJS may be broken.
import plotly.io as pio
import plotly.express as px
import plotly.offline as py
pio.renderers.default = "notebook"
df = px.data.iris()
fig = px.scatter(df, x="sepal_width", y="sepal_length", color="species", size="sepal_length")
fig
And here we demonstrate xarray
to ensure that it shows up properly.
import xarray as xr
import numpy as np
data = xr.DataArray(
np.random.randn(2, 3),
dims=("x", "y"),
coords={"x": [10, 20]}, attrs={"foo": "bar"}
)
data
<xarray.DataArray (x: 2, y: 3)> array([[ 0.39212079, -0.25085855, -2.64879738], [-1.04200005, 1.08321885, -0.28360253]]) Coordinates: * x (x) int64 10 20 Dimensions without coordinates: y Attributes: foo: bar
- x: 2
- y: 3
- 0.3921 -0.2509 -2.649 -1.042 1.083 -0.2836
array([[ 0.39212079, -0.25085855, -2.64879738], [-1.04200005, 1.08321885, -0.28360253]])
- x(x)int6410 20
array([10, 20])
- foo :
- bar
API documentation and generated content¶
This page contains general code elements that are common for package documentation.
Autosummary table and API stub pages¶
|
Drop specified labels from rows or columns. |
|
Group DataFrame using a mapper or by a Series of columns. |
The ExtensionArray of the data backing this Series or Index. |
Inline module documentation¶
numpy.linalg
¶
numpy.linalg
¶
The NumPy linear algebra functions rely on BLAS and LAPACK to provide efficient low level implementations of standard linear algebra algorithms. Those libraries may be provided by NumPy itself using C versions of a subset of their reference implementations but, when possible, highly optimized libraries that take advantage of specialized processor functionality are preferred. Examples of such libraries are OpenBLAS, MKL (TM), and ATLAS. Because those libraries are multithreaded and processor dependent, environmental variables and external packages such as threadpoolctl may be needed to control the number of threads or specify the processor architecture.
OpenBLAS: https://www.openblas.net/
threadpoolctl: https://github.com/joblib/threadpoolctl
Please note that the most-used linear algebra functions in NumPy are present in
the main numpy
namespace rather than in numpy.linalg
. There are:
dot
, vdot
, inner
, outer
, matmul
, tensordot
, einsum
,
einsum_path
and kron
.
Functions present in numpy.linalg are listed below.
multi_dot matrix_power
cholesky qr svd
eig eigh eigvals eigvalsh
norm cond det matrix_rank slogdet
solve tensorsolve lstsq inv pinv tensorinv
LinAlgError
-
numpy.linalg.
eig
(a)¶ Compute the eigenvalues and right eigenvectors of a square array.
- Parameters
- a(…, M, M) array
Matrices for which the eigenvalues and right eigenvectors will be computed
- Returns
- w(…, M) array
The eigenvalues, each repeated according to its multiplicity. The eigenvalues are not necessarily ordered. The resulting array will be of complex type, unless the imaginary part is zero in which case it will be cast to a real type. When a is real the resulting eigenvalues will be real (0 imaginary part) or occur in conjugate pairs
- v(…, M, M) array
The normalized (unit “length”) eigenvectors, such that the column
v[:,i]
is the eigenvector corresponding to the eigenvaluew[i]
.
- Raises
- LinAlgError
If the eigenvalue computation does not converge.
See also
eigvals
eigenvalues of a non-symmetric array.
eigh
eigenvalues and eigenvectors of a real symmetric or complex Hermitian (conjugate symmetric) array.
eigvalsh
eigenvalues of a real symmetric or complex Hermitian (conjugate symmetric) array.
scipy.linalg.eig
Similar function in SciPy that also solves the generalized eigenvalue problem.
scipy.linalg.schur
Best choice for unitary and other non-Hermitian normal matrices.
Notes
New in version 1.8.0.
Broadcasting rules apply, see the numpy.linalg documentation for details.
This is implemented using the
_geev
LAPACK routines which compute the eigenvalues and eigenvectors of general square arrays.The number w is an eigenvalue of a if there exists a vector v such that
a @ v = w * v
. Thus, the arrays a, w, and v satisfy the equationsa @ v[:,i] = w[i] * v[:,i]
for \(i \in \{0,...,M-1\}\).The array v of eigenvectors may not be of maximum rank, that is, some of the columns may be linearly dependent, although round-off error may obscure that fact. If the eigenvalues are all different, then theoretically the eigenvectors are linearly independent and a can be diagonalized by a similarity transformation using v, i.e,
inv(v) @ a @ v
is diagonal.For non-Hermitian normal matrices the SciPy function scipy.linalg.schur is preferred because the matrix v is guaranteed to be unitary, which is not the case when using eig. The Schur factorization produces an upper triangular matrix rather than a diagonal matrix, but for normal matrices only the diagonal of the upper triangular matrix is needed, the rest is roundoff error.
Finally, it is emphasized that v consists of the right (as in right-hand side) eigenvectors of a. A vector y satisfying
y.T @ a = z * y.T
for some number z is called a left eigenvector of a, and, in general, the left and right eigenvectors of a matrix are not necessarily the (perhaps conjugate) transposes of each other.References
G. Strang, Linear Algebra and Its Applications, 2nd Ed., Orlando, FL, Academic Press, Inc., 1980, Various pp.
Examples
>>> from numpy import linalg as LA
(Almost) trivial example with real e-values and e-vectors.
>>> w, v = LA.eig(np.diag((1, 2, 3))) >>> w; v array([1., 2., 3.]) array([[1., 0., 0.], [0., 1., 0.], [0., 0., 1.]])
Real matrix possessing complex e-values and e-vectors; note that the e-values are complex conjugates of each other.
>>> w, v = LA.eig(np.array([[1, -1], [1, 1]])) >>> w; v array([1.+1.j, 1.-1.j]) array([[0.70710678+0.j , 0.70710678-0.j ], [0. -0.70710678j, 0. +0.70710678j]])
Complex-valued matrix with real e-values (but complex-valued e-vectors); note that
a.conj().T == a
, i.e., a is Hermitian.>>> a = np.array([[1, 1j], [-1j, 1]]) >>> w, v = LA.eig(a) >>> w; v array([2.+0.j, 0.+0.j]) array([[ 0. +0.70710678j, 0.70710678+0.j ], # may vary [ 0.70710678+0.j , -0. +0.70710678j]])
Be careful about round-off error!
>>> a = np.array([[1 + 1e-9, 0], [0, 1 - 1e-9]]) >>> # Theor. e-values are 1 +/- 1e-9 >>> w, v = LA.eig(a) >>> w; v array([1., 1.]) array([[1., 0.], [0., 1.]])
-
numpy.linalg.
matrix_power
(a, n)¶ Raise a square matrix to the (integer) power n.
For positive integers n, the power is computed by repeated matrix squarings and matrix multiplications. If
n == 0
, the identity matrix of the same shape as M is returned. Ifn < 0
, the inverse is computed and then raised to theabs(n)
.Note
Stacks of object matrices are not currently supported.
- Parameters
- a(…, M, M) array_like
Matrix to be “powered”.
- nint
The exponent can be any integer or long integer, positive, negative, or zero.
- Returns
- a**n(…, M, M) ndarray or matrix object
The return value is the same shape and type as M; if the exponent is positive or zero then the type of the elements is the same as those of M. If the exponent is negative the elements are floating-point.
- Raises
- LinAlgError
For matrices that are not square or that (for negative powers) cannot be inverted numerically.
Examples
>>> from numpy.linalg import matrix_power >>> i = np.array([[0, 1], [-1, 0]]) # matrix equiv. of the imaginary unit >>> matrix_power(i, 3) # should = -i array([[ 0, -1], [ 1, 0]]) >>> matrix_power(i, 0) array([[1, 0], [0, 1]]) >>> matrix_power(i, -3) # should = 1/(-i) = i, but w/ f.p. elements array([[ 0., 1.], [-1., 0.]])
Somewhat more sophisticated example
>>> q = np.zeros((4, 4)) >>> q[0:2, 0:2] = -i >>> q[2:4, 2:4] = i >>> q # one of the three quaternion units not equal to 1 array([[ 0., -1., 0., 0.], [ 1., 0., 0., 0.], [ 0., 0., 0., 1.], [ 0., 0., -1., 0.]]) >>> matrix_power(q, 2) # = -np.eye(4) array([[-1., 0., 0., 0.], [ 0., -1., 0., 0.], [ 0., 0., -1., 0.], [ 0., 0., 0., -1.]])
-
numpy.linalg.
norm
(x, ord=None, axis=None, keepdims=False)¶ Matrix or vector norm.
This function is able to return one of eight different matrix norms, or one of an infinite number of vector norms (described below), depending on the value of the
ord
parameter.- Parameters
- xarray_like
Input array. If axis is None, x must be 1-D or 2-D, unless ord is None. If both axis and ord are None, the 2-norm of
x.ravel
will be returned.- ord{non-zero int, inf, -inf, ‘fro’, ‘nuc’}, optional
Order of the norm (see table under
Notes
). inf means numpy’s inf object. The default is None.- axis{None, int, 2-tuple of ints}, optional.
If axis is an integer, it specifies the axis of x along which to compute the vector norms. If axis is a 2-tuple, it specifies the axes that hold 2-D matrices, and the matrix norms of these matrices are computed. If axis is None then either a vector norm (when x is 1-D) or a matrix norm (when x is 2-D) is returned. The default is None.
New in version 1.8.0.
- keepdimsbool, optional
If this is set to True, the axes which are normed over are left in the result as dimensions with size one. With this option the result will broadcast correctly against the original x.
New in version 1.10.0.
- Returns
- nfloat or ndarray
Norm of the matrix or vector(s).
See also
scipy.linalg.norm
Similar function in SciPy.
Notes
For values of
ord < 1
, the result is, strictly speaking, not a mathematical ‘norm’, but it may still be useful for various numerical purposes.The following norms can be calculated:
ord
norm for matrices
norm for vectors
None
Frobenius norm
2-norm
‘fro’
Frobenius norm
–
‘nuc’
nuclear norm
–
inf
max(sum(abs(x), axis=1))
max(abs(x))
-inf
min(sum(abs(x), axis=1))
min(abs(x))
0
–
sum(x != 0)
1
max(sum(abs(x), axis=0))
as below
-1
min(sum(abs(x), axis=0))
as below
2
2-norm (largest sing. value)
as below
-2
smallest singular value
as below
other
–
sum(abs(x)**ord)**(1./ord)
The Frobenius norm is given by [1]:
\(||A||_F = [\sum_{i,j} abs(a_{i,j})^2]^{1/2}\)
The nuclear norm is the sum of the singular values.
Both the Frobenius and nuclear norm orders are only defined for matrices and raise a ValueError when
x.ndim != 2
.References
- 1
G. H. Golub and C. F. Van Loan, Matrix Computations, Baltimore, MD, Johns Hopkins University Press, 1985, pg. 15
Examples
>>> from numpy import linalg as LA >>> a = np.arange(9) - 4 >>> a array([-4, -3, -2, ..., 2, 3, 4]) >>> b = a.reshape((3, 3)) >>> b array([[-4, -3, -2], [-1, 0, 1], [ 2, 3, 4]])
>>> LA.norm(a) 7.745966692414834 >>> LA.norm(b) 7.745966692414834 >>> LA.norm(b, 'fro') 7.745966692414834 >>> LA.norm(a, np.inf) 4.0 >>> LA.norm(b, np.inf) 9.0 >>> LA.norm(a, -np.inf) 0.0 >>> LA.norm(b, -np.inf) 2.0
>>> LA.norm(a, 1) 20.0 >>> LA.norm(b, 1) 7.0 >>> LA.norm(a, -1) -4.6566128774142013e-010 >>> LA.norm(b, -1) 6.0 >>> LA.norm(a, 2) 7.745966692414834 >>> LA.norm(b, 2) 7.3484692283495345
>>> LA.norm(a, -2) 0.0 >>> LA.norm(b, -2) 1.8570331885190563e-016 # may vary >>> LA.norm(a, 3) 5.8480354764257312 # may vary >>> LA.norm(a, -3) 0.0
Using the axis argument to compute vector norms:
>>> c = np.array([[ 1, 2, 3], ... [-1, 1, 4]]) >>> LA.norm(c, axis=0) array([ 1.41421356, 2.23606798, 5. ]) >>> LA.norm(c, axis=1) array([ 3.74165739, 4.24264069]) >>> LA.norm(c, ord=1, axis=1) array([ 6., 6.])
Using the axis argument to compute matrix norms:
>>> m = np.arange(8).reshape(2,2,2) >>> LA.norm(m, axis=(1,2)) array([ 3.74165739, 11.22497216]) >>> LA.norm(m[0, :, :]), LA.norm(m[1, :, :]) (3.7416573867739413, 11.224972160321824)
-
numpy.linalg.
tensorinv
(a, ind=2)¶ Compute the ‘inverse’ of an N-dimensional array.
The result is an inverse for a relative to the tensordot operation
tensordot(a, b, ind)
, i. e., up to floating-point accuracy,tensordot(tensorinv(a), a, ind)
is the “identity” tensor for the tensordot operation.- Parameters
- aarray_like
Tensor to ‘invert’. Its shape must be ‘square’, i. e.,
prod(a.shape[:ind]) == prod(a.shape[ind:])
.- indint, optional
Number of first indices that are involved in the inverse sum. Must be a positive integer, default is 2.
- Returns
- bndarray
a’s tensordot inverse, shape
a.shape[ind:] + a.shape[:ind]
.
- Raises
- LinAlgError
If a is singular or not ‘square’ (in the above sense).
See also
numpy.tensordot
,tensorsolve
Examples
>>> a = np.eye(4*6) >>> a.shape = (4, 6, 8, 3) >>> ainv = np.linalg.tensorinv(a, ind=2) >>> ainv.shape (8, 3, 4, 6) >>> b = np.random.randn(4, 6) >>> np.allclose(np.tensordot(ainv, b), np.linalg.tensorsolve(a, b)) True
>>> a = np.eye(4*6) >>> a.shape = (24, 8, 3) >>> ainv = np.linalg.tensorinv(a, ind=1) >>> ainv.shape (8, 3, 24) >>> b = np.random.randn(24) >>> np.allclose(np.tensordot(ainv, b, 1), np.linalg.tensorsolve(a, b)) True
C++ API¶
-
type
MyType
¶ Some type
-
template<typename
T
, std::size_tN
>
classstd
::
array
¶ Some cpp class
-
float
Sphinx
::
version
¶ The description of Sphinx::version.
-
int
version
¶ The description of version.
-
typedef std::vector<int>
List
¶ The description of List type.
JavaScript API¶
Link to
ModTopLevel()
-
class
module_a.submodule.
ModTopLevel
()¶ Link to
mod_child_1()
Link to
ModTopLevel.mod_child_1()
-
ModTopLevel.
mod_child_1
()¶ Link to
mod_child_2()
-
ModTopLevel.
mod_child_2
()¶
Link to
ModTopLevel()
-
class
module_b.submodule.
ModNested
()¶ -
ModNested.
nested_child_1
()¶ Link to
nested_child_2()
-
ModNested.
nested_child_2
()¶ Link to
nested_child_1()
-
Optional parameter args¶
At this point optional parameters cannot be generated from code. However, some projects will manually do it, like so:
This example comes from django-payments module docs.
-
class
payments.dotpay.
DotpayProvider
(seller_id, pin[, channel=0[, lock=False], lang='pl'])¶ This backend implements payments using a popular Polish gateway, Dotpay.pl.
Due to API limitations there is no support for transferring purchased items.
- Parameters
seller_id – Seller ID assigned by Dotpay
pin – PIN assigned by Dotpay
channel – Default payment channel (consult reference guide)
lang – UI language
lock – Whether to disable channels other than the default selected above
Data¶
-
numpy.linalg.
Data_item_1
¶ -
numpy.linalg.
Data_item_2
¶ -
numpy.linalg.
Data_item_3
¶ Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce congue elit eu hendrerit mattis.
Some data link Data_item_1
.
Lists & Tables¶
Lists¶
Enumerated Lists¶
Arabic numerals.
lower alpha)
(lower roman)
upper alpha.
upper roman)
Lists that don’t start at 1:
Three
Four
C
D
iii
iv
List items may also be auto-enumerated.
Definition Lists¶
- Term
Definition
- Termclassifier
Definition paragraph 1.
Definition paragraph 2.
- Term
Definition
Option Lists¶
For listing command-line options:
- -a
command-line option “a”
- -b file
options can have arguments and long descriptions
- --long
options can be long also
- --input=file
long options can also have arguments
- --very-long-option
The description can also start on the next line.
The description may contain multiple body elements, regardless of where it starts.
- -x, -y, -z
Multiple options are an “option group”.
- -v, --verbose
Commonly-seen: short & long options.
- -1 file, --one=file, --two file
Multiple options with arguments.
- /V
DOS/VMS-style options too
There must be at least two spaces between the option and the description.
Field list¶
- Author
David Goodger
- Address
123 Example Street Example, EX Canada A1B 2C3
- Contact
- Authors
Me; Myself; I
- organization
humankind
- date
$Date: 2012-01-03 19:23:53 +0000 (Tue, 03 Jan 2012) $
- status
This is a “work in progress”
- revision
$Revision: 7302 $
- version
1
- copyright
This document has been placed in the public domain. You may do with it as you wish. You may copy, modify, redistribute, reattribute, sell, buy, rent, lease, destroy, or improve it, quote it at length, excerpt, incorporate, collate, fold, staple, or mutilate it, or do anything else to it that your or anyone else’s heart desires.
- field name
This is a generic bibliographic field.
- field name 2
Generic bibliographic fields may contain multiple body elements.
Like this.
- Dedication
For Docutils users & co-developers.
- abstract
This document is a demonstration of the reStructuredText markup language, containing examples of all basic reStructuredText constructs and many advanced constructs.
Bullet Lists¶
A bullet list
Nested bullet list.
Nested item 2.
Item 2.
Paragraph 2 of item 2.
Nested bullet list.
Nested item 2.
Third level.
Item 2.
Nested item 3.
inline literall
inline literall
inline literall
Second list level¶
here is a list in a second-level section.
-
here is an inner bullet
oh
one more
with an inline literally
. yahooheh heh. child. try to beat this embed:
1 2 3 4 5 6 7 8 9 10
# -*- coding: utf-8 -*- """Test Module for sphinx_rtd_theme.""" class Foo: """Docstring for class Foo. This text tests for the formatting of docstrings generated from output ``sphinx.ext.autodoc``. Which contain reST, but sphinx nests it in the
and another. yahoo
hi
and hehe
Hlists¶
|
|
Hlist with images
|
|
Numbered List¶
One,
Two.
Three with long text. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed feugiat sagittis neque quis eleifend. Duis rutrum lectus sit amet mattis suscipit.
Using bullets and letters. (A)
Using bullets and letters. (B)
Using bullets and letters. (C)
Tables¶
Grid Tables¶
Here’s a grid table followed by a simple table:
Header row, column 1 (header rows optional) |
Header 2 |
Header 3 |
Header 4 |
---|---|---|---|
body row 1, column 1 |
column 2 |
column 3 |
column 4 |
body row 2 |
Cells may span columns. |
||
body row 3 |
Cells may span rows. |
|
|
body row 4 |
|||
body row 5 |
Cells may also be
empty: |
Inputs |
Output |
|
---|---|---|
A |
B |
A or B |
False |
False |
False |
True |
False |
True |
False |
True |
True |
True |
True |
True |
Giant Tables¶
Header 1 |
Header 2 |
Header 3 |
Header 1 |
Header 2 |
Header 3 |
Header 1 |
Header 2 |
Header 3 |
Header 1 |
Header 2 |
Header 3 |
---|---|---|---|---|---|---|---|---|---|---|---|
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
body row 1 |
column 2 |
column 3 |
List Tables¶
List table |
Header 1 |
Header 2 |
Header 3 long. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam sit amet mauris arcu. |
---|---|---|---|
Stub Row 1 |
Row 1 |
Column 2 |
Column 3 long. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam sit amet mauris arcu. |
Stub Row 2 |
Row 2 |
Column 2 |
Column 3 long. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam sit amet mauris arcu. |
Stub Row 3 |
Row 3 |
Column 2 |
Column 3 long. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam sit amet mauris arcu. |
![]() This is a short caption for a figure.¶ |
![]() This is a long caption for a figure. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec porttitor dolor in odio posuere, vitae ornare libero mattis. In lobortis justo vestibulum nibh aliquet, non.¶ |
Markdown pages¶
Sphinx can also generate pages from Markdown, using the Recommonmark package.
This page demonstrates that this is possible! Note that in order for your
in-page table of contents to work properly, in-page Markdown headings (except for
the title) should have at least two #
symbols.
A markdown section¶
This section was created with ## A markdown section
- note that this is ATX-style
heading, which rST usually does not support. However, with recommonmark it is possible!
First top-level¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris sed sapien et sem scelerisque bibendum sed eget velit. Duis ut ultricies enim, vitae ultricies neque. Proin in purus eget orci posuere consequat sit amet ac augue. Aliquam dolor turpis, ultricies ac dui vel, maximus eleifend enim. Suspendisse eleifend orci non lectus mattis, pharetra blandit eros feugiat. Maecenas sed orci eget orci pharetra posuere id vel enim. Aenean ac lobortis lorem. Sed malesuada urna non consectetur condimentum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
First second-level¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris sed sapien et sem scelerisque bibendum sed eget velit. Duis ut ultricies enim, vitae ultricies neque. Proin in purus eget orci posuere consequat sit amet ac augue. Aliquam dolor turpis, ultricies ac dui vel, maximus eleifend enim. Suspendisse eleifend orci non lectus mattis, pharetra blandit eros feugiat. Maecenas sed orci eget orci pharetra posuere id vel enim. Aenean ac lobortis lorem. Sed malesuada urna non consectetur condimentum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
Second top-level¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris sed sapien et sem scelerisque bibendum sed eget velit. Duis ut ultricies enim, vitae ultricies neque. Proin in purus eget orci posuere consequat sit amet ac augue. Aliquam dolor turpis, ultricies ac dui vel, maximus eleifend enim. Suspendisse eleifend orci non lectus mattis, pharetra blandit eros feugiat. Maecenas sed orci eget orci pharetra posuere id vel enim. Aenean ac lobortis lorem. Sed malesuada urna non consectetur condimentum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
Second second-level¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris sed sapien et sem scelerisque bibendum sed eget velit. Duis ut ultricies enim, vitae ultricies neque. Proin in purus eget orci posuere consequat sit amet ac augue. Aliquam dolor turpis, ultricies ac dui vel, maximus eleifend enim. Suspendisse eleifend orci non lectus mattis, pharetra blandit eros feugiat. Maecenas sed orci eget orci pharetra posuere id vel enim. Aenean ac lobortis lorem. Sed malesuada urna non consectetur condimentum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
Third top-level¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris sed sapien et sem scelerisque bibendum sed eget velit. Duis ut ultricies enim, vitae ultricies neque. Proin in purus eget orci posuere consequat sit amet ac augue. Aliquam dolor turpis, ultricies ac dui vel, maximus eleifend enim. Suspendisse eleifend orci non lectus mattis, pharetra blandit eros feugiat. Maecenas sed orci eget orci pharetra posuere id vel enim. Aenean ac lobortis lorem. Sed malesuada urna non consectetur condimentum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
Third second-level¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Mauris sed sapien et sem scelerisque bibendum sed eget velit. Duis ut ultricies enim, vitae ultricies neque. Proin in purus eget orci posuere consequat sit amet ac augue. Aliquam dolor turpis, ultricies ac dui vel, maximus eleifend enim. Suspendisse eleifend orci non lectus mattis, pharetra blandit eros feugiat. Maecenas sed orci eget orci pharetra posuere id vel enim. Aenean ac lobortis lorem. Sed malesuada urna non consectetur condimentum. Orci varius natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus.
Pandas example - Indexing and selecting data¶
Note
This is an example page with excerpts from the pandas docs, for some “real world” content. But including it here apart from the rest of the pandas docs will mean that some of the links won’t work, and not all code examples are shown with their complete outputs.
The axis labeling information in pandas objects serves many purposes:
Identifies data (i.e. provides metadata) using known indicators, important for analysis, visualization, and interactive console display.
Enables automatic and explicit data alignment.
Allows intuitive getting and setting of subsets of the data set.
In this section, we will focus on the final point: namely, how to slice, dice, and generally get and set subsets of pandas objects. The primary focus will be on Series and DataFrame as they have received more development attention in this area.
Note
The Python and NumPy indexing operators []
and attribute operator .
provide quick and easy access to pandas data structures across a wide range
of use cases. This makes interactive work intuitive, as there’s little new
to learn if you already know how to deal with Python dictionaries and NumPy
arrays. However, since the type of the data to be accessed isn’t known in
advance, directly using standard operators has some optimization limits. For
production code, we recommended that you take advantage of the optimized
pandas data access methods exposed in this chapter.
Warning
Whether a copy or a reference is returned for a setting operation, may
depend on the context. This is sometimes called chained assignment
and
should be avoided. See Returning a View versus Copy.
Different choices for indexing¶
Object selection has had a number of user-requested additions in order to support more explicit location based indexing. Pandas now supports three types of multi-axis indexing.
.loc
is primarily label based, but may also be used with a boolean array..loc
will raiseKeyError
when the items are not found. Allowed inputs are:A single label, e.g.
5
or'a'
(Note that5
is interpreted as a label of the index. This use is not an integer position along the index.).A list or array of labels
['a', 'b', 'c']
.A slice object with labels
'a':'f'
(Note that contrary to usual python slices, both the start and the stop are included, when present in the index!)A boolean array
A
callable
function with one argument (the calling Series or DataFrame) and that returns valid output for indexing (one of the above).
See more at Selection by Label.
.iloc
is primarily integer position based (from0
tolength-1
of the axis), but may also be used with a boolean array..iloc
will raiseIndexError
if a requested indexer is out-of-bounds, except slice indexers which allow out-of-bounds indexing. (this conforms with Python/NumPy slice semantics). Allowed inputs are:An integer e.g.
5
.A list or array of integers
[4, 3, 0]
.A slice object with ints
1:7
.A boolean array.
A
callable
function with one argument (the calling Series or DataFrame) and that returns valid output for indexing (one of the above).
.loc
,.iloc
, and also[]
indexing can accept acallable
as indexer. See more at Selection By Callable.
Getting values from an object with multi-axes selection uses the following
notation (using .loc
as an example, but the following applies to .iloc
as
well). Any of the axes accessors may be the null slice :
. Axes left out of
the specification are assumed to be :
, e.g. p.loc['a']
is equivalent to
p.loc['a', :, :]
.
Object Type |
Indexers |
---|---|
Series |
|
DataFrame |
|
Basics¶
As mentioned when introducing the data structures in the last section,
the primary function of indexing with []
(a.k.a. __getitem__
for those familiar with implementing class behavior in Python) is selecting out
lower-dimensional slices. The following table shows return type values when
indexing pandas objects with []
:
Object Type |
Selection |
Return Value Type |
---|---|---|
Series |
|
scalar value |
DataFrame |
|
|
Here we construct a simple time series data set to use for illustrating the indexing functionality:
>>> dates = pd.date_range('1/1/2000', periods=8)
>>> df = pd.DataFrame(np.random.randn(8, 4),
... index=dates, columns=['A', 'B', 'C', 'D'])
...
>>> df
A B C D
2000-01-01 0.469112 -0.282863 -1.509059 -1.135632
2000-01-02 1.212112 -0.173215 0.119209 -1.044236
2000-01-03 -0.861849 -2.104569 -0.494929 1.071804
2000-01-04 0.721555 -0.706771 -1.039575 0.271860
2000-01-05 -0.424972 0.567020 0.276232 -1.087401
2000-01-06 -0.673690 0.113648 -1.478427 0.524988
2000-01-07 0.404705 0.577046 -1.715002 -1.039268
2000-01-08 -0.370647 -1.157892 -1.344312 0.844885
Note
None of the indexing functionality is time series specific unless specifically stated.
Thus, as per above, we have the most basic indexing using []
:
>>> s = df['A']
>>> s[dates[5]]
-0.6736897080883706
You can pass a list of columns to []
to select columns in that order.
If a column is not contained in the DataFrame, an exception will be
raised. Multiple columns can also be set in this manner:
>>> df
A B C D
2000-01-01 0.469112 -0.282863 -1.509059 -1.135632
2000-01-02 1.212112 -0.173215 0.119209 -1.044236
2000-01-03 -0.861849 -2.104569 -0.494929 1.071804
2000-01-04 0.721555 -0.706771 -1.039575 0.271860
2000-01-05 -0.424972 0.567020 0.276232 -1.087401
2000-01-06 -0.673690 0.113648 -1.478427 0.524988
2000-01-07 0.404705 0.577046 -1.715002 -1.039268
2000-01-08 -0.370647 -1.157892 -1.344312 0.844885
>>> df[['B', 'A']] = df[['A', 'B']]
>>> df
A B C D
2000-01-01 -0.282863 0.469112 -1.509059 -1.135632
2000-01-02 -0.173215 1.212112 0.119209 -1.044236
2000-01-03 -2.104569 -0.861849 -0.494929 1.071804
2000-01-04 -0.706771 0.721555 -1.039575 0.271860
2000-01-05 0.567020 -0.424972 0.276232 -1.087401
2000-01-06 0.113648 -0.673690 -1.478427 0.524988
2000-01-07 0.577046 0.404705 -1.715002 -1.039268
2000-01-08 -1.157892 -0.370647 -1.344312 0.844885
You may find this useful for applying a transform (in-place) to a subset of the columns.
Warning
pandas aligns all AXES when setting Series
and DataFrame
from .loc
, and .iloc
.
This will not modify df
because the column alignment is before value assignment.
>>> df[['A', 'B']]
A B
2000-01-01 -0.282863 0.469112
2000-01-02 -0.173215 1.212112
2000-01-03 -2.104569 -0.861849
2000-01-04 -0.706771 0.721555
2000-01-05 0.567020 -0.424972
2000-01-06 0.113648 -0.673690
2000-01-07 0.577046 0.404705
2000-01-08 -1.157892 -0.370647
>>> df.loc[:, ['B', 'A']] = df[['A', 'B']]
>>> df[['A', 'B']]
A B
2000-01-01 -0.282863 0.469112
2000-01-02 -0.173215 1.212112
2000-01-03 -2.104569 -0.861849
2000-01-04 -0.706771 0.721555
2000-01-05 0.567020 -0.424972
2000-01-06 0.113648 -0.673690
2000-01-07 0.577046 0.404705
2000-01-08 -1.157892 -0.370647
The correct way to swap column values is by using raw values:
>>> df.loc[:, ['B', 'A']] = df[['A', 'B']].to_numpy()
>>> df[['A', 'B']]
A B
2000-01-01 0.469112 -0.282863
2000-01-02 1.212112 -0.173215
2000-01-03 -0.861849 -2.104569
2000-01-04 0.721555 -0.706771
2000-01-05 -0.424972 0.567020
2000-01-06 -0.673690 0.113648
2000-01-07 0.404705 0.577046
2000-01-08 -0.370647 -1.157892
Attribute access¶
You may access an index on a Series
or column on a DataFrame
directly
as an attribute:
sa = pd.Series([1, 2, 3], index=list('abc'))
dfa = df.copy()
sa.b
dfa.A
>>> sa.a = 5
>>> sa
a 5
b 2
c 3
dtype: int64
>>> dfa.A = list(range(len(dfa.index))) # ok if A already exists
>>> dfa
A B C D
2000-01-01 0 -0.282863 -1.509059 -1.135632
2000-01-02 1 -0.173215 0.119209 -1.044236
2000-01-03 2 -2.104569 -0.494929 1.071804
2000-01-04 3 -0.706771 -1.039575 0.271860
2000-01-05 4 0.567020 0.276232 -1.087401
2000-01-06 5 0.113648 -1.478427 0.524988
2000-01-07 6 0.577046 -1.715002 -1.039268
2000-01-08 7 -1.157892 -1.344312 0.844885
>>> dfa['A'] = list(range(len(dfa.index))) # use this form to create a new column
>>> dfa
A B C D
2000-01-01 0 -0.282863 -1.509059 -1.135632
2000-01-02 1 -0.173215 0.119209 -1.044236
2000-01-03 2 -2.104569 -0.494929 1.071804
2000-01-04 3 -0.706771 -1.039575 0.271860
2000-01-05 4 0.567020 0.276232 -1.087401
2000-01-06 5 0.113648 -1.478427 0.524988
2000-01-07 6 0.577046 -1.715002 -1.039268
2000-01-08 7 -1.157892 -1.344312 0.844885
Warning
You can use this access only if the index element is a valid Python identifier, e.g.
s.1
is not allowed. See here for an explanation of valid identifiers.The attribute will not be available if it conflicts with an existing method name, e.g.
s.min
is not allowed, buts['min']
is possible.Similarly, the attribute will not be available if it conflicts with any of the following list:
index
,major_axis
,minor_axis
,items
.In any of these cases, standard indexing will still work, e.g.
s['1']
,s['min']
, ands['index']
will access the corresponding element or column.
If you are using the IPython environment, you may also use tab-completion to see these accessible attributes.
You can also assign a dict
to a row of a DataFrame
:
>>> x = pd.DataFrame({'x': [1, 2, 3], 'y': [3, 4, 5]})
>>> x.iloc[1] = {'x': 9, 'y': 99}
>>> x
x y
0 1 3
1 9 99
2 3 5
You can use attribute access to modify an existing element of a Series or column of a DataFrame, but be careful;
if you try to use attribute access to create a new column, it creates a new attribute rather than a
new column. In 0.21.0 and later, this will raise a UserWarning
:
>>> df = pd.DataFrame({'one': [1., 2., 3.]})
>>> df.two = [4, 5, 6]
UserWarning: Pandas doesn't allow Series to be assigned into nonexistent columns - see https://pandas.pydata.org/pandas-docs/stable/indexing.html#attribute_access
>>> df
one
0 1.0
1 2.0
2 3.0
Selection by label¶
Warning
Whether a copy or a reference is returned for a setting operation, may depend on the context.
This is sometimes called chained assignment
and should be avoided.
See Returning a View versus Copy.
Warning
.loc
is strict when you present slicers that are not compatible (or convertible) with the index type. For example using integers in aDatetimeIndex
. These will raise aTypeError
.
dfl = pd.DataFrame(np.random.randn(5, 4),
columns=list('ABCD'),
index=pd.date_range('20130101', periods=5))
dfl
>>> dfl.loc[2:3]
TypeError: cannot do slice indexing on <class 'pandas.tseries.index.DatetimeIndex'> with these indexers [2] of <type 'int'>
String likes in slicing can be convertible to the type of the index and lead to natural slicing.
dfl.loc['20130102':'20130104']
pandas provides a suite of methods in order to have purely label based indexing. This is a strict inclusion based protocol.
Every label asked for must be in the index, or a KeyError
will be raised.
When slicing, both the start bound AND the stop bound are included, if present in the index.
Integers are valid labels, but they refer to the label and not the position.
The .loc
attribute is the primary access method. The following are valid inputs:
A single label, e.g.
5
or'a'
(Note that5
is interpreted as a label of the index. This use is not an integer position along the index.).A list or array of labels
['a', 'b', 'c']
.A slice object with labels
'a':'f'
(Note that contrary to usual python slices, both the start and the stop are included, when present in the index! See Slicing with labels.A boolean array.
A
callable
, see Selection By Callable.
>>> s1 = pd.Series(np.random.randn(6), index=list('abcdef'))
>>> s1
a 1.431256
b 1.340309
c -1.170299
d -0.226169
e 0.410835
f 0.813850
dtype: float64
>>> s1.loc['c':]
c -1.170299
d -0.226169
e 0.410835
f 0.813850
dtype: float64
>>> s1.loc['b']
1.3403088497993827
Note that setting works as well:
>>> s1.loc['c':] = 0
>>> s1
a 1.431256
b 1.340309
c 0.000000
d 0.000000
e 0.000000
f 0.000000
dtype: float64
With a DataFrame:
>>> df1 = pd.DataFrame(np.random.randn(6, 4),
.... index=list('abcdef'),
.... columns=list('ABCD'))
....
>>> df1
A B C D
a 0.132003 -0.827317 -0.076467 -1.187678
b 1.130127 -1.436737 -1.413681 1.607920
c 1.024180 0.569605 0.875906 -2.211372
d 0.974466 -2.006747 -0.410001 -0.078638
e 0.545952 -1.219217 -1.226825 0.769804
f -1.281247 -0.727707 -0.121306 -0.097883
>>> df1.loc[['a', 'b', 'd'], :]
A B C D
a 0.132003 -0.827317 -0.076467 -1.187678
b 1.130127 -1.436737 -1.413681 1.607920
Slicing with labels¶
When using .loc
with slices, if both the start and the stop labels are
present in the index, then elements located between the two (including them)
are returned:
>>> s = pd.Series(list('abcde'), index=[0, 3, 2, 5, 4])
>>> s.loc[3:5]
3 b
2 c
5 d
dtype: object
If at least one of the two is absent, but the index is sorted, and can be compared against start and stop labels, then slicing will still work as expected, by selecting labels which rank between the two:
>>> s.sort_index()
0 a
2 c
3 b
4 e
5 d
dtype: object
>>> s.sort_index().loc[1:6]
2 c
3 b
4 e
5 d
dtype: object
However, if at least one of the two is absent and the index is not sorted, an
error will be raised (since doing otherwise would be computationally expensive,
as well as potentially ambiguous for mixed type indexes). For instance, in the
above example, s.loc[1:6]
would raise KeyError
.
Selection by position¶
Warning
Whether a copy or a reference is returned for a setting operation, may depend on the context.
This is sometimes called chained assignment
and should be avoided.
See Returning a View versus Copy.
Pandas provides a suite of methods in order to get purely integer based indexing. The semantics follow closely Python and NumPy slicing. These are 0-based
indexing. When slicing, the start bound is included, while the upper bound is excluded. Trying to use a non-integer, even a valid label will raise an IndexError
.
The .iloc
attribute is the primary access method. The following are valid inputs:
An integer e.g.
5
.A list or array of integers
[4, 3, 0]
.A slice object with ints
1:7
.A boolean array.
A
callable
, see Selection By Callable.
>>> s1 = pd.Series(np.random.randn(5), index=list(range(0, 10, 2)))
>>> s1
0 0.695775
2 0.341734
4 0.959726
6 -1.110336
8 -0.619976
dtype: float64
>>> s1.iloc[:3]
0 0.695775
2 0.341734
4 0.959726
dtype: float64
>>> s1.iloc[3]
-1.110336102891167
Note that setting works as well:
s1.iloc[:3] = 0
s1
With a DataFrame:
df1 = pd.DataFrame(np.random.randn(6, 4),
index=list(range(0, 12, 2)),
columns=list(range(0, 8, 2)))
df1
Select via integer slicing:
df1.iloc[:3]
df1.iloc[1:5, 2:4]
Select via integer list:
df1.iloc[[1, 3, 5], [1, 3]]
df1.iloc[1:3, :]
df1.iloc[:, 1:3]
# this is also equivalent to ``df1.iat[1,1]``
df1.iloc[1, 1]
For getting a cross section using an integer position (equiv to df.xs(1)
):
df1.iloc[1]
Out of range slice indexes are handled gracefully just as in Python/Numpy.
# these are allowed in python/numpy.
x = list('abcdef')
x
x[4:10]
x[8:10]
s = pd.Series(x)
s
s.iloc[4:10]
s.iloc[8:10]
Note that using slices that go out of bounds can result in an empty axis (e.g. an empty DataFrame being returned).
dfl = pd.DataFrame(np.random.randn(5, 2), columns=list('AB'))
dfl
dfl.iloc[:, 2:3]
dfl.iloc[:, 1:3]
dfl.iloc[4:6]
A single indexer that is out of bounds will raise an IndexError
.
A list of indexers where any element is out of bounds will raise an
IndexError
.
>>> dfl.iloc[[4, 5, 6]]
IndexError: positional indexers are out-of-bounds
>>> dfl.iloc[:, 4]
IndexError: single positional indexer is out-of-bounds
Selection by callable¶
.loc
, .iloc
, and also []
indexing can accept a callable
as indexer.
The callable
must be a function with one argument (the calling Series or DataFrame) that returns valid output for indexing.
>>> df1 = pd.DataFrame(np.random.randn(6, 4),
.... index=list('abcdef'),
.... columns=list('ABCD'))
....
>>> df1
A B C D
a -0.023688 2.410179 1.450520 0.206053
b -0.251905 -2.213588 1.063327 1.266143
c 0.299368 -0.863838 0.408204 -1.048089
d -0.025747 -0.988387 0.094055 1.262731
e 1.289997 0.082423 -0.055758 0.536580
f -0.489682 0.369374 -0.034571 -2.484478
>>> df1.loc[lambda df: df['A'] > 0, :]
A B C D
c 0.299368 -0.863838 0.408204 -1.048089
e 1.289997 0.082423 -0.055758 0.536580
>>> df1.loc[:, lambda df: ['A', 'B']]
A B
a -0.023688 2.410179
b -0.251905 -2.213588
c 0.299368 -0.863838
d -0.025747 -0.988387
e 1.289997 0.082423
f -0.489682 0.369374
>>> df1.iloc[:, lambda df: [0, 1]]
A B
a -0.023688 2.410179
b -0.251905 -2.213588
c 0.299368 -0.863838
d -0.025747 -0.988387
e 1.289997 0.082423
f -0.489682 0.369374
>>> df1[lambda df: df.columns[0]]
a -0.023688
b -0.251905
c 0.299368
d -0.025747
e 1.289997
f -0.489682
Name: A, dtype: float64
You can use callable indexing in Series
.
df1['A'].loc[lambda s: s > 0]
Using these methods / indexers, you can chain data selection operations without using a temporary variable.
bb = pd.read_csv('data/baseball.csv', index_col='id')
(bb.groupby(['year', 'team']).sum()
.loc[lambda df: df['r'] > 100])
Boolean indexing¶
Another common operation is the use of boolean vectors to filter the data.
The operators are: |
for or
, &
for and
, and ~
for not
.
These must be grouped by using parentheses, since by default Python will
evaluate an expression such as df['A'] > 2 & df['B'] < 3
as
df['A'] > (2 & df['B']) < 3
, while the desired evaluation order is
(df['A > 2) & (df['B'] < 3)
.
Using a boolean vector to index a Series works exactly as in a NumPy ndarray:
>>> s = pd.Series(range(-3, 4))
>>> s
0 -3
1 -2
2 -1
3 0
4 1
5 2
6 3
dtype: int64
>>> s[s > 0]
4 1
5 2
6 3
dtype: int64
>>> s[(s < -1) | (s > 0.5)]
0 -3
1 -2
4 1
5 2
6 3
dtype: int64
>>> s[~(s < 0)]
3 0
4 1
5 2
6 3
dtype: int64
You may select rows from a DataFrame using a boolean vector the same length as the DataFrame’s index (for example, something derived from one of the columns of the DataFrame):
df[df['A'] > 0]
List comprehensions and the map
method of Series can also be used to produce
more complex criteria:
df2 = pd.DataFrame({'a': ['one', 'one', 'two', 'three', 'two', 'one', 'six'],
'b': ['x', 'y', 'y', 'x', 'y', 'x', 'x'],
'c': np.random.randn(7)})
# only want 'two' or 'three'
criterion = df2['a'].map(lambda x: x.startswith('t'))
df2[criterion]
# equivalent but slower
df2[[x.startswith('t') for x in df2['a']]]
# Multiple criteria
df2[criterion & (df2['b'] == 'x')]
With the choice methods Selection by Label, Selection by Position you may select along more than one axis using boolean vectors combined with other indexing expressions.
df2.loc[criterion & (df2['b'] == 'x'), 'b':'c']
The query()
Method¶
DataFrame
objects have a query()
method that allows selection using an expression.
You can get the value of the frame where column b
has values
between the values of columns a
and c
. For example:
n = 10
df = pd.DataFrame(np.random.rand(n, 3), columns=list('abc'))
df
# pure python
df[(df['a'] < df['b']) & (df['b'] < df['c'])]
# query
df.query('(a < b) & (b < c)')
Do the same thing but fall back on a named index if there is no column
with the name a
.
df = pd.DataFrame(np.random.randint(n / 2, size=(n, 2)), columns=list('bc'))
df.index.name = 'a'
df
df.query('a < b and b < c')
If instead you don’t want to or cannot name your index, you can use the name
index
in your query expression:
df = pd.DataFrame(np.random.randint(n, size=(n, 2)), columns=list('bc'))
df
df.query('index < b < c')
Note
If the name of your index overlaps with a column name, the column name is given precedence. For example,
df = pd.DataFrame({'a': np.random.randint(5, size=5)})
df.index.name = 'a'
df.query('a > 2') # uses the column 'a', not the index
You can still use the index in a query expression by using the special identifier ‘index’:
df.query('index > 2')
If for some reason you have a column named index
, then you can refer to
the index as ilevel_0
as well, but at this point you should consider
renaming your columns to something less ambiguous.
MultiIndex
query()
Syntax¶
You can also use the levels of a DataFrame
with a
MultiIndex
as if they were columns in the frame:
n = 10
colors = np.random.choice(['red', 'green'], size=n)
foods = np.random.choice(['eggs', 'ham'], size=n)
colors
foods
index = pd.MultiIndex.from_arrays([colors, foods], names=['color', 'food'])
df = pd.DataFrame(np.random.randn(n, 2), index=index)
df
df.query('color == "red"')
If the levels of the MultiIndex
are unnamed, you can refer to them using
special names:
df.index.names = [None, None]
df
df.query('ilevel_0 == "red"')
The convention is ilevel_0
, which means “index level 0” for the 0th level
of the index
.
query()
Use Cases¶
A use case for query()
is when you have a collection of
DataFrame
objects that have a subset of column names (or index
levels/names) in common. You can pass the same query to both frames without
having to specify which frame you’re interested in querying
df = pd.DataFrame(np.random.rand(n, 3), columns=list('abc'))
df
df2 = pd.DataFrame(np.random.rand(n + 2, 3), columns=df.columns)
df2
expr = '0.0 <= a <= c <= 0.5'
map(lambda frame: frame.query(expr), [df, df2])
query()
Python versus pandas Syntax Comparison¶
Full numpy-like syntax:
df = pd.DataFrame(np.random.randint(n, size=(n, 3)), columns=list('abc'))
df
df.query('(a < b) & (b < c)')
df[(df['a'] < df['b']) & (df['b'] < df['c'])]
Slightly nicer by removing the parentheses (by binding making comparison
operators bind tighter than &
and |
).
df.query('a < b & b < c')
Use English instead of symbols:
df.query('a < b and b < c')
Pretty close to how you might write it on paper:
df.query('a < b < c')
The in
and not in
operators¶
query()
also supports special use of Python’s in
and
not in
comparison operators, providing a succinct syntax for calling the
isin
method of a Series
or DataFrame
.
# get all rows where columns "a" and "b" have overlapping values
df = pd.DataFrame({'a': list('aabbccddeeff'), 'b': list('aaaabbbbcccc'),
'c': np.random.randint(5, size=12),
'd': np.random.randint(9, size=12)})
df
df.query('a in b')
# How you'd do it in pure Python
df[df['a'].isin(df['b'])]
df.query('a not in b')
# pure Python
df[~df['a'].isin(df['b'])]
You can combine this with other expressions for very succinct queries:
# rows where cols a and b have overlapping values
# and col c's values are less than col d's
df.query('a in b and c < d')
# pure Python
df[df['b'].isin(df['a']) & (df['c'] < df['d'])]
Note
Note that in
and not in
are evaluated in Python, since numexpr
has no equivalent of this operation. However, only the in
/not in
expression itself is evaluated in vanilla Python. For example, in the
expression
df.query('a in b + c + d')
(b + c + d)
is evaluated by numexpr
and then the in
operation is evaluated in plain Python. In general, any operations that can
be evaluated using numexpr
will be.
Special use of the ==
operator with list
objects¶
Comparing a list
of values to a column using ==
/!=
works similarly
to in
/not in
.
df.query('b == ["a", "b", "c"]')
# pure Python
df[df['b'].isin(["a", "b", "c"])]
df.query('c == [1, 2]')
df.query('c != [1, 2]')
# using in/not in
df.query('[1, 2] in c')
df.query('[1, 2] not in c')
# pure Python
df[df['c'].isin([1, 2])]
Returning a view versus a copy¶
When setting values in a pandas object, care must be taken to avoid what is called
chained indexing
. Here is an example.
dfmi = pd.DataFrame([list('abcd'),
list('efgh'),
list('ijkl'),
list('mnop')],
columns=pd.MultiIndex.from_product([['one', 'two'],
['first', 'second']]))
dfmi
Compare these two access methods:
dfmi['one']['second']
dfmi.loc[:, ('one', 'second')]
These both yield the same results, so which should you use? It is instructive to understand the order
of operations on these and why method 2 (.loc
) is much preferred over method 1 (chained []
).
dfmi['one']
selects the first level of the columns and returns a DataFrame that is singly-indexed.
Then another Python operation dfmi_with_one['second']
selects the series indexed by 'second'
.
This is indicated by the variable dfmi_with_one
because pandas sees these operations as separate events.
e.g. separate calls to __getitem__
, so it has to treat them as linear operations, they happen one after another.
Contrast this to df.loc[:,('one','second')]
which passes a nested tuple of (slice(None),('one','second'))
to a single call to
__getitem__
. This allows pandas to deal with this as a single entity. Furthermore this order of operations can be significantly
faster, and allows one to index both axes if so desired.
Why does assignment fail when using chained indexing?¶
The problem in the previous section is just a performance issue. What’s up with
the SettingWithCopy
warning? We don’t usually throw warnings around when
you do something that might cost a few extra milliseconds!
But it turns out that assigning to the product of chained indexing has inherently unpredictable results. To see this, think about how the Python interpreter executes this code:
value = None
dfmi.loc[:, ('one', 'second')] = value
# becomes
dfmi.loc.__setitem__((slice(None), ('one', 'second')), value)
But this code is handled differently:
dfmi['one']['second'] = value
# becomes
dfmi.__getitem__('one').__setitem__('second', value)
See that __getitem__
in there? Outside of simple cases, it’s very hard to
predict whether it will return a view or a copy (it depends on the memory layout
of the array, about which pandas makes no guarantees), and therefore whether
the __setitem__
will modify dfmi
or a temporary object that gets thrown
out immediately afterward. That’s what SettingWithCopy
is warning you
about!
Note
You may be wondering whether we should be concerned about the loc
property in the first example. But dfmi.loc
is guaranteed to be dfmi
itself with modified indexing behavior, so dfmi.loc.__getitem__
/
dfmi.loc.__setitem__
operate on dfmi
directly. Of course,
dfmi.loc.__getitem__(idx)
may be a view or a copy of dfmi
.
Sometimes a SettingWithCopy
warning will arise at times when there’s no
obvious chained indexing going on. These are the bugs that
SettingWithCopy
is designed to catch! Pandas is probably trying to warn you
that you’ve done this:
def do_something(df):
foo = df[['bar', 'baz']] # Is foo a view? A copy? Nobody knows!
# ... many lines here ...
# We don't know whether this will modify df or not!
foo['quux'] = value
return foo
Yikes!
Evaluation order matters¶
When you use chained indexing, the order and type of the indexing operation partially determine whether the result is a slice into the original object, or a copy of the slice.
Pandas has the SettingWithCopyWarning
because assigning to a copy of a
slice is frequently not intentional, but a mistake caused by chained indexing
returning a copy where a slice was expected.
If you would like pandas to be more or less trusting about assignment to a
chained indexing expression, you can set the option
mode.chained_assignment
to one of these values:
'warn'
, the default, means aSettingWithCopyWarning
is printed.'raise'
means pandas will raise aSettingWithCopyException
you have to deal with.None
will suppress the warnings entirely.
dfb = pd.DataFrame({'a': ['one', 'one', 'two',
'three', 'two', 'one', 'six'],
'c': np.arange(7)})
# This will show the SettingWithCopyWarning
# but the frame values will be set
dfb['c'][dfb['a'].str.startswith('o')] = 42
This however is operating on a copy and will not work.
>>> pd.set_option('mode.chained_assignment','warn')
>>> dfb[dfb['a'].str.startswith('o')]['c'] = 42
Traceback (most recent call last)
...
SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
A chained assignment can also crop up in setting in a mixed dtype frame.
Note
These setting rules apply to all of .loc/.iloc
.
This is the correct access method:
dfc = pd.DataFrame({'A': ['aaa', 'bbb', 'ccc'], 'B': [1, 2, 3]})
dfc.loc[0, 'A'] = 11
dfc
This can work at times, but it is not guaranteed to, and therefore should be avoided:
dfc = dfc.copy()
dfc['A'][0] = 111
dfc
This will not work at all, and so should be avoided:
>>> pd.set_option('mode.chained_assignment','raise')
>>> dfc.loc[0]['A'] = 1111
Traceback (most recent call last)
...
SettingWithCopyException:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_index,col_indexer] = value instead
Warning
The chained assignment warnings / exceptions are aiming to inform the user of a possibly invalid assignment. There may be false positives; situations where a chained assignment is inadvertently reported.
Top-level headers and the TOC¶
Your right table of contents will behave slightly differently depending on whether your page has one top-level header, or multiple top-level headers. See below for more information.
An example with multiple top-level headers¶
If a page has multiple top-level headers on it, then the in-page Table of Contents will show each top-level header. On this page, there are multiple top-level headers. As a result, the top-level headers all appear in the right Table of Contents. Here’s an example of a page structure with multiple top-level headers:
My first header
===============
My sub-header
-------------
My second header
================
My second sub-header
--------------------
And here’s a second-level header¶
Notice how it is nested underneath “Top-level header 2” in the TOC.
An example with a single top-level header¶
If the page only has a single top-level header, it is assumed to be the page title, and only the headers underneath the top-level header will be used for the right Table of Contents.
On most pages in this documentation, only a single top-level header is used. For example, they have a page structure like:
My title
========
My header
---------
My second header
----------------
Section with sub-pages¶
To create an additional level of nesting in the sidebar, construct a
nested toctree
:
Sub-page 1¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec lorem neque, interdum in ipsum nec, finibus dictum velit. Ut eu efficitur arcu, id aliquam erat. In sit amet diam gravida, imperdiet tellus eu, gravida nisl. Praesent aliquet odio eget libero elementum, quis rhoncus tellus tincidunt. Suspendisse quis volutpat ipsum. Sed lobortis scelerisque tristique. Aenean condimentum risus tellus, quis accumsan ipsum laoreet ut. Integer porttitor maximus suscipit. Mauris in posuere sapien. Aliquam accumsan feugiat ligula, nec fringilla libero commodo sed. Proin et erat pharetra.
Sub-page 2¶
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec lorem neque, interdum in ipsum nec, finibus dictum velit. Ut eu efficitur arcu, id aliquam erat. In sit amet diam gravida, imperdiet tellus eu, gravida nisl. Praesent aliquet odio eget libero elementum, quis rhoncus tellus tincidunt. Suspendisse quis volutpat ipsum. Sed lobortis scelerisque tristique. Aenean condimentum risus tellus, quis accumsan ipsum laoreet ut. Integer porttitor maximus suscipit. Mauris in posuere sapien. Aliquam accumsan feugiat ligula, nec fringilla libero commodo sed. Proin et erat pharetra.
Changelog¶
See the GitHub Releases for the changelog of each release.