Intro to Data Structures
We’ll start with a quick, non-comprehensive overview of the fundamental data structures in pandas to get you started. The fundamental behavior about data types, indexing, and axis labeling / alignment apply across all of the objects. To get started, import numpy and load pandas into your namespace:
In [1]: import numpy as np In [2]: import pandas as pd
Here is a basic tenet to keep in mind: data alignment is intrinsic. The link between labels and data will not be broken unless done so explicitly by you.
We’ll give a brief intro to the data structures, then consider all of the broad categories of functionality and methods in separate sections.
Series
Warning
In 0.13.0 Series
has internally been refactored to no longer sub-class ndarray
but instead subclass NDFrame
, similarly to the rest of the pandas containers. This should be a transparent change with only very limited API implications (See the Internal Refactoring)
Series
is a one-dimensional labeled array capable of holding any data type (integers, strings, floating point numbers, Python objects, etc.). The axis labels are collectively referred to as the index. The basic method to create a Series is to call:
>>> s = pd.Series(data, index=index)
Here, data
can be many different things:
- a Python dict
- an ndarray
- a scalar value (like 5)
The passed index is a list of axis labels. Thus, this separates into a few cases depending on what data is:
From ndarray
If data
is an ndarray, index must be the same length as data. If no index is passed, one will be created having values [0, ..., len(data) - 1]
.
In [3]: s = pd.Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e']) In [4]: s Out[4]: a -2.7828 b 0.4264 c -0.6505 d 1.1465 e -0.6631 dtype: float64 In [5]: s.index Out[5]: Index([u'a', u'b', u'c', u'd', u'e'], dtype='object') In [6]: pd.Series(np.random.randn(5)) Out[6]: 0 0.2939 1 -0.4049 2 1.1665 3 0.8420 4 0.5398 dtype: float64
Note
Starting in v0.8.0, pandas supports non-unique index values. If an operation that does not support duplicate index values is attempted, an exception will be raised at that time. The reason for being lazy is nearly all performance-based (there are many instances in computations, like parts of GroupBy, where the index is not used).
From dict
If data
is a dict, if index is passed the values in data corresponding to the labels in the index will be pulled out. Otherwise, an index will be constructed from the sorted keys of the dict, if possible.
In [7]: d = {'a' : 0., 'b' : 1., 'c' : 2.} In [8]: pd.Series(d) Out[8]: a 0.0 b 1.0 c 2.0 dtype: float64 In [9]: pd.Series(d, index=['b', 'c', 'd', 'a']) Out[9]: b 1.0 c 2.0 d NaN a 0.0 dtype: float64
Note
NaN (not a number) is the standard missing data marker used in pandas
From scalar value If data
is a scalar value, an index must be provided. The value will be repeated to match the length of index
In [10]: pd.Series(5., index=['a', 'b', 'c', 'd', 'e']) Out[10]: a 5.0 b 5.0 c 5.0 d 5.0 e 5.0 dtype: float64
Series is ndarray-like
Series
acts very similarly to a ndarray
, and is a valid argument to most NumPy functions. However, things like slicing also slice the index.
In [11]: s[0] Out[11]: -2.7827595933769937 In [12]: s[:3] Out[12]: a -2.7828 b 0.4264 c -0.6505 dtype: float64 In [13]: s[s > s.median()] Out[13]: b 0.4264 d 1.1465 dtype: float64 In [14]: s[[4, 3, 1]] Out[14]: e -0.6631 d 1.1465 b 0.4264 dtype: float64 In [15]: np.exp(s) Out[15]: a 0.0619 b 1.5318 c 0.5218 d 3.1472 e 0.5153 dtype: float64
We will address array-based indexing in a separate section.
Series is dict-like
A Series is like a fixed-size dict in that you can get and set values by index label:
In [16]: s['a'] Out[16]: -2.7827595933769937 In [17]: s['e'] = 12. In [18]: s Out[18]: a -2.7828 b 0.4264 c -0.6505 d 1.1465 e 12.0000 dtype: float64 In [19]: 'e' in s Out[19]: True In [20]: 'f' in s Out[20]: False
If a label is not contained, an exception is raised:
>>> s['f'] KeyError: 'f'
Using the get
method, a missing label will return None or specified default:
In [21]: s.get('f') In [22]: s.get('f', np.nan) Out[22]: nan
See also the section on attribute access.
Vectorized operations and label alignment with Series
When doing data analysis, as with raw NumPy arrays looping through Series value-by-value is usually not necessary. Series can be also be passed into most NumPy methods expecting an ndarray.
In [23]: s + s Out[23]: a -5.5655 b 0.8529 c -1.3010 d 2.2930 e 24.0000 dtype: float64 In [24]: s * 2 Out[24]: a -5.5655 b 0.8529 c -1.3010 d 2.2930 e 24.0000 dtype: float64 In [25]: np.exp(s) Out[25]: a 0.0619 b 1.5318 c 0.5218 d 3.1472 e 162754.7914 dtype: float64
A key difference between Series and ndarray is that operations between Series automatically align the data based on label. Thus, you can write computations without giving consideration to whether the Series involved have the same labels.
In [26]: s[1:] + s[:-1] Out[26]: a NaN b 0.8529 c -1.3010 d 2.2930 e NaN dtype: float64
The result of an operation between unaligned Series will have the union of the indexes involved. If a label is not found in one Series or the other, the result will be marked as missing NaN
. Being able to write code without doing any explicit data alignment grants immense freedom and flexibility in interactive data analysis and research. The integrated data alignment features of the pandas data structures set pandas apart from the majority of related tools for working with labeled data.
Note
In general, we chose to make the default result of operations between differently indexed objects yield the union of the indexes in order to avoid loss of information. Having an index label, though the data is missing, is typically important information as part of a computation. You of course have the option of dropping labels with missing data via the dropna function.
Name attribute
Series can also have a name
attribute:
In [27]: s = pd.Series(np.random.randn(5), name='something') In [28]: s Out[28]: 0 0.5410 1 -1.1747 2 0.1287 3 0.0430 4 -0.4287 Name: something, dtype: float64 In [29]: s.name Out[29]: 'something'
The Series name
will be assigned automatically in many cases, in particular when taking 1D slices of DataFrame as you will see below.
New in version 0.18.0.
You can rename a Series with the pandas.Series.rename()
method.
In [30]: s2 = s.rename("different") In [31]: s2.name Out[31]: 'different'
Note that s
and s2
refer to different objects.
DataFrame
DataFrame is a 2-dimensional labeled data structure with columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dict of Series objects. It is generally the most commonly used pandas object. Like Series, DataFrame accepts many different kinds of input:
- Dict of 1D ndarrays, lists, dicts, or Series
- 2-D numpy.ndarray
- Structured or record ndarray
- A
Series
- Another
DataFrame
Along with the data, you can optionally pass index (row labels) and columns (column labels) arguments. If you pass an index and / or columns, you are guaranteeing the index and / or columns of the resulting DataFrame. Thus, a dict of Series plus a specific index will discard all data not matching up to the passed index.
If axis labels are not passed, they will be constructed from the input data based on common sense rules.
From dict of Series or dicts
The result index will be the union of the indexes of the various Series. If there are any nested dicts, these will be first converted to Series. If no columns are passed, the columns will be the sorted list of dict keys.
In [32]: d = {'one' : pd.Series([1., 2., 3.], index=['a', 'b', 'c']), ....: 'two' : pd.Series([1., 2., 3., 4.], index=['a', 'b', 'c', 'd'])} ....: In [33]: df = pd.DataFrame(d) In [34]: df Out[34]: one two a 1.0 1.0 b 2.0 2.0 c 3.0 3.0 d NaN 4.0 In [35]: pd.DataFrame(d, index=['d', 'b', 'a']) Out[35]: one two d NaN 4.0 b 2.0 2.0 a 1.0 1.0 In [36]: pd.DataFrame(d, index=['d', 'b', 'a'], columns=['two', 'three']) Out[36]: two three d 4.0 NaN b 2.0 NaN a 1.0 NaN
The row and column labels can be accessed respectively by accessing the index and columns attributes:
Note
When a particular set of columns is passed along with a dict of data, the passed columns override the keys in the dict.
In [37]: df.index Out[37]: Index([u'a', u'b', u'c', u'd'], dtype='object') In [38]: df.columns Out[38]: Index([u'one', u'two'], dtype='object')
From dict of ndarrays / lists
The ndarrays must all be the same length. If an index is passed, it must clearly also be the same length as the arrays. If no index is passed, the result will be range(n)
, where n
is the array length.
In [39]: d = {'one' : [1., 2., 3., 4.], ....: 'two' : [4., 3., 2., 1.]} ....: In [40]: pd.DataFrame(d) Out[40]: one two 0 1.0 4.0 1 2.0 3.0 2 3.0 2.0 3 4.0 1.0 In [41]: pd.DataFrame(d, index=['a', 'b', 'c', 'd']) Out[41]: one two a 1.0 4.0 b 2.0 3.0 c 3.0 2.0 d 4.0 1.0
From structured or record array
This case is handled identically to a dict of arrays.
In [42]: data = np.zeros((2,), dtype=[('A', 'i4'),('B', 'f4'),('C', 'a10')]) In [43]: data[:] = [(1,2.,'Hello'), (2,3.,"World")] In [44]: pd.DataFrame(data) Out[44]: A B C 0 1 2.0 Hello 1 2 3.0 World In [45]: pd.DataFrame(data, index=['first', 'second']) Out[45]: A B C first 1 2.0 Hello second 2 3.0 World In [46]: pd.DataFrame(data, columns=['C', 'A', 'B']) Out[46]: C A B 0 Hello 1 2.0 1 World 2 3.0
Note
DataFrame is not intended to work exactly like a 2-dimensional NumPy ndarray.
From a list of dicts
In [47]: data2 = [{'a': 1, 'b': 2}, {'a': 5, 'b': 10, 'c': 20}] In [48]: pd.DataFrame(data2) Out[48]: a b c 0 1 2 NaN 1 5 10 20.0 In [49]: pd.DataFrame(data2, index=['first', 'second']) Out[49]: a b c first 1 2 NaN second 5 10 20.0 In [50]: pd.DataFrame(data2, columns=['a', 'b']) Out[50]: a b 0 1 2 1 5 10
From a dict of tuples
You can automatically create a multi-indexed frame by passing a tuples dictionary
In [51]: pd.DataFrame({('a', 'b'): {('A', 'B'): 1, ('A', 'C'): 2}, ....: ('a', 'a'): {('A', 'C'): 3, ('A', 'B'): 4}, ....: ('a', 'c'): {('A', 'B'): 5, ('A', 'C'): 6}, ....: ('b', 'a'): {('A', 'C'): 7, ('A', 'B'): 8}, ....: ('b', 'b'): {('A', 'D'): 9, ('A', 'B'): 10}}) ....: Out[51]: a b a b c a b A B 4.0 1.0 5.0 8.0 10.0 C 3.0 2.0 6.0 7.0 NaN D NaN NaN NaN NaN 9.0
From a Series
The result will be a DataFrame with the same index as the input Series, and with one column whose name is the original name of the Series (only if no other column name provided).
Missing Data
Much more will be said on this topic in the Missing data section. To construct a DataFrame with missing data, use np.nan
for those values which are missing. Alternatively, you may pass a numpy.MaskedArray
as the data argument to the DataFrame constructor, and its masked entries will be considered missing.
Alternate Constructors
DataFrame.from_dict
DataFrame.from_dict
takes a dict of dicts or a dict of array-like sequences and returns a DataFrame. It operates like the DataFrame
constructor except for the orient
parameter which is 'columns'
by default, but which can be set to 'index'
in order to use the dict keys as row labels.
DataFrame.from_records
DataFrame.from_records
takes a list of tuples or an ndarray with structured dtype. Works analogously to the normal DataFrame
constructor, except that index maybe be a specific field of the structured dtype to use as the index. For example:
In [52]: data Out[52]: array([(1, 2.0, 'Hello'), (2, 3.0, 'World')], dtype=[('A', '<i4'), ('B', '<f4'), ('C', 'S10')]) In [53]: pd.DataFrame.from_records(data, index='C') Out[53]: A B C Hello 1 2.0 World 2 3.0
DataFrame.from_items
DataFrame.from_items
works analogously to the form of the dict
constructor that takes a sequence of (key, value)
pairs, where the keys are column (or row, in the case of orient='index'
) names, and the value are the column values (or row values). This can be useful for constructing a DataFrame with the columns in a particular order without having to pass an explicit list of columns:
In [54]: pd.DataFrame.from_items([('A', [1, 2, 3]), ('B', [4, 5, 6])]) Out[54]: A B 0 1 4 1 2 5 2 3 6
If you pass orient='index'
, the keys will be the row labels. But in this case you must also pass the desired column names:
In [55]: pd.DataFrame.from_items([('A', [1, 2, 3]), ('B', [4, 5, 6])], ....: orient='index', columns=['one', 'two', 'three']) ....: Out[55]: one two three A 1 2 3 B 4 5 6
Column selection, addition, deletion
You can treat a DataFrame semantically like a dict of like-indexed Series objects. Getting, setting, and deleting columns works with the same syntax as the analogous dict operations:
In [56]: df['one'] Out[56]: a 1.0 b 2.0 c 3.0 d NaN Name: one, dtype: float64 In [57]: df['three'] = df['one'] * df['two'] In [58]: df['flag'] = df['one'] > 2 In [59]: df Out[59]: one two three flag a 1.0 1.0 1.0 False b 2.0 2.0 4.0 False c 3.0 3.0 9.0 True d NaN 4.0 NaN False
Columns can be deleted or popped like with a dict:
In [60]: del df['two'] In [61]: three = df.pop('three') In [62]: df Out[62]: one flag a 1.0 False b 2.0 False c 3.0 True d NaN False
When inserting a scalar value, it will naturally be propagated to fill the column:
In [63]: df['foo'] = 'bar' In [64]: df Out[64]: one flag foo a 1.0 False bar b 2.0 False bar c 3.0 True bar d NaN False bar
When inserting a Series that does not have the same index as the DataFrame, it will be conformed to the DataFrame’s index:
In [65]: df['one_trunc'] = df['one'][:2] In [66]: df Out[66]: one flag foo one_trunc a 1.0 False bar 1.0 b 2.0 False bar 2.0 c 3.0 True bar NaN d NaN False bar NaN
You can insert raw ndarrays but their length must match the length of the DataFrame’s index.
By default, columns get inserted at the end. The insert
function is available to insert at a particular location in the columns:
In [67]: df.insert(1, 'bar', df['one']) In [68]: df Out[68]: one bar flag foo one_trunc a 1.0 1.0 False bar 1.0 b 2.0 2.0 False bar 2.0 c 3.0 3.0 True bar NaN d NaN NaN False bar NaN
Assigning New Columns in Method Chains
New in version 0.16.0.
Inspired by dplyr’s mutate
verb, DataFrame has an assign()
method that allows you to easily create new columns that are potentially derived from existing columns.
In [69]: iris = pd.read_csv('data/iris.data') In [70]: iris.head() Out[70]: SepalLength SepalWidth PetalLength PetalWidth Name 0 5.1 3.5 1.4 0.2 Iris-setosa 1 4.9 3.0 1.4 0.2 Iris-setosa 2 4.7 3.2 1.3 0.2 Iris-setosa 3 4.6 3.1 1.5 0.2 Iris-setosa 4 5.0 3.6 1.4 0.2 Iris-setosa In [71]: (iris.assign(sepal_ratio = iris['SepalWidth'] / iris['SepalLength']) ....: .head()) ....: Out[71]: SepalLength SepalWidth PetalLength PetalWidth Name sepal_ratio 0 5.1 3.5 1.4 0.2 Iris-setosa 0.6863 1 4.9 3.0 1.4 0.2 Iris-setosa 0.6122 2 4.7 3.2 1.3 0.2 Iris-setosa 0.6809 3 4.6 3.1 1.5 0.2 Iris-setosa 0.6739 4 5.0 3.6 1.4 0.2 Iris-setosa 0.7200
Above was an example of inserting a precomputed value. We can also pass in a function of one argument to be evalutated on the DataFrame being assigned to.
In [72]: iris.assign(sepal_ratio = lambda x: (x['SepalWidth'] / ....: x['SepalLength'])).head() ....: Out[72]: SepalLength SepalWidth PetalLength PetalWidth Name sepal_ratio 0 5.1 3.5 1.4 0.2 Iris-setosa 0.6863 1 4.9 3.0 1.4 0.2 Iris-setosa 0.6122 2 4.7 3.2 1.3 0.2 Iris-setosa 0.6809 3 4.6 3.1 1.5 0.2 Iris-setosa 0.6739 4 5.0 3.6 1.4 0.2 Iris-setosa 0.7200
assign
always returns a copy of the data, leaving the original DataFrame untouched.
Passing a callable, as opposed to an actual value to be inserted, is useful when you don’t have a reference to the DataFrame at hand. This is common when using assign
in chains of operations. For example, we can limit the DataFrame to just those observations with a Sepal Length greater than 5, calculate the ratio, and plot:
In [73]: (iris.query('SepalLength > 5') ....: .assign(SepalRatio = lambda x: x.SepalWidth / x.SepalLength, ....: PetalRatio = lambda x: x.PetalWidth / x.PetalLength) ....: .plot(kind='scatter', x='SepalRatio', y='PetalRatio')) ....: Out[73]: <matplotlib.axes._subplots.AxesSubplot at 0x11cee0c90>
Since a function is passed in, the function is computed on the DataFrame being assigned to. Importantly, this is the DataFrame that’s been filtered to those rows with sepal length greater than 5. The filtering happens first, and then the ratio calculations. This is an example where we didn’t have a reference to the filtered DataFrame available.
The function signature for assign
is simply **kwargs
. The keys are the column names for the new fields, and the values are either a value to be inserted (for example, a Series
or NumPy array), or a function of one argument to be called on the DataFrame
. A copy of the original DataFrame is returned, with the new values inserted.
Warning
Since the function signature of assign
is **kwargs
, a dictionary, the order of the new columns in the resulting DataFrame cannot be guaranteed to match the order you pass in. To make things predictable, items are inserted alphabetically (by key) at the end of the DataFrame.
All expressions are computed first, and then assigned. So you can’t refer to another column being assigned in the same call to assign
. For example:
In [74]: # Don't do this, bad reference to `C` df.assign(C = lambda x: x['A'] + x['B'], D = lambda x: x['A'] + x['C']) In [2]: # Instead, break it into two assigns (df.assign(C = lambda x: x['A'] + x['B']) .assign(D = lambda x: x['A'] + x['C']))
Indexing / Selection
The basics of indexing are as follows:
Operation | Syntax | Result |
---|---|---|
Select column | df[col] | Series |
Select row by label | df.loc[label] | Series |
Select row by integer location | df.iloc[loc] | Series |
Slice rows | df[5:10] | DataFrame |
Select rows by boolean vector | df[bool_vec] | DataFrame |
Row selection, for example, returns a Series whose index is the columns of the DataFrame:
In [75]: df.loc['b'] Out[75]: one 2 bar 2 flag False foo bar one_trunc 2 Name: b, dtype: object In [76]: df.iloc[2] Out[76]: one 3 bar 3 flag True foo bar one_trunc NaN Name: c, dtype: object
For a more exhaustive treatment of more sophisticated label-based indexing and slicing, see the section on indexing. We will address the fundamentals of reindexing / conforming to new sets of labels in the section on reindexing.
Data alignment and arithmetic
Data alignment between DataFrame objects automatically align on both the columns and the index (row labels). Again, the resulting object will have the union of the column and row labels.
In [77]: df = pd.DataFrame(np.random.randn(10, 4), columns=['A', 'B', 'C', 'D']) In [78]: df2 = pd.DataFrame(np.random.randn(7, 3), columns=['A', 'B', 'C']) In [79]: df + df2 Out[79]: A B C D 0 -1.9160 -0.9862 -2.4213 NaN 1 0.9651 1.6767 0.3298 NaN 2 -1.6622 2.1966 -1.9169 NaN 3 -0.1887 0.7653 -0.0010 NaN 4 -1.0760 0.3969 -1.1774 NaN 5 2.8104 -0.1792 -0.5705 NaN 6 -1.2272 0.1963 0.5312 NaN 7 NaN NaN NaN NaN 8 NaN NaN NaN NaN 9 NaN NaN NaN NaN
When doing an operation between DataFrame and Series, the default behavior is to align the Series index on the DataFrame columns, thus broadcasting row-wise. For example:
In [80]: df - df.iloc[0] Out[80]: A B C D 0 0.0000 0.0000 0.0000 0.0000 1 2.3859 1.3585 1.2234 -2.1065 2 2.1047 1.7004 1.3268 -0.6895 3 1.8741 2.7181 2.3819 -0.7597 4 2.1988 0.9662 0.8265 0.0932 5 4.9966 1.1967 1.3303 -0.2855 6 1.2632 0.5778 1.0712 -0.5254 7 3.4625 0.6322 1.0626 -0.4427 8 2.6802 3.1629 1.2977 -1.8177 9 1.3038 0.1957 3.5899 -0.8671
In the special case of working with time series data, and the DataFrame index also contains dates, the broadcasting will be column-wise:
In [81]: index = pd.date_range('1/1/2000', periods=8) In [82]: df = pd.DataFrame(np.random.randn(8, 3), index=index, columns=list('ABC')) In [83]: df Out[83]: A B C 2000-01-01 0.0627 -0.0284 0.4436 2000-01-02 -0.2688 -1.5776 1.8502 2000-01-03 0.6381 -0.5566 -0.0712 2000-01-04 -0.5114 0.1563 -1.0756 2000-01-05 1.6636 -0.4377 -0.0773 2000-01-06 0.0292 0.1790 1.7401 2000-01-07 -0.7290 -0.8980 -0.3142 2000-01-08 -0.0481 -0.8756 0.1691 In [84]: type(df['A']) Out[84]: pandas.core.series.Series In [85]: df - df['A'] Out[85]: 2000-01-01 00:00:00 2000-01-02 00:00:00 2000-01-03 00:00:00 \ 2000-01-01 NaN NaN NaN 2000-01-02 NaN NaN NaN 2000-01-03 NaN NaN NaN 2000-01-04 NaN NaN NaN 2000-01-05 NaN NaN NaN 2000-01-06 NaN NaN NaN 2000-01-07 NaN NaN NaN 2000-01-08 NaN NaN NaN 2000-01-04 00:00:00 ... 2000-01-08 00:00:00 A B C 2000-01-01 NaN ... NaN NaN NaN NaN 2000-01-02 NaN ... NaN NaN NaN NaN 2000-01-03 NaN ... NaN NaN NaN NaN 2000-01-04 NaN ... NaN NaN NaN NaN 2000-01-05 NaN ... NaN NaN NaN NaN 2000-01-06 NaN ... NaN NaN NaN NaN 2000-01-07 NaN ... NaN NaN NaN NaN 2000-01-08 NaN ... NaN NaN NaN NaN [8 rows x 11 columns]
Warning
df - df['A']
is now deprecated and will be removed in a future release. The preferred way to replicate this behavior is
df.sub(df['A'], axis=0)
For explicit control over the matching and broadcasting behavior, see the section on flexible binary operations.
Operations with scalars are just as you would expect:
In [86]: df * 5 + 2 Out[86]: A B C 2000-01-01 2.3135 1.8579 4.2178 2000-01-02 0.6561 -5.8882 11.2510 2000-01-03 5.1903 -0.7830 1.6438 2000-01-04 -0.5571 2.7814 -3.3781 2000-01-05 10.3180 -0.1886 1.6135 2000-01-06 2.1460 2.8950 10.7003 2000-01-07 -1.6451 -2.4900 0.4289 2000-01-08 1.7596 -2.3780 2.8455 In [87]: 1 / df Out[87]: A B C 2000-01-01 15.9483 -35.1931 2.2545 2000-01-02 -3.7205 -0.6339 0.5405 2000-01-03 1.5672 -1.7966 -14.0388 2000-01-04 -1.9553 6.3984 -0.9297 2000-01-05 0.6011 -2.2845 -12.9363 2000-01-06 34.2568 5.5863 0.5747 2000-01-07 -1.3717 -1.1136 -3.1826 2000-01-08 -20.8019 -1.1421 5.9134 In [88]: df ** 4 Out[88]: A B C 2000-01-01 1.5457e-05 6.5188e-07 3.8707e-02 2000-01-02 5.2191e-03 6.1948e+00 1.1718e+01 2000-01-03 1.6575e-01 9.5982e-02 2.5745e-05 2000-01-04 6.8412e-02 5.9663e-04 1.3386e+00 2000-01-05 7.6595e+00 3.6712e-02 3.5708e-05 2000-01-06 7.2612e-07 1.0268e-03 9.1678e+00 2000-01-07 2.8246e-01 6.5029e-01 9.7473e-03 2000-01-08 5.3406e-06 5.8781e-01 8.1783e-04
Boolean operators work as well:
In [89]: df1 = pd.DataFrame({'a' : [1, 0, 1], 'b' : [0, 1, 1] }, dtype=bool) In [90]: df2 = pd.DataFrame({'a' : [0, 1, 1], 'b' : [1, 1, 0] }, dtype=bool) In [91]: df1 & df2 Out[91]: a b 0 False False 1 False True 2 True False In [92]: df1 | df2 Out[92]: a b 0 True True 1 True True 2 True True In [93]: df1 ^ df2 Out[93]: a b 0 True True 1 True False 2 False True In [94]: -df1 Out[94]: a b 0 False True 1 True False 2 False False
Transposing
To transpose, access the T
attribute (also the transpose
function), similar to an ndarray:
# only show the first 5 rows In [95]: df[:5].T Out[95]: 2000-01-01 2000-01-02 2000-01-03 2000-01-04 2000-01-05 A 0.0627 -0.2688 0.6381 -0.5114 1.6636 B -0.0284 -1.5776 -0.5566 0.1563 -0.4377 C 0.4436 1.8502 -0.0712 -1.0756 -0.0773
DataFrame interoperability with NumPy functions
Elementwise NumPy ufuncs (log, exp, sqrt, ...) and various other NumPy functions can be used with no issues on DataFrame, assuming the data within are numeric:
In [96]: np.exp(df) Out[96]: A B C 2000-01-01 1.0647 0.9720 1.5582 2000-01-02 0.7643 0.2065 6.3611 2000-01-03 1.8928 0.5732 0.9312 2000-01-04 0.5996 1.1692 0.3411 2000-01-05 5.2783 0.6455 0.9256 2000-01-06 1.0296 1.1960 5.6977 2000-01-07 0.4824 0.4074 0.7304 2000-01-08 0.9531 0.4166 1.1842 In [97]: np.asarray(df) Out[97]: array([[ 0.0627, -0.0284, 0.4436], [-0.2688, -1.5776, 1.8502], [ 0.6381, -0.5566, -0.0712], [-0.5114, 0.1563, -1.0756], [ 1.6636, -0.4377, -0.0773], [ 0.0292, 0.179 , 1.7401], [-0.729 , -0.898 , -0.3142], [-0.0481, -0.8756, 0.1691]])
The dot method on DataFrame implements matrix multiplication:
In [98]: df.T.dot(df) Out[98]: A B C A 4.0471 -0.0390 0.1783 B -0.0390 4.6207 -2.5806 C 0.1783 -2.5806 7.9431
Similarly, the dot method on Series implements dot product:
In [99]: s1 = pd.Series(np.arange(5,10)) In [100]: s1.dot(s1) Out[100]: 255
DataFrame is not intended to be a drop-in replacement for ndarray as its indexing semantics are quite different in places from a matrix.
Console display
Very large DataFrames will be truncated to display them in the console. You can also get a summary using info()
. (Here I am reading a CSV version of the baseball dataset from the plyr R package):
In [101]: baseball = pd.read_csv('data/baseball.csv') In [102]: print(baseball) id player year stint ... hbp sh sf gidp 0 88641 womacto01 2006 2 ... 0.0 3.0 0.0 0.0 1 88643 schilcu01 2006 1 ... 0.0 0.0 0.0 0.0 .. ... ... ... ... ... ... ... ... ... 98 89533 aloumo01 2007 1 ... 2.0 0.0 3.0 13.0 99 89534 alomasa02 2007 1 ... 0.0 0.0 0.0 0.0 [100 rows x 23 columns] In [103]: baseball.info() <class 'pandas.core.frame.DataFrame'> RangeIndex: 100 entries, 0 to 99 Data columns (total 23 columns): id 100 non-null int64 player 100 non-null object year 100 non-null int64 stint 100 non-null int64 team 100 non-null object lg 100 non-null object g 100 non-null int64 ab 100 non-null int64 r 100 non-null int64 h 100 non-null int64 X2b 100 non-null int64 X3b 100 non-null int64 hr 100 non-null int64 rbi 100 non-null float64 sb 100 non-null float64 cs 100 non-null float64 bb 100 non-null int64 so 100 non-null float64 ibb 100 non-null float64 hbp 100 non-null float64 sh 100 non-null float64 sf 100 non-null float64 gidp 100 non-null float64 dtypes: float64(9), int64(11), object(3) memory usage: 18.0+ KB
However, using to_string
will return a string representation of the DataFrame in tabular form, though it won’t always fit the console width:
In [104]: print(baseball.iloc[-20:, :12].to_string()) id player year stint team lg g ab r h X2b X3b 80 89474 finlest01 2007 1 COL NL 43 94 9 17 3 0 81 89480 embreal01 2007 1 OAK AL 4 0 0 0 0 0 82 89481 edmonji01 2007 1 SLN NL 117 365 39 92 15 2 83 89482 easleda01 2007 1 NYN NL 76 193 24 54 6 0 84 89489 delgaca01 2007 1 NYN NL 139 538 71 139 30 0 85 89493 cormirh01 2007 1 CIN NL 6 0 0 0 0 0 86 89494 coninje01 2007 2 NYN NL 21 41 2 8 2 0 87 89495 coninje01 2007 1 CIN NL 80 215 23 57 11 1 88 89497 clemero02 2007 1 NYA AL 2 2 0 1 0 0 89 89498 claytro01 2007 2 BOS AL 8 6 1 0 0 0 90 89499 claytro01 2007 1 TOR AL 69 189 23 48 14 0 91 89501 cirilje01 2007 2 ARI NL 28 40 6 8 4 0 92 89502 cirilje01 2007 1 MIN AL 50 153 18 40 9 2 93 89521 bondsba01 2007 1 SFN NL 126 340 75 94 14 0 94 89523 biggicr01 2007 1 HOU NL 141 517 68 130 31 3 95 89525 benitar01 2007 2 FLO NL 34 0 0 0 0 0 96 89526 benitar01 2007 1 SFN NL 19 0 0 0 0 0 97 89530 ausmubr01 2007 1 HOU NL 117 349 38 82 16 3 98 89533 aloumo01 2007 1 NYN NL 87 328 51 112 19 1 99 89534 alomasa02 2007 1 NYN NL 8 22 1 3 1 0
New since 0.10.0, wide DataFrames will now be printed across multiple rows by default:
In [105]: pd.DataFrame(np.random.randn(3, 12)) Out[105]: 0 1 2 3 4 5 6 \ 0 1.225021 -0.528620 0.448676 0.619107 -1.199110 -0.949097 2.169523 1 -1.753617 0.992384 -0.505601 -0.599848 0.133585 0.008836 -1.767710 2 -0.461585 -1.321106 1.745476 1.445100 0.991037 -0.860733 -0.870661 7 8 9 10 11 0 0.302230 0.919516 0.657436 0.262574 -0.804798 1 0.700112 -0.020773 -0.302481 0.347869 0.179123 2 -0.117845 -0.046266 2.095649 -0.524324 -0.610555
You can change how much to print on a single row by setting the display.width
option:
In [106]: pd.set_option('display.width', 40) # default is 80 In [107]: pd.DataFrame(np.random.randn(3, 12)) Out[107]: 0 1 2 \ 0 -1.280951 1.472585 -1.001914 1 0.130529 -1.603771 -0.128830 2 -1.084566 -0.515272 1.367586 3 4 5 \ 0 1.044770 -0.050668 -0.013289 1 -1.869301 -0.232977 -0.139801 2 0.963500 0.224105 -0.020051 6 7 8 \ 0 -0.291893 2.029038 -1.117195 1 -1.083341 -0.357234 -0.818199 2 0.524663 0.351081 -1.574209 9 10 11 0 1.598577 -0.397325 0.151653 1 -0.886885 1.238885 -1.639274 2 -0.486856 -0.545888 -0.927076
You can adjust the max width of the individual columns by setting display.max_colwidth
In [108]: datafile={'filename': ['filename_01','filename_02'], .....: 'path': ["media/user_name/storage/folder_01/filename_01", .....: "media/user_name/storage/folder_02/filename_02"]} .....: In [109]: pd.set_option('display.max_colwidth',30) In [110]: pd.DataFrame(datafile) Out[110]: filename \ 0 filename_01 1 filename_02 path 0 media/user_name/storage/fo... 1 media/user_name/storage/fo... In [111]: pd.set_option('display.max_colwidth',100) In [112]: pd.DataFrame(datafile) Out[112]: filename \ 0 filename_01 1 filename_02 path 0 media/user_name/storage/folder_01/filename_01 1 media/user_name/storage/folder_02/filename_02
You can also disable this feature via the expand_frame_repr
option. This will print the table in one block.
DataFrame column attribute access and IPython completion
If a DataFrame column label is a valid Python variable name, the column can be accessed like attributes:
In [113]: df = pd.DataFrame({'foo1' : np.random.randn(5), .....: 'foo2' : np.random.randn(5)}) .....: In [114]: df Out[114]: foo1 foo2 0 0.909160 1.360298 1 -0.667763 -1.603624 2 -0.101656 -1.648929 3 1.189682 0.145121 4 -0.090648 -2.536359 In [115]: df.foo1 Out[115]: 0 0.909160 1 -0.667763 2 -0.101656 3 1.189682 4 -0.090648 Name: foo1, dtype: float64
The columns are also connected to the IPython completion mechanism so they can be tab-completed:
In [5]: df.fo<TAB> df.foo1 df.foo2
Panel
Panel is a somewhat less-used, but still important container for 3-dimensional data. The term panel data is derived from econometrics and is partially responsible for the name pandas: pan(el)-da(ta)-s. The names for the 3 axes are intended to give some semantic meaning to describing operations involving panel data and, in particular, econometric analysis of panel data. However, for the strict purposes of slicing and dicing a collection of DataFrame objects, you may find the axis names slightly arbitrary:
- items: axis 0, each item corresponds to a DataFrame contained inside
- major_axis: axis 1, it is the index (rows) of each of the DataFrames
- minor_axis: axis 2, it is the columns of each of the DataFrames
Construction of Panels works about like you would expect:
From 3D ndarray with optional axis labels
In [116]: wp = pd.Panel(np.random.randn(2, 5, 4), items=['Item1', 'Item2'], .....: major_axis=pd.date_range('1/1/2000', periods=5), .....: minor_axis=['A', 'B', 'C', 'D']) .....: In [117]: wp Out[117]: <class 'pandas.core.panel.Panel'> Dimensions: 2 (items) x 5 (major_axis) x 4 (minor_axis) Items axis: Item1 to Item2 Major_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 Minor_axis axis: A to D
From dict of DataFrame objects
In [118]: data = {'Item1' : pd.DataFrame(np.random.randn(4, 3)), .....: 'Item2' : pd.DataFrame(np.random.randn(4, 2))} .....: In [119]: pd.Panel(data) Out[119]: <class 'pandas.core.panel.Panel'> Dimensions: 2 (items) x 4 (major_axis) x 3 (minor_axis) Items axis: Item1 to Item2 Major_axis axis: 0 to 3 Minor_axis axis: 0 to 2
Note that the values in the dict need only be convertible to DataFrame. Thus, they can be any of the other valid inputs to DataFrame as per above.
One helpful factory method is Panel.from_dict
, which takes a dictionary of DataFrames as above, and the following named parameters:
Parameter | Default | Description |
---|---|---|
intersect | False | drops elements whose indices do not align |
orient | items | use minor to use DataFrames’ columns as panel items |
For example, compare to the construction above:
In [120]: pd.Panel.from_dict(data, orient='minor') Out[120]: <class 'pandas.core.panel.Panel'> Dimensions: 3 (items) x 4 (major_axis) x 2 (minor_axis) Items axis: 0 to 2 Major_axis axis: 0 to 3 Minor_axis axis: Item1 to Item2
Orient is especially useful for mixed-type DataFrames. If you pass a dict of DataFrame objects with mixed-type columns, all of the data will get upcasted to dtype=object
unless you pass orient='minor'
:
In [121]: df = pd.DataFrame({'a': ['foo', 'bar', 'baz'], .....: 'b': np.random.randn(3)}) .....: In [122]: df Out[122]: a b 0 foo -1.264356 1 bar -0.497629 2 baz 1.789719 In [123]: data = {'item1': df, 'item2': df} In [124]: panel = pd.Panel.from_dict(data, orient='minor') In [125]: panel['a'] Out[125]: item1 item2 0 foo foo 1 bar bar 2 baz baz In [126]: panel['b'] Out[126]: item1 item2 0 -1.264356 -1.264356 1 -0.497629 -0.497629 2 1.789719 1.789719 In [127]: panel['b'].dtypes Out[127]: item1 float64 item2 float64 dtype: object
Note
Unfortunately Panel, being less commonly used than Series and DataFrame, has been slightly neglected feature-wise. A number of methods and options available in DataFrame are not available in Panel. This will get worked on, of course, in future releases. And faster if you join me in working on the codebase.
From DataFrame using to_panel
method
This method was introduced in v0.7 to replace LongPanel.to_long
, and converts a DataFrame with a two-level index to a Panel.
In [128]: midx = pd.MultiIndex(levels=[['one', 'two'], ['x','y']], labels=[[1,1,0,0],[1,0,1,0]]) In [129]: df = pd.DataFrame({'A' : [1, 2, 3, 4], 'B': [5, 6, 7, 8]}, index=midx) In [130]: df.to_panel() Out[130]: <class 'pandas.core.panel.Panel'> Dimensions: 2 (items) x 2 (major_axis) x 2 (minor_axis) Items axis: A to B Major_axis axis: one to two Minor_axis axis: x to y
Item selection / addition / deletion
Similar to DataFrame functioning as a dict of Series, Panel is like a dict of DataFrames:
In [131]: wp['Item1'] Out[131]: A B C D 2000-01-01 0.835993 -0.621868 -0.173710 -0.174326 2000-01-02 -0.354356 2.090183 -0.736019 -1.250412 2000-01-03 -0.581326 -0.244477 0.917119 0.611695 2000-01-04 -1.576078 -0.528562 -0.704643 -0.481453 2000-01-05 1.085093 -1.229749 2.295679 -1.016910 In [132]: wp['Item3'] = wp['Item1'] / wp['Item2']
The API for insertion and deletion is the same as for DataFrame. And as with DataFrame, if the item is a valid python identifier, you can access it as an attribute and tab-complete it in IPython.
Transposing
A Panel can be rearranged using its transpose
method (which does not make a copy by default unless the data are heterogeneous):
In [133]: wp.transpose(2, 0, 1) Out[133]: <class 'pandas.core.panel.Panel'> Dimensions: 4 (items) x 3 (major_axis) x 5 (minor_axis) Items axis: A to D Major_axis axis: Item1 to Item3 Minor_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00
Indexing / Selection
Operation | Syntax | Result |
---|---|---|
Select item | wp[item] | DataFrame |
Get slice at major_axis label | wp.major_xs(val) | DataFrame |
Get slice at minor_axis label | wp.minor_xs(val) | DataFrame |
For example, using the earlier example data, we could do:
In [134]: wp['Item1'] Out[134]: A B C D 2000-01-01 0.835993 -0.621868 -0.173710 -0.174326 2000-01-02 -0.354356 2.090183 -0.736019 -1.250412 2000-01-03 -0.581326 -0.244477 0.917119 0.611695 2000-01-04 -1.576078 -0.528562 -0.704643 -0.481453 2000-01-05 1.085093 -1.229749 2.295679 -1.016910 In [135]: wp.major_xs(wp.major_axis[2]) Out[135]: Item1 Item2 Item3 A -0.581326 -1.271582 0.457167 B -0.244477 -0.861256 0.283861 C 0.917119 -0.597879 -1.533955 D 0.611695 -0.118700 -5.153265 In [136]: wp.minor_axis Out[136]: Index([u'A', u'B', u'C', u'D'], dtype='object') In [137]: wp.minor_xs('C') Out[137]: Item1 Item2 Item3 2000-01-01 -0.173710 2.381645 -0.072937 2000-01-02 -0.736019 -2.413161 0.305002 2000-01-03 0.917119 -0.597879 -1.533955 2000-01-04 -0.704643 -1.536019 0.458746 2000-01-05 2.295679 0.181524 12.646732
Squeezing
Another way to change the dimensionality of an object is to squeeze
a 1-len object, similar to wp['Item1']
In [138]: wp.reindex(items=['Item1']).squeeze() Out[138]: A B C D 2000-01-01 0.835993 -0.621868 -0.173710 -0.174326 2000-01-02 -0.354356 2.090183 -0.736019 -1.250412 2000-01-03 -0.581326 -0.244477 0.917119 0.611695 2000-01-04 -1.576078 -0.528562 -0.704643 -0.481453 2000-01-05 1.085093 -1.229749 2.295679 -1.016910 In [139]: wp.reindex(items=['Item1'], minor=['B']).squeeze() Out[139]: 2000-01-01 -0.621868 2000-01-02 2.090183 2000-01-03 -0.244477 2000-01-04 -0.528562 2000-01-05 -1.229749 Freq: D, Name: B, dtype: float64
Conversion to DataFrame
A Panel can be represented in 2D form as a hierarchically indexed DataFrame. See the section hierarchical indexing for more on this. To convert a Panel to a DataFrame, use the to_frame
method:
In [140]: panel = pd.Panel(np.random.randn(3, 5, 4), items=['one', 'two', 'three'], .....: major_axis=pd.date_range('1/1/2000', periods=5), .....: minor_axis=['a', 'b', 'c', 'd']) .....: In [141]: panel.to_frame() Out[141]: one two three major minor 2000-01-01 a 0.445900 -1.286198 -1.023189 b -0.574496 -0.407154 0.591682 c 0.872979 0.068084 -0.008919 d 0.297255 -2.157051 -0.415572 2000-01-02 a -1.022617 -0.443982 -0.772683 b 1.091870 -0.881639 -0.516197 c 1.831444 0.851834 0.626655 d 1.271808 -1.352515 0.269623 2000-01-03 a -0.472876 0.228761 1.709250 b -0.279340 0.416858 -0.830728 c 0.495966 0.301709 -0.290244 d 0.367858 0.569010 -1.588782 2000-01-04 a -1.530917 -0.047619 0.639406 b -0.285890 0.413370 1.055533 c 0.943062 0.573056 -0.260898 d 1.361752 -0.154419 -0.289725 2000-01-05 a 0.210373 0.987044 0.279621 b -1.945608 0.063191 0.454423 c 2.532409 0.439086 -0.065750 d 0.373819 1.657475 1.465709
Panel4D (Experimental)
Panel4D
is a 4-Dimensional named container very much like a Panel
, but having 4 named dimensions. It is intended as a test bed for more N-Dimensional named containers.
- labels: axis 0, each item corresponds to a Panel contained inside
- items: axis 1, each item corresponds to a DataFrame contained inside
- major_axis: axis 2, it is the index (rows) of each of the DataFrames
- minor_axis: axis 3, it is the columns of each of the DataFrames
Panel4D
is a sub-class of Panel
, so most methods that work on Panels are applicable to Panel4D. The following methods are disabled:
join , to_frame , to_excel , to_sparse , groupby
Construction of Panel4D works in a very similar manner to a Panel
From 4D ndarray with optional axis labels
In [142]: p4d = pd.Panel4D(np.random.randn(2, 2, 5, 4), .....: labels=['Label1','Label2'], .....: items=['Item1', 'Item2'], .....: major_axis=pd.date_range('1/1/2000', periods=5), .....: minor_axis=['A', 'B', 'C', 'D']) .....: In [143]: p4d Out[143]: <class 'pandas.core.panelnd.Panel4D'> Dimensions: 2 (labels) x 2 (items) x 5 (major_axis) x 4 (minor_axis) Labels axis: Label1 to Label2 Items axis: Item1 to Item2 Major_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 Minor_axis axis: A to D
From dict of Panel objects
In [144]: data = { 'Label1' : pd.Panel({ 'Item1' : pd.DataFrame(np.random.randn(4, 3)) }), .....: 'Label2' : pd.Panel({ 'Item2' : pd.DataFrame(np.random.randn(4, 2)) }) } .....: In [145]: pd.Panel4D(data) Out[145]: <class 'pandas.core.panelnd.Panel4D'> Dimensions: 2 (labels) x 2 (items) x 4 (major_axis) x 3 (minor_axis) Labels axis: Label1 to Label2 Items axis: Item1 to Item2 Major_axis axis: 0 to 3 Minor_axis axis: 0 to 2
Note that the values in the dict need only be convertible to Panels. Thus, they can be any of the other valid inputs to Panel as per above.
Slicing
Slicing works in a similar manner to a Panel. []
slices the first dimension. .ix
allows you to slice arbitrarily and get back lower dimensional objects
In [146]: p4d['Label1'] Out[146]: <class 'pandas.core.panel.Panel'> Dimensions: 2 (items) x 5 (major_axis) x 4 (minor_axis) Items axis: Item1 to Item2 Major_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 Minor_axis axis: A to D
4D -> Panel
In [147]: p4d.ix[:,:,:,'A'] Out[147]: <class 'pandas.core.panel.Panel'> Dimensions: 2 (items) x 2 (major_axis) x 5 (minor_axis) Items axis: Label1 to Label2 Major_axis axis: Item1 to Item2 Minor_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00
4D -> DataFrame
In [148]: p4d.ix[:,:,0,'A'] Out[148]: Label1 Label2 Item1 1.127489 0.015494 Item2 -1.650400 0.130533
4D -> Series
In [149]: p4d.ix[:,0,0,'A'] Out[149]: Label1 1.127489 Label2 0.015494 Name: A, dtype: float64
Transposing
A Panel4D can be rearranged using its transpose
method (which does not make a copy by default unless the data are heterogeneous):
In [150]: p4d.transpose(3, 2, 1, 0) Out[150]: <class 'pandas.core.panelnd.Panel4D'> Dimensions: 4 (labels) x 5 (items) x 2 (major_axis) x 2 (minor_axis) Labels axis: A to D Items axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 Major_axis axis: Item1 to Item2 Minor_axis axis: Label1 to Label2
PanelND (Experimental)
PanelND is a module with a set of factory functions to enable a user to construct N-dimensional named containers like Panel4D, with a custom set of axis labels. Thus a domain-specific container can easily be created.
The following creates a Panel5D. A new panel type object must be sliceable into a lower dimensional object. Here we slice to a Panel4D.
In [151]: from pandas.core import panelnd In [152]: Panel5D = panelnd.create_nd_panel_factory( .....: klass_name = 'Panel5D', .....: orders = [ 'cool', 'labels','items','major_axis','minor_axis'], .....: slices = { 'labels' : 'labels', 'items' : 'items', .....: 'major_axis' : 'major_axis', 'minor_axis' : 'minor_axis' }, .....: slicer = pd.Panel4D, .....: aliases = { 'major' : 'major_axis', 'minor' : 'minor_axis' }, .....: stat_axis = 2) .....: In [153]: p5d = Panel5D(dict(C1 = p4d)) In [154]: p5d Out[154]: <class 'pandas.core.panelnd.Panel5D'> Dimensions: 1 (cool) x 2 (labels) x 2 (items) x 5 (major_axis) x 4 (minor_axis) Cool axis: C1 to C1 Labels axis: Label1 to Label2 Items axis: Item1 to Item2 Major_axis axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 Minor_axis axis: A to D # print a slice of our 5D In [155]: p5d.ix['C1',:,:,0:3,:] Out[155]: <class 'pandas.core.panelnd.Panel4D'> Dimensions: 2 (labels) x 2 (items) x 3 (major_axis) x 4 (minor_axis) Labels axis: Label1 to Label2 Items axis: Item1 to Item2 Major_axis axis: 2000-01-01 00:00:00 to 2000-01-03 00:00:00 Minor_axis axis: A to D # transpose it In [156]: p5d.transpose(1,2,3,4,0) Out[156]: <class 'pandas.core.panelnd.Panel5D'> Dimensions: 2 (cool) x 2 (labels) x 5 (items) x 4 (major_axis) x 1 (minor_axis) Cool axis: Label1 to Label2 Labels axis: Item1 to Item2 Items axis: 2000-01-01 00:00:00 to 2000-01-05 00:00:00 Major_axis axis: A to D Minor_axis axis: C1 to C1 # look at the shape & dim In [157]: p5d.shape Out[157]: (1, 2, 2, 5, 4) In [158]: p5d.ndim Out[158]: 5
© 2008–2012, AQR Capital Management, LLC, Lambda Foundry, Inc. and PyData Development Team
Licensed under the 3-clause BSD License.
https://pandas.pydata.org/pandas-docs/version/0.18.1/dsintro.html