BUG/MAINT: Change default of inplace to False in pd.eval #16732

gfyoung · 2017-06-20T04:30:41Z

Deprecated back in 0.18.0.

Also patches bug where we were improperly handling the inplace=False condition, as we were assuming that target input was non-None when that wasn't necessarily enforced.

gfyoung · 2017-06-20T04:59:32Z

Apparently, >200 (!) of our tests didn't take heed of the warning to pass in inplace=True

gfyoung · 2017-06-20T05:11:28Z

Also, I must say that the docstring for inplace is not very intuitive IMO. I would have expected that inplace=False would return the result of the computation and not the other way around.

@jreback @jorisvandenbossche : Thoughts?

jorisvandenbossche

Adding an example to the docstring might clarify a lot what this function actually does

jorisvandenbossche · 2017-06-20T07:16:16Z

pandas/core/computation/eval.py

-        in a future version, will default to False.  Use inplace=True
-        explicitly rather than relying on the default.
+    inplace : bool, default False
+        If the expression mutates, whether to modify the expression inplace


I think adding an example might clarify it further eg: "If the expression mutates (e.g. creates a new column), ..."

jorisvandenbossche · 2017-06-20T07:17:35Z

pandas/core/computation/eval.py

-        in a future version, will default to False.  Use inplace=True
-        explicitly rather than relying on the default.
+    inplace : bool, default False
+        If the expression mutates, whether to modify the expression inplace


You changed object to expression in "whether to modify object inplace", but I think object is actually better.

Looking at the test example:

In [87]: df = DataFrame(np.random.randn(5, 2), columns=list('ab')) In [88]: df.eval('c = a + b', inplace=False) Out[88]: a b c 0 -0.742101 2.084071 1.341970 1 -0.823597 -1.686791 -2.510388 2 0.277927 -0.076424 0.201503 3 1.159483 0.188500 1.347983 4 0.010328 -0.890899 -0.880571 In [89]: df Out[89]: a b 0 -0.742101 2.084071 1 -0.823597 -1.686791 2 0.277927 -0.076424 3 1.159483 0.188500 4 0.010328 -0.890899

it is the df object that does or not get modified, not the "expression"

jorisvandenbossche · 2017-06-20T07:19:00Z

pandas/core/computation/eval.py

+    inplace : bool, default False
+        If the expression mutates, whether to modify the expression inplace
+        or return a copy of it with mutation. In other words, it will return
+        the result of the expression if `inplace=True` and `target` otherwise.


"and target otherwise." is not fully correct. It can also mutate the calling object.

WIth the same example:

In [89]: df Out[89]: a b 0 -0.742101 2.084071 1 -0.823597 -1.686791 2 0.277927 -0.076424 3 1.159483 0.188500 4 0.010328 -0.890899 In [90]: df.eval('c = a + b', inplace=True) In [91]: df Out[91]: a b c 0 -0.742101 2.084071 1.341970 1 -0.823597 -1.686791 -2.510388 2 0.277927 -0.076424 0.201503 3 1.159483 0.188500 1.347983 4 0.010328 -0.890899 -0.880571

jorisvandenbossche · 2017-06-20T07:23:12Z

Whoops, sorry, all my comments were in light of me thinking it was about DataFrame.eval method, not top-level pandas.eval.

Anyhow, the frame method also has to be updated, and there the comments will be sensible :-) (edit: but actually the docstring there is already much clearer!)

gfyoung · 2017-06-20T07:47:33Z

@jorisvandenbossche : Actually ignore my previous comment. That doesn't make much sense to me at all still. The documentation says "whether to return a new DataFrame or mutate the existing." Which condition means return a new DataFrame?

If you say it's inplace=False, then why are we returning target in pd.eval (and not ret)? Also, why are we returning anything when inplace=True in pd.eval?

IMO, it should be that ret is returned when inplace=False, and nothing is returned when inplace=True because target should be modified. In fact, I might go as far as to say that inplace=True only when target is not a primitive (and raise otherwise). Otherwise, not really sure what this means here:

>>> import pandas as pd
>>> c = pd.eval('1 + 2', inplace=True)
>>> c
3
>>> c = pd.eval('1 + 2', inplace=False)
>>> c
>>>

jorisvandenbossche · 2017-06-20T08:14:10Z

cc @chris-b1

chris-b1 · 2017-06-20T13:21:43Z

My thought is that inplace should only be part of the api for DataFrame.eval - there it works as expected, and consistent with other uses of inplace. For pd.eval it is simply an implementation detail (since DataFrame.eval calls it) - I should not have added it to the docstring.

In fact, pd.eval should probably raise if inplace is passed in (so keep the default = None) without a target.

gfyoung · 2017-06-20T15:34:22Z

@chris-b1 : I think we are mostly aligned in our thought process, and I think (if nobody objects) that we should address the issue now in this PR instead of waiting.

I don't think adding inplace to the doc-string was a bad idea. It's just that I don't understand why it behaves so counter-intuitively to what I would think inplace does in pd.eval. Why is it that target is returned when inplace=False? It should be the other way around.

I agree that we should only allow inplace=True when target is not a primitive. Furthermore, we should be returning anything when inplace=True because we assume that target itself has been modified.

What do you think?

jorisvandenbossche · 2017-06-20T16:07:00Z

The question we should ask: what is the usecase of inplace? (is there actually a usecase?)
I don't see one, so I agree with @chris-b1 we better remove it from the docstring / document it as no effect.

Possible usecases:

expression without assignment:
- this should always return something, as there is no object to return to (passing a target also, logically, does not do anything with the target). So inplace makes no sense in this case.
- The reason that inplace=False actually suppresses the return value is just a bug in the current implementation. The code does:
```
if not inplace and inplace is not None:
    return target

return ret
```
  which fails for inplace=False without a target.
expression with assignment:
- for this case, you are obliged to pass a target, so why would you not want to modify this target?
- That's my first thought, but OK, here might indeed be a possible usecase: eg you want to add a column to an existing dataframe, but still want to take a copy of it first, and then return the modified result instead of modifying inplace. Similar as we do with eg assign

So actions I would take:

Fix the bug with inplace in case of no assignment. And I would even additionally raise an error if inplace is specified in such a case
Update the docstring to make it clear that inplace is only relevant for the case where a target is passed.

gfyoung · 2017-06-20T17:21:45Z

@jorisvandenbossche : Well that touches upon #16529, but in any case, it can be used when target is a DataFrame object, so I wouldn't say that it has no use case at this point.

I agree that we should update the docstring to say that inplace has no meaning when target isn't a non-primitive object (passing in target=1 also makes no sense).

Also, the fix for that bug should be the following:

<if target is None or is some primitive>:
   inplace = False
...
if not inplace:
   return ret

There should be no condition for returning anything if inplace=True because we assume that target itself is modified inplace.

How does that sound? @chris-b1 thoughts?

chris-b1 · 2017-06-20T23:57:09Z

I would suggest

DataFrame.eval, change inplace default to False
pd.eval
- keep default for inplace as None
- if inplace kw is passed with no target, raise

gfyoung · 2017-06-21T00:21:59Z

@chris-b1 : I agree with inplace=False, but I don't agree with your suggestion about inplace=None. The only time I would consider raising is when inplace=True (inplace=False shouldn't care about target) and target is something that you can't modify in-place. Also, I think we should default inplace=False to pd.eval, as that is consistent with what we have for other inplace defaults.

codecov · 2017-06-21T06:43:07Z

Codecov Report

Merging #16732 into master will increase coverage by 0.02%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #16732      +/-   ##
==========================================
+ Coverage   90.94%   90.97%   +0.02%     
==========================================
  Files         161      161              
  Lines       49272    49283      +11     
==========================================
+ Hits        44811    44834      +23     
+ Misses       4461     4449      -12

Flag	Coverage Δ
#multiple	`88.74% <100%> (+0.02%)`	⬆️
#single	`40.16% <5%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/computation/eval.py	`96.93% <100%> (+2.37%)`	⬆️
pandas/core/frame.py	`97.71% <100%> (+0.04%)`	⬆️
pandas/io/common.py	`67.79% <0%> (-2.12%)`	⬇️
pandas/core/indexes/base.py	`95.8% <0%> (-0.01%)`	⬇️
pandas/core/window.py	`96.49% <0%> (ø)`	⬆️
pandas/core/groupby.py	`92.07% <0%> (ø)`	⬆️
pandas/core/indexes/multi.py	`96.65% <0%> (ø)`	⬆️
pandas/core/dtypes/common.py	`94.79% <0%> (+0.04%)`	⬆️
pandas/core/indexes/timedeltas.py	`90.58% <0%> (+0.19%)`	⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 329fdaa...c6f5bcd. Read the comment docs.

jorisvandenbossche · 2017-06-21T07:21:23Z

pandas/core/computation/eval.py

+    # "Primitives" cannot be modified inplace because
+    # they have no attributes that we can modify.
+    #
+    # The only exceptions are lists and dictionaries,


I don't think it is possible to modify lists -> __getitem__ is used, and it is a string expression, so you can never work with a list

Or do you have an example where this actually works?

(anyway, it would also be good to add some tests for those statements, also for the dictionaries)

Ah, I see the diff I was looking at is already outdated :-)

@jorisvandenbossche : This is very much WIP. I'm trying to test that the changes I make actually don't break anything, let alone > 200 tests like last time. 😄

Once I have the implementation in a way that works, I will add the tests (I have them in a separate file that I'm running to test before pushing).

jorisvandenbossche · 2017-06-21T07:33:21Z

pandas/core/computation/eval.py

+    modifiable = hasattr(target, "__setitem__")
+
+    if inplace and not modifiable:
+        raise ValueError("Cannot modify the provided target inplace")


Another possibility is to just raise depending on parsed_expr.assigner. If there is no assignment in the expression, raise when inplace is set to True

Because now you could pass a target, and then set inplace=True (then this error will not be triggered), while this makes no sense when there is no assignment

Because now you could pass a target, and then set inplace=True

What do you mean? How can you do those two things sequentially?

Just using both as keyword arguments: pd.eval(expr, target=df, inplace=True)

that only makes sense when expr contains an assignment, and that you check with parsed_expr.assigner

That's fair. I think I'll just put a try-except around the item assignment so that we don't have to do all of these checks along the way, as the check itself is not robust to things like list.

For me the try excepts are not needed. When trying to set an object that does not support assigment with strings like a list, you already get a clear error message (and this is really a corner case, people should actually only pass dataframes)

For me the try excepts are not needed.

I think a uniform error message is needed. Passing in np.ndarray provides an error message that is not immediately clear regarding item assignment.

people should actually only pass dataframes

Well then perhaps pd.eval should not exist 😄 because as is, there's nothing wrong with me passing in a dictionary or some other dict-like object.

gfyoung · 2017-06-21T21:47:17Z

@jreback @jorisvandenbossche : Could you run the failing build on Travis? It doesn't seem related to my changes at all whatsoever.

gfyoung · 2017-06-22T08:17:51Z

Thanks to whoever reran the build! Everything is green now and ready for review.

chris-b1 · 2017-06-22T13:32:50Z

pandas/core/computation/eval.py

 def eval(expr, parser='pandas', engine=None, truediv=True,
         local_dict=None, global_dict=None, resolvers=(), level=0,
-         target=None, inplace=None):
+         target=None, inplace=False):


Defer to @jreback, @jorisvandenbossche, but I am still of the opinion that API would be cleaner if inplace was only allowed with a target and raised otherwise. pd.eval('1 + 2', inplace=True) == None is surprising.

@chris-b1 : True, but "with a target" is not well-defined either. As you can see by my documentation change on target in pandas.eval, you first need to ensure that item assignment with strings is supported, but how do you check that without modifying the object?

That's something I imagined would create unnecessary overhead. Also, if you go down this path, you should then check if there is item assignment in the first place if you want inplace=True to not surprise anyone.

That's why I decided to allow the user to pass in inplace=True even if there is no real target. I will in fact document that behavior just in case.

As I said before, I don't think we need to check the modifyability of target. Leave it to the user.

Yeah, sure, I have no problem leaving it as is.

jreback · 2017-06-22T14:39:37Z

I don't really recall why there is a target object for assignment here at all. maybe @cpcloud remembers.

This seems odd to have target AND inplace. Since inplace is our standard, would deprecate target, default inplace=False

jorisvandenbossche · 2017-06-23T07:53:32Z

pandas/core/frame.py

-            explicitly rather than relying on the default.
+        inplace : bool, default False
+            If the expression contains an assignment, whether to perform the
+            operation inplace and mutate the existing DataFrame. Otherwise,


Maybe make the "Otherwise, " -> "By default, " ? (to stress more that returning a new dataframe is the default) Or could also append something to the sentence.

I'm not sure why we need to emphasize this. It's pretty clear that the default is False from the declaration of inplace on line 2235.

jorisvandenbossche · 2017-06-23T07:56:55Z

pandas/tests/computation/test_eval.py

+        expression = "a = 1 + 2"
+
+        with tm.assert_raises_regex(ValueError, msg):
+            self.eval(expression, target=invalid_target, inplace=False)


It is a bit a pity that the copy check has to come first, because you now get a not super informative message when passing an invalid target (about not having copy, while you just can't pass it).

(which would be the same if we wouldn't catch those, I know :-))

True, but I can also pass in objects that are constructed to support item assignment but fail on copy. It seems reasonable to put a custom error message for that case.

jorisvandenbossche · 2017-06-23T08:06:04Z

pandas/core/computation/eval.py

+    ------
+    ValueError : There are many instances where such an error can be raised:
+        - An invalid target is provided, and
+          the expression is multiline.


Can you put this on one line? (I think it fits)

Further, why the "and the expression is multiline" ?

Meant to go for the condition of multi_line and target=None

jorisvandenbossche · 2017-06-23T08:09:33Z

pandas/core/computation/eval.py


+    Raises
+    ------
+    ValueError : There are many instances where such an error can be raised:


Can you put the "There are many instances where such an error can be raised:" also on the next line? (and then the colon : is not needed)

gfyoung · 2017-06-23T19:06:09Z

@jreback @jorisvandenbossche @chris-b1 : PTAL

gfyoung · 2017-06-29T03:23:23Z

@jreback @jorisvandenbossche @chris-b1 : Any updates or thoughts on this?

jreback

just some minor comments.

jreback · 2017-06-30T06:55:48Z

doc/source/whatsnew/v0.21.0.txt

 ^^^^^^^^^^^^^^^^^

 - Moved definition of ``MergeError`` to the ``pandas.errors`` module.
+- :func:`eval` will now raise a ``ValueError`` in cases where we cannot operate properly on the ``target`` parameter, or when an inplace operation is specified but there is no item assignment in the expression (:issue:`16732`)


this is a little obtuse maybe try to be slightly more verbose

I think it makes sense to have a couple of examples in a sub-section on what is happening. Also feel free to expand the Examples section of the doc-string.

I tried to clarify the wording a bit. Done.

jreback · 2017-06-30T20:51:24Z

pandas/core/computation/eval.py

    level : int, optional
        The number of prior stack frames to traverse and add to the current
        scope. Most users will **not** need to change this parameter.
    target : a target object for assignment, optional, default is None


this doc-string is not numpy doc (IOW the expl should be on the next line), specify what the target object can be (DataFrame?)

It really can be anything that supports string item-assignment, per the docstring I put.

jreback · 2017-06-30T20:51:57Z

pandas/core/computation/eval.py

-        WARNING: inplace=None currently falls back to to True, but
-        in a future version, will default to False.  Use inplace=True
-        explicitly rather than relying on the default.
+        This is essentially passed into the resolver and is used when there


rm essentially.

Makes, sense. Done.

jreback · 2017-06-30T20:52:26Z

pandas/core/computation/eval.py

+        There are many instances where such an error can be raised:
+
+        - `target=None`, but the expression is multiline.
+        - The expression is multiline, but not all them have item assignment.


expand what this means (for 2nd item)

Sure, done.

jreback · 2017-06-30T20:53:05Z

pandas/core/computation/eval.py

+                try:
+                    target = env.target.copy()
+                except AttributeError:
+                    raise ValueError("Cannot return a copy of the target")


do we have a test for this?

Absolutely. There is an entire test devoted to this!

gfyoung · 2017-07-01T10:35:46Z

@jreback @jorisvandenbossche @chris-b1 : Addressed all comments. PTAL.

jreback · 2017-07-02T17:43:26Z

doc/source/whatsnew/v0.21.0.txt

 ^^^^^^^^^^^^^^^^^

 - Moved definition of ``MergeError`` to the ``pandas.errors`` module.
+- :func:`eval` will now raise a ``ValueError`` when item assignment malfunctions, or inplace operations are specified, but there is no item assignment in the expression (:issue:`16732`)


can you add a small sub-section with an example of what is changing

Added it to the breaking section, since technically there is a breaking change, as passing in inplace=True without item assignment is disallowed now.

nice can u make title shorter

gfyoung · 2017-07-03T01:12:06Z

@jreback @jorisvandenbossche @chris-b1 : Addressed comment. PTAL.

jreback · 2017-07-03T04:56:39Z

doc/source/whatsnew/v0.21.0.txt

+change, the error message is now this:
+
+.. ipython :: python
+


i think u need an okexception or something here
(or a code block)

Used code block.

jreback · 2017-07-03T04:56:56Z

doc/source/whatsnew/v0.21.0.txt

+
+.. ipython :: python
+
+   pd.eval("1 + 2", target=arr, inplace=True)


jreback · 2017-07-03T04:57:28Z

doc/source/whatsnew/v0.21.0.txt

 ^^^^^^^^^^^^^^^^^

 - Moved definition of ``MergeError`` to the ``pandas.errors`` module.
+- :func:`eval` will now raise a ``ValueError`` when item assignment malfunctions, or inplace operations are specified, but there is no item assignment in the expression (:issue:`16732`)


nice can u make title shorter

Deprecated in 0.18.0. xref pandas-devgh-11149. Also patches bug where we were improperly handling the inplace=False condition, as we were assuming that target input was non-None when that wasn't necessarily enforced.

gfyoung · 2017-07-03T07:24:58Z

@jreback @jorisvandenbossche @chris-b1 : Addressed comments. PTAL.

gfyoung · 2017-07-03T07:26:16Z

Also, just a reminder: xref #6581 (deprecation log) and #11149 (PR with deprecation)

gfyoung · 2017-07-05T23:12:21Z

@jreback @jorisvandenbossche @chris-b1 : Any updates or thoughts on this?

jreback

some tiny doc comments. ping when pushed.

jreback · 2017-07-06T10:12:53Z

doc/source/whatsnew/v0.21.0.txt

+  IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`)
+  and integer or boolean arrays are valid indices
+
+This is a very long way of saying numpy arrays don't support string-item indexing. With this


don't -> doesn't

It's plural ("arrays"), so "don't" is the correct conjugation there.

jreback · 2017-07-06T10:13:23Z

doc/source/whatsnew/v0.21.0.txt

+  Out[4]: 3
+
+However, this input does not make much sense because the output is not being assigned to
+the target. Now, a `ValueError` will be raised when such an input is passed in:


double-backticks

jreback · 2017-07-06T22:46:04Z

thanks @gfyoung

gfyoung force-pushed the eval-inplace-false branch from 5b38b03 to bf6fbe5 Compare June 20, 2017 04:59

gfyoung force-pushed the eval-inplace-false branch from bf6fbe5 to b7cee03 Compare June 20, 2017 05:14

jorisvandenbossche reviewed Jun 20, 2017

View reviewed changes

gfyoung force-pushed the eval-inplace-false branch from b7cee03 to d52e95d Compare June 20, 2017 07:59

gfyoung force-pushed the eval-inplace-false branch from d52e95d to a59bee6 Compare June 21, 2017 05:12

gfyoung changed the title ~~MAINT: Change default of inplace to False in pd.eval~~ BUG/MAINT: Change default of inplace to False in pd.eval Jun 21, 2017

gfyoung force-pushed the eval-inplace-false branch from a59bee6 to 5bfa90b Compare June 21, 2017 06:09

gfyoung force-pushed the eval-inplace-false branch from 5bfa90b to d1004eb Compare June 21, 2017 07:25

jorisvandenbossche reviewed Jun 21, 2017

View reviewed changes

gfyoung force-pushed the eval-inplace-false branch 2 times, most recently from ccb2b58 to 4874b13 Compare June 21, 2017 15:24

chris-b1 reviewed Jun 22, 2017

View reviewed changes

jreback added API Design Deprecate Functionality to remove in pandas labels Jun 22, 2017

gfyoung force-pushed the eval-inplace-false branch 3 times, most recently from 56ff112 to 3e6cb8c Compare June 23, 2017 04:44

jorisvandenbossche reviewed Jun 23, 2017

View reviewed changes

gfyoung force-pushed the eval-inplace-false branch from 3e6cb8c to 1c5619f Compare June 23, 2017 08:30

gfyoung force-pushed the eval-inplace-false branch from 1c5619f to 7144144 Compare June 24, 2017 16:00

chris-b1 approved these changes Jun 29, 2017

View reviewed changes

TomAugspurger added this to the 0.21.0 milestone Jun 30, 2017

jreback approved these changes Jun 30, 2017

View reviewed changes

gfyoung force-pushed the eval-inplace-false branch from 7144144 to b6bdad3 Compare July 1, 2017 08:37

jreback reviewed Jul 2, 2017

View reviewed changes

gfyoung force-pushed the eval-inplace-false branch from b6bdad3 to 2eaa714 Compare July 2, 2017 19:28

jreback reviewed Jul 3, 2017

View reviewed changes

MAINT/BUG: Default inplace to False in pd.eval

2a09efb

Deprecated in 0.18.0. xref pandas-devgh-11149. Also patches bug where we were improperly handling the inplace=False condition, as we were assuming that target input was non-None when that wasn't necessarily enforced.

gfyoung force-pushed the eval-inplace-false branch from 2eaa714 to 2a09efb Compare July 3, 2017 05:25

jreback mentioned this pull request Jul 6, 2017

DEPR: deprecations log for removed issues #13777

Closed

jreback approved these changes Jul 6, 2017

View reviewed changes

DOC: Double-tick ValueError in whatsnew

c6f5bcd

jreback merged commit 8d197ba into pandas-dev:master Jul 6, 2017

gfyoung deleted the eval-inplace-false branch July 7, 2017 02:26

simonjayhawkins mentioned this pull request Apr 27, 2020

DOC/CLN: remove outdated warnings in enhancingperf.rst #33831

Merged


		.. ipython :: python

		pd.eval("1 + 2", target=arr, inplace=True)

Uh oh!

BUG/MAINT: Change default of inplace to False in pd.eval #16732

BUG/MAINT: Change default of inplace to False in pd.eval #16732

Uh oh!

Conversation

gfyoung commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gfyoung commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gfyoung commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorisvandenbossche left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jorisvandenbossche commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gfyoung commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jorisvandenbossche commented Jun 20, 2017

Uh oh!

chris-b1 commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gfyoung commented Jun 20, 2017

Uh oh!

jorisvandenbossche commented Jun 20, 2017

Uh oh!

gfyoung commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chris-b1 commented Jun 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gfyoung commented Jun 21, 2017

Uh oh!

codecov bot commented Jun 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gfyoung Jun 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gfyoung Jun 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gfyoung Jun 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gfyoung commented Jun 21, 2017

Uh oh!

gfyoung commented Jun 20, 2017 •

edited

Loading

gfyoung commented Jun 20, 2017 •

edited

Loading

gfyoung commented Jun 20, 2017 •

edited

Loading

jorisvandenbossche commented Jun 20, 2017 •

edited

Loading

gfyoung commented Jun 20, 2017 •

edited

Loading

chris-b1 commented Jun 20, 2017 •

edited

Loading

gfyoung commented Jun 20, 2017 •

edited

Loading

chris-b1 commented Jun 20, 2017 •

edited

Loading

codecov bot commented Jun 21, 2017 •

edited

Loading

gfyoung Jun 21, 2017 •

edited

Loading

gfyoung Jun 21, 2017 •

edited

Loading

gfyoung Jun 21, 2017 •

edited

Loading

gfyoung Jun 22, 2017 •

edited

Loading