Skip to content

Support merging DataFrames on a combo of columns and index levels (GH 14355) #17484

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Dec 1, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
da94fdb
Support merging frames on a combo of columns and index levels (GH 14355)
Jul 19, 2017
f8c8c53
Cleanup for review
Sep 10, 2017
368844a
revert implementation (but keep documentation and tests)
Sep 11, 2017
1c4699e
Simplify and refactor column/level logic in merge
Sep 11, 2017
ac1189b
PEP8 cleanup
Sep 11, 2017
d90ed78
Extract column/level ambiguity warning logic into utility method
Sep 11, 2017
27b2d25
Add newline and add :ref: entry for new doc section
Sep 11, 2017
de6f4b1
docstring / comment cleanup
Sep 11, 2017
39d0bba
Merge branch 'master' into enh_14355
Oct 2, 2017
5b1b100
Documentation updates
Oct 9, 2017
dfc6cf7
Fix errors in _drop_columns_or_levels
Oct 9, 2017
03e3c2e
Refactor and parametrize test cases
Oct 9, 2017
bf5d349
Moved label/level helpers up to NDFrame, added axis support, and adde…
Oct 10, 2017
7da39aa
PEP8
Oct 11, 2017
f5a16ff
Revert accidental change to merging.rst
Oct 12, 2017
aa099ea
Use fixtures for new TestMergeColumnAndIndex tests
Oct 13, 2017
3be43a4
Merge branch 'master' into enh_14355
Oct 13, 2017
b655e30
Merge branch 'master' into enh_14355
Oct 20, 2017
b7e2cc2
Merge branch 'master' into enh_14355
Nov 1, 2017
e9f02b1
Update documentation for a 0.22 release
Nov 1, 2017
0cd4ef5
Merge remote-tracking branch 'upstream/master' into enh_14355
Nov 2, 2017
e029f7b
Documentation updates
Nov 6, 2017
fdddbd3
Moved test_label_or_level_utils to pandas/tests/generic
Nov 6, 2017
89061b9
Refactored level_or_level test cases to use fixtures
Nov 6, 2017
090b3e8
Moved label_or_level utils on Series and DataFrame to NDFrame
Nov 6, 2017
47ff8b8
fix test comment typo
Nov 6, 2017
59f2dce
PEP8ify
Nov 6, 2017
4c4dbd0
Merge remote-tracking branch 'upstream/master' into enh_14355
Nov 6, 2017
1d7e570
Moved column and index tests to new file
Nov 6, 2017
dd289a6
Remove test class and convert to using fixtures
Nov 6, 2017
313d2c3
Rename new test file
Nov 6, 2017
0b0397b
Documentation and testing review updates
Nov 7, 2017
bc53bef
Merge remote-tracking branch 'upstream/master' into enh_14355
Nov 7, 2017
cd17c42
Merge remote-tracking branch 'upstream/master' into enh_14355
Nov 23, 2017
1a4e3e4
Merge remote-tracking branch 'upstream/master' into enh_14355
Nov 23, 2017
a49012c
Fix generator/list lint issues
Nov 23, 2017
6fd9760
Allow non-None hashable objects to reference index levels (not just s…
Nov 26, 2017
f7e04f5
Reduce parameterized test case count by removing how fixture
Nov 26, 2017
cf8e654
Refactor warning code and add stacklevel
Nov 26, 2017
e874f04
Use single backticks to reference method params in docstrings
Nov 26, 2017
13ce87c
Add tests and docstring updates for using index levels as `on` param …
Nov 26, 2017
b5cb4c1
PEP8
Nov 26, 2017
f3b95fe
Fixed Note->Notes in docstring
Dec 1, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'master' into enh_14355
  • Loading branch information
Jon M. Mease committed Oct 2, 2017
commit 39d0bba2fd8e556cdb03c42c4b1a80c8484d7fff
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.21.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,7 @@ Other Enhancements
- :func:`date_range` now accepts 'Y' in addition to 'A' as an alias for end of year (:issue:`9313`)
- Integration with `Apache Parquet <https://parquet.apache.org/>`__, including a new top-level :func:`read_parquet` and :func:`DataFrame.to_parquet` method, see :ref:`here <io.parquet>`. (:issue:`15838`, :issue:`17438`)
- :func:`DataFrame.add_prefix` and :func:`DataFrame.add_suffix` now accept strings containing the '%' character. (:issue:`17151`)
- Read/write methods that infer compression (:func:`read_csv`, :func:`read_table`, :func:`read_pickle`, and :meth:`~DataFrame.to_pickle`) can now infer from non-string paths, such as ``pathlib.Path`` objects (:issue:`17206`).
- :func:`DataFrame.merge` now accepts index level names as `on`, `left_on`, and `right_on` parameters, allowing frames to be merged on a combination of columns and index levels (:issue:`14355`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need a sub-section for this. its a pretty big change. give the example from the whatsnew.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a new subsection. Should I remove this one?

- `read_*` methods can now infer compression from non-string paths, such as ``pathlib.Path`` objects (:issue:`17206`).
- :func:`pd.read_sas()` now recognizes much more of the most frequently used date (datetime) formats in SAS7BDAT files (:issue:`15871`).
Expand Down
3 changes: 2 additions & 1 deletion pandas/core/groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -2776,7 +2776,8 @@ def is_in_obj(gpr):

elif is_in_axis(gpr): # df.groupby('name')
if gpr in obj:
obj._check_column_or_level_ambiguity(gpr)
if validate:
obj._check_column_or_level_ambiguity(gpr)
in_axis, name, gpr = True, gpr, obj[gpr]
exclusions.append(name)
elif obj._is_index_reference(gpr):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should fail on a Series the way it works currently

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing this out @jreback. I'm still working on a full set of updates for this PR but in the meantime, I added a test case to #17843 that would have caught this.

Expand Down
You are viewing a condensed version of this merge commit. You can view the full changes here.