Repository navigation
BUG: groupby with as_index=False drops groupings when names collide - #71970
Open
Oscar-man-shrestha wants to merge 4 commits into
Open
Oscar-man-shrestha wants to merge 4 commits into
Oscar-man-shrestha wants to merge 4 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What happened to labels 6 and 7
The external grouping Series is named
aand labels the rows[6, 6, 7]. The DataFrame also has a data column namedawith values[1, 1, 2], so the sums are 2 for group 6 and 2 for group 7. Withas_index=False, pandas skipped inserting the external group labels because the result already had a column namedaholding[2, 2]. The old output showeda = [2, 2]and omitted the group labels.Why
ValueErrormatchesreset_index()hereWith
as_index=True, the labels sit in an index namedanext to a data column nameda. Callingreset_index()raisesValueError: cannot insert a, already exists. Both sets of values can be kept if the grouping key is renamed. With the name left asa, though, pandas would have to invent a new name or produce duplicateacolumns. Raising the same error fromas_index=Falselets the caller choose the name instead of having pandas guess.Changes
_insert_inaxis_groupernow raisesValueErrorwhen a named external grouper collides with a result column, instead of skipping the group labels.DataFrameGroupBy.aggregatewith a callable returns the result of_python_agg_generaldirectly, since that already inserts the grouping columns; previously they would be inserted twice.Behavior change:
test_groupby_as_index_agggrouped by an external Series namedjimthat collides with a column.mean,max,count,idxmaxandallnow raiseValueErrorthere, matchingreset_index().cumsumis a transform, inserts no grouping column, and is unchanged.Tests cover the collision, the renamed key (
external.rename("group")keeps[6, 7]next to the sums), and a callable aggregation inserting the key once.closes BUG: groupby with as_index=False drops groupings when names collide #58337
Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added an entry in the latest
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.I have reviewed and followed all the contribution guidelines
Check exactly one of the following, per the automated contributions policy:
AGENTS.md, I have reviewed and understood every change, and I have described above how I used it and exactly which tool, model version, and effort setting — e.g.claude opus 4.8 (xhigh), not justclaude.