Skip to content

API: New copy / view semantics using Copy-on-Write #46958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Aug 20, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ea0bff8
API: New copy / view semantics using Copy-on-Write
jorisvandenbossche May 6, 2022
74981f9
fix more tests
jorisvandenbossche May 7, 2022
36faac9
Handle CoW in BM.iset
jorisvandenbossche May 9, 2022
ffa24a4
Handle CoW in xs
jorisvandenbossche May 9, 2022
8c00bbd
add bunch of todo comments and usage warnings
jorisvandenbossche May 9, 2022
af5a02c
Insert None ref in BM.insert
jorisvandenbossche May 9, 2022
a64f310
Ensure to not assume copy_on_write is set in case of ArrayManager
jorisvandenbossche May 9, 2022
e6c0baa
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche May 10, 2022
977cbb8
Handle refs in BM._combine / test CoW in select_dtypes
jorisvandenbossche May 10, 2022
cc367f5
handle get_numeric_data for single block manager
jorisvandenbossche May 10, 2022
53a8273
fix test_internals (get_numeric_data now uses CoW)
jorisvandenbossche May 11, 2022
d3d26f4
handle refs in consolidation
jorisvandenbossche May 20, 2022
5360e57
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche May 20, 2022
e431bc4
fix deep=None for ArrayManager
jorisvandenbossche May 20, 2022
a2d8fde
update copy/view tests from other PR
jorisvandenbossche May 20, 2022
81f6eae
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche May 23, 2022
b2a1428
clean-up fast_xs workarounds now it returns a SingleBlockManager
jorisvandenbossche May 23, 2022
4b1ccf6
tracks refs in to_frame
jorisvandenbossche May 23, 2022
40f2b24
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Jun 8, 2022
9339f15
fixup after updata main and column_setitem + iloc inplace setitem cha…
jorisvandenbossche Jul 16, 2022
358deaf
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Jul 16, 2022
085d15b
fix inplace fillna + fixup new tests
jorisvandenbossche Jul 16, 2022
377f789
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Jul 17, 2022
7dcefe8
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Jul 21, 2022
773f03f
address comments + update some todo comments
jorisvandenbossche Jul 21, 2022
bfc2117
Update pandas/core/internals/managers.py
jorisvandenbossche Jul 21, 2022
89c4fff
fixup linting
jorisvandenbossche Jul 21, 2022
b2e56ea
update new copy_view tests to use get_array helper
jorisvandenbossche Jul 21, 2022
fdeb154
add comment to setitem
jorisvandenbossche Jul 21, 2022
c521161
switch default to False, ensure CoW copies only happen when enabled +…
jorisvandenbossche Jul 21, 2022
fb297df
update type annotations
jorisvandenbossche Jul 21, 2022
9ab9df5
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Aug 10, 2022
14f753e
Fix stata issue to avoid SettingWithCopyWarning in read_stata
jorisvandenbossche Aug 10, 2022
f7389b1
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Aug 17, 2022
764838f
update type + option comment
jorisvandenbossche Aug 17, 2022
efde1bf
Merge remote-tracking branch 'upstream/main' into blockmanager-cow
jorisvandenbossche Aug 18, 2022
3d1377b
fixup new rename test
jorisvandenbossche Aug 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
update type + option comment
  • Loading branch information
jorisvandenbossche committed Aug 17, 2022
commit 764838fdca3deb599b20430ef6e413f5e3ad1535
7 changes: 2 additions & 5 deletions pandas/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,7 @@
from decimal import Decimal
import operator
import os
from typing import (
Callable,
Literal,
)
from typing import Callable

from dateutil.tz import (
tzlocal,
Expand Down Expand Up @@ -1844,7 +1841,7 @@ def using_array_manager():


@pytest.fixture
def using_copy_on_write() -> Literal[False]:
def using_copy_on_write() -> bool:
"""
Fixture to check if Copy-on-Write is enabled.
"""
Expand Down
4 changes: 2 additions & 2 deletions pandas/core/config_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -544,8 +544,8 @@ def use_inf_as_na_cb(key) -> None:
copy_on_write_doc = """
: bool
Use new copy-view behaviour using Copy-on-Write. Defaults to False,
unless overridden by the 'PANDAS_COPY_ON_WRITE' environment variable (needs
to be set before pandas is imported).
unless overridden by the 'PANDAS_COPY_ON_WRITE' environment variable
(if set to "1" for True, needs to be set before pandas is imported).
"""


Expand Down