Add --disallow-any-generics to --strict #5685

TV4Fun · 2018-09-27T01:03:04Z

This is a tiny bit hacky as --disallow-any-generics is not defined
through add_invertible_flag. Resolves #5684.

This is a tiny bit hacky as --disallow-any-generics is not defined through `add_invertible_flag`.

TV4Fun · 2018-09-27T01:11:21Z

BTW, I think --disallow-any-generics is not a good name for this flag, as it implies that it doesn't allow generics explicitly parameterized with Any. I think a better name would be --disallow-unparameterized-generics.

ilevkivskyi

Thanks! Could you please add a test for this (see check-flags.test)?

ilevkivskyi · 2018-10-04T15:06:41Z

mypy/main.py

+             'type parameters').dest
+    assert dest is not None
+    strict_flag_names.append('--disallow-any-generics')
+    strict_flag_assignments.append((dest, True))


Why can't we just use add_invertible_flag here? Like, for example, --disallow-subclassing-any above.

Yes, that does make more sense. Fixed to do that now.

In preparation of working on PEP 420 namespaces, here's some initial work. IT DOES NOT IMPLEMENT NAMESPACES YET. What it does do: - Move nearly 400 lines from build.py to a new file, `modulefinder.py`. This includes `SearchPaths`, `BuildSource`, `FindModuleCache`, `mypy-path`, `default_lib_path`, `get_site_packages` (now without underscore!), and `compute_search_paths`. - Slight refactor to `FindModuleCache` so that the `search_paths` are passed to the constructor instead of to `find_module`. - Removed `search_paths` and `python_executable` from the signature of `find_module` and `find_modules_recursive`, and also from the cache key (**this may be the most controversial change** -- it's not just a refactor). - Add a (non-functional) `--namespace-packages` flag to `main.py` and add new global config option `namespace_packages`. (These are not used yet.) I'm presenting this as a separate PR because it's a significant refactor without change of functionality -- everything I plan to do after this will mostly tweak the code in `modulefinder.py`. It seems fairer to reviewers to be able to review this massive code move without having to worry too much about "what else changed". (Note that this is part of my general plan to move functionality out of `build.py` -- that file is (still) way too bulky.)

* Get rid of trivial type_check_only() function * Remove bin_dir -- it's no longer used * Fix wrong import in testcheck.py * Fix indent of compute_search_paths() args * Minor cleanup in mypy/test/testcheck.py * Cleanups in mypy/dmypy_server.py and mypy/server/ * Fix lint * Fix lint fix

…_subclass__ (python#5677) Implements python#604 (comment).

Helps with issue discussed in python#5688.

python#5702)

…on#5711) This commit fixes two issues with said code sample: - The math import was missing. - The function argument didn't match the variable name used in the function body. Also made comment spacing consistent.

In conjunction with a change that makes mypyc able to directly generate calls to `__bool__`, this allows us to generate a much more direct check for truthiness of TypeInfo.

Fixes python#4872 Fixes python#3876 Fixes python#2678 Fixes python#5199 Fixes python#5493 (It also fixes a bunch of similar issues previously closed as duplicates, except one, see below). This PR fixes a problems when mypy commits to soon to using outer context for type inference. This is done by: * Postponing inference to inner (argument) context in situations where type inferred from outer (return) context doesn't satisfy bounds or constraints. * Adding a special case for situation where optional return is inferred against optional context. In such situation, unwrapping the optional is a better idea in 99% of cases. (Note: this doesn't affect type safety, only gives empirically more reasonable inferred types.) In general, instead of adding a special case, it would be better to use inner and outer context at the same time, but this a big change (see comment in code), and using the simple special case fixes majority of issues. Among reported issues, only python#5311 will stay unfixed.

Replace the `var_arg` and `kw_arg` attributes with methods in `CallableType`. This speeds up type checking slightly and improves memory use, as we only construct the value as needed. Subtype checks for callables are now slower, but that's fine since we construct callables at least 10x as frequently as we perform callable subtype checks. Also remove a somewhat expensive assertion. This seems to give about 2-3% performance improvement on self-check (not compiled).

This implements various small optimizations to make AST conversion from the typed_ast AST faster in compiled mypy. This makes compiled mypy about 4% faster on self-check, and it also makes non-compiled mypy around 1% faster on self-check. Notable included optimizations: * Implement visit methods more efficiently. Don't inherit visitors from typed_ast visitor classes. * Refactor decorated methods away, as they are slow with mypyc. * Refactor away many non-native attribute lookups. * Refactor away some nested functions as they are slow with mypyc. Also removed some dead code. I'll implement the same optimizations for Python 2 AST conversions in a separate PR.

Tentative implementation of PEP 420. Fixes python#1645. Clarification of the implementation: - `candidate_base_dirs` is a list of `(dir, verify)` tuples, laboriously pre-computed. - The old code just looped over this list, looking for `dir/<module>`, and if found, it would verify that there were `__init__.py` files in all the right places (except if `verify` is false); the first success would be the hit; - In PEP 420 mode, if the above approach finds no hits, we do something different for those paths that failed due to `__init__` verification; essentially we narrow down the list of candidate paths by checking for `__init__` files from the top down. Hopefully the last test added clarifies this.

This seems to speed up compiled mypy by up to about 4%, with marginal changes to memory use.

Note that python#5728 is not fixed by this yet! The behavior is unchanged except it is no longer listed in the `--help` output. Also add `file=sys.stderr` to a few of the `print()` calls for other deprecated options. Also mark it deprecated in the docs (but don't delete those docs yet).

…any-generics

This reverts commit fc89540.

TV4Fun · 2018-10-04T22:19:04Z

Added a couple of test cases for --disallow-any-generics. Interestingly, though there were already two test cases for --disallow-any-generics, neither actually tested for an error condition, so added that. I wanted to add a test for --strict as there aren't any now, but that seemed to create some errors unrelated to my test case:

----------------------------- Captured stderr call -----------------------------
Expected:
  main:7: error: Missing type parameters for generic type (diff)
Actual:
  tmp/builtins.pyi:8: error: Function is missing a type annotation (diff)
  tmp/builtins.pyi:34: error: Call to untyped function "object" in typed context (diff)
  main:7: error: Missing type parameters for generic type (diff)

Alignment of first line difference:
  E: main:7: error: Missing type parameters for generic type
  A: tmp/builtins.pyi:8: error: Function is missing a type annotation

The test that caused this was:

[case testCheckStrictAnyGeneric]
# flags: --strict
from typing import TypeVar, Callable

T = TypeVar('T')
C = Callable[[], T]

def f(c: C) -> None:  # E: Missing type parameters for generic type
    pass
[builtins fixtures/list.pyi]
[out]

Not sure if this is a known issue or not.

JelleZijlstra · 2018-10-04T23:12:10Z

This is probably because the stubs we use for builtins in tests aren't --strict-clean. Might be worth fixing.

ilevkivskyi

Thanks for adding the tests!

I think your issue with errors in builtins will be fixed by not using the builtins fixture you don't need.

Alternatively, you can just fix the errors in that file (fixtures/list.pyi).

ilevkivskyi · 2018-10-06T21:20:49Z

test-data/unit/check-flags.test

+
+def f(c: C):
+    pass
+[builtins fixtures/list.pyi]


I don't think you need the fixture here and in other tests.

You are correct, removed.

test-data/unit/check-flags.test

ilevkivskyi · 2018-10-06T21:29:00Z

(Also would be good to fix the commit history, but it is not essential since we will squash anyway.)

TV4Fun · 2018-10-10T20:55:42Z

Comments addressed. I agree the commit history is a little messed up, but as you said, you are going to squash anyway.

ilevkivskyi

Thanks! Looks ready now. I have only one suggestion about one test.

test-data/unit/check-flags.test

ilevkivskyi · 2018-10-11T19:26:12Z

@gvanrossum Does it make sense to cherry-pick in the release branch? The point is that it should only affect people who use --strict, and those are presumably fine with the extra errors this may cause.

Edit: merge -> cherry-pick.

gvanrossum · 2018-10-12T15:04:58Z

I'm sorry, the 0.640 release train has already closed its doors and is firing up its engine. It's too late too get on board. Take the next one in 3-4 weeks!

This flag is enabled by "mypy --strict" since mypy 0.650 so I want to enable it in the default configuration too. See python/mypy#5685

Add --disallow-any-generics to --strict

94403b4

This is a tiny bit hacky as --disallow-any-generics is not defined through `add_invertible_flag`.

ilevkivskyi reviewed Oct 4, 2018

View reviewed changes

gvanrossum and others added 25 commits October 4, 2018 14:32

Remove unused imports in build module (python#5693)

2347f63

Remove unused imports (python#5694)

29152c1

Make return type implicitly None for type checked __init__ and __init…

85c173b

…_subclass__ (python#5677) Implements python#604 (comment).

Add mention of runtests script to tests readme (python#5709)

4892882

Helps with issue discussed in python#5688.

Use sys.executable for invoking mypy in runtests if it is >=Python 3.4 (

0b6260b

python#5702)

Micro-optimization of TypeInfo.__bool__ (python#5712)

652ec81

In conjunction with a change that makes mypyc able to directly generate calls to `__bool__`, this allows us to generate a much more direct check for truthiness of TypeInfo.

Sync typeshed (python#5717)

7b4ed0f

Tweak TypeState to avoid getattrs under mypyc (python#5726)

3133f01

Use a raw string for a regex (python#5729)

0a52c2e

Increase the GC threshold 3x (python#5734)

549f71d

This seems to speed up compiled mypy by up to about 4%, with marginal changes to memory use.

DataDrivenTestCase: only use .file, not .path (python#5731)

45a2a4e

Specify --disallow-any-generics via add_invertible_flag

7c0bf9b

Update typeshed

fc89540

Merge branch 'master' of github.com:python/mypy into strict-dissalow-…

5b11056

…any-generics

Revert "Update typeshed"

2a38d35

This reverts commit fc89540.

Shorten too long line

3bfcf2f

Add unit tests for --disallow-any-generics

2ea3224

ilevkivskyi reviewed Oct 6, 2018

View reviewed changes

Add test and remove unnecessary test fixtures

e561452

ilevkivskyi approved these changes Oct 11, 2018

View reviewed changes

test-data/unit/check-flags.test Outdated Show resolved Hide resolved

Update testCheckAllowAnyGenericAnyGeneric

408bcfe

ilevkivskyi merged commit d5b4f74 into python:master Oct 11, 2018

suutari-ai pushed a commit to suutari/meterelf that referenced this pull request Jan 27, 2019

setup.cfg: Enable disallow_any_generics

e66736b

This flag is enabled by "mypy --strict" since mypy 0.650 so I want to enable it in the default configuration too. See python/mypy#5685

TV4Fun deleted the strict-dissalow-any-generics branch July 9, 2021 13:34

Uh oh!

Add --disallow-any-generics to --strict #5685

Add --disallow-any-generics to --strict #5685

Uh oh!

Conversation

TV4Fun commented Sep 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

TV4Fun commented Sep 27, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi Oct 4, 2018

Choose a reason for hiding this comment

Uh oh!

TV4Fun Oct 4, 2018

Choose a reason for hiding this comment

Uh oh!

TV4Fun commented Oct 4, 2018

Uh oh!

JelleZijlstra commented Oct 4, 2018

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

ilevkivskyi Oct 6, 2018

Choose a reason for hiding this comment

Uh oh!

TV4Fun Oct 10, 2018

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ilevkivskyi commented Oct 6, 2018

Uh oh!

TV4Fun commented Oct 10, 2018

Uh oh!

ilevkivskyi left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ilevkivskyi commented Oct 11, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum commented Oct 12, 2018

Uh oh!

Uh oh!

TV4Fun commented Sep 27, 2018 •

edited

Loading

TV4Fun commented Sep 27, 2018 •

edited

Loading

ilevkivskyi commented Oct 11, 2018 •

edited

Loading