Possible unnecessary OverflowError in random.getrandbits

[Nokhrin]

Documentation

Problem

Actual upper bound of a random.Random.randbytes is 2**28 - 1

Following docs and exception message I would expect the limit of 2 ** 32 - 1 as for C 4-bytes integer

    ## -------------------- bytes methods ---------------------

    def randbytes(self, n):
        """Generate n random bytes."""
        return self.getrandbits(n * 8).to_bytes(n, 'little')

>>> bytes_string = random.randbytes(2 ** 28)
Traceback (most recent call last):
  File "C:\Program Files\Python39\lib\code.py", line 90, in runcode
    exec(code, self.locals)
  File "<input>", line 1, in <module>
  File "C:\Program Files\Python39\lib\random.py", line 286, in randbytes
    return self.getrandbits(n * 8).to_bytes(n, 'little')
OverflowError: Python int too large to convert to C int
>>> bytes_string = random.randbytes(2 ** 28 - 1)
>>> len(bytes_string)
268435455

Reproduced on
Python 3.9.4 (tags/v3.9.4:1f2e308, Apr 6 2021, 13:40:21) [MSC v.1928 64 bit (AMD64)] on win32
Python 3.11.2 (main, Jul 19 2024, 12:24:02) [GCC 12.2.0] on linux

Will you please point me - if there is a bug in docs/implementation, or this is my misunderstanding only.
Thank you in advance for you patience.

Linked PRs

• gh-133489: Document random.randbytes bounds. #133529
• gh-133489: Remove size restrictions on getrandbits() and randbytes() #133658
• [3.14] gh-133489: Remove size restrictions on getrandbits() and randbytes() (GH-133658) #134964
• [3.13] gh-133489: Remove size restrictions on getrandbits() and randbytes() (GH-133658) #134965

[rhettinger]

@serhiy-storchaka The overflow occurs in the upstream call to random.getrandbits(8 * 2 ** 28 - 1).

The argument clinic spec reads:

_random.Random.getrandbits

  self: self(type="RandomObject *")
  k: int
  /

Should k be switched to object and converted with PyLong_AsLongLongAndOverflow? The 32-bit signed int limitation on Windows seems unnecessarily restrictive.

[skirpichev]

Why not have int32 converter in the AC? Now we have PyLong_AsInt32/AsUInt32/etc.

[rhettinger]

@skirpichev We want something like ssize_t so that we can ask for arrays as big as we can allocate. The limit should be driven by memory constraints rather than function argument constraints. 32-bit signed is insufficient.

[serhiy-storchaka]

Why not have int32 converter in the AC?

It was in my plans.

For this case it is better to use the size_t converter (automatic ValueError for negative value), or even the uint64 converter (to be able creating bytes objects up to 2 GiB on 32-bit platforms).

[skirpichev]

or even the uint64 converter

Yes, int32 was just a typo - I meant converters for fixed-size (unsigned) integers. 8-byte type looks better, of course, for the given case.

[serhiy-storchaka]

See #133583.

[serhiy-storchaka]

#133658 makes getrandbits() using 64-bit bits count. Since in CPython a hard limit for bits count in integers is 64-bit, this is the right type.

What should we do with other versions, @Raymond? I think that it is not late to backport it to 3.14, and the backport is strightforward, because the uint64 converter exists in 3.14. In 3.13 we could use unsigned_long_long, which is practically the same. Or we can document the limitation if it is too late for backport.

[gpshead]

raising OverflowError feels like a bug so back porting all the way to 3.13 as a bug fix makes sense to me.

[tornaria]

There is an (unintended?) api break. In python 3.13.3, getrandbits() accepts an "integer-like" object as in:

Python 3.13.3 (main, Apr 11 2025, 13:41:26) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import random ; r = random.Random()
>>> from numpy import int64
>>> r.getrandbits(int64(20))
582096

but in python 3.13.4, getrandbits() fails as in:

Python 3.13.4 (main, Jun  8 2025, 19:43:01) [GCC 14.2.1 20250405] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import random ; r = random.Random()
>>> from numpy import int64
>>> r.getrandbits(int64(20))
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    r.getrandbits(int64(20))
    ~~~~~~~~~~~~~^^^^^^^^^^^
TypeError: an integer is required

This causes trouble for sagemath, where integers are by default a special type ZZ.

[gpshead]

Can you please open a new issue to track that?

[skirpichev]

Ah, that's because _PyLong_UnsignedLongLong_Converter() has no fallback to index-like objects.

In 3.14+ we are using PyLong_AsNativeBytes for uint64 converter and it's not an issue. Maybe just add a uint64 converter for 3.13 (it has PyLong_AsNativeBytes) or this is too much?