././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/CHANGELOG.md0000644000000000000000000000444414725335663012434 0ustar00# Changelog ## Aiolimiter 1.2.1 (2024-12-08) ### Bugfixes - Issue a `RuntimeWarning` and reset internal waiter state when being reused across asyncio loops. ([#292](https://github.com/mjpieters/aiolimiter/issues/292)) ## Aiolimiter 1.2.0 (2024-12-01) ### Bugfixes - Improve performance by using a single timeout and a heapq for blocked tasks. This ensures only a single task needs to wake up per 'drip' of the bucket, instead of creating timeouts for every task. ([#73](https://github.com/mjpieters/aiolimiter/issues/73)) ## Aiolimiter 1.1.1 (2024-11-30) ### Bugfixes - Only include CHANGELOG.md in the source distribution. ([#206](https://github.com/mjpieters/aiolimiter/issues/206)) - Fixed wait time calculation for waiting tasks, making acquisition faster (PR by @schoennenbeck) ([#217](https://github.com/mjpieters/aiolimiter/issues/217)) ### Misc - [#139](https://github.com/mjpieters/aiolimiter/issues/139), [#288](https://github.com/mjpieters/aiolimiter/issues/288) ## Aiolimiter 1.1.0 (2023-05-08) ### Features - Add ``__slots__`` to the ``AsyncLimiter`` class, reducing memory requirements. ([#85](https://github.com/mjpieters/aiolimiter/issues/85)) ### Deprecations and Removals - Dropped support for Python 3.6 ([#62](https://github.com/mjpieters/aiolimiter/issues/62)) ### Misc - [#95](https://github.com/mjpieters/aiolimiter/issues/95) ## Aiolimiter 1.0.0 (2021-10-15) ### Bugfixes - Avoid warnings on Python 3.8 and up by not passing in the loop to ``asyncio.wait_for()``. ([#46](https://github.com/mjpieters/aiolimiter/issues/46)) ## Aiolimiter 1.0.0b1 (2019-12-01) ### Improved Documentation - Corrected build process to ensure CHANGELOG.md is updated on release. ([#4](https://github.com/mjpieters/aiolimiter/issues/4)) ## Aiolimiter 1.0.0b0 (2019-11-30) _No significant changes_. ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/LICENSE.txt0000644000000000000000000000206014725335663012436 0ustar00MIT License Copyright (c) 2019 Martijn Pieters Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/README.md0000644000000000000000000000633614725335663012104 0ustar00# aiolimiter [![Azure Pipelines status for master branch][azure_badge]][azure_status] [![codecov.io status for master branch][codecov_badge]][codecov_status] [![Latest PyPI package version][pypi_badge]][aiolimiter_release] [![Latest Read The Docs][rtd_badge]][aiolimiter_docs] [azure_badge]: https://dev.azure.com/mjpieters/aiolimiter/_apis/build/status/CI?branchName=master [azure_status]: https://dev.azure.com/mjpieters/aiolimiter/_build/latest?definitionId=4&branchName=master "Azure Pipelines status for master branch" [codecov_badge]: https://codecov.io/gh/mjpieters/aiolimiter/branch/master/graph/badge.svg [codecov_status]: https://codecov.io/gh/mjpieters/aiolimiter "codecov.io status for master branch" [pypi_badge]: https://badge.fury.io/py/aiolimiter.svg [aiolimiter_release]: https://pypi.org/project/aiolimiter "Latest PyPI package version" [rtd_badge]: https://readthedocs.org/projects/aiolimiter/badge/?version=latest [aiolimiter_docs]: https://aiolimiter.readthedocs.io/en/latest/?badge=latest "Latest Read The Docs" ## Introduction An efficient implementation of a rate limiter for asyncio. This project implements the [Leaky bucket algorithm][], giving you precise control over the rate a code section can be entered: ```python from aiolimiter import AsyncLimiter # allow for 100 concurrent entries within a 30 second window rate_limit = AsyncLimiter(100, 30) async def some_coroutine(): async with rate_limit: # this section is *at most* going to entered 100 times # in a 30 second period. await do_something() ``` It was first developed [as an answer on Stack Overflow][so45502319]. ## Documentation https://aiolimiter.readthedocs.io ## Installation ```sh $ pip install aiolimiter ``` The library requires Python 3.8 or newer. ## Requirements - Python >= 3.8 ## License `aiolimiter` is offered under the [MIT license](./LICENSE.txt). ## Source code The project is hosted on [GitHub][]. Please file an issue in the [bug tracker][] if you have found a bug or have some suggestions to improve the library. ## Developer setup This project uses [poetry][] to manage dependencies, testing and releases. Make sure you have installed that tool, then run the following command to get set up: ```sh poetry install --with docs && poetry run doit devsetup ``` Apart from using `poetry run doit devsetup`, you can either use `poetry shell` to enter a shell environment with a virtualenv set up for you, or use `poetry run ...` to run commands within the virtualenv. Tests are run with `pytest` and `tox`. Releases are made with `poetry build` and `poetry publish`. Code quality is maintained with `flake8`, `black` and `mypy`, and `pre-commit` runs quick checks to maintain the standards set. A series of `doit` tasks are defined; run `poetry run doit list` (or `doit list` with `poetry shell` activated) to list them. The default action is to run a full linting, testing and building run. It is recommended you run this before creating a pull request. [leaky bucket algorithm]: https://en.wikipedia.org/wiki/Leaky_bucket [so45502319]: https://stackoverflow.com/a/45502319/100297 [github]: https://github.com/mjpieters/aiolimiter [bug tracker]: https://github.com/mjpieters/aiolimiter/issues [poetry]: https://poetry.eustace.io/ ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/pyproject.toml0000644000000000000000000000432014725335663013530 0ustar00[tool.poetry] name = "aiolimiter" version = "1.2.1" description = "asyncio rate limiter, a leaky bucket implementation" license = "MIT" authors = ["Martijn Pieters "] readme = "README.md" homepage = "https://github.com/mjpieters/aiolimiter" repository = "https://github.com/mjpieters/aiolimiter" documentation = "http://aiolimiter.readthedocs.org/en/stable/" keywords = ["asyncio", "rate-limiting", "leaky-bucket"] classifiers = [ "Framework :: AsyncIO", "Intended Audience :: Developers", ] include = [ { path = "CHANGELOG.md", format = "sdist" }, ] [tool.poetry.urls] "CI: Azure Pipelines" = "https://dev.azure.com/mjpieters/aiolimiter/_build" "Coverage: codecov" = "https://codecov.io/github/aiolimiter/aiosignal" "GitHub: issues" = "https://github.com/mjpieters/aiolimiter/issues" [tool.poetry.dependencies] python = "^3.8" [tool.poetry.group.dev.dependencies] pytest = ">=7,<9" flake8 = "^5.0.3" flake8-bugbear = "^23.1.20" pytest-asyncio = ">=0.24,<0.25" pytest-cov = ">=4,<6" tox = ">=3.14.1,<5.0.0" pre-commit = ">=2.9.2,<4.0.0" doit = ">=0.34,<0.37" isort = "^5.2.1" toml = "^0.10.0" twine = ">=4,<7" towncrier = ">=22.8,<25.0" [tool.poetry.group.dev.dependencies.black] version = "^24.4.2" markers = "platform_python_implementation != 'PyPy'" [tool.poetry.group.dev.dependencies.mypy] version = "^1.1" markers = "platform_python_implementation != 'PyPy'" [tool.poetry.group.docs] optional = true [tool.poetry.group.docs.dependencies] sphinx = ">=4.2.0,<8.0.0" aiohttp-theme = "^0.1.6" sphinx-autodoc-typehints = ">=1.10.3,<3.0.0" sphinxcontrib-spelling = ">=4.3,<8.0" toml = "^0.10.0" [tool.black] target-version = ["py37", "py38", "py39", "py310", "py311", "py312", "py313"] [tool.isort] line_length = 88 known_third_party = ["pytest"] [tool.towncrier] directory = "changelog.d/" template = "changelog.d/towncrier_template.md" filename = "CHANGELOG.md" package_dir = "src" package = "aiolimiter" title_format = "## {name} {version} ({project_date})" issue_format = "[#{issue}](https://github.com/mjpieters/aiolimiter/issues/{issue})" start_string = "\n" underlines = ["", "", ""] [build-system] requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api" ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/src/aiolimiter/__init__.py0000644000000000000000000000043714725335663015657 0ustar00# SPDX-License-Identifier: MIT # Copyright (c) 2019 Martijn Pieters # Licensed under the MIT license as detailed in LICENSE.txt from importlib.metadata import version # type: ignore from .leakybucket import AsyncLimiter __version__ = version("aiolimiter") __all__ = ["AsyncLimiter"] ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/src/aiolimiter/leakybucket.py0000644000000000000000000001604614725335663016426 0ustar00# SPDX-License-Identifier: MIT # Copyright (c) 2019 Martijn Pieters # Licensed under the MIT license as detailed in LICENSE.txt import asyncio import os import sys import warnings from contextlib import AbstractAsyncContextManager from functools import partial from heapq import heappop, heappush from itertools import count from types import TracebackType from typing import List, Optional, Tuple, Type LIMITER_REUSED_ACROSS_LOOPS_WARNING = ( "This AsyncLimiter instance is being re-used across loops. Please create " "a new limiter per event loop as re-use can lead to undefined behaviour." ) if sys.version_info >= (3, 12): # pragma: no cover _warn_reuse = partial( warnings.warn, message=LIMITER_REUSED_ACROSS_LOOPS_WARNING, category=RuntimeWarning, skip_file_prefixes=(os.path.dirname(__file__),), ) else: # no support for dynamic stack levels, disable stack location _warn_reuse = partial( warnings.warn, message=LIMITER_REUSED_ACROSS_LOOPS_WARNING, category=RuntimeWarning, stacklevel=0, ) class AsyncLimiter(AbstractAsyncContextManager): """A leaky bucket rate limiter. This is an :ref:`asynchronous context manager `; when used with :keyword:`async with`, entering the context acquires capacity:: limiter = AsyncLimiter(10) for foo in bar: async with limiter: # process foo elements at 10 items per minute :param max_rate: Allow up to `max_rate` / `time_period` acquisitions before blocking. :param time_period: duration, in seconds, of the time period in which to limit the rate. Note that up to `max_rate` acquisitions are allowed within this time period in a burst. """ __slots__ = ( "max_rate", "time_period", "_rate_per_sec", "_level", "_last_check", "_event_loop", "_waiters", "_next_count", "_waker_handle", ) max_rate: float #: The configured `max_rate` value for this limiter. time_period: float #: The configured `time_period` value for this limiter. def __init__(self, max_rate: float, time_period: float = 60) -> None: self.max_rate = max_rate self.time_period = time_period self._rate_per_sec = max_rate / time_period self._level = 0.0 self._last_check = 0.0 # timer until next waiter can resume self._waker_handle: asyncio.TimerHandle | None = None # min-heap with (amount requested, order, future) for waiting tasks self._waiters: List[Tuple[float, int, "asyncio.Future[None]"]] = [] # counter used to order waiting tasks self._next_count = partial(next, count()) @property def _loop(self) -> asyncio.AbstractEventLoop: self._event_loop: asyncio.AbstractEventLoop try: loop = self._event_loop if loop.is_closed(): # limiter is being reused across loops; make a best-effort # attempt at recovery. Existing waiters are ditched, with # the assumption that they are no longer viable. loop = self._event_loop = asyncio.get_running_loop() self._waiters = [ (amt, cnt, fut) for amt, cnt, fut in self._waiters if fut.get_loop() == loop ] _warn_reuse() except AttributeError: loop = self._event_loop = asyncio.get_running_loop() return loop def _leak(self) -> None: """Drip out capacity from the bucket.""" now = self._loop.time() if self._level: # drip out enough level for the elapsed time since # we last checked elapsed = now - self._last_check decrement = elapsed * self._rate_per_sec self._level = max(self._level - decrement, 0) self._last_check = now def has_capacity(self, amount: float = 1) -> bool: """Check if there is enough capacity remaining in the limiter :param amount: How much capacity you need to be available. """ self._leak() return self._level + amount <= self.max_rate async def acquire(self, amount: float = 1) -> None: """Acquire capacity in the limiter. If the limit has been reached, blocks until enough capacity has been freed before returning. :param amount: How much capacity you need to be available. :exception: Raises :exc:`ValueError` if `amount` is greater than :attr:`max_rate`. """ if amount > self.max_rate: raise ValueError("Can't acquire more than the maximum capacity") loop = self._loop while not self.has_capacity(amount): # Add a future to the _waiters heapq to be notified when capacity # has come up. The future callback uses call_soon so other tasks # are checked *after* completing capacity acquisition in this task. fut = loop.create_future() fut.add_done_callback(partial(loop.call_soon, self._wake_next)) heappush(self._waiters, (amount, self._next_count(), fut)) self._wake_next() await fut self._level += amount # reset the waker to account for the new, lower level. self._wake_next() return None def _wake_next(self, *_args: object) -> None: """Wake the next waiting future or set a timer""" # clear timer and any cancelled futures at the top of the heap heap, handle, self._waker_handle = self._waiters, self._waker_handle, None if handle is not None: handle.cancel() while heap and heap[0][-1].done(): heappop(heap) if not heap: # nothing left waiting return amount, _, fut = heap[0] self._leak() needed = amount - self.max_rate + self._level if needed <= 0: heappop(heap) fut.set_result(None) # fut.set_result triggers another _wake_next call return wake_next_at = self._last_check + (1 / self._rate_per_sec * needed) self._waker_handle = self._loop.call_at(wake_next_at, self._wake_next) def __repr__(self) -> str: # pragma: no cover args = f"max_rate={self.max_rate!r}, time_period={self.time_period!r}" state = f"level: {self._level:f}, waiters: {len(self._waiters)}" if (handle := self._waker_handle) and not handle.cancelled(): microseconds = int((handle.when() - self._loop.time()) * 10**6) if microseconds > 0: state += f", waking in {microseconds} \N{MICRO SIGN}s" return f"" async def __aenter__(self) -> None: await self.acquire() return None async def __aexit__( self, exc_type: Optional[Type[BaseException]], exc: Optional[BaseException], tb: Optional[TracebackType], ) -> None: return None ././@PaxHeader0000000000000000000000000000003400000000000010212 xustar0028 mtime=1733671859.4536474 aiolimiter-1.2.1/src/aiolimiter/py.typed0000644000000000000000000000016114725335663015237 0ustar00# This package supports type hinting, # see https://www.python.org/dev/peps/pep-0561/#packaging-type-information aiolimiter-1.2.1/PKG-INFO0000644000000000000000000001062400000000000011640 0ustar00Metadata-Version: 2.1 Name: aiolimiter Version: 1.2.1 Summary: asyncio rate limiter, a leaky bucket implementation Home-page: https://github.com/mjpieters/aiolimiter License: MIT Keywords: asyncio,rate-limiting,leaky-bucket Author: Martijn Pieters Author-email: mj@zopatista.com Requires-Python: >=3.8,<4.0 Classifier: Framework :: AsyncIO Classifier: Intended Audience :: Developers Classifier: License :: OSI Approved :: MIT License Classifier: Programming Language :: Python :: 3 Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 Classifier: Programming Language :: Python :: 3.12 Classifier: Programming Language :: Python :: 3.13 Project-URL: CI: Azure Pipelines, https://dev.azure.com/mjpieters/aiolimiter/_build Project-URL: Coverage: codecov, https://codecov.io/github/aiolimiter/aiosignal Project-URL: Documentation, http://aiolimiter.readthedocs.org/en/stable/ Project-URL: GitHub: issues, https://github.com/mjpieters/aiolimiter/issues Project-URL: Repository, https://github.com/mjpieters/aiolimiter Description-Content-Type: text/markdown # aiolimiter [![Azure Pipelines status for master branch][azure_badge]][azure_status] [![codecov.io status for master branch][codecov_badge]][codecov_status] [![Latest PyPI package version][pypi_badge]][aiolimiter_release] [![Latest Read The Docs][rtd_badge]][aiolimiter_docs] [azure_badge]: https://dev.azure.com/mjpieters/aiolimiter/_apis/build/status/CI?branchName=master [azure_status]: https://dev.azure.com/mjpieters/aiolimiter/_build/latest?definitionId=4&branchName=master "Azure Pipelines status for master branch" [codecov_badge]: https://codecov.io/gh/mjpieters/aiolimiter/branch/master/graph/badge.svg [codecov_status]: https://codecov.io/gh/mjpieters/aiolimiter "codecov.io status for master branch" [pypi_badge]: https://badge.fury.io/py/aiolimiter.svg [aiolimiter_release]: https://pypi.org/project/aiolimiter "Latest PyPI package version" [rtd_badge]: https://readthedocs.org/projects/aiolimiter/badge/?version=latest [aiolimiter_docs]: https://aiolimiter.readthedocs.io/en/latest/?badge=latest "Latest Read The Docs" ## Introduction An efficient implementation of a rate limiter for asyncio. This project implements the [Leaky bucket algorithm][], giving you precise control over the rate a code section can be entered: ```python from aiolimiter import AsyncLimiter # allow for 100 concurrent entries within a 30 second window rate_limit = AsyncLimiter(100, 30) async def some_coroutine(): async with rate_limit: # this section is *at most* going to entered 100 times # in a 30 second period. await do_something() ``` It was first developed [as an answer on Stack Overflow][so45502319]. ## Documentation https://aiolimiter.readthedocs.io ## Installation ```sh $ pip install aiolimiter ``` The library requires Python 3.8 or newer. ## Requirements - Python >= 3.8 ## License `aiolimiter` is offered under the [MIT license](./LICENSE.txt). ## Source code The project is hosted on [GitHub][]. Please file an issue in the [bug tracker][] if you have found a bug or have some suggestions to improve the library. ## Developer setup This project uses [poetry][] to manage dependencies, testing and releases. Make sure you have installed that tool, then run the following command to get set up: ```sh poetry install --with docs && poetry run doit devsetup ``` Apart from using `poetry run doit devsetup`, you can either use `poetry shell` to enter a shell environment with a virtualenv set up for you, or use `poetry run ...` to run commands within the virtualenv. Tests are run with `pytest` and `tox`. Releases are made with `poetry build` and `poetry publish`. Code quality is maintained with `flake8`, `black` and `mypy`, and `pre-commit` runs quick checks to maintain the standards set. A series of `doit` tasks are defined; run `poetry run doit list` (or `doit list` with `poetry shell` activated) to list them. The default action is to run a full linting, testing and building run. It is recommended you run this before creating a pull request. [leaky bucket algorithm]: https://en.wikipedia.org/wiki/Leaky_bucket [so45502319]: https://stackoverflow.com/a/45502319/100297 [github]: https://github.com/mjpieters/aiolimiter [bug tracker]: https://github.com/mjpieters/aiolimiter/issues [poetry]: https://poetry.eustace.io/