Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: rate limit middleware #44

Open
joscha opened this issue Sep 9, 2024 · 1 comment
Open

Feature request: rate limit middleware #44

joscha opened this issue Sep 9, 2024 · 1 comment

Comments

@joscha
Copy link
Member

joscha commented Sep 9, 2024

This package should provide a rate limit middleware that respects the API rate limits

  • throttle requests based on the actual free API requests that are sent with each response:
       "x-ratelimit-limit-org": "unlimited",
       "x-ratelimit-limit-org-remaining": "unlimited",
       "x-ratelimit-limit-org-reset": "2143843",
       "x-ratelimit-limit-user": "900",
       "x-ratelimit-limit-user-remaining": "863",
       "x-ratelimit-limit-user-reset": "26",
* automatic retry/requeue requests that failed because of a `429` response.
@joscha
Copy link
Member Author

joscha commented Oct 8, 2024

With a combination of V1 api and V2 api, the best workaround is currently:

import { Affinity } from "@planet-a/affinity-node/v1";
import {
  createConfiguration,
  helpers,
  PersonsApi,
} from "@planet-a/affinity-node/v2";
import assert from "assert";
import pThrottle from "p-throttle";

const {
  paginator: { paginated },
} = helpers;

const config = createConfiguration({
  authMethods: {
    bearerAuth: {
      tokenProvider: {
        getToken: async () => {
          assert(process.env.AFFINITY_API_KEY);
          return process.env.AFFINITY_API_KEY;
        },
      },
    },
  },
});

assert(process.env.AFFINITY_API_KEY);
const affinity = new Affinity(process.env.AFFINITY_API_KEY);
const {
  rate: {
    api_key_per_minute: { limit },
  },
} = await affinity.rateLimit.get();

const throttle = pThrottle({
  limit,
  interval: 60 * 1000,
});

const personsApi = new PersonsApi(config);
const personFn = personsApi.getV2Persons.bind(personsApi);
const throttledPersonFn = throttle(personFn);
const personIterator = paginated(throttledPersonFn);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant