mirror of
https://github.com/OpenBB-finance/OpenBB.git
synced 2026-10-29 13:32:08 +00:00
7b7f012aa9
* fix(finra): use persistent session to prevent TooManyRedirects Resolves #6972. The FINRA OTC data fetching functions (get_finra_weeks, get_finra_data, get_full_data) previously created new HTTP sessions for each request. When making dozens of sequential API calls to aggregate weekly data, this caused Cloudflare's bot protection on api.finra.org to trigger redirect loops, resulting in a TooManyRedirects error. Changes: - Refactored get_finra_weeks and get_finra_data to accept an optional session parameter via kwargs. - Updated get_full_data to create a single requests.Session and perform a pre-flight GET to finra.org to establish Cloudflare cookies before making API calls. - The session is reused across all requests within a single data fetch operation, preventing the redirect loop. - Removed hardcoded User-Agent from get_finra_data in favor of the platform's standard user agent set by get_requests_session. - Increased timeout for weeklyDownloadDetails from 3s to 20s for consistency and reliability. * Apply Black Formatting * perf(finra): use async requests for concurrent weekly data fetching Addresses reviewer feedback on PR #7443. The get_full_data function previously made up to 52 sequential POST requests to fetch weekly OTC data, resulting in poor performance. This commit adds async equivalents (aget_finra_weeks, aget_finra_data, aget_full_data) that use the platform's amake_request utility and asyncio.gather to fetch all weeks concurrently through a shared aiohttp session. The FinraOTCAggregateFetcher now uses aextract_data, which is natively supported by the platform's Fetcher base class. The pre-flight GET to finra.org for Cloudflare cookie establishment is preserved in the async path. * fix(finra): return all rows from data fetching instead of just the first one * run black, add symbol field to data model, and recapture test cassette * mypy --------- Co-authored-by: Danglewood <85772166+deeleeramone@users.noreply.github.com>