#include "rate_controlling_url_async_fetcher.h"
Public Member Functions | |
RateControllingUrlAsyncFetcher (UrlAsyncFetcher *fetcher, int max_global_queue_size, int per_host_outgoing_request_threshold, int per_host_queued_request_threshold, ThreadSystem *thread_system, Statistics *statistics) | |
virtual bool | SupportsHttps () const |
virtual bool | Fetch (const GoogleString &url, MessageHandler *message_handler, AsyncFetch *fetch) |
Static Public Member Functions | |
static void | Initialize (Statistics *statistics) |
Initializes statistics variables associated with this class. | |
Static Public Attributes | |
static const char | kQueuedFetchCount [] |
static const char | kDroppedFetchCount [] |
static const char | kCurrentGlobalFetchQueueSize [] |
Friends | |
class | CustomFetch |
Fetcher which limits the number of outgoing fetches per domain. If the fetch is for a user-facing request, this sends the request out anyway and updates the count for number of outgoing fetches. For non-user facing requests, this checks that the number of outgoing fetches for this domain is less than the limit. If less than the limit, it sends the fetch out and updates the count. If greater than the per-domain limit, and if the global queue size is within the limit, it queues the request up. However, if the global queue size is above the limit, it drops the request.
virtual bool net_instaweb::RateControllingUrlAsyncFetcher::Fetch | ( | const GoogleString & | url, | |
MessageHandler * | message_handler, | |||
AsyncFetch * | fetch | |||
) | [virtual] |
Fetch with AsyncFetch interface.
Default implementation uses StreamingFetch method and calls HeadersComplete right before the first call to Write, Flush or Done.
Future implementations ought to call HeadersComplete directly.
Return value is the same as StreamingFetch. (Returns true iff callback has already been called by the time Fetch returns.)
Reimplemented from net_instaweb::UrlAsyncFetcher.
virtual bool net_instaweb::RateControllingUrlAsyncFetcher::SupportsHttps | ( | ) | const [inline, virtual] |
Determine if the fetcher supports fetching using HTTPS. By default we assume a fetcher can.
Reimplemented from net_instaweb::UrlAsyncFetcher.