Page Speed Optimization Libraries
1.8.31.3
|
#include "serf_url_async_fetcher.h"
Public Types | |
enum | WaitChoice { kThreadedOnly, kMainlineOnly, kThreadedAndMainline } |
Public Member Functions | |
SerfUrlAsyncFetcher (const char *proxy, apr_pool_t *pool, ThreadSystem *thread_system, Statistics *statistics, Timer *timer, int64 timeout_ms, MessageHandler *handler) | |
SerfUrlAsyncFetcher (SerfUrlAsyncFetcher *parent, const char *proxy) | |
virtual void | ShutDown () |
virtual bool | SupportsHttps () const |
virtual void | Fetch (const GoogleString &url, MessageHandler *message_handler, AsyncFetch *callback) |
int | Poll (int64 max_wait_ms) |
bool | WaitForActiveFetches (int64 max_milliseconds, MessageHandler *message_handler, WaitChoice wait_choice) |
void | FetchComplete (SerfFetch *fetch) |
void | ReportCompletedFetchStats (SerfFetch *fetch) |
Update the statistics object with results of the (completed) fetch. | |
apr_pool_t * | pool () const |
serf_context_t * | serf_context () const |
void | PrintActiveFetches (MessageHandler *handler) const |
virtual int64 | timeout_ms () |
ThreadSystem * | thread_system () |
void | set_list_outstanding_urls_on_error (bool x) |
bool | track_original_content_length () const |
void | set_track_original_content_length (bool x) |
bool | SetHttpsOptions (StringPiece directive) |
void | SetSslCertificatesDir (StringPiece dir) |
const GoogleString & | ssl_certificates_dir () const |
void | SetSslCertificatesFile (StringPiece file) |
const GoogleString & | ssl_certificates_file () const |
Static Public Member Functions | |
static void | InitStats (Statistics *statistics) |
static bool | ValidateHttpsOptions (StringPiece directive, GoogleString *error_message) |
Protected Types | |
typedef Pool< SerfFetch > | SerfFetchPool |
typedef std::vector< SerfFetch * > | FetchVector |
Protected Member Functions | |
bool | allow_https () const |
Determines whether https is allowed in the current configuration. | |
bool | allow_self_signed () const |
bool | allow_unknown_certificate_authority () const |
bool | allow_certificate_not_yet_valid () const |
void | set_https_options (uint32 https_options) |
void | Init (apr_pool_t *parent_pool, const char *proxy) |
bool | SetupProxy (const char *proxy) |
bool | StartFetch (SerfFetch *fetch) |
virtual bool | AnyPendingFetches () |
int | ApproximateNumActiveFetches () |
void | CancelActiveFetches () |
void | CancelActiveFetchesMutexHeld () |
bool | WaitForActiveFetchesHelper (int64 max_ms, MessageHandler *message_handler) |
void | CleanupFetchesWithErrors () |
bool | shutdown () const |
These must be accessed with mutex_ held. | |
void | set_shutdown (bool s) |
Protected Attributes | |
apr_pool_t * | pool_ |
ThreadSystem * | thread_system_ |
Timer * | timer_ |
ThreadSystem::CondvarCapableMutex * | mutex_ |
mutex_ protects serf_context_ and active_fetches_. | |
serf_context_t * | serf_context_ |
SerfFetchPool | active_fetches_ |
SerfFetchPool | completed_fetches_ |
SerfThreadedFetcher * | threaded_fetcher_ |
Variable * | active_count_ |
Friends | |
class | SerfFetch |
To access stats variables below. |
virtual bool net_instaweb::SerfUrlAsyncFetcher::AnyPendingFetches | ( | ) | [protected, virtual] |
AnyPendingFetches is accurate only at the time of call; this is used conservatively during shutdown. It counts fetches that have been requested by some thread, and can include fetches for which no action has yet been taken (ie fetches that are not active).
int net_instaweb::SerfUrlAsyncFetcher::ApproximateNumActiveFetches | ( | ) | [protected] |
ApproximateNumActiveFetches can under- or over-count and is used only for error reporting.
void net_instaweb::SerfUrlAsyncFetcher::CleanupFetchesWithErrors | ( | ) | [protected] |
This cleans up the serf resources for fetches that errored out. Must be called only immediately after running the serf event loop. Must be called with mutex_ held.
virtual void net_instaweb::SerfUrlAsyncFetcher::Fetch | ( | const GoogleString & | url, |
MessageHandler * | message_handler, | ||
AsyncFetch * | fetch | ||
) | [virtual] |
Asynchronously fetch a URL, set the response headers and stream the contents to fetch and call fetch->Done() when the fetch finishes.
There is an unchecked contract that response_headers are set before the response_writer or callback are used. Caution, several implementations do not satisfy this contract (but should).
TODO(sligocki): GoogleString -> GoogleUrl or at least StringPiece.
TODO(sligocki): Include the URL in the fetch, like the request headers.
Implements net_instaweb::UrlAsyncFetcher.
void net_instaweb::SerfUrlAsyncFetcher::FetchComplete | ( | SerfFetch * | fetch | ) |
Remove the completed fetch from the active fetch set, and put it into a completed fetch list to be cleaned up.
int net_instaweb::SerfUrlAsyncFetcher::Poll | ( | int64 | max_wait_ms | ) |
Indicates that Serf should enumerate failing URLs whenever the underlying Serf library reports an error.
bool net_instaweb::SerfUrlAsyncFetcher::SetHttpsOptions | ( | StringPiece | directive | ) |
Indicates that direct HTTPS fetching should be allowed, and how picky to be about certificates. The directive is a comma separated list of these keywords: enable disable allow_self_signed allow_unknown_certificate_authority allow_certificate_not_yet_valid Returns 'false' if the directive does not parse properly.
virtual void net_instaweb::SerfUrlAsyncFetcher::ShutDown | ( | ) | [virtual] |
Stops all active fetches and prevents further fetches from starting (they will instead quickly call back to ->Done(false).
Reimplemented from net_instaweb::UrlAsyncFetcher.
bool net_instaweb::SerfUrlAsyncFetcher::StartFetch | ( | SerfFetch * | fetch | ) | [protected] |
Start a SerfFetch. Takes ownership of fetch and makes sure callback is called even if fetch fails to start.
mutex_ must be held before calling StartFetch.
virtual bool net_instaweb::SerfUrlAsyncFetcher::SupportsHttps | ( | ) | const [virtual] |
Determine if the fetcher supports fetching using HTTPS. By default we assume a fetcher can.
Reimplemented from net_instaweb::UrlAsyncFetcher.
virtual int64 net_instaweb::SerfUrlAsyncFetcher::timeout_ms | ( | ) | [inline, virtual] |
Returns a maximum time that we will allow fetches to take, or kUnspecifiedTimeout (the default) if we don't promise to timeout fetches.
Reimplemented from net_instaweb::UrlAsyncFetcher.
bool net_instaweb::SerfUrlAsyncFetcher::track_original_content_length | ( | ) | const [inline] |
Indicates that Serf should track the original content length for fetched resources.
static bool net_instaweb::SerfUrlAsyncFetcher::ValidateHttpsOptions | ( | StringPiece | directive, |
GoogleString * | error_message | ||
) | [inline, static] |
Validates the correctness of an https directive. Exposed as a static method for early exit on mis-specified pagespeed.conf.
Variable* net_instaweb::SerfUrlAsyncFetcher::active_count_ [protected] |
This is protected because it's updated along with active_fetches_, which happens in subclass SerfThreadedFetcher as well as this class.