net_instaweb::SerfUrlAsyncFetcher Class Reference

#include "serf_url_async_fetcher.h"

Inheritance diagram for net_instaweb::SerfUrlAsyncFetcher:
net_instaweb::UrlPollableAsyncFetcher net_instaweb::UrlAsyncFetcher

List of all members.

Public Types

enum  WaitChoice { kThreadedOnly, kMainlineOnly, kThreadedAndMainline }

Public Member Functions

 SerfUrlAsyncFetcher (const char *proxy, apr_pool_t *pool, ThreadSystem *thread_system, Statistics *statistics, Timer *timer, int64 timeout_ms, MessageHandler *handler)
 SerfUrlAsyncFetcher (SerfUrlAsyncFetcher *parent, const char *proxy)
virtual void ShutDown ()
virtual bool SupportsHttps () const
virtual bool Fetch (const GoogleString &url, MessageHandler *message_handler, AsyncFetch *callback)
virtual int Poll (int64 max_wait_ms)
bool WaitForActiveFetches (int64 max_milliseconds, MessageHandler *message_handler, WaitChoice wait_choice)
void FetchComplete (SerfFetch *fetch)
apr_pool_t * pool () const
serf_context_t * serf_context () const
void PrintActiveFetches (MessageHandler *handler) const
virtual int64 timeout_ms ()
ThreadSystemthread_system ()
void set_force_threaded (bool x)
void set_list_outstanding_urls_on_error (bool x)

Static Public Member Functions

static void Initialize (Statistics *statistics)

Protected Types

typedef Pool< SerfFetch > SerfFetchPool
typedef std::vector< SerfFetch * > FetchVector

Protected Member Functions

void Init (apr_pool_t *parent_pool, const char *proxy)
bool SetupProxy (const char *proxy)
bool StartFetch (SerfFetch *fetch)
virtual bool AnyPendingFetches ()
int ApproximateNumActiveFetches ()
void CancelActiveFetches ()
void CancelActiveFetchesMutexHeld ()
bool WaitForActiveFetchesHelper (int64 max_ms, MessageHandler *message_handler)
void CleanupFetchesWithErrors ()
bool shutdown () const
 These must be accessed with mutex_ held.
void set_shutdown (bool s)

Protected Attributes

apr_pool_t * pool_
ThreadSystemthread_system_
Timertimer_
ThreadSystem::CondvarCapableMutexmutex_
 mutex_ protects serf_context_ and active_fetches_.
serf_context_t * serf_context_
SerfFetchPool active_fetches_
SerfFetchPool completed_fetches_
SerfThreadedFetcher * threaded_fetcher_
Variableactive_count_

Friends

class SerfFetch
 To access stats variables below.

Detailed Description

Todo:
TODO(sligocki): Serf does not seem to act appropriately in IPv6 environments, fix and test this. Specifically: (1) It does not attempt to fall-back to IPv4 if IPv6 connection fails; (2) It may not correctly signal failure, which causes the incoming connection to hang.

Member Function Documentation

virtual bool net_instaweb::SerfUrlAsyncFetcher::AnyPendingFetches (  )  [protected, virtual]

AnyPendingFetches is accurate only at the time of call; this is used conservatively during shutdown. It counts fetches that have been requested by some thread, and can include fetches for which no action has yet been taken (ie fetches that are not active).

int net_instaweb::SerfUrlAsyncFetcher::ApproximateNumActiveFetches (  )  [protected]

ApproximateNumActiveFetches can under- or over-count and is used only for error reporting.

void net_instaweb::SerfUrlAsyncFetcher::CleanupFetchesWithErrors (  )  [protected]

This cleans up the serf resources for fetches that errored out. Must be called only immediately after running the serf event loop. Must be called with mutex_ held.

virtual bool net_instaweb::SerfUrlAsyncFetcher::Fetch ( const GoogleString url,
MessageHandler message_handler,
AsyncFetch fetch 
) [virtual]

Fetch with AsyncFetch interface.

Default implementation uses StreamingFetch method and calls HeadersComplete right before the first call to Write, Flush or Done.

Future implementations ought to call HeadersComplete directly.

Return value is the same as StreamingFetch. (Returns true iff callback has already been called by the time Fetch returns.)

Todo:
TODO(sligocki): Stick all other params into AsyncFetch object (url, request_headers, response_headers, message_handler).

Reimplemented from net_instaweb::UrlAsyncFetcher.

void net_instaweb::SerfUrlAsyncFetcher::FetchComplete ( SerfFetch *  fetch  ) 

Remove the completed fetch from the active fetch set, and put it into a completed fetch list to be cleaned up.

virtual int net_instaweb::SerfUrlAsyncFetcher::Poll ( int64  max_wait_ms  )  [virtual]

Poll the active fetches, returning the number of fetches still outstanding.

Implements net_instaweb::UrlPollableAsyncFetcher.

void net_instaweb::SerfUrlAsyncFetcher::set_force_threaded ( bool  x  )  [inline]

By default, the Serf fetcher will call UrlAsyncFetcher::Callback::EnableThreaded() to determine whether a particular URL fetch should be executed in the fetcher thread.

Setting this variable causes the fetches to be threaded independent of the value of UrlAsyncFetcher::Callback::EnableThreaded().

void net_instaweb::SerfUrlAsyncFetcher::set_list_outstanding_urls_on_error ( bool  x  ) 

Indicates that Serf should enumerate failing URLs whenever the underlying Serf library reports an error.

virtual void net_instaweb::SerfUrlAsyncFetcher::ShutDown (  )  [virtual]

Stops all active fetches and prevents further fetches from starting (they will instead quickly call back to ->Done(false).

Reimplemented from net_instaweb::UrlAsyncFetcher.

bool net_instaweb::SerfUrlAsyncFetcher::StartFetch ( SerfFetch *  fetch  )  [protected]

Start a SerfFetch. Takes ownership of fetch and makes sure callback is called even if fetch fails to start.

mutex_ must be held before calling StartFetch.

virtual bool net_instaweb::SerfUrlAsyncFetcher::SupportsHttps (  )  const [inline, virtual]

Determine if the fetcher supports fetching using HTTPS. By default we assume a fetcher can.

Reimplemented from net_instaweb::UrlAsyncFetcher.

virtual int64 net_instaweb::SerfUrlAsyncFetcher::timeout_ms (  )  [inline, virtual]

Returns a maximum time that we will allow fetches to take, or kUnspecifiedTimeout (the default) if we don't promise to timeout fetches.

Reimplemented from net_instaweb::UrlAsyncFetcher.


Member Data Documentation

This is protected because it's updated along with active_fetches_, which happens in subclass SerfThreadedFetcher as well as this class.


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines
Generated on Tue May 29 16:33:46 2012 for Page Speed Optimization Libraries by  doxygen 1.6.3