net_instaweb::ResourceManager Class Reference

#include "resource_manager.h"

Inheritance diagram for net_instaweb::ResourceManager:
net_instaweb::ApacheResourceManager

List of all members.

Public Member Functions

 ResourceManager (RewriteDriverFactory *factory)
void SetDefaultLongCacheHeaders (const ContentType *content_type, ResponseHeaders *header) const
void SetDefaultLongCacheHeadersWithCharset (const ContentType *content_type, StringPiece charset, ResponseHeaders *header) const
void SetContentType (const ContentType *content_type, ResponseHeaders *header)
 Changes the content type of a pre-initialized header.
void set_filename_prefix (const StringPiece &file_prefix)
void set_statistics (Statistics *x)
void set_rewrite_stats (RewriteStats *x)
void set_relative_path (bool x)
void set_lock_manager (NamedLockManager *x)
void set_http_cache (HTTPCache *x)
void set_page_property_cache (PropertyCache *x)
void set_client_property_cache (PropertyCache *x)
void set_metadata_cache (CacheInterface *x)
void set_message_handler (MessageHandler *x)
StringPiece filename_prefix () const
Statisticsstatistics () const
NamedLockManagerlock_manager () const
RewriteDriverFactoryfactory () const
ThreadSynchronizerthread_synchronizer ()
bool Write (const ResourceVector &inputs, const StringPiece &contents, const ContentType *content_type, StringPiece charset, OutputResource *output, MessageHandler *handler)
void ApplyInputCacheControl (const ResourceVector &inputs, ResponseHeaders *headers)
bool IsPagespeedResource (const GoogleUrl &url)
 Is this URL a ref to a Pagespeed resource?
bool IsImminentlyExpiring (int64 start_date_ms, int64 expire_ms) const
void ComputeSignature (RewriteOptions *rewrite_options)
Hasherhasher () const
const Hasherlock_hasher () const
const Hashercontents_hasher () const
FileSystemfile_system ()
void set_file_system (FileSystem *fs)
FilenameEncoderfilename_encoder () const
void set_filename_encoder (FilenameEncoder *x)
UrlNamerurl_namer () const
void set_url_namer (UrlNamer *n)
StaticJavascriptManagerstatic_javascript_manager () const
void set_static_javascript_manager (StaticJavascriptManager *manager)
Schedulerscheduler () const
void set_scheduler (Scheduler *s)
bool has_default_system_fetcher ()
UrlAsyncFetcherDefaultSystemFetcher ()
Timertimer () const
HTTPCachehttp_cache () const
PropertyCachepage_property_cache () const
PropertyCacheclient_property_cache () const
CriticalImagesFindercritical_images_finder () const
void set_critical_images_finder (CriticalImagesFinder *finder)
const UserAgentMatcheruser_agent_matcher ()
void set_user_agent_matcher (UserAgentMatcher *n)
BlinkCriticalLineDataFinderblink_critical_line_data_finder () const
void set_blink_critical_line_data_finder (BlinkCriticalLineDataFinder *finder)
CacheInterfacemetadata_cache ()
bool store_outputs_in_file_system ()
void set_store_outputs_in_file_system (bool store)
void RefreshIfImminentlyExpiring (Resource *resource, MessageHandler *handler) const
RewriteStatsrewrite_stats () const
MessageHandlermessage_handler () const
void ReadAsync (Resource::NotCacheablePolicy not_cacheable_policy, Resource::AsyncCallback *callback)
NamedLockMakeCreationLock (const GoogleString &name)
bool TryLockForCreation (NamedLock *creation_lock)
 Attempt to obtain a named lock without blocking. Return true if we do so.
void LockForCreation (NamedLock *creation_lock, QueuedWorkerPool::Sequence *worker, Function *callback)
void set_hasher (Hasher *hasher)
 Setters should probably only be used in testing.
void set_default_system_fetcher (UrlAsyncFetcher *fetcher)
bool HandleBeacon (const StringPiece &unparsed_url)
RewriteDriverdecoding_driver () const
RewriteOptionsglobal_options ()
RewriteOptionsNewOptions ()
 Makes a new, empty set of RewriteOptions.
RewriteDriverNewRewriteDriver ()
RewriteDriverNewUnmanagedRewriteDriver (bool is_custom, RewriteOptions *options)
RewriteDriverNewCustomRewriteDriver (RewriteOptions *custom_options)
void ReleaseRewriteDriver (RewriteDriver *rewrite_driver)
ThreadSystemthread_system ()
UsageDataReporterusage_data_reporter ()
void set_metadata_cache_readonly ()
bool metadata_cache_readonly () const
void ShutDownDrivers ()
void MergeNonCachingResponseHeaders (const ResourcePtr &input, const OutputResourcePtr &output)
void MergeNonCachingResponseHeaders (const ResponseHeaders &input_headers, ResponseHeaders *output_headers)
 Entry-point with the same functionality, exposed for easier testing.
QueuedWorkerPoolhtml_workers ()
 Pool of worker-threads that can be used to handle html-parsing.
QueuedWorkerPoolrewrite_workers ()
 Pool of worker-threads that can be used to handle resource rewriting.
QueuedWorkerPoollow_priority_rewrite_workers ()
size_t num_active_rewrite_drivers ()
void InitWorkersAndDecodingDriver ()
bool response_headers_finalized () const
void set_response_headers_finalized (bool x)

Static Public Member Functions

static bool IsExcludedAttribute (const char *attribute)

Static Public Attributes

static const int64 kGeneratedMaxAgeMs
 The lifetime for cache-extended generated resources, in milliseconds.
static const char kResourceEtagValue []
static const char kCacheKeyResourceNamePrefix []
static const char kStatisticsGroup []
 Default statistics group name.

Friends

class ResourceManagerTest

Detailed Description

Todo:
TODO(jmarantz): Rename this class to ServerContext, as it no longer contains much logic about resources -- that's been moved to RewriteDriver, which should be renamed RequestContext.

Member Function Documentation

void net_instaweb::ResourceManager::ApplyInputCacheControl ( const ResourceVector &  inputs,
ResponseHeaders headers 
)

Computes the most restrictive Cache-Control intersection of the input resources, and the provided headers, and sets that cache-control on the provided headers. Does nothing if all of the resources are fully cacheable, since in that case we will want to cache-extend.

Disregards Cache-Control directives other than max-age, no-cache, no-store, and private, and strips them if any resource is no-cache or private. By assumption, a resource can only be no-store if it is also no-cache.

RewriteDriver* net_instaweb::ResourceManager::decoding_driver (  )  const [inline]

Returns a RewriteDriver* suitable for decoding pagespeed URLs. This driver does not necessarily have up-to-date Options, especially in the case of a cache flush. However, this doesn't affect its ability to correctly decode options.

UrlAsyncFetcher* net_instaweb::ResourceManager::DefaultSystemFetcher (  )  [inline]

Note: for rewriting user content, you want to use RewriteDriver's async_fetcher() instead, as it may apply session-specific optimizations.

RewriteOptions* net_instaweb::ResourceManager::global_options (  ) 

Returns a pointer to the master global_options. These are not used directly in RewriteDrivers, but are Cloned into the drivers as they are created. We generally do not expect global_options() to change once the system is processing requests, except in Apache when someone does a cache-flush by touching a file "cache.flush" in the file-cache directory.

bool net_instaweb::ResourceManager::HandleBeacon ( const StringPiece &  unparsed_url  ) 

Handles an incoming beacon request by incrementing the appropriate variables. Returns true if the url was parsed and handled correctly; in this case a 204 No Content response should be sent. Returns false if the url could not be parsed; in this case the request should be declined.

Hasher* net_instaweb::ResourceManager::hasher (  )  const [inline]
Todo:
TODO(jmarantz): check thread safety in Apache.
void net_instaweb::ResourceManager::InitWorkersAndDecodingDriver (  ) 

A ResourceManager may be created in one phase, and later populated with all its dependencies. This populates the worker threads and a RewriteDriver used just for quickly decoding (but not serving) URLs.

static bool net_instaweb::ResourceManager::IsExcludedAttribute ( const char *  attribute  )  [static]

Returns whether or not this attribute can be merged into headers without additional considerations.

bool net_instaweb::ResourceManager::IsImminentlyExpiring ( int64  start_date_ms,
int64  expire_ms 
) const

Returns true if the resource with given date and TTL is going to expire shortly and should hence be proactively re-fetched.

void net_instaweb::ResourceManager::LockForCreation ( NamedLock creation_lock,
QueuedWorkerPool::Sequence worker,
Function callback 
)

Attempt to obtain a named lock. When the lock has been obtained, queue the callback on the given worker Sequence. If the lock times out, cancel the callback, running the cancel on the worker.

QueuedWorkerPool* net_instaweb::ResourceManager::low_priority_rewrite_workers (  )  [inline]

Pool of worker-threads that can be used to handle low-priority/high CPU portions of resource rewriting.

NamedLock* net_instaweb::ResourceManager::MakeCreationLock ( const GoogleString name  ) 

Allocate an NamedLock to guard the creation of the given resource. If the object is expensive to create, this lock should be held during its creation to avoid multiple rewrites happening at once. The lock will be unlocked when creation_lock is reset or destructed.

void net_instaweb::ResourceManager::MergeNonCachingResponseHeaders ( const ResourcePtr input,
const OutputResourcePtr output 
) [inline]

Take any headers that are not caching-related, and not otherwise filled in by SetDefaultLongCacheHeaders or SetContentType, but were* set on the input resource, and copy them to the output resource. This allows user headers to be preserved. This must be called as needed by individual filters, prior to Write().

Note that this API is only usable for single-input rewriters. Combiners will need to execute some kind of merge, union, or intersection policy, if we wish to preserve origin response headers.

Note: this does not call ComputeCaching() on the output headers, so that method must be called prior to invoking any caching predicates on the output's ResponseHeader. In theory we shouldn't mark the caching bits dirty because we are only adding headers that will not affect caching, but at the moment the dirty-bit is set independent of that.

Todo:
TODO(jmarantz): avoid setting caching_dirty bit in ResponseHeaders when the header is not caching-related.
CacheInterface* net_instaweb::ResourceManager::metadata_cache (  )  [inline]

Cache for small non-HTTP objects.

Note that this might share namespace with the HTTP cache, so make sure your key names do not start with http://.

RewriteDriver* net_instaweb::ResourceManager::NewCustomRewriteDriver ( RewriteOptions custom_options  ) 

Like NewUnmanagedRewriteDriver, but uses supplied options instead of RewriteDriver's global_options().

NOTE: This does not merge custom_options with global_options(), the caller must do that if they want them merged.

Filters allocated using this mechanism have their filter-chain already frozen (see AddFilters()).

Takes ownership of 'custom_options'.

RewriteDriver* net_instaweb::ResourceManager::NewRewriteDriver (  ) 

Generates a new managed RewriteDriver using the RewriteOptions managed by this class. Each RewriteDriver is not thread-safe, but you can generate a RewriteDriver* for each thread. The returned drivers manage themselves: when the HTML parsing and rewriting is done they will be returned to the pool.

Filters allocated using this mechanism have their filter-chain already frozen (see AddFilters()).

RewriteDriver* net_instaweb::ResourceManager::NewUnmanagedRewriteDriver ( bool  is_custom,
RewriteOptions options 
)

Generates a new unmanaged RewriteDriver using the RewriteOptions managed by this class. Each RewriteDriver is not thread-safe, but you can generate a RewriteDriver* for each thread. The returned drivers must be explicitly deleted by the caller.

RewriteDrivers allocated using this mechanism have not yet frozen their filters, and so callers may explicitly enable individual filters on the driver -- beyond those indicated in the options. After all extra filters are added, AddFilters must be called to freeze them and instantiate the filter-chain.

Takes ownership of 'options'.

size_t net_instaweb::ResourceManager::num_active_rewrite_drivers (  ) 

Returns the number of rewrite drivers that we were aware of at the time of the call. This includes those created via NewCustomRewriteDriver and NewRewriteDriver, but not via NewUnmanagedRewriteDriver.

void net_instaweb::ResourceManager::ReadAsync ( Resource::NotCacheablePolicy  not_cacheable_policy,
Resource::AsyncCallback callback 
)

Loads contents of resource asynchronously, calling callback when done. If the resource contents are cached, the callback will be called directly, rather than asynchronously. The resource will be passed to the callback, with its contents and headers filled in.

void net_instaweb::ResourceManager::ReleaseRewriteDriver ( RewriteDriver rewrite_driver  ) 

Puts a RewriteDriver back on the free pool. This is intended to be called by a RewriteDriver on itself, once all pending activites on it have completed, including HTML Parsing (FinishParse) and all pending Rewrites.

Todo:
TODO(jmarantz): this cannot recycle RewriteDrivers with custom rewrite options, which is a potential performance issue for Apache installations that set custom options in .htaccess files, where essentially every RewriteDriver will be a custom driver. To resolve this we need to make a comparator for RewriteOptions so that we can determine option-equivalence and, potentially, keep free-lists for each unique option-set.
bool net_instaweb::ResourceManager::response_headers_finalized (  )  const [inline]

Determines whether we can assume that the response headers we see in rewrite_drivers when filters are applied reflect the final form from the origin. In proxy applications, this is generally true. But in Apache, it depends when the output_filter is applied relative to mod_headers and mod_expires.

The default-value is 'true'.

void net_instaweb::ResourceManager::set_metadata_cache_readonly (  )  [inline]

Calling this method will stop results of rewrites being cached in the metadata cache. This is meant for the shutdown sequence.

void net_instaweb::ResourceManager::SetDefaultLongCacheHeaders ( const ContentType content_type,
ResponseHeaders header 
) const [inline]

Set time and cache headers with long TTL (including Date, Last-Modified, Cache-Control, Etags, Expires).

Also sets Content-Type headers if content_type is provided. If content_type is null, the Content-Type header is omitted.

void net_instaweb::ResourceManager::SetDefaultLongCacheHeadersWithCharset ( const ContentType content_type,
StringPiece  charset,
ResponseHeaders header 
) const

As above, but also sets charset if it's non-empty and content_type is non-NULL.

void net_instaweb::ResourceManager::ShutDownDrivers (  ) 

Waits a bounded amount of time for all currently running jobs to complete. This is meant for use when shutting down processing, so that jobs running in background do not access objects that are about to be deleted. If there are long-running outstanding tasks, the drivers may stay running past this call.

Todo:
TODO(jmarantz): Change New*RewriteDriver() calls to return NULL when run after shutdown. This requires changing call-sites to null-check their drivers and gracefully fail.
bool net_instaweb::ResourceManager::store_outputs_in_file_system (  )  [inline]

Whether or not dumps of rewritten resources should be stored to the filesystem. This is meant for testing purposes only.

bool net_instaweb::ResourceManager::Write ( const ResourceVector &  inputs,
const StringPiece &  contents,
const ContentType content_type,
StringPiece  charset,
OutputResource output,
MessageHandler handler 
)

Writes the specified contents into the output resource, and marks it as optimized. 'inputs' described the input resources that were used to construct the output, and is used to determine whether the result can be safely cache extended and be marked publicly cacheable. 'content_type' and 'charset' specify the mimetype and encoding of the contents, and will help form the Content-Type header. 'charset' may be empty when not specified.

Note that this does not escape charset.

Callers should take care that dangerous types like 'text/html' do not sneak into content_type.


Member Data Documentation

This value is a shared constant so that it can also be used in the Apache-specific code that repairs our caching headers downstream of mod_headers.


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines
Generated on Tue May 29 16:34:03 2012 for Page Speed Optimization Libraries by  doxygen 1.6.3