Page Speed Optimization Libraries  1.3.25.1
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines
Classes | Public Types | Public Member Functions | Static Public Member Functions | Static Public Attributes | Protected Member Functions | Protected Attributes
net_instaweb::RewriteTestBase Class Reference
Inheritance diagram for net_instaweb::RewriteTestBase:
net_instaweb::RewriteOptionsTestBase< OptionsClass > net_instaweb::HtmlParseTestBaseNoAlloc net_instaweb::CriticalImagesFinderTestBase net_instaweb::CssRewriteTestBase net_instaweb::ProxyInterfaceTestBase net_instaweb::RewriteContextTestBase

List of all members.

Classes

class  CssLink
 Representation for a CSS <link> tag. More...

Public Types

enum  ActiveServerFlag { kPrimary, kSecondary }

Public Member Functions

 RewriteTestBase (Statistics *statistics)
 RewriteTestBase (std::pair< TestRewriteDriverFactory *, TestRewriteDriverFactory * > factories)
virtual void SetUp ()
virtual void TearDown ()
virtual bool AddBody () const
virtual TestRewriteDriverFactoryMakeTestFactory ()
void AddRecompressImageFilters ()
void AddFilter (RewriteOptions::Filter filter)
 Add a single rewrite filter to rewrite_driver_.
void AddOtherFilter (RewriteOptions::Filter filter)
 Add a single rewrite filter to other_rewrite_driver_.
void AddRewriteFilter (RewriteFilter *filter)
void AddFetchOnlyRewriteFilter (RewriteFilter *filter)
void AddOtherRewriteFilter (RewriteFilter *filter)
void SetBaseUrlForFetch (const StringPiece &url)
ResourcePtr CreateResource (const StringPiece &base, const StringPiece &url)
Timertimer ()
void AppendDefaultHeaders (const ContentType &content_type, GoogleString *text)
 Append default headers to the given string.
void AppendDefaultHeaders (const ContentType &content_type, int64 original_content_length, GoogleString *text)
void ServeResourceFromManyContexts (const GoogleString &resource_url, const StringPiece &expected_content)
void ServeResourceFromManyContextsWithUA (const GoogleString &resource_url, const StringPiece &expected_content, const StringPiece &user_agent)
void ServeResourceFromNewContext (const GoogleString &resource_url, const StringPiece &expected_content)
virtual RewriteDriverhtml_parse ()
void DefaultResponseHeaders (const ContentType &content_type, int64 ttl_sec, ResponseHeaders *response_headers)
 Set default headers for a resource with content_type and Cache ttl_sec.
bool FetchResource (const StringPiece &path, const StringPiece &filter_id, const StringPiece &name, const StringPiece &ext, GoogleString *content)
bool FetchResource (const StringPiece &path, const StringPiece &filter_id, const StringPiece &name, const StringPiece &ext, GoogleString *content, ResponseHeaders *response)
bool FetchResourceUrl (const StringPiece &url, GoogleString *content, ResponseHeaders *response)
bool FetchResourceUrl (const StringPiece &url, GoogleString *content)
bool TryFetchResource (const StringPiece &url)
 Just check if we can fetch a resource successfully, ignore response.
void SetUseManagedRewriteDrivers (bool use_managed_rewrite_drivers)
GoogleString CssLinkHref (const StringPiece &url)
void CollectCssLinks (const StringPiece &id, const StringPiece &html, StringVector *css_links)
 Collects the hrefs for all CSS <link>s on the page.
void CollectCssLinks (const StringPiece &id, const StringPiece &html, CssLink::Vector *css_links)
 Collects all information about CSS links into a CssLink::Vector.
void EncodePathAndLeaf (const StringPiece &filter_id, const StringPiece &hash, const StringVector &name_vector, const StringPiece &ext, ResourceNamer *namer)
 Encode the given name (path + leaf) using the given pagespeed attributes.
StringVector MultiUrl (const StringPiece &url1)
StringVector MultiUrl (const StringPiece &url1, const StringPiece &url2)
StringVector MultiUrl (const StringPiece &url1, const StringPiece &url2, const StringPiece &url3)
StringVector MultiUrl (const StringPiece &url1, const StringPiece &url2, const StringPiece &url3, const StringPiece &url4)
GoogleString Encode (const StringPiece &path, const StringPiece &filter_id, const StringPiece &hash, const StringPiece &name, const StringPiece &ext)
GoogleString Encode (const StringPiece &path, const StringPiece &filter_id, const StringPiece &hash, const StringVector &name_vector, const StringPiece &ext)
GoogleString EncodeNormal (const StringPiece &path, const StringPiece &filter_id, const StringPiece &hash, const StringPiece &name, const StringPiece &ext)
 Same as Encode but specifically using UrlNamer not TestUrlNamer.
GoogleString EncodeNormal (const StringPiece &path, const StringPiece &filter_id, const StringPiece &hash, const StringVector &name_vector, const StringPiece &ext)
GoogleString EncodeWithBase (const StringPiece &base, const StringPiece &path, const StringPiece &filter_id, const StringPiece &hash, const StringPiece &name, const StringPiece &ext)
GoogleString EncodeWithBase (const StringPiece &base, const StringPiece &path, const StringPiece &filter_id, const StringPiece &hash, const StringVector &name_vector, const StringPiece &ext)
GoogleString AddOptionsToEncodedUrl (const StringPiece &url, const StringPiece &options)
 Takes an already-encoded URL and adds options to to it.
void SetupWaitFetcher ()
void CallFetcherCallbacks ()
RewriteOptionsoptions ()
RewriteOptionsother_options ()
void TestServeFiles (const ContentType *content_type, const StringPiece &filter_id, const StringPiece &rewritten_ext, const StringPiece &orig_name, const StringPiece &orig_content, const StringPiece &rewritten_name, const StringPiece &rewritten_content)
 Helper method to test all manner of resource serving from a filter.
TestRewriteDriverFactoryfactory ()
TestRewriteDriverFactoryother_factory ()
void UseMd5Hasher ()
void SetDefaultLongCacheHeaders (const ContentType *content_type, ResponseHeaders *header)
void SetFetchResponse (const StringPiece &url, const ResponseHeaders &response_header, const StringPiece &response_body)
void SetResponseWithDefaultHeaders (const StringPiece &relative_url, const ContentType &content_type, const StringPiece &content, int64 ttl_sec)
 Add content to mock fetcher (with default headers).
bool LoadFile (const StringPiece &filename, GoogleString *contents)
void AddFileToMockFetcher (const StringPiece &url, const StringPiece &filename, const ContentType &content_type, int64 ttl_sec)
 Add the contents of a file to mock fetcher (with default headers).
void AddToResponse (const StringPiece &url, const StringPiece &name, const StringPiece &value)
void SetFetchResponse404 (const StringPiece &url)
void SetFetchFailOnUnexpected (bool fail)
void FetcherUpdateDateHeaders ()
void ClearFetcherResponses ()
virtual void ClearStats ()
void ClearRewriteDriver ()
MockUrlFetchermock_url_fetcher ()
Hasherhasher ()
DelayCachedelay_cache ()
LRUCachelru_cache ()
Statisticsstatistics ()
MemFileSystemfile_system ()
HTTPCachehttp_cache ()
PropertyCachepage_property_cache ()
MockMessageHandlermessage_handler ()
RewriteDriverrewrite_driver ()
RewriteDriverother_rewrite_driver ()
MockSchedulermock_scheduler ()
 The scheduler used by rewrite_driver.
int64 start_time_ms () const
bool ReadFile (const char *filename, GoogleString *contents)
bool WriteFile (const char *filename, const StringPiece &contents)
ServerContextserver_context ()
ServerContextother_server_context ()
CountingUrlAsyncFetchercounting_url_async_fetcher ()
void SetMockHashValue (const GoogleString &value)
void SetCacheDelayUs (int64 delay_us)
RewriteDriverMakeDriver (ServerContext *server_context, RewriteOptions *options)
GoogleString AbsolutifyUrl (const StringPiece &in)
 Converts a potentially relative URL off kTestDomain to absolute if needed.
void TestRetainExtraHeaders (const StringPiece &name, const StringPiece &filter_id, const StringPiece &ext)
const UrlSegmentEncoderFindEncoder (const StringPiece &id) const
void SetUseTestUrlNamer (bool use_test_url_namer)
 Switch url namers as specified.
GoogleString EncodeCssName (const StringPiece &name, bool supports_webp, bool can_inline)
bool ReadIfCached (const ResourcePtr &resource)
void InitiateResourceRead (const ResourcePtr &resource)
HTTPCache::FindResult HttpBlockingFind (const GoogleString &key, HTTPCache *http_cache, HTTPValue *value_out, ResponseHeaders *headers)
HTTPCache::FindResult HttpBlockingFindStatus (const GoogleString &key, HTTPCache *http_cache)
void SetXhtmlMimetype ()
 Sets the response-headers Content-Type to "application/xhtml+xml".
void SetHtmlMimetype ()
 Sets the response-headers Content-Type to "text/html".
void SetMimetype (const StringPiece &mimetype)
 Sets the response-headers Content-Type as specified.
void CheckFetchFromHttpCache (StringPiece url, StringPiece expected_contents, int64 expected_expiration_ms)
void SetupCohort (PropertyCache *cache, const GoogleString &cohort)
 Setup statistics for the given cohort and add it to the give PropertyCache.
MockPropertyPageNewMockPage (const StringPiece &key)
 Returns a new mock property page for the page property cache.
MockPropertyPageNewMockClientPage (const StringPiece &key)
 Returns a new mock property page for the client property cache.

Static Public Member Functions

static GoogleString ChangeSuffix (GoogleString old_url, bool append_new_suffix, StringPiece old_suffix, StringPiece new_suffix)

Static Public Attributes

static const char kTestData []
 Testdata directory.

Protected Member Functions

void Init ()
virtual RequestContextPtr CreateRequestContext ()
void CallFetcherCallbacksForDriver (WaitUrlAsyncFetcher *fetcher, RewriteDriver *driver)
void PopulateDefaultHeaders (const ContentType &content_type, int64 original_content_length, ResponseHeaders *headers)
void SetActiveServer (ActiveServerFlag server_to_use)
void AdvanceTimeUs (int64 delay_ms)
void AdvanceTimeMs (int64 delay_ms)
void SetTimeUs (int64 time_us)
void SetTimeMs (int64 time_ms)
void AdjustTimeUsWithoutWakingAlarms (int64 time_us)
 Adjusts time ignoring any scheduler callbacks. Use with caution.
LoggingInfo * logging_info ()
GoogleString AppliedRewriterStringFromLog ()
void SetCurrentUserAgent (const StringPiece &user_agent)
 Sets current_user_agent_.

Protected Attributes

MockUrlFetcher mock_url_fetcher_
 The mock fetcher & stats are global across all Factories used in the tests.
scoped_ptr< Statisticsstatistics_
scoped_ptr
< TestRewriteDriverFactory
factory_
scoped_ptr
< TestRewriteDriverFactory
other_factory_
ServerContextserver_context_
RewriteDriverrewrite_driver_
ServerContextother_server_context_
RewriteDriverother_rewrite_driver_
scoped_ptr< HtmlWriterFilterother_html_writer_filter_
ActiveServerFlag active_server_
bool use_managed_rewrite_drivers_
StringPiece current_user_agent_
MD5Hasher md5_hasher_
RewriteOptionsoptions_
 owned by rewrite_driver_.
RewriteOptionsother_options_
 owned by other_rewrite_driver_.
UrlSegmentEncoder default_encoder_
ResponseHeaders response_headers_

Member Enumeration Documentation

Specifies which server should be "active" in that rewrites and fetches will use it. The data members affected are those returned by:

  • factory() / other_factory()
  • server_context() / other_server_context()
  • rewrite_driver() / other_rewrite_driver()
  • options() / other_options()
Enumerator:
kPrimary 

Use the normal data members.

kSecondary 

Use all the other_ data members.


Constructor & Destructor Documentation

net_instaweb::RewriteTestBase::RewriteTestBase ( std::pair< TestRewriteDriverFactory *, TestRewriteDriverFactory * >  factories) [explicit]

Specifies alternate factories to be initialized on construction. By default, TestRewriteDriverFactory is used, but you can employ your own subclass of TestRewriteDriverFactory using this constructor. If you do, you probably also want to override MakeTestFactory.


Member Function Documentation

virtual bool net_instaweb::RewriteTestBase::AddBody ( ) const [inline, virtual]

In this set of tests, we will provide explicit body tags, so the test harness should not add them in for our convenience. It can go ahead and add the <html> and </html>, however.

Implements net_instaweb::HtmlParseTestBaseNoAlloc.

Reimplemented in net_instaweb::RewriteContextTestBase.

Adds a custom rewriter filter but does not register it for HTML rewriting, only for fetches.

Add a custom rewrite filter (one without a corresponding option) to other_rewrite_driver and enable it.

Adds kRecompressJpeg, kRecompressPng, kRecompressWebp, kConvertPngToJpeg, kConvertJpegToWebp and kConvertGifToPng.

Add a custom rewrite filter (one without a corresponding option) to rewrite_driver and enable it.

void net_instaweb::RewriteTestBase::AdvanceTimeUs ( int64  delay_ms) [protected]

Advances time forward using the mock scheduler. Note that time is not advanced directly in the mock_timer; the scheduler must be used.

void net_instaweb::RewriteTestBase::AppendDefaultHeaders ( const ContentType content_type,
int64  original_content_length,
GoogleString text 
)

Append default headers to the given string, including X-Original-Content-Length for tests that depend on this.

Convenience method for retrieving the computed applied rewriters string from the current request context's log record. Thread-safe.

Calls callbacks on given wait fetcher, making sure to properly synchronize with async rewrite flows given driver.

static GoogleString net_instaweb::RewriteTestBase::ChangeSuffix ( GoogleString  old_url,
bool  append_new_suffix,
StringPiece  old_suffix,
StringPiece  new_suffix 
) [static]

If append_new_suffix is true, appends new_suffix to old_url. If append_new_suffix is false, replaces old_suffix at the end of old_url with new_suffix. Either way, precondition: old_url ends with old_suffix

void net_instaweb::RewriteTestBase::CheckFetchFromHttpCache ( StringPiece  url,
StringPiece  expected_contents,
int64  expected_expiration_ms 
)

Verifies that the specified URL can be fetched from HTTP cache, and that its cache TTL and contents are as specified.

Calls Clear() on the rewrite driver and does any other necessary clean-up so the driver is okay for a test to reuse.

Override this if the test fixture needs to use a different RequestContext subclass.

GoogleString net_instaweb::RewriteTestBase::Encode ( const StringPiece &  path,
const StringPiece &  filter_id,
const StringPiece &  hash,
const StringPiece &  name,
const StringPiece &  ext 
) [inline]

Helper function to encode a resource name from its pieces using whatever encoding we are testing, either UrlNamer or TestUrlNamer.

GoogleString net_instaweb::RewriteTestBase::EncodeCssName ( const StringPiece &  name,
bool  supports_webp,
bool  can_inline 
)

Helper function which instantiates an encoder, collects the required arguments and calls the virtual Encode().

GoogleString net_instaweb::RewriteTestBase::EncodeWithBase ( const StringPiece &  base,
const StringPiece &  path,
const StringPiece &  filter_id,
const StringPiece &  hash,
const StringPiece &  name,
const StringPiece &  ext 
) [inline]

Same as Encode but specifying the base URL (which is used by TestUrlNamer but is unused by UrlNamer so for it results in exactly the same as Encode).

bool net_instaweb::RewriteTestBase::FetchResource ( const StringPiece &  path,
const StringPiece &  filter_id,
const StringPiece &  name,
const StringPiece &  ext,
GoogleString content 
)

Helper function to test resource fetching, returning true if the fetch succeeded, and modifying content. It is up to the caller to EXPECT_TRUE on the status and EXPECT_EQ on the content.

const UrlSegmentEncoder* net_instaweb::RewriteTestBase::FindEncoder ( const StringPiece &  id) const

Find the segment-encoder for the filter found via 'id'. Some test filters are not registered with RewriteDriver so for those we use the default encoder.

This definition is required by HtmlParseTestBase which defines this as pure abstract, so that the test subclass can define how it instantiates HtmlParse.

Implements net_instaweb::HtmlParseTestBaseNoAlloc.

While our production cache model is non-blocking, we use an in-memory LRU for tests that calls its callback directly from Get. Thus we can make a convenient blocking cache wrapper to make it easier to write tests.

The same as the above function, but doesn't need an HTTPValue or ResponseHeaders.

Variation on ReadIfCached that is used when we expect the resource not to be in present in cache, but instead we are looking to initiate the resource-rewrite process so that a subsequent call to ReadIfCached succeeds.

bool net_instaweb::RewriteTestBase::LoadFile ( const StringPiece &  filename,
GoogleString contents 
)

Load a test file (from testdata/) into 'contents', returning false on failure.

LoggingInfo* net_instaweb::RewriteTestBase::logging_info ( ) [protected]

Convenience method to pull the logging info proto out of the current request context's log record. The request context owns the log record, and if the log record has a non-NULL mutex, it will need to be locked for this call.

Creates a RewriteDriver using the passed-in options, object, but does *not* finalize the driver. This gives individual _test.cc files the chance to add filters to the options prior to calling driver->AddFilters().

Makes a TestRewriteDriverFactory. This can be overridden in subclasses if you need a factory with special properties.

Todo:
TODO(jmarantz): This is currently only used in ServeResourceFromNewContext, but should be used for factory_ and other_factory_. This would requuire a refactor, because those are created at construction; too early for subclass overrides to take effect. To deal with that, an alternate constructor is provided above so that the proper sort of factories can be passed in.
void net_instaweb::RewriteTestBase::PopulateDefaultHeaders ( const ContentType content_type,
int64  original_content_length,
ResponseHeaders headers 
) [protected]

Populate the given headers based on the content type and original content length information.

Helper function for legacy tests that used this now-extinct interface. In general we don't support this flow in production but we rely on it in tests for obliquely covering some cases relating to resource pathnames.

Todo:
TODO(jmarantz): These abstractions are not satisfactory long-term where we want to have driver-lifetime in tests be reflective of how servers work. But for now we use these accessors.

Note that the *rewrite_driver() methods are not valid during construction, so any test classes that need to use them must do so from SetUp() methods.

void net_instaweb::RewriteTestBase::ServeResourceFromNewContext ( const GoogleString resource_url,
const StringPiece &  expected_content 
)

Test that a resource can be served from an new server that has not already constructed it.

Set the "active" server to that specified; the active server is used for rewriting and serving pages.

void net_instaweb::RewriteTestBase::SetBaseUrlForFetch ( const StringPiece &  url)

Sets the active context URL for purposes of XS checks of fetches on the main rewrite driver.

Overrides the async fetcher on the primary context to be a wait fetcher which permits delaying callback invocation. CallFetcherCallbacks can then be called to let the fetches complete and call the callbacks.

void net_instaweb::RewriteTestBase::SetUseManagedRewriteDrivers ( bool  use_managed_rewrite_drivers)

Use managed rewrite drivers for the test so that we see the same behavior in tests that we see in real servers. By default, tests use unmanaged drivers so that _test.cc files can add options after the driver was created and before the filters are added.

void net_instaweb::RewriteTestBase::TestRetainExtraHeaders ( const StringPiece &  name,
const StringPiece &  filter_id,
const StringPiece &  ext 
)

Tests that non-caching-related response-header attributes are propagated to output resources.

'name' is the name of the resource.

Returns the main factory Timer*, which can be used for calling NowUs and NowMs. To set the time, use (Advance|Set)Time(Ms|Us), which wake up any scheduler alarms. See also AdjustTimeUsWithoutWakingAlarms which should be used with extreme care.


Member Data Documentation

We have two independent RewriteDrivers representing two completely separate servers for the same domain (say behind a load-balancer).

Server A runs rewrite_driver_ and will be used to rewrite pages and serves the rewritten resources.


The documentation for this class was generated from the following file:
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines