Page Speed Optimization Libraries  1.8.31.3
Namespaces | Classes | Typedefs | Enumerations | Functions | Variables
net_instaweb Namespace Reference

Unit-test framework for wget fetcher. More...

Namespaces

namespace  css_util

Classes

class  ApacheConfig
class  ApacheMessageHandler
class  ApacheRequestContext
class  ApacheRewriteDriverFactory
 Creates an Apache RewriteDriver. More...
class  ApacheServerContext
class  ApacheThreadSystem
class  ApacheWriter
 Writer object that writes to an Apache Request stream. More...
class  AprFileSystem
class  AprTimer
class  InstawebContext
class  ApacheFetch
class  InstawebHandler
class  ModSpdyFetchController
class  ModSpdyFetcher
class  CacheHtmlFlow
class  FlushEarlyFlow
class  HtmlDetector
class  ProxyFetchFactory
class  ProxyFetchPropertyCallback
class  ProxyFetchPropertyCallbackCollector
class  ProxyFetch
class  ProxyInterface
class  ProxyUrlNamer
class  MockFilter
class  CreateFilterCallback
class  BackgroundFetchCheckingAsyncFetch
class  BackgroundFetchCheckingUrlAsyncFetcher
class  ProxyInterfaceTestBase
class  FileRewriter
class  StaticRewriter
class  RewriteOptionsManager
class  FileDriver
class  FileStatisticsLog
 Statistics logger that sends its output to a file. More...
class  LoggingFilter
 Counts some basic statistics observed as HTML is parsed. More...
class  StatisticsLog
class  AsyncFetch
class  StringAsyncFetch
class  AsyncFetchUsingWriter
class  SharedAsyncFetch
class  FallbackSharedAsyncFetch
class  ConditionalSharedAsyncFetch
class  AsyncFetchWithLock
class  CacheUrlAsyncFetcher
class  CountingUrlAsyncFetcher
class  ExternalUrlFetcher
class  FetcherTest
class  HTTPCache
class  HttpDumpUrlAsyncWriter
class  HttpDumpUrlFetcher
class  HttpResponseParser
class  HTTPValue
class  HTTPValueWriter
class  InflatingFetch
class  AbstractLogRecord
class  LogRecord
 Simple AbstractLogRecord implementation which owns a LoggingInfo protobuf. More...
class  CopyOnWriteLogRecord
struct  ImageRewriteInfo
class  MockLogRecord
 A class which helps mock the methods of LogRecord for testing. More...
class  ExpectStringAsyncFetch
class  MockUrlFetcher
class  RateController
class  RateControllingUrlAsyncFetcher
class  ReflectingTestFetcher
class  RequestContext
class  SimulatedDelayFetcher
class  SyncFetcherAdapterCallback
 Class to help run an asynchronous fetch synchronously with a timeout. More...
class  UserAgentSensitiveTestFetcher
class  UrlAsyncFetcher
class  UrlAsyncFetcherStats
class  WaitUrlAsyncFetcher
class  WgetUrlFetcher
class  WriteThroughHTTPCache
 Composes two cache interfaces to form a two level http cache. More...
class  ImageTestingPeer
class  AddHeadFilter
class  AddInstrumentationFilter
 Injects javascript instrumentation for monitoring page-rendering time. More...
class  AssociationTransformer
class  AssociationSlot
class  BaseTagFilter
class  BeaconCriticalImagesFinder
class  BeaconCriticalLineInfoFinder
class  CacheExtender
class  CacheHtmlFilter
class  CacheHtmlInfoFinder
 Manages the cache lifetimes of CacheHtmlInfo. More...
class  CacheableResourceBase
class  CollectFlushEarlyContentFilter
class  CommonFilter
class  ComputeVisibleTextFilter
class  CriticalCssBeaconFilter
class  CriticalCssFilter
class  CriticalCssFinder
 Finds critical CSS rules (i.e. CSS needed for the initial page load). More...
struct  BeaconMetadata
class  CriticalImagesBeaconFilter
struct  CriticalImagesInfo
class  CriticalImagesFinder
class  TestCriticalImagesFinder
class  CriticalImagesFinderTestBase
class  CriticalLineInfoFinder
class  CriticalSelectorFilter
struct  CriticalSelectorInfo
class  CriticalSelectorFinder
class  BeaconCriticalSelectorFinder
class  CssAbsolutify
class  CssCombineFilter
class  CssFilter
class  CssFlattenImportsContext
 Context used by CssFilter under async flow that flattens . More...
class  CssHierarchy
class  CssImageRewriter
class  CssInlineFilter
 Inline small CSS files. More...
class  CssInlineImportToLinkFilter
class  CssMinify
class  CssMoveToHeadFilter
class  CssOutlineFilter
 Filter to take explicit <style> and <script> tags and outline them to files. More...
class  CssResourceSlot
 A place storing a rewritable URL inside a CSS AST. More...
class  CssResourceSlotFactory
class  CssRewriteTestBase
class  CssSummarizerBase
class  CssTagScanner
class  RewriteDomainTransformer
class  CssUrlCounter
class  CssUrlEncoder
class  CssUrlExtractor
class  CustomRewriteTestBase
class  DataUrlInputResource
class  DebugFilter
class  DecodeRewrittenUrlsFilter
class  DedupInlinedImagesFilter
class  DeferIframeFilter
class  DelayImagesFilter
class  DeterministicJsFilter
class  DeviceProperties
class  DomStatsFilter
 Counts some basic statistics observed as HTML is parsed. More...
class  DomainLawyer
class  DomainRewriteFilter
class  DownstreamCachePurger
class  DownstreamCachingDirectives
class  ExperimentMatcher
class  FakeFilter
class  FileInputResource
class  FileLoadMapping
class  FileLoadMappingRegexp
class  FileLoadMappingLiteral
class  FileLoadPolicy
class  FileLoadRule
class  FileLoadRuleRegexp
class  FileLoadRuleLiteral
class  FixReflowFilter
class  FlushEarlyContentWriterFilter
class  FlushEarlyInfoFinder
class  MeaningfulFlushEarlyInfoFinder
class  FlushHtmlFilter
class  ScriptEditor
 Edit a substring in a script element. More...
class  GoogleAnalyticsFilter
class  GoogleFontCssInlineFilter
class  GoogleFontServiceInputResource
class  HandleNoscriptRedirectFilter
class  Image
class  ImageCombineFilter
class  ImageRewriteFilter
class  ImageTestBase
class  ImageUrlEncoder
class  InPlaceRewriteResourceSlot
class  InPlaceRewriteContext
 Context that is used for an in-place rewrite. More...
class  RecordingFetch
class  InlineRewriteContext
class  InsertDnsPrefetchFilter
class  InsertGAFilter
class  JavascriptRewriteConfig
class  JavascriptCodeBlock
class  JavascriptFilter
class  JavascriptSourceMapFilter
class  JavascriptLibraryIdentification
class  JsCombineFilter
class  JsDeferDisabledFilter
class  JsDisableFilter
class  JsInlineFilter
 Inline small Javascript files. More...
class  JsOutlineFilter
 Filter to take explicit <style> and <script> tags and outline them to files. More...
class  LazyloadImagesFilter
class  LocalStorageCacheFilter
class  MetaTagFilter
class  MockCriticalCssFinder
class  MockCriticalImagesFinder
class  MockResourceCallback
class  OutputResource
class  PedanticFilter
class  ProcessContext
class  RedirectOnSizeLimitFilter
class  RequestProperties
class  Resource
struct  TimedBool
 A boolean with an expiration date. More...
class  ResourceCombiner
class  ResourceFetch
class  ResourceNamer
class  ResourceSlot
class  FetchResourceSlot
class  HtmlResourceSlot
class  HtmlResourceSlotComparator
class  RewriteContext
class  TrimWhitespaceRewriter
 Simple test filter just trims whitespace from the input resource. More...
class  TrimWhitespaceSyncFilter
class  UpperCaseRewriter
 A similarly structured test-filter: this one just upper-cases its text. More...
class  NestedFilter
class  CombiningFilter
class  RewriteContextTestBase
class  RewriteDriver
class  OptionsAwareHTTPCacheCallback
class  RewriteDriverFactory
class  RewriteDriverPool
class  RewriteFilter
class  RewriteGflags
class  RewriteOptions
class  RewriteOptionsTestBase
class  RewriteQuery
class  RewriteStats
 Collects a few specific statistics variables related to Rewriting. More...
class  RewriteTestBase
class  RewrittenContentScanningFilter
class  ScanFilter
class  ScriptTagScanner
class  ServerContext
class  SimpleTextFilter
class  SingleRewriteContext
class  SplitHtmlBeaconFilter
 Inject JavaScript for detecting the below-the-fold HTML panels. More...
struct  XpathUnit
class  SplitHtmlConfig
class  SplitHtmlState
 Represents the filter state necessary to perform the split. More...
class  SplitHtmlFilter
class  SplitHtmlHelperFilter
class  StaticAssetManager
class  StripNonCacheableFilter
class  StripScriptsFilter
 Remove all scripts from a web page. More...
class  SupportNoscriptFilter
class  SuppressPreheadFilter
class  TestDistributedFetcher
class  TestRewriteDriverFactory
class  TestUrlNamer
class  UrlInputResource
class  UrlLeftTrimFilter
class  UrlNamer
class  UrlPartnership
class  UsageDataReporter
class  AddHeadersFetcher
class  AprMemCache
class  InPlaceResourceRecorder
class  LoopbackRouteFetcher
 See file comment. More...
struct  SerfStats
class  SerfUrlAsyncFetcher
class  SystemCachePath
class  SystemCaches
class  SystemRequestContext
class  SystemRewriteDriverFactory
 A server context with features specific to a psol port on a unix system. More...
class  SystemRewriteOptions
class  SystemServerContext
 A server context with features specific to a PSOL port on a unix system. More...
class  SystemThreadSystem
class  AbstractPropertyStoreGetCallback
class  CachePropertyStore
class  FallbackPropertyPage
class  MockPropertyPage
class  PropertyValue
 Holds the value & stability-metadata for a property. More...
class  PropertyCache
 Adds property-semantics to a raw cache API. More...
class  AbstractPropertyPage
 Abstract interface for implementing a PropertyPage. More...
class  PropertyPage
class  PropertyStore
class  PropertyStoreGetCallback
class  TwoLevelPropertyStore

Typedefs

typedef Callback2< StringPiece,
bool * > 
HeaderPredicateFn
typedef RefCountedPtr
< RequestContext
RequestContextPtr
typedef std::map< GoogleString,
const Panel * > 
PanelIdToSpecMap
typedef std::multimap
< GoogleString, std::pair
< GoogleString, const int >
, StringCompareInsensitive > 
AttributesToNonCacheableValuesMap
typedef std::map< GoogleString,
std::pair< int32, int32 > > 
RenderedImageDimensionsMap
typedef RefCountedPtr
< CssResourceSlot
CssResourceSlotPtr
typedef RefCountedPtr< ResourceResourcePtr
typedef std::vector< ResourcePtr > ResourceVector
typedef Callback2< const
GoogleUrl &, bool * > 
ResourceUrlClaimant
typedef RefCountedPtr
< ResourceSlot
ResourceSlotPtr
typedef RefCountedPtr
< HtmlResourceSlot
HtmlResourceSlotPtr
typedef std::vector
< ResourceSlotPtr > 
ResourceSlotVector
typedef std::set
< HtmlResourceSlotPtr,
HtmlResourceSlotComparator
HtmlResourceSlotSet
typedef RefCountedPtr
< OutputResource
OutputResourcePtr
typedef std::vector
< OutputResourcePtr > 
OutputResourceVector
typedef std::vector< XpathUnitXpathUnits
typedef std::map< GoogleString,
XpathUnits * > 
XpathMap
 Map of xpath to XpathUnits.
typedef bool(* WebpProgressHook )(int percent, void *user_data)
 Progress hook for WebP conversions.
typedef std::vector
< PropertyPage * > 
PropertyPageStarVector

Enumerations

enum  BeaconStatus { kDoNotBeacon, kBeaconNoNonce, kBeaconWithNonce }
enum  OutputResourceKind { kRewrittenResource, kOnTheFlyResource, kOutlinedResource }
enum  PropertyCacheDecodeResult { kPropertyCacheDecodeNotFound, kPropertyCacheDecodeExpired, kPropertyCacheDecodeParseError, kPropertyCacheDecodeOk }
enum  PropertyCacheUpdateResult { kPropertyCacheUpdateNotFound, kPropertyCacheUpdateEncodeError, kPropertyCacheUpdateOk }
enum  RewriteResult { kRewriteFailed, kRewriteOk, kTooBusy }

Functions

void SlurpUrl (ApacheServerContext *server_context, request_rec *r)
void AprReportError (MessageHandler *message_handler, const char *filename, int line, const char *message, int error_code)
void ApacheRequestToRequestHeaders (const request_rec &request, RequestHeaders *request_headers, HeaderPredicateFn *predicate)
void ApacheRequestToRequestHeaders (const request_rec &request, RequestHeaders *request_headers)
 Fully converts apache request header structure into RequestHeaders.
void ApacheRequestToResponseHeaders (const request_rec &request, ResponseHeaders *headers, ResponseHeaders *err_headers)
void ResponseHeadersToApacheRequest (const ResponseHeaders &response_headers, request_rec *request)
void ErrorHeadersToApacheRequest (const ResponseHeaders &err_response_headers, request_rec *request)
void DisableDownstreamHeaderFilters (request_rec *request)
 Remove downstream filters that might corrupt our caching headers.
void PrintHeaders (request_rec *request)
 Debug utility for printing Apache headers to stdout.
void DisableCachingRelatedHeaders (request_rec *request)
 Updates headers related to caching (but not Cache-Control).
void DisableCacheControlHeader (request_rec *request)
template<class T >
apr_status_t apache_cleanup (void *object)
 Generic deleter meant to be used with apr_pool_cleanup_register().
void attach_mod_spdy ()
 Needs to be called from a ap_hook_optional_fn_retrieve hook.
int mod_spdy_get_spdy_version (conn_rec *conn)
spdy_slave_connection_factory * mod_spdy_create_slave_connection_factory (conn_rec *master_connection)
void mod_spdy_destroy_slave_connection_factory (spdy_slave_connection_factory *factory)
spdy_slave_connection * mod_spdy_create_slave_connection (spdy_slave_connection_factory *factory, ap_filter_rec_t *input_filter, void *input_filter_ctx, ap_filter_rec_t *output_filter, void *output_filter_ctx)
void mod_spdy_run_slave_connection (spdy_slave_connection *conn)
void mod_spdy_destroy_slave_connection (spdy_slave_connection *conn)
bool mod_ssl_is_https (conn_rec *conn)
Matcher< ImageRewriteInfoLogImageRewriteActivityMatcher (Matcher< const char * > id, Matcher< const GoogleString & > url, Matcher< RewriterApplication::Status > status, Matcher< bool > is_image_inlined, Matcher< bool > is_critical_image, Matcher< bool > is_url_rewritten, Matcher< int > size, Matcher< bool > try_low_res_src_insertion, Matcher< bool > low_res_src_inserted, Matcher< ImageType > low_res_image_type, Matcher< int > low_res_data_size)
bool ValidateAndExpireNonce (int64 now_ms, StringPiece nonce, CriticalKeys *critical_keys)
void GetCriticalKeysFromProto (int64 support_percentage, const CriticalKeys &critical_keys, StringSet *keys)
void UpdateCriticalKeys (bool require_prior_support, const StringSet &new_set, int support_value, CriticalKeys *critical_keys)
bool ShouldBeacon (const CriticalKeys &proto, const RewriteDriver &driver)
void WriteCriticalKeysToPropertyCache (const StringSet &new_keys, StringPiece nonce, int support_interval, bool should_replace_prior_result, bool require_prior_support, StringPiece property_name, const PropertyCache *cache, const PropertyCache::Cohort *cohort, AbstractPropertyPage *page, MessageHandler *message_handler, Timer *timer)
void PrepareForBeaconInsertionHelper (CriticalKeys *proto, NonceGenerator *nonce_generator, RewriteDriver *driver, bool using_candidate_key_detection, BeaconMetadata *result)
bool UpdateCandidateKeys (const StringSet &keys, CriticalKeys *proto, bool clear_rebeacon_timestamp)
bool IsBeaconDataAvailable (const CriticalKeys &proto)
ImageNewImage (const StringPiece &original_contents, const GoogleString &url, const StringPiece &file_prefix, Image::CompressionOptions *options, Timer *timer, MessageHandler *handler)
ImageBlankImageWithOptions (int width, int height, ImageType type, const StringPiece &tmp_dir, Timer *timer, MessageHandler *handler, Image::CompressionOptions *options)
int CharToInt (char c)
 char to int *without sign extension*.
int JpegIntAtPosition (const StringPiece &buf, size_t pos)
int GifIntAtPosition (const StringPiece &buf, size_t pos)
int PngIntAtPosition (const StringPiece &buf, size_t pos)
bool PngSectionIdIs (const char *hdr, const StringPiece &buf, size_t pos)
const PropertyValueDecodeFromPropertyCacheHelper (const PropertyCache *cache, AbstractPropertyPage *page, const PropertyCache::Cohort *cohort, StringPiece property_name, int64 cache_ttl_ms, PropertyCacheDecodeResult *status)
template<typename T >
T * DecodeFromPropertyCache (const PropertyCache *cache, AbstractPropertyPage *page, const PropertyCache::Cohort *cohort, StringPiece property_name, int64 cache_ttl_ms, PropertyCacheDecodeResult *status)
template<typename T >
T * DecodeFromPropertyCache (RewriteDriver *driver, const PropertyCache::Cohort *cohort, StringPiece property_name, int64 cache_ttl_ms, PropertyCacheDecodeResult *status)
PropertyCacheUpdateResult UpdateInPropertyCache (const protobuf::MessageLite &value, const PropertyCache::Cohort *cohort, StringPiece property_name, bool write_cohort, AbstractPropertyPage *page)
PropertyCacheUpdateResult UpdateInPropertyCache (const protobuf::MessageLite &value, RewriteDriver *driver, const PropertyCache::Cohort *cohort, StringPiece property_name, bool write_cohort)
bool OptimizeWebp (const GoogleString &original_jpeg, int configured_quality, WebpProgressHook progress_hook, void *progress_hook_data, GoogleString *compressed_webp, MessageHandler *message_handler)
bool ReduceWebpImageQuality (const GoogleString &original_webp, int quality, GoogleString *compressed_webp)
apr_pool_t * AprCreateThreadCompatiblePool (apr_pool_t *parent_pool)

Variables

const char kPagespeedOriginalUrl [] = "mod_pagespeed_original_url"
const char kModPagespeedFilterName [] = "MOD_PAGESPEED_OUTPUT_FILTER"
 Filter used for HTML rewriting.
const char kModPagespeedFixHeadersName [] = "MOD_PAGESPEED_FIX_HEADERS_FILTER"
 Filter used to fix headers after mod_headers runs.
const char kModPagespeedInPlaceFilterName [] = "MOD_PAGESPEED_IN_PLACE_FILTER"
const char kModPagespeedInPlaceCheckHeadersName [] = "MOD_PAGESPEED_IN_PLACE_CHECK_HEADERS_FILTER"
 Second filter checks headers for cacheability.
const char kPageUrl [] = "page.html"
const char kBackgroundFetchHeader [] = "X-Background-Fetch"
const char kGaSnippetGetTracker [] = "_modpagespeed_getRewriteTracker("
const char kGaSnippetPrefix [] = " var pageTrackerMethodNames = [\n"
const char kGaSnippetSuffix [] = "})();\n"
const int64 kBeaconTimeoutIntervalMs = Timer::kMinuteMs
const int64 kHighFreqBeaconCount = 3
const int64 kLowFreqBeaconMult = 100
const int64 kNonceExpirationLimit = 5
const char kGAExperimentSnippet []
const char kGAJsSnippet []
const char kGASpeedTracking []
const int kNoQualityGiven = -1

Detailed Description

Unit-test framework for wget fetcher.

Mock PropertyPage for use in unit tests.

extern "C"

Base class for tests which want a ServerContext.

for NULL

for scoped_ptr

for StringSet, etc

Some common routines and constants for tests dealing with Images.

for GoogleString

A filter which does not modify the DOM, but counts statistics about it.

Base class for tests which do rewrites within CSS.

namespace Css

for ResourcePtr

DO NOT EDIT. Generated by ./google_analytics_snippet_gen.py.

Callbacks used for testing.

TimingInfo.

for size_t

for FILE

for apr_status_t The httpd header must be after the instaweb_context.h. Otherwise, the compiler will complain "strtoul_is_not_a_portable_function_use_strtol_instead".

html_filter that passes data through unmodified, but logs statistics about the data as it goes by. It should be possible to create many instances of this class and insert them at different points in the rewriting flow Goal is to log: NUM_EXPLICIT_CLOSED - <tag> </tag> pairs NUM_IMPLICIT_CLOSED - <tag> for implicitly-closed tag NUM_BRIEF_CLOSED - </tag> NUM_CLOSED - Sum of above three NUM_UNCLOSED - <tag> without matching </tag> NUM_SPURIOUS_CLOSED - </tag> without preceding <tag>; UNCOUNTED RIGHT NOW! NUM_TAGS - Total number of opening tags NUM_CDATA - cdata sections NUM_COMMENTS - comments NUM_DIRECTIVES - directives NUM_DOCUMENTS - started documents NUM_IE_DIRECTIVES - ie directives Reporting: We report this information via a StatisticsLog: filter.ToString(log) Two sets of statistics (eg before and after processing) can be compared using before.Equals(after),

Todo:
TODO(sligocki): Find a way to forward declare FileSystem::InputFile.
Todo:
TODO(gee): Should this be in public? Do we really care?
Todo:
TODO(gee): Hmm, this sort of sucks.

If your .cc file needs to use the types declared in logging_proto.h, you must also include net/instaweb/http/public/logging_proto_impl.h See that header file for an explanation of why this is necessary.

Copyright 2011 Google Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http:///www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Helper class to make RewriteTestBase tests that use a custom options subclass.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Search for synchronous loads of Google Analytics similar to the following:

<script type="text/javascript"> var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www."); document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js type='text/javascript'" + "%3E%3C/script%3E")); </script> <script type="text/javascript"> try { var pageTracker = _gat._getTracker("UA-XXXXX-X"); pageTracker._trackPageview(); } catch(err) {} </script>

Replace the document.write with a new snippet that loads ga.js asynchronously. Also, insert a replacement for _getTracker that converts any calls to the synchronous API to the asynchronous API. The _getTracker replacement is a new function that returns a mock tracker object. Anytime a synchronous API method is called, the mock tracker fowards it to a _gaq.push(...) call.

An alternative approach would been to find all the API calls and rewrite them to the asynchronous API. However, to be done properly, it would have had the added complication of using a JavaScript compiler.

Unit-test the RewriteContext class. This is made simplest by setting up some dummy rewriters in our test framework.

for pair

We need to include rewrite_driver.h due to covariant return of html_parse() We need to include mock_timer.h to allow upcast to Timer*.

Todo:
TODO(skerner): #include image_spriter.pb.h is needed to allow use of enum ImageFormat. Break this dependency and remove the include.
Todo:
TODO(huibao): Move LibpngImageLibrary to pagespeed/kernel/image/. Refactor LibpngImageLibrary. This class can be split into 3 parts: reader, writer, and canvas creater. The first two parts can be merged with png_optimizer.cc and png_optimizer.h.

This class is a simple wrapper around another fetcher that adds headers to requests based on settings in the rewrite options before passing them on to the backend fetcher.


Typedef Documentation

typedef Callback2<StringPiece, bool*> net_instaweb::HeaderPredicateFn

Defines a predicate function used to select which request-headers to copy. The callback sets the bool* arg (.second) to true if it wants to include the header.

The StringPiece is the name of the header.

typedef Callback2<const GoogleUrl&, bool*> net_instaweb::ResourceUrlClaimant

Sometimes some portions of URL space need to be handled differently by dedicated resource subclasses. ResourceProvider callbacks are used to teach RewriteDriver about these, so it knows not to build regular UrlInputResource objects.


Enumeration Type Documentation

Enumerator:
kRewrittenResource 

derived from some input resource URL or URLs.

kOnTheFlyResource 

derived from some input resource URL or URLs in a very inexpensive way --- it makes no sense to cache the output contents.

kOutlinedResource 

derived from page HTML.

Enumerator:
kPropertyCacheDecodeNotFound 

includes property cache not being enabled.

Enumerator:
kPropertyCacheUpdateNotFound 

can't find existing value to update

Used to signal whether optimization was successful or not to RewriteContext::RewriteDone.

Enumerator:
kRewriteFailed 

rewrite is impossible or undesirable

kRewriteOk 

rewrite went fine

kTooBusy 

the system is temporarily too busy to handle this rewrite request; no conclusion can be drawn on whether it's worth trying again or not.


Function Documentation

void net_instaweb::ApacheRequestToRequestHeaders ( const request_rec &  request,
RequestHeaders *  request_headers,
HeaderPredicateFn *  predicate 
)

Converts Apache header structure into RequestHeaders, selecting only those for which the predicate sets its bool* argument to true. If the predicate is NULL, then all the headers are transferred.

The predicate should be created with NewPermanentCallback and stored in a scoped_ptr<Callback2>, so that it is deleted after this function completes.

void net_instaweb::ApacheRequestToResponseHeaders ( const request_rec &  request,
ResponseHeaders *  headers,
ResponseHeaders *  err_headers 
)

Converts Apache header structure (request.headers_out) into ResponseHeaders headers. If err_headers is not NULL then request.err_headers_out is copied into it. In the event that headers == err_headers, the headers from request.err_headers_out will be appended to the list of headers, but no merging occurs.

apr_pool_t* net_instaweb::AprCreateThreadCompatiblePool ( apr_pool_t *  parent_pool)

Creates a pool that can be used in any thread, even when run in Apache prefork.

1) This method must be called from startup phase only 2) Each pool must be accessed only from a single thread (or otherwise have its access serialized) 3) Different pools returned by this function may be safely used concurrently. 4) It's OK to just use ap_pool_create to create child pools of this one from multiple threads; those will be re-entrant too (but pools created merely as children of Apache's pools will not be reentrant in prefork)

In short, pools returned by this method are not fully threadsafe, but at least they are not thread-hostile, which is what you get with apr_pool_create in Prefork.

Note: the above is all about the release version of the pool code, the checking one has some additional locking!

WARNING: you must not call apr_pool_clear on the returned pool. The returned pool can be used to create sub-pools that can be accessed in distinct threads, due to a mutex injected into the allocator. However, if you call apr_pool_clear on the returned pool, the allocator's mutex will be freed and the pointer to it will be dangling. Subsequent allocations are likely to crash.

Image* net_instaweb::BlankImageWithOptions ( int  width,
int  height,
ImageType  type,
const StringPiece &  tmp_dir,
Timer *  timer,
MessageHandler *  handler,
Image::CompressionOptions *  options 
)

Creates a blank image of the given dimensions and type. For now, this is assumed to be an 8-bit 4-channel image transparent image.

int net_instaweb::CharToInt ( char  c) [inline]

char to int *without sign extension*.

The following four helper functions were moved here for testability. We ran into problems with sign extension under different compiler versions, and we'd like to catch regressions on that front in the future.

template<typename T >
T* net_instaweb::DecodeFromPropertyCache ( const PropertyCache *  cache,
AbstractPropertyPage *  page,
const PropertyCache::Cohort *  cohort,
StringPiece  property_name,
int64  cache_ttl_ms,
PropertyCacheDecodeResult *  status 
)

Decodes a protobuf of type T from the property named 'property_name' in the cohort 'cohort_name' in the given property cache, and makes sure it has not exceeded its TTL of 'cache_ttl_ms' (a value of -1 will disable this check).

status will denote the decoding state; if it's kPropertyCacheDecodeOk then a pointer to a freshly allocated decoded proto is returned; otherwise NULL is returned.

*status set by helper

template<typename T >
T* net_instaweb::DecodeFromPropertyCache ( RewriteDriver *  driver,
const PropertyCache::Cohort *  cohort,
StringPiece  property_name,
int64  cache_ttl_ms,
PropertyCacheDecodeResult *  status 
)

Wrapper version of the above function that gets the property cache and the property page from the given driver.

const PropertyValue* net_instaweb::DecodeFromPropertyCacheHelper ( const PropertyCache *  cache,
AbstractPropertyPage *  page,
const PropertyCache::Cohort *  cohort,
StringPiece  property_name,
int64  cache_ttl_ms,
PropertyCacheDecodeResult *  status 
)

Returns PropertyValue object for given cohort and property name, setting *status and returning NULL if any errors were found.

void net_instaweb::DisableCacheControlHeader ( request_rec *  request)

Updates caching headers to ensure the resulting response is not cached. Removes any max-age specification, and adds max-age=0, no-cache.

void net_instaweb::ErrorHeadersToApacheRequest ( const ResponseHeaders &  err_response_headers,
request_rec *  request 
)

Converts ResponseHeaders into Apache request err_headers. This function does not alter the status code or the major/minor version of the Apache request.

void net_instaweb::GetCriticalKeysFromProto ( int64  support_percentage,
const CriticalKeys &  critical_keys,
StringSet *  keys 
)

Generate a list of the critical keys from a proto, storing it into keys. Takes into account legacy keys that may have been added before. A key is considered critical if its support is at least support_percentage of the maximum possible support value (which ramps up as beacon results arrive). When support_percentage = 0, any support is sufficient; when support_percentage = 100 all beacon results must support criticality.

bool net_instaweb::IsBeaconDataAvailable ( const CriticalKeys &  proto) [inline]

Based on the CriticalKeys data seen so far, describe whether beacon metadata is available. This returns false until data is received.

Matcher<ImageRewriteInfo> net_instaweb::LogImageRewriteActivityMatcher ( Matcher< const char * >  id,
Matcher< const GoogleString & >  url,
Matcher< RewriterApplication::Status >  status,
Matcher< bool >  is_image_inlined,
Matcher< bool >  is_critical_image,
Matcher< bool >  is_url_rewritten,
Matcher< int >  size,
Matcher< bool >  try_low_res_src_insertion,
Matcher< bool >  low_res_src_inserted,
Matcher< ImageType >  low_res_image_type,
Matcher< int >  low_res_data_size 
)

A custom matcher to match more than 10 arguments allowed by MOCK_METHOD* macros.

spdy_slave_connection_factory* net_instaweb::mod_spdy_create_slave_connection_factory ( conn_rec *  master_connection)

See mod_spdy's slave_connection_api.h for description of the methods below. These are merely forwarding wrappers with some CHECKS. Note that this method will return NULL if the relevant mod_spdy methods weren't found registered with Apache. Others, however, will CHECK-fail (since there is no sensible way to call them if this method failed); except you can always safely mod_spdy_destroy_slave_connection_factory(NULL).

int net_instaweb::mod_spdy_get_spdy_version ( conn_rec *  conn)

If the connection is using SPDY with mod_spdy, returns the protocol version. Otherwise, returns 0.

bool net_instaweb::mod_ssl_is_https ( conn_rec *  conn)

Returns true if given connection is using HTTPS. (This is actually a mod_ssl function).

Image* net_instaweb::NewImage ( const StringPiece &  original_contents,
const GoogleString &  url,
const StringPiece &  file_prefix,
Image::CompressionOptions *  options,
Timer *  timer,
MessageHandler *  handler 
)

Image owns none of its inputs. All of the arguments to NewImage(...) (the original_contents in particular) must outlive the Image object itself. The intent is that an Image is created in a scoped fashion from an existing known resource.

The options should be set via Image::SetOptions after construction, before the image is used for anything but determining its natural dimension size.

Todo:
TODO(jmarantz): It would seem natural to fold the ImageOptions into the Image object itself.
bool net_instaweb::OptimizeWebp ( const GoogleString &  original_jpeg,
int  configured_quality,
WebpProgressHook  progress_hook,
void *  progress_hook_data,
GoogleString *  compressed_webp,
MessageHandler *  message_handler 
)

Optimizer in the style of pagespeed/kernel/image/jpeg_optimizer.h that creates a webp-formatted image in compressed_webp from the jpeg image in original_jpeg. Indicates failure by returning false, in which case compressed_webp may be filled with junk.

void net_instaweb::PrepareForBeaconInsertionHelper ( CriticalKeys *  proto,
NonceGenerator *  nonce_generator,
RewriteDriver *  driver,
bool  using_candidate_key_detection,
BeaconMetadata *  result 
)

Given a set of candidate critical keys, decide whether beaconing should take place. We should *always* beacon if there's new critical key data. Otherwise re-beaconing is based on a time and request interval, and 2 modes of beaconing frequency are supported. At first, beaconing occurs at a high frequency until we have collected kHighFreqBeaconCount beacons; after that, we transition into low frequency beaconing mode, where beaconing occurs less often. We also track the number of expired nonces since the last valid beacon was received to see if beaconing is set up correctly, and if it looks like it isn't, only do low frequency beaconing. Sets status and nonce appropriately in *result (nonce will be empty if no nonce is required). If candidate keys are not required, keys may be empty (but new candidate detection will not occur). If result->status != kDontBeacon, caller should write proto back to the property cache using UpdateInPropertyCache.

bool net_instaweb::ReduceWebpImageQuality ( const GoogleString &  original_webp,
int  quality,
GoogleString *  compressed_webp 
)

Reduce the quality of the webp image. Indicates failure by returning false. WebP quality varies from 1 to 100. Original image will be returned if input quality is <1.

void net_instaweb::ResponseHeadersToApacheRequest ( const ResponseHeaders &  response_headers,
request_rec *  request 
)

Converts the ResponseHeaders to the output headers. This function does not alter the status code or the major/minor version of the Apache request.

void net_instaweb::SlurpUrl ( ApacheServerContext *  server_context,
request_rec *  r 
)

Loads the URL based on the fetchers and other infrastructure in the factory.

bool net_instaweb::UpdateCandidateKeys ( const StringSet &  keys,
CriticalKeys *  proto,
bool  clear_rebeacon_timestamp 
)

Update the candidate key set in proto. If new candidate keys are detected, they are inserted into proto with a support value of 0, and true is returned. Otherwise returns false. If clear_rebeacon_timestamp is set, the rebeacon timestamp field in the proto is cleared to force rebeaconing on the next request.

void net_instaweb::UpdateCriticalKeys ( bool  require_prior_support,
const StringSet &  new_set,
int  support_value,
CriticalKeys *  critical_keys 
)

Add support for new_set to existing support. The new_set should be obtained from a fully-validated beacon result -- this means PrepareForBeaconInsertion should have been called if required, and the resulting nonce should have been checked. If require_prior_support then there must be an existing support entry (possibly 0) for new support to be registered.

PropertyCacheUpdateResult net_instaweb::UpdateInPropertyCache ( const protobuf::MessageLite &  value,
RewriteDriver *  driver,
const PropertyCache::Cohort *  cohort,
StringPiece  property_name,
bool  write_cohort 
) [inline]

Updates the property 'property_name' in cohort 'cohort_name' of the property cache managed by the rewrite driver with the new value of the proto T. If 'write_cohort' is true, will also additionally write out the cohort to the cache backing.

bool net_instaweb::ValidateAndExpireNonce ( int64  now_ms,
StringPiece  nonce,
CriticalKeys *  critical_keys 
)

Check whether the given nonce is valid, invalidating any expired nonce entries we might encounter. To avoid the need to copy and clear the nonce list, we invalidate the entry used and any expired entries by clearing the nonce value and timestamp. These entries will be reused by AddNonceToCriticalSelectors.

void net_instaweb::WriteCriticalKeysToPropertyCache ( const StringSet &  new_keys,
StringPiece  nonce,
int  support_interval,
bool  should_replace_prior_result,
bool  require_prior_support,
StringPiece  property_name,
const PropertyCache *  cache,
const PropertyCache::Cohort *  cohort,
AbstractPropertyPage *  page,
MessageHandler *  message_handler,
Timer *  timer 
)

Update the property cache with a new set of keys. This will update the support value for the new keys. If require_prior_support is set, any keys that are not already present in the property cache will be ignored (to prevent spurious keys from being injected). Note that it only increases the support value for the new keys, it does not decay values that are not present. PrepareForBeaconInsertion should have been called previously if !should_replace_prior_result and nonces must be checked.


Variable Documentation

const int64 net_instaweb::kBeaconTimeoutIntervalMs = Timer::kMinuteMs

The amount of time after generating a nonce that we will accept it as valid. This keeps an attacker from accumulating large numbers of valid nonces to send many beacon responses at once.

The number of valid beacons received that will switch from high frequency to low frequency beaconing.

The multiplier to apply to RewriteOptions::beacon_reinstrument_time_sec() to determine the low frequency beaconing interval. For example, the default value rebeaconing value is 5 seconds, so we will rebeacon every 5 seconds in high frequency mode, and every 500 seconds (~8 minutes) in low frequency mode.

const char net_instaweb::kModPagespeedInPlaceFilterName[] = "MOD_PAGESPEED_IN_PLACE_FILTER"

Filters used for In-Place Resource Optimization. First filter stores un-gzipped contents.

The limit on the number of nonces that can expire before we stop trying to do high frequency beaconing. This is a signal that beacons are not configured correctly and so we drop into low frequency beaconing mode.

Quality taken when no quality is passed through flags or when no quality is retrieved from JpegUtils::GetImageQualityFromImage.

 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Defines