2016-12-12 28 views
1

我正在尝试使用Prerender服务为爬网程序设置服务器端的Angular 1.5应用程序。Angular with Prerender(.htaccess设置)

对于内部页面,一切正常,但主页面呈现出现问题 - 抓取工具看到404页面而不是主页面。

我想有一个在我的.htaccess一些其他规则的问题 - 除了预渲染的规则,我使用所有的页面其他两个规则:

  • 重写URL没有结尾的斜杠上的网址与尾随斜杠
  • 上的网址中添加www重写URL加www

将是感激任何提示!

这里是Apache serveer我的.htaccess文件

RequestHeader set X-Prerender-Token "MyToken" 

    RewriteEngine On 

    RewriteCond %{HTTP_HOST} ^www.example.com$ [NC] 
    RewriteRule ^(.*)$ http://example.com/$1 [R=301,L] 


    # If an existing asset or directory is requested go to it as it is 
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR] 
    RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d 
    RewriteRule^- [L] 

    RewriteCond %{REQUEST_URI} ^/$ 
    RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$ 
    RewriteRule ^(.*)$ /snapshots/%1? [NC,L] 

    RewriteCond %{REQUEST_FILENAME} !-f 
    RewriteRule ^(.*[^/])$ /$1/ [L,R=301] 

    <IfModule mod_proxy_http.c> 
     RewriteCond %{HTTP_USER_AGENT} Googlebot|bingbot|Googlebot-Mobile|Baiduspider|Yahoo|YahooSeeker|DoCoMo|Twitterbot|TweetmemeBot|Twikle|Netseer|Daumoa|SeznamBot|Ezooms|MSNBot|Exabot|MJ12bot|sogou\sspider|YandexBot|bitlybot|ia_archiver|proximic|spbot|ChangeDetection|NaverBot|MetaJobBot|magpie-crawler|Genieo\sWeb\sfilter|Qualidator.com\sBot|Woko|Vagabondo|360Spider|ExB\sLanguage\sCrawler|AddThis.com|aiHitBot|Spinn3r|BingPreview|GrapeshotCrawler|CareerBot|ZumBot|ShopWiki|bixocrawler|uMBot|sistrix|linkdexbot|AhrefsBot|archive.org_bot|SeoCheckBot|TurnitinBot|VoilaBot|SearchmetricsBot|Butterfly|Yahoo!|Plukkie|yacybot|trendictionbot|UASlinkChecker|Blekkobot|Wotbox|YioopBot|meanpathbot|TinEye|LuminateBot|FyberSpider|Infohelfer|linkdex.com|Curious\sGeorge|Fetch-Guess|ichiro|MojeekBot|SBSearch|WebThumbnail|socialbm_bot|SemrushBot|Vedma|alexa\ssite\saudit|SEOkicks-Robot|Browsershots|BLEXBot|woriobot|AMZNKAssocBot|Speedy|oBot|HostTracker|OpenWebSpider|WBSearchBot|FacebookExternalHit [NC,OR] 
     RewriteCond %{QUERY_STRING} _escaped_fragment_ 

     # Only proxy the request to Prerender if it's a request for HTML 
     RewriteRule ^(?!.*?(\.js|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff))(.*) http://service.prerender.io/http://example.com/$2 [P,L] 
    </IfModule> 

    # If the requested resource doesn't exist, use index.html 
    RewriteRule^/index.html 

回答

0

你有这样的节:

RewriteCond %{REQUEST_URI} ^/$ 
    RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$ 
    RewriteRule ^(.*)$ /snapshots/%1? [NC,L] 

这将尝试从/快照/目录下的文件服务,如果_escaped_fragment_是在URL 。这与Prerender.io没有任何关系,因此您可能需要删除该部分,因为它可能是404的原因。

您还正在通过其用户代理检查Googlebot和Bingbot是一个坏主意,因为他们可能会惩罚你的隐形。