Results 1 to 9 of 9

Thread: Looking for Caching Software

  1. #1
    TopDogger's Avatar
    TopDogger is online now Über Hund
    Join Date
    Jan 2009
    Location
    Hellfire, AZ
    Posts
    3,077
    Thanks
    347
    Thanked 913 Times in 697 Posts

    Looking for Caching Software

    I am working on a project where I need to cache static HTML copies of dynamic pages. I need to dramatically cut down on requests from a remote database.

    It needs to runs on Linux and I need to be able to set a lengthy timeout, such as 8 to 12 hours.

    A PHP solution is preferred.

    Does anyone have any recommendations for a freebie software option that is easy to learn and install?
    "Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -- Benjamin Franklin


  2. #2
    TopDogger's Avatar
    TopDogger is online now Über Hund
    Join Date
    Jan 2009
    Location
    Hellfire, AZ
    Posts
    3,077
    Thanks
    347
    Thanked 913 Times in 697 Posts
    I'm looking for a recommendation for a system that someone here has used. Source Forge is a good source for free software, but a lot of it is crap with long learning curves. Unfortunately, I don't really have the time to test a half-dozen caching systems.

    There are a lot of caching options out there, but I am really looking for one that is easy to use and works well.
    "Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -- Benjamin Franklin


  3. #3
    Andy101's Avatar
    Andy101 is offline Code Otaku
    Join Date
    Aug 2009
    Location
    Kanazawa
    Posts
    1,263
    Thanks
    173
    Thanked 304 Times in 231 Posts
    For WordPress I use WP Super Cache which is easy to use and works well.

    For my own code such as where I might be sourcing an RSS feed, I find it fairly easy to code a cache. It entails saving the feed to a text file and also saving a time stamp value for the last update (which could be got from the attributes of the saved file) or to the database. Then the script compares the stored time to the current time to check if the feed should be refreshed.

    Writing this time stamp to a text file is not much of a time burden since MySQL uses text files to store data on a server anyway.

    If you had a custom script developed, I don't think it is much work to get the cache feature added i.e. it shouldn't cost much extra to pay your programmer to implement it.

    Sorry, I don't know of any specific software solutions for this. I think that the admin code would be far more complex than a custom, hard-coded solution.

  4. Thanked by:

    TopDogger (3 November, 2010)

  5. #4
    TopDogger's Avatar
    TopDogger is online now Über Hund
    Join Date
    Jan 2009
    Location
    Hellfire, AZ
    Posts
    3,077
    Thanks
    347
    Thanked 913 Times in 697 Posts
    This is not a WordPress project and I am the programmer.

    This is an existing dynamic PHP site that uses the .html extension for the pages. We need to cut down on the requests from a remote XML database. The data is not in a local database. Caching for extended time periods appears to be the best option.

    Thanks for the help, but I am really looking for a recommendation from someone who has already been down this path.
    "Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -- Benjamin Franklin


  6. #5
    Will.Spencer's Avatar
    Will.Spencer is offline Retired
    Join Date
    Dec 2008
    Posts
    5,033
    Blog Entries
    1
    Thanks
    1,010
    Thanked 2,329 Times in 1,259 Posts
    Quote Originally Posted by TopDogger View Post
    We need to cut down on the requests from a remote XML database. The data is not in a local database.
    Are you sure that page level caching is the right solution? If you're using a decent XML database, it should have something like MySQL's Query Caching built in. Which XLM database are you using? As examples, both Sedna and Ozone offer caching, although the implementation details are beyond my skill level.

    Alternatively, you could also set up Squid in front of your web servers. It's the oldest and most respected caching proxy in existence. I've used Squid and I've competed against it in support of a commercial product -- it's good stuff.
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

  7. #6
    TopDogger's Avatar
    TopDogger is online now Über Hund
    Join Date
    Jan 2009
    Location
    Hellfire, AZ
    Posts
    3,077
    Thanks
    347
    Thanked 913 Times in 697 Posts
    It is not a local database, so I have no control over database caching. I'm trying to reduce requests from Amazon's live XML database.

    Pulling all of the data into a MySQL database is not a good option, because there are millions of products and prices can change often. Plus, I don't want to slow the site down with a behemoth database.

    It appears that page caching is the best method for reducing requests. For example, a popular page might request the same data 500 times in a day, but only once or twice if I cache the page for 12 hour durations.

    I have heard of Squid, but have not looked into it. Is it an option for my scenario?
    "Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -- Benjamin Franklin


  8. #7
    Will.Spencer's Avatar
    Will.Spencer is offline Retired
    Join Date
    Dec 2008
    Posts
    5,033
    Blog Entries
    1
    Thanks
    1,010
    Thanked 2,329 Times in 1,259 Posts
    Quote Originally Posted by TopDogger View Post
    It is not a local database, so I have no control over database caching. I'm trying to reduce requests from Amazon's live XML database.
    You can implement caching when calling a remote database -- that's actually when it makes the most sense. But... the database client has to support it.

    I don't know jack about Amazon's system. What is your interface to it?

    Quote Originally Posted by TopDogger View Post
    Pulling all of the data into a MySQL database is not a good option, because there are millions of products and prices can change often. Plus, I don't want to slow the site down with a behemoth database.
    Oh yeah, definitely agree.

    Quote Originally Posted by TopDogger View Post
    It appears that page caching is the best method for reducing requests. For example, a popular page might request the same data 500 times in a day, but only once or twice if I cache the page for 12 hour durations.

    I have heard of Squid, but have not looked into it. Is it an option for my scenario?
    Squid will sit in front of the web servers and serve pages for the web servers. Because the web servers will never see the page requests, they will not make requests to the database server. Of course, that can screw with your stats.
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

  9. #8
    TopDogger's Avatar
    TopDogger is online now Über Hund
    Join Date
    Jan 2009
    Location
    Hellfire, AZ
    Posts
    3,077
    Thanks
    347
    Thanked 913 Times in 697 Posts
    Quote Originally Posted by Will.Spencer View Post
    You can implement caching when calling a remote database -- that's actually when it makes the most sense. But... the database client has to support it.

    I don't know jack about Amazon's system. What is your interface to it?

    I use REST to retrieve an XML file with Amazon.

    I suspect Amazon does not support caching. It is hard to get any answers out of them. They advise against using caching because they want the data displayed to be fresh. However, their XML DB servers can slow down considerably during the Christmas season and it frustrates visitors to affiliate sites. I'm trying to speed up my sites for the holidays, which means I need something I can set up quickly.


    Quote Originally Posted by Will.Spencer View Post
    Squid will sit in front of the web servers and serve pages for the web servers. Because the web servers will never see the page requests, they will not make requests to the database server. Of course, that can screw with your stats.
    It sounds like you are talking about clustering software. Is that how squid works?

    Can that be used with my VPS server?

    Are you using it yourself?
    "Democracy is two wolves and a lamb voting on what to have for lunch. Liberty is a well-armed lamb contesting the vote." -- Benjamin Franklin


  10. #9
    Will.Spencer's Avatar
    Will.Spencer is offline Retired
    Join Date
    Dec 2008
    Posts
    5,033
    Blog Entries
    1
    Thanks
    1,010
    Thanked 2,329 Times in 1,259 Posts
    Quote Originally Posted by TopDogger View Post
    I use REST to retrieve an XML file with Amazon.
    Ahhh... now it makes sense.

    RESTWiki has bad things to say about caching support in Amazon's implementation of REST.

    Here's someone caching Amazon REST requests in PERL.

    Quote Originally Posted by TopDogger View Post
    It sounds like you are talking about clustering software. Is that how squid works?
    Squid isn't a cluster, it's more of a reverse proxy when used as a proxy cache.

    Quote Originally Posted by TopDogger View Post
    Can that be used with my VPS server?
    I think so, although I have only ever used dedicated Squid servers.

    Quote Originally Posted by TopDogger View Post
    Are you using it yourself?
    I have in the past, but am not currently. I was using Squid for a very different purpose back then. We were trying to optimize bandwidth utilization over limited WAN circuits.
    Submit Your Webmaster Related Sites to the NB Directory
    I swear, by my life and my love of it, that I will never live for the sake of another man, nor ask another man to live for mine.

Similar Threads

  1. Web Design Software?
    By GekiDan in forum Web Design
    Replies: 14
    Last Post: 29 September, 2014, 12:13 PM
  2. Web Design Software?
    By forceclosed in forum Web Graphics
    Replies: 21
    Last Post: 24 July, 2012, 16:14 PM
  3. Which are Best Software for SEO?
    By Nic SEO in forum Promoting
    Replies: 27
    Last Post: 29 July, 2011, 21:33 PM
  4. Google Not Caching Pages (And AdSense PSAs)
    By Zash in forum On-Site SEO
    Replies: 4
    Last Post: 10 August, 2010, 02:15 AM
  5. What software are you using?
    By Coelho in forum Community Building
    Replies: 9
    Last Post: 29 June, 2010, 01:02 AM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •