Optimizing SQUID in Linux

cache_mem

This is one of the top two options as far as performance impact goes. The default value here is 8 MB, which is a safe option for just about any system size. But since your box is not lacking in memory it can safely be raised to 32 MB, and even 48 if you have stripped all other services off of your server. If your box is lacking in memory, go back to the hardware section and reread the part about memory…you need a bunch of memory for a fast web cache. You should be aware that cache_mem does not limit the process size of squid. This sets how much memory squid is allowed to set aside for “hot objects” which are the most recently used objects. Having said all that, keep in mind that the buffer cache in Linux is also quite good, so the gains you’ll see by raising this are very small, but still measurable.

cache_dir

This is where you set the directories you will be using. You should have already mkreiserfs’d your cache directory partitions, so you’ll have an easy time deciding the values here. First, you will want to use about 60% or less of each cache directory for the web cache. If you use any more than that you will begin to see a slight degradation in performance. Remember that cache size is not as important as cache speed, since for maximum effectiveness your cache needs only store about a weeks worth of traffic. You’ll also need to define the number of directories and subdirectories. The formula for deciding that is this:

x=Size of cache dir in KB (i.e. 6GB=~6,000,000KB) y=Average object size

(just use 13KB z=Number of directories per first level directory

`(((x / y) / 256) / 256) * 2 = # of directories

As an example, I use 6GB of each of my 13GB drives, so:

6,000,000 / 13 = 461538.5 / 256 = 1802.9 / 256 = 7 * 2 = 14

So my cache_dir line would look like this:

cache_dir 6000 14 256

Those are really the two important ones. Everything else is just picking nits. You may wish to turn off the store log since there isn’t much one can do with it anyway:

cache_store_log none

If, after all of this you still find your squid being overloaded at times, try setting the max_open_disk_fds to some low number. For a single disk cache, 900 is a good choice. For a dual disk cache, try 1400. To get the perfect number for your cache you’ll need to keep an eye on the info page from the cache manager during a particularly heavy load period. Watch the file descriptor usage, and then set this number about 10%below it. This can help your cache ride out the heaviest loads without getting too bogged down.

Also, the following may be set to improve performance marginally:

half_closed_clients off
maximum_object_size 1024 KB

——————————————————–

Disk space and memory

A cache can always use more disk space, but as the size of your disk-cache grows, you will need more memory to index it. There’s a straightforward rule for memory.

Divide the size of your disk cache by 13 Kbytes, and multiply that by 130 bytes. Add the size of cache_mem, and add about 2.5 Mbytes more for executable files, libraries, and other sundry overhead. For example: We have a 10-Gbyte drive, and a cache_mem of 8 Mbytes.

10 Gbytes/13 Kbytes = 769,230
769,230 x 130 bytes = 99,999,900 bytes (or 97,656 Kbytes)
97,656 Kbytes + 2.5 Mbytes + 8 Mbytes = 10,849,656 Kbytes or about 108 Mbytes

The example server needs 108 Mbytes available to Squid to support 10 Gbytes of cache_dir.

Provide as much disk space as you can provide RAM to support it. Squid performs very badly when it starts to swap. Remember to set aside memory for anything else on the machine (DNS, cron, operating system, etc.).

Refresh patterns

Refresh patterns determine the lifetime of the object. Within an object lifetime, Squid will serve the object without requesting an IMS (“if modified since”) request. Once the lifetime is exceeded, Squid will keep the object but will send an IMS request to the origin server. If the object has been modified since it was first cached, Squid requests the new copy. If not, it keeps the old copy. Either way, the object is marked as fresh again.

Here’s our (default) basic refresh pattern:

refresh_pattern . 0 20% 4320

The dot (.) is the the regular expression pattern, and matches anything. It uses POSIX regular expressions. (See man 7 regex).

The zero (0) is the minimum freshness time. If it’s anything other than zero, it will override any expiration headers given with the object. If the content provider actually provided an expiration header, we should usually honor it.

The last term (4320) is the maximum freshness time. The object becomes stale after this many minutes in the cache.

The 20 percent is used for our default case, for when there’s no information from the content provider about the lifetime of the object. Squid takes x percent (20 percent in this example) of the difference between the last-modified time of the object and the current time, and uses that as the object lifetime. If the object lifetime is less than the minimum set by the refresh_pattern, it is increased to at least that. If it’s greater than the supplied maximum, it’s reduced to that.

Non-standard files

Some kinds of files can be maintained much longer than others. Zip, tar.gz, tgz, and .exe files rarely change content without also changing name. Using regular expressions, we can create a set of refresh patterns like this:

refresh_pattern -i exe$ 0 50% 999999 refresh_pattern -i zip$ 0 50% 999999 refresh_pattern -i tar.gz$ 0 50% 999999 refresh_pattern -i tgz$ 0 50% 999999

Refresh pattern options

Note that these options violate the HTTP standard. Do not use them lightly.

override-expire pretends there is no expiration header on the object and calculates purely based on last-modified times. This permits you to cache sites that abuse the use of expiration headers, but also inhibits updates of frequently changed content (such as news sites).

ignore-reload prevents the object being refreshed when the user presses the refresh button on their browser. This does not perform well when the object has no content length — you may wind up with a broken object that the users cannot reload.

reload-into-ims transforms reloads into validations. Beware: Web servers may permit an object to be updated without the last-modified time being altered. The server may then insist that the object is still valid when it actually is not.

DO increase maximum_object_size. 40 Mbytes is not too large. 800 Mbytes might cache the large downloads.

DO increase the ipcache_size and fqdncache_size.

Source: http://www.linux-faqs.com/squid.php

Source: http://www.linuxdevcenter.com/pub/a/linux/2001/07/26/squid.html?page=2

Install WordPress in Debian Etch

WordPress is a state-of-the-art semantic personal publishing platform with a focus on aesthetics, web standards, and usability. What a mouthful. WordPress is both free and priceless at the same time. WordPress Requirements

* Apache Web server

* PHP 4.2 or greater

* MySQL 3.23.23 or greater

* The mod_rewrite Apache module

First you need to install apache2 with php support and mysql with php support

Apache with PHP Support check here

Mysql Database installation check here

Mysql with php support you need to run the following command

#apt-get install php4-mysql

after that you need to install wordpress using the follwoing command

#apt-get install wordpress

This will install the wordpress now you need to go to /usr/share/doc/wordpress/example folder to setup apache to point to /usr/share/wordpress. See the examples/apache.conf

Create a file called wordpress under /etc/apache2/sites-available/ and enter the following and save the file

Alias /blog /usr/share/wordpress
<Directory /usr/share/wordpress>
Options FollowSymLinks
AllowOverride Limit Options FileInfo
DirectoryIndex index.php
</Directory>
Now you need to enable this site using the following command

#a2ensite wordpress

Now reload apache using the following command

#/etc/init.d/apache2 restart

Database setup can be done with the help of a script in examples/setup-mysql

To setup mysql database for wordpress you need to follow this

setup-mysql [-n NAME] [-h | -d | -b] FQDN

Creates by default a WordPress mysql configuration depending on required fully
qualified domain name(FQDN).

Options:
-n name for the mysql user and database
-h help
-d destroy and purge
-b backup

Example: You want your blog to be served from http://blog.example.com for user ‘wordpress’.

Then run:
sudo sh setup-mysql -n wordpress blog.example.com

Example:-

#sh setup-mysql -n wordpress 172.20.2.18

After finishing this you need to go to http://your-server-ip/blog/ you should see the following message

It doesn’t look like you’ve installed WP yet. Try running install.php.

Click on install.php link and you should see the following screen saying welcome to wordpress installation in this click on First Step

Source: http://www.debianadmin.com/install-wordpress-in-debian-etch.html

Recover root password under linux with single user mode

How-To: Recover root password under linux with single user mode

<!– google_alternate_ad_url = “http://www.debuntu.org/altadd/rectangle-300.php”; google_ad_client = “pub-3316878663386307″; google_ad_type = “text_image”; google_ad_channel = “8357962815″; google_ad_width = 336; google_ad_height = 280; google_ad_format = “336×280_as”; google_color_border = “ffffff”; google_color_bg = “FAFCFF”; google_color_link = “0A8FBC”; google_color_url = “008000″; google_color_text = “000000″; //–>

It happens sometime that you can’t remember root password. On Linux, recovering root password can be done by booting Linux under a specific mode: single user mode.
This tutorial will show how to boot Linux in single user mode when using GRUB and finally how to change root password.

During normal usage, a Linux OS runs under runlevels between 2 and 5 which corresponds to various multi-user modes. Booting Linux under runlevel 1 will allow one to enter into a specific mode, single user mode. Under such a level, you directly get a root prompt. From there, changing root password is a piece of cake.

1. Entering runlevel 1

Some Linux distribution, such as Ubuntu for instance, offer a specific boot menu entry where it is stated “Recovery Mode” or “Single-User Mode”. If this is your case, selecting this menu entry will boot your machine into single user mode, you can carry on with the next part. If not, you might want to read this part.

Using GRUB, you can manually edit the proposed menu entry at boot time. To do so, when GRUB is presenting the menu list (you might need to press ESC first), follow those instructions:

• use the arrows to select the boot entry you want to modify.
• press e to edit the entry
• use the arrows to go to kernel line
• press e to edit this entry
• at the end of the line add the word single
• press ESC to go back to the parent menu
• press b to boot this kernel

The kernel should be booting as usual (except for the graphical splash screen you might be used to), and you will finally get a root prompt (sh#).

Here we are, we have gained root access to the filesystem, let’s finally change the password.

2. Changing root password

As root, changing password does not ask for your old password, therefore running the command:

# passwd

will prompt you for your new password and will ask you to confirm it to make sure there is no typo.

That’s it, you can now reboot your box and gain root access again

Source: http://www.debuntu.org/recover-root-password-single-user-mode-and-grub

Ubuntu Linux shutdown the X server

“How do I shutdown the X server, so that I have a command line prompt on the console?”

If you are using GNOME:
sudo /etc/init.d/gdm stop

Again to start GNOME desktop:
sudo /etc/init.d/gdm start

source : http://theos.in/ubuntu-linux-shutdown-the-x-server.html

Calamaris

The Calamaris is a software written in Perl that effects the generation of reports detailed of the use of the InterNet using the archives of logs of some servers well proxy, as the NetCache, Inktomi Traffic Server, Oops! proxy server, Novell InterNet Caching System, Compaq Tasksmart, Netscape/iplanet Web Proxy server and are clearly the Squid. The generated reports are well simple in the presentation, however very rich in extracted details of the archives of logs, they can be generated in same format HTML or in text to be sent way email.

The use of this software is very simple, first has that to lower the version most recent of it in http://cord.de/tools/squid/calamaris/Welcome.html.en, to unpack the archive in the directory of its preference, in ours in case that /usr/local/calamaris, after this already can use it. Below we have necessary a simple example of comando1 for generation of the reports of log of the Squid.# /usr/local/calamaris/calamaris -a -F html /var/log/squid/access.log >/srv/www/default/html/calamaris/index.html

The command above already is enough for generation of excellent reports of the analysis of logs. In this in case that we use the option – the one that says to the Calamaris to be generated all the reports, – F HTML specifies the format of the report that we want, in the case in HTML, /var/log/squid/access.log is where it is located the archive of log of the Squid and /srv/www/default/html/calamaris/index.html the localization of the generated report, in this case a folder in the tree of the Apache of form that can be analyzed of any station of the net.

Source: http://www.howtoforge..com/monitoring_squid

How to use apt-get behind proxy server (Ubuntu/Debian)

If you’re using Synaptic

Open up your Synaptic package manager (usually as root), go to Settings-> Preference -> Network. Enter your proxy server details like : username:password@proxyserver.net, and put the proxy server port (usually 8080).

If you’re using command-line apt-get

Edit your /etc/bash.bashrc file as root.

Put these line at the end of your /etc/bash.bashrc file :

export http_proxy=http://username:password@proxyserver.net:port/ export ftp_proxy=http://username:password@proxyserver.netport/ 

You can omit the username:password, if your proxy server has no password.

The lower-case export lines with quotes worked for me:
export http_proxy=”http://username:password@proxyserver.net:port”
export ftp_proxy=”http://username:password@proxyserver.net:port”
But I appended them to the file /etc/profile instead of /etc/bash.bashrc so that it would effect synaptic from the menus (and lots of other system programs).

Source : http://blog.mypapit.net/2006/02/how-to-use-apt-get-behind-proxy-server-ubuntudebian.html