Monday, October 1, 2012

Varnish reports error 503 for large files

I have been working with a startup where I had to build a network of Varnish servers, of course in front of the dynamic storage. It worked absolutely fine for smaller files of any kind, e.g. image, video, binary, etc.
But for large files, say greater than 1G, it just hung and after sometimes report error 503. This error generally directs me that Varnish is no longer there but after checking on the system, it was still there and accepting requests.
Next thing I wanted to look in was the VCL (Varnish Configuration Language) since when we talk about Varnish, we are talking about various VCLs we actually configured.
Varnish gets exhausted for large files and we have to "pipe" the requests instead of "lookup". It means that it will stop investigating the request and just sends the data back and forth. It won't cache, it streams.
So your VCL will look like this:
added to vcl_recv:
/* Bypass cache for large files. The x-pipe header is
set in vcl_fetch when a too large file is detected. */
if (req.http.x-pipe && req.restarts > 0) {
remove req.http.x-pipe;
return (pipe);

added to vcl_fetch:
# don't cache files larger than 10MB
/* Don't try to cache too large files. It appears
Varnish just crashes if we don't filter them. */
if (beresp.http.Content-Length ~ "[0-9]{8,}" ) {
set req.http.x-pipe = "1";
return (restart);
Hope it helps!!!

Thursday, March 1, 2012

Linux system administration and RHCE preparation

Thought of teaching came into my mind N number of times but I somehow couldn't initiate this task earlier. Being a Linux professional, I always looked for some tips and knowledge from the experts and professionals in the industry. Unfortunately, it did not happen because industry professionals don't have time to teach and traditional teachers don't have industry experience.
The fact is that theoretical knowledge helps very little in real time projects. So I have decided to share my knowledge with Linux newbies and professionals whom I can help with the experience I carry.
I am learning and will always be and now its your time to learn - from the professional.

Please enroll yourself into the first of the many to come online courses by simply clicking on

Happy Learning!!!

Wednesday, February 29, 2012

Linux administration and Networking commands

Here are some of the basic linux administration and networking commands. These commands are often used by system admins in day to day work. I hope this post will be helpful for some of the admins.

Display filesystem information
Most common use: df -h

Display usage
Most common use, under a specific directory: du -a
Easily and quickly identify the size of files/programs in certain directories. A word of caution is that you should not run this command from the / directory. It will actually display size for every file on the entire Linux harddisk.

Find locations of files/directories quickly across entire filesystem
Most common use: find / -name appname -type d -xdev

You may also try the commands: locate
or try slocate

Command line tool to configure or check all network cards/interfaces
Most common uses: ifconfig and also ifconfig eth0
Using the plain ifconfig command will show you the details of all the already configured network cards or interfaces. This is a great way to get a check that your network hardware is working properly.

Summary of network connections and status of sockets
Most common uses: netstat and also netstat |head and also netstat -r
Netstat command simply displays all sockets and server connections. The top few lines are usually most helpful regarding webserver administration. Therefore if you are doing basic webserver work, you can quickly read the top lines of the netstat output by including the |head (pipe and head commands). Using the -r option gives you a very good look at the network routing addresses. This is directly linked to the route command.

Checks the domain name and IP information of a server
Most common use: nslookup
You are bound to need this command for one reason or another. When performing server installation and configuration this command gives you the existing root server IP and DNS information and can also provide details from other remote servers. Therefore, it is also a very useful security command where you can lookup DNS information regarding a particular host IP that you may see showing up on your server access logs

Sends test packets to a specified server to check if it is responding properly
Most common use: ping (replace the with a true IP address)
This is an extremely useful command that is necessary to test network connectivity and response of servers. It creates a series of test packets of data that are then bounced to the server and back giving an indication whether the server is operating properly.

Lists all existing processes on the server
Most common uses: ps and also ps -A |more
The simple command will list every process associated with the specific user running on the server. This is helpful in case you run into problems and need to for instance kill a particular process that is stuck in memory.

Removes/deletes directories and files
Most common use: rm -r name (replace name with your file or directory name)
The -r option forces the command to also apply to each subdirectory within the directory. This will work for even non-empty directories.

Lists the routing tables for your server
Most common use: route -v
This is pretty much the exact same output as the command netstat -r. You can suit yourself which you prefer to run.

The super-user do command that allows you to run specific commands that require root access.
Most common use: sudo command (replace command with your specific one)
This command is useful when you are logged into a server and attempt a command that requires super-user or root privileges. In most cases, you can simply run the command through sudo, without having to log in as root. In fact, this is a very beneficial way to administer your server without daily use of the root login, which is potentially dangerous.  

Displays many system statistics and details regarding active processes
Most common use: top
This is a very useful system administrator tool that basically gives you a summary view of the system including number of users, memory usage, CPU usage, and active processes.
Allows you to change the timestamp on a file.
Most common use: touch filename
Using the basic touch command, as above, will simply force the current date and time upon the specified file. This is helpful, but not often used. 

Traces the existing network routing for a remote or local server
Most common use: traceroute hostname
(replace hostname with the name of your server such as
This is a very powerful network command that basically gives the exact route between your machine and a server. In some cases you can actually watch the network hops from country to country across an ocean, through data centers, etc.

An extension of the who command that displays details of all users currently on the server
Most common uses: w
This is a very important system admin tool I use commonly to track who is on the server and what processes they are running. It is obviously most useful when run as a superuser.

Tool used to monitor who is on the system and many other server related characteristics
Most common uses: who and also who -q and also who -b
The plain command just lists the names of users currently on the server. Using the -q option allows you to quickly view just the total number of users on the system. Using the -b option reminds you how long it has been since you rebooted that stable Linux server! 

This command is used to kill a running process on the server. You need to process id to execute the kill command. Consider you run " pstree -p " which will list all the process with its respective process id. To kill a particular process use the command " kill ".

Friday, February 24, 2012

Linux filesystem and directory structure

A file is a collection of data items stored on disk. It contains data of type picture, music, video, text, etc. Anything which is stored on disk must be in form of a file.
A directory is a set of files. In fact, Linux treats directory as a file. There is a root directory ( / ) on every linux system followed by number of sub-directories.

Linux Filesystem:
Following are the filesystems supported on Linux operating systems.

1. ext2 - This is like UNIX filesystem. It has the concept of blocks, inodes and directories.
2. ext3 - It is an enhanced version of ext2 with journaling capabilities.
3. ext4 - Ext4 has been added to Linux Kernel 2.6.28 onwards. It is a deeper improvement over ext3 than ext3 was over ext2. Ext4 modifies important data structures of the filesystem. It is a filesystem with an improved design, better performance, reliability and features
4. proc - It is a virtual filesystem present on every Linux system. It acts as an interface to internal data structures of the system
5. lsofs: A filesystem used by CDROM.
6. Sysfs: It is a ram-based filesystem initially based on ramfs. It is use to exporting kernel objects so that end user can use it easily.

Directory Structure:
Here is the Linux directory structure with brief description of each directory.
/bin - System binaries, including the command shell
/boot - Boot-up routines
/dev - Device files for all your peripherals
/etc - System configuration files
/home - User directories
/lib - Shared libraries and modules
/lost+found - Lost-cluster files, recovered from a disk-check
/mnt - Mounted file-systems
/opt - Optional software
/proc - Kernel-processes pseudo file-system
/root - Administrator’s home directory
/sbin - System administration binaries
/usr - User-oriented software
/var - Various other files: mail, spooling and logging

Learn how to setup private Git server under 10 minutes