There are quite a few products on the market that you can use. Some hardware will come with backup software (Although it tends to be a lite version - meaning you will have to pay for the full version) You need to establish how much data you have to backup before choosing your hardware.

If you have just MS Word files or spreadsheets then you may find that a usb memory stick is all you need. You can simply copy your files to the memory stick and have it safe in TWO places.

If you have downloaded music or a keen photographer then you are unlikely to be able to get away with just a memory stick. The next option would be an external hard drive based system. Basically its a hard drive in a fancy case. The more flashing lights doesnt mean its any better than one in an ordinary case. The prices can be wildly different though for the same "guts" 

Third option is a tape drive. These are expensive and for most people would probably not be the right solution. It basically can backup your data to a tape. This is faster than a usb drive but no more safe than the others.

Another option is to backup to another machine if you have more than one machine at home or work. The down side of this is the backup is not off site. However if you backed each machine to another you then have access to backups without buying extra hardware.

Back Up Software