There's 2 steps required to take a full backup.
- Backup your content
- Backup your files
To backup your content, you can use the WordPress export tool.
Or you can take a database dump or export copy of your database directly from phpMyAdmin which is the best method.
Another way to backup your content is to install a plugin which also copies your database in a SQL file format and some plugins also copy your files.
You can do this manually using FTP or using a plugin.
If you manually download a copy of your files, you would be advised to download the entire folder which includes your WordPress installation, themes, plugins and uploads folder which includes all your images/media etc.
Some themes also enable you to backup your theme settings which isn't a bad idea as well.
Note: Its suggested you store your backup away from your server preferably on your local system or an external hard disk.
If your server gets hacked, you can lose everything and most hosts don't guarantee backup.
Another option is to use a backup service like Vaultpress who specialize in WordPress.