vBulletin comes fairly secure these days, though since Internet Brands took over development, it's debatable about the security. This article though will walk you through some different steps you can use to beef up your forum security to prevent spam bots, XSS flaws, common security exploits and more.
Luckily vBulletin has had some robust security features built in from the 3.8 and prior days that hasn't been removed yet, and this can help stop exploitation before it happens.
<strong class='bbc'>Ensure you're running the latest version.[/b]
You don't have to be using vBulletin 4.0 branch to stay up to date on security. Make sure you're vBulletin is at its latest available release for your branch. This ensures that all known security exploits have been found and fixed. Also make sure you run supported versions that aren't EOL'd and don't get security updates any longer. vBulletin 2.x and vBulletin 3.0.x haven't received security patches in a while, so run them at your own risk.
<strong class='bbc'>Only install modifications from trusted sources that you must have.[/b]
One of the biggest security holes for any software is once it'd been modded. Adding modifications opens your forum up to additional exploits. Modifications released for vBulletin aren't checked for quality or security before being available for download. By using modifications you're forum doesn't need, you are leaving traces and additional possible back door exploits for hackers to access your site.
<strong class='bbc'>Run the latest server software.[/b]
This only applies to those who host on a VPS or higher. But using old out of date PHP, MySQL, and Apache (or whatever combination you use), almost always leads to exploits being carried out on the server level. Hardening your server can prevent basic brute force attempts, and script kiddies from exploiting you.
<strong class='bbc'>Set your account to undeletable/unalterable.
[/b]In includes/config.php set your administrator account to be protected from accidental (or purposful) deletion. Protecting your account makes it easier to recover, and if you're quick after an exploit makes it easier to clean up.
<strong class='bbc'>Use strong passwords.[/b]
One of the easiest ways to exploit a forum is by a weak password from staff members. Ensure that all staff (your self included sparky...) are using secure passwords. Try to use a alphanumeric character, a numeric character, and a capitalized character in your passwords.
<strong class='bbc'>Rename modcp and admincp folders.[/b]
Hiding these folders, is just an extra step to frustrate script kiddie hackers. Pick something totally random and something no one else would guess.*
<strong class='bbc'>Use question and answer member validation.[/b]
CAPTCHA and ReCAPTCHA have a failure rate against automated spam bots; CAPTCHA has a really high failure rate. The human and question verification has a 100% success rate against automated spam. Nothing will stop human spammers, so don't worry about the occasional one that gets through the cracks. Be sure to use multiple Q&A questions, to keep things random and spammers guess. Pro tip: Don't make the question to difficult as to lose members!
<strong class='bbc'>Create off server backups.[/b]
This is probably the best advice anyone can give you, and the easiest way for you to recover from a disaster. Backing up your information isn't good enough, it needs to be moved off server to a more secure location like your home computer. If you're server ever becomes compromised, and the entire thing wiped clean, then any backup there would be useless wouldn't it?