For all the WordPress sites I maintain, I always install the following plugins:

• BackWPup: To make full backups of my WordPress sites, and upload them to my Dropbox account. It supports a lot of other backup destinations as well. The free version is excellent, but if you like it, buy the pro version.
  My typical settings are:
  General: Archive Format: zip
  Schedule: Start job with: WordPress Cron
  Schedule: Scheduler: Type: Weekly (at eg. 02:35 in the morning – never set it to the top of the hour, because other people probably backup at that time)
  To Dropbox: File deletion: 1 (because Dropbox then keeps the past month’s backups anyways)
• Avoid Google’s Cache: Because I don’t like that Google and Internet Archive make copies of my content.
  I wrote it because a politician regretted something he wrote on his WordPress site.
• WP Updates Notifier: Sends you an email every time WordPress, a plugin or a theme has an update.
• Open external links in a new window: Once a user clicks on an external link, a new tab is opened, instead of opening the link in the current tab. This is to keep visitors on the current website. It is pure JavaScript, and doesn’t affect SEO.

Let me know if you have plugins that I should add to this list.