Using Gmail as SMTP host for YouTrack

Unless you're comfortable with using your personal gmail account, create a new account that will be used only for sending notifications from YouTrack.
Open YouTrack administrative settings, and enable Email:

SMTP host	smtp.gmail.com
SMTP port	465
Mail protocol	SMTP+SSL
SMTP login	name@host
SMTP password	pAsSwOrD
Server 'from' email	name@host

Try sending a test email, if your server is not running from the same IP/Area from where you initially logged into the gmail account test will fail. The error message suggests to use browser to login, but in my case YouTrack is running on headless minimalistic remote server, installing a browser and all it dependencies would bloat the machine. SSH SOCKS proxy to the rescue!
ssh -D 2222 user@yourserver
This will create a local listening port 2222 on your desktop that can be used to proxy your browser through the server that runs YouTrack.
Set your favorite browser to use this temporary proxy, on FireFox it can be done by navigating to "about:config" and setting the following fields:

network.proxy.socks	127.0.0.1
network.proxy.socks_port	2222
network.proxy.type	1

Verify that your settings are correct by checking your browser IP address, and then login to GMail through the proxy.  Now the test email should go through.