WordPress: The plugin generated 1 characters of unexpected output

Sometimes, when you activate a plugin in the admin panel, you may see an error message like this:
The plugin generated 1 characters of unexpected output during activation. If you notice “headers already sent” messages, problems with syndication feeds or other issues, try deactivating or removing this plugin.
This is usually caused by additional white spaces or empty lines at the beginning and/or end in one of the plugin's PHP files. PHP files should start with the <?php tag in line 1 column 1 and end with a ?> on the very last line. Additional empty lines before the starting- or after the closing tag can cause this (basically harmless) error.
