Should I delete disabled modules in Drupal?

A good reason to remove inactive modules: drupal_system_listing(). In some recent on a Drupal 7 site, I kept noticing errors when running database updates via drush, which looked something like this:

<h1>Additional uncaught exception thrown while handling exception.</h1><h2>Original</h2><p>PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away: SELECT r.filename AS filename
FROM 
{registry} r
WHERE  (r.name LIKE :db_condition_placeholder_0 ESCAPE &amp;#039;\\&amp;#039;) AND (r.type = :db_condition_placeholder_1) ; Array
(
    [:db_condition_placeholder_0] =&amp;gt; DrupalAPCCache
    [:db_condition_placeholder_1] =&amp;gt; interface
)
 in _registry_check_code() (line 3175 of /var/www/my-site/htdocs/includes/bootstrap.inc).</p>

How could this be? We disabled and uninstalled the APC module a long time ago in favor of using Memcache instead. I believe that this is because drupal_system_listing() parses all module data of any module files that exist in the code, regardless of the module's active status. This might be a good reason to delete old modules that are no longer in use on a Drupal site. And it will speed up your site as well!

Just remember that disabling a module in Drupal isn't the same as uninstalling it. When you disable a module, it leaves all it's configuration settings in place. If you plan to permanently remove a module in Drupal, you should run it's uninstall script in Modules > Uninstall, or use the drush command

$ drush pm-uninstall [module_name]

In fact, you should probably just follow the guide on uninstalling modules on drupal.org.