kernel config based on lsmod output

Update (October 7th, 2011): Kernels from 2.6.32 onwards should have the option to “make localmodconfig” and “make localyesconfig”. This post describes a dirty hack to get a result similar to “make localyesconfig”. If possible, use the official way instead of my python script.

After reading about the amazing 5-seconds bootup I decided to once again compile a kernel myself. Compiling a kernel is almost trivial these days, but customizing the configuration can still be quite confusing. For example, the names of the modules in lsmod aren’t the ones you select as config options. To map them, I found some scripts in the LQWiki, but they weren’t that easily to use, and also programming in bash is just painful.

So i wrote a python variant, that takes an input config(for example your distributions config) and changes the reply to “y” for all config options if the respective module is loaded(ie. if ext3 is loaded, CONFIG_EXT3_FS=y will be set).

In almost all cases you still want to tweak the resulting config file with make menuconfig. Also keep in mind that some things don’t work that easily if compiled in, for example if firmware has to be loaded from a file, the disk should be accessible.

#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script to modify kernel config based on lsmod output
Copyright (C) 2008  Andreas Goelzer
 
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
 
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
 
You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 
Based on a previous config and the output of lsmod, this script determines
which modules could be compiled in and generates a new config.
See http://andreas.goelzer.de/kernel-config-based-on-lsmod-output for updates
"""

import re;
from optparse import OptionParser;
from os import popen;
from sys import stderr, stdout, stdin;

parser = OptionParser(version="%prog 0.31");
parser.add_option("-i", "--infile", dest="cfgfile",
                  help="input config file", default=".config", metavar="FILE");
parser.add_option("-o", "--outfile", dest="outfile",
                  help="output config file", default="-", metavar="FILE");
parser.add_option("-l", "--logfile", dest="logfile",
                  help="file to log errors to", default="-", metavar="FILE");
parser.add_option("-s", "--sourcedir", dest="sourcedir",
                  help="kernel source tree", default=".", metavar="DIR");
#parser.add_option("-v", "--verbose",
                  #action="store_true", dest="verbose", default=False,
                  #help="print debug messages");

(options, args) = parser.parse_args();
if(options.outfile == '-'): of = stdout;
else: of = open(options.outfile, 'w');
if(options.logfile != '-'): stderr = open(options.logfile, 'w');


loadedmods=popen('lsmod | tail -n+2').readlines();
getmodname=re.compile(r"^(?P<modname>\w*)");

#prob. need to replace kernel with sth. like (kernel|ubuntu) for an ubuntu kernel source
parsepath=re.compile(r"/kernel(?P<path>/.*/)(?P<file>.*).ko")

wantin=[];
for module in loadedmods:
	modname = re.search(getmodname,module).group('modname');
	moduleprops=re.search(parsepath,popen('modinfo -n ' + modname).read());
	if(moduleprops):
		#search the makefile for the module name
		try:
			f=open(options.sourcedir + moduleprops.group('path') + 'Makefile' , 'r');
		except IOError:
			stderr.write('Could not find Makefile for ' + modname + '\n');
			continue
		cont=f.read();
		f.close();
		m=re.search(r"obj-\$\((?P<cfgname>[A-Z0-9_]*)\)\W*\+=\W*"+moduleprops.group('file')+r"\.o",cont);
		if(m):wantin.append(m.group('cfgname'));
		else:stderr.write('Could not determine config name for ' + modname + '\n');
	else:
		stderr.write('Could not parse modinfo for ' + modname + '\n');



if(options.cfgfile != '-'): 
	f=open(options.cfgfile, 'r');
	lines=f.readlines();
	f.close();
else:
	lines=stdin.readlines();

confparse = re.compile(r"\W*(?P<iscomment>#?)\W*(?P<cfgname>CONFIG_[A-Z0-9_]*)\W*=?\W*(?P<answer>[nmy]?)");

for line in lines:
	matches = re.search(confparse,line);
	if(matches and matches.group('cfgname') in wantin): 
		of.write(matches.group('cfgname')+'=y\n');
	else: 
		#if(matches and matches.group('answer') == 'm'):of.write(matches.group('cfgname')+'=n\n');
		of.write(line);

filetype customconfig.py (3.41 kiB, 2009-05-19)

To use the script, call it for example like ./customconfig.py -i /boot/config-2.6.24-21-generic -o .config if it is located in a linux kernel directory. Then modify the resulting config to suit your needs with make menuconfig, and then you can proceed to compile your kernel.

This entry was posted in Computers and tagged , , . Bookmark the permalink.

10 Responses to kernel config based on lsmod output

  1. Giacomo says:

    Thank you, very very much
    I was going to write down a similar script, than I’ve found this

  2. KaZe says:

    Hi, i’d like to configure my own kernel, but when i tried to run this script i get the follow error:

    facu@facu-ubunchu:~/Escritorio$ ./customconfig.py -i /boot/config-2.6.28-11-generic -o .config
    Traceback (most recent call last):
    File “./customconfig.py”, line 61, in
    f=open(options.sourcedir + moduleprops.group(‘path’) + ‘Makefile’ , ‘r’);
    IOError: [Errno 2] No such file or directory: ‘./fs/isofs/Makefile’

  3. Andreas Goelzer says:

    I replaced the script with an updated version. Now that missing Makefile error should not prevent the script from finishing, modules it cannot find won’t get compiled in – that means, if that isofs module is crucial for your setup, make sure to select it manually.

    I’m sorry, noticed that bug a while ago, but somehow forgot to update the script here :/

  4. Robert says:

    Great script! I’m trying to understand the config process and wondered if you could answer some questions. I’m trying to debloat my kernel compiles on Ubuntu 8.04 Hardy and am still fairly new at this.

    Something that hasn’t been clear to me is if lsmod lists all modules regardless of whether they’re compiled into the kernel or compiled as loadable modules or just lists loadable modules.

    Second question: Ubuntu splits it modules ( I don’t know why ) between the kernel and the lum ( linux-ubuntu-modules ) package such that most of my modules as reported by modinfo are in:

    /lib/modules/$(uname -r)/kernel/

    but some such as the ALSA sound subsystem are in:

    /lib/modules/$(uname -r)/ubuntu/

    which each correspond to the kernel and lum packages respectively.

    I store my source and configs for these two packages in

    ~/kernel-source/linux-2.6.24/.config
    ~/kernel-source/linux-ubuntu-modules-2.6.24-2.6.24/.config

    If I wanted to modify your script to work with this setup what would I need to do? I’m used to perl and don’t know the first thing about python.

  5. Andreas Goelzer says:

    lsmod only lists the modules that were dynamically loaded, not the things you compiled in. so by compiling the kernel yourself, you can really clean up things.

    if you’d want to have more than one kernel source dir, you could try nesting try…except (i think that is not the right way to do it, but i’m not that familiar with python myself), ie. something like:

    sourcedir2=…
    try:
    f=open(options.sourcedir + moduleprops.group(‘path’) + ‘Makefile’ , ‘r’);
    except IOError:
    try:
    f=open(sourcedir2 + moduleprops.group(‘path’) + ‘Makefile’ ,
    except IOError:
    stderr.write(‘Could not find Makefile for ‘ + modname + ‘\n’);
    continue

    take care of proper whitespacing :)

    As to the format of the two config files, i have no idea, maybe you can just cat them together or so for an input file, and then use the resulting file for both. or use /boot/config-2.6.XX-YY-generic as an input config.

    note that you can also compile vanilla kernels the usual way in ubuntu, works fine (no support though, but afaik also not with self-compiled ubuntu kernels)

  6. Robert says:

    Hi again, quick heads up. There is a bug in the pattern match which may miss some config options. I noticed this while writing my own perl version of your program. Look at:

    kernel, drivers/video/console, bitblit
    ./linux-2.6.24/drivers/video/console/Makefile

    You’ll see that sometimes there is more than one object on the line:

    obj-$(CONFIG_FRAMEBUFFER_CONSOLE) += fbcon.o bitblit.o font.o softcursor.o

    To account for this you need to re-write your match pattern to allow anything between the += and the object file name:

    m=re.search(r”obj-\$\((?P[A-Z0-9_]*)\)\W*\+=.*”+moduleprops.group(‘file’)+r”\.o”,cont);

    At least that’s what I did in Perl and I assume it should work in python too.

    Another interesting thing I noticed is that there can be either a many to one or one to many relationship between modules and config options. For instance CONFIG_FRAMEBUFFER_CONSOLE causes four different modules to be built whereas the drivers/cdrom/cdrom.o object has 5 different config option which all turn it on.

    I’m trying to see now if there is a way to automate lum configuration, although I have the feeling that one may have to be done by hand.

  7. Just wanted to say I loved the information you had here will absolutely help me out. Thanks

  8. Greetings! This is my first visit to your blog! We are a collection of volunteers and starting a new project in a community in the same niche. Your blog provided us valuable information to work on. You have done a marvellous job!

  9. sherif says:

    Hi thanks for this one ( I also read about the 5 second boot and wanted to optimize my system :D )
    There is a make localmodconfig option in the current kernel
    may be you can update your post

    Best regards sherif.

  10. Andreas Goelzer says:

    Thanks sherif, I updated the post.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>