K's Lair » python http://andreas.goelzer.de Electronics and small programs and other stuff Thu, 02 Sep 2010 12:30:24 +0000 en hourly 1 http://wordpress.org/?v=3.0.1 GPS plasmoid http://andreas.goelzer.de/gps-plasmoid http://andreas.goelzer.de/gps-plasmoid#comments Thu, 09 Apr 2009 16:16:20 +0000 Andreas Goelzer http://andreas.goelzer.de/?p=133 Continue reading →]]> Having recently acquired a bluetooth GPS receiver for as little as 20 €, and just having switched to KDE 4, i tried to combine those two, and write a little plasmoid to show the current position on the desktop.

And that is basically everything to say, enjoy the little applet.

Installation

Make sure gpsd is installed and set up properly. Also make sure the python bindings are installed.
In Ubuntu you would need to:

sudo apt-get install gpsd python-gps
sudo dpkg-reconfigure gpsd
sudo apt-get install python-plasma python-dev

After those are installed, you can proceed to install the plasmoid with plasmapkg -i.

Problems and TODOs

This is all rather dirty and bloated. The correct way to do it would probably be to write a data engine for plasma, which would provide the positional data. Maybe someone will do it, especially if Marble will one day be able to support overlays integrate GeoClue or something similar.

Right now the data travels from the GPS device to the PC in the form of NMEA sentences, gets then converted into gpsd’s own format, python-gps connects to gpsd over TCP port 2947, interprets the gpsd output, hands it over to the applet, which reformats the data yet again. For something so useless that just wastes too many CPU cycles. Interpreting GPS data entirely in python is not hard, but still a bit harder than with gpsd, especially if one wants to cover a lot of GPS devices. Don’t get me wrong, gpsd does its job quite well, it is just a bit overkill for this task.

Another improvement would be readable coordinates. Something like “10 km northwest of Zürich” or even “200 meters south of Sternen Oerlikon, Zürich” would really be much more useful.

Update February 2010: Version 0.16 now uses KDE’s geolocation data engine. This drops the (direct) dependency on gpsd, and allows for ip-based location – unfortnately i now can no longer get my gps to work with it, hoping for kubuntu lucid.

filetype gps-plasmoid_0.16.plasmoid (5.85 kiB, 2010-02-13)
filetype gps-plasmoid_0.15.plasmoid (2.59 kiB, 2009-04-11)
filetype gps-plasmoid012.png (911.48 kiB, 2009-04-09)
]]> http://andreas.goelzer.de/gps-plasmoid/feed 6 GPS Mouse in Python http://andreas.goelzer.de/gps-mouse-in-python http://andreas.goelzer.de/gps-mouse-in-python#comments Sat, 08 Nov 2008 00:15:49 +0000 Andreas Goelzer http://andreas.goelzer.de/?p=74 Continue reading →]]> Some months ago I aquired a cheap gps mouse (Royaltek RGM-2000, really cheap, 2 € + shipping) at ebay. My plans to use it in combination with an microcontroller and a display haven’t worked out, it is a lot of work and the garmin units do that job fine.
However, playing around with it in python lead to better results. Getting the data to google earth works nicely, and at least qlandkarte can read the generated gpx tracks.

Usage

  • Start the Program, add a parameter for the port your nmea device is attached to (/dev/ttyUSB0 is the default
  • Start google earth or something, load the generated kml file (/tmp/posdate by default)
  • Exit the program by pressing ctrl-c, track data will be written to /tmp/track-some time.gpx

Guess I should code something with extra geek value now.
It might be comfortable to have google earth automatically reload the file, so instead open sth. like

1
2
3
4
5
6
7
8
9
10
11
12

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.1">
<NetworkLink>
	<name>GPS</name>
	<flyToView>1</flyToView>
	<Url>
		<href>/tmp/posdata.kml</href>
		<refreshMode>onInterval</refreshMode>
		<refreshInterval>10</refreshInterval>
	</Url>
</NetworkLink>
</kml>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133

#!/usr/bin/python
"""
Script to read nmea data from a GPS mouse
Copyright (C) 2008  Andreas Goelzer
 
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
 
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
 
You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
This program reads data from a nmea device (uses the GGA sentence),
and creates a google kml file with the current position and the path
leading there. Additionally, a gpx file is generated containing the
track data with time information.
"""
 
from serial import Serial;
from time import strftime, gmtime;
from optparse import OptionParser;
 
 
hist = "";
gpx = "";
 
kmlmask="""<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://earth.google.com/kml/2.0">
<Document>
	<Placemark>
		<name>Your Position</name>
		<Point>
			<coordinates>%s</coordinates>
		</Point>
	</Placemark>
	<Placemark>
		<name>Previous positions</name>
		<description>Path that lead you here</description>
		<LineString>
			<coordinates>%s</coordinates>
		</LineString>
	</Placemark>
</Document>
</kml>"""
#			<altitudeMode>absolute</altitudeMode>
 
gpxmask = """<?xml version="1.0" encoding="UTF-8" standalone="no" ?><gpx xmlns="http://www.topografix.com/GPX/1/1" xmlns:gpxx="http://www.garmin.com/xmlschemas/GpxExtensions/v3" xmlns:gpxtpx="http://www.garmin.com/xmlschemas/TrackPointExtension/v1" creator="Colorado 300" version="1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.topografix.com/GPX/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd http://www.garmin.com/xmlschemas/GpxExtensions/v3 http://www.garmin.com/xmlschemas/GpxExtensions/v3/GpxExtensionsv3.xsd http://www.garmin.com/xmlschemas/TrackPointExtension/v1 http://www.garmin.com/xmlschemas/TrackPointExtension/v1/TrackPointExtensionv1.xsd">
<trk><name>%s</name><trkseg>%s</trkseg></trk></gpx>"""
 
def gpstime():
	return strftime("%Y-%m-%dT%H:%M:%SZ",gmtime());
 
def readlatlon(value,inverse):
	lat = float(value);
	lat2 = int(lat) / 100;
	lat = lat2 + (lat - 100 * lat2)/ 60; 
	if inverse:
		lat = -lat;
	return lat;
 
def calcchecksum(checkstr):
	cs = ord(checkstr[0]);
	for i in range(1,len(checkstr)):
		cs ^= ord(checkstr[i])
	return cs;
 
 
class sentencevalidator:
	"Check checksum and other stuff and split to fields"
	def __init__(self,line):
		if(line[0] == '$' and line[-5] == '*' and calcchecksum(line[1:-5]) ==  int(line[-4:-2],16)):
			self.valid = True;
			self.sentence = line[3:6];
			self.data = line[7:-5].split(',');
		else :
			self.valid = False;
 
parser = OptionParser(version="%prog 0.1");
parser.add_option("-s", "--serial", dest="serial",
                  help="serial terminal of the gps unit", default='/dev/ttyUSB0', metavar="FILE");
parser.add_option("-g", "--gpxfile", dest="gpxfile",
                  help="GPX output file", default="/tmp/track" + gpstime() + ".gpx", metavar="FILE");
parser.add_option("-k", "--kmlfile", dest="kmlfile",
                  help="KML output file", default="/tmp/posdata.kml", metavar="FILE");
parser.add_option("-v", "--verbose",
                  action="store_true", dest="verbose", default=False,
                  help="print debug messages");
 
(options, args) = parser.parse_args();
 
gps = Serial(options.serial, 4800, timeout=1);
 
 
#on sirf chips, activate all sentences
for i in range(1,5):
	cmd = "PSRF103,0%d,00,05,01" % i;
	realcmd="$%s*%x\r\n" % (cmd,calcchecksum(cmd));
	print realcmd;
	gps.write(realcmd)
 
try:
	while 1:
		line = gps.readline()
		if(options.verbose): print line,
		s = sentencevalidator(line);
		if(s.valid and s.sentence == "GGA" and int(s.data[5]) != 0):
			lat = readlatlon(s.data[1],s.data[2]=='S');
			lon = readlatlon(s.data[3],s.data[4]=='W');
			alt = float(s.data[8]);
 
			gpx += "\n" + '<trkpt lat="%f" lon="%f"><ele>%f</ele><time>%s</time></trkpt>' % (lon,lat,alt,gpstime())
 
			position =  "%f, %f, %f" % (lon,lat,alt);
			print position
			hist += "\n" + position;
			f=open("/tmp/posdata.kml",'w');
			f.write(kmlmask % (position,hist));
			f.close();
 
except:
	0
 
f=open(options.gpxfile,'w');
f.write(gpxmask % (gpstime(),gpx));
f.close();
 
gps.close()
filetype readgps.py (4.46 kiB, 2008-11-08)

]]> http://andreas.goelzer.de/gps-mouse-in-python/feed 4 kernel config based on lsmod output http://andreas.goelzer.de/kernel-config-based-on-lsmod-output http://andreas.goelzer.de/kernel-config-based-on-lsmod-output#comments Mon, 06 Oct 2008 11:35:20 +0000 Andreas Goelzer http://andreas.goelzer.de/?p=23 Continue reading →]]> After reading about the amazing 5-seconds bootup I decided to once again compile a kernel myself. Compiling a kernel is almost trivial these days, but customizing the configuration can still be quite confusing. For example, the names of the modules in lsmod aren’t the ones you select as config options. To map them, I found some scripts in the LQWiki, but they weren’t that easily to use, and also programming in bash is just painful.

So i wrote a python variant, that takes an input config(for example your distributions config) and changes the reply to “y” for all config options if the respective module is loaded(ie. if ext3 is loaded, CONFIG_EXT3_FS=y will be set).

In almost all cases you still want to tweak the resulting config file with make menuconfig. Also keep in mind that some things don’t work that easily if compiled in, for example if firmware has to be loaded from a file, the disk should be accessible.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script to modify kernel config based on lsmod output
Copyright (C) 2008  Andreas Goelzer
 
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
 
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
 
You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 
Based on a previous config and the output of lsmod, this script determines
which modules could be compiled in and generates a new config.
See http://andreas.goelzer.de/kernel-config-based-on-lsmod-output for updates
"""
 
import re;
from optparse import OptionParser;
from os import popen;
from sys import stderr, stdout, stdin;
 
parser = OptionParser(version="%prog 0.31");
parser.add_option("-i", "--infile", dest="cfgfile",
                  help="input config file", default=".config", metavar="FILE");
parser.add_option("-o", "--outfile", dest="outfile",
                  help="output config file", default="-", metavar="FILE");
parser.add_option("-l", "--logfile", dest="logfile",
                  help="file to log errors to", default="-", metavar="FILE");
parser.add_option("-s", "--sourcedir", dest="sourcedir",
                  help="kernel source tree", default=".", metavar="DIR");
#parser.add_option("-v", "--verbose",
                  #action="store_true", dest="verbose", default=False,
                  #help="print debug messages");
 
(options, args) = parser.parse_args();
if(options.outfile == '-'): of = stdout;
else: of = open(options.outfile, 'w');
if(options.logfile != '-'): stderr = open(options.logfile, 'w');
 
 
loadedmods=popen('lsmod | tail -n+2').readlines();
getmodname=re.compile(r"^(?P<modname>\w*)");
 
#prob. need to replace kernel with sth. like (kernel|ubuntu) for an ubuntu kernel source
parsepath=re.compile(r"/kernel(?P<path>/.*/)(?P<file>.*).ko")
 
wantin=[];
for module in loadedmods:
	modname = re.search(getmodname,module).group('modname');
	moduleprops=re.search(parsepath,popen('modinfo -n ' + modname).read());
	if(moduleprops):
		#search the makefile for the module name
		try:
			f=open(options.sourcedir + moduleprops.group('path') + 'Makefile' , 'r');
		except IOError:
			stderr.write('Could not find Makefile for ' + modname + '\n');
			continue
		cont=f.read();
		f.close();
		m=re.search(r"obj-\$\((?P<cfgname>[A-Z0-9_]*)\)\W*\+=\W*"+moduleprops.group('file')+r"\.o",cont);
		if(m):wantin.append(m.group('cfgname'));
		else:stderr.write('Could not determine config name for ' + modname + '\n');
	else:
		stderr.write('Could not parse modinfo for ' + modname + '\n');
 
 
 
if(options.cfgfile != '-'): 
	f=open(options.cfgfile, 'r');
	lines=f.readlines();
	f.close();
else:
	lines=stdin.readlines();
 
confparse = re.compile(r"\W*(?P<iscomment>#?)\W*(?P<cfgname>CONFIG_[A-Z0-9_]*)\W*=?\W*(?P<answer>[nmy]?)");
 
for line in lines:
	matches = re.search(confparse,line);
	if(matches and matches.group('cfgname') in wantin): 
		of.write(matches.group('cfgname')+'=y\n');
	else: 
		#if(matches and matches.group('answer') == 'm'):of.write(matches.group('cfgname')+'=n\n');
		of.write(line);
filetype customconfig.py (3.41 kiB, 2009-05-19)

To use the script, call it for example like ./customconfig.py -i /boot/config-2.6.24-21-generic -o .config if it is located in a linux kernel directory. Then modify the resulting config to suit your needs with make menuconfig, and then you can proceed to compile your kernel.

]]> http://andreas.goelzer.de/kernel-config-based-on-lsmod-output/feed 7 Feeding a screensaver from the digikam database http://andreas.goelzer.de/feeding-a-screensaver-from-the-digikam-database http://andreas.goelzer.de/feeding-a-screensaver-from-the-digikam-database#comments Mon, 02 Jun 2008 20:17:09 +0000 Andreas Goelzer http://andreas.goelzer.de/?p=12 Continue reading →]]> After several months of living without any screensaver, I decided it just would be nice to once again have something nice to watch and show while idling. I really like the photo slideshow that ships with windows vista, but as my laptop runs linux only that was not possible. The next best thing i found was glslideshow from the xscreensaver compilation. Like most slideshow programs, glslideshow will look into a given directory, and show files out of it in random order. However, there are downsides to that simple approach: My photo folder contains some private pictures, and also some pictures that are just ugly. So i decided to connect it to my digikam database, which was surprisingly simple.

By default, xscreensaver takes the directory out of its configfile ~/.xscreensaver, and passes it to a helper script xscreensaver-getimage-file, which then prints the complete path to a random file in that directory. So all one needs to do to add another source for pictures is to replace that file, there is already xscreensaver-getimage-flickr
, which pulls photos off flickr. So I wrote a similar script in python to print a random file from my digikam database. Maybe there are more?

Installation

The script is written in python, so that has to be installed (which is usually the case), it also requires sqlite and python bindings for it (apt-get install python-pysqlite2 in debian and ubuntu). If those are installed, backup the original xscreensaver-getimage-file, and copy the script there. Check that it is executable.
the overall process would look like(assuming xscreensaver-glslideshow already works)

sudo apt-get install python-pysqlite2
cd ~
wget http://andreas.goelzer.de/download/xscreensaver-getimage-digikam.py
chmod +x xscreensaver-getimage-digikam.py
cd /usr/bin
sudo mv xscreensaver-getimage-file xscreensaver-getimage-file-original
sudo mv ~/xscreensaver-getimage-digikam.py /usr/bin/
ln -s xscreensaver-getimage-digikam.py xscreensaver-getimage-file

If you had previously pointed xscreensaver to your digikam photo folder, that should work now. If not, edit ~/.xscreensaver and add a line pointing to that directory, in my case:

imageDirectory: /home/goelzera/Documents/Bilder

Modification

Right now this is all a bit hacky. at the beginning of the file, there is a line

bad_tags = "('private','me','Nophoto','Notmeanttobenice')"

If called, the script will take a random picture out of the database, then check if it has one of those tags. If it is tagged with one of those, it will be dismissed, similarly, if it has an extension indicating xscreensaver won’t be able to display it. So, set those line to the tags you do not want in your slideshow. For everything else you need to change the sql queries or the source code.

Running

Set your screensaver to glslideshow and wait for it to start

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102

#!/usr/bin/python
# -*- coding: utf-8 -*-
"""
Script to choose random pictures from the digikam database
Copyright (C) 2008  Andreas Goelzer
 
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
 
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
 
You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 
This program takes one random picture out of a sqlite-database
made by digikam.
This is useful for screensavers, xscreensaver-glslideshow in particular.
"""
 
#import sys;
#ef=open('/home/goelzera/Documents/Programme/python/xscreensaver-errlog','a')
#sys.stderr = ef
 
good_extensions = ('jpg', 'jpeg', 'pjpeg', 'pjpg', 'png', 'gif', 'tif', 'tiff', 'xbm', 'xpm')
bad_tags = "('private','me','Nophoto','Notmeanttobenice')"
#good_tags = "('public')"
good_tags = ""
attempts = 200
 
 
import sqlite3;
from sys import argv, exit;
from os.path import splitext;
 
digikamversion = "4";
 
if(len(argv) > 1):
	dirname = argv[len(argv) - 1]
else:
	dirname = "/home/goelzera/Documents/Pictures"
 
con = sqlite3.connect(dirname + "/digikam" + digikamversion + ".db")
if(digikamversion == "4"):
	url = "relativePath"
	dirid = "album"
else:
	url = "url"
	dirid = "dirid"
 
#from time import time
#f=open('/home/goelzera/Documents/Programme/python/xscreensaver-log','a')
#f.write(str(time()) + '\t' + str(argv) + '\t')
 
#get tagids for bad tags
def getTagIdsFromNames(names):
	tag_ids = ' ';
	for row in con.execute("SELECT id FROM Tags WHERE name IN "+ names):
		tag_ids += str(row[0]) + ','
	tag_ids = '(' + tag_ids[:-1] + ')'
	return tag_ids
 
if(bad_tags != ""): bad_tag_ids = getTagIdsFromNames(bad_tags)
if(good_tags != ""): good_tag_ids = getTagIdsFromNames(good_tags)
 
file = 'none';
#search for images, take a random one out of the db and see if it fulfills the criteria
for attempt in range(attempts):
	#get a random picture, i guess this is the wrong way, performs horrible
	#truely random, but painfully slow
	#row = con.execute("SELECT id, name, $dirid FROM Images ORDER BY RANDOM() LIMIT 0,1").fetchone()
	#fast, but not as random
	row = con.execute("SELECT id, name, "+ dirid+" FROM Images WHERE id >= (abs(RANDOM()) % (SELECT max(id) FROM Images)) LIMIT 0,1").fetchone()
	#f.write(str(row[0]) + ',')
 
	#check for bad tag
	if(bad_tags != ""):
		if(con.execute("SELECT 1 FROM ImageTags WHERE imageid = " + str(row[0]) + " AND tagid IN "+ bad_tag_ids).fetchone()): continue
 
	#check for good tag
	if(good_tags != ""):
		if(not con.execute("SELECT 1 FROM ImageTags WHERE imageid = " + str(row[0]) + " AND tagid IN "+ good_tag_ids).fetchone()): continue
 
 
	#check for invalid extension
	ext = splitext(row[1])
	ext = ext[1][1:].lower()
	if(not ext in good_extensions): continue
 
	#not rejected, get directory
	drow = con.execute("SELECT "+url+" FROM Albums WHERE id = " + str(row[2])).fetchone()
	file = dirname + drow[0] + '/' + row[1]
	break
 
print file.encode('utf-8')
 
#f.write(file.encode('latin-1') + '\n')
filetype xscreensaver-getimage-digikam.py (3.29 kiB, 2009-08-06)

]]> http://andreas.goelzer.de/feeding-a-screensaver-from-the-digikam-database/feed 6