Skip to content

Archive for February, 2010

Saving settings in python with YAML (an XML alternative)

Feb 25 10
by mat

When transferring information or settings between computers and programs is is useful to use something simple and program independent. For a recent project I need to send a settings file between a PHP script and a python program, I needed something that would support trees so an INI file was out of the question, and I naturally thought of using XML, however when looking for more information I stumbled upon YAML (seemingly forgotten alternative to XML).

YAML is a recursive acronym that stands for YAML Ain’t Markup Language and below is a simplified example taken from the YAML site. The layout is similar to python with indents representing the nested layers. Semicolons are used to seperate variable names and value pairs.

name   : Joe Bloggs
product:
    - sku         : BL394D
      quantity    : 4
      description : Basketball
      price       : 450.00
    - sku         : BL4438H
      quantity    : 1
      description : Super Hoop
      price       : 2392.00
tax  : 251.42
total: 4443.5

To use YAML in python you will need the PyYAML library avaliable here or in via your package manager in linux. (I.E. “sudo apt-get install http://pyyaml.org/wiki/PyYAML” in ubuntu/debian).

The code for then using YAML is very simple, first we import the yaml library, then we open our settings file we have created and then use yaml’s load feature to load our settings into a python dictionary, and then finally printing it out so we can see what it contains

import yaml
f=open('settings.cfg')
settings=yaml.load(f)
print settings

Using the example YAML file above saved to settings.cfg this python script will output the following when run:

{'product': [{'sku': 'BL394D', 'price': 450.0, 'description': 
'Basketball', 'quantity': 4}, {'sku': 'BL4438H', 'price': 2392.0, 
'description': 'Super Hoop', 'quantity': 1}], 'total': 4443.5, 'tax':
251.41999999999999, 'name': 'Joe Bloggs'}

We can now access all the information in the file very easily (I.E. settings['name'] will give us the name Joe Bloggs).

YAML supports much much more than what is discussed here and further information can be found on the YAML website.

Compiling and running CUDA 2.3 SDK and toolkit on ubuntu 9.10 x64 (64-bit)

Feb 20 10
by mat

I’ve heard a lot about CUDA, such as how it is 10,000% faster at cracking wireless passwords over a conventional program/hardware, but never really got around to testing it out before now. This post details the steps required to compile and setup CUDA 2.3 SDK and toolkit on ubuntu 9.10.

Downloads
You are required to have an Nvidia graphics driver (relatively new version) already installed. First download the CUDA toolkit and CUDA sdk from the Nvidia CUDA 2.3 download page.

Install the toolkit

# Make file executable
chmod +x cudatoolkit_2.3_linux_64_ubuntu9.04.run
# Run it as superuser
sudo ./cudatoolkit_2.3_linux_64_ubuntu9.04.run

You now need to edit your .bashrc file in your home directory to include the paths (so your CUDA binaries can be found by the system)

export PATH=${PATH}:/usr/local/cuda/bin
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/local/cuda/lib64

Note if you are using 32bit then “lib64″ should be replaced with just “lib”

Install the SDK

# Make file executable
chmod +x cudasdk_2.3_linux.run
# Run it as normal user
./cudasdk_2.3_linux.run

You should now have a NVIDIA_GPU_Computing_SDK folder in your home directory. Change directory into the C folder inside this one.

cd NVIDIA_GPU_Computing_SDK/C

In this folder is a make file which will compile all the Nvidia SDK and all the demos, in order for this to work in ubuntu 9.10 (x64) you will need to install several dependencies. By installing these before attempting to make will save you a lot of time, if you are getting errors please scroll down to the problems section to see if they are already covered.

# Install the necessary libraries
sudo apt-get install freeglut3 freeglut3-dev libx11-dev mesa-common-dev libxmu6

Making and running demos

You can then run the make command, once this is ran all of the executables will be placed in NVIDIA_GPU_Computing_SDK/C/bin/linux/released . We can check that our computer has an useable CUDA device install by running the deviceQuery program:

cd ~/NVIDIA_GPU_Computing_SDK/C/bin/linux/released
./deviceQuery

This should output something similar to the following:

# ./deviceQuery
CUDA Device Query (Runtime API) version (CUDART static linking)
There is 1 device supporting CUDA

Device 0: "GeForce GTX 260"
  CUDA Driver Version:                           2.30
  CUDA Runtime Version:                          2.30
  CUDA Capability Major revision number:         1
  CUDA Capability Minor revision number:         3
  Total amount of global memory:                 938803200 bytes
  Number of multiprocessors:                     27
  Number of cores:                               216
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       16384 bytes
  Total number of registers available per block: 16384
  Warp size:                                     32
  Maximum number of threads per block:           512
  Maximum sizes of each dimension of a block:    512 x 512 x 64
  Maximum sizes of each dimension of a grid:     65535 x 65535 x 1
  Maximum memory pitch:                          262144 bytes
  Texture alignment:                             256 bytes
  Clock rate:                                    1.47 GHz
  Concurrent copy and execution:                 Yes
  Run time limit on kernels:                     Yes
  Integrated:                                    No
  Support host page-locked memory mapping:       Yes
  Compute mode:                                  Default (multiple host threads can use this device simultaneously)

Test PASSED

Now that we can see CUDA is successfully installed and a suitable device is found we can run some of nvidia’s more ascetically pleasing demos:

./fluidsGL

CUDA SDK example fluidsGL on ubuntu 9.10 x64

CUDA SDK example fluidsGL on ubuntu 9.10 x64

./smokeParticles

CUDA SDK example smokeparticles on ubuntu 9.10 x64

CUDA SDK example smokeparticles on ubuntu 9.10 x64

./particles

CUDA SDK example particles on ubuntu 9.10 x64

CUDA SDK example particles on ubuntu 9.10 x64

./postProcessGL

CUDA SDK example postProcessGL on ubuntu 9.10 x64 (teapot)

CUDA SDK example postProcessGL on ubuntu 9.10 x64 (teapot)

Problems


libxi (Nvidia forum link)

make[1]: Leaving directory `/home/mat/NVIDIA_GPU_Computing_SDK/C/common'
make[1]: Entering directory `/home/mat/NVIDIA_GPU_Computing_SDK/C/common'
In file included from ./../common/inc/paramgl.h:24,
                 from src/paramgl.cpp:19:
./../common/inc/GL/glut.h:60:20: error: GL/glu.h: No such file or directory
make[1]: *** [obj/release/paramgl.cpp.o] Error 1
make[1]: Leaving directory `/home/mat/NVIDIA_GPU_Computing_SDK/C/common'
make: *** [lib/libparamgl.so] Error 2
sudo apt-get install freeglut3 freeglut3-dev libx11-dev mesa-common-dev
/usr/include/bits/mathcalls.h:350: error: inline function ‘int __signbitf(float)’ cannot be declared weak
/usr/include/bits/mathcalls.h:350: error: inline function ‘int __signbitl(long double)’ cannot be declared weak
/usr/include/bits/mathinline.h:36: error: inline function ‘int __signbitf(float)’ cannot be declared weak
/usr/include/bits/mathinline.h:42: error: inline function ‘int __signbit(double)’ cannot be declared weak
/usr/include/bits/mathinline.h:48: error: inline function ‘int __signbitl(long double)’ cannot be declared weak
/usr/local/cuda/bin/../include/math_functions.h:442: error: inline function ‘int __signbitl(long double)’ cannot be declared weak
make[1]: *** [obj/release/particleSystem.cu.o] Error 255
make[1]: Leaving directory `/home/mat/NVIDIA_GPU_Computing_SDK/C/src/particles'
make: *** [src/particles/Makefile.ph_build] Error 2

The problem is due to having gcc 4.4 installed rather than 4.3, it is possible to install the older version of this compiler but it is simpler to modify common/common.mk and add the following extra flag (Nvidia forum link):

# Change:
NVCCFLAGS += --compiler-options -fno-strict-aliasing
# To:
NVCCFLAGS += --compiler-options -fno-strict-aliasing --compiler-options -fno-inline

and change the -O2

# Change:
COMMONFLAGS += -O2
# To: 
COMMONFLAGS += -O0

The two remaining errors you may encounter are very similar and arrise from missing libraries:

libxi (Nvidia forum link)

/usr/bin/ld: cannot find -lXi
collect2: ld returned 1 exit status
make[1]: *** [../../bin/linux/release/particles] Error 1
sudo apt-get install libxi-dev

libxmu (Nvidia forum link)

/usr/bin/ld: cannot find -lXmu
collect2: ld returned 1 exit status
make[1]: *** [../../bin/linux/release/particles] Error 1
sudo apt-get install libxmu-dev libxmu6

Advancing PyMan (using python’s pygame to recreate pacman)

Feb 18 10
by mat

A while back I followed a few tutorials on creating a pacman game in python using pygame. I took the resulting code from these tutorials and added several enhancements. Unfortunately I haven’t had enough time to continue with the code, so hopefully this code is of use to someone learning python to play with and further enhance (Who knows might end up feature complete!).

Screen shot of PyMan in action

Screen shot of PyMan in action

Enhancements over original code

  • start menu
  • sounds
  • multiple ghosts
  • not having to hold down a directional button constantly
  • can press a direction before a turn and will turn at next opertunity (like real pacman)
  • probably more stuff that I’ve forgotten about by now

Source
Source Files – The source and files, to run the game just run “python PyMan.py”

References
I believe the site where I got the original code from was learningpython.com (not 100% sure) and the following links to the tutorials:
Tutorial 1
Tutorial 2
Tutorial 3

Utilising the notification system in KDE or Gnome in bash scripts (ubuntu 9.10 / linux)

Feb 18 10
by mat

So you want to use the nice notification features available in your desktop environment (KDE or Gnome) from a script you wrote? Below explains how to do just that for the two different environments.

KDE

The code below will use kdialog (should be installed along with kde) to create a popup message that displays for 3 seconds before closing:

kdialog --passivepopup 'notification message!' 3

This should look like the image below:

kdialog passive popup notification

kdialog passive popup notification

Source: stackoverflow

Gnome

A similar tool is available for gnome, but to the best of my knowledge (I don’t really use gnome) you need to install a package. The libnotify-

sudo apt-get install libnotify-bin

The command notify-send can then be used to create notifications from your script.

notify-send -t 3000 "notification title" "notification text"

Where 3000 is the timeout in milliseconds (so 3 seconds). notify-send features some nice additional options such as the ability to include images eg:

notify-send -i /home/user/exampleicon.png -t 3000 "notification title" "notification text"

Source: Coder’s Talk

Supporting multiple phones and screensizes in your android applications

Feb 11 10
by mat

When releasing an android application it is often desirable to release your application to the largest amount of users as possible. By developing an application with android 1.5 as the target, later versions are automatically supported (1.6, 2.0, and 2.1). However different screen-sizes were introduced with android 1.6 so by default an application will not support smaller screens (larger screens are automatically supported).

Android Versions Marketshare
The image below shows the current market share for the different version of android:

Android version percentages

Android version marketshare

Android Platform Percentage of Devices
1.1 0.3%
1.5 27.7%
1.6 54.2%
2.0 2.9%
2.0.1 14.8%

Devices with small screens include the following phones T-Mobile G1, Samsung I7500, and the HTC Tattoo. Unfortunately I cannot find any statistics regarding the percentage of small screen android users (if you can please let me know). However as the fix is very simple it seems stupid not to.

Which version of android are you using?

View Results

Loading ... Loading ...

What screen size are you running android on

View Results

Loading ... Loading ...

Adding Support
To add support for different screen sizes we need to add a supports-screens tag in our android_manifest.xml. This should look like the following:

  <supports-screens
          android:largeScreens="true"
          android:normalScreens="true"
          android:smallScreens="true"
          android:anyDensity="false" />

This tells android that the application should work on large, normal and small screens. Note that only setting small to false will prevent it from been visible in the market to small screened phones, for normal and large it will simply change the method of which android interprets the display of this application. anyDensity fix me

However in order to use the supports-screens tag the target of your application must be android 1.6 (or Sdk version 4). We can allow our program which is now targetted for android 1.6 still be avaliable for android 1.5 (Sdk version 3) by adding android:minSdkVersion=”3″ to our uses-sdk tag.

<uses-sdk android:minSdkVersion="3" android:targetSdkVersion="4"/>

Now your application can be built for 1.6 supporting smaller screen sizes and still work on 1.5.

Demonstration
Below are screenshots from two AVD’s, one with a normal screen size, and one with a small screen size:

android small screen wordcube

small screen android AVD running wordcube


android medium screen wordcube

Medium screen android AVD running wordcube

Note: Eclipse bug
Unfortunately eclipse currently does not acknowledge minSdkVersion and will not give you the option to test your application in any android virtual devices (AVD’s), while this is not fixed you can manually install the package into your AVD. For this you need to know the name of your emulator instance, to do this first run the AVD and then use the followin command to find out the name.

$adb devices
List of devices attached
emulator-5554   device

So we can see that the emulator instance I need is called emulator-5554. We can now use adb to install a package into this emulator. The below example shows installing wordcubefree from the bin folder in my project to my AVD using adb.

adb -s emulator-5554 install ~/android/workspace/WordCubeFree/bin/WordCubeFree.apk
1019 KB/s (39683 bytes in 0.038s)
Can't dispatch DDM chunk 46454154: no handler defined
Can't dispatch DDM chunk 4d505251: no handler defined
        pkg: /data/local/tmp/WordCubeFree.apk
Success

Note that this will fail if the application is already installed and you will need to uninstall the application before using this command (if anyone knows a switch to force install or alternative method please let me know)

Going Further
It is possible to delve into this is more detail by providing layouts for small, medium and large screen sizes. This can be explored in more detail in the google dev support guide

Android: Using SVN with your app’s project (and eclipse)

Feb 11 10
by mat

When creating any non-trivial program using a versioning system is essential, especially when working in part of a group. This guide aims to be a quick tutorial to the SVN (subversion) tool for versioning and how to use it with an android project.

Assumptions

  • You will need SVN installed on your computer. This can be done using your package manager or by the following command in ubuntu / debian based systems:

    sudo apt-get install subversion
  • You already have an SVN repository configured. If not please view a tutorial like this or if you have a nice webhost like me (thanks dreamhost :P ) there may be a simple tool to do this automatically for you in your panel.

Note
Don’t include the files inside /bin or /gen as they are just build from the source code and will simply fill up a lot space in your SVN. But do include the folders themselves as the project will fail to build without them.

Command line (recommended)
For SVN, I am a great fan of the command line. From the few SVN GUI applications that I have used in the past I can recommend Turtoise SVN for windows and kdesvn for linux (kde) but I still prefer the command line.

The following code will checkout the project from your server. This will create a new folder called “projectname” on your computer and download the project from your server (at this point it is most likely an empty folder).

# Checkout the SVN directory
svn co svn.yourdomain.com/projectname projectname

You can then copy or create your android project in this directory. In our project folder there are two folders which contain generated files (as opposed to source files) there is no point uploading these to the svn as you will simply take up space and bandwidth. Before you decided to upload your changes to the server you should empty the bin and gen folders:

# Empty bin and gen folders
rm -rf ./projectname/bin/*
rm -rf ./projectname/gen/*

Each time you add a new file to the project you will need to add (`svn add filename` for single files or `svn add *` for all files):

# Tell SVN we want to be versioning these files
svn add projectname/*

When you are happy with your changes you can commit (`svn commit -m “message”`) your changes to the svn to create a new version, it is mandatory to include a message with each revision and it is best to be as detailed as possible with the changes made. This makes it much easier to hunt down where a bug or regression was introduced.

# Save the changes and upload to repository
svn commit -m "Initial import of projectname"

Each time you commit or wish to upgrade what is stored locally to the latest version on the server you need to use the following:

# Update the locally stored version
svn update projectname

Further points
Eclipse has a plugin to manage SVN download and install instructions can be found here.

Python: interfacing with an arduino

Feb 3 10
by mat

So what is an arduino?
An arduino is an open source open hardware programmable controller with several inputs and outputs. The image below shows an Ardunio Dicemella.

Ardunio Dicemella Annotated Photo

Ardunio Dicemella Annotated Photo

It (Arduino Dicemella) has 14 digital input/output pins (of which 6 can be used as PWM outputs), 6 analog inputs, a 16 MHz crystal oscillator, a USB connection, a power jack, an ICSP header, and a reset button. It contains everything needed to support the microcontroller; simply connect it to a computer with a USB cable or power it with a AC-to-DC adapter or battery to get started.

They are very useful for people who know how to program but have little experience with hardware interaction.

Programming the arduino
This post will not contain in-depth detail on how to program the arduino, instead focussing briefly on setting up serial (over serial or usb cable) communications in order to talk to a python script. The arduino can be programmed via a IDE provided by the creators in a C-style hardware language.

Code example

int ledPin = 13;            // choose the pin for the LED
int inputPin = 2;          // choose the input pin (for a pushbutton)
int val = 0;                // variable for reading the pin status

void setup() {
  pinMode(ledPin, OUTPUT);      // declare LED as output
  pinMode(inputPin, INPUT);     // declare pushbutton as input
}

void loop(){
  val = digitalRead(inputPin);  // read input value
  if (val == HIGH) {            // check if the input is HIGH
    digitalWrite(ledPin, HIGH);  // turn LED ON
  } else {
    digitalWrite(ledPin, LOW); // turn LED OFF
  }
}

Arduino LED switch circuit off

Arduino LED switch circuit off


Arduino LED switch circuit on

Arduino LED switch circuit on

Now we add a few lines to enable the writing of information from our arduino over the serial connection. We first need to set up the transfer speed in our setup (Serial.begin(9600);). Then we can simply send messages over serial using Serial.print(“message\n”);. You can choose between print and println with the difference been that the latter automatically appends the newline char, so we would use the former to write multiple things to the same line. Below is our modified code:

Serial write example

int ledPin = 13;           // choose the pin for the LED
int inputPin = 2;         // choose the input pin (for a pushbutton)
int val = 0;               // variable for reading the pin status

void setup() {
  pinMode(ledPin, OUTPUT);      // declare LED as output
  pinMode(inputPin, INPUT);     // declare pushbutton as input
  Serial.begin(9600);
  Serial.print("Program Initiated\n");  
}

void loop(){
  val = digitalRead(inputPin);  // read input value
  if (val == HIGH) {            // check if the input is HIGH
    digitalWrite(ledPin, HIGH);  // turn LED ON
    Serial.print("LED Activated\n");
  } else {
    digitalWrite(ledPin, LOW); // turn LED OFF
  }
}

We now add into this code the ability to receive information via serial. Below is the modified example which removes the action of the button and replaces it by activating the LED when ‘Y’ is sent via serial.

Serial read example

int ledPin = 13;  // choose the pin for the LED
int val = 0;      // variable for reading the pin status
char msg = '  ';   // variable to hold data from serial


void setup() {
  pinMode(ledPin, OUTPUT);      // declare LED as output
  Serial.begin(9600);
  Serial.print("Program Initiated\n");  
}

void loop(){
        // While data is sent over serial assign it to the msg
	while (Serial.available()>0){ 
		msg=Serial.read();
	}

  // Turn LED on/off if we recieve 'Y'/'N' over serial 
  if (msg=='Y') {            
    digitalWrite(ledPin, HIGH);  // turn LED ON
    Serial.print("LED Activated\n");
    msg=' ';
  } else if (msg=='N') {
    digitalWrite(ledPin, LOW); // turn LED OFF
  }
}

Interaction with python

First we import the serial library to python in order to communicate with the arduino (this includes talking over usb).

import serial

We then attempt to connect to our arduino on /dev/ttyUSB0, using try and except to catch an exception if we are unable to find the arduino on USB0. The 9600 corresponds to the baud rate (speed of communication) that we are using with the arduino and should be the same as set in the program on the arduino otherwise your communication may appear garbled.

try:
	arduino = serial.Serial('/dev/ttyUSB0', 9600)
except:
	print "Failed to connect on /dev/ttyUSB0"

The address will be /dev/ttyUSB# where # is replaced by a number for arduinos connected via usb and /dev/ttyS# where # is replaced by a number for arduinos connected via serial. If you are not sure of the location of your arduino, it can be found in the arduino IDE or you can write some python to scroll through possible locations until a response is found

locations=['/dev/ttyUSB0','/dev/ttyUSB1','/dev/ttyUSB2','/dev/ttyUSB3',
'/dev/ttyS0','/dev/ttyS1','/dev/ttyS2','/dev/ttyS3']

for device in locations:
	try:
		arduino = serial.Serial(device, 9600)
	except:
		print "Failed to connect on",device

You may need to be careful as other devices can be connected. For example if I try to connect to /dev/ttyS0 I will connect to the wacom tablet on my laptop.

Once you have connected to your arduino successfully you can write information to it using write and read information sent from it using read (you will need to import time to use the sleep function). If your arduino does not send any messages via serial then attempting to readline will result in your program hanging until it receives a message.

try:
	arduino.write('Y')
	time.sleep(1)
	print arduino.readline()
except:
	print "Failed to send!"

So the python code should now look like the following and we should be able to control the LED over serial.

import serial
import time

locations=['/dev/ttyUSB0','/dev/ttyUSB1','/dev/ttyUSB2','/dev/ttyUSB3',
'/dev/ttyS0','/dev/ttyS1','/dev/ttyS2','/dev/ttyS3']  
  
for device in locations:  
	try:  
		print "Trying...",device
		arduino = serial.Serial(device, 9600) 
		break
	except:  
		print "Failed to connect on",device   

try:  
    arduino.write('Y')  
    time.sleep(1)
    print arduino.readline()
except:  
    print "Failed to send!" 

The above will send the character ‘Y’ (Y for Yes please turn on the LED) to the arduino wait for 1 second and then read from the arduino which will have hopefully posted a response to our ‘Y’. Using the program on this should turn the LED on, and report LED Activated back via serial to our python program. This should be enough for people to get started with ardunios and communicating with them in python.

References

  • Arduino – The arduino website with everything you are likely to need (programming examples and reference guide, and hardware information)
  • Arduino tutorial – a basic and easy to understand tutorial on programming the arduino
  • Python port of arduino-serial.c – By John Wiseman from which I based my program.
  • original arduino-serial.c – by Tod E. Kurt.
  • Sparkfun – Here is a good place to purchase ardunio and other electronics parts. Try coolcomponents if your from the uk like me
  • Dealextreme – Hong Kong based retailer that sells a lot of cheap DIY electronics and also has worldwide free delivery with no min spend (crazy). Does take about two weeks to arrive though (uk).

How to tell android which volume (media/ringtone etc.) should be controlled by your app

Feb 1 10
by mat

In my android apps I was getting annoying problems that whilst playing sound the volume buttons would control the media volume, but when no sound was playing they would control the ringtone volume. I found the following by trial and error, hopefully this post will help people with the same problem.

This is done by placing a call of setVolumeControlStream in the onCreate part of your activity which takes on of the following values

  • AudioManager.STREAM_MUSIC
  • AudioManager.STREAM_RING
  • AudioManager.STREAM_ALARM
  • AudioManager.STREAM_NOTIFICATION
  • AudioManager.STREAM_SYSTEM
  • AudioManager.STREAM_VOICECALL

Below shows the code required to set your applications default volume control to any of the above values:

Media Volume (Music/Video etc…)

this.setVolumeControlStream(AudioManager.STREAM_MUSIC);

Ringtone Volume

this.setVolumeControlStream(AudioManager.STREAM_RING);

Alarm Volume

this.setVolumeControlStream(AudioManager.STREAM_ALARM);

Notification Volume

this.setVolumeControlStream(AudioManager.STREAM_NOTIFICATION);

System Volume

this.setVolumeControlStream(AudioManager.STREAM_SYSTEM);

Voicecall Volume

this.setVolumeControlStream(AudioManager.STREAM_VOICECALL);

I imagine this is also possible via and XML settings, if anyone knows of this I’d be very grateful to know how!