Tinkering Is Fun

Using NetworkManager from the command line

2012-01-19T23:10:00.002+01:00

As much as Network Manager can be annoying at times, it's often quite useful, especially if your network configuration is not fixed.

But what if you can't boot into X for whatever reason, or if for some reason you can't start nm-applet or equivalent? If you leave network setup to NetworkManager, you have no network now. This also means no way to google for help. :-)
You can find many guides telling you to just configure the network yourself. If it's a wired network this is quite easy (with DHCP just dhclient eth0 would often just do the right thing). But for wireless networks this gets more complicated. So you might be advised to use iwconfig (for open or WEP networks), or wpa_supplicant (for all).

But if you already have the network configured in NetworkManager (and the NetworkManager service is running, or you manage to start it), I suggest you just use that. There's a command line utility to interact with the NetworkManager, called nmcli.

Here are the basic usecases you might need:

nmcli con list

lists all the configured networks NetworkManager knows about; notice the columns NAME and UUID.

To bring up a connection use either:

nmcli con up id NAME

yes, I also find it a bit confusing - but you really are supposed to put a value from the name column after the id :-), e.g. nmcli con up id MyNetwork

nmcli con up uuid UUID

you normally don't need to use this

To bring a connection down use down instead of up.

For more details see man nmcli, and nmcli --help.

Linux 3.2.1 on ALIX 2d3

2012-01-17T22:38:00.001+01:00

It's that time of the year again... Here is a linux 3.2.1 config file for ALIX 2. It's a basic config. It includes upport for the integrated Ethernet, USB, GPIO and watchdog. Also included is WiFi stack + an Atheros card, but you probably want to change this to suit your needs. Compared to the 3.0 version, this has working LEDs thanks to the new platform driver by Ed Wildgoose. The button should be accessible via raw gpio, but not via the evdev interface. Hopefully the platform driver will be extended to cover this, but this requires an GPIO evdev driver first.

HTTP proxy and apt

2011-07-03T23:57:00.001+02:00

This might be well-known, but I though I'd mention it anyway, as it seems a bit mysterious at first sight.
If you live behind a HTTP proxy you probably are used to the two ways of making your system use the proxy: by http_proxy and no_proxy environment variables and by the GNOME Network Proxy Preferences (gnome-network-properties). But either way it can happen that simple sudo aptitude or sudo apt-get install foo fail because it tries to access the internet directly, ignoring the proxy setting. So, why is that?

Both ways end up setting the http_proxy environment variable (and possibly others). But sudo, at least on Ubuntu 11.04, is set up to reset the environment. You can see it in the /etc/sudoers file:

Defaults env_reset

See sudoers(5) for details.
This option causes the http_proxy environment variable to be dropped from the environment of the command that you start, e.g. try:

$ export http_proxy=test
$ env | grep http_proxy
http_proxy=test
$ sudo env | grep http_proxy
$

On the other hand, if you start sudo via sudo -i and are setting the http_proxy variable via /etc/profile.d/ then the proxy setting is correctly propagated to the environment, because the newly launched root shell sources /etc/profile (see sudo(8)).
This means you can work around the non-working sudo aptitude by sudo -i followed by aptitude.
But this workaround doesn't solve everything. It seems that Jockey loses the http_proxy variable in the process of installing packages.
So, what is the correct solution?
It turns out you can explicitly set the http proxy in apt configuration. Create a file in /etc/apt/apt.conf.d with a name like http-proxy (arbitrary) with the following content:

Acquire::http::Proxy "http://proxy:8080";

See apt.conf(5) for more info.
This way, no matter how is apt fired up (sudo, jockey, ...), it always uses the proxy.
Btw. it would be also possible to whitelist the http_proxy variable in /etc/sudoers, but resetting the environment and setting the proxy via apt.conf seems a (little) bit safer.

Eclipse CDT: removing non-existent include paths

2011-07-03T00:48:00.001+02:00

It seems that Eclipse CDT doesn't like it when you have a non-existent path in the list of project include paths. Fair enough. You can selectively remove the non-existent paths from your project via some mouse-clicking.

But if you are generating your Eclipse project via, say, CMake and you're unfortunate enough that you can't reasonably change the CMake project files (think of evil upstream and you not wanting to patch their overly complex build system changing with every release), what do you do? After a bunch of project regenerations removing of the include paths by hand becomes annoying.

As I was experiencing this issue, I decided to hack together an Eclipse plugin to resolve this for me. And here is the result: fixincludes Eclipse CDT plugin.

It's pretty basic. Nevertheless it proved to not to be all that easy to write, as it seems the CDT devel side lacks (good, up-to-date) documentation. Or I have not found it.

Anyway, there is both a binary archive and a source archive. It should work with both Eclipse Indigo + CDT 8.0, or Eclipse Helios + CDT 7.0.

To install the binary one: Just grab the binary jar and put it into the dropins subdirectory inside your Eclipse install directory. After starting Eclipse you should see a new item in the Project menu: Fix Includes.... Clicking that should fire up a dialog asing you for CDT projects to operate on. In turn it should remove all non-existent include paths. Or at least most of them. :-)

To get the sources: You need to have the Eclipse Plug-in Development Environment installed. In File/Import... select Plug-in Development/Plug-ins and Fragments. Then point the import wizard to the directory where you downloaded the source jar (Import From: Directory). Import As: Project with source folders. Then select the project in the next screen and you should be ready to go.

P.S. To install the Plug-in Development Environment click Help/Install New Software... (Obvious, isn't it? :-)) and then select General Purpose Tools/Eclipse Plug-in Development Environment. To build the jars, use File/Export..., Plug-in Development/Deployable plug-ins and fragments.

P.P.S. While there are some bugs and some TODOs, it works enough for me. I'm putting it here in case someone else finds that useful too.

Linux 3.0 on ALIX 2d3

2011-07-02T22:32:00.001+02:00

Update: Due to the way the current ALIX LEDs driver is implemented it cannot be used at the same time the generic GPIO driver is loaded. :-/ Luckily there's some work going on to resolve the issue - see this commit in linux-next.

Just an update: Here's a linux 3.0-rc4 config file for ALIX 2, more specifically for the ALIX 2d3.
This should have all the basic things on the board running, including Ethernet, USB, ~~LEDs~~, GPIO (not tested), watchdog.
You most certainly want to hand-tune it for your particular setup. No WiFi driver is enabled in the referenced config -- if you have one (or two :-)) wireless NICs in your ALIX, you certainly want to add that to your configuration.

Creating animated SVG plots with GNU Octave and gnuplot

2011-05-15T15:33:00.001+02:00

It's relatively easy to create an animated plot as either a video (AVI, MPEG, ...) file or an animated GIF file via GNU Octave or gnuplot -- one needs only to create a sequence of images and then use a tool like mencoder or ffmpeg to combine these into a single video file.

But how about an animated SVG file? As plots are vector by nature, an animated vector file sounds cool. Gnuplot is capable of producing output graphs in SVG format, but I was not successful in finding a way of converting these into a single animated SVG.

So I hacked together something for the task. If you have a recent-enough browser (tested in Firefox 4.0.1 and Chromium 11.0.696.65) you can have a look at a trivial example.
If you're interested, grab the scripts here: ani-plot

Windows XP both native and in VirtualBox

2011-02-11T01:47:00.006+01:00

There are several good tutorials about similar topics, namely:

but I couldn't find any that would describe what I was trying to achieve.

What I had: Windows XP installed on a physical partition. What I wanted: Being able to run the installation both inside VirtualBox and also natively. I did not want to reinstall the WinXP installation nor did I want to have to fiddle with any Windows Genuine Disadvantage stuff. And I didn't want to break the native boot in the process. Also, as this was some über-corporate install I didn't want to have to ever use the WinXP install CD + its rescue console.

The second link above ([2]) was almost what I needed, but it expected that you're starting with a new windows installation (and it went the other way round: from VirtualBox to native). Nevertheless, most of what the guide says is applicable here too.

Btw. I'm not much of a windows person and many things (like the windows boot) are quite opaque to me. Probably some steps are not needed/might be adapted, but the instructions described here worked for me.

I strongly suggest you first read the above mentioned HOWTOs. Also have a look at the VirtualBox User Manual. Back up your installation. It might be useful in case you're not lucky. Also watch out -- using raw partitions with VirtualBox (or any other virtualization software for that matter) can be dangerous: A typo in the process and your data is history. Also, if you're reading this, read this whole first, before trying anything out on your box.

The outline is as follows:

1. Prepare windows for the shock

Boot into windows (yuck) and prepare the system for the shock. Take inspiration from [1]:

Run MergeIDE (There might be a way to avoid that -- see below, but in general just proceed this way.)
Create a new hardware profile to be used for the VirtualBox boot (My Computer, Properties, Hardware). The idea is to have two profiles -- one for native boot and another for VirtualBox boot. See [2], section IV, 2. Turn off timeout so that you don't end up with the wrong profile by accident and give the profiles a reasonable names. I'm not sure how badly would windows suffer if you used the other profile but I was too afraid to try. :-)

In [1] there are some rumours that you should watch out for some .sys files (namely agp440.sys, intelppm.sys) and remove/disable these. On the box I was doing this procedure there was intelppm.sys. I moved it away from windows directory. After the process was finished I moved it back and it has not caused any trouble. I don't know if it would have affected the process had I not moved the file away.

2. Allow the user who will be running the VirtualBox instance R/W access to the windows partition

There are several ways how to do this. A way I chose was:

Create a special group for this:

addgroup vbox-disk
Add myself to the group:

adduser $USER vbox-disk
Create an udev rule to change the group of the right device file to the new group:

create /etc/udev/rules.d/vbox-disk.rules with the following contents (assuming sda1 is the windows partition):

KERNEL=="sda1", GROUP="vbox-disk"

Hm, I yet have to figure out how to make udev actually change the ownership without a reboot. Anyway, a reboot works. :-)

3. Create a VMDK file that refers to the real windows partition.

You could theoretically use a whole-disk VMDK file, but unless the disk is really dedicated to windows I'd advise against that. Restrict the VM to only what it needs to use. You don't want windows to overwrite your linux partition in a fit of rage. :-)

The process is described in the VirtualBox User Manual. Assuming that windows live on the first partition of the first hard disk (typical scenario) you would do something like this:

VBoxManage internalcommands createrawvmdk -filename /path/to/file.vmdk -rawdisk /dev/sda -partitions 1 -relative

If you don't have read access to the whole disk device you need to run this as root. Then change the ownership so that the file is owned by the user who will be running the VirtualBox instance.

Make sure you're not accessing the windows partition from linux while the VM will be running. While some guides say mounting the partition read-only is ok, I would advice against doing that. I could imagine this resulting in your host oopsing. Use VirtualBox's shared folders to exchange data between host and guest.

4. Create a new virtual machine

Mostly follow what [2] has to say. Just watch out: As this is not a new install, you need to have the right setting for the IO APIC. As explained in [1] windows are quite sensitive to this change. Most probably you will need to enable it.

Set IO APIC to match your PC.
Attach the raw VMDK file as the virtual HDD, attach it to IDE controller.
Enable exactly one Ethernet adapter.

It might be possible to use SATA adapter instead of the IDE one but then I gather you'd have to have a very similar adapter to the one that VirtualBox is emulating (it seems to work ok with the Intel Matrix Storage SATA drivers -- so if you're running with these in the real machine it might just work). On the machine I was doing this procedure there were SATA drivers installed but they did not work with the VirtualBox virtual SATA adapter.

Unfortunately the Intel Matrix Storage STA driver refuses to install in native environment if you don't have a supported adapter so it seems impossible to prepare the windows installation for the virtual SATA adapter in advance. But with MergeIDE windows can be prepared at least for the VirtualBox IDE adapter. It's possible to move the VMDK to virtual SATA adapter later.

5. Fine-tune virtual machine settings

In order to not bump into the ridiculousness of the Windows Genuine Disadvantage thing you (I gather) need to copy the DMI data from your real PC into the virtual machine. Follow [2] (section I.2 - DMI BIOS settings).

Furthermore [2], section VII - Windows activation suggests:

Make a backup of %WINDIR%\system32\wpa.dbl - hopefully if you bump into the WGA ridiculousness, you can revert and try again.
Set the MAC address of the one emulated Ethernet adapter to that of your real Ethernet adapter. If you've got more of them, you might be in for some fun. See the above-referenced section VII in [2].

6. Restore windows MBR

If you have Grub in the MBR, booting the VM as it is now, provided you allowed access only to the windows partition, would result in the rescue prompt of Grub as it can't access the linux partition with the data it requires.

Luckily the VMDK file allows for its own MBR. The MBR is not written to the real disk (if it's not a whole-disk file, but a file referencing only partitions) and the data is not discarded either -- it's kept in the VMDK file. So you can have different MBR in the virtual machine and in the real box.

To boot windows all you need is to overwrite the VMDK's MBR with something that will correctly start NTLDR on the windows partition. One way to do this is to use ms-sys. Luckily SystemRescueCd contains ms-sys, so all you need to do is download SystemRescueCd image, attach it to the virtual machine and boot from the ISO image. Once you get the SystemRescueCd prompt, check that the hard drive looks as expected (VirtualBox should not allow you to access any partitions that were not listed when creating the VMDK file) and then do something like:

ms-sys -m -w /dev/sda

This effectively nukes Grub in the VMDK image. Shut down the virtual machine and detach the ISO image. After a fresh start you should see windows trying to boot.

Note: If you have the right MBR on hand, you could have used the -mbr parameter when creating the VMDK file. I said: read this whole first! :-)

7. Making windows boot to the GUI

You should now see at least the HW profile-selection menu of windows. Select the profile for VirtualBox boot and hope... Depending on how lucky you are or are not, windows will either boot into GUI or will hang/BSOD.

In my case I saw the following behaviour:

If IO APIC was disabled: The boot would hang up after loading mup.sys -- this is visible in safe mode boot.
If IO APIC was enabled: After loading mup.sys the system would seemingly do nothing for a while and then I'd get a BSOD.

Btw. to ease troubleshooting it might be (moderately) helpful to add the option /SOS into the boot.init file. Just watch out -- don't mount the partition while the VM is running and unmount it before you start the VM again.

The BSOD made me unsure about the IO APIC setting for my box and I was considering the option should be unchecked on my setup. But it turns out that, as expected, IO APIC on was the right setting, despite the BSOD.

I'm not exactly sure what caused the BSOD, but the following seems to have helped:

Select different IDE controller type. Setting it to ICH6 worked for me. The default, that was PIIX4, constantly produced bluescreens on startup for me.
Increase VM RAM size. I guess I was too harsh on windows. I increased it from 512M (on the 2G RAM box) to 1G.

After I got windows to boot I tried decreasing the memory size back to 512M to see if it would BSOD but it was not reproducible, so I'm not at all sure about the influence of the memory size, but I can't rule that out either.

If you're lucky then you got windows to boot into the login screen at this point. If not... try to play with the VM setting. Determine (for sure) if windows are using the APIC or PIC HAL and adjust the VM settings accordingly. Check that MergeIDE was ran. Change the IDE adapter type or maybe even try the SATA one. Use Google.

8. Polishing

Try native boot. If you're lucky it should still work. (Naturally use the HW profile designated for native boot.)

If you got that far then most of the work is done. What else can be done:

Install the VirtualBox guest additions. It increases the performance and usability of the guest. There are some rumours that this would have negative impact on the native boot of windows. It worked ok for me, but if you wish you can run the additions only inside the VM -- see [2], section IV, 8.
Enable VirtualBox shared folders to share files between the host and the quest.
Move the VMDK image to virtual SATA controller. Now I'm not sure to what extent is that true, but rumour has it that virtual SATA has superior performance to virtual IDE. To move to virtual SATA proceed like this:

Shut down the VM.
Add a SATA adapter in the VM config. There is no need for any image to be connected to that. Leave the VMDK image hooked to the IDE adapter for now.
Boot the VM. You should see that windows detected new hardware. Dismiss all the dialogs and run the Intell Matrix Storage driver installer instead. Note: This mostly comes from a guide on r3dux.org. There's a warning that the current Intel Matrix Storage driver might not work with VirtualBox any more and a link to an older version is provided. I used that version and it worked for me. I don't know if the current one would or would not work.
Shut down the VM.
Now you can move the VMDK image to the SATA adapter.
That's it. Boot the VM again and hope it works.

I was lucky and managed all that while the windows installation can still boot natively. (Not that I need it, but it's not my computer, so I didn't want to break it.)

Good luck!

Feel free to post correction to comments. Thanks. :-)

Using Glib::RefPtr with STL sorted containers

2010-08-20T00:36:00.002+02:00

It turns there's a bug in glibmm that prevents one from using Glib::RefPtr<T> with any of the STL sorted containers, e.g. std::map or std::set.

The issue is a side effect of Glib::RefPtr<T>:

declaring operators == and !=
declaring operator bool()
not declaring operators <, <=, >, >=

The operators == and != compare pointers for equality and work as expected. The operator bool() is defined to allow the following usage of a RefPtr:

if (ref_ptr)
    puts("is non-null");
else
    puts("is null");

STL sorted containers need operator< on the type they contain. Assuming p1 and p2 are of type Glib::RefPtr<T>, the following compiles just fine:

p1 < p2

This in turn means that e.g. std::set<Glib::RefPtr<T> > my_map; compiles. Unfortunately, with current version of glibmm, this breaks badly at runtime.

Why? As there's no custom operator<, the compiler interprets p1 < p2 like this:

(bool)p1 < (bool)p2

This is true iff p1 is NULL and p2 is non-NULL. The operators <=, >, >= also work on the results of operator bool(). So, depending on the pointers wrapped in p1 and p2, it's possible that the following would be true:

p1 != p2 && !(p1 < p2) && !(p1 > p2)

(this is true iff p1 and p2 wrap two different non-NULL pointers)

There's a bug report for the issue.

A workaround for the time being is to use a custom comparator like this:

template<typename T>
class RefPtrLess
{
public:
    bool operator() (const Glib::RefPtr<T> &a,
        const Glib::RefPtr<T> &b)
    {
        const T* const a_ptr = a.operator->();
        const T* const b_ptr = b.operator->();

        return a_ptr < b_ptr;
    }
};

typedef std::set<Glib::RefPtr<MyType>, RefPtrLess<MyType> > TMyTypeSet;

Sending git patches from a different box

2010-07-15T22:58:00.001+02:00

Here's a solution to a problem that's been bothering me for a while: I wanted to create git patches from a box where there is no real MTA and so git send-email can't be used.

One option is setting up real mail MTA or something simpler like msmtp or nullmailer. While this is the preferable, "true unix" way, I think plenty of ordinary linux users don't have this either.

If, for whatever reason, this is not an option, but you have access to another box, where a MTA is set up correctly, you can do this:

commit the change(s) into the git tree
create the patch emails with: git format-patch --to to@addr ...
transfer the produced patch email files to the box where you can send mails from
send the email(s) with: /usr/sbin/sendmail -oi -t < mail-formatted.patch

The first time I suggest you send the patch to yourself to verify it's all working as expected.

This approach has a downside: You don't see the mails you sent in your MUA sent mail folder. Depending on your MUA, there are other ways of sending the mail. If you're using mutt, you might be in luck. But in general MUAs tend to mangle the patches. :-(

Configuring linux kernel for use on ALIX 2 -- update

2010-07-11T02:51:00.000+02:00

Here is a short update on my older post on configuring linux kernel for use on ALIX2.

The general points are still valid. The specific locations of the options might have changed. But to provide something more up to date, here is a working linux 2.6.34.1 configuration for ALIX 2.

Use this only as a starting point to get a configuration that works well for you. There are plenty of options that are specific to my setup and you certainly want those changed. But you should be able to boot with this one.

Linux kernel configuration -- using configuration from an older version

2010-07-11T02:42:00.000+02:00

As you probably know if you ever configured the linux kernel yourself (possibly by make menuconfig, or if you've been with linux long enough, by make config), the current kernel configuration that would be used for build is stored in the file .config.

The options in the .config file depend on the kernel version the file was produced with. So, you can't directly use a config file from an older kernel with a newer one. But what you can do is use make oldconfig. This way, the config options that are still present in the current kernel keep their values and you are asked to configure only what options were added.

To use this, just place the old .config file into the new kernel source tree and issue make oldconfig.

Yeah, this post was inspired by the comment asking for newer ALIX 2 kernel config. A comment that I missed. Sorry. Mike. :-(

Soft-float (FPU emulation) with DJGPP

2010-04-11T21:02:00.000+02:00

If you build a program using floating-point arithmetic with DJGPP and then try to run it on a box without a FPU (such as the 386, or 486SX), the program will die at the first attemt to use the floats.

The solution for this is to tell DJGPP to use floating point emulation.

While there is the -msoft-float GCC option, which seems to be recognized in the DJGPP port too, the option does not work under DJGPP.

There is a DJGPP FAQ item about FPU emulation. It turns out the easiest way is to just add -lemu to linker options.

Also, if you target the 386, add -march=i386 to compiler options. Otherwise the program might crash, as DJGPP can use an instruction that is not supported on the 386. (Not sure about 486. Is Pentium is the default target? Or the 486? Either way, it is good practice to specify -march as the lowest CPU the app needs to run on.)

TurboVision with DJGPP and DOSEMU

2010-04-11T20:44:00.001+02:00

There's a nice port of the TUI library TurboVision to GCC. The port works on unix-like systems including linux, but it also still supports DOS (with DJGPP toolchain).

Here's a guide on how to build the DOS version.

First, grab the sources. I suggest you use the latest CVS version:

cvs -d:pserver:anonymous@tvision.cvs.sourceforge.net:/cvsroot/tvision login

(use empty password)

cvs -z3 -d:pserver:anonymous@tvision.cvs.sourceforge.net:/cvsroot/tvision co -P tvision

Then you need a DOS environment with DJGGP in it. DOSEMU + DJGPP worked ok for me (as detailed in my previous posts on the topic: 1 and 2).

Have a look into doc/install/djgpp.txt. As the file explains, you will need some extra GNU tools installed inside your DJGPP directory. Here is the list of the files that worked for me at the time of this writing:

djdev203.zip - basic DJGPP runtime
bnu219b.zip - GNU Binutils
gcc442b.zip - gcc
gpp442b.zip - g++
mak3791b.zip - GNU Make
fil41b.zip - GNU fileutils
txt20b.zip - GNU textutils
shl2011b.zip - GNU sh-utils
perl588b.zip - Perl

The TurboVision doc further mentions you will need GNU gettext. This is not a hard requirement -- you can build TurboVision without gettext support. If you do want to use gettext, be warned the procedure of installing GNU gettext is a bit complicated -- see the doc/install/djgpp.txt file and gnu/gtxt-010.40/djgpp/README inside the gettext archive (gtxt040b.zip).

So now all you need to do is put a copy of the TurboVision sources in a place that can be reached from your DOS environment, change to that directory and fire up configure.bat. (If building without gettext, add the --no-intl option.)

Unfortunately, you might just get this:

C:\tvision>configure
Configuring Turbo Vision v2.2.0 library

Unknown OS, you must do things by yourself at conflib.pl line 903.
Determining OS:

The reason here is that the "uname" command (part of GNU sh-utils) prints out:

c:\tvision>uname
??Unknow

instead of what the TurboVision configure script expects ("MS-DOS").

You either have to patch the configure script (hack the test if ($os=~/MS\-DOS/) to always pass, e.g. into if (1)) or change the uname command. (Quick and dirty hack to the uname command. Use the binary to overwrite the original djgpp\bin\uname.exe)

Either way, now the configure script should pass:

C:\tvision>configure --no-intl
Configuring Turbo Vision v2.2.0 library

Determining OS: DOS [djgpp]
Looking for a working gcc: gcc OK
C flags: -O2 -Wno-packed
C++ flags: -O2 -Wno-packed
Looking for the C++ compiler: gpp
Checking Architecture: x86
Looking for pointer size:  32 bits
Looking for prefix: c:/djgpp
Looking for GNU make: make
Looking for GNU ar: ar
Looking for install tool: install
Looking for xgettext: no
Checking DJGPP version: 2.0.3 OK
Checking for international support: disabled by user request.
Looking for allegro library: Bad command or filename - "allegro-config".
 no, disabling AlCon
Checking endianess: little endian

Generating Makefile
Configuring makefiles: intl/dummy/Makefile 
Configuring RHIDE: makes/rhide.env compat/rhide.env 
Configuring RHIDE: examples/rhide.env 
Generating configuration header: no changes
Extracting from makes/librhtv.imk: processing
Extracting from compat/compat.imk: processing
Processing winnt/bccmake.in => winnt/Makefile
Processing winnt/msvcmake.in => winnt/Makefile.nmk
Makefiles for examples.
Makefiles for translations.
Processing intl/gnumake.in => intl/Makefile
Processing redhat/librhtv.spec.in => redhat/librhtv-2.2.0.spec
Processing qnxrtp/tvision.qpg.in => qnxrtp/tvision.qpg

Succesful configuration!

Then fire up make. With the default DOSEMU configuration you might end up with this error:

cc1plus.exe: out of memory allocating 65536 bytes after a total of 6390696 bytes
make.exe[1]: *** [../makes/obj/iffilelen.o] Error 1
make.exe[1]: Leaving directory `c:/tvision/makes'
make.exe: *** [static-lib] Error 2

The solution is to increase the amount of DPMI memory in /etc/dosemu/dosemu.conf, e.g.: $_dpmi = (0x10000) instead of the default 0x5000.

You can that use make examples to build some sample code. Start up e.g. examples\demo\demo.exe to see if the library compiled and works fine (works best with xdosemu, don't expect it to work in dumb mode :-)).

To build your own programs with the TurboVision library, just add -IC:\tvision\include to compiler options and -Lc:\tvision\makes -lrhtv to linker options. (Substitute c:\tvision with your TurboVision sources path.)

Building for DOS, part 2: Setting up DJGPP under DOSEMU

2010-01-21T00:28:00.003+01:00

First edit your /etc/dosemu/dosemu.conf: bump up the size of the available DPMI memory from the default 0x5000 KiB to something like 0x10000 or even 0x20000, as you might otherwise run out of memory on more complicated source files.
Now you can proceed with DJGPP install. -- I suggest you use the Zip File Picker, as I already mentioned in the previous post.

The Zip File Picker suggests you make the zip extracting from inside DOSEMU. But in my experience extracting from linux works ok and it's more comfortable. Do read the readme. But the first part of the installation can be as simple as this (you could pick a different target directory name, as long as it's not dev or dev/something):

get all the needed ZIP files in a directory

$ cd ~/.dosemu/drive_c
$ mkdir djgpp
$ for i in /path/to/*.zip; do unzip $i -d djgpp; done

Next step is setting the DJGPP environment variable and adding c:\djgpp\bin into the PATH. The easiest way to do this with dosemu is by editing autoexec.bat. A word of warning -- if the file is a symlink and you can't edit the file the symlink points to (as is the case with default Ubuntu install), you have to change the symlink into a (editable) copy first, e.g.:

$ cd ~/.dosemu/drive_c
$ cp autoexec.bat autoexec.bat.copy; mv autoexec.bat.copy autoexec.bat

As the file uses DOS line end characters (CR LF), if might look odd in some linux text editors (like mcedit). The easiest way is to edit the file from within dosemu -- fire up DOSEMU, then type:

edit autoexec.bat

The default dosemu autoexec.bat file sets up some environment variables and then plays some DOSEMU-specific tricks. So before the unix and lredir tricks insert these two lines:

set DJGPP=C:\DJGPP\DJGPP.ENV
set PATH=C:\DJGPP\BIN;%PATH%

The last step is adapting config.sys (use edit config.sys). Zip File Picker suggests config.sys is to contain these three lines:

files=40
fcbs=40,0
shell=c:\dos\command.com c:\dos /e:2048 /p

On my ubuntu install, the "files=40" line is already present. So just "fcbs=40,0" needs to be added. The file also contains:

shellhigh=z:\command.com /e:1024 /p

It worked ok for me if I left it at "shellhigh" and just enlarged the environment size from 1024 to DJGPP-suggested 2048:

shellhigh=z:\command.com /e:2048 /p

Restart DOSEMU (exitemu in dosemu, then fire up DOSEMU again). If everything worked ok you should be able to build and run C (and optionally other languages, if you installed those too) programs in DOSEMU:

C:\test>gcc test1.c -o test1.exe
C:\test>test1
Hello World!

Note: On one of my boxes (running Ubuntu 9.10, DOSEMU 1.4.0+svn.1828-2ubuntu2) I was unable to get g++ versions 4.4.1 and 4.4.2 working. -- But 4.4.2 is working fine for me on Gentoo with dosemu-1.4.1_pre20091009. Here is the bug report.

It seems to be possible to work around the issue for the time being: Set $_cpu_emu = "vm86sim" in /etc/dosemu/dosemu.conf and use GCC version 4.3.2.

Building for DOS, part 1

2010-01-11T23:38:00.000+01:00

And now for something completely different. :)

The unthinkable happened -- I had to build some C++ program of mine for FreeDOS. I understand that DOS is by all means a dead platform, but if you happen to have to target DOS, then read on. :)

The good news is that there is a port of GCC for DOS. It's called DJGPP. The website pretty much looks like it looked 10 years ago, which might imply it's dead. But in fact DJGPP is still alive, as the newsgroups and mailing lists prove.

There are more ways how you could use DJGPP: You could install it on a real DOS machine (yuck!). Or you could install linux to DJGPP cross-compiler (haven't tried, but doc implies it's tricky). Or you could install DJGGP inside DOSEMU. That is the way I chose.

Determining what files you need to download is a bit daunting, but DJGPP offers a handy tool, the Zip File Picker.

For DOSEMU you want to select "Build and run programs with DJGPP", then "DOSEMU". Select the languages you need. Unless you're feeling adventurous, you don't need any DOS GUI, you can stick to editing the files from linux, and just build in DOSEMU.

There are in fact more things than what the Zip File Picker offers that you can install. This includes various utilities and libraries. You can even get Perl for DOS. To see what's available, have a look into Getting djgpp - pick a mirror, then go to the "current" directory.

In second part I describe the configuration of DOSEMU for DJGPP.

Debugging non-native programs with QEMU + GDB

2009-12-14T00:38:00.004+01:00

Here, I expand further on the topic of QEMU user-mode emulation.
You probably noticed that QEMU doc mentions the option "-g port", which mentions GDB, without much further info.

Also, if you've been curious enough, you might have tried running GDB within your (say) ARM Debian chroot. If you have not, let me demonstrate:

This GDB was configured as "arm-linux-gnueabi".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
...
Reading symbols from /root/test...done.
(gdb) break main
Breakpoint 1 at 0x83ac: file test.c, line 5.
(gdb) start
Temporary breakpoint 2 at 0x83ac: file test.c, line 5.
Starting program: /root/test
qemu: Unsupported syscall: 26
Hello World!
During startup program exited normally.
(gdb)

So - gdb doesn't work because of unsupported syscall sys_ptrace. But as QEMU supports the GDB remote protocol, you can use that for debugging.

Apart form QEMU you also need a cross GDB built for your target. You can either build your own, or look here for some tips for Gentoo or Debian. Or get it from CodeSourcery.

It might be a good idea to look into the GDB manual for some remote debugging background first.

Start QEMU like this (either on a static binary, or inside foreign chroot):

$ qemu-arm -g 1234 your-binary

You can use any port number you want (as long as it's not occupied already). Then you have to use the same when telling GDB to connect to this instance of QEMU.

Launch the cross GDB:

$ armel-unknown-linux-gnu-gdb
GNU gdb (Gentoo 7.0 p1) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=armel-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.gentoo.org/>.
(gdb)

Now you need to tell GDB a few things:

what file you're debugging: use the file command for that
where to look for dynamic libraries: use the command set sysroot (you can get around this if the binary you're debugging is static; for older gdb use set solib-absolute-prefix)
where to look for sources: use the command dir (if it's not in the current directory)
how to connect to QEMU: use target remote localhost:1234 (host:port is the general syntax)
it seems GDB doesn't take sysroot into account when looking for debug symbols in case those were separated from the main libraries - so tell GDB explicitly where to look for them with set debug-file-directory

This implies you need to see (or have a current copy of) the binary and possible libraries the binary uses. In case of the foreign chroot that's easy -- you use the chroot path as the solib-absolute-prefix. Here's an example:

(gdb) file /mnt/data/debian-armel/root/test
Reading symbols from /mnt/data/debian-armel/root/test...done.
(gdb) set sysroot /mnt/data/debian-armel/
(gdb) set debug-file-directory /mnt/data/debian-armel/usr/lib/debug
(gdb) dir /mnt/data/debian-armel/root/
Source directories searched: /mnt/data/debian-armel/root:$cdir:$cwd
(gdb) target remote localhost:1234
Remote debugging using localhost:1234
warning: Can not parse XML target description; XML support was disabled at compile time
[New Remote target]
[Switching to Remote target]
0x400817e0 in ?? () from /mnt/data/debian-armel/lib/ld-linux.so.3
(gdb) break main
Breakpoint 1 at 0x83ac: file test.c, line 5.
(gdb) c
Continuing.

Breakpoint 1, main () at test.c:5
5 puts("Hello World!");
(gdb) c
Continuing.

Program exited normally.
(gdb) quit

You can automate the set up via a .gdbinit file.

Running ARM Linux on your desktop PC: The foreign chroot way

2009-12-12T02:08:00.002+01:00

Here is how you can run a non-native (ARM used here as an example) linux distro on your PC with the help of QEMU user emulation and binfmt_misc.

With QEMU user emulation you can run non-native executables. I.e. with QEMU configured for arm-linux-user you can run arm binaries. Unfortunately, without any further configuration, you can run only static executables. For dynamic executables you need to have all the libraries the executable depends on, built for the same architecture as the main executable (ARM here). This at first sight poses a problem -- you need pretty much the same standard libraries that you already have in your system (glibc, ...), but for a different architecture.

Such situation (wanting a different set of libraries) can be solved by a chroot. It would also nicely isolate your new ARM linux install. But ordinary chroot must keep the same architecture. Without one more ingredient to the mix you would only get:

chroot: failed to run command `/bin/bash': Exec format error

The last magic ingredient is binfmt_misc. Binfmt_misc generalizes the classical shebang, allowing you to associate custom interpreters to specific file types. Yes, you guessed it -- it allows us to associate ARM (or some other arch) ELF executables with our qemu. After setting up the association the executables can be run the same way as native executables, without specifying QEMU on the command line.

So, to recap: We use

qemu user emulation to run the ARM executables
chroot as a place to keep the installed distro including the required dynamic libraries
binfmt_misc to tell the kernel to run ARM ELF executables with the help of qemu

There's one last thing to resolve: We need a static build of QEMU, so that QEMU itself doesn't need any libraries. -- With dynamic QEMU, we'd find ourselves with the need of native libraries inside the chroot, which would defeat the purpouse.

Most distributions nowadays offer QEMU in their repositories, but static QEMU is not so common. But building it from the source is easy. Either grab a release (I tested 0.12) or, if you're feeling adventurous, clone QEMU git repo.

Configure QEMU with:

$ ./configure --disable-kvm --target-list=arm-linux-user --static

Invoking make should produce arm-linux-user/qemu-arm. If you want, grab a trivial ARM static executable (source) to test your qemu:

$ arm-linux-user/qemu-arm hello_world-arm-static
Hello world!

You can also check if the resulting QEMU binary is indeed static:

$ file arm-linux-user/qemu-arm
arm-linux-user/qemu-arm: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 2.6.9, not stripped

Let's set up binfmt_misc now. For ARM ELF it is (taken from QEMU's qemu-binfmt-conf.sh):

# echo ':arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/PATH/TO/QEMU:' > /proc/sys/fs/binfmt_misc/register

I find it convenient to put the qemu binary into /usr/local/bin/qemu-arm-static. You can put it anywhere you like, just keep in mind that later you want to use the same path inside your chroot. (Technically you could do with the qemu binary in the chroot only, but at least for this test we need it outside the chroot too.)

If everything worked well, you should get:

$ ./hello_world-arm-static
Hello world!

Now to the chroot: The easiest way to create one is to get a linux distribution for the target architecture. Here we use Debian, because it supports plenty or architectures, including ARM, and it super-easy to set up in a chroot. All you need is debootstrap -- many (even non-debian) distros have it in their repositories.

Because we're installing a foreign arch, we ask the installer to do only the first stage of the installation now (substitute $TARGET_PATH with your chosen target path):

# debootstrap --arch=armel --foreign --variant=minbase sid $TARGET_PATH http://ftp.cz.debian.org/debian

Now, we want to chroot to $TARGET_PATH and run the second stage of the install. For that, we need to put the QEMU binary into the chroot, so that the system can find it when starting /bin/bash inside the chroot. In my case I'd do

# mkdir -p debian-armel/usr/local/bin
# cp /usr/local/bin/qemu-arm-static debian-armel/usr/local/bin

Then we need to do some standard chroot preparations [1], [2]:

# mount --bind /dev $TARGET_PATH/dev
# mount --bind /dev/pts $TARGET_PATH/dev/pts
# mount --bind /dev/shm $TARGET_PATH/dev/shm
# mount --bind /proc $TARGET_PATH/proc
# mount --bind /sys $TARGET_PATH/sys

Now we can finally enter the chroot:

# chroot $TARGET_PATH

A word of warning: If the system can't find the interpreting binary (QEMU) inside the chroot, the error message you get is a bit unintuitive (took me a while to figure out what's wrong the first time I ran into it):

chroot: failed to run command `/bin/bash': No such file or directory

Once you enter the chroot, you should run the second stage of the Debian install:

# /debootstrap/debootstrap --second-stage

Congratulations! You've got your very own ARM Debian install on your PC! :-)

Now, there are some more things to do, but they are the same for ordinary chroot environment:

copy /etc/recolv.conf from your real machine over $TARGET_PATH/etc/resolv.conf to get DNS working
create /etc/apt/sources.list, e.g.: echo "deb http://ftp.cz.debian.org/debian/ unstable main" > /etc/apt/sources.list
do an apt-get update to make apt happy
install additional packages you want to have inside the chroot
automate the binfmt_misc (un)registration and the chroot entering/leaving: Here are scripts I'm using. You could also find schroot helpful.

Running ARM Linux on your desktop PC

2009-12-08T23:43:00.002+01:00

It comes as no surprise that you can run entire operating system inside another with the help of virtualization -- either by some sort of hypervisor, or for the more desktop-ish of us, through a virtual machine like QEMU, Bochs, VirtualBox or the proprietary VMware.

But in the typical setup the OS you're running in the virtual machine (guest) is targeted at the same architecture as the host. -- Most typical being host == guest == x86.

But some of these tools can do more:

Bochs can be used to run x86 or x86_64 code on any reasonable host, because it emulates even the CPU.
QEMU, like Bochs, can also emulate the CPU. QEMU seems to be able to achieve higher emulation speeds and supports more target architectures, albeit less host architectures.

So, the point is, with QEMU you can experiment with interesting architectures on your (most probably x86/x86_64) PC. Now, who could resist that? ;-) Ever wanted to know how does Debian for ARM feel like?

To make matters even more interesting -- with QEMU's "user emulation" and a little help from binfmt_misc you can chroot into a non-native distro install.

More on this interesting subject in a later post.

Configuring linux kernel for use on ALIX 2

2009-12-04T00:36:00.001+01:00

I'm a happy owner of two ALIX boards (2c3 and newer 2d3).

Searching the net for info about how to configure the kernel for this sweet device doesn't bring much good, so I thought I'd add my take on it.

The CPU used is an AMD Geode LX800, so it's x86-compatible. And the board itself is almost PC-compatible. This means you can run stock x86 Debian on it.

To give you some idea on the insides, here's the output of lspci:

00:01.0 Host bridge: Advanced Micro Devices [AMD] CS5536 [Geode companion] Host Bridge (rev 33)
00:01.2 Entertainment encryption device: Advanced Micro Devices [AMD] Geode LX AES Security Block
00:09.0 Ethernet controller: VIA Technologies, Inc. VT6105M [Rhine-III] (rev 96)
00:0a.0 Ethernet controller: VIA Technologies, Inc. VT6105M [Rhine-III] (rev 96)
00:0b.0 Ethernet controller: VIA Technologies, Inc. VT6105M [Rhine-III] (rev 96)
00:0f.0 ISA bridge: Advanced Micro Devices [AMD] CS5536 [Geode companion] ISA (rev 03)
00:0f.2 IDE interface: Advanced Micro Devices [AMD] CS5536 [Geode companion] IDE (rev 01)
00:0f.4 USB Controller: Advanced Micro Devices [AMD] CS5536 [Geode companion] OHC (rev 02)
00:0f.5 USB Controller: Advanced Micro Devices [AMD] CS5536 [Geode companion] EHC (rev 02)

If you want to customize the kernel yourself (instead of sticking to whatever your distro provides) here are some tips based on my experience:

Turn on General setup/Configure standard kernel features (for small systems). This lets you kill off some features that are useless on the ALIX anyway, like PC-speaker and Virtual Terminal support.
You probably want to disable any sort of namespaces/virtualization support.
In Processor type and features:

Select Geode GX/LX as the Processor family.
There is neither HPET nor MTRR, so turn that off.

There's no ACPI/APM, so you can turn that off in Power management and ACPI options. But if you intend to use selective USB suspend, you need to keep Power Management support on. (All other options in this submenu can be off.)
In Bus options: There is PCI, so that needs to be enabled. Also turn on NatSemi SCx200 support. But don't turn on NatSemi SCx200 27MHz High-Resolution Timer Support. Enable Geode Multi-Function General Purpose Timer (MFGPT) events. (As the help says this can be use as high-precision timer, turning on Processor type and features/High Resolution Timer Support might be a good idea(?))
In Device drivers:

Decide if you wish to use the legacy ATA way or the newer libata way. I've been using the libata approach for a long time without any problem, so I suggest you use that. But beware -- switching between those two results in the CF card being visible as either /dev/hda or /dev/sda and this can break you boot.
For the old ATA way choose: ATA/ATAPI/MFM/RLL support/CS5536 chipset support (or AMD CS5535 chipset support in older kernels)
For the libata way choose: Serial ATA (prod) and Parallel ATA (experimental) drivers/ATA SFF support/CS5536 PATA support
Enable Network device support/Ethernet (10 or 100Mbit)/EISA, VLB, PCI and on board controllers/VIA Rhine support. You can also enable Use MMIO instead of PIO for VIA Rhine. (Works fine for me.)
Input device support: Enable if you plan to connect any such thing to your ALIX. If it's just gonna sit on your shelf chewing some network traffic you don't need that.
Character devices: Feel free to disable Virtual terminal as it's of no use on an ALIX 2 anyway. To use the HW random number generator that the Geode contains enable Hardware Random Number Generator Core support/AMD Geode HW Random Number Generator support. Don't forget to enable the serial console: Serial drivers/8250/16550 and compatible serial support and Console on 8250/16550 and compatible serial port.
Enable Hardware Monitoring support/National Semiconductor LM90 and compatibles to be able to see what's going on inside your ALIX. (You also need to enable I2C Support for this.)
There's hadware watchdog too. To use it, enable Watchdog Timer Support/AMD Geode CS5535/CS5536 Watchdog. In debian, look into the package "watchdog" for an app to use it.
Graphics support: Feel free to uncheck everything there. :-)
Sound card support: The same, unless you buy some USB sound thingy. :-)
USB support: To get USB functionality enable EHCI HCD (USB 2.0) support and OHCI HCD support.
To get access to the ALIX leds, enable LED Support/LED Support for ALIX.2 and ALIX.3 series and also select some LED Triggers.
You can freely uncheck Real Time Clock. While there's an option to add a battery to the board so that it keeps the time, even if I enable RTC, I don't get /dev/rtc. Either way, without the battery you'll always boot into Jan 1st 2000, so I think RTC support is useless here.
You can enable GPOI Support/AMD CS5535/CS5536 GPIO (Geode Companion Device) -- This can probably used to read the status of the front ALIX button (?) Also I2C Support/I2C Hardware Bus support/Geode ACCESS.bus support seems relevant. But I don't know more and so far was too lazy to look for it. Any info is welcome. :-)

To use the hardware AES crypto engine enable Cryptographic API/Hardware crypto devices/Support for the Geode LX AES engine.
I suggest you enable Magic SysRq key (Kernel hacking/Magic SysRq key), as it works also over serial (instead of SysRq, you send a break, see Documentation/sysrq.txt).

(The menu items are as per the 2.6.31.6 kernel, but they should be reasonably similar for any kernel version that is not too distant.)

The other options pretty much depend on your specific requirements and there's not anything board-specific that influences them.

Here is a working kernel configuration file for ALIX 2 that can be used as a base for further customization.

Compiling linux kernel for x86 on x86_64

2009-12-03T00:09:00.001+01:00

Now, that's nothing overly exotic. Yet a search through the web suggests some wild solutions like setting up a real x86_64 -> x86 cross-compiler or even setting up a 32-bit linux distribution inside a chroot.

The reasoning is simple:

If you have 32-bit chroot environment with a compiler, you can just compile the 32-bit kernel there in the usual fashion.
If you have the right cross-compiler, you can use the standard kernel cross-compilation procedure.

The downside is that both options are gonna eat some space off your HDD and take some time to set up.

But if you're on x86_64, you can install the multilib version of gcc that adds support for building 32-bit binaries with a simple "-m32" option.

Then I found this hint:

$ (echo ‘#! /bin/sh’; echo ‘exec gcc -m32 “$@”‘) >~/bin/i486-linux-gnu-gcc
$ chmod +x ~/bin/i486-linux-gnu-gcc
$ for i in ar ld nm objcopy strip; do
$ ln -s `which $i` ~/bin/i486-linux-gnu-$i
$ done

This basically creates a "fake" i486-linux-gnu cross-toolchain out of the multilib x86_64/x86 toolchain. Then it can be used with kernel Makefile like this:

make ARCH=i386 CROSS_COMPILE=i486-linux-gnu- <target>

That's a way I've been using for some time. But as I was preparing this post, I fired up google once again and discovered this very brief answer:

make CROSS-COMPILE=i686-pc-linux-gnu- ARCH=i386 <target>

And that command works despite me having neither real nor "fake" i686-pc-linux-gnu- cross-toolchain installed. Experimenting further reveals that all that is needed is:

make ARCH=i386 <target>

Just don't forget you need to keep that ARCH=i386 parameter throughout the make invocations (modules_install doesn't seem to need them), e.g.:

make ARCH=i386 menuconfig
make ARCH=i386 bzImage modules

In this case of building the kernel on a different machine than the one it's supposed to run on, you certainly don't want to install the built modules into /lib/modules, so invoke the make like this:

make INSTALL_MOD_PATH=prefix modules_install

Also, don't do any of the steps as root. -- There's no need to. If you build the kernel as root, you risk accidentally overwriting your host modules or causing some other damage.

You knew it was coming

2009-12-02T23:12:00.000+01:00

I was never a big fan of blogs. And I'd never think I'd actually start one.

Anyway, the reason I hopped on is simple. I'm a software engineer who likes to tinker with computers. This quite naturally means I like free software. Specifically, I enjoy fiddling with linux, doing all kinds of odd things with it. I sometimes manage to run into things that I'm not able to find good answers to and figure out stuff in the process. Up to now, the results of my tinkering were staying locked up in my installation of Zim. But I was thinking that maybe someone else might find them useful too.

So this is the premise. Let's see how this evolves.