Naming Names, Part 1

I frequently go off on tangents. While watching some youtube videos I'd downloaded, I got the notion that it'd be neater to see them all grouped into the teams they are on. I was about to use my trusty, crusty, old bash function to rename them when it occurred to me that it was past due for an update. For reference, here is the function I've been using in my profile since Forever:-

function filenamesed () {
   if [ "$1" = "" -o "$2" = "" ]; then
      echo "Usage: filenamesed 'sedcmds' filename(s)"
      return
   fi
   SEDCMD="$1"
   shift
   while [ "$1" != "" ]; do
      F="$1"
      NF=`echo "$F" | sed "$SEDCMD"`
      if [ "$F" != "$NF" ]; then
         echo "$F -> $NF"
         mv -i "$F" "$NF"
      fi
      shift
   done
}

I won't explain it in depth. Suffice to say that it takes a GNU sed command as its first argument, and a bunch of files as other arguments, and then applies that sed command to the filenames. For example, when I want to change all the underscores in filenames into spaces, I would run:-

filenamesed 's/_/ /g;' videos/youtube/*.mp4

So if the original goal was "Watch some MindCrack UHC" and Tangent #1 was "rename those files", Tangent #2 is "update your crusty old shell script" and we finally come to the maximally-tangental task of bringing you lovely people along for the ride. Let's do this in Perl 5, piecemeal, and explain our thought processes along the way.

Waking Up Is Difficult

I previously posted about my new samsung n220 that I'm using for development on the go. One problem with it that's plagued me for quite a while is suspend, and more specifically, resume. And it's hardly a consolation prize that the laptop can go to sleep so easily if it can't wake up again!

The trouble was that it only periodically failed to resume, which makes it a nightmare to debug as you can never 100% trust that your latest tweaks have actually fixed the problem or not.

Well, I'm posting this update to say that I'm almost maybe probably possibly definitely mostly 99% certain that I have a fix. It's not really addressing the base cause of the problem, which I suspect is a buggy ACPI BIOS, but it serves to prevent the strange failure state that it kept getting stuck in.

C State. C State run. Run, state, run!

Intel C States are the terms used to describe how active or sleepy the processor is at any given moment. C0 means the processor is busy working at full capacity, and higher numbers refer to increasing amounts of laziness in the name of power efficiency. A lot of the time while you are using your laptop, it really isn't doing anything special, and will sit in the lowest power state - the highest numbered C state. This isn't a problem, because under normal circumstances, it takes mere microseconds to snap back to life when needed.

Unless, maybe, it goes into this deep sleep mode just as the laptop is also going into an ACPI sleep mode and maybe it's just too sleepy to wake up and just five more minutes, please.

To look at how the C states are being used, we can use powertop , a lovely little utility made by Intel (but entirely usable on non-Intel machines) to track power usage. It's great for finding ways to increase your battery life, but also handy to illustrate what your CPU is doing and when. Here's what it looks like when starting it:-

Pushing the right arrow key takes us to the Idle stats page, which shows the percentage of time spent in each state.

Unfortunately, it seems that that max C4 state is interacting badly with whatever ACPI bug I also have. So the fix is to simply tell Linux to never let the processor daydream that hard, and to give it a good poke if it tries to doze off.

Editing the /etc/default/grub file allows us to change what kernel parameters are used at boot. The line to change is the GRUB_CMDLINE_LINUX one, and we want to add the intel_idle.max_cstate=3 parameter to prevent us from ever going into the C4 state. My /etc/default/grub now looks like this:-

# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=10
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
GRUB_CMDLINE_LINUX="acpi_osi=Linux acpi_backlight=vendor intel_idle.max_cstate=3"

# Uncomment to enable BadRAM filtering, modify to suit your needs
# This works with Linux (no patch required) and with any kernel that obtains
# the memory map information from GRUB (GNU Mach, kernel of FreeBSD ...)
#GRUB_BADRAM="0x01234567,0xfefefefe,0x89abcdef,0xefefefef"

# Uncomment to disable graphical terminal (grub-pc only)
#GRUB_TERMINAL=console

# The resolution used on graphical terminal
# note that you can use only modes which your graphic card supports via VBE
# you can see them in real GRUB with the command `vbeinfo'
#GRUB_GFXMODE=640x480

# Uncomment if you don't want GRUB to pass "root=UUID=xxx" parameter to Linux
#GRUB_DISABLE_LINUX_UUID=true

# Uncomment to disable generation of recovery mode menu entries
#GRUB_DISABLE_RECOVERY="true"

# Uncomment to get a beep at grub start
#GRUB_INIT_TUNE="480 440 1"

Save the file, run sudo update-grub afterwards as directed, and reboot. There won't be any noticeable difference, but checking PowerTOP will show:-

We're no longer spending any time in the C4 state. And months and months of cautious use will show that it's actually working and hasn't failed to resume once!

Obviously, not entering the maximal power-saving state means slightly increased power draw on average and slightly reduced battery life. I haven't noticed a huge difference, personally; the power use tends to hover around 10W±1 and I still get more battery life from a full charge than I spend in a typical outing. I hope someone out there with similar problems finds this information useful. It's always so hard to find a good solution to these kinds of problems online, because they're always very specific to individual hardware and software configurations. What worked fine for one person may be completely useless for another; I tried blacklisting modules for various bits of possibly-buggy hardware, and tried different acpi-related kernel params. Just as I decided that I had finally found the solution, it would get locked up again. But I've finally got it sorted out and there's just one more problem to fix: The 3G modem!

Mixing up Minecraft

So I've put the Curses / Perl 5 stuff on hold for now while I work on a new secret project. I've been busy doing C++ and Qt stuff, making a nice GUI app to help people make remixes of their favourite Minecraft texture packs.

I started this around the new year, and was racing to get something done before the Minecraft 1.5 release. Well, so much for that! The thing with the Minecraft 1.5 release is that it splits the terrain and items sheets up into individual files, which are generally easier for people to work with. In fact, pretty much as soon as I had the idea to make this tool, Mojang announced they'd be changing how texture packs work to make things easier for users. Damn their black hearts!

Anyway, I think my tool still has a lot of potential to reduce the headache of merging bits of texture packs together and generally tweaking things. Certainly there's a few extra features on my to-do list now: the new version supports animations for everything, so that'll be a fun feature to work on.

Here's a screenshot of my not-yet ready alpha version. Dokucraft is on the left, and the default Minecraft on the right, but you could load any number of packs and arrange them how you please. The central area is where you'll be able to drag and drop tiles into to create your own custom pack, and the program will remember the steps you took to make it simpler when Minecraft and the texture packs inevitably get updates.

There's still much to do of course, but I've finally got most of the "infrastructure" code out of the way that I'm happy enough with it to show people. If you've got some special workflow or tweak that you like to apply to your Minecraft texture packs, leave a comment!

Neko's New Netbook

Around the end of last year, I decided I could really use a small portable laptop to do work on. I still have and love my old macbook, but while it is certainly "portable" it is not the sort of machine I would want to just grab and go to the park with. It has a glossy glass screen, which is ideal for crystal-clear viewing indoors but completely useless outdoors.

The Project

My n220 with its default matte display.

I started to wonder - e-ink type displays are ideal for the outdoors. And writing doesn't require a high refresh rate, really. Were there any notebooks with that sort of screen and a go-anywhere attitude? Surely there were plenty of writers out there who would love such a thing. Well, my initial searches concluded that yes, there were people who also wanted something like that - who pose the question only to get shot down on forums because "why would you want that". It's true that e-ink displays have a refresh rate problem, and perhaps that would have some implications even for typing text. I was undaunted and continued searching.

Then I found out about Pixel Qi displays. These are the same displays that were designed for the One Laptop Per Child project, and the designer is also making the small 10-inch displays available to commercial partners or interested hackers. Indoors, they work as a traditional LCD - colour and everything. The cool thing is, direct sunlight doesn't obliterate the image. Instead, it gracefully 'degrades' to a greyscale e-paper-like display, which looks great.

I learned I could shell out $800 for a "Sunbook" made by one of these partners, but the price was a bit steep for the specs, and it was basically just a Samsung n110 fitted with the screen.

Testing the screen just after replacing it.

Perhaps I could do better if I bought a netbook compatible with the display and fitted it myself! And this is exactly what I did. I trawled the Make forum for the Pixel Qi display, gathered as much info as I could, and bought a second-hand Samsung n220 from eBay. The screen, obtained via MakerShed, ended up costing me more than the laptop itself! But all up, it was still cheaper, more powerful, and more interesting to buy the parts myself and assemble them.

Up until then, I had never modified a laptop more than switching hard drives or RAM. Prising the plastic covers off the body and screen involved many scary snapping noises. Have I broken it?!? I've broken it, haven't I?! The LVDS connector was fiddly, as expected. However, all went well and I've levelled up my Hardware skill.

My New Toy

So, how happy am I with the Pixel Qi screen and the Samsung n220? Pretty damn happy! It enables me to just casually grab the netbook and go out for a walk - to one of the local parks, the library, wherever - set up and do some programming. It means that if home is too noisy to get any work done, it doesn't matter as much; the whole world is my office!

There are a few minor quibbles, certainly. There's a tiny bit of light bleed around the edge of the display (when it's dark enough that the light is even necessary, of course!). It doesn't have quite the same colour range or viewing angle that the original samsung LCD had. The netbook itself is from 2010, and while capable, isn't the fastest machine around.

But those things don't really matter. I'm not playing games on this thing, I'm working. I don't mind if code takes a little longer to compile. Most of my time is spent writing code, and that's where this netbook shines. Quite literally, in direct sunlight, because the display becomes easier to read. I can't emphasise enough how good it is to be able to use it outside. Especially since (what with the health problems) I've been super deficient on vitamin D and need to get lots more of that. But also simply because it's nice to get out into the fresh air.

Battery life was already pretty good to begin with at around 5 hours. Now that I've installed xubuntu, the screen and a small SSD, the battery indicator says I have 8 hours of charge to play with! I have never managed to use up all that charge in one go, so I'd say it's perfect.

System of Operations

Out in the park with the Pixel Qi display.

I tried out a bunch of distro combinations and settled on xubuntu 12.10, the 64-bit version. There wasn't a huge difference between 32-bit and 64-bit, they both have a slight edge over each other performance wise, but 64-bit is the future and if the CPU supports it that's really what I should be using. Memory usage isn't a terrible problem; I don't plan to keep the same Firefox habits as I do on the macbook and leave 30+ tabs open from months ago. That said, the XFCE desktop environment was much lighter on memory usage and much snappier than anything else I tried, and suits this machine ideally. I can customise the panels to maximise screen real estate, and installed Synapse for a lightning-fast little program launcher.

The other cool thing about this laptop is that it has a built-in 3G modem. It shows up as an internal USB device, and you can (in theory) configure Network Manager to build a Mobile Broadband connection using it. I haven't got this working yet; I don't think it's Linux' fault, the only SIM card I have is for my phone and perhaps there is some subtle difference I'm missing in the setup details. Perhaps when I'm earning some money from selling software here on the interwebs, I can justify spending for little extras like this. If I get it working, I'll be sure to edit this post.

Alas, I haven't yet pinned down the problem I have with pretty much all linux machines I've ever used:- Suspend and Resume. It's a game of Russian Roulette when suspending this machine, you have no idea whether it'll wake up properly or not. Happily, shutdown and boot are pretty fast, so you don't absolutely need it. Still, it'd be nice to preserve state between walkabouts. Again, I'm doing some research and suspect it's a misbehaving module that's to blame, but the infuriating thing about debugging these problems is that they're intermittent and you never know for sure if you've fixed it or not (spoiler: you haven't).

Update: I've found a workaround for the suspend/resume problem! See my latest post here.

Technical Details

To enable the display brightness controls for this laptop in Linux, you need to pass the acpi_osi=Linux acpi_backlight=vendor options to the kernel. You can put this in /etc/default/grub to get it active permanently. Fn+Up/Down arrows will now control the backlight brightness under X, but it doesn't let you go all the way down to "off", which is what you need to switch the Pixel Qi into pure reflective mode and maybe save a bit of power. To fix this, I found that you can also control what the intel graphics driver is telling the panel for its brightness level. These are two independent brightness controls, so what I've done for now is just bind a key (I used Fn-F5, XF86Launch1) to toggle the brightness completely on or off via the intel driver. Here's the script I used:-

#!/bin/bash

# To run this from a keybind without sudo wanting a password, you should make the
# file /etc/sudoers.d/qi_backlight with the contents:-
#
# james ALL=(root)NOPASSWD:/usr/bin/tee /sys/class/backlight/intel_backlight/brightness
#
# and make the file 440. Maybe make the file with a ~ on the name first, then
# finally mv it into place, or keep a root shell around; sudo likes to fall
# over and die if even the slightest thing is out of place.


# ---- Must be using correct hardware ----
[[ -f /sys/class/backlight/intel_backlight/max_brightness ]] || exit

MAX_BRIGHTNESS=`cat /sys/class/backlight/intel_backlight/max_brightness`
CURR_BRIGHTNESS=`cat /sys/class/backlight/intel_backlight/brightness`

if (( $CURR_BRIGHTNESS > 0 )); then
 NEW_BRIGHTNESS=0
else
 NEW_BRIGHTNESS="$MAX_BRIGHTNESS"
fi
echo "Changing brightness from $CURR_BRIGHTNESS to $NEW_BRIGHTNESS."

echo "$NEW_BRIGHTNESS" | sudo tee /sys/class/backlight/intel_backlight/brightness

As the comments in the script say, be careful if you're setting this up to let you sudo it without a password! A mangled sudoers file will lock you out of any root activities until you fix it (via booting in single user mode, liveUSB rescue, etc).

I'm really loving this laptop, and can't understand why the makers of Pixel Qi aren't pushing for it to be used in more tech. It's great, but if it's not easy to get a hold of, its popularity will remain limited.

Yes, there's glare in the photo, it looks much nicer in real life.

Getting Testy

Been a while 

I'd apologise about the long interval between this post and the last, but posts apologising about lack of posts gets a bit tiresome on a blog. I've continued to work on programming bit by bit, and it's been code that is necessary but just isn't terribly interesting to talk about.

However, today I decided to bite the bullet and add some testing code to my library. I think Michael Schwern sums it up nicely on the Test::Tutorial perldoc page:-

AHHHHHHH!!!! NOT TESTING! Anything but testing! Beat me, whip me, send me to Detroit, but don't make me write tests! 

Suffice to say, Test::Simple goes out of its way to make things easier on the programmer, to encourage them to actually write tests. So I've been learning how it's done, and I'll share the basics with you.

Conventions

Apparently the general convention for perl testing modules is to make a tests/ directory, wherein your test code lives with a '.t' suffix. The 'ok()' sub from the Test::Simple package does most of the work for you. The most basic test would look something like this:- 

#!/usr/bin/perl

use warnings;
use strict;
use Test::Simple tests => 1;

ok( 1 + 1 == 2, "basic addition" );

Running the script will then tell you if any of the conditions in the ok() calls returned false. The string parameter to ok() is just a convenient description to use to help find out what went wrong.

A more practical example

Checking to see if basic mathematics has failed us is all well and good, but what if we want to test our important library code? That's a little more complicated for me, since I'm not testing a system library so it's not in @INC, perl's library search path. But wouldn't you know it, perl has a module for just such an eventuality: FindBin.

#!/usr/bin/perl

use warnings;
use strict;
use utf8;
use feature 'unicode_strings';

use FindBin qw($RealBin);
use lib "$RealBin/../../";

use Test::Simple tests => 1;


# As Hierarchical is a Role, we need to mix it into a class before using it.
{
        package HierTest;
        use Moose;
        with 'Decurse::Hierarchical';
}

my $h1 = HierTest->new();

ok( $h1->isa('Decurse::Hierarchical'), "new()" );

The good news is, this no longer complained about being unable to find my module. The bad news was, the test was failing - I had made an error which in hindsight was pretty obvious, but let's walk through it together:-

$ perl tests/Hierarchical.t 
1..1
not ok 1 - new()
#   Failed test 'new()'
#   at tests/Hierarchical.t line 27.
# Looks like you failed 1 test of 1.

So, Test::Simple can tell us what test failed, but it doesn't necessarily know why. What we can do here is drop in Test::Simple's more advanced sibling, Test::More. It has the same 'ok()' sub with a few nice extra variations. 'is()' is used when your test basically boils down to a comparison, and 'isa_ok()' can do the type check for us. Omitting the preamble, my test now looked like this:-

use Test::More tests => 1;


# As Hierarchical is a Role, we need to mix it into a class before using it.
{
        package HierTest;
        use Moose;
        with 'Decurse::Hierarchical';
}

my $h1 = HierTest->new();

isa_ok( $h1, 'Decurse::Hierarchical', "new()" );

Running it reveals the mistake I made when writing the test. Hey, nobody's perfect!

james@qitest4: ~/source/perl/aitl/libs/Decurse
$ perl tests/Hierarchical.t 
1..1
not ok 1 - new() isa Decurse::Hierarchical
#   Failed test 'new() isa Decurse::Hierarchical'
#   at tests/Hierarchical.t line 27.
#     new() isn't a 'Decurse::Hierarchical' it's a 'HierTest'
# Looks like you failed 1 test of 1.

I had mistakenly assumed the 'isa' relationship would include what roles a class consumed, but it only tests the actual class name itself, which in my test is 'HierTest'. I needed a completely different way to check the object was properly consuming that role. Naturally, Moose has a way to do that, and I only needed to change the test to read:-

ok( $h1->meta->does_role('Decurse::Hierarchical'), "new() does Hierarchical" );

After fixing that, my test suite (of one whole test) now works!

james@qitest4: ~/source/perl/aitl/libs/Decurse
$ perl tests/Hierarchical.t 
1..1
ok 1 - new() does Hierarchical

Now all I have to do is fill in a bunch more tests.

But why bother?

You may be wondering why I want to formalise these tests at all; I tested it once, it worked, good - why not move on to other things? Well, the important deal with writing test cases is not to verify that your code works the first time after you've written it. The point is to be able to repeatedly verify that you haven't broken anything in the future! Perhaps you refactor some code, which changes some semantics about a method call somewhere, which has a knock-on effect in a class you'd forgotten about. Having testing code, and running those tests frequently, will ensure that you catch that badness early and tell you exactly what went wrong.

That's all for now. I hope this post encourages you to write your own testing code - it doesn't have to be a big chore, after all! Next update I will probably write about the laptop I've bought specifically for programming work, I'm very excited about it.

A wild Moose appears!

Going to be occupied with basic architecture stuff for a while, and won't have much to show off. But I want to retain as much momentum as possible, so here's a short post about some cool Moose features!

Method Modifiers

Moose makes it really easy for a subclass, role, or other part of your class structure modify the methods you would ordinarily get. The manual defines method modifiers better than I can, so instead I'll show you an example of where I'm using them.

I'm making a library class called "Decurse::DecurseApp" that forms the basis of a curses-based perl application with a main loop and widgets and input handling and all that sort of thing. I want to be able to use that in application code that I write, and have that code just add the extra main loop business it needs on top of the DecurseApp structure.

In other object systems, there's plenty of patterns you can use to allow for 'hooks' into DecurseApp code, but Moose provides a neat, simple way of doing it. Here's the basics of DecurseApp:-

package Decurse::DecurseApp;
use utf8;
use Moose;

# Set $self->exit(1) for graceful exit from the main loop.
has 'should_exit' => (is => 'rw', isa => 'Bool', writer => 'exit');

# Augmentable mainloop.
sub mainloop
{
   my $self = shift;

   while ( ! $self->should_exit()) {
      # Render everything that needs to be rendered
      $self->draw();

      # Wait a moment to see if there's any input.
      $self->handle_input();

      # User application specifics can go here by augmenting mainloop.
      inner();
   }
}

sub draw
{
   my $self = shift;
   # Do all the basic widget-drawing infrastructure here.
}

sub handle_input
{
   my $self = shift;
   # Do the complicated keyboard-reading stuff here.
}

1;

Once the class is fleshed out, users of the code could define a perfectly functional DecursesApp class just by writing:-

package MyApp;
use utf8; 
use Moose; 

extends 'Decurse::DecurseApp';

1;

... but that's no fun. The whole point of structuring things like this is so that MyApp can do things inside the main loop that are specific to the application we want to write. We can use Moose's method modifiers to augment the mainloop method with our own code at the point where inner() gets called, and we can also define things that should happen before or after the usual draw() and handle_input() methods.

package MyApp;
use utf8;
use Moose; 

extends 'Decurse::DecurseApp';

augment 'mainloop' => sub
{
   my $self = shift;
   # Game event processing goes here!
};

after 'draw' => sub
{
   my $self = shift;
   # Just my own little test junk for now.
   Decurse::Palette::set_colour("bold red on black");
   Curses::addstr "This text is red on black.\n";
};

before 'handle_input' => sub
{
   my $self = shift;
   # Since the real handle_input isn't actually implemented yet...
   Curses::getch();
   $self->exit(1);
};

1;

If the syntax looks a little awkward, there's the MooseX::Declare module that enhances things with keywords like 'class', 'role' and 'method'. I'm not using that yet since I'm only just learning the Moose basics, and want to see how it works with the vanilla perl5 syntax first.

Random side-note

The Blogger interface kinda sucks when trying to put <code> or <pre> tagged text in here. Surely there's a better way?

Perlish Curses

So let's write silly little test programs. First thing to do is write a program that reads keystrokes and reports back what the keycode is.

Input

For a terminal program, anything above the standard alphanumeric keys can get exceedingly tricky, since there are many different keyboards and terminal programs available. Keystrokes like F1 and the left arrow key get returned as a sequence of escape characters: ^[OP and ^[[D in my case, but other terminal software may use different sequences. Happily, setting Curses::keypad(1); will tell ncurses to do its best at interpreting these special sequences for me, and calls to Curses::getch() will return a special value for those special keys. Less happily, as one Perl Monk discovered, the function will return characters and ctrl-keys as-is along with these special numeric values, so it can be a bit awkward to differentiate between them. Never mind, at least the rest of the library protects us from some of the C cruft inherited from curses.h - so addstr, waddstr, mvaddstr and mvwaddstr are all thankfully rolled into one.

Since we're attempting some semblance of Unicode support this time around, there's also some really !!Fun!! things to consider: What happens if I copy-paste some unicode characters (e.g. Chinese) into my terminal? And how the hell are we going to handle extended character input methods?

After playing with my keycode-reading program for a little, it seems like it won't be such a terrible problem after all. There's some special key sequences that feed a bunch of bytes into the program, and the byte sequence corresponds to the UTF-8 for the characters I pasted / entered. SCIM (or I guess it's ibus that I'm using nowadays?) seems to fire up just fine and lets me type Chinese into gnome-terminal. Curses doesn't interpret these byte sequences specially, so I'll have to do some parsing and reading of my own, but it's not the end of the world.

Output

As for "printing" characters to the screen - fun terminology that dates back to the early teletype machines where your output was literally printed onto a roll of paper -  it seems fairly easy to do in perl. There's significantly less arcane invocation required to configure things for utf-8. Here's the start of my program:-

#!/usr/bin/perl

use warnings;
use strict;
use utf8;
use Carp;
binmode(STDOUT, ":utf8");
binmode(STDERR, ":utf8");
    
use Text::CharWidth qw(mbwidth mbswidth mblen);
use Curses;

So far, so good. A little bit of standard Curses initialisation and we can print unicode text, no problem. It seems that I was worrying too much - if you can give Curses some multi-byte UTF-8 character sequence, it'll happily pass it on to your UTF-8 enabled terminal without looking too hard at it.

That's not to say things will be easy. Oh no.

Hurdles

Here's what I've got so far:-

The top line is perl warning me about a potential bug. I forget what triggered it exactly, possibly the odd numeric/character duality of what getch() gives us. The nice thing is, however, that I have a way to show them without messing up the rest of the screen.

Ordinarily, messages from warn and die would just get crapped onto the screen wherever curses left the cursor position last, and possibly get overwritten by more output. By installing a few signal handlers and shoving the message into a variable, I can print it out and ensure that future screen refreshes will still show the last error:-

my $topline = "Perl-based keycode reading program! Press q to quit.";
BEGIN {
   $SIG{__WARN__} = sub { Curses::addstr(0, 0, $_[0]); $topline = $_[0]; };
   $SIG{__DIE__} = sub { Curses::endwin(); print STDERR "We crumble!\n"; };
}

Magic.


I also print out whatever useful information I can find; the termname will be handy later when we attempt to test our programs on terminals other than gnome-terminal (and start tearing our hair out because they all have subtle differences). I've also made it so that I can toggle curses' keypad mode with ctrl-K, letting me see either the raw or interpreted character sequences.

Then we've got some boxes. This is me experimenting with how wide characters affect the coordinate system curses uses; all coordinates are given as (row, column) but the interaction between single-width and full-width characters in screen coordinates is not specified.

As it turns out, the coordinates presume single-width character cells for the screen but still let you render a full-width character - it just spills into the adjacent cell. At least, this is true for my version of Curses.pm, my version of libncursesw.so, and my version of gnome-terminal. With all those levels of indirection, it becomes difficult to control the output with any degree of certainty.

One flaw I've found so far is that when you attempt to draw a wide character inbetween two other wide characters, you get garbage hanging around on the screen. Here's a .gif of me moving a little 人 character around the rest of the screen. The garbage characters you see being left behind aren't really 'there' - forcing curses to redraw the entire screen fixes it - but it's still a nuisance. It might not really be curses, or the terminal's, or anyone's fault; asking to draw a character inbetween two others is a pretty odd request and you should expect odd results.

I started to think that I might not be able to use perl's curses after all, but in hindsight it's not so bad. You cannot always control what characters you're going to have to draw in what positions, especially with user input being a factor, but a bug like this can be mitigated:-

• For a text editor, we're likely going to be drawing each line in one pass anyway, rather than flitting around the screen and repositioning the cursor a lot.
• Likewise, if I were to make a silly little roguelike game - which I'm considering - I'd have fairly tight control over the cells used to display the game world, and could reasonably expect that I wouldn't accidentally hit this overdrawing problem.
• In all other cases, such as needing to draw some sort of "dialog box" atop previous content, a quick call to Curses::clearok(1); Curses::refresh(); Curses::clearok(0); should fix things.

Note that I don't want to just set clearok(1) and be done with it. In this mode, curses makes no assumptions about existing screen content and redraws everything from scratch. Even with today's modern computers, if I set that and then hold the arrow keys down I'm going to notice some flicker as the entire screen updates. I don't want that. I certainly don't want it over a potentially slow remote shell!

There was one final problem I ran into but I'm hoping it won't be so important - I noticed that when attempting to use some (allegedly) full-width line drawing characters for the boxes, they were being treated as though they were single-width. So I tried using the Chinese characters you see above, and things worked fine - possibly somewhere along the line some part of the chain got the wrong idea about the widths.

Otherwise, I'm pretty happy with how this turned out and will continue experimenting with perl and curses.

Curses!

So, to get back into the programming groove I am going to be remaking my clunky old text editor. Because I'm such a Qt fanboy, the logical choice would be to make a quick Qt GUI around QTextDocument and call it a day. It really would be pretty damn easy.

But! there are many instances in which a terminal-based text editor is the superior choice. SSHing between machines, you frequently don't want to bother doing X forwarding and load a heavy UI over the network. And of course, there's those rare instances when X isn't working and you need to fix it...

This means I'm going to have to get to grips with the NCurses library again. Only this time, my goal is to do things "properly" and handle Unicode and so-called "wide" characters. This is where things get Interesting, since NCurses (or at least, the System V curses library it's based on) predates Unicode and the wide character stuff was bolted on later. The documentation makes little reference to the wide version of the various functions, and there's special things to consider like ensuring the locale is set to UTF-8 and #defineing the _XOPEN_SOURCE_EXTENDED macro above all your source files. It's not going to be easy.

Decisions, Decisions...

I have a couple of options open to me to get started on this. One is, I could get my head around the C API for libncursesw5-dev, and write a nice big C++ wrapper around the bits I need so that I never have to worry about it again. It could even use Qt - not the GUI parts, but Qt Core, which has nice things like QString in it. Damn, I love QString. I love a lot of the Qt library, since it actually covers a lot more than just painting pretty widgets on the screen. Making an interface to NCurses that takes QStrings and the like may seem a little bizarre, but if bizarre keeps me entertained, then let's do that.

The other option is my old friend Perl. I'd love to try out some perl6 - I wish it were ready but it's not quite there yet, we'll leave that for another day. But perl5 is solid as a rock, has Curses bindings, and I could also take the opportunity to learn some Moose at the same time. Moose is a nice high-level object system for perl, which I'll need if I'm making more than the single-file swiss-army-sledgehammer perl scripts that I usually come up with.

The only questions are: Will wrestling with wchar_t and ncurses in C++ drive me crazy? Do the perl Curses bindings even support unicode perl strings? Why did my fresh 12.04 install fail to render Chinese in my C++ test program, when the same program works fine on my 11.04 laptop? And why do I need setlocale, and why is wcwidth not in ncurses, and... which language do I pick?

Lower your expectations

The answer to the language conundrum is simply "try both". Let's aim lower. Let's make silly little test programs that barf a few characters on screen, and maybe listen for some input. Programs to print out what keycode the user just entered are always handy for debugging. And by aiming lower, we reduce that awful overhead of planning out the entire project in one lump.

It is no good to have grand plans if you can't even get started due to your planning process spinning away in the background obsessing over details that aren't implemented yet. Just... just go write something. Something bad. Something poorly thought out. Something which you can get done and then learn from the mistakes in the next iteration.