10 November 2012

Getting Testy

Been a while 

I'd apologise about the long interval between this post and the last, but posts apologising about lack of posts gets a bit tiresome on a blog. I've continued to work on programming bit by bit, and it's been code that is necessary but just isn't terribly interesting to talk about.

However, today I decided to bite the bullet and add some testing code to my library. I think Michael Schwern sums it up nicely on the Test::Tutorial perldoc page:-
AHHHHHHH!!!! NOT TESTING! Anything but testing! Beat me, whip me, send me to Detroit, but don't make me write tests! 
Suffice to say, Test::Simple goes out of its way to make things easier on the programmer, to encourage them to actually write tests. So I've been learning how it's done, and I'll share the basics with you.

Conventions

Apparently the general convention for perl testing modules is to make a tests/ directory, wherein your test code lives with a '.t' suffix. The 'ok()' sub from the Test::Simple package does most of the work for you. The most basic test would look something like this:- 
#!/usr/bin/perl

use warnings;
use strict;
use Test::Simple tests => 1;

ok( 1 + 1 == 2, "basic addition" );
Running the script will then tell you if any of the conditions in the ok() calls returned false. The string parameter to ok() is just a convenient description to use to help find out what went wrong.

A more practical example

Checking to see if basic mathematics has failed us is all well and good, but what if we want to test our important library code? That's a little more complicated for me, since I'm not testing a system library so it's not in @INC, perl's library search path. But wouldn't you know it, perl has a module for just such an eventuality: FindBin.
#!/usr/bin/perl

use warnings;
use strict;
use utf8;
use feature 'unicode_strings';

use FindBin qw($RealBin);
use lib "$RealBin/../../";

use Test::Simple tests => 1;


# As Hierarchical is a Role, we need to mix it into a class before using it.
{
        package HierTest;
        use Moose;
        with 'Decurse::Hierarchical';
}

my $h1 = HierTest->new();

ok( $h1->isa('Decurse::Hierarchical'), "new()" );
The good news is, this no longer complained about being unable to find my module. The bad news was, the test was failing - I had made an error which in hindsight was pretty obvious, but let's walk through it together:-
$ perl tests/Hierarchical.t 
1..1
not ok 1 - new()
#   Failed test 'new()'
#   at tests/Hierarchical.t line 27.
# Looks like you failed 1 test of 1.
So, Test::Simple can tell us what test failed, but it doesn't necessarily know why. What we can do here is drop in Test::Simple's more advanced sibling, Test::More. It has the same 'ok()' sub with a few nice extra variations. 'is()' is used when your test basically boils down to a comparison, and 'isa_ok()' can do the type check for us. Omitting the preamble, my test now looked like this:-
use Test::More tests => 1;


# As Hierarchical is a Role, we need to mix it into a class before using it.
{
        package HierTest;
        use Moose;
        with 'Decurse::Hierarchical';
}

my $h1 = HierTest->new();

isa_ok( $h1, 'Decurse::Hierarchical', "new()" );
Running it reveals the mistake I made when writing the test. Hey, nobody's perfect!
james@qitest4: ~/source/perl/aitl/libs/Decurse
$ perl tests/Hierarchical.t 
1..1
not ok 1 - new() isa Decurse::Hierarchical
#   Failed test 'new() isa Decurse::Hierarchical'
#   at tests/Hierarchical.t line 27.
#     new() isn't a 'Decurse::Hierarchical' it's a 'HierTest'
# Looks like you failed 1 test of 1.
I had mistakenly assumed the 'isa' relationship would include what roles a class consumed, but it only tests the actual class name itself, which in my test is 'HierTest'. I needed a completely different way to check the object was properly consuming that role. Naturally, Moose has a way to do that, and I only needed to change the test to read:-
ok( $h1->meta->does_role('Decurse::Hierarchical'), "new() does Hierarchical" );
After fixing that, my test suite (of one whole test) now works!
james@qitest4: ~/source/perl/aitl/libs/Decurse
$ perl tests/Hierarchical.t 
1..1
ok 1 - new() does Hierarchical
Now all I have to do is fill in a bunch more tests.

But why bother?

You may be wondering why I want to formalise these tests at all; I tested it once, it worked, good - why not move on to other things? Well, the important deal with writing test cases is not to verify that your code works the first time after you've written it. The point is to be able to repeatedly verify that you haven't broken anything in the future! Perhaps you refactor some code, which changes some semantics about a method call somewhere, which has a knock-on effect in a class you'd forgotten about. Having testing code, and running those tests frequently, will ensure that you catch that badness early and tell you exactly what went wrong.

That's all for now. I hope this post encourages you to write your own testing code - it doesn't have to be a big chore, after all! Next update I will probably write about the laptop I've bought specifically for programming work, I'm very excited about it.

20 July 2012

A wild Moose appears!

Going to be occupied with basic architecture stuff for a while, and won't have much to show off. But I want to retain as much momentum as possible, so here's a short post about some cool Moose features!

Method Modifiers

Moose makes it really easy for a subclass, role, or other part of your class structure modify the methods you would ordinarily get. The manual defines method modifiers better than I can, so instead I'll show you an example of where I'm using them.

I'm making a library class called "Decurse::DecurseApp" that forms the basis of a curses-based perl application with a main loop and widgets and input handling and all that sort of thing. I want to be able to use that in application code that I write, and have that code just add the extra main loop business it needs on top of the DecurseApp structure.

In other object systems, there's plenty of patterns you can use to allow for 'hooks' into DecurseApp code, but Moose provides a neat, simple way of doing it. Here's the basics of DecurseApp:-

package Decurse::DecurseApp;
use utf8;
use Moose;

# Set $self->exit(1) for graceful exit from the main loop.
has 'should_exit' => (is => 'rw', isa => 'Bool', writer => 'exit');

# Augmentable mainloop.
sub mainloop
{
   my $self = shift;

   while ( ! $self->should_exit()) {
      # Render everything that needs to be rendered
      $self->draw();

      # Wait a moment to see if there's any input.
      $self->handle_input();

      # User application specifics can go here by augmenting mainloop.
      inner();
   }
}

sub draw
{
   my $self = shift;
   # Do all the basic widget-drawing infrastructure here.
}

sub handle_input
{
   my $self = shift;
   # Do the complicated keyboard-reading stuff here.
}

1;


Once the class is fleshed out, users of the code could define a perfectly functional DecursesApp class just by writing:-

package MyApp;
use utf8; 
use Moose; 

extends 'Decurse::DecurseApp';

1;


... but that's no fun. The whole point of structuring things like this is so that MyApp can do things inside the main loop that are specific to the application we want to write. We can use Moose's method modifiers to augment the mainloop method with our own code at the point where inner() gets called, and we can also define things that should happen before or after the usual draw() and handle_input() methods.

package MyApp;
use utf8;
use Moose; 

extends 'Decurse::DecurseApp';

augment 'mainloop' => sub
{
   my $self = shift;
   # Game event processing goes here!
};

after 'draw' => sub
{
   my $self = shift;
   # Just my own little test junk for now.
   Decurse::Palette::set_colour("bold red on black");
   Curses::addstr "This text is red on black.\n";
};

before 'handle_input' => sub
{
   my $self = shift;
   # Since the real handle_input isn't actually implemented yet...
   Curses::getch();
   $self->exit(1);
};

1;


If the syntax looks a little awkward, there's the MooseX::Declare module that enhances things with keywords like 'class', 'role' and 'method'. I'm not using that yet since I'm only just learning the Moose basics, and want to see how it works with the vanilla perl5 syntax first.

Random side-note

The Blogger interface kinda sucks when trying to put <code> or <pre> tagged text in here. Surely there's a better way?

25 June 2012

Perlish Curses

So let's write silly little test programs. First thing to do is write a program that reads keystrokes and reports back what the keycode is.

Input

For a terminal program, anything above the standard alphanumeric keys can get exceedingly tricky, since there are many different keyboards and terminal programs available. Keystrokes like F1 and the left arrow key get returned as a sequence of escape characters: ^[OP and ^[[D in my case, but other terminal software may use different sequences. Happily, setting Curses::keypad(1); will tell ncurses to do its best at interpreting these special sequences for me, and calls to Curses::getch() will return a special value for those special keys. Less happily, as one Perl Monk discovered, the function will return characters and ctrl-keys as-is along with these special numeric values, so it can be a bit awkward to differentiate between them. Never mind, at least the rest of the library protects us from some of the C cruft inherited from curses.h - so addstr, waddstr, mvaddstr and mvwaddstr are all thankfully rolled into one.

Since we're attempting some semblance of Unicode support this time around, there's also some really !!Fun!! things to consider: What happens if I copy-paste some unicode characters (e.g. Chinese) into my terminal? And how the hell are we going to handle extended character input methods?

After playing with my keycode-reading program for a little, it seems like it won't be such a terrible problem after all. There's some special key sequences that feed a bunch of bytes into the program, and the byte sequence corresponds to the UTF-8 for the characters I pasted / entered. SCIM (or I guess it's ibus that I'm using nowadays?) seems to fire up just fine and lets me type Chinese into gnome-terminal. Curses doesn't interpret these byte sequences specially, so I'll have to do some parsing and reading of my own, but it's not the end of the world.

Output

As for "printing" characters to the screen - fun terminology that dates back to the early teletype machines where your output was literally printed onto a roll of paper -  it seems fairly easy to do in perl. There's significantly less arcane invocation required to configure things for utf-8. Here's the start of my program:-

#!/usr/bin/perl

use warnings;
use strict;
use utf8;
use Carp;
binmode(STDOUT, ":utf8");
binmode(STDERR, ":utf8");
    
use Text::CharWidth qw(mbwidth mbswidth mblen);
use Curses;

So far, so good. A little bit of standard Curses initialisation and we can print unicode text, no problem. It seems that I was worrying too much - if you can give Curses some multi-byte UTF-8 character sequence, it'll happily pass it on to your UTF-8 enabled terminal without looking too hard at it.

That's not to say things will be easy. Oh no.

Hurdles

Here's what I've got so far:-

Screenshot of curses test program

The top line is perl warning me about a potential bug. I forget what triggered it exactly, possibly the odd numeric/character duality of what getch() gives us. The nice thing is, however, that I have a way to show them without messing up the rest of the screen.

Ordinarily, messages from warn and die would just get crapped onto the screen wherever curses left the cursor position last, and possibly get overwritten by more output. By installing a few signal handlers and shoving the message into a variable, I can print it out and ensure that future screen refreshes will still show the last error:-

my $topline = "Perl-based keycode reading program! Press q to quit.";
BEGIN {
   $SIG{__WARN__} = sub { Curses::addstr(0, 0, $_[0]); $topline = $_[0]; };
   $SIG{__DIE__} = sub { Curses::endwin(); print STDERR "We crumble!\n"; };
}

Magic.

I also print out whatever useful information I can find; the termname will be handy later when we attempt to test our programs on terminals other than gnome-terminal (and start tearing our hair out because they all have subtle differences). I've also made it so that I can toggle curses' keypad mode with ctrl-K, letting me see either the raw or interpreted character sequences.

Then we've got some boxes. This is me experimenting with how wide characters affect the coordinate system curses uses; all coordinates are given as (row, column) but the interaction between single-width and full-width characters in screen coordinates is not specified.

As it turns out, the coordinates presume single-width character cells for the screen but still let you render a full-width character - it just spills into the adjacent cell. At least, this is true for my version of Curses.pm, my version of libncursesw.so, and my version of gnome-terminal. With all those levels of indirection, it becomes difficult to control the output with any degree of certainty.

Animation of some garbage text being left behind in curses
One flaw I've found so far is that when you attempt to draw a wide character inbetween two other wide characters, you get garbage hanging around on the screen. Here's a .gif of me moving a little 人 character around the rest of the screen. The garbage characters you see being left behind aren't really 'there' - forcing curses to redraw the entire screen fixes it - but it's still a nuisance. It might not really be curses, or the terminal's, or anyone's fault; asking to draw a character inbetween two others is a pretty odd request and you should expect odd results.

I started to think that I might not be able to use perl's curses after all, but in hindsight it's not so bad. You cannot always control what characters you're going to have to draw in what positions, especially with user input being a factor, but a bug like this can be mitigated:-
  • For a text editor, we're likely going to be drawing each line in one pass anyway, rather than flitting around the screen and repositioning the cursor a lot.
  • Likewise, if I were to make a silly little roguelike game - which I'm considering - I'd have fairly tight control over the cells used to display the game world, and could reasonably expect that I wouldn't accidentally hit this overdrawing problem.
  • In all other cases, such as needing to draw some sort of "dialog box" atop previous content, a quick call to Curses::clearok(1); Curses::refresh(); Curses::clearok(0); should fix things.

Note that I don't want to just set clearok(1) and be done with it. In this mode, curses makes no assumptions about existing screen content and redraws everything from scratch. Even with today's modern computers, if I set that and then hold the arrow keys down I'm going to notice some flicker as the entire screen updates. I don't want that. I certainly don't want it over a potentially slow remote shell!

There was one final problem I ran into but I'm hoping it won't be so important - I noticed that when attempting to use some (allegedly) full-width line drawing characters for the boxes, they were being treated as though they were single-width. So I tried using the Chinese characters you see above, and things worked fine - possibly somewhere along the line some part of the chain got the wrong idea about the widths.

Otherwise, I'm pretty happy with how this turned out and will continue experimenting with perl and curses.






09 June 2012

Curses!

So, to get back into the programming groove I am going to be remaking my clunky old text editor. Because I'm such a Qt fanboy, the logical choice would be to make a quick Qt GUI around QTextDocument and call it a day. It really would be pretty damn easy.

But! there are many instances in which a terminal-based text editor is the superior choice. SSHing between machines, you frequently don't want to bother doing X forwarding and load a heavy UI over the network. And of course, there's those rare instances when X isn't working and you need to fix it...

This means I'm going to have to get to grips with the NCurses library again. Only this time, my goal is to do things "properly" and handle Unicode and so-called "wide" characters. This is where things get Interesting, since NCurses (or at least, the System V curses library it's based on) predates Unicode and the wide character stuff was bolted on later. The documentation makes little reference to the wide version of the various functions, and there's special things to consider like ensuring the locale is set to UTF-8 and #defineing the _XOPEN_SOURCE_EXTENDED macro above all your source files. It's not going to be easy.

Decisions, Decisions...

I have a couple of options open to me to get started on this. One is, I could get my head around the C API for libncursesw5-dev, and write a nice big C++ wrapper around the bits I need so that I never have to worry about it again. It could even use Qt - not the GUI parts, but Qt Core, which has nice things like QString in it. Damn, I love QString. I love a lot of the Qt library, since it actually covers a lot more than just painting pretty widgets on the screen. Making an interface to NCurses that takes QStrings and the like may seem a little bizarre, but if bizarre keeps me entertained, then let's do that.

The other option is my old friend Perl. I'd love to try out some perl6 - I wish it were ready but it's not quite there yet, we'll leave that for another day. But perl5 is solid as a rock, has Curses bindings, and I could also take the opportunity to learn some Moose at the same time. Moose is a nice high-level object system for perl, which I'll need if I'm making more than the single-file swiss-army-sledgehammer perl scripts that I usually come up with.

The only questions are: Will wrestling with wchar_t and ncurses in C++ drive me crazy? Do the perl Curses bindings even support unicode perl strings? Why did my fresh 12.04 install fail to render Chinese in my C++ test program, when the same program works fine on my 11.04 laptop? And why do I need setlocale, and why is wcwidth not in ncurses, and... which language do I pick?

Lower your expectations

The answer to the language conundrum is simply "try both". Let's aim lower. Let's make silly little test programs that barf a few characters on screen, and maybe listen for some input. Programs to print out what keycode the user just entered are always handy for debugging. And by aiming lower, we reduce that awful overhead of planning out the entire project in one lump.

It is no good to have grand plans if you can't even get started due to your planning process spinning away in the background obsessing over details that aren't implemented yet. Just... just go write something. Something bad. Something poorly thought out. Something which you can get done and then learn from the mistakes in the next iteration.

30 May 2012

The Shoemaker Has No Shoes

My name is James. Some call me Neko. Everyone who knows me knows that I am a Lazy Cat.

I am a programmer, so the natural thing for me to blog about is programming. In particular, I'm starting up a little business of my own, with the hope that I could maybe sell some of the software I create and earn a little income from the thing I enjoy doing. But this takes time, and there are many things I need to get in place before people could even see an "alpha" version of my labours.

Firstly, I need a website so that people know of my existence. I've registered lazycat.com.au for myself, but as yet haven't decided on what hosting meets my needs. For now, this blog will suffice.

Next, I want to update the tools of my craft. Since my university days, I've been using a little text editor that I rolled up myself as an experimental foray into the ncurses library. It's been a long time since I touched the code, and frankly the source code has reached critical mass and is unmaintainable. There's a couple of options for me at this point:-
  1. I could switch to using Qt Creator for everything. Qt Creator is a pretty awesome IDE which I've used on several occasions where I have a big, complex code base to wade through. It's quite decent, although I honestly prefer to not be tied down to any particular IDE.
  2. I could take sides in the Vim/Emacs holy war. They're both great, but none quite do things exactly the way I'd like.
  3. I could succumb to the urge that lies deep within every programmer and reinvent the wheel for the nth time, rewriting my clunky old editor from scratch and taking the opportunity to add some shiny new things in the process. This could be a nice way to get my motivation up, just dabbling on some project for the pure fun of it.

Clearly, option 3 is the way to go. I might even blog about some of the inevitable problems I'm going to face, getting to grips with ncurses again and trying to persuade it to permit wide characters this time.