Chuck

Ode to a beach house

2011-03-30T23:33:00.000-07:00

The beach is cool.
The beach is nice.
The waves on my feet
Are like sugar and spice.

*jazz hands*

Change

2011-01-12T22:06:00.001-08:00

Sometimes you see something

And you know it's what you want.

And that's where you'll go.

That's where you'll be.

You know it.

You've decided.

In the same way, this extends

To experiences.

To have done

Or have had the taste of something,

To know what it's like

And that it's different from your norm,

That it's better in some way,

Even if only in its difference,

Can be inspiration enough to change.

And you decide your future,

As simple as that.

It's a beautiful thing.

How to use YUI Slider

2010-06-12T15:40:00.000-07:00

Once upon a time...

At IMVU, we've come to rely heavily on YUI Library. We use it in the website for features such as the avatar card, the message widget, and the music store. It also plays a major supporting role in the user interface of our 3D client application (we embed XULRunner).

IMVU's agile atmosphere is such that the day we discover a new and cool technology can be the same day we ship product which includes that new and cool technology. As a result, we don't always RTFM. We learn by doing. We learn by failing. It's exhilarating, although sometimes the road is a bumpy one. YUI's Slider component was one of the bumps along the way.

It's easy, right?

Our 3D application has a tabbed interface, not unlike all popular web browsers. Internally, we refer to these tabs as modes. In our Settings mode, we needed a slider for adjusting the volume of voice chat. We chose YUI's Slider to satisfy that need:

The code was simple:

<div id="sliderbg">
   <div id="sliderthumb"><img src="thumb.png" /></div>
</div>

YAHOO.widget.Slider.getHorizSlider(
   'sliderbg', 'sliderthumb', 0, 200
);

Unfortunately, we found that the slider's thumb was always 20 pixels out of alignment! It fell too far off the left edge, the right edge, or didn't go far enough. The hacks and programming acrobatics that ensued in attempts to compensate were as far from simplicity as they were from sanity.

It's easy if you pay attention.

Fast forward a week or more later, and I decided to crack open the YUI Slider documentation. It took me less than five minutes of reading to realize exactly how we were doing it wrong. They key passage was this:

"In typical implementations you will need to make the background width equal to the total number of pixels you want the slider to be able to move plus the width of the thumb element."

It was accompanied by the following diagram:

Who knows why the initial maximum value had been set to 200. Everyone who came after assumed it was correct. It's a nice and even number. How could it be wrong? We failed to consider the relevance of the source art's dimensions:

The background was 237x24 and the thumb was 17x24. This of course meant that the maximum value we should have been using was 237 - 17 = 220. Suddenly a certain magic number wasn't so magical anymore.

Art isn't fartsy. It's important!

If you're mindful of the dimensions of your source art and the simple math described above, you'll have no problem using YUI's Slider component.

Subsequent to this discovery, I re-cut our source art and removed the padding around the edges of our chat volume background and thumb art. This wasn't strictly necessary, but it allowed the use of a maximum value of 200. And 200 is such a nice and even number! I'm a software engineer. I have my reasons...

Below are some examples of correct and incorrect maximum values for their associated background and thumb art. I also threw in something a little more interesting for fun. View the source for details.

Enjoy!

http://is.gd/cN5dc

Poisonous devices

2010-02-14T03:46:00.000-08:00

Phones

Sometimes, it's nice when friends and family call. I've even been pleasantly surprised by strangers before. Other times, people call me for the wrong reasons and at the wrong times. I used to believe that when phones ring they must be answered, but that's no longer the case. A phone is an assistive device. If it's not assisting me then what value does it provide?

I used to be terrified of phones. After a part-time job which required a lot of phone time with strangers, that terror went away. Except that in hindsight I don't think it did. It was always stressful. I simply grew to accept that stress.

Over the years I've been tuning my phone usage. I used to answer every call because it was about facing my fear. Then I answered every call because... that's just what you do. It's a phone! Then I realized that it's not my job to answer the phone. This was an important realization for me, because that part-time job had programmed me to believe that it was. I began answering the phone less and less.

These days, I hardly ever answer my phone. One huge source of stress: gone.

This sometimes means I miss calls from friends and family, but ultimately the people I know and love have many ways to reach me. If a chat on the phone is absolutely necessary, it will happen. Eventually. If it's a call from a stranger about random crap I don't care about, guess what: I win. You are unlikely to call me again.

One of my favorite things to do is to turn off my phone. Especially when I go to sleep. Often I'll turn it off when I get home after work as well. And it's fantastic. Especially the part where I'm never jarred awake in the middle of the night.

Clocks

I used to have a watch and I would look at it all the time. If I'm at the bus stop waiting for the bus, I'm checking my watch. If I'm at work and wishing that lunch would come, I'm checking my watch. If I'm wishing work is over so I can go home, I'm checking my watch. If I'm waiting for a friend to come over, if I'm waiting for a show to come on, if I'm trying not to stay up too late...

I decided that looking at clocks is stressful. How about I just change how I'm living so I don't need to look at them? That's what I do now. I engineer my life to alleviate stress and to optimize fun and learning.

For example:

I don't look at bus schedules anymore. Instead, I get enough sleep, wake up refreshed, and walk over to the bus stop when I'm good and ready. This is usually hours before I need to be at work, so I'm never in a rush. I listen to music while I wait, enjoy the cool morning air, and watch the birds while I munch on an apple. Some mornings are foggy and wet and some are bright and warm. I notice and appreciate them because I'm not late for work, I'm not tapping my toe, and I'm not constantly looking at my watch.
There are dozens of ways to alleviate stress at work, and I ruthlessly identify and crush them. As a result, I'm not wishing lunch would come quickly so I can get a moment's relief. I'm not wishing the day would end so I can go home and get a few more moments of relief. I've made my work incredibly satisfying. When lunch rolls around, it's usually a happy surprise. Same thing for the end of the day. So much so that I come into the office on weekends because it's an enjoyable part of my life that I want to experience daily.
It turns out that once I alleviated so much stress in other parts of my life, I also alleviated boredom. For me, boredom was the end result of living in a constant state of stress. This may not make immediate sense, but when I was in a constant state of stress, I was always doing one of two things: stressing or recovering. In that mode of living, I was never actually spending time on myself, either by having fun or by feeding my mind with fulfilling personal learning. Even when I thought I was doing these things, I really wasn't. In that severe state of stress, I believe there was no fun or personal learning. There was a heavy weight which pushed me down and which permeated everything I did. What I thought was fun or learning was instead "escape". Enter boredom: when escape velocity was not sufficient to outmatch stress.

Whenever I think about clocks, I think about the rabbit from Alice in Wonderland. I don't want to be That Guy, running around, worrying his head off, with precious little time to spend on himself. I'm valuable to myself, so I make plenty of time for me.

Alarms

Alarms are like clocks, plus a punch in the face. If I don't like waking up in the middle of the night when I'm not done sleeping, then I don't like waking up in the morning when I'm not done sleeping either. How about I just get enough sleep so that I don't need an alarm?

I have also learned that caffeine is the catalyst for my own personal hell (another poisonous device!) It kicks off a vicious cycle. I drink caffeine. I am amped. I come down off the high. I drink more. It's late and I'm full of energy and my body's burning calories so I eat. I now have even more energy. I stay up late. I have to get into work early so I set an alarm. Too few hours later, my alarm punches me in the face. I drink caffeine to jump start my system. And away we go...

As I mentioned, I turn my phone off when I go to sleep. It's not entirely a coincidence that my phone is also my alarm (iPhone.) I turn it off with purpose. When I'm sleeping, phone calls and alarms are equivalent. Depriving myself of sleep does not improve my life. I control my phone, therefore I choose to improve my life by turning it off when I'm sleeping.

Alarms should be used for alarming circumstances. Like fires. Not burning to death is a good way to add value to my life. Other alarms, not so much.

Right the wrongs

2010-02-11T16:03:00.000-08:00

It's difficult and painful and time-consuming to switch from a habitual lifestyle of bitching over to one of fixing, but it's possible to change. My first impulse when encountering foreign and confusing code has always been to complain about it to someone, immediately. It's such a dumb and mindless thing to do, but I've done it a million times. On some level it's about camaraderie. Wah-wah, it sucks, let me cry on your shoulder. You understand how awful this is, right?

My first attempts at correcting this behavior focused on the wrong thing: demeanor. It's not bitching if I'm calm, collected, and diplomatic when I say it right? It's okay if I try to trick myself into thinking I made an attempt, but was mostly silently seething to myself, and then came to you for help right? What a passive-agressive waste of someone else's time. There's help and then there's hand-holding. Instead, my focus needed to be on solving problems, not the injustice of hard ones and trying to foist them on others.

This video http://www.youtube.com/watch?v=k2h2lvhzMDc (Ed Catmull, Pixar) reminded me of useless bitching (the kind I envision "mediocre teams" would participate in) and that my programmed response in the face of difficulty should be one of reason and action rather than victimization. It's not that someone intended to punch me in the face with their code, it's that the target has shifted and I need to correct course. It's not about personal vendetta, it's about what works and what doesn't work.

This is my reminder to myself to be intelligent and thoughtful and to right wrongs rather than wallow, sidestep, or avoid. I righted wrongs today, and it's awesome.

I am posting to my blog. Stop.

2010-02-02T09:52:00.001-08:00

I am kind of amused by people who end their emails with their name. Your name is in my contact list. I can read the From: field. I know who you are.

Even more amusing is when they end it with a single letter. Is that supposed to be a signature? I don't get it. If you're going to use a signature, make it cool.

For example, there's this guy who "signs" all of his CSS with this:

c(~)

It's a mug of beer with a handle and sloshitude inside, see? Awesome.

I don't like IDEs anymore.

2009-09-19T20:41:00.000-07:00

I spent seven or eight years using Eclipse as my primary development environment. I never really questioned it until I switched jobs and saw some of the mavericks using Emacs, Vim, and command shells. I decided to become fluent in Emacs, and have been using it daily for the past couple years now.

Thinking about it now, it's most interesting to me that I never looked back and said "Gee, life was so much better back in the IDE days." Hell no. In my mind, Eclipse is a sofa and Emacs is a treadmill. I have a goal, I'm on task, I know what I want to do and I do it.

My big fancy IDE mentality was all about victimization, blame, and laziness. "Big fancy IDE crash! No my fault." "Big fancy IDE get confoozed and lie! I do bad thing, oops!!" "Big fancy IDE slow on big project, me play while wait!" "Big fancy IDE big and fancy! It break, I no no why. I confoozed."

My Emacs mentality is scientific, hands-on, and active. I can count on one hand the number of times GNU Emacs has ever crashed on me. Emacs doesn't happily chew on gigs of files in the background, caching data that I could give two shits about. When I want something, I ask for it explicitly. My UX isn't impacted by background tasks that are attempting to read my mind, because there's only ever one task: the one I'm doing right now.

Being born into royalty and pampered for so many years feels nice, but over time it just turned me into a fat lazy slob with a soft mushy brain. It trained me to surrender my critical thinking skills. Emacs and command shells forced me to learn how to tie my own shoes and make my own meals. I enjoy the independence and self-reliance.

I tried Komodo today, at the recommendation of a coworker, and it has a nice polished look, but it reminds me too much of Eclipse. The editor looks as though it is based on Scintilla, which is cool. I am a fan of SciTE. The background churn as it attempts to cache everything under the sun makes the UI sluggish and choppy, which blows my mind. I'm running a quad core with buttloads of memory on an SSD. Come on, man. Seriously?

These are only my first impressions, so I'm going to keep at it for at least a couple weeks to get a real feel for it. Maybe I can right some wrongs, but I get the sense it's never going to feel as light and razor sharp as Emacs.

Celebr-Asian

2009-02-21T02:54:00.001-08:00

I've been fascinated by Eastern culture for as long as I can remember. With every passing year, more of that culture seems to become a part of my life. This post is one part enumeration and one part celebration.

I love fighting anime and have seen every episode of Dragon Ball Z. I have seen all of One Piece (personal all-time favorite,) Bleach, and Naruto, and I stay current.
I taught myself to use chopsticks both left- and right-handed.
I own a Zojirushi rice cooker and eat a plain bowl of rice daily.
I own a Zojirushi water boiler and use it frequently for oatmeal.
Shin Ramyun are my favorite instant noodles and I keep my kitchen stocked.
I spent three months in New York with my Fujian girlfriend (now ex) and her three Asian roommates.
I teach myself Mandarin (slowly) and have a subscription to ChinesePod. I've memorized Liang Shan Bo Yu Zhu Li Ye and have sung it before during karaoke with my ex-girlfriend and a dozen or so of her Asian friends.
I have been a paying member of crunchyroll for over two years.
I have been drinking bubble tea almost daily for the past 3 years. I get most of it from Loving Hut, which is on the same block as my office in downtown Palo Alto, CA. I learned how to make it at home as well.
I do Pincha Mayurasana daily, and can hold it for either two minutes or 10 pushups worth. Lesser known is that my inspiration to do so came from the character Zoro in One Piece.

Eastern culture truly permeates my life, and I appreciate the role it plays on many levels. In tribute, I summon my rhyming and songwriting skills! Sing along with me, won't you?

Sweatdrops on foreheads and tidings from kittens;
Tigers a-crouching while dragons lay hidden;
Red paper envelopes brimming with bling; Asia is full of my favorite things.

Rose-colored petals and climactic doodles?
Green tea and pearl tea and bowls full of noodles;
High kicks that fly from the legs of Bruce Lee;
Asia is full of my favorite things.

Girls in school uniform, pigtails, and glasses
Saving the earth while they ditch all their classes;
Chinese and Japanese insanity;
Asia is full of my favorite things!

When the dubs bite,
When the plot stinks,
When I'm feeling had,
I simply remember that Asia is king
And then I don't feel so bad!

^_^;

Timeboxing results

2009-02-20T02:43:00.000-08:00

This is part eight of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

In my kickoff post on timeboxing yesterday, I challenged myself to crank out 7 blog posts in one "night," with the constraint that I spend 30min on each. Herein are the results of that challenge.

Summary

So, was I able to pull it off?

It got difficult toward the end, but I totally did it. And I am really happy I did, because it was incredibly satisfying.

To summarize quickly, the first 3 or 4 posts were kept within 30min, and they started taking 45min to an hour afterward. I found I had to take 15min breathers to recollect my thoughts and try to stir up additional topics and calls to action.

The hump

There was one pronounced hump around 2am when I had 2 posts left to go, and I was seriously doubting whether or not I wanted to stay up and finish them. Partly because I was tired, but also because I was running out of steam.

I finally decided to do it, and I think having pronounced that I was going to do it helped a lot with this. I had just announced I was going to do, so not doing it felt like wussing out. My goal was pushing it, but it was definitely attainable.

Once I had made up my mind, a lot of stress vanished. I find that rather interesting, in hindsight. Just the power of pulling up my britches and getting serious actually made it feel easier.

Overall quality

I was dreading reading back over my late night 7 post scramble a little bit. However, having just reviewed them all now, I gotta say: I'm pretty damn impressed. Everything was 90% of the way there, except perhaps my last post, which felt good, but aborted.

I added some content and changed a bit in a few of the posts, but by and large I was making typo corrections and adding in useful headers to draw attention and break things up better. Sometimes I would use certain words repeatedly, so I'd cut them and balance it out. Sometimes I'd draw attention to something above when I meant to do so for something below. Really trivial stuff.

Unmitigated power!

Here's what I liked the most about this timeboxed posting spree: it really focused me and made me feel like I was exercising a lot of untapped power.

What if I did this every night, in the same way I rhyme, draw, or work on my posture? It's a compelling thought, because I can see developing a high level of skill in writing in a short amount of time. It really is a feeling of power!

And it also makes me think. What other skills could I have if I really wanted them? And what doors would they open?

Try it!

I'm going to start applying timeboxing more in my life, due to my findings in this little experiment. I would urge you to do the same, even if for a short period, if only to feel what this is like. Sometimes all we need is to experience what things could be like in order to ignite change.

Timebox a series of tasks, with a sincere intent and a dutiful hand. You won't be disappointed!

My quality of life, part 6

2009-02-19T03:33:00.000-08:00

This is part seven of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

In this post I'm going to share a bit of imagery that helped motivate me to continue improving my health, and share some thoughts on the stretchiness of stomachs!

Neck rings & lip plates

Orange skin was a big eye-opener for me, but that was nothing compared to what you see pictured below.

I believe I was watching an episode of National Geographic at the time. In fact, here's a National Geographic video on neck rings. I was fascinated and perhaps even a little disturbed by the elongated necks of these women.

At the same time, this was an amazing testament to the flexibility and durability of the human body. I had no idea something like this was even possible. And if these women could stretch their necks to such an extreme extent with rings as a tool, I could do something as mundane as shed a few (dozen) pounds with food as my tool.

Shrink the fleshy balloon!

In a lot of the "get healthy" literature I would read, there was often mention of how to "shrink your stomach." I don't actually know if that's the proper terminology to use, or if it's even possible, but the idea seemed reasonable.

Images such as the one below for some reason have always made me think of the stretchability of the stomach.

While removing the lip plate and leaving it out might not ever shrink those lips back to their original size, I could easily imagine it contracting at least a bit without the plate to keep it taut. It was easy for me to visualize the lip as a stomach and the plate as the food filling it.

As far as I can tell, the stomach is as open to "suggestion" as the body parts in both of the above examples. Perhaps even moreso. It's a bit like a fleshy balloon. The mass of the balloon may be fixed, but you can force an awful lot into it -- far more than you might assume if you had never before seen a balloon inflated.

The flip side of this is that the longer your stomach is filled with a certain amount, the more relaxed it becomes at that size. Some degree of resistance is lost. The stomach may shrink again when emptied, but whereas a balloon will quickly deflate, your stomach is not so trivially evacuated.

So, this was some of my thinking. Exposure to these sorts of images have inspired thought in me over the years, and helped me to reason about health.

How to shop for awesome and compatible PC parts the easy way

2009-02-19T02:04:00.000-08:00

This is part six of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

I'm no hardware guru, but I have assembled a dozen or so PCs. I recently upgraded most of the guts of my computer and spent a little time developing a simple and straightforward way to find compatible PC parts without having to know many technical details.

1. Use Newegg

First, do all of your shopping through Newegg. If you're a real penny pincher, you might be able to find better deals by scouring the net, but it's just not worth the extra effort for whatever marginally better price you may find.

Newegg is on the level, and will always be within the same ballpark. Newegg is reputable, reliable, and other words that start with the letter R. Newegg rocks. And I am a big nerd, so you should believe what I say!

2. Leverage knowledge of what most other customers bought

If you're dumb as a bag of hammers when it comes to PC hardware, a really good way to hone in on compatible parts is to find a popular and highly rated motherboard that you like and let all remaining decisions be driven by the "Customers Also Bought" feature.

3. Lean on the nerds.

In computer programming, the phrase "Lean on the compiler." is used to help remind us that we can leverage our compilers for many types of hide-and-seek grunt work and save ourselves a lot of time.

In searching for compatible PC parts, you need to "Lean on the nerds." Most of the people spending money on Newegg know what they're doing. If people who buy one highly rated motherboard also buy a certain other highly rated CPU, there is an extremely high chance those parts are compatible.

4. [Freedom to] Ignore the bad comments

If you shop this way and only consider 4 or 5 star products, you are able to (and should) ignore the bad comments. The people crying foul are in the minority for one, so it is unlikely you will experience their difficulties. Additionally, my observation from reading many of these comments is that most of these people are simply buying incompatible parts. Slightly incompatible or huge incompatible, but incompatible.

5. Find "buying buddies" in the good comments

You will find themes in the good comments, and those are what you want to feel out. This is like looking for apartment roommates. You may have some requirements, or maybe you'll know it when you see it, but you'll want to get a feel for what you're getting yourself into before you commit. And the comments that are in the majority will tell you that.

So, be smart. And by smart, I mean lazy. Lean on the nerds!

One way to build better posture

2009-02-19T01:00:00.001-08:00

This is part five of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

I'm a computer programmer and I spend most of my time sitting down. And when you spend most of your time sitting down, you want to spend that time comfortably. For me, this meant a lot of reclining, slouching, and hunching.

Unfortunately, this seemed to plague me with back and neck pains. A bit of reading led me to believe that better posture might help this situation, so looked into ways to improve mine. Since then I've settled on a few methods, the easiest of which I'll share here.

As with my posts on developing rhyming and artistic skill, the key is to simply practice. A lot. And I think the best way to get that sort of practice is to integrate the skill you want to acquire into your daily routine. For rhyming, I constantly goofed off online in an IRC channel. For lettering, while reading technical manuals I transcribed a lot of the content.

For developing better posture, I do this: I sit up straight on the bus.

Mundane, sure, but do it on your daily commute to work for a month and you'll be surprised how strong the muscles in your back become. If you drive to work, do it there. If you are constantly on a plane, do it there. If you don't do any of these things, you're screwed.

I kid.

Really, the idea is simple. Find a relatively short interval of time which you would otherwise spend on auto-pilot, zoning out. You can probably figure out a way to make that time worth more to you, such as going toward improving your posture. Sitting up straight on the bus was one of mine.

The activity doesn't have to directly focus on the skill -- it can support it. Think of Mr. Miyagi from The Karate Kid. "Wax on. Wax off." didn't seem like karate to Daniel, and it wasn't. But it turned out to be a valuable activity that supported his pursuit of karate.

I still slouch and hunch, but I do it far less often than I used to. As a result, I find I very rarely develop neck and back pains any more.

Don't believe me? Try it yourself!

Getting the videos you want quickly with DC++

2009-02-19T00:04:00.001-08:00

This is part four of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

YouTube

Free, Streaming, ugly, mixed bag

These days almost everybody knows what YouTube is. You can stream videos on demand. Maybe not the best videos in the world, but sometimes they've got television shows and snippets of movies. The average quality is pretty low, but hey: it's fast.

Paid services

Paid, streaming, high quality, comprehensive

For higher quality streaming media, people tend to turn to paid services, such as Amazon or iTunes. If you don't have a lot of money to spend, you can sometimes find services with higher average quality than YouTube, but that's fairly rare and the selection of material is usually very limited.

Torrents

Free, fast (if it's current,) high quality, comprehensive

If you want something of good quality, for free, and quickly, you're probably going to turn to torrents.

You can't stream videos you're downloading with torrents, but on the flip side if you've got a really fast connection, you can usually max it out and get the entire video in such a short amount of time that it's almost as instantly satisfying as streaming the content.

The thing about torrents maxing out your connection, however is that this is usually only true for very current media, such as movies or television episodes that are relatively brand-spanking new. Without going into details, the older something becomes, the less likely you are to be able to get it quickly, or perhaps even at all. If you stay current and "ride the wave," this is a great solution.

eMule

Free, wildly varying speeds and quality, comprehensive

If you don't ride the wave, or come late to the game, you may turn to peer-to-peer clients such as eMule. This sort of service can work well, but has the reputation of still being fairly slow for older media.

DirectConnect

Free, fast, high quality, comprehensive

If you are a pack rat like I am, and download and save tons of videos for your "digital library," then one really good alternative is the DirectConnect protocol. My client of choice is DC++.

How it works

How DirectConnect works is you join what are basically "chat rooms," where you and other people are sharing all of the videos (and other files) in your personal stashes. The chat rooms will have basic requirements such as "You must allow at least 2 or more people to download from you at once" or "You must be sharing at least 5 gigabytes of files to join the chat room." or even "You must be sharing at least 5 gigabytes of science fiction to join the chat room." Once you have met the requirements and join a chat room, you can search all of the files shared by the people in that chat room.

Unlock more content by having more content

The requirements part is very interesting, because once you start to find chat rooms with large base requirements such as 150 gigabytes of files shared, you will notice that, if you are able to meet that requirement, you suddenly have an amazing selection of videos at your fingertips.

How quality is maintained

Because these are actually chat rooms, they are not so much the "wild west" where quality and selection can vary extremely, but are much smaller and more tightly knit, with "officers" who inspect the files people share and boot them out of the chat room if the files they are sharing are just blatantly in violation of quality standards.

Other benefits

One other great thing about DC++ is that a lot of people from Europe, such as Sweden where extremely fast connections (like fiber!) flow like wine, hang out in these sorts of channels. I don't know why, but that's the way it is. And that means that the people in the chat room that you are downloading from often have very fast connections. Which means you get your files faster!

One downside DC++ used to have was that you could only download a certain file from one person at a time, so it was very important the other person have a fast connection. But having started using DC++ again recently, I have discovered that this is no longer the case, and you can download from multiple people at once, just like in eMule or as with torrents. Which now means that if you find a good chat room with large base share requirements and a lot of Swedes in it... you just hit the jackpot.

Worth it? Yes!

It took me a while to figure out how to use DC++ well, but it is very much worth the effort. My experience is that it is a good middle ground between the sheer speed of torrents for new media versus the slow speed of eMule for old things. DC++ is populated with pack rats like me, many of them aficionados in specific genres such as anime or scifi, judging by what they share.

In the near future I'll share some specific tips on how to use DC++ well, but in the meanwhile, if you're also a pack rat and have a decent connection and a lot of hard drive space, you should check it out!

How to develop artistic skill in lettering

2009-02-18T23:34:00.000-08:00

This is part three of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

I've always been artistic and interested in drawing, and probably would have been an artist if I'd never found computers and programming, but even though I never followed that particular path, I still draw almost daily.

Developing the skill is easy. As I mentioned in my post on how to rhyme, all it takes is just a hell of a lot of practice.

Here's an example from my Flickr account:

As cool as this is (to some people,) you might be surprised to know that I don't actually consider this very talented or even skillful work. Anyone can do this sort of thing if you practice it as much as I do.

Of course, you have to find time to do these things. And that can be no small task when you're a busy person. We only have so many hours in a day, after all!

I did take art lessons back in grade school and high school, but once I was out and got hooked on programming, I stopped really nurturing it. And the sort of art pictured above isn't anything like the sort of art I practiced in school. The art above is all straight lines and relatively few curves. In school I practiced people and realism, with lots of shading and far more curves and detail than the angular and rapid strokes above.

It's interesting to me how I actually came into this sort of lettering. It was a pretty incremental and natural process that... just kind of happened.

When I got heavy into programming, I began reading a lot of technical texts. "Teach yourself programming in 21 days!" and all that jazz. Perhaps because I was musical and into rhyming and therefore songwriting and poetry, I was used to writing a lot. Lyrics, etc. When I began teaching myself to program and reading all of these books, I began to transcribe a lot of the text and example programs I read. Much like the monks back in super old-timey times (yes, that is a technical term) would transcribe bibles I suppose. Sometimes verbatim, sometimes brainstorming alterations I might soon attempt.

I'll put pictures up at some point possibly, but I literally wrote all day long. Tomes and tomes of college-rule paper. It's like I was trying to forcibly imprint the contents of these books into my brain. All of the insightful passages and clever bits of code slowly made the trek from paper, into my eyes, my brain, and then back out of my pen again.

As a side effect of this, I developed very legible handwriting. Teachers would often comment on it in notes when grading handwritten papers I would submit for school assignments. Friends would comment on it when coming over to hang out. I didn't realize until much later that this was excellent training for developing the steady and decisive strokes needed for drawing.

You'll notice the picture above has a lot of hatchwork. It's very similar to a lot of lowercase Ls, very close together. The letters are just enlarged letters in my own writing style, with a very limited sort of "this vs. thick" characteristic. The only real artistic license in any of it is the flair at the base or extremities of the letters. And those are mostly just easy quick flicks of the wrist that are then capped off with straight lines. And I'll add fling fan out a few of them here and there. Then, I trace it. And nothing is easier than tracing!

I still write on paper a lot to this day, and I still mindlessly do art as pictured above almost daily, often in 10min or less. During lulls in meetings or after meals, while brainstorming, whenever. But the cool thing is, I found a way to take a mundane skill like writing, and turn it into something fun and at least a little pleasing to the eye.

How to develop rhyming skill

2009-02-18T23:02:00.001-08:00

This is part two of a timeboxed series. [ 1 2 3 4 5 6 7 8 ]

I hang out in IRC a lot. I have for over 13 years or so. I'm not exactly sure why, but I assume a lot of it is simply the camaraderie. A lot of like-minded individuals with common interests hang out and goof off. And sometimes rhyme!

There is a specific channel on a specific server that I consider to be my third place. I think part of the reason I keep coming back is because the feeling I have when I'm there is one of unrestrained creativity.

I am somewhat musical, so when I'm not making "your mom" jokes (a favorite pastime of ours, who knows why,) I switch between actually participating "fruitfully" in conversation -- answering technical questions, being "serious" -- and treating every word anyone utters as a sort of challenge. A challenge to warp what they've said in a silly way, to cleverly confuse its meaning, or -- and this is my favorite challenge -- to rhyme! The channel is my palette, and their words are my paint.

A lot of the time, this sort of challenge comes explicitly, because I have a reputation of rhyming things out of the blue and for no good reason. I go off on a tangent and spam the channel with random rhymes that are streaming mostly unfiltered from my brain.

Lately, our channel is fascinated with LimerickDB. Here's one that I find particularly hilarious, but it is dirty, so please avert your eyes if you are sensitive! (I apologize to certain friends and family.)

#17
There was a young sailor from Brighton,
Who said to his girl, "You're a tight one."
She replied, "Bless my soul,
You're in the wrong hole;
There's plenty of room in the right one."

Love it. And there are plenty more where that came from. As soon as this was pasted into the channel, the gauntlet was thrown down. After thinking for a moment, and considering some recent drama directed at my workplace, I decided on a topic and about 20min later out popped the following:

There is a famed blogger named Atwood,
But his posts aren't really all /that/ good.
His tweets are inane
And his podcasts are lame.
He does make me LOL, but a cat could.

I was pretty pleased with myself, since rhyming with "Atwood" was a little difficult. I don't really loathe Atwood all that much, since I've historically openly admired his blog, but nonetheless it's fun to simply to "do battle on the field" when he has clearly belittled "my team."

So, how do I do come up with such awesome rhymes in so little time? Ha-ha-ha, well Johnny. I'm glad you asked. It's simple really. I practice all the damn time.

Seriously, I do this almost daily. And boy do I have more misses than hits. Every once in a while, I come up with some great stuff. Most of it low-brow and lewd, sure. But hey, I'm not arguing with results.

The great thing about the IRC channel I frequent is that it's sort of like a museum of oddities. A couple dozen nerds funneling all of their bizarre findings into one place. There is endless material with which to rhyme, so it's a really good place to practice the art. Of course, it helps a lot that I'm among friends who can stand me. But another great thing about IRC is you can be random on a daily basis without annoying people too much. They pay attention when they feel like it, much like twitter.

So, I practice a lot. But it also helps that I sing a lot. In my first post during the "blood pact" that I am participating in, I mentioned that I also sing frequently to myself in gibberish. This is another incredibly useful tool in learning how to rhyme because you make all of the sounds of the rainbow! Whatever sounds cool. Or maybe it sounds really bad. But you're exercising those sounds all the time. And when you sing in gibberish, you begin to develop the ability to view words less as a whole and more as their component parts. The vowels, the consonants -- those are just all of your gibberish, constructed in a meaningful way.

Another thing gibberish and singing can help you with is learning how to "stretch" vowels in whatever way you see fit. To shoehorn words that might not otherwise rhyme if spoken normally. Almost every singer under the sun does this. It's sort of like Neo in the matrix. He could bend the rules to pull off some amazing stunts!

Timeboxing

2009-02-18T22:30:00.000-08:00

This is part one of a timeboxed series. [ 1 2 3 4 5 6 7 8]

Some coworkers of mine and some friends are writing blogs daily for a month as part of a so-called blood pact. I'm about 7 posts behind today, so I decided to conduct an experiment. I'm going to write 7 posts tonight, and timebox each one by 30 minutes. Can I do it? I think so. I dare me to prove me wrong! So, I'm going to put it to the test. And describe my thinking and see if it holds up around 1 or 1:30am.

What better way to inaugurate the session than with a post on timeboxing!

So what is timeboxing? And it it useful? I'll elaborate. Also, I claim: yes.

At IMVU, the awesome place where I work, we are fairly agile and practice scrum. The way I understand it, this is still a relatively new methodology, so there isn't One True Way to perform scrum. But one of the keystones is rapid iteration.

A good way to rapidly iterate is to kick off any given feature with an exploratory "spike." What is a spike?

Well, from the perspective of a "non-agile" developer, this is probably what you would consider the normal way to do anything. You don't write things like automated tests, up front or otherwise. You spend a little time fleshing out an idea, have some meetings, make your estimates, and then start hacking away.

I realize there are of course companies that spend a lot of time up front doing planning down to excruciating details and to very exacting standards, using the so-called waterfall model. But for many of us who do relatively straight-forward web development for small companies -- which is what I did for 7 years prior to my employment at IMVU -- I think it's pretty safe to say that 90 to 100% of the design done up front and in meetings. Perhaps even one big meeting.

At any rate, the idea with a spike is that you spend minimal time with up front planning, and simply start executing on what seems to be the most straight-forward solution. The idea being, you will quickly get a feel for what most of the roadblocks will be simply by trying to do what you want to do. This is also often described as "writing one to throw away." Lessons learned, and all that. And even if it doesn't turn out to be "all the way there," chances are good it'll be 80% of the way there or more.

The way things usually seem to work is that you spike something and then, seeing as you're already 80% of the way there and how you did put a decent amount of effort into it to get it to that point afterall, you keep it. I'm curious to see if that will hold true for these blog posts, because usually I spend at least 4 hours on each one, perhaps half or more of it spent slamming out content, and then the last half revising and cutting a ton. Condensing it down, polishing it.

I've been resistant to this, historically, since I like to wrap my brain around something fully before proceeding, but lately I've been timeboxing tasks at work and, all things being equal, I like how it feels to timebox something. Maybe you finish it, maybe not. But you decide on a reasonable amount of effort and move on. You don't spend as much time down rabbit holes because you are mindful of the relatively short time period you've set for yourself to achieve your goal. If you sincerely attempt it, you begin to think much more decisively, because you need to.

I remember reading some DragonLance books once, wherein Raistlin, a mage, was learning magic from a sort of warmaster mage in the field. Raistlin was all about the laborious incantations and exacting spell components. He was amazed how the warmaster could, with very limited components and by not even completing spells, still "magically" (ha-ha) execute partial offensive spells. Sure, they weren't complete, they weren't elegant, but they still packed a wallop!

My quality of life, part 5

2009-02-17T22:28:00.000-08:00

My previous post talked about how I went about teaching myself patience, which I felt was a missing ingredient in my quest for improved health. Today I'm going to wrap things up with a few final words on revisiting an old habit (one of my worst,) deconstructing it, and how I went about replacing it with a new one.

Culture of completion

As with many people I know, I was raised to eat all of the food on my plate. "There are starving children in some 3rd world country!" These were common and well-intentioned words, but for me became rote. I obeyed. And like many things long-practiced, they pass out of consciousness and into habit. Unfortunately, habits are seldom devoted rational thought. Much like learning to speak, to play a musical instrument, or to drive a car, you memorize most of the rules so that, to a large degree, you can forget them. Soon the rules are "in your bones" and you act on them without effort. For me, this meant that I dutifully carried out the habit of finishing everything on my plate regardless of the amount I took or was given. The habit knew nothing of volume. It ate what existed. My eyes told me when I was finished. And therein lies the disconnect. Some people have performed interesting studies on eating habits like this, so I won't bother with belaboring too many details. I will say that it became a point of pride to finish my plate, partly because our "familial culture" was happy and status quo when I did so, and disdainful if waste was produced. I'm not casting blame, but simply recognizing that little thought entered into it. Cooking for many begat big meals begat many servings begat exhortation to finish begat finishing. Much like prolonged exposure to very high volume music can cause deafness (desensitization to sound,) prolonged force-feeding can cause obesity (desensitization to nourishment.) Through my habit, I had lost my "ear for eating." It no longer mattered even if I tried to listen. What I heard was unclear. If my ear for eating had grown deaf, I needed to rely on other tools to get me to the place I wanted to be.

Hearing aids?

Being mindful of how the plate cleaning rule had affected me was an important step. What am I doing? Why am I doing it? Having answered these questions, it caused me to begin to pause and think before meals. I began to pay attention. Before, I had been mechanically shoveling food to complete a task, and my plate was one unit of work. Now I was thinking, and always questioning whether the work really needed to be done. Making the analogy with sound helped me to treat these pauses before meals sort of like a dial for volume. What if I just served myself a little bit less than normal? I don't have to crank it full blast to enjoy it, and the next meal is always right around the corner. Put another way, you will eat again, in your life, you know. By applying this same tactic over and over (discipline and patience ever-present,) I reduced my intake little by little. I ritualized the visualization of that volume dial. My prior habit had been finishing the entire plate so I decided to make it my new habit to "cleverly" serve myself just a little bit less, or eat everything on my plate except the last little bit. I feel like part of the reason this worked for me is that I shifted the focus from that of a show of strength to a show of skill. It became a matter of whether or not I had the chops to sustain such a simple behavior.

Was I able to stop myself from eating that last little bit?
Could I brave the ridicule by wasting some of each meal?
Would I cave to peer and familial pressure?

It was my own personal inner game.

Kicking it to the curb

Changing old habits is hard. But if you take the time to reason about and understand how they were formed, you can turn big, impenetrable, and forbidding habits into what they really often are: reasonable outcomes due to specific circumstances or experiences. Once you know what the circumstances and experiences are that have contributed to your habits, you can set out to change them and create new ones, respectively. You can develop strategies against and do battle with each of them individually and eat the elephant one bite at a time. ...but don't eat him all in one sitting.

My quality of life, part 4

2009-02-14T01:50:00.000-08:00

Yesterday, I said discipline improved my life the most. Sickness gave me motivation to change, but sustaining that motivation was difficult because I was missing what turns out to be an important tool.

Patience

I was led to the following conclusion: I needed patience. Prior to my hospitalization, I would have been dismissive of this. But my serious lack of health put me in a serious frame of mind, and so I decided to treat the idea seriously and to explore ways to obtain it. How did I use patience in my daily life otherwise? Maybe I was already exercising patience in ways I didn't realize, and that would help me make some comparisons and better strategize.

My daily routine

I would get up and eat, usually in a rush because I had stayed up into the wee hours and was running late for work. Maybe I wouldn't eat at all if I was late enough. I would wait on the bus, I supposed, for 30 minutes. That's a sort of patience, right? Well, I had little choice. Distance has inescapable qualities, it turns out. Okay, I'm at work. I'm programming. There's deadlines. I'm rushing to meet them. Lunch can't arrive quickly enough. Waiting for lunch is patience, right? Lunch comes and goes and I'm working again, rushing like before. I need to be done yesterday. I have a bad track record with deadlines, so I usually work late. I take the bus home eventually, half-relieved and half-exhausted. Arriving home, I do what I want. I eat what I want, as much as I want. I watch whatever I want. Anything I want to do, I pretty much do it. Activities such as trips to the grocery store might require patience, I guess, but I classified them more as annoyances. I stay up late, because I want as much of all that as I can get. Which of course feeds into the next day, with the lateness and the rushing all over again. That was my standard day. Ultimately, when I mulled it over, most of what I did, I did in a hurry. And when left to my own devices, most of what I did was as a direct result of whim or a desire for immediate satisfaction. "I'm hungry!" "I'm bored." And the ever-popular "I'm bored. Time to eat!"

Less frequent happenings

So, little patience to be found in my day-to-day. What sort of things did I wait for that took more than a day? Paychecks came every two weeks, though again, not something I had much control over. I could stockpile cash for cool gadgets, but honestly I hardly ever did so. If I wanted something, I bought it ASAP. I used credit if necessary. Holidays rolled around every so often, though usually they came as a surprise, since I was always neck-deep in work. I remember coming to work on numerous occasions, not having realized we had the day off. The list goes on but, no, I realized patience wasn't part of my life.

Orange skin

While the idea of patience scuttled around in the back of my mind, I was otherwise mostly occupying my time by actively reading many "get healthy" books. You come across a lot of bizarre and fascinating stories when reading literature about ailments and how to heal them, but one example that stood out for me was to do with consuming too many carrots. In high enough quantities and in long enough duration, your skin can actually turn orange. It's not dangerous, it's just funny. However, it was also something of an epiphany for me. This made much more "real" to me the fact that our bodies are constantly growing and rebuilding themselves out of what we eat and drink. And that those choices can have very material effects. Further, that it takes time -- orange skin doesn't happen if you decide to "O.D. on carrots" one day. It takes frequent and prolonged exposure. And so, it seemed obvious to me at that point that any changes I wanted to occur in my own body would take equally diligent application. And certainly it would take much more time than the daily or even bi-monthly cycles of "patience" that I was accustomed to.

Cake & steaks

This gave me a sort of ruler to help gauge what sorts of foods I should eat, and how frequently I should eat them. What would happen if I ate nothing but chocolate cake for an entire month? How would I feel? What if I ate nothing but steaks? What would it feel like to be built out of, say, apples? I had my own idea about how different things would affect me, but I actually started putting it to the test. I wanted to find some staple foods to base most of my diet on, so my criteria became this: I should be able to eat something, almost exclusively, for weeks at a time and feel well. If any food caused me significant physical discomfort after prolonged exposure, I would stop eating it. FYI, cake and steaks lost (after about three days and frequent chest-clutching.) As some of my friends and coworkers know, apples won a month-long round, and continue to be a staple of mine.

Run with it.

I'm not going to dispute I was a bit crazy due to my willingness to explore extremes in this way (remember those steep slopes I mentioned?), but there was and is a method to my madness and it has provided me with numerous valuable insights about my body's needs and limitations. The interesting thing to me, looking back over the past 6 years, is that the basis for most of what has become my "health rationale" came from the idea of "orange skin." It was just this curious edge-case scenario I read about once, yet it was novel enough that I kept revisiting it, and it became a focal point through which I developed many personally useful ideas and tools. If I were to offer advice on maintaining motivation, it would be to find something personally interesting and milk it for all it's worth. Perhaps obvious, but super effective. It's kind of amazing what sort of mileage you can get out of even simple ideas. Maybe one day you'll be able to blog about how orange skin taught you patience.

My quality of life, part 3

2009-02-13T01:47:00.000-08:00

In my previous posts, I talked a little bit about my personality and some of my history. I had a brush with death and that changed me, I kept trying trying to improve my health in every way possible and never gave up, I've got health and energy now, etc. Great. Here's what has improved my life the most:

Discipline

It took me a long time to realize that discipline is not about taking the hard line, brutally enforcing strict rules which must be obeyed to the letter. The word "discipline" conjured visions of stoic figures in military uniform and business suits, disapproving school teachers with rulers, and general discomfort due to deprivation. It wasn't a positive feeling. However, I was conflating discipline itself with sensationalized depictions of circumstances surrounding discipline. For what it's worth, revisiting these pre-programmed feelings and breaking them down logically was instrumental in getting me on the path to self-improvement.

Sustainable effort

The truth is, you choose how to learn what you learn, and you can choose a steep slope or a gentle one. I was never able to sustain the effort required to scale steep slopes. And so, I chose gentle slopes. Of course, even gentle slopes had two primary downsides for me:

Change took substantially longer to notice and I would begin to question whether worthwhile change was occurring at all.
Being a man filled with testosterone and machismo, there was far less to brag about. And bragging rights were like rocket fuel for continued effort. Without that, forget it. Or so I felt at the time.

Machismo was the easiest to leave behind, once I made the decision that intelligence was a superior problem-solving tool. Gradual, nearly imperceptible change was the real challenge.

Finding motivation

A small, incremental and theoretically "good" change to your habits is hard to gauge the effects of from day to day. So much so that it feels like you are being deprived of essential feedback and therefore robbed of motivation to continue nurturing it. In my next post, I'll talk about how my "health rationale" has evolved, and how it has helped to keep me motivated and consistent.

My quality of life, part 2

2009-02-08T12:21:00.000-08:00

Part of the reason I've had so many inquiries about my energy and health in recent years can be explained by clicking here.

In the scraggly version, I led a very sedentary lifestyle and, while cheery, I would not have been described as energetic. In the cheese-master photo, I was jogging regularly and had completely transformed my diet.

So, why the change? Or perhaps more usefully: how?

A brief history

I have been hospitalized three times with a diagnosis of ITP, and eventually chronic ITP. The causes for and cure to ITP are unknown, but to sum it up, my body began attacking and destroying my platelets and I would bleed and bruise very easily.

I am 5'10", and prior to my first hospitalization I had reached 240lbs. I spent most of my waking hours indoors in front of a computer, and I ate very little raw fruits and vegetables. Attempts to eat salad resulted in gagging and vomiting. I drank mostly soda and juice, and very little water. What water I did drink tasted extremely bitter.

As I learned shortly after my first hospitalization, the thing about ITP is: they don't know what causes it or how to cure it. I could also provide a lot of gruesome details about my first hospital stay, but instead I'll simply say that the there was a lot of pain, a brain hemorrhage, and a week-long catheter.

The important part is: big wake-up call.

Health tactics and strategy

My hospital stay shook me. I knew I wasn't living healthily, and so I began to read and listen.

I read about ITP and vegetables and exercise. I read about vegans, vegetarians, and fruititarians. I tried all three. I ate nothing but apples for an entire month and reached (a pallid) 170lbs. I fasted for nearly three days. I did push-ups. I jogged. I lifted weights. I spent a day in a state that I can only describe as euphoric, and experienced some of the most genuine unprovoked happiness I've ever known.

I attacked the problem from all angles and experimented on myself. I didn't try one thing and give up. I didn't try many things and give up. I felt strongly that if I could only develop the right habits, do all the things everybody knows you /should/ do, that my body would correct itself. And ultimately? It did.

The two hospital stays subsequent to my first were disheartening, but I also took note that the first was six months after, and the second was a year after that. I was getting stronger. All the while, I was experiencing significant changes in how I felt and the level of energy I had. That "re-heartened" me.

A simple analysis

I thought it would be interesting to visualize a handful of salient points related to my health over the years, so I spent some time thinking and graphing. See below for what I came up with.

This is only a rough depiction of what I remember, but it feels like a good approximation.

The three dips in my health reflect my hospital stays. You can see that after my first stay, I began to start incorporating raw (fruits and) vegetables into my diet. An interesting note is that not long after my second hospitalization I not only liked salad, I craved it.

My weight was the worst leading up to to my first hospital stay, and the lowest point was during my apple extravaganza. I've normalized at around 190lbs for the past couple years.

I leave you to draw your own conclusions, but apart from the novelty, it reaffirms for me what I think everyone already knows:

Good health isn't rocket-science.

My quality of life, part 1

2009-02-07T04:00:00.000-08:00

If you know me, you know I'm a happy guy. I'm quick to laugh, eager to goof off, and enthusiastic about life in general. That's me. I'm also fit and very healthy. And incredibly sexy. And I have a deep voice. /And/ everyone who knows me thinks I am awesome. Or at least, some of those things might be true. The first one for sure. Anyway. It wasn't always so. For various reasons, family and friends have been fishing for details about my energy and health with increasing frequency over the past few years. To help satisfy that curiosity, I'm going to explore some of those details here. First, I'll explain a few things about my personality. Long story short, I attribute most of "me" to my parents. They serve an important role as my safety net, and they have never not been there for me. They also have an abundance of talent, creativity, and love and a lot of it rubbed off. In RPG terms, I rolled high base stats with lots of bonuses.

With that safety net, it's easy to become care-free. It's like savegames. I'm not worried about screwing up because I can always reload. That's how it feels. And that feeling counts for a lot. I've always felt empowered to explore, to try, and to fail with impunity. To explore, to try, and to fail is to learn, and so: my parents have played a large role in my own sense of freedom /to/ learn.

What has that sense of freedom gotten me? Well...

My parents are musical. It is no surprise then that I have an affinity for playing instruments, singing, and songwriting. Freedom to learn is what kept me practicing the guitar for weeks, even when my cuticles were bloody from strumming with poor aim. Rather than an obstacle, they became a point of pride and a milestone.

I'm fascinated by language and manner of speech, and I think the sense of freedom to learn that my parents fostered in me is part of why I reflexively mimic interesting voices when I hear them, and why I continue to teach myself Mandarin, which is tonal.

I often talk to myself in complete gibberish with various accents. Even if it's ridiculous, it exercises a muscle I don't want to atrophy. I also sing frequently in this way. My freedom to learn has taught me that even nonsense, if genuinely expressed, can develop skill. To me, song is more parts emotion evoked from beautiful sounds than it is lyrics. Because of this, I tend to listen to music with female vocalists more than anything else, because they tend to have voices better capable of evoking that emotion.

I've noticed I sometimes unconsciously begin to mimic the voices of new people I meet, especially if I like them and especially if they have an accent. Two particular instances of this come to mind.

One was years back when I was a teen. Friend of a friend. Scrappy guy. Bit of a mechanic. Super strong. He had an interesting high pitched voice and an underbite. I didn't realize I was mimicking his voice until I got home and began talking with my parents. They didn't seem too thrilled.

The other was much more recently, two years ago. A coworker from Ireland, with whom I still work. He's the friendliest and most thoughtful guy. My voice began to develop the same calm and relaxed pitch. I had to suppress the urge to speak with the same accent when in his presence. I was afraid he'd think I was mocking him. Usually, you practice to learn, but I had to practice to /stop/ learning. In that less appropriate context, anyway.

So, some of what contributes to my energy is the constant positive feedback from chronic learning.

If you catch me grinning goofily to myself, I could be telling myself a joke or fondly recalling a blog I read once with boobies in it. But chances aren't bad that I'm singing gibberish in an Irish accent either.

Make a call to action.

2008-12-10T14:19:00.000-08:00

I just learned something about my coworkers that I didn't realize. Email which doesn't make a call to action is ignored.

I sent out a message earlier today explaining that hitting the database in most of our website tests was very costly, timewise. I provided a small example with some numbers. My final remark was basically "To be continued..."

Not long after, I went out to get coffee with a few of the guys. The constructive criticism was: "So what? I know they're costly. You know they're costly. As an engineer, I don't really care that you hate that they're costly. I want to know what you or I can do about it. There is no call to action."

I stated that my intention explicitly wasn't to make a call to action -- that would come later. But then it was explained that our company mailing lists have so much noise, any messages which don't have a call to action are effectively ignored, if for no other reason than to protect oneself from wasting time reading fluff.

It does make sense. I can't read all of our mail. If it's not immediately clear whether a message is important or relates to me, I move on.

If you want to affect change in your company, don't broadcast what amounts to a rant. I realize that's all my message was at this stage. People care about pain points, but they have their own focus and problems to solve. Their time, and yours, is a valuable ration of limited supply. We all need as much of it as possible in order to feel productive. Don't waste it with bitching.

So you want to improve the situation? Do the legwork and find a course of action. Share that. Most of the time, everyone already knows what is wrong. What they'd really like to know is how to make it better. Even more than than that, they like simple actionable items.

Lesson learned!

Better maintainability with an IFRAME?

2008-10-23T19:42:00.001-07:00

We have this huge admin page at work that manages various things about our customers. It's like 5,500 lines long. At some point, we decided never to add any additional functionality to the page without first improving it. Today, I had to add to it. After a little brainstorming, my pair and I decided to just rip out the sub-section we needed to add to, stuff it into its own page, and embed it within the original page using an IFRAME. There were a few minor kinks to work out, such as auto-population of fields as the result of running a search from the parent page, but overall it was surprisingly easy to do. The original page doesn't even look that much different than before. The section we ripped out was a little over 1,000 lines, so we took a pretty nice bite out of the original monster. Another problem was that the test coverage for all of the features in this admin page were scattered all over the place. So I also took the opportunity to hunt them all down and put them in one place, as well as to add a few missing tests. Not only is the parent page now smaller and this sub-section more manageable, but the test coverage for these features is now in the first place my fellow engineers would expect it to be (a file of the same name with an "ftest_" prefix, per an internal standard). This was a small incremental improvement toward better maintainability of the page and its features. It's smaller, easier to find the relevant tests, and the UI acts and feels almost exactly the same. We don't intend to convert every subsection into an IFRAME and call it quits, necessarily. But as far as making a decision to improve and executing on it, I was delighted with the straightforwardness. This isn't what I'd call an elegant solution -- perhaps marginally clever -- but it is a simple and effective one. I've been programmed to hate IFRAMEs over the years. In this case, however, they've proven to be a pretty handy tool!

Learning to write tests that matter

2008-10-20T20:06:00.000-07:00

I've been grinding away for a few weeks now on FORCE, my VERGE 3 map editor. I set aside a little time each day to spend on it. Development is primarily test-driven, and I commit small changes frequently. It's a nice change of pace from the workplace where frequent commits aren't as feasible and a multitude of other concerns are involved, such as monitoring time-consuming builds and pushes, checking the production environment, etc.

Test explosion on the way to Testopia

My first tests were testing functionality at a ridiculous level of granularity. I haven't developed many projects of my own test-first, so it has been valuable to operate mindlessly in that mode for a while. Fluently writing tests, even bad ones, was worthwhile. I didn't know what level of testing was most valuable. So I started at the finest level I could think of and let time and experience work it all out. Writing a lot of small tests all of the time is a great way to jar yourself into thinking more critically. Or at least, patterns began to emerge very quickly. A lot of my tests turned out to be incredibly similar, to the point of uselessness. I became incentivized to work smarter. For example, while reading VERGE 3 map headers, there are a number of 256-char strings that contain the map's name, the tileset filename, music filename, etc. I added tests for reading each individual string, and implemented a method to read each string as I went. Testing mindlessly at this level is easy, and I ended up with a bunch of crap like this:

def test_read_name(self):
    self.map.read_name(StringIO(VERGE3Map.padded_string(self.name)))
    self.assertEqual(self.map.name, self.name)
def test_reading_name_fails_if_too_few_bytes(self):
    self.assertRaisesBecauseStringTooShort(self.map.read_name, self.name)

def test_read_vsp_filename(self):
    self.map.read_vsp_filename(StringIO(VERGE3Map.padded_string(self.vsp_filename)))
    self.assertEqual(self.map.vsp_filename, self.vsp_filename)
def test_reading_vsp_filename_fails_if_too_few_bytes(self):
    self.assertRaisesBecauseStringTooShort(self.map.read_vsp_filename, self.vsp_filename)

def test_read_music_filename(self):
    self.map.read_music_filename(StringIO(VERGE3Map.padded_string(self.music_filename)))
    self.assertEqual(self.map.music_filename, self.music_filename)
def test_reading_music_filename_fails_if_too_few_bytes(self):
    self.assertRaisesBecauseStringTooShort(self.map.read_music_filename, self.music_filename)

Write too many tests like this and you begin to wonder why you're picking specks of dirt up off the floor instead of using a vacuum. There are a few things I realized after revisiting these tests with some hundred or more commits worth of hindsight:

These tests are all reading padded strings. I should have tests for reading padded strings in the appropriate location. An I/O testing class would be better than the VERGE 3 map loading test class.
I don't know what I was thinking with the "fails if too few bytes" tests. Those should also go in an I/O testing class, if anywhere. They probably shouldn't exist at all. I don't know what scenario these were intended to account for, other than my knowledge that a file could somehow not have enough bytes. A common phrase in my workplace is "Test to your level of paranoia." I can conceive of a lot of crazy failure cases when reading from disk, but I'm not actually worried about all of them. Particularly not this early in the game. Don't write tests for scenarios just because you can. Write tests that support a specific business need. Or at the very least, likely scenarios, preferably ones that you've actually experienced before.
Mostly, I just want to know that I can read a VERGE 3 map header accurately here. All of these tests are sort of over-specific paths toward that goal. All I really needed was a test that reads a full header and asserts I was able to get the expected values. A couple other scenarios of interest are "Is this actually a VERGE 3 map? What if the signature is wrong?" and "Is the version correct? What happens if this is a version I don't support?" These are the kinds of questions that need tests. I don't need tests that ask over and over again "Can I read a string? Can I read a string? Can I read a string?"

After many rewrites, I'm much more satisfied now with what I'm testing and the value that is being provided:

def readHeader(self, header):
    self.map.read_header(io.fakefile(header.pack()))
def assertErrorsEqual(self, errors):
    self.assertEqual(errors, self.map.errors())

def test_bad_signature_in_header(self):
    self.readHeader(VERGE3MapHeader(signature='123456'))
    self.assertErrorsEqual([BadSignatureError('123456')])

def test_bad_version_in_header(self):
    self.readHeader(VERGE3MapHeader(signature=VERGE3_MAP_SIGNATURE, version=123))
    self.assertErrorsEqual([BadVersionError(123)])

def test_can_read_header(self):
    self.readHeader(VERGE3MapHeader(name='Test map',
                                    script_offset=1,
                                    vsp='test.vsp',
                                    music='test.mod',
                                    rstring='2,R,3,1',
                                    start_script='test_startup',
                                    start_position=(3, 4)))
    self.assertEqual(self.map.header.name, 'Test map')
    self.assertEqual(self.map.header.script_offset, 1)
    self.assertEqual(self.map.header.vsp, 'test.vsp')
    self.assertEqual(self.map.header.music, 'test.mod')
    self.assertEqual(self.map.header.rstring, '2,R,3,1')
    self.assertEqual(self.map.header.start_script, 'test_startup')
    self.assertEqual(self.map.header.start_position, (3, 4))

These tests cover a few different specific scenarios that I care about, rather than testing a lot of minutiae that is covered elsewhere or covered naturally as the side effect of a broader test. Testing in this way may be incredibly obvious to you, but the value I've personally gained here is the capacity to recognize a more intelligent solution by spending loads of time brute forcing a mindless one. Most of the time when I don't know what to test or even how to test something, it turns out that I just haven't "done it wrong" enough. I haven't slogged it out in the trenches long enough to give me the necessary experience to conceive of what the next step should be. I suppose this is all along the lines of "You must learn to walk before you can run." Sometimes, you realize you haven't even learned how to crawl yet!

How to run SWT apps on a Mac

2008-09-23T22:16:00.001-07:00

Talk about victory at the 11th hour. I had almost given up hope. I just needed to brush up on my Google-fu and read a little more carefully. I found a thread in the Apple mailing lists with some useful information. I haven't fully explored the details, since I'm mostly happy that I got this to work at all, but it has to do with Java on the Mac not running from the main thread. Somehow that doesn't jive with Carbon or Coca (one of the two.) The solution is to override the default behavior with a JVM switch. Running Java from a terminal, you would normally specify this like so:

java -XstartOnFirstThread myapp

With Jython, it's only slightly different:

jython -J-XstartOnFirstThread myapp.py

What a load off. Victory is mine!

Installing SWT

The solution above assumes you've already installed SWT. Here's how you do that. There isn't a formal installation process. You just download it and copy it somewhere. You can get it here. Make sure you download the OS X version. I downloaded the most recent version and simply copied the swt.jar file directly into my development folder. Then I ran the following commands:

JYTHONPATH=.:swt.jar
export JYTHONPATH

This makes the current working directory and swt.jar visible to Jython. You do need to specify both. You'll probably want to put that in your shell's startup script. I run zsh, so I dropped those two commands into my home folder's .zshrc file. For reference, I'm running Java 1.5.0_13 and Jython 2.5a3.

Swing wins over SWT on the Mac

2008-09-23T21:37:00.000-07:00

At least for me! I spent the last day or so trying to figure out how to get even a simple barebones SWT application running on this Mac mini. No dice. I can launch a SWT application, but I am inevitably met with the Mac's swirling lollipop of death, as I've come to call it. While it's true that I'm no Mac or *nix expert, and I was trying to do this all from a terminal, I'm still fairly surprised this has proven so difficult. I even went so far as to install Eclipse and try and follow the instructions of a step-by-step SWT tutorial. I just feel mired and sapped of all energy! So, I'm going to use Swing.

I really like stackoverflow.com

2008-09-22T12:02:00.000-07:00

I followed the podcasts leading up to the public beta, and was very curious to see the end result. I really like how it's shaping up. I submitted my first question a couple days ago. I had already discovered an answer, but I felt the information was valuable enough to share with others. I think the answers and votes worked out well. My answer, which was the most specific and handled all of the cases in the question, had the most votes. Whereas another answer which didn't, but which still imparted useful information, had about half as many votes. I like it. I wrestled a bit with the specificity of the question's title. I opted for a shorter and less exacting title, with more details in the body. I'm not decided on whether that was the best route to take, but it was a good first effort.

These are a few of my favorite things.

Having only used the site for a few days, here are my impressions: It's simple, clean, and functional. Contrary to various remarks I've read about it being "ugly", I like how it looks. As a site that answers programming-related questions, it does a great job. You could call it "intuitive", but I think a better description would be "obvious". The reputation system teaches me how to become a better citizen. When I first posted my question and got the more generic response, my immediate reaction was to downvote the answer. And I did. But of course, the attempt notified me that my reputation was not yet high enough. This gave me a moment to reflect and realize that the question, while not the exact solution I was looking for, was still useful information. You've got to actually use the site in order to gain reputation enough to influence it. During this time you learn how to become a good citizen. "I can't downvote? Why?" You poke around and you discover why. Rinse and repeat. The notification bars were a great move. They appear as overlays at the top of the page. When you're new to the site, one appears which links you to the FAQ. It also draws attention to achievements, such as when you earn badges for your participation. It's a nice little tip of the hat. "We see what you did there." As a geek and a programmer, it's a fun place to visit. Apart from finding answers to questions of interest, there some humorous programming-related questions, such as What's your favorite "programmer" cartoon? My favorite is definitely this one. The site is great. Very nice work!

Map editor domain model

2008-09-21T20:46:00.000-07:00

What is this "domain model" jazz that I keep talking about? It's really simple. I have various features planned for my map editor. The core data structures and logic for those features is my domain model. For example, I plan to support the ability to undo actions in the editor, such as the plotting of tiles. Part of the "model" I've created to provide this ability is the CommandTimeline class. This class concerns itself only with the mechanics of an undoable history of commands. Another ability I plan to support is, of course, the plotting of tiles. The model I've created for this exists in the Plotter class. You'll notice these classes have fairly light dependencies. "Light" in the sense that they aren't written within the context of UI or heavily dependent on some third party library, such as, say, pygame. One of the benefits of this independence is code reuse. Another is testability.

The Plotter class

I'm growing to love this class. It might not look like it at first, but it's pretty flexible. It can:

plot individual tiles
fill a rectangular area with the same tile
plot a brush (a rectangle composed of different tiles)
fill a rectangular area with a brush

The signature for the plot() method looks like this:

def plot(self, map, start_inclusive, end_exclusive = None):

The interesting thing here is the first argument, map. This isn't a "map" in the VERGE 3 sense of the word. Think of it as a grid, which can be as small as 1x1. Refer to my old blog for a more detailed explanation. The plot() method can plot any object to any grid, as long as that object has the following interface:

def get_tile(self, x, y):

The plot() method sets every value in the grid from the starting coordinate to the ending coordinate to a value returned by this method. This had interesting ramifications for the Tile class. Up until I decided on the interface requirement for the plot() method, there really wasn't yet a reason for a Tile class to exist. I'm letting tests drive my design, so this class may eventually include an image. Then again, it might not! We'll see where the tests take me.

Plotting a single tile

So of course when the plot() method is called with a Tile instance and a single starting position, the logic in this method will calculate the ending point such that the area iterated over is only one tile. The call to get_tile() will return itself and the end result will be that a single tile is modified in the destination map.

Filling a rectangular area with a tile

If you call plot() with a Tile instance and both a starting and ending position specified, the entire region between those points will be covered with the same tile because get_tile() always returns the same thing.

Plotting a brush

The Map class also implements get_tile(), except that it returns values corresponding to positions within the map. Calling plot() with a Map instance and a starting and ending position that matches its size will result in that collection of tiles being applied to the destination map at the appropriate location.

Filling a rectangular area with a brush

In addition to returning coordinates that correspond to position within the map, the Map class implements get_tile() in such a way that coordinates which fall "outside" the map are wrapped back around in a uniform way. The end result is that calling plot() with a Map instance and starting and ending positions that are larger than its dimensions will fill the "extra space" with a repeating pattern of the map. There are some edge cases I've yet to consider, but I'm really happy with this so far.

It's cool.

Working in this way -- isolated away from other concerns such as UI or third party libraries -- has been very rewarding so far. Without all of that other cruft getting in my way, my train of thought feels much more simple and concise. There are less distractions. I might even go so far as to say it's a less stressful way to develop. I'm juggling far less concerns and funneling the leftover brain cycles into the quality of each behavior I'm modeling. The real test will be how everything develops as I introduce more complexity into the domain, and also what will ultimately happen when it comes time to leverage the model at a higher level, such as in the context of UI. Right now I can't see how it will be anything other than a breeze.

Python's struct.unpack() is awesome

2008-09-15T21:27:00.000-07:00

I just realized my endian-ness "problem" with Jython really isn't a problem at all. If you refer to the end of my previous post, here's the solution:

struct.unpack('<i', version)

The lesser-than indicates that a little-endian interpretation is desired. I've been cackling maniacally for the last 5 minutes. I can't believe my problem was solved with one character.

Jython 2.5 wins

2008-09-15T20:49:00.000-07:00

I'm already spoiled on Python 2.5, so this was a no-brainer. I have no idea how I missed the alpha on the home page, but thanks to a helpful reader I am now thoroughly enjoying myself. Everything is reverted back to its former glory. The only issue I have now is switching the default endian-ness when opening files with open(). Java defaults to big-endian, basically. Here's the problem in action!

E:\dev\force-me>jython
Jython 2.5a3 (trunk:5315:5317, Sep 10 2008, 20:54:23)
[Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_03
Type "help", "copyright", "credits" or "license" for more information.
>>> import struct
>>> f = open('maps/grue0030.vsp', 'rb')
>>> f.seek(4)
>>> version = f.read(4)
>>> struct.unpack('i', version)
(100663296,)
>>>

The same steps in Python yield an unpacked value of (6,), which is the version of VERGE 3 tile sets. I remember from my Java days being able to set the endian-ness on Buffer instances, but I'm not sure how that knowledge helps me here. Or perhaps this is related more directly to struct.unpack()? Couldn't say. Nothing a little RTFM-ing it up won't fix...

Working with Jython

2008-09-15T01:02:00.001-07:00

I decided to actually run all of my tests in Jython a few hours ago. A lot of tests failed. But that was somewhat expected. I've been developing with Python 2.5, and Jython is roughly on par with Python 2.2. The conversion took an hour or so, which wasn't too bad. Perhaps the most annoying to me is the lack of decorators. I have some static methods in a couple classes, such as:

@staticmethod
def flip_vertical(data):
  return data[::-1]

Which of course became:

def flip_vertical(data):
  return data[::-1]
flip_vertical = staticmethod(flip_vertical)

Python 2.2 also doesn't have the third param for slice increments, so I couldn't use the [::-1] idiom for reversing a sequence, and had to roll my own:

def reversed(list):
  result = list[:]
  result.reverse()
  return result

def flip_vertical(data):
  return reversed(data)
flip_vertical = staticmethod(flip_vertical)

There were also a few little things like having to replace assertTrue() and assertFalse() with failUnless() and failIf(), as well as no [chars] param for split() and no [fillchar] param for ljust(). You can see the whole commit here. This was a relatively painless conversion since I did it early. Several people have also recommended IronPython to me, but I'm going to give Jython an honest try first.

Map editor timeline

2008-09-14T14:50:00.000-07:00

So right now my ETA for "feature complete" is the 1st of March '09. The rough number I came up with was 40 days, so I spread that out across 4 weekends per month. This takes into account the fact that I'll usually only be devoting time on the weekends to work on the editor. Some weekends are longer than others and I'll probably work on it some week days too, so I'm sure it evens out somehow in the end. I'll have a functional editor before then, but of course it won't have all the bells and whistles that VERGE users are accustomed to. I'll come up with timelines for milestones between now and then in the near future. Here was the list I used to help me with this first quick pass:

Load and save VERGE 3 MAP, VSP, CHR 3 days
Scrollable map viewport 3 days
Scrollable tile selector 3 days
Layer manager 3 days
Plotting tools: pencil, rect, fill, eye-dropper, eraser 5 days
Minimap 2 days
Undo 4 days (distributed cost)
Cut and paste 2 days
Brushes 3 days
Map layer renderer 2 days
Tile selector renderer 2 days
Brush renderer 2 days
Obstruction layer renderer 2 days
Zone layer renderer 2 days
Entities renderer 2 days

Very rough, but it helps you get a sense for the general scope of the project. I'm learning to estimate high, in part because I've found that on every project I've ever worked on there are "hidden costs" that arise, such as the effort involved in integrating components into a cohesive whole.

Map inspection tool

2008-09-13T17:41:00.001-07:00

I've setup a new project on Google Code for FORCE. You can check in on the commits here. My next post will start elaborating a bit on the domain model and where I'm going with it. I've started a little bit on some VERGE 3 map format loaders as well. To help start exercising them, I wrote a little command-line map inspection tool. Here's what it looks like in action:

E:\dev\force-me>inspect maps/*

E:\dev\force-me>python inspect.py maps/*

:maps\bumville.map (84708 bytes)
         Script offset > 84555 (153 bytes)
              Map name > Bumsville: City of Bums
                   VSP > grue0030.vsp
        Music filename >
         Render string > 2,1,3,7,8,E,6,4,5,R
        Startup script > start
         Player starts > (0, 0)
              # Layers > 8

:maps\city01.map (33009 bytes)
         Script offset > 32615 (394 bytes)
              Map name >
                   VSP > city01.vsp
        Music filename >
         Render string > 1,2,3,E,4,5,R
        Startup script > start
         Player starts > (0, 0)
              # Layers > 5

:maps\city01.vsp (674436 bytes)
        Not a map. Skipping...

:maps\grue0030.vsp (102750 bytes)
        Not a map. Skipping...

As you can see, it supports globbing. This tool will eventually report on the dimensions and names of layers, entities, and zones. When globbing, it will calculate totals as well. Think of it as a VERGE 3 file format dir or ls equivalent. I also want to extend this or create an additional tool with search capabilities for locating maps that use specific tile sets or contain certain entities. If, like me, you have a ton of maps and tile sets lying around -- and you're horrible at organizing them -- this will help you find what you're looking for.

It's probably not as hard as you think.

2008-09-10T18:18:00.000-07:00

This is a reminder to myself that usually when I don't want to write a test or a function, because it seems too difficult, I need to ask myself if I've actually tried. A lot of the time I'm just talking myself out of it because code "looks scary" or because I'm regurgitating lore from other engineers who said it was scary. I recently implemented a new payment method at work, and today a fellow engineer needed a simple test function to create a fulfilled order with that payment method. He needed this because he was updating our revenue report to include the numbers for the newly added payment method. No such function existed in any of my automated tests -- I was hitting all of the pages in this new payment method's checkout flow and producing full orders this way. It's slow -- hitting the network for each page along the way -- but it works. The problem here of course is that someone who wanted to deal with the new payment method I had just created didn't have the tools he needed to do so. Since I wrote the implementation, it made sense for us to pair on the creation of this new test function. Now the fun part: Our payment system has a lot of legacy code in it. And I mean legacy in both the Feathers sense of the word and the other commonly associated sense -- it has no tests and is also ancient. It hasn't received much refactoring love over the years, so there is a lot of logic dispersed across numerous pages in a dozen or more payment methods. A lot of it is duplicated as well. When we started writing this function, we weren't sure what questions to ask really. The uncertainty of what we were fidgeting around with induced a little bit of development paralysis. So, we walked over to a tech lead to get his thoughts on the matter. Ultimately, we were talked into doing what we were hesitant to do. With a mild reluctance, we decided okay. We're going to pay this debt. It might take a while, and we're going to have to do some spelunking, but we're going to do it. The amazing thing was, we had figured it all out within the next 30 minutes or less. We drove the development of the function with tests of course. It never ceases to amaze me how quickly you reach the end of the tunnel. Especially in situations like this where you are so unsure of what comes next. You knuckle down, poke around, and think critically enough to actually write a test -- and before you know it the test is passing. And your response is inevitably "...oh. It's done... That's all it took?" And you sit there for a moment kind of flabbergasted. And then you commit! Very rewarding. It feels like Good Things come early and often to those who are disciplined. Never talk yourself out of doing something because it seems difficult or someone told you it was. Take a whack at it. Pair with someone. Pull in another head before too long if things start getting complicated. Success can be small and incremental. Also, a more experienced engineer can add a lot of clarity to the situation even without involved knowledge of what it is your tackling, merely by knowing the right questions to ask and helping to put a finer point on what parts of the problem you actually should care about solving. Often you will try to solve a problem that is bigger than necessary. Most of the time all you want is a small incremental improvement.

I'm going to ship a map editor.

2008-09-09T21:17:00.000-07:00

So in the spirit of actually picking a project, sticking with it, and shipping a product that real people actually use and enjoy, I'm going to write a map editor. There is a clear need for a good one in the VERGE community and it's something I of course have personal interest in. I've obviously had plans for something like this for a long time. And I obviously haven't finished anything. What has changed now? I'm discarding the mental mindset of "this is just research". This will be The Real McCoy. I've had shiny ball syndrome all these years and I kind of already knew it. My laid-back attitude is probably why I've kept it for so long. Looking back now, I think that really holds you back. It's time to finish something now. I've made a lot of clever acronyms in my time, but I'm going to stick with FORCE for this offering. It stands for "Fully Operational Retro Console Editor". I had originally chosen choosetheforce.com as my domain with this in mind, intending to eventually replace the final E with "Engine". But I'll cross that bridge when I get to it. On to the gameplan. I was going to create a pretty awesome World of Warcraft-esque talent chart graphic to help illustrate, but I'm not fluent in any software that would allow me to do so in a short amount of time. So I'll stick with what I know: lists. Here's a sort of high-level breakdown and long-term roadmap of development stages/buckets, some of which can be worked on in parallel, some not.

The domain model. I've already started on this one, since all of the other stages depend on it. This is all the core data and operations. Tiles, tile sets, brushes, map layers, maps, undo, plotting with tiles & brushes, copy & paste, the eye-dropper -- all of that. If my jargon alarms you, "domain model" is just a more formal term for "my data structures 'n stuff." I've been reading Eric Evans' book Domain Driven Design for quite some time now, and will often refer to it as my experience and learning increase. This is one of my Bibles. Jeffrey Palermo's The Onion Architecture is also a great articulation of an architecture I agree with and which I use as a sort of guiding light.
Renderers. Doesn't exactly roll off the tongue... maybe "visualization" is better. The rendering of tiles, map layers, maps as a whole, obstruction tiles, zone tiles, entities, and all that goes here. Of course this relies on the domain model.
File formats. I plan to support VERGE 3 file formats first. This is its own little island and is sort of a highly specific domain model. Understanding all the little details of the VERGE map format in isolation is a good thing. This can be worked on somewhat in parallel with other stages, and also allows me to ship other lower-hanging fruit before shipping a full GUI -- command-line tools, such as converters, etc. I've done extensive work with loading and saving code for all 3 of VERGE's map (and all its other) formats in the past. I've already got a leg-up here.
Simple map viewer. This will be the stage where I start crafting an actual UI and rendering maps in a scrollable area. You can think of this as the "image viewer" stage in development. This won't be entirely useful for anything other than making sure I can render maps correctly within this context, and that it doesn't explode in the face of corrupted maps. There will probably be crude support for a few things like layer visibility toggling.
Advanced map viewer. I'll probably focus on improving rendering speed in the UI here, and add in things like zooming and dragging of the viewport. I'll probably also add in the minimap at this stage.
Simple map editor. Here of course simple editing will be implemented, probably only with the "pencil" tool, an eye-dropper, undo, and the other basic amenities. This will be the first stage where I'll have a deliverable that could be considered usable, insomuch as it will satisfy the requirements of a VERGE map editor: You will be able to load a map, modify it, and save it back again, while plotting and undoing things all along the way.
Advanced map editor. This will be where I add in all the creature comforts such as a broader repertoire of plotting tools like rectangles, lines, flood-fill, brushes, etc. There will be simple editing abilities for all of the data specific to the VERGE map formats as well, like zones, obstructions, and entity data.

I'll be diving into greater detail on each of these as I tackle them. You'll see me breaking them down further and even producing some timelines for deliverables *gasp*.

I just discovered Jython.

2008-09-08T10:01:00.001-07:00

And it is glorious! Mind you, I have a history with Java. If you've followed my old blog, you know this. I devoted quite a number of posts to a Java-based tile map editor named SMEE. I've always loved Swing, even though most people have a lot of disdain for its L&F. I like that it was designed to "breathe". That is, the layout managers are sensitive to font-sizes. I also developed a fondness for the JScrollPane and how easy it was to extend if speed was a concern. Fast-forward from 8 years ago and I've spent about a year now tinkering with Python. However, most of the UI libraries I've tried have been a little too rough around the edges. Swing never had that native look or polish, but they put a hell of a lot of effort into it. You can imagine my surprise when I discovered there was a Python interpreter written in Java. I have access to Swing and a stellar 2D graphics API... from Python? In-credible. Oh yeah, and there's SWT, so apparently I can have my cake and eat it too... on top of already having my cake and eating it too with Jython. Witness the power of this fully-operational Pythonic Java death-star! More on this soon. I just needed to gush a little bit in advance.

I taught myself how to finish.

2008-09-03T22:33:00.001-07:00

Go read this right now. I stumbled across the above post after having composed the list below -- I'd been putting a lot of thought into it myself recently. Here are a few of my own observations on getting things done:

Multiple heads are better than one.
This is particularly true when you are in unfamiliar territory. Finding someone to ask the questions that need to be asked, to help you understand, and to challenge your understanding is invaluable.
Confidence plays a major role in problem-solving.
Facing unknowns with calm inquisitiveness will teach you the most in the shortest amount of time. Anger and frustration severely impede learning and therefore progress. Increasing your learning increases your confidence.
Coding without a concrete goal is like running a race without a finish line.
If you don't know where the finish line is, it's easy to talk yourself out of finishing. Articulate your goal somewhere, anywhere, and break it down. Get a sense for what's actually involved. Now you have a measuring stick.

Some brief thoughts on each: Every time I collaborate, particularly with someone who is technically savvy and knows more than I do, I develop a more thorough understanding of the problem I'm attempting to solve. That builds confidence. It was interesting to read Rands' words on confidence in particular. Did you read his post yet? He does a much better job explaining it than I ever could. Rands' bug database? That's an articulated goal. It also has the valuable property of being broken down into small manageable tasks. 537 of them. A goal which is broken down in this way is a great confidence-building ruler. I've been putting all of this to the test at work. My most recent project just shipped -- one that seemed vague and hand-wavy for the first week. Once I actually articulated a goal, grabbed someone to talk to, and came up with a clear strategy for the endgame, my confidence went through the roof. My progress from that point onward was focused and consistent. Bizarre as this may sound, I've never had this experience with a project before. It's a personal breakthrough. The revelation is: it's not even difficult. The mind boggles. I'm never going back. Now to teach myself to finish on time...

Finishing a project is tough.

2008-09-02T23:34:00.000-07:00

Especially if you've never developed the habit. I've started a lot of projects over the years and I can't think of one off the top of my head that I've actually completed. That's about 14 years now! Ouch. I have a lot of enthusiasm, but I've noticed I tend to develop in spurts. I'll bang away at something for a while and then drift onto something else in an ever-so-non-committal sort of way. I think this is partially because my attitude has been a very laid-back one. "No stress man, I'm just fooling around..." Not enjoying that so much anymore. Where are the fruits of my motherloving labor? I'm getting to a point in my life where I want to start actually completing some projects. Projects of my own. I want to point anybody to a website and say "Here's what I've done. It's shipped. It's out there. Shazam!" I'll make jazz hands. It'll be great. At work I've been nearing completion of a new payment method within the mothership that is our payment system. I've had a good amount of help getting started, but I've written most of it myself at this point. And all I can say is: Gadzooks! What an exercise in self-discipline. Happily, development hasn't been so much rough as it was extended. As in, even beyond my teach-lead assisted estimate. I said X hours. He said, "Haha, no. Multiply that by 3." I said, "Fine by me! CAKE." Fast-forward to 4x my initial estimate and I'm almost done. The good news is I've started to accrue enough good development habits that even though this took longer than expected, the process hasn't been a fraction as nerve-wracking as smaller projects I've attempted to tackle at my previous employ. I'm calm, thoughtful, diligent, and -- best of all -- confident. All that literature out there about "habitual excellence" doesn't seem so much like mumbo-jumbo to me anymore. I want habits that will set me up for success. I actively seek them out. Every day. Speaking of success, that's another thing I'm learning: you gotta give yourself a lot of early successes. Don't try and go whole-hog outta the gate. All that agile crap about doing the simplest thing that'll possibly work isn't all that crappy. YAGNI and all of its friends in the agile Acronomicon are trying to jam this into your head for a reason. They're arrows along the trail that point you in the right direction: a finished product. I could keep going. And I will in my next post! I'll share some thoughts on one of the things I think is a key difficulty in finishing a project: breaking a project down into small enough tasks so that you can get a sense for its scope, and how that actually empowers you to make better estimates, as well as defer tasks for the next release when deadlines loom.

Think concretely, develop incrementally

2008-08-21T12:42:00.000-07:00

I was reading Jeff Atwood's Check In Early, Check In Often post this morning and I thought "Pfft, you've already talked about this crap." And sure enough, he does mention the previous post it is similar to. But guess what I didn't do last night? I didn't commit pending changes I had. I figured I'd do it during lunch today. And I did. And two of our slaves on BuildBot went red. Way to go, dolt! What I should have done is committed each few conversions as I went. If I encountered a breaking change I didn't want to deal with, I should have reverted all of my changes and moved on to the next test. If I was still working on some changes before going home, I should have reverted them all. Last night I was running around like a crazy person converting a bunch of XPath locators in our Selenium test coverage to use cssQuery style selectors. We run Selenium under IE7 and the XPath implementation is surprisingly slow for locators of even mild complexity. It's an order of magnitude slower. I wasn't in a hurry per se, but I was a little ADD. If certain XPaths started breaking a test and it wasn't immediately obvious why, I'd move on to another test. Unfortunately for me, I didn't revert all of my changes to each of these files. By the time lunch rolled around today I'd completely forgotten I'd left some of these tests broken. This is one of the perils of not checking in all of your changes frequently. You forget where you left off. You end up saying "Oops!" and "My bad..." a lot. "ADD" really is an apt description of my primary "mode of development" prior to working at an agile company. I spent most of my time tackling the shiniest problem or way too much time polishing pointless features to make them shiny. There were goals to reach for and deadlines to hit, but my path along the way was very erratic and full of detours. I've noticed that test-driven development helps me pave a more direct path toward my goal. I spend a lot more time working on code that is directly related to fulfilling business requirements and getting one step closer to project completion. Every time I deviate from this path by plowing ahead without tests, I inevitably begin to lose clarity and spend time flim-flamming around, poking and prodding my code to "see what happens." "Coding from the hip" has always appealed to me, but in practice I keep finding out that it's really not so great. I shoot up the place and hit a lot of things I meant to hit, but usually a bunch of things I didn't mean to hit either. It's just plain sloppy, and a waste of good ammunition! I'm a slow and ponderous developer, and like to really wrap my head around a problem before proceeding, so perhaps that's part of why I value writing tests first so much. They help me articulate the problems I need to solve. They help me think concretely, and keep me focused. This focus also helps me stick to one problem at a time, and therefore to develop incrementally. Automated tests are sort of like guardian angels, too. When you screw up and break everything all to hell with your hair-brained schemes later on, they will often swoop in and save you from a predicament by giving you advance notice that you're going to detonate a bomb if you push your changes to production because you just inadvertently left a trail of gunpowder to and from a bunch of subsystems and lit fuses here, here, and here. It's good to know when you've screwed up before it gets into customers' hands. You have time to react and prevent disaster! So, this is perhaps a personal reminder to myself to stay focused and tackle one small problem at a time. Write tests first! Do it! DO IT NOW.

Layer management, Refactoring, The road ahead

2008-05-29T08:46:00.000-07:00

There are about 20 posts left to import from the old blog, for those curious. Here is today's treatment:

Build 15 Cameth: I give the UI a little more polish here, add the ability create and remove layers, erase tiles, and a few other misc. items like an eye-dropper.
Build 16: This build was devoted to some heavy refactoring. I continue to make small UI improvements. This is a long and detailed post about the nitty gritty details of event handling within the context of Java. I talk about challenges I faced, what choices I made, and the lessons I learned along the way.
SMEE Roadmap: This is a quick summary of where I had intended to take SMEE.

Multiple Layers, Bugfixes, and Polishing Hubcaps

2008-05-23T08:45:00.000-07:00

More unearthed entries:

SMEE RULZ: Haven't you heard?
Multiple Layers, Part 1: I introduce support for layering into SMEE here, and give a high level overview of its implementation.
Multiple Layers, Part 2: I add the ability to actually plot tiles to the currently selected layer here. I muse a little bit about the overall architecture of SMEE, and how the subsystems interact with one another. I also take a moment to share some thoughts on test-driven development, which was very new to me at the time.
Shrinking Scroll Pane Bugfix: The switch from JDK 1.5 to 1.6 gave me a little trouble with Swing. SMEE's viewport would shrink to a tiny speck when the map being displayed was larger than the window. Since that would likely be the common case, I fix it.
Build 15 Cometh: SMEE starts to look like something like a Real Application in this post. I steal some classy icons and cursors, add a "currently selected tiles" panel for viewing the tiles that will plot if you press the left, middle, or right mouse buttons, and implement drag and drop reordering in the layer manager, as well as include layer management actions in the undo system.

On the value of meetings

2008-05-16T15:50:00.000-07:00

I just got out of a meta meeting. You may be familiar with meta meetings. They are usually of the recurring daily or weekly variety. Instead of serving whatever purpose the meeting had originally intended, you end up talking about the meeting itself. Should we change the nature of the meeting? Should we meet at different times? Should we cancel the meeting? Code can develop smells over time. I think meetings can too. In our case, the meta nature of this meeting has begun to surface due to the growing size of our company. I am reminded of the Single Responsibility Principle. In the same way that a function or a class should have a distinct purpose which is easily described and understood, I think the same holds true for meetings. Otherwise they eventually become vague, noisy, and more trouble than they are worth. This may be a stretch, but it seems a little like each person you add to a meeting is a bit like adding another line of code to a function. It will simply become unruly after a while. And whenever that happens, you probably need to... Refactor!

Investigating backward compatibility

2008-05-15T11:24:00.000-07:00

Part of our 3D client stopped working recently. There are some issues with a newer version of our inventory panel, which is written in Flash (versus C++). That is to say, ~~it doesn't show up at all~~ using one particular tab within the panel causes the Flash to quickly become unresponsive. Oh yeah, and this only happens under Windows 98. I am tasked with figuring out whether or not this is worth fixing. Our statistics tell us that Windows 98 makes up about 0.3% of our user base. That doesn't appear significant, superficially, but it all comes down to the numbers. How much does it cost to fix it? How much money we will (potentially) lose if those people using Windows 98 decide to stop using our product if we abandon compatibility? Out of curiosity, I took a look at the general user base of Windows 98 on a global scale. It's .99% at the time of this posting. We've never actively kept an eye out for what does and doesn't work in our 3D client under Windows 98. It's a rather happy coincidence it runs at all under Windows 98 at this point (well, we did invest a little time in this a while back). My concern is that something completely random, tiny, and obscure may have been introduced which has thrown a monkey wrench into the works. This may be incredibly difficult to diagnose. I'm going to spend an hour or so today putting some feelers out. I'll report on my findings later in the comments here.

Tile ripping, Faking isometric, and Viewport dragging

2008-05-12T22:21:00.000-07:00

I just finished moving out into the sticks! Luckily, they have the intarwebs here. One of the best things about this place: trampoline. 'Nuff said. And now, for your reading pleasure... More hits, all the way FROM THE YEAR 2000!*

Tile Ripping: This was a fun little exercise in tile ripping. Any hobbyist game maker wants to rip tiles at some point or another. I also dream up an excuse to play with MD5. Furthermore, a stripper is involved. But don't worry. It's SFW.
Isometric: I do a quick proof of an isometric view here, and I do it without all the crazy math you usually see in isometric tiling tutorials.
Viewport Dragging: This is another monster of a post. I was in the process of switching from Java 1.5 to 1.6, and I make mention of the manhandling I had to do to my fast custom viewport scroller. Dare to read this post and you shall be privy to such vocabularic poeticisms as "semi-wackified".

* Actually from the year 2005. Years on blog may appear smaller than they are. Void where prohibited.

Learn by doing, viewport optimizations, and MVC

2008-05-07T22:36:00.000-07:00

Today's selection of oldies-but-goodies:

Unit Testing Mu, Part 3: I start off with a motivational pep talk about how you should "just do it" and learn something. I muse a bit about unit testing, then dive right into some concrete testing and the questions I found myself asking.
The Need For Speed: This post covers a topic I spent a lot of time learning about and wrestling with the details of-- fast viewport scrolling. Some simple improvements to do with tile caching and tile plotting are covered as well. At a later date I eventually wrote a custom implementation of JViewport which allows fast scrolling diagonally. This is something Java does not support with scroll panes by default. I'll dig that up and post about it soon. It was an epic adventure!
Model-View-Controller: In this post I talk a bit about design patterns before plunging into MVC. This was around the time I had discovered design patterns. Apparently I'd also been reading some Sun Tzu. Interesting to come across this post not two days after Coding Horror covered the topic. Serendipitous!

That last guy is a monster of a post! I only briefly skimmed bits and pieces of all three of these while reformatting them, but old memories were firing left and right. I'm finding myself eager to re-read these after so long. I remember being rather excited at the time, devouring literature on patterns and testing and best practices. It will be interesting to compare notes with the me of the past!

Should you close that PHP tag?

2008-05-07T02:30:00.000-07:00

Disclaimer: This is not a burning Problem in need of a Solution. For many people, this is akin to arguing about whitespace versus tabs. I do have a reasoned opinion however, and some relevant data points worth sharing. I had an interesting discussion recently with a coworker.

"How difficult would it be to write a script that scanned all of our PHP includes and remove any closing tags (?>) and whitespace at the end of a file?" he inquired.

My reaction:

"What? Why? Don't do that."

If you've developed in PHP long enough or with codebases of sufficient size, you'll know why someone would want to do this. I'll spare you the details, but it has to do with page headers. The script was proposed as a solution to this problem. This is a Bad Idea. It is far from something I would defend to the death, but-- left to my own devices-- I will always close all opened tags. I recognize that PHP allows you to omit the closing tag, and the Zend coding standard goes so far as to actually forbid closing this tag for PHP-only files. Two things:

Opened tags want to be closed. An unclosed tag runs counter to the enforced (and therefore ingrained) balance of all other constructs. Some obvious examples:
- parentheses
- curly braces
- square brackets
- string literals
- multi-line comments
You name it, closure is enforced. Failing to do so results in a compile-time error. Omitting the closing tag is an unintuitive solution to a technical problem achieved through an abuse of syntax. During the course of the discussion, I compared this to the natural tendency most people have to adjust a crooked picture frame. Ultimately, this sort of solution is more a crutch than it is holistic-- once you rely on it, guess what happens when somebody unknowingly readjusts your clever asymmetry? You're back to square one, with your site breaking in non-obvious ways. On a (rather morbid) related note, I remember a TV show wherein a war veteran recalled how villagers in enemy territory took advantage of this same tendency. Before fleeing their homes, they would rig purposely crooked picture frames with explosives. Sure enough, some troops couldn't help themselves-- it was impulsive and habitual. An extreme comparison, but there is a parallel.
The root cause of the technical problem is not addressed. The proposed solution only partially treats the symptom it attempts to fix. It covers up one of the manholes PHP developers tend to fall into. It may serve well as a short-term band-aid between now and the time when you figure out how to "heal" your codebase, but I do not believe it is a good long-term solution. Our codebase contains over 1 million lines. We have several dozen include files batched together in one master include which is monstrously large. For N included files, there are at most N chances to accidentally fall into this particular manhole due to accidentally introduced trailing whitespace at the end of a PHP-only include. Here's the flip-side to that coin which the proposed fix does not address: for each included file there are N number of functions with M code paths that may echo content. N*M hurts more than N. In a tangle of 1 million lines, I would argue that-- due to the way our code is constructed-- we have an even greater chance to inadvertently trigger one these content echoing code paths before sending headers.

So what is the root cause of all the trouble here? The problem is that every time you include a file that closes its PHP tag, you are potentially manufacturing another manhole. Omitting your closing PHP tags may appear to be an advantageous solution, but it is a very superficial treatment. And it is not intuitive to others until they are baptized over to your religion and begin drinking your delicious koolaid. And woe to the pagans! There are two better ways to fix this.

Write tests which enforce the convention of omitting closing PHP tags for PHP-only files. Ayende recently wrote up a great post about this. This is still only a step in the right direction because it only keeps people from accidentally reintroducing closing PHP tags. You are still entirely prone to code paths prematurely echoing output "inline" before your redirect header is issued.
Realize that redirects by way of header() are naturally prone to this pitfall, and treat them as the special-case they are. Impose the rule that header() may only be called, if at all, at the top of "root" pages and cannot be called after includes, perhaps even from within any function at all. Actively enforce these rules with automated tests. Now you have exactly 1 chance to prematurely inject output into your page before headers are sent-- before the opening PHP tag. No spaces before an opening PHP tag is also an easily enforced convention through tests. Your chances to inject output prematurely become zero.

At 1 million lines, the cost of retrofitting existing code toward this design outweighs the benefit. Nonetheless, I contend that it is the "better" solution. It is much more robust. And it isn't rocket-science! It's trivial to follow and enforce.

I totally dissed Jeff Atwood

2008-05-06T22:15:00.000-07:00

...but I didn't mean to! It was honest ignorance, but I still feel a little dirty. I've been recently catching up on a lot of his older posts. I recently read his post on the meaning of "Coding Horror". Not long after I was reviewing some of my first posts, when I found the following in my second post on this new blog:

I'm a programmer and a nerd, so Coding Horror and The Daily WTF are great. Like any nerd, I need my daily dose of LOLs and WTFs. I don't feel so compelled to experience these "cover to cover." I read them in the same way I might read icanhascheezburger. "LOL!" "WTF..." And cut.

Just wow. Not even close. I hadn't been reading his blog for long at the time. There are funny posts, but I was obviously reading his blog through a LOL-colored lense. His blog has really grown on me. I find this interesting because I don't remember enjoying it much at all initially. Perhaps that was my own fault-- I was expecting fluff. Coding Horror is now one of my favorite blogs, if not the favorite. He covers a wide range of useful and geeky topics on both hardware and software. And he's been around long enough that he has posts on almost every computing topic you might care about. Jeff, I done ya wrong. And you have my sincerest apology!

Refactoring, focus management, and composite undo

2008-05-04T23:19:00.000-07:00

Avast! More old blog entries off the starboard bow!

Tightening Screws: Taking a breath to regroup and consider some refactorings. Looks like this was around the time I discovered MVC also!
Hocus Focus: Some attention toward the refactorings mentioned.
Composite Undo: I polish the undo implementation a bit here. I discuss the implementation in general, and provide examples!

It hurts when you don't test-drive

2008-05-03T23:20:00.000-07:00

The Character class in Sprytes was becoming a pain, so I yanked it. I think not test-driving it may have led me down...

The wrong path

The core problem has to do with the Character class containing four different sprites. Two for idle left and right frames, and two for walking left and right frames. Basically, because sprites each have their own coordinates and velocities, I either have to update all four every time I move, or be clever about copying positions and velocities between sprites when the "current sprite" being used and displayed changes. I tried to be clever, but was failing pretty miserably! I got to a point where I had ripped out a bunch of the physics code, then wrote some tests for left and right movement, and the tests were passing, but actually running the app had odd behavior where I would move right for a while, then tap left and my position would snap back to my initial location. After some tinkering I was able to figure out how to fix the behavior, but with a spike rather than a test. I had some sense for what was wrong-- the syncing-- but nothing firm enough to write a test in a reasonable amount of time that I understood. The more I worked on it the more I felt like Character's implementation was introducing unnecessary busywork to keep in proper working order as features were added or modified. So, back to my original point-- I don't think my Character should contain multiple sprites. It should contain multiple Animations, or some construct that doesn't maintain position and velocities. I only need one set of those for the Character.

Duck typing

This tango had me lamenting not using interfaces, since I come from a Java background, so naturally I went trolling the Python documentation for how interfaces work in Python. Basically, however, I learned that Python doesn't use interfaces. I've been reading up on duck typing for the last little while and starting to get an inkling of how it works. I'm liking it. Or perhaps I'm liking that I'm understanding it. Interfaces are more about what your objects are (in the sense that using an interface in Java requires you specify it as part of a class's definition), but duck typing is all about what your objects can do. I turned to the Wikipedia page for Duck typing first, of course. This was the line that turned on the first light bulb:

...duck typing allows polymorphism without inheritance.

I had always linked polymorphism with inheritance in my mind, so it's nice being exposed to the possibility that it can exist separately from inheritance. This did make me feel a bit uneasy though. I felt in my gut that I'd need a much more thorough understanding of what's going on so that I wouldn't be shooting myself in the foot, stabbing myself in the eye, etc. Lo and behold, the Criticism section on the wiki page has a paragraph that begins:

One issue with duck typing is that it forces the programmer to have a much wider understanding of the code he or she is working with at any given time. In a strongly and statically typed language that uses type hierarchies and parameter type checking, it's much harder to supply an unexpected object type to a class.

I also got a kick out of this analogy, which I think is pretty great:

In essence, the problem is that, "if it walks like a duck and quacks like a duck", it could be a dragon doing a duck impersonation. You may not always want to let dragons into a pond, even if they can impersonate a duck.

Well, at least my gut feeling appears to be in line with reality. Although the paragraph to follow came as some small relief:

In practice, the issue is equivalent to not mixing dissimilar objects for duck-typing and is handled almost transparently as part of the knowledge of the codebase required to maintain it.

Basically, I just need to become a little more disciplined. And that's a Good Thing!

Abort, retry, fail?win!

So, since I've yanked the Character class, I'll probably ponder the design a little bit more before making another attempt. Although right now I'm thinking an Animation class is probably a good direction to head in. And this time I'll be letting tests drive the implementation! I'll also definitely be reading some more literature on duck typing. I'm hoping some of my favorite bloggers have already treated the topic. Heads down mode, engage!

I'll tell you where you can('t) put it.

2008-05-02T21:44:00.001-07:00

I'm blogging about blogging here-- you are forewarned!

There are a lot of outlets for expressing yourself and to be heard online. I sometimes struggle with picking the correct medium. Do I blog it? Tweet it? Stick it in a stew? And once I do, how much or how little do I share?

In the large, I get it. Tweet small, blog big.

How big is too big? How much is too much?

To be truthful, I don't often ask myself whether I should tweet something versus blog about it. I more often struggle with the depth and detail to publish on my blog. Too much detail bores some people to tears.

Some people blog atypically, insomuch as they don't always "blog big."

Ayende has never shied away from blogging one-liners. Though perhaps that may change. I am not too surprised at his initial perplexity with Twitter. He was basically tweeting long before Twitter came on the scene.

Even before I was brought to the fold, I found his usage pattern interesting. Ayende uses his blog like a Swiss Army knife. You want informations? He haz dem. Sometimes a one-liner is all you need. And that's okay.

Excessive detail considered harmful

Back to my struggle, I have a ton of stuff I want to blog about. I wanted more detail. So last month I tested the waters. I really let it rip.

I'll role-play the sort of responses this elicited. One moment.

Chuck walks into the corner of the room. Barely audible are whispers of self-encouragement. He nods his head, forming his resolve. He walks to the center of the room, shaking his hands at his sides.

Chuck exhales, shedding all doubt. Inward he draws a deep, deep breath. Framing his mouth with his hands, he bellows loudly:

"BOOOOOO-RIIIIIING!"

Even though it's not what I wanted to hear, this was good feedback to receive. I'm still getting my sea legs and this sort of discovery is part of that process.

Sprytes commit "diary"

I still want this detailed information somewhere, but that somewhere is probably not here. Since the comment-ability of a blog is appealing to me, perhaps a dedicated blog is what I want. I'll not be spamming everyone with minutiae that way.

What I really want is to annotate most of my commits to the Sprytes main branch, explaining why I chose this or that and what I was thinking at the time. This would be a detailed project narrative, sort of like director commentary.

This is mostly for my benefit, and probably boring for others. Cliché #8 applies. Perhaps others will derive value from it in the distant future!

Advice

My only advice here is that expressing yourself can be tough, especially if you've got a lot to say. You've got to find your own equilibrium. And you'll figure it out quickest if you do so frequently, fearlessly, and humbly!

"You should always be failing some of the time."

Sprytes: Jumping and simple physics

2008-04-30T23:55:00.000-07:00

If you've been following my commits, you'll notice I've been fiddling around with simple physics code for jumping. I haven't written any tests around it, so I've been thinking about how I would test-drive jumping. Warning: Remedial physics math ahead! What little math I've been using is mostly leftover knowledge in my brain from tinkering with this sort of code in the past. I knew roughly that an object's speed is referred to as its velocity. For jumping, I can apply a velocity to my Y coordinate to move it. I knew I could give the hulk's Y velocity a large initial value when the user hits a key, such as the space bar. This velocity would be the thrust that propels the hulk upward. I would simply decay the velocity by a constant amount to simulate the pull of gravity.

Simulating gravity: An example

To put into concrete terms (that are arbitrary here, for the sake of explanation), assume simple pixel coordinates: the origin of (0, 0) is the upper left pixel of the screen, the X axis increases toward the right, and the Y axis increases toward the bottom. The "floor" is at a Y coordinate of 100. The Y velocity starts at 0. The Y velocity is constantly applied to the Y coordinate. Since the Y velocity is 0 by default, this has no effect most of the time. When the user presses the space bar, I give the Y velocity a value of -10. Because Y coordinates increases in the downward direction, a velocity of -10 will subtract 10 from the Y coordinate and move the hulk upward. To simulate gravity, I want to constantly apply force in the downward direction-- to counteract the upward thrust I've just created. This is achieved by constantly adding a positive value to the Y velocity. If this doesn't immediately make sense, think it through: if you're constantly adding a positive value to a negative starting value, it will eventually wrap from negative to positive. Applying this math to my hulk character, onscreen it manifests as the hulk shooting upward due to the large initial Y velocity, slowing gradually, hitting an apex point briefly and then propelling faster and faster downward toward the floor. To keep the hulk from shooting off the bottom of the screen altogether I simply limit the hulk's Y coordinate from every being greater than the Y coordinate which represents the floor-- in this example, a value of 100.

Test-driving the jumping math

The basic principle is fairly simple, but for some reason in my mind the tests I might have written seemed like they would be difficult. I decided to finally break down and do what everyone with a question does: I asked Google. I eventually found this page, which provides decent enough coverage of the basic forces involved. On a related note, I can't wait until Jeff Atwood and Joel Spolsky make this sort of intarweb-powered learning even easier. In any event, after soaking up some of this intarweb goodness, I realized that this would be easier to test if I simply broke down the math into more discrete testable units. I don't need to write a test that verifies the jumping math as a whole works-- I simply need to write tests that verify the basic principles. Whenever I have trouble formulating a test, I almost always find that I don't have enough knowledge of the problem domain. Once I research a topic well enough and become sufficiently familiar with it, I invariably come to the same conclusion: I can write better, simpler tests than I had originally expected. This simplicity comes from a knowledge that is deep enough to yield awareness of many smaller components of a more complex whole. I can write tests for:

Thrust - When the user presses the space bar, verify that a character is given an initial Y velocity of a sufficient amount. The test will be simple at first, but I can see developing it further in the future to ensure that a thrust of T is enough to make the character to reach a specific apex in pixels.
Velocity - Verify that a character is moved X or Y pixels in the appropriate direction when the character's X or Y velocities are non-zero.
Gravity - Given a Y velocity of N, verify that applying gravity to a character increases its Y velocity by G when calling some sort of "gravity application" function, where G is the amount of gravity.
Impact - Verify that a character's Y velocity is nullified and its Y coordinate reset to the ground level value if it attempts to exceed that value.

I'll be reporting back soon on how the process works out. This will be the first Sprytes functionality that I'm going to develop fully test-first. That's right-- instead of increasing my suck factor I will instead be countering it with awesomeness.

Tile selectors, clipboards, and doodads

2008-04-29T14:24:00.000-07:00

I've imported a few more old blog entries.

Tile Selector: Fake layering with 64x64 tiles vs. SMEE's standard 32x32
Doodads: Misc. additions, including a tile grid, zooming, and dialogs
Clipboard Tomfoolery: Pasting images from the clipboard into the tileset

Sprytes: Idle vs. walking animations

2008-04-26T02:05:00.000-07:00

In the current revision of Sprytes, you can move the hulk around on the screen with the left and right keys. While standing still, the idle animations cycle. When in motion, I've added walking animations. Watch it in action below. The interesting revisions are:

r37 - This is the first rough implementation of idle vs. walking animations.
r38 - I make the left and right movement timed here. Working on both fast and slow machines helped make the need for this evident.
r39 - Here I swap out one approach for managing the current sprite with another, better approach. Basically, immediate calculation of current_sprite vs. state in two variables: facing and walking. The code is much clearer with the latter.
r40 - The walking and idle animations are of different heights-- the walking animation is taller. The switch from idle to walking was jarring because of it. In lieu of a formal "hotspot anchoring" sort of solution, I aligned the sprites along their bottom edge.

Next I'm thinking of doing jumping and some simple obstructions. Probably with a different character, since the Hulk is in limited supply. If there's something you have specific interest in seeing, speak up!

Inefficiency proficiency

2008-04-25T03:26:00.000-07:00

I had used Eclipse for many years prior to working at IMVU. I worked regularly on projects containing hundreds of files. I never gave much thought to navigating the expansive list of files. I'd scroll, sometimes tap a letter key to help jump into the general location of interest. My natural dexterity and love of first-person shooters perhaps worked against me here. Scrolling around the list was annoying, but not quite annoying enough. Now that I deal with projects that contain thousands of files on a regular basis, that behavior doesn't scale well at all. If you deal with projects that have a large number of files, hunting for files like this can actually take up quite a lot of time throughout the day. But that didn't stop me from blindly doing so! I spent all 7 years at my previous job performing brute force scrolling & hunting-- I developed a blind spot by mechanically performing the same action over and over. I had become proficient at being inefficient.

CTRL+SHIFT+R

In Eclipse, CTRL+SHIFT+R alleviates this headache-- you can jump quickly to any file by simply starting to type in its name. I hadn't known about this shortcut until a coworker let me in on the secret. Briefly, here's how it works. Key in the combo to get a list of all the files in your workspace: As you key in the name of a file, the results narrow: The scenario pictured is a simple one, with only one match. If your project has multiple files with the same name, all matches will be listed along with their paths, so you can differentiate. This is far more immediate than searching through dozens of folders or pages of files on larger projects.

Are you desensitized? Know your tools.

It is all too easy to become desensitized to process complexity. Next time you sit down to develop, watch yourself-- how many little memorized tasks do you mindlessly perform? Are they truly necessary? Is there a better way? Had I taken even just a few minutes to RTFM or given it more than a passing thought during all those years working in Eclipse, I'd have discovered CTRL+SHIFT+R in no time. Consider what benefits you may reap if you do the same!

Another old blog entry: Infinite undo

2008-04-24T01:09:00.000-07:00

This one has role-playing, oh boy!

Infinite Undo

Beware of silliness. It bites.

Help me pick a blog name

2008-04-23T15:50:00.000-07:00

Scott Bellware was tweeting earlier today and caught my attention:

i really despise blogs that have titles and don't identify the author

This made me think for a second. I've used my IRC handle "aen" for so many years (13+) that I never really thought about it. While I go to no trouble to obscure my identity, I don't go to any lengths to reveal it either. A while later:

@CaffeinatedTwit a blogger's identity goes in the top left corner of the browser viewport, where everyone expects it

I checked Coding Horror, possibly my favorite blog, to verify-- yep, name is right there. You'll notice I've added it here as well. It's about time I pick a good, descriptive name. I'm looking for a blog name that fits well with my personality and motivations. Most of you who read my blog know me well enough, though I welcome input from any person-- Ideas?

More old blog entries, Part 2

2008-04-22T23:06:00.001-07:00

I just reconstructed another 4 entries. Looks like I start hitting the unit testing hard, then launch straight into SMEE (Java, FYI.) Amazing.

It's interesting to compare the style in my SMEE posts with some of the wiki pages I've been authoring at my day job. No wonder it felt natural. I've started using a (somewhat) more mature tone in this new incarnation of my blog, but skimming over some of the old entries, such as "Humble Beginnings", had me belly laughing. I realized-- charging forward with my ass on fire is pretty awesome.

Feeling overwhelmed?

2008-04-22T16:11:00.000-07:00

Feeling overwhelmed by too many posts that are too detailed? It's been mentioned to me. It's probably true, but I'm taking the Twitter attack vector, as per my rapid fire post. I also realize that the content here has been fairly remedial. It'll be a little while before I get to more complex topics. It also serves a couple purposes. I'm forcing myself into the habit of posting when something comes to me. I'm also detailing Sprytes development at a very fine grain to see if I really do think things through. You can all watch it evolve-- if you've got the stamina to keep up! After my rapid fire post, I somehow ended up reading codinghorror's post on how to become a successful blogger. One of the points was, roughly, "keep jabbing". Loved the post. There are definitely a lot of jabs coming your way. Don't worry though, with any luck there'll be some haymakers too!

Sprytes commit log feed

2008-04-22T03:17:00.000-07:00

I went on one wild and woolly adventure figuring out how to tweak Blogger's "layout tags" to get the feed for Sprytes you see on the left displaying how I wanted. Another, less woolly, adventure involved setting up the same thing on my website using the Google AJAX Feed API.

Blogger layout tags

Blogger does have a "Feed" widget by default, which you can add. But it only display's the title of the item in the feed. I wanted to display the commit logs and the date. There's an option to display dates, but that wasn't working for the Sprytes feed. Apparently it uses a "published" date by default, and my feed was using the "updated" date. Long story short: Layout > Edit HTML > Expand Widget Templates. Then edited my feed widget to look like this:

<b:widget id='Feed1' locked='false' title='Sprytes main branch' type='Feed'>
  <b:includable id='main'>
    <h2><data:title/></h2>
    <b:loop values='data:feedData.items' var='i'>
      <div class='entry'>
        <a expr:href='data:i.alternate.href'><data:i.title/></a>
        <data:i.summary/>
        <small style='color:gray;'><data:i.str_updated/></small>
      </div>
    </b:loop>
    <b:include name='quickedit'/>
  </b:includable>
</b:widget>

Wacky, man. That's all I have to say about that. And of course I styled it a little bit:

#Feed1 div, #Feed1 p {display:inline;}
#Feed1 .entry {display:block;margin:0 0 0.333em 0;}

This went in the Edit HTML inside one of the style blocks in the head. It's a little hacky. The first line is because, for some inexplicable reason, Blogger generates two nested <div> tags around a <p> for the feed "summary". I am not a fan.

Google AJAX Feed API

This was fairly straight-forward. You can see the results here. You signup here to get a key for your domain. Then you splat something like this out in your <head>:

<script type="text/javascript" src="http://www.google.com/jsapi?key=ABQIAAAA1sHFvW85UQOkGMSRRwOSExQTqjlf4jCB3EssuPDRD_oNPRkUDRQEgzkJSgggZiBks_kGEsQuq0NTog"></script>
<script type="text/javascript">
google.load("feeds", "1");
function initialize() {
  var feed = new google.feeds.Feed("http://feeds.launchpad.net/~choosetheforce/sprytes/main/branch.atom");
  feed.setNumEntries(5);
  feed.load(function(result) {
    if (result.error) return;
    var container = document.getElementById("sprytes-main");
    for (var i = 0; i < result.feed.entries.length; i++) {
      var entry = result.feed.entries[i];
      var div = document.createElement("div");
      div.innerHTML = "<div class=\"entry\"><a href=\"" + entry.link + "\">" + entry.title + "</a></strong> - " + entry.contentSnippet + "<br /><small>" + entry.publishedDate + "</small></div>";
      container.appendChild(div);
      }
    });
  }
  google.setOnLoadCallback(initialize);
</script>

The styling was along the same lines:

#sprytes-main {width:20em; float:right;}
#sprytes-main h2 {margin:0 0 0.5em 0; padding:0;}
#sprytes-main .entry {margin-bottom:0.5em;}

You can view the source of my homepage if this is getting you all hot and bothered. Admit it. You are. I know you are. No? Fine. See if I care!!!

Surfacing more activity

2008-04-21T13:47:00.000-07:00

I want to surface my source control commits here on my blog and website. I tend to commit first, let things simmer, then compose a post elaborating on my choices. Broadcasting my commits more actively is one small way to hint at what's in the pipe. I like the "most recent" tweets script I use here. My company has also started migrating over to Google Sites for our wiki, which has a navigation bar that lists most recent pages modified. Something similar for commits would be great. Now that I think about it, this is probably trivial for my Launchpad project. Commits are published as RSS, and I've actually tested it out with Google Reader. I really love the Twitter model. I find myself wanting to aggregate more and more of what I do. The ability to easily broadcast any activities I choose-- particularly those which are intrinsically digital-- to selected parties and with selective visibility is such a boon.

More old blog entries

2008-04-20T23:16:00.000-07:00

Still can't believe I was doing unit testing back then. Awesome!

My old development blog

2008-04-20T18:04:00.000-07:00

I recently hooked up an old drive and found my old development blog, from back in the days when I was using Serendipity and being hosted somewhere way less cool than asmallorange. I lost all of the post dates, but all of the posts themselves remain intact. I've also got all of the screenshots, source files, and other resources. In my spare time I'll be putting up entries on my website. Speaking of which, I have a website now!

http://choosetheforce.com

It's sparse, but contains links to everything I care about. The old development blog entries were saved in the form of an XML RSS feed, which has various bizarre tags in it. I'm reviewing each one, restructuring the HTML, and making sure all of the links work. Otherwise, the content is left untouched. I'll link to the old posts throughout the reconstruction process.

Sprytes r27-r28

2008-04-20T16:13:00.000-07:00

Links: r27, r28 Here I've begun adding test coverage. There are quite a few more dependencies than I had realized, though so far it's been proving straight-forward to put under test.

Mocking "services"

Our 3D client application at IMVU has a "services" class for certain activities that we'd prefer to be able to fake for testing purposes. That's similar to what I've done here for loading and blitting images with pygame and PIL. The idea is that a base class with some implementation of method X can be overriden in a descendant class Y, and then you pass around an instance of Y to your API while testing it. Basically, I am suppressing certain behaviors so that I can test the flow of surrounding code.

The tests

At IMVU, we keep our test coverage in separate files, which I've done here as well. I'm adopting the convention that tests for foo.py will always be in foo_tests.py. I've got two tests so far:

Test that attempting to paint a sprite that has no images raises an exception.
Test that attempting to paint a sprite that has at least one image does not raise an exception.

Next likely tests will be around verifying the mechanics of animation.

Sprytes r26

2008-04-20T15:11:00.000-07:00

Link: r26 Here I introduce a Character class which is intended to represent the idea of an in-game character that is potentially controlled by the player. Currently it contains sprites for left and right animations, digests user input, and does a couple other simple tasks.

HTML scraper for Launchpad's Bazaar repository diff view

I wanted to be able to display some of the diffs of my work from time to time for those who don't really want to be bothered to click through to the Launchpad project, so I whipped up a quick HTML scraper in PHP. I'm not sure if there are any issues with using an <iframe> for this (I know, I'm a bad man). Doesn't always seem to show up while I'm previewing this post, or even after publishing it. See! Here's a direct link to the above, if it's not showing up for you. Does it show up for you all the time? Any ideas as to why it might not always work? It's rough, but it kinda works! Not in an <iframe> on this blog, obviously. But it is theoretically awesome. It's going to need some tweaking to look good in an an RSS aggregator, once I get it showing up in here. I use cURL to get the HTML and then some regexes and replacements to extract what I want. Here's what the source looks like. I leave its comprehension to you, Dear Reader.

Sprytes r23-r25

2008-04-20T14:49:00.000-07:00

I'm eventually going to have my hulk dude walking back and forth, but before that I just wanted him to accept user input and face left or right. Usually when you want to do this sort of thing you load separate sequences of frames for facing left and right. However, a lot of the time it's "good enough" to just mirror the animation for the "other" direction. That's an option I want at my fingertips. Links: r23, r24, r25. r23 was sort of a false start. r24 and r25 are a bit more interesting.

Infinite loop vs. memory exhaustion

r25 fixes a Big problem where I ended up in an infinite loop. It was kind of funny because I was also allocating memory in the loop. Instead of locking up completely it keeled over from memory exhaustion first. I'm not sure if unit tests would have saved me here. But it made me realize I'm going to want to start writing some soon.

Sprite mirroring

At first I was going to start cramming the notion of which direction you were facing into the Sprite class. After thinking about it, however, that's silly. I want sprites to remain very simple-- they are relocatable images or animations, and that's it. I may eventually support silhouetting, tinting, or other simple transformations, but I intended to keep sprites as a sort of "base type" for whatever other higher level concepts I being articulating. With that in mind, I instead took the approach of creating a flip_sprite_horizontal() helper to clone a sprite, but flip all its images horizontally. The implementation for this mostly required adding new code, instead of doing a bunch of crazy tweaks to my existing implementation, which I like!

Sprytes r22

2008-04-20T14:36:00.000-07:00

This revision was very simple. I bring it up by itself mainly because it's an issue I'll have to solve more formally at a later date. The problem is, I gobble up 100% of CPU if I let the rendering loop run at full speed. Just pygame.time.wait()ing is a quick and dirty temporary solution. I can do with impunity because the animation code compensates as necessary. Of course, it does slow down the mouse again a little bit. There are possibly two contributing factors here:

I'm refreshing the screen way faster than the refresh rate. Any faster than that is not visually perceptible by the user, and I'm just wasting cycles.
The way the timing loop works, I'm executing the same block of code many times in order to play catch up. I could probably just make some simple calculations and do a little math, incrementing millisecond in animate() by the elapsed time, and wrapping the value as necessary with modulus or something.

In any event, it's pretty silly for such a simple app to take up so much CPU. I think that means... I need "FPS throttling"?

Sprytes r21

2008-04-20T13:32:00.000-07:00

I did a handful of refactorings in the r21 merge, which ultimately ended in moving all of the PIL-specific code for loading animated GIFs into its own module.

Getting my DVCS sea legs

I'll have to read up on Bazaar's versioning a bit more. I commit both from home and from work, and my private key is setup on both machines. Each time I push from one or the other, it seems as though the timeline of versions reported by Launchpad switches around a bit, and displays the versions from the machine I've just pushed from as the "dominant" line. Bazaar remembers all revisions and their commit logs when merging-- r21 is a merge, so you can see all of the particulars from whomever else's branch I merged from. The r21 detail on Launchpad links to r12.1.16, which provides a link to view the history of that revision. From there you can see all of the commits merged from the other branch. I suppose the mildly confusing part at first was that these are both "me". Of course, as far as commits go, they are coming from two distinct machines, so I somewhat get why it works as it does. Each of these machines has its own history of local commits, just as if I were collaborating with someone else. Still getting the hang of DVCS!

DVCS benefits

The ability to commit as frequently as I like is glorious. I don't have to have an active internet connection or any sort of connection to any other box. That is hugely beneficial. Everything is a branch, and commits are always local. The equivalent to an SVN commit to trunk, for example, would be a "push" to the central branch. You only need a connection for pushes. All of your commit history in the meanwhile is contained within the push. If I could do this in SVN at work, I would be able to reveal all of the incremental steps that I'm batching up for commits. We could still do the entire "central repository thing", but the ability to benefit from versioning at all times, and to be able to expose the full level of detail at which I (and others) are working, would be hugely beneficial for reviewing commits and root-causing issues in production.

Rapid fire

2008-04-18T18:25:00.000-07:00

I remember reading through Ayende's blog for the first time and thinking to myself "Wow, this guy is a machine!" He is prolific, but he also streams consciousness. No interesting encounter is left unblogged, however brief. Some posts are a single sentence. He uses his blog like many people use Twitter. Perhaps it's even a bit closer to Pownce-- a few words plus some screenshots or other media. I like his format. It's more... agile! I sometimes dread working on longer posts, even though I've historically tended toward them, because I somehow felt you had to share something staggering and brilliant with every post. Not necessary. It's most often the case that I'd simply like to share a quick insight or issue and move on. I want to get back to the good stuff-- coding! It seems to work pretty well for him, so I'm going to adopt the same strategy. Try it-- I suspect you will be liberated!

Sprytes, v1aen, sullyhc, and SMEE!

2008-04-17T23:56:00.000-07:00

Stuff I'm up to:

Sprytes

I've turned my pygame sprite system tinkering into a Launchpad project named Sprytes (don't laugh!) and devoted a small homepage to it. My future posts on Sprytes will be shorter, and probably refer largely to a specific revision range. Bazaar is pretty cool. I recently saw the Linus talk about git, which got me excited about DVCS. But git is hardcore! Bazaar is a nice median road for me. Sprytes satisfies my Python urges.

v1aen & sullyhc

I also have a Google Code project named v1aen that I've neglected recently but will be picking up again. v1aen is a rewrite of the VERGE1 codebase with test coverage (using UnitTest++). There's also a sullyhc project in the same repository that is a rewrite of the VERGE pack-in demo "Sully" in "hardcoded" form. This will be the first in a series of hardcoded games that I will develop in a test-driven fashion to help me determine a core set of functionalities for a new game creation system that is still in the planning stages. These two projects satisfy my C++ (and STL and boost) urges.

SMEE

Then there's SMEE (Simplest Map Editor Ever), which I had been developing on my old blog and which I'll also be resurrecting soon. I'll be putting up my old development blog entries somewhere as well. SMEE is a map editor for VERGE1 maps (and will probably grow to handle VERGE2 and VERGE3). I have this checked into a Subversion repository on a co-located server I share with friends. SMEE satisfies my Java (and Swing) urges. This may seem like a lot of different directions, but I assure you there's method to my madness!

Seventh pygame: Die, fluctuating border!

2008-04-16T01:31:00.000-07:00

Full source here. Last time I vowed to get rid of the ugly border around my GIF animation code. It's the only thing holding this back from a being a decent GIF animation loader. Why did that ugly border even exist around my animation? I tried loading a bunch of different GIF animations and found that it wasn't that uncommon. Some of them had huge borders. I rooted around in the PIL documentation a bit. This page has an example with a "tile" property. It also mentioned a GIF file's "logical screen size". Wasn't sure what that was, so I took a look at the GIF specification.

18. Logical Screen Descriptor.

      a. Description.  The Logical Screen Descriptor contains the parameters
      necessary to define the area of the display device within which the
      images will be rendered.  The coordinates in this block are given with
      respect to the top-left corner of the virtual screen; they do not
      necessarily refer to absolute coordinates on the display device.  This
      implies that they could refer to window coordinates in a window-based
      environment or printer coordinates when a printer is used.

The GIF spec has a section titled the "Logical Screen Descriptor". What it boils down to is animated GIFs are encoded as if there were some virtual screen size in which the animation was taking place. Usually it's the smallest possible dimensions that can "hold" all of the frames, but that's not always true. For example, the logical screen size could be 100 pixels wide and 300 pixels tall. The actual animation might only be taking place in the top 200 pixels, and the bottom 100 will just be entirely "padding". The important bit of information is that each frame in the animated sequence has a descriptor which indicates the "window" for that frame. From what I can tell, this is almost always the minimum possible rectangle within the logical screen size (ie. within the GIF dimensions) that holds the image data for that frame. If you take a look in the "Image Descriptor" part of the GIF spec, you'll see that it keeps track of each image's upper left position and size.

20. Image Descriptor.

      ...

      The Image Descriptor contains the parameters necessary to process a table
      based image. The coordinates given in this block refer to coordinates
      within the Logical Screen, and are given in pixels.

      ...

      7 6 5 4 3 2 1 0        Field Name                    Type
     +---------------+
  0  |               |       Image Separator               Byte
     +---------------+
  1  |               |       Image Left Position           Unsigned
     +-             -+
  2  |               |
     +---------------+
  3  |               |       Image Top Position            Unsigned
     +-             -+
  4  |               |
     +---------------+
  5  |               |       Image Width                   Unsigned
     +-             -+
  6  |               |
     +---------------+
  7  |               |       Image Height                  Unsigned
     +-             -+
  8  |               |
     +---------------+
  9  | | | |   |     |                      See below
     +---------------+

After reading all of that, the example on the PIL documentation page made a bit more sense. The PIL example was this:

if im.tile[0][0] == "gif":
    # only read the first "local image" from this GIF file
    tag, (x0, y0, x1, y1), offset, extra = im.tile[0]
    im.size = x1-x0, y1-y0
    im.tile = [(tag, (0, 0) + im.size, offset, extra)]

The x0, y0, x1, and y1 fields are the useful part. All I needed to do then was to figure out how to extract only that portion of each image and copy only that part into each image that I put into my sprites. Here's what I came up with:

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        sprite.time_between_frames = pil_image.info['duration']
        palette = to_triplets(pil_image.getpalette())
        try:
            while 1:
                (x0, y0, x1, y1) = (0, 0) + pil_image.size
                if len(pil_image.tile) > 0:
                    (x0, y0, x1, y1) = pil_image.tile[0][1]
                image = pygame.image.fromstring(pil_image.tostring(), pil_image.size, 'P')
                image.set_palette(palette)
                image.set_colorkey(pil_image.info['transparency'])
                new_image = pygame.Surface(pil_image.size, pygame.SRCALPHA)
                new_image.blit(image, (x0, y0), (x0, y0, x1 - x0, y1 - y0))
                sprite.images.append(new_image)
                pil_image.seek(pil_image.tell() + 1)
        except EOFError:
            pass # end of sequence
    else:
        sprite.images.append(pygame.image.load(image_filename))
    return sprite

I basically load each image normally, but then create an entirely new image and copy the "window" region for each image into it. That entirely new image is added to the sprite. My original approach was to write a function that just took the image I loaded and cleared all the pixels outside the window to be transparent. I didn't like it very much though, since it was per-pixel. I could have also just used fill() to splat 4 rectangles of transparency around the edges, but that seemed like an awful lot of shenanigans. I opted for the simplest approach in the end. And that's all she wrote! GIF animation loader: case closed. I'll be iterating a bit more rapidly from here on out, in a slightly different format. It'll contain all the cream-filled goodness, but less of the banter. It'll be awesome, you'll see!

Sixth pygame: Refactoring FTW

2008-04-13T20:39:00.000-07:00

Full source here. There's a few things I want to clean up with the code I've written so far.

Consistent separation of concerns

First, I've separated the responsibilities of animation and rendering in the Sprite class, but they're still tied together in the Scene class. I'll split that out for consistency, and because I don't want to neglect SoC.

class Scene:
    ...
    def paint(self):
        for sprite in self.sprites:
            sprite.paint()
        pygame.display.flip()
    def animate(self):
        for sprite in self.sprites:
            sprite.animate()

...

while not should_quit():
    scene.paint()
    scene.animate()
    player.position = pygame.mouse.get_pos()

Distribution of complexity

The other thing which is bothering me, and may be somewhat stylistic, is that I'm calculating the elapsed time since the last call to animate() for every sprite. I'll feel better moving that calculating up into the scene. I can calculate the elapsed time once, then call all of the sprite animate() methods the same number of times:

class Sprite:
    def __init__(self):
        self.position = (0, 0)
        self.images = []
        self.current_image = 0
        self.time_between_frames = 0
        self.millisecond = 0
    ...
    def animate(self):
        self.millisecond += 1
        if  self.millisecond == self.time_between_frames:
            self.millisecond = 0
            self.current_image += 1
            self.current_image %= len(self.images)

class Scene:
    def __init__(self, sprites):
        self.sprites = sprites
        self.last_animated = 0
    def paint(self):
        for sprite in self.sprites:
            sprite.paint()
        pygame.display.flip()
    def animate(self):
        milliseconds_elapsed = pygame.time.get_ticks() - self.last_animated
        for real_millisecond in range(milliseconds_elapsed):
            for sprite in self.sprites:
                sprite.animate()
        self.last_animated = pygame.time.get_ticks()

Of course, this has the natural consequence that if I don't use a scene object and instead work with individual sprites, then I'll have to manage real time myself, all over again. I can live with this choice, because I do not ever intend for sprites to be manipulated outside of a scene. I want the scene to contain all of the more "complex" knowledge, such as the particulars of double-buffering or page flipping, and how to sync animation properly with real time. Another partial reason is to help distribute the complexity of my system. The Scene class was pretty spare, which is fine, but if I can think of a good reason to remove some complexity from one part of my system and deposit that it into another part, I achieve two things:

I reduce the complexity of one part, making that part easier to understand.
I give another part of my system more of a reason to exist. Sometimes when you have skeleton classes that do very little, it's a code smell that's telling you the class may not need to exist at all.

I do not recommend juggling code between classes for the mere sake of maintaining some sort of code equilibrium by volume. However, it is possible to do so advantageously. Occasionally take a step back and check your assumptions. Consider the overall design. Does it still make as much sense as when you first started? Could its clarity be improved? Could it be simplified? Refactoring code is crucial to maintaining a lean and understandable codebase over time.

Removing duplication

There is a small amount of duplication here in the scene's animate() method. Can you see it?

def animate(self):
    milliseconds_elapsed = pygame.time.get_ticks() - self.last_animated
    for real_millisecond in range(milliseconds_elapsed):
        for sprite in self.sprites:
            sprite.animate()
    self.last_animated = pygame.time.get_ticks()

I am calling pygame.time.get_ticks() twice here when I don't really need to be. This is an exceedingly minor quibble, but my animation is also slightly less accurate because of it. It is trivial to remove, so I will remove it. If you think about it, because I am calling pygame.time.get_ticks() twice, all of the code in between those two calls is unaccounted for when determining how many milliseconds have elapsed since the last call. As the number of sprites grow and the inner loop takes more time, my animations will become more an more inaccurate (slower). It would probably take a large number of sprites to cause this to be noticeable, but again, it depends upon the machine. I can alleviate this duplication and remove this sliding scale of inaccuracy by recording the current time once and then reusing it:

def animate(self):
    now = pygame.time.get_ticks()
    milliseconds_elapsed = now - self.last_animated
    for real_millisecond in range(milliseconds_elapsed):
        for sprite in self.sprites:
            sprite.animate()
    self.last_animated = now

Cool beans. Everything is still running smoothly and behaving as I expect it to. Next time, I'm finally going to get rid of that annoying border around my animation!

Fifth pygame: Better animation

2008-04-12T02:52:00.000-07:00

Full source here. You need Python, pygame, and PIL to run this. The images are here. In my fourth pygame, I discovered that pygame.time.wait() isn't such a great approach to animation. Maybe it would be if no interaction with the user was necessary and I was only animating one sprite. My goals here are a little loftier than that. I had taken a step back from the problem and roughed out a simple conceptual model for how I thought the mechanics of animation should work:

millisecond = 0
duration = 100
while 1:
    millisecond += 1
    if  millisecond == duration:
        sprite.animate()
        millisecond = 0

Realizing my model

I want this "ideal" model surfaced in my code, so a good place to start as any would be to define the animate() method that is being hinted at. This new method should advance to the next image in the animation. I already have code for that inside paint(), so I'll simply move it over. I'll also remove the call to pygame.time.wait(), since my goal is to replace that with a better method of animation that doesn't pause the entire program while waiting between frames.

class Sprite:
    ...
    def paint(self):
        screen.blit(self.images[self.current_image], self.position)
    def animate(self):
        self.current_image += 1
        self.current_image %= len(self.images)

If I run this, my animation stops working! Well, obviously. I moved the animation code into a new method which I'm not even calling yet. It will be easiest to call it inside the scene rendering loop:

class Scene:
    ...
    def paint(self):
        for sprite in self.sprites:
            sprite.paint()
            sprite.animate()
        pygame.display.flip()

Now the animation is working again, although I'm back to hyperactive spasms that are a blur of motion because it's animating so quickly.

Think about it. Think, think about it.^[1]

When I think about what it means to animate a sprite, it seems as though the timing code should be part of that. Rendering the sprite is one responsibility, and it makes sense that blitting the sprite's images to the display is all that is required to fulfill that responsibility. Animating the sprite is another responsibility. I think it makes sense that this should include both the notion of moving to the next frame and deciding when to move to the next frame. This means I should probably pull some more of the code from my conceptual model into animate(). My model code loops infinitely. Clearly, I don't want to pull an infinite loop in. However, it makes sense that the animate() method is going to be called in a loop in order to animate my sprites. And in fact, I've already placed animate() inside the scene rendering loop with my most recent change. The meaty part of my conceptual model is the part inside the loop where I'm incrementing milliseconds, so I'm going to pull that into animate(). I'm kind of turning my model inside-out, but that's okay. I'm reasoning out my decisions, and this feels like the logical choice. I'll make millisecond one of the data members of my sprite class so I can keep track of the current millisecond count between each call. The duration is already being tracked-- it's called time_between_frames. I think that name is a little more descriptive than just "duration", so I'll stick with it.

class Sprite:
    def __init__(self):
        ...
        self.time_between_frames = 0
        self.millisecond = 0
    ...
    def animate(self):
        self.millisecond += 1
        if  self.millisecond == self.time_between_frames:
            self.millisecond = 0
            self.current_image += 1
            self.current_image %= len(self.images)

With this small change, I can move my mouse around smoothly once more! But... the animation is incredibly slow.

Syncing with real time

The problem here is that I'm not acknowledging real time in my code yet-- only indirectly with clever variable names. As a result, my animate() method could end up being called many times a millisecond. It could be called only once every few milliseconds. It's going to run at different speeds on different machines, and will be entirely dependent upon the CPU and GPU. It's time to review the pygame time documentation again. What are my options? They are:

pygame.time.get_ticks - get the time in milliseconds
pygame.time.wait - pause the program for an amount of time
pygame.time.delay - pause the program for an amount of time
pygame.time.set_timer - repeatedly create an event on the event queue
pygame.time.Clock - create an object to help track time.

I've been using pygame.time.wait, and that didn't pan out, so I doubt pygame.time.delay is the answer. It does the same thing but with more accuracy. In my previous post, I had described what I needed like so:

I need to keep track of what time the current frame was initially displayed. Then I can compare that against the current time on an ongoing basis. When the difference between the two becomes equal to the duration, then I can move to the next frame.

Of all the above options, the first seems to be the best fit. I want milliseconds, you've got milliseconds... let's make a deal! As for the other two: pygame.time.set_timer creates "an event" on the event queue, and I have no idea what that is. Don't want to bother finding out. pygame.time.Clock creates an object to help track time... but if I can already get the time in milliseconds directly from from pygame.time.get_ticks(), there's no reason for me to use some specialized "time object". There's no reason for me to use object-oriented concepts here. Back to the problem at hand, I've got myself an animate() method which is keeping track of "imaginary milliseconds". I need to make those Real Life Milliseconds.

Bridging the gap

If it is true that different code runs at different speeds on different machines, then what I need to be able to do is determine how much time has actually elapsed between calls to animate(). If I can figure that out, I can speed up or slow down my imaginary milliseconds to match. To track how many milliseconds have elapsed since the last call to animate(), I'll add a "last animated" field to my sprite class, and start recording that at the end of the method:

class Sprite:
    def __init__(self):
        ...
        self.last_animated = 0
    ...
    def animate(self):
        self.millisecond += 1
        if  self.millisecond == self.time_between_frames:
            self.millisecond = 0
            self.current_image += 1
            self.current_image %= len(self.images)
        self.last_animated = pygame.time.get_ticks()

Now that I know when this method was last called, I need to calculate how much time has elapsed at the top, so that I can compensate as necessary:

def animate(self):
    milliseconds_elapsed = pygame.time.get_ticks() - self.last_animated
    ...
    self.last_animated = pygame.time.get_ticks()

All I'm doing here is getting the difference between the current time and the last time this method was called. Easy peasy^[2]. Now that I know how many milliseconds have elapsed since last time, all I need to do is wrap most of the code in a loop. The code is incrementing one imaginary millisecond at a time, so it makes sense that I just have to make it increment my imaginary milliseconds enough to match how many milliseconds have actually elapsed:

def animate(self):
    milliseconds_elapsed = pygame.time.get_ticks() - self.last_animated
    for real_millisecond in range(milliseconds_elapsed):
        self.millisecond += 1
        if  self.millisecond == self.time_between_frames:
            self.millisecond = 0
            self.current_image += 1
            self.current_image %= len(self.images)
    self.last_animated = pygame.time.get_ticks()

I ran this and-- by God-- the results ain't too shabby!^[3] The hulk is animating at a reasonable speed and I can move him around with my mouse very smoothly. Kick-ass!

Retrospective

I didn't expect decent timing code to take two full posts. Time can be a tough subject, if you really buckle down and try to articulate and reason out your decisions and strategies. I have a lot of historical familiarity with timing code, but forcing myself to explain why I was making various choices was an interesting an educational exercise. Over time it is not uncommon to simply memorize patterns of code, without an intuitive understanding or deeper knowledge of how or why it works. Making baby steps as I have here, and forcing myself to make logical choices that flow from one to the other challenged me. In many ways I was verifying that I knew what I thought I knew, you know? So! I've come out the other end with increased clarity and more L. There are plenty of directions I could go with my next pygame. We'll see what I can cook up next time! Removing that annoying border would be a start... [1] So good. [2] Pumpkin pie! [3] This actually didn't work initially. After reviewing the pygame.time.get_ticks() description, I realized it counts the milliseconds since pygame.init() is called-- and I wasn't calling that! I've seen many pygame.init() calls in other tutorials, but I like that I discovered an actualy criteria that requires you call it. This has been included in the full source link provided.

Fourth pygame: Animation + Timing = Shenanigans!

2008-04-11T03:59:00.000-07:00

Full source here. Picking up from last time, I'm going to need a way to store multiple images if I want to read in all of the frames from a GIF animation. The simplest thing I can think of to accomplish this is to turn Sprite's self.image into a list of images. I'll also move all of the member data initialization into the ctors. I want this initialization to take place for every instance.^[1]

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        ...
        image = pygame.image.fromstring(pil_image.tostring(), pil_image.size, 'P')
        image.set_palette(palette)
        image.set_colorkey(pil_image.info['transparency'])
        sprite.images.append(image)
    else:
        sprite.images.append(pygame.image.load(image_filename))
    return sprite

class Sprite:
    def __init__(self):
        self.position = (0, 0)
        self.images = []
    def paint(self):
        screen.blit(self.images[0], self.position)

class Scene:
    def __init__(self, sprites):
        self.sprites = sprites
    ...

The main thing I've actually changed here is reworking make_sprite() and Sprite to "play nice" with an image list instead of a single image. This is most evident in the paint() method where I simply blit self.images[0]. I'm just "priming the pump" for the change I really want: animation! I have two simple objectives right now:

Grab all of the images inside a GIF and add them to my sprite.
Loop through those images when drawing my sprite!

Dance, pilgrim!

I consult briefly with the PIL tutorial on image sequences and take a crack at it:

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        palette = to_triplets(pil_image.getpalette())
        try:
            while 1:
                image = pygame.image.fromstring(pil_image.tostring(), pil_image.size, 'P')
                image.set_palette(palette)
                image.set_colorkey(pil_image.info['transparency'])
                sprite.images.append(image)
                pil_image.seek(pil_image.tell() + 1)
        except EOFError:
            pass # end of sequence
    else:
        sprite.images.append(pygame.image.load(image_filename))
    return sprite

class Sprite:
    def __init__(self):
        self.position = (0, 0)
        self.images = []
        self.current_image = 0
    def paint(self):
        screen.blit(self.images[self.current_image], self.position)
        self.current_image += 1
        self.current_image %= len(self.images)

The changes are actually quite small. I'm just wrapping the PIL image loading code with a loop and adding a "current image" member to the Sprite class that increments every time paint() is called. You can't get much simpler than that! The modulus here

self.current_image %= len(self.images)

keeps the current frame wrapping within the correct range. If my GIF has 10 images, this loops from 0 to 9 over and over. There is an interesting tangent I could go off on here in regards to error handling, but I'll save that for another post. I run these changes. It kinda works! Never mind that the hulk is having a 100 mile a minute seizure and there's some weird fluctuating border around the edges-- I've achieved my objectives! I've extracted all of the images from the GIF and I'm looping through them when drawing my sprite.

Slow down! You move too fast.^[2]

The weird border is one problem, but I'll put that aside for the moment. I'm mostly happy I've managed to load all of the frames. First I'll focus on slowing down the animation to its intended pace. Then I can tackle that funky border. The GIF already "knows" how quickly the frames in the animation should be cycling. As with the transparency index, PIL makes this information available through the info properties. The property of interest here is the duration. The duration is specified in milliseconds. This means that in my code I want to switch to the next image in my sprite every X milliseconds, where X is the duration. If we take a look at the pygame documentation, there are several options. After briefly examining these options, pygame.time.wait() or pygame.time.delay() seem like the no-brainers. All I want to do is wait for a certain amount of time before moving to the next frame. I'll need to grab the duration property (i.e. the "animation speed") from the GIF first. Then I should be able to wait for the required amount of time after displaying each frame. pygame.time.wait() takes the number of milliseconds to wait as the parameter. The GIF duration property is described as the "Time between frames in an animation (in milliseconds)". These two are a match made in heaven! Here's an attempt at throwing them together:

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        sprite.time_between_frames = pil_image.info['duration']
        ...
    else:
        sprite.images.append(pygame.image.load(image_filename))
    return sprite

class Sprite:
    def __init__(self):
        self.position = (0, 0)
        self.images = []
        self.current_image = 0
        self.time_between_frames = 0
    def paint(self):
        screen.blit(self.images[self.current_image], self.position)
        pygame.time.wait(self.time_between_frames)
        self.current_image += 1
        self.current_image %= len(self.images)

Running this has a pretty nice payoff: it works!

Slugging it out

Moving the mouse around the window proves to be a little disappointing, however. The movement is sluggish! Before, it was animating like crazy and the mouse movement was very smooth. Why is it so slow now? You may be able to intuit the answer if you think about it for a moment. I'm not always that quick on my toes, so many times in situations like this I will simply begin plugging different values into my code to observe what happens. This helps me get a better "feel" for what going on. For example, what if we replace the animation speed with a literal value instead of loading it from the GIF?

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        sprite.animation_speed = 50
        ...

If you run this, you'll notice the animation becomes faster and the mouse movement is a bit smoother. It's still a little "rough", but it's definitely smoother than before. The sluggishness of the mouse and the how long you wait between frames seem to be related! Here's a riddle. What do you think would happen if I had two hulks animating at the same time? Or three? Try it.

player = make_sprite('hulk.gif')
clone_billy = make_sprite('hulk.gif')
clone_bob = make_sprite('hulk.gif')
scene = Scene([background, player, clone_billy, clone_bob])

This makes the animations quite a bit slower! And the mouse is back to being very sluggish! Perhaps not what you'd expect. Referring back to the pygame documentation, the description for pygame.time.wait() reads: pause the program for an amount of time. While animating a single sprite, this pauses the entire program! Clearly I don't want this to be happening for every single sprite. Imagine I'm animating 10 hulks, all at the same speed. They should cycle through their frames in unison. However, I am pausing the entire program for the full duration every time paint() is called on each sprite. Instead of pausing once up front and then moving all 10 sprites ahead one frame, I am pausing 10 times in a row! I could move the waiting outside of the sprite's paint() method and to the end of the scene's paint() method, which would solve the problem of everything becoming slower and slower as more sprites are added, but then I would still have the problem where slower animations make the mouse more sluggish. What I need here is a way to move to the next frame in an animation at the right time without pausing the entire program. I need to keep track of what time the current frame was initially displayed. Then I can compare that against the current time on an ongoing basis. When the difference between the two becomes equal to the duration, then I can move to the next frame.

Time to regroup. Evasive mental maneuvers!

Let's pretend I'm not even talking about time here for a moment. Milliseconds shmilliseconds. Not important. Conceptually, all I want is to perform an action after accumulating some sort of critical mass. Consider the following code:

accumulator = 0
critical_mass = 100
while 1:
    accumulator += 1
    if  accumulator == critical_mass:
        do_something()
        accumulator = 0

I realize how remedial this is, but let's step through it. This code loops infinitely. Before the loop it sets up an accumulator and a "critical mass". Each time through the loop the accumulator is incremented. Every time the critical mass is reached, I call do_something() and reset the accumulator. What if I modify the code slightly:

millisecond = 0
duration = 100
while 1:
    millisecond += 1
    if  millisecond == duration:
        sprite.animate()
        millisecond = 0

Same code, different names. Soak this in for a second. It's a pretty good conceptual model for my intention, which is to animate a sprite every time X milliseconds have elapsed. In this case, every 100 milliseconds.

Conceptual model seeks concrete implementation.

Roughing out plans as above can allow you to make more intelligent choices about your needs. The verbiage above is already telling me that it is natural to separate the code for animation-- animate()-- from the code for rendering-- paint(). Next time, I'll take a closer look at the pygame.time documentation and suss out a better solution to this timing dilemma by using my conceptual model as a guide. The question is: How will I make millisecond in my conceptual model reflect real time? [1] I was previously declaring these variables in class scope, which results in their values only being initialized once when the class definition is loaded. Subsequent instances would receive whatever values had accumulated from previous instances -- not what I want! [2] You got to make the morning last.

What makes Python beautiful to me

2008-04-08T02:18:00.000-07:00

I was trying to sleep, but my mind started wandering and I began thinking about what makes Python beautiful to me. A recent post by codinghorror made observations about the book "Beautiful Code". Code isn't beautiful, he realized, but rather it is the "architecture, the ideas, the grander algorithms and strategies that code represents." Lying face-down on my pillow, I found myself nodding my head in agreement just like the first time I read his words. My mind lingered briefly on convention over configuration. There are several conventions in Python syntax that I enjoy because they feel balanced and reasoned. Here are a few of my favorite things:

The reliance on indentation for logical grouping requires less code, so there is that low-level simplicity of expression. If there's anything I find beautiful in code, it's simplicity of expression.
There is no ternary operator, as in C++. This requires more code to express certain ideas. But on the flip side, I feel this keeps with simplicity of expression by decreasing the number of ways in which it is possible to express those same ideas.
The same simplicity in limited options is present with the assignment operators such as += and -=. There are no ++ or -- operators in Python, nor the subtleties that come with them. Post- and prefix increment and decrement can produce physically smaller code, but the more "spelled out" nature of the code you are forced to write in their absence fosters readability, IMO.

That's my short list. They are among the least "fancy" tools present in Python's repertoire, but to me these fundamental choices are the most beautiful. List comprehensions are something I am only recently growing to fully appreciate. What about code do you find most beautiful?

Third pygame: Hulk want animation!

2008-04-07T22:01:00.000-07:00

Full source here. Shortly after my last post, I found discovered that pygame had its own default sprite system. However, it left me with a bad taste in my mouth. It's not all that bad a system, but when I discovered it I already had some simply animation code working and many sprites onscreen. pygame's sprite system by default does not support ordering of sprites. I would have at least expected the painter's algorithm to be the default. I could have tried to fix the lack of ordering in pygame's system with more code -- I even found some examples -- but in the end it felt wrong. pygame's sprite system was getting in my way. My system uses the painter's algorithm and is extremely simple, so I decided to continue cultivating it. The mouse cursor being displayed alongside my image as I move it around on the screen isn't very pretty, so I remove it:

screen = pygame.display.set_mode((640, 480))
pygame.mouse.set_visible(False)

I'm also tired of clicking on the window's X button to close my app:

def should_quit():
    event = pygame.event.poll()
    return event.type == pygame.QUIT or event.type == pygame.KEYDOWN

This makes it exit on any key press. That'll work out fine while I'm tinkering. It wasn't long before I started thinking about animation. I was surfing around on the web looking for cool free sprites, and noticed a lot of animated GIF images I wish I could use easily. For example, I found this guy: Being able to quickly plug in cool stuff like this without a bunch of extra work slicing up images and setting timings would be awesome. FYI, I have no idea if this animation is public domain or not. I will take it down if anybody asks me to. I found a simple background too. This one I know is public domain. I found it in the Creative Commons on Flickr: You can click on the image to see who deserves the credit. Made a few tweaks to my code for this stuff:

screen = pygame.display.set_mode((500, 375))
pygame.mouse.set_visible(False)

background = make_sprite_from_image('red-forest.jpg')
player = make_sprite_from_image('hulk.gif')
scene = Scene([background, player])

while not should_quit():
    scene.paint()
    player.position = pygame.mouse.get_pos()

I figure I might as well start referring to the sprite I'm controlling as the player. Unfortunately, turns out pygame won't animate it for me afterall! Bummer. So I've got to do the animating myself. pygame doesn't seem to actually load all of the frames in an animated GIF, so after a little Googling I found the Python Imaging Library (PIL). After consulting the first example in the PIL tutorial, I made a few tweaks in a naive attempt to plug it right into my code:

import pygame
import Image

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        sprite.image = Image.open(image_filename)
    else:
        sprite.image = pygame.image.load(image_filename)
    return sprite

I shortened the function name down to just make_sprite() because it's pretty obvious by looking at calls to it that it's making a sprite from an image. I don't need to keep saying make_sprite_from_image(). I made the call to the PIL code a special case only for files with the GIF extension because I just want to test the waters in a specific case with this new library. If I discover it's the best thing since sliced bread I can later just replace the entirety of make_sprite() with pure PIL image loading. So, awesome thing here is that this code will die with a stack trace! Not quite what I was hoping for. Image.open() returns a PIL-specific flavor of image object which is not compatible with pygame and pygame's screen.blit() function. I had to figure out how to convert PIL images into pygame images. After inspecting the PIL Image class documentation for a little while, and then some more wild Google searching, I found a thread with enough to get me going. This much got rid of the stack trace, but I was getting a black image:

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        sprite.image = pygame.image.fromstring(pil_image.tostring(), pil_image.size, 'P')
    else:
        sprite.image = pygame.image.load(image_filename)
    return sprite

More searching revealed my problem. Apparently tostring() sets a blank palette (a.k.a. "colormap") by default. Yet more searching clued me in on the getpalette() method of PIL image instances, which is conspiciously absent from PIL's Image module documentation. On the pygame side of things, its documentation describes a set_palette() function for setting the palette of an image. The only fly in the ointment is that PIL's getpalette() returns the palette in a flat 3*256 byte list, whereas pygame's set_palette() takes a 256 item list of RGB triplets. I wrote a quick converter and plugged it in:

def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        sprite.image = pygame.image.fromstring(pil_image.tostring(), pil_image.size, 'P')
        pil_palette = pil_image.getpalette()
        palette = []
        for i in range(0, len(pil_palette), 3):
            rgb = pil_palette[i:i+3]
            palette.append(rgb)
        sprite.image.set_palette(palette)
    else:
        sprite.image = pygame.image.load(image_filename)
    return sprite

Much to my surprise, it worked. Of course, the transparency didn't seem to be working. Some more quick searching revealed I had to specify which color index should be used for transparency. It's usually index 0, so I tried that:

sprite.image.set_palette(palette)
sprite.image.set_colorkey(0)

Woops, my shaggy mane disappeared! Ah, yes. GIF images can use any index they want for transparency. I'm not sure what the reasoning is, but that's the way it works. Turns out to be a pretty easy fix:

sprite.image.set_colorkey(pil_image.info['transparency'])

I've been doing a whole lot of RTFM tonight. Who knew that sort of thing works? I don't like that crazy palette conversion loop stuffed in there though. I pulled it out into its own function and made it a little more compact:

def to_triplets(pil_palette):
    palette = []
    for i in range(0, len(pil_palette), 3):
        palette.append(pil_palette[i:i+3])
    return palette
    
def make_sprite(image_filename):
    sprite = Sprite()
    if image_filename.endswith(".gif"):
        pil_image = Image.open(image_filename)
        sprite.image = pygame.image.fromstring(pil_image.tostring(), pil_image.size, 'P')
        sprite.image.set_palette(to_triplets(pil_image.getpalette()))
        sprite.image.set_colorkey(pil_image.info['transparency'])
    else:
        sprite.image = pygame.image.load(image_filename)
    return sprite

After consulting a friend, I went one better: I compressed the loop down into a tidy list comprehension. Thanks dude!

def to_triplets(pil_palette):
    return [pil_palette[i:i+3] for i in range(0, len(pil_palette), 3)]

I'm pretty satisfied with that. Now that I've got my snazzy jolly green giant that I can move around over my snazzy red forest background, I just need to extract his animating goodness! This page reveals a little bit about handling GIF images in PIL, and this section of the PIL tutorial reveals even more about image "sequences". Shenanigans are afoot! I'll stop here before things get too crazy. Bonus points if you can figure out how to extract all the frames from the GIF using PIL before my next post!

Second pygame: a sprite system

2008-04-05T20:34:00.000-07:00

I started learning how to display images with pygame. I snooped around a little and came up with this:

import pygame

def should_quit():
    event = pygame.event.poll()
    return event.type == pygame.QUIT
    
screen = pygame.display.set_mode((640, 480))
sprite = pygame.image.load('gamer.jpg')

while not should_quit():
    screen.blit(sprite, (0, 0))
    pygame.display.flip()

The image is a huge image of me that fills the entire window, because I'm vain like that. I want to create a sprite system where the specifics of how and where things are blitted is taken care of behind the scenes. I don't want to leave those specifics exposed, so I abstracted that stuff away into a Sprite class:

...

class Sprite:
    def paint(self):
        screen.blit(self.image, (0, 0))

screen = pygame.display.set_mode((640, 480))
sprite = Sprite()
sprite.image = pygame.image.load('gamer.jpg')

while not should_quit():
    sprite.paint()
    pygame.display.flip()

Now I have an awesome self.image. Ba-dum bum. I don't want to have to remember to assign to sprite.image every time I load an image into a sprite, so I moved that into the sprite's constructor:

...

class Sprite:
    def __init__(self, image_filename):
        self.image = pygame.image.load(image_filename)
    def paint(self):
        screen.blit(self.image, (0, 0))

screen = pygame.display.set_mode((640, 480))
sprite = Sprite('gamer.jpg')

...

This is a little bit simpler, but now I've made my Sprite class dependent on pygame.image.load(). I don't really want the concept of resource management stuffed in there, it muddies my definition. I want that responsibility handled elsewhere, so I moved it out into its own helper function:

...

def make_sprite_from_image(image_filename):
    sprite = Sprite()
    sprite.image = pygame.image.load(image_filename)
    return sprite
        
...

screen = pygame.display.set_mode((640, 480))
sprite = make_sprite_from_image('gamer.jpg')

...

I like this a lot better. Now I wanna display another sprite on top of this bad boy:

...

screen = pygame.display.set_mode((640, 480))
background = make_sprite_from_image('gamer.jpg')
axe = make_sprite_from_image('axe.png')

while not should_quit():
    background.paint()
    axe.paint()
    pygame.display.flip()

Not bad. I grabbed a little axe dude image from the interwebs and he's displaying on top of my background with proper transparency and everything. I want to make it a little more interesting by moving my axe dude around with the mouse. This means I need to introduce the concept of positioning somehow. It's pretty easy to add that to my definition of sprites:

...
        
class Sprite:
    position = (0, 0)
    def paint(self):
        screen.blit(self.image, self.position)

...

while not should_quit():
    background.paint()
    axe.paint()
    axe.position = pygame.mouse.get_pos()
    pygame.display.flip()

I friggin' love Python syntax. It's pretty close to how I pseudocode, though knowledge of Python in the first place influenced my examples in prior posts. Now I can swizzle my mizzle around the wizzle and stand, mouth agizzle at my ingenizzle! Mouse over any confounding izzles for enlightenmizzle. What I don't like here though, is that I'm starting to mix rendering and the logic for movement together. I could move my movement logic below the rendering and put a space in there to separate the two groups, true. But in particular I want to abstract away all the details of rendering, as I did with my Sprite class. Mr. pygame.display.flip() is one of those details, and it's staring me down! What I'm wanting is the ability to dump all my sprites into a black box and have it figure out how to render everything and display it. I think I'll make a Scene. Ba-dum bum! Someone stop me, I'm too good at this.

...

class Scene:
    sprites = []
    def paint(self):
        for sprite in self.sprites:
            sprite.paint()
        pygame.display.flip()

...

background = make_sprite_from_image('gamer.jpg')
axe = make_sprite_from_image('axe.png')
scene = Scene()
scene.sprites.append(background)
scene.sprites.append(axe)

while not should_quit():
    scene.paint()
    axe.position = pygame.mouse.get_pos()

This is nice, but I don't like all that sprites.append() BS. It's exposing the implementation details of my Scene. Exposing these details means this code is dependent on scene.sprites being implemented as a list. What if I want to change that later? What if I decide I want the Scene to specially manage the ordering of its sprites, instead of always tacking sprites onto the end of the list? I'd have to revisit all of he code that was adding sprites like this and update it. I want an explicit interface for adding sprites to my scene:

...

class Scene:
    sprites = []
    def paint(self):
        for sprite in self.sprites:
            sprite.paint()
        pygame.display.flip()
    def add_sprites(self, sprites):
        self.sprites += sprites

...

background = make_sprite_from_image('gamer.jpg')
axe = make_sprite_from_image('axe.png')
scene = Scene()
scene.add_sprites([background, axe])

while not should_quit():
    scene.paint()
    axe.position = pygame.mouse.get_pos()

Better, but I already know what sprites I want in my scene at the time I'm creating it. I want to toss them all in right when I declare the scene. A constructor achieves that easily enough:

...

class Scene:
    sprites = []
    def __init__(self, sprites):
        self.add_sprites(sprites)
    ...
    def add_sprites(self, sprites):
        self.sprites += sprites

...

background = make_sprite_from_image('gamer.jpg')
axe = make_sprite_from_image('axe.png')
scene = Scene([background, axe])

...

I don't really need add_sprites() now, though I could see the need for it in the future. I'll let that future need determine when I add it back in though. For now I'm ripping it out and keeping things simple:

...

class Scene:
    sprites = []
    def __init__(self, sprites):
        self.sprites += sprites
    def paint(self):
        for sprite in self.sprites:
            sprite.paint()
        pygame.display.flip()

...

10 iterations will do for now! Good times. I haven't spent much time with Python in months. Learning more about it with pygame is The Fun Way. Look at me, flingin' sprites around and jazz. I'm not test-driving my code, but it's definitely need-driven. How XP of me! I like to keep it real, you know? Real lean! Ba-dum-bum-bum *ching* Thank youuu, and good night!

First pygame program

2008-04-05T19:11:00.000-07:00

I started playing around with pygame today. I googled for "pygame tutorial" links you to this tutorial. I didn't like it so much. I know most people enjoy diving right into a small working example with shiny sparkly awesomeness, but that tutorial has so much cruft. I was looking for something simpler. The second result led me to this tutorial, but it wasn't much better. I just wanted a simple bare-bones example. I tried to distill down the examples to a minimum and got this:

import pygame

def should_quit():
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            return True
    return False
    
pygame.init()
screen = pygame.display.set_mode((640, 480))

while not should_quit():
    pass

I tried searching for "simple pygame tutorial" instead, which led me to this guy. A lot better. I noticed I didn't have to init pygame or do that event loop either. Changed my program to this:

import pygame

def should_quit():
    event = pygame.event.poll()
    return event.type == pygame.QUIT
    
screen = pygame.display.set_mode((640, 480))

while not should_quit():
    pass

I like that a lot. It doesn't do anything very useful at the moment, but it's a happier place for me to start. Let the pygaming begin!

Safari vs. Firefox for Mac

2008-04-05T13:08:00.000-07:00

During the late night hours, codinghorror is a fountain of tweets filled with cool links. About 11 hours ago he linked to an analysis between Safari vs. Firefox on the Mac. It was an interesting read for me, especially because I'm going to be purchasing a Mac in the near future. One quibble the author had was about the location bar:

When I click the mouse in the middle of a URL, I just want to place the insertion point. I don’t want to select the entire URL. If I wanted to select the entire URL, I’d double-click. Click to place, double-click to select — just like any other text field.

I've experienced frustration with this before, but only when Firefox is slogging along in fits and starts due to excessive memory usage. At that point any frustrations are compounded. 90% of the time when I'm clicking into the location bar, all I want is to copy the entire address. One click and a CTRL+C and I'm good to go. I don't know if I want the location bar to behave "just like any other text field." I've noticed IE7 and Opera do this as well. Anybody else have 2¢ to contribute? Update: After using Firefox for a while today, it's probably more accurate for me to say that half my 90% is spent copying addresses and the other half is spent entering in new addresses. Both of these actions are easier when the full address is automatically selected.

Just lay one brick, dude.

2008-04-03T20:50:00.000-07:00

With some 10 or more months of earnest agile practice and learning under my belt, I continue to be impressed by the impact of its disciplines in my life. I am thinking more clearly, achieving goals with more regularity, and generally feeling more empowered than ever before. We accrue many memories and experiences over the years and it's always interesting to me when things long since forgotten resurface with new clarity or meaning. I'm not having an epiphany so much as I'm experiencing a great and satisfying feeling of being on The Right Path. Below are 5 events which have been swirling around in my mind recently. I have no great insight to share other than to say that for me it feels as though they support and reinforce one another.

I remember watching an interview with Will Smith some years ago where he recounted the story of rebuilding a large brick wall. The part that stuck with me was the banishment of all other concerns. Focus on laying one brick. The magnitude of the task you're undertaking is secondary and inevitable. As a Star Wars fan, this reference is obligatory: Do, or do not. There is no try. I always feel like I'm doing when following agile practices. I spend more time acting than hesitating. I lay one brick at a time, the best way I know how. At the end of the day I've put down a lot of bricks. And that's pretty satisfying.
It was exciting to discover source control for the first time. I can undo any mistake, no matter how big or long ago? Awesome. Of course, in practice execution is everything. It wasn't until I learned about the value of small logical change sets that I was able to fully appreciate versioning. I had found my one brick equivalent.
Not long after my employment at IMVU, the CTO had remarked once as we chatted that his preference was for others to use an active voice. This really stuck with me and continues to spur me to action. All the ways in which I communicate with others have been heavily influenced by the words exchanged during that encounter. I find myself now consciously making effort to express opinions clearly and without reservation. Champion those things which you have a strong belief in or great passion for.
I recently rediscovered Twitter. I think it's brilliant now and I use it daily. This will seem quite hilarious I'm sure, but when I "tweet" I am reminded of committing code to a source control repository. Even further, that I am doing so in an agile way: small, frequent commits that succinctly express what I'm doing. The restriction of 140 characters per tweet is an interesting study in sharpening one's focus. It almost feels like agile pushups. You frequently find yourself tasked with distilling responses and exclamations down to their core expressivity. It's a skill. If you don't believe me, follow Scott Bellware for a while. I dare you not to be impressed by his tweeting mastery.
This week is a "hack week" at IMVU. After each 2 month cycle, a week is given to everyone in the company do whatever we think will improve our product or process in some way to delight customers. We are all free to pursue whatever hair-brained get-rich-quick schemes we may have in mind, management be damned!^[1] This is the first hack week I've participated in where I have actually completed a project and will be presenting. It was a small feature, to be sure, but as a study I kept a "wiki diary" of the entire development process. It was fully test-driven and there is full disclosure of all the obstacles I encountered. During the process of keeping this log, I was again reminded of source control and even Twitter (we are starting to use Google Sites as the company wiki and many people are subscribed to all changes that occur.) I was more than a little shocked by how focused and straight-forward my work became: even before writing a test I would briefly articulate my goal^[2] and then save the page. My current goal was always right there in front of me.

If you don't feel like you're on The Right Path, I challenge you to go catch that rainbow! [1] Management cannot tell us what to do, but we are of course subject to some forms of quality control if we wish to ship to a large number of customers. We don't have free reign to wreak havoc on the site with impunity. ;-) [2] What should I call this approach? "Double Down" TDD? DDTDD sure is a mouthful!

Exploring GCS infrastructure commonalities, Part 2

2008-03-25T09:21:00.000-07:00

Picking up from last time, how can Text be fit back into the Scene mold? Let's try an object-oriented approach. Instead of thinking in terms of data structures and functions that operate on them, let's instead think in terms of classes whose methods operate solely on each respective class's data. The scene renderer we had built deals explicitly in sprites:

DrawScene(scene):
    for each (sprite in scene.sprites):
        DrawImage(sprite.image, sprite.x, sprite.y)

Since we want Text to be treated as a Sprite, we might consider inheritance:

Text extends Sprite:
    sprites[]

However, text doesn't really have this relationship with a sprite. Text is not a type of sprite. We could be clever and generate an image that represents the full text when calling CreateText(), but that's an awful lot of work to satisfy a contrived model. If we shoehorn text into an image, at that point there is little value in creating a class specifically for Text. Object-orientation is all about what things are and what they do. We know what sprites and text are, but what do we want to be able to do with them? Sprites can be:

Drawn
Moved

These are the same abilities we want for text. How can we express those abilities? One option is to use interfaces. I'll leave the gory details to better sources if you're not familiar. The crash course: interfaces are a way to describe abilities that some category of objects can have. In the scenario of our scene render, the only ability we're interested in is how to draw all of the sprites. But I want to be able to draw text as well! Instead of thinking in terms of concrete things which can be drawn, interfaces allow us to think in terms of abstract things which can be drawn. All we're interested in is the "drawability" of those things. We can express drawability as an interface:

interface Paintable:
    Paint()

I'll use painting terminology because it is a common analogy. It also rolls off the tongue a little better! We could go on to use this interface in our Sprite and Text types:

Sprite is Paintable:
    image
    x, y
    Paint():
        DrawImage(image, x, y)

Text is Paintable:
    sprites[]
    Paint():
        for each (sprite in sprites):
            sprite.Paint()

Now I could modify the scene and scene rendering algorithm like so:

Scene:
    paintables[]

DrawScene(scene):
    for each (paintable in scene.paintables):
        paintable.Paint()

Back when I first was learning about interfaces, thinking in this way was a real eye-opener for me. The pseudocode is surprisingly simple. Real code isn't too far removed from this. And actually, I'm still clinging to data structures and functions above. So let's go the full monty:

Scene is Paintable:
    paintables[]
    Paint():
        for each (paintable in paintables):
            paintable.Paint()

A scene itself is paintable, afterall. The great thing here is we can merrily toss any class that implements the Paintable interface into the scene and it'll Just Work. Instances of Sprite and Text know how to draw themselves, and that's all that's important. Through their shared interface, we are able to take advantage of that commonality to great effect. So what about moving sprites and text? What scenarios would a shared interface for movement between the two benefit from? And how would you define it?

Exploring GCS infrastructure commonalities

2008-03-17T01:14:00.001-07:00

I often brainstorm the design of game creation system infrastructure. I'm going to share a bit of how this process unfolds for me and how agile techniques have come to influence it. My favorite puzzle to solve is graphics infrastructure. I've written many of my own systems over the years and have spent egregious amounts of time exploring and attempting to grok open sourced commercial solutions -- notably Wolfenstein, Quake, Abuse, and Golgotha. I've always had some sense of commonality between many of the graphics subsystems, but never a disciplined approach in comparing, contrasting, and distilling those commonalities. My tack was always very rooted in concretes and limited to the concepts expressed by language-specific syntax. Sure, I'd squeeze out some cleverness here and there and pat myself on the back. However, these triumphs were most often over immediate technical obstacles. True wisdom was seldom won. The wisdom I'm referring to is architectural. For example:

Solving common classes of problems through abstract thinking
Maintainable systems that are resilient against and amenable to change
Sustainable systems that don't collapse beneath complexity and growth

I am still cultivating wisdom like this, but I can report some skill in each of these areas. Let me illustrate. I'll use pseudocode to flesh out some concepts. For the sake of simplicity, I'll start with some basic data structures and functions that operate on them. In the realm of 2D gaming, I'll make the generalization that any screen is composed with a collection of positioned images. Forget special effects and everything else. Assume a simple image:

Image:
   bytes[]
   width, height

Very simple: data is the image data. width and height are the dimensions. I'll refer to positioned images as sprites:

Sprite:
   image
   x, y

A sprite is simply an image and an associated position. I'll refer to the collection of positioned images onscreen as a scene:

Scene:
   sprites[]

Given these types, it's fairly simple to imagine a painting algorithm:

DrawScene(scene):
   for each (sprite in scene.sprites):
       DrawImage(sprite.image, sprite.x, sprite.y)

I've got a simple sprite rendering system at this point. Now I might decide I want support for text rendering. Here's a font definition:

Font:
   cells[]

The cells[] are a collection of Image types. A painting algorithm might be:

DrawText(text, x, y, font):
   for each (character in text):
       DrawImage(font.cells[character], x, y)
       x += font.cells[character].width

I am of course glossing over the details of character lookup. It is sufficient to say that a font is a repository of images. Each image maps to a letter in the alphabet. As each image is drawn, we move to the right. Viola, font rendering. Here is where my agile senses start tingling. One of the key agile practices is the separation of concerns. This is simply a technique whereby you isolate code that performs one duty from code that performs other unrelated duties. This may sound obvious. After all, you separate your font rendering from your sprite rendering, your menu system from your map loading code, and your audio system from your scripting engine. How hard can it be? Some separations aren't so obvious. And often what you separate depends upon your goals. Take another look at the DrawText function. Do you consider that function to be managing multiple responsibilities? Would any value come from breaking up those responsibilities? How would you do it? Really think about this for a few moments. When I look at this function, I see two responsibilities:

Rendering
A rendering algorithm

One of my specific goals is code reuse. I already have a sprite rendering system, and the rendering inside DrawText smells a little like duplication. The text that's being rendered could easily be represented by a collection of sprites. What if, instead of relying on a DrawText function to render text, I instead write a function that simply converts text into sprites? Then I can feed those sprites into my existing sytem. Here's a crack at it:

CreateText(text, x, y, font):
   for each (character in text):
       sprites[] = new Sprite(font.cells[character], x, y)
       x += font.cells[character].width
   return sprites

This function does effectively what DrawText does, minus the actual rendering. Now instead of a main game loop that would call both DrawScene and DrawText, I can turn any text into sprites via CreateText and toss them into the scene. I gain a few things from this separation:

Code reuse. I am leveraging an existing system.
Clarity. All rendering occurs in one place only.
Speed. The rendering algorithm would no longer eat CPU every frame.

All of these benefits are worthy of consideration. Of course, there are other considerations here. For example, by tossing the product of CreateText into the sprite pool, the text loses its identity. We no longer have a way to treat that text as a single entity. What happens when we want to remove the text from the scene or move it around, such as for scrolling? This is solved easily enough by creating a data structure for text:

Text:
   sprites[]

Text creation receives minimal changes to accommodate this new structure:

CreateText(text, x, y, font):
   text = new Text
   for each (character in text):
       text.sprites[] = new Sprite(font.cells[character], x, y)
       x += font.cells[character].width
   return text

With its identity regained, text is easily moved:

MoveText(text, x, y):
   for each (sprite in text.sprites):
       sprite.x = x
       sprite.y = y
       x += sprite.width

Here of course the text parameter is an instance of Text and not a string of characters. The trouble with this Text type is that it does not fit into the Scene mold. It contains sprites but is not a sprite itself. Can Text be fit back into that mold? Should it be? What would you do and why?

Subversion Snippets

2008-03-10T04:28:00.000-07:00

Embedding code fragments here is a huge pain in the ass. Months ago I was searching for a way to embed syntax highlighted <code> blocks into my posts. Eventually I stumbled across a solution that was fairly painless and didn't look too shabby. What's been eating at me is the fact that even though I can do this now, it's still a hassle to paste code fragments in, tag them as necessary to receive the highlighting, and then keep the whitespace intact during editing. For some reason blogger.com likes to slowly nibble away at my indentation as I switch between the Edit Html and Compose tabs, or back and forth from the Preview mode. It's very irritating. My primary intention when embedding snippets is sharing code I've already written, so why can't I just reference the original work? Some years ago I remember reading about a technical author who was frequently annoyed by code samples in books which did not compile or were inaccurate. His solution was to build the retrieval of source code fragments directly into the assembly of his books through an automated process. The fragments were gauranteed to be accurate because they were actual production code. This in mind, I spent a little time and hammered out a quick proof-of-concept. Google provides an open source project hosting service with source control. A browsable subversion repository is provided, and you can view the direct raw contents of any file from any revision. My PHP script curls the address for one of the raw files in my v1aen repository, extracts the lines I'm interested in, and applies the syntax highlighting to it. It's quick and dirty and works like a charm. My final solution may make use of Yahoo UI's asyncRequest to retrieve the raw file. From there it should be trivial to filter and embed the code. Like all desperate people in similar situations, I did actually try out an <iframe> first but that's hacky as sin and incredibly difficult to size correctly. Cool beans, hmm? Let the code fragments flow like gravy!

Bad tests, bad practices: they're social problems

2008-03-08T03:14:00.000-08:00

Gojko enumerates a handful of symptoms that are signs TDD has "gone bad." As many of the comments point out, these symptoms are not TDD-specific and apply to automated testing in general. TDD is not about the mere existence of tests, it is about driving your design by writing tests first. Nonethless, it is a useful list worth reading. Ultimately, these are all social problems which are best solved by high visibility into root causes and a culture that actively seeks to improve its process and the health of its product(s). The first and third points in Gojko's list are bad practices. The first I thought deserved mention:

Disabling tests to make a build pass: If the build is failing because of a test, developers disable the test to make the build pass. This is a bad practice because tests become irrelevant or get lost — people don’t remember to fix and enable them later. If the test is deprecated, then delete it completely. If it is not deprecated, don’t disable it but make it pass.

Very good points. However, there is a distinction to be made regarding the nature of the failure. If a test is failing reliably and you are disabling it to make a build pass, you are most likely causing harm to the health of the project. If the test no longer provides valuable coverage, by all means: delete it. If it does provide value, it needs to be fixed. If you don't know whether or not it's valuable and you're disabling it, you are ~~stupid~~ completely unprofessional. Intermittent test failures are a special breed. Disabling them is still dubious, but it is far less harmful to do so when there is high visibility into commits and your process fosters accountability toward their repair in relatively short order. You will certainly be in much better shape than silently gutting valuable test coverage. In such a case, a commented test can serve as a reminder. At IMVU, every engineer who causes a test failure must ask why 5 times. The result of this is shared with all other engineers via a mailing list to which we all subscribe. The idea is to spread lessons learned and thus improve our overall code quality. We have a "build master" of sorts who tracks follow-up work from these lessons learned. The build master keeps responsible parties mindful of their follow-up work and helps to ensure any disabled tests are fixed and reinstated in the automated testing suite. Another idea we will be experimenting with soon is revoking said parties' commit privileges until their follow-up work has been completed. How's that for accountability!

I want a Mac.

2008-03-06T20:51:00.000-08:00

I've realized recently that I'm really enamored with Apple in general. I've flirted with the idea of buying a Mac over the years but never did anything about it. I'd see an ad or watch a short video and think to myself "Man, that'd be cool." Well, guess what. Macs are cool. And I want one. I don't want a cutting edge box, swell as that would be. I just want a decent box I can tinker around with, do some development on, and get the whole Mac experience. It's about time I start getting familiar with some different operating systems. And OS X is hot. My price range is around $1k. I don't want to drop fat wads of cash until I'm fluent and comfortable and know what I like and dislike. I've been directed to the Mac Buyer's Guide by a friend, which is great. I've also been told that if I'm going to be spending over $1k I should get an ADC Student Membership, which would qualify me for around 10% off my purchase. Anyone have any further tips to share?

Never stop learning! Seek an agile environment

2008-03-05T23:54:00.000-08:00

I've come to realize I'll never really get "caught up" with my blogroll.

It was overwhelming at first, but the more I think about it the more I appreciate that fact. Having new and interesting things to read and learn every day is certainly something to look forward to.

I still have a bit of a problem, however -- all of the literature I'm reading is so good and I'm so excited to read and internalize it that I feel compelled to stay up later and later so I can read "just one more" post...

C'est la vie! Life could be worse. And in fact it used to be.

I could be working somewhere that doesn't follow any sort of agile development process. I spent 7 years working in such an environment. I've said it before, but automated testing is the #1 cause of decreased stress in my life. And guess what goes hand-in-hand with agile development? Please believe I am genuine when I say that automated testing gave me back something I'd lost: confidence.

The people I really empathize with are those who are working in a non-agile environment and who see the need for and want to change it but feel powerless to do so. It's a very tough thing to do when you're one developer among many and your plate is full putting out fires.

Toward the end of my stay with my former employer, I began to develop an acute sense of this. I was reading about agile techniques, flirting with Ruby on Rails, and generally attempting to expand my knowledge base in any way that might help give me more confidence in the vast seas of code I was cranking out.

Like many programmers, I was more than happy to be told to implement any sort of fandangled whatchamacallit. I would happily run off into a corner and make it happen, as if it were a dare. But my fantastical devices always aged poorly. My mountains of magical code inevitably became a maintenance nightmare.

It's easy to solve technical problems with technical solutions -- just about any programming monkey with a good command of syntax in any language can figure out how to stuff the round pegs in the round holes and the square pegs in the square holes -- but in my experience writing maintainable solutions is really friggin' hard.

One of the key things that drew me to agile literature was that word: "confidence." As the amount of code I wrote on any given project grew, my confidence shrank. In any complex system, little changes can have far-reaching and subtle impacts, and those are what always get you in the end. I needed a way to keep a tab on all the details and automated tests appeared to me to provide that ability.

Time I spent reading agile material before I left my job: about 1 year.

Percentage of that material I ever tried at work: <10%.

It's really tough to get the ball rolling, especially when you're not even sure how to roll your crap into a ball-like form to begin with. This is partly because I dealt with a lot of legacy systems, but largely because I always felt under the gun to produce. The underlying current guiding my every action was naked human fear. I don't know what I was afraid of specifically. I suppose looking bad and/or getting fired. I would tell myself that I wasn't getting paid to learn, I was getting paid to ship.

I don't know how typical this scenario is, but I can say I've read a lot of blogs where developers express similar frustrations and lack of confidence. I don't have all the answers, but I can offer some small advice: never stop learning. You might not be able to affect valuable change at your current job, as I found myself unable to, but actively learning and saturating your brain with new ideas will prime the pump for the time when you do move on. You'll have a better idea of what sort of environment you'd like to work at.

I can also tell you from experience that landing head-first in an environment that is very agile is incredibly liberating. Instead of attempting to rub against the grain, as a learning "agilist" would in a non-agile environment, running with the herd makes it easy to absorb an incredible amount of information in a short amount of time.

I'm not evangelizing the virtues of becoming a sheep, but simply drawing attention to the fact that it's easier and swifter to learn when among like-minds. It can provide an amazing jump-start.

I've been working at my new job for less than a year now, and we practice agile techniques daily. All of my development is driven by agile practices in some way. I've had the chance to try out all sorts of things I'd only ever read about before, and I've been able to experience the benefits of agile practices first-hand on countless occassions.

I am an agile cheerleader, I won't lie, but even so there are a whole set of values and techniques that come along with agile development that have nothing to do with automated testing but have everything to do with making you a better programmer.

I personally believe that you will only be able to appreciate agile techniques fully if you are able to actively use them in a production environment. It's still possible to get bitten by the agile bug without doing so, and to experience the value of agile practices, but in a production environment with millions of customers there is a sharp point on what practices prove out to have value and which do not. Your learning will become even further accelerated, if for no other reason than that fucking up is costly.

If you aren't in an agile environment, I strongly urge you to seek one out. Don't take everything I've said here for granted. Try it on for size. If you're experiencing deja vu or having a Keanu moment and feel the need for change, I'm telling you -- the grass is indeed greener over here!

Games and automated testing, Part 4

2008-03-04T23:56:00.000-08:00

In Part 3, I demonstrated one way of mocking out an interface: with function pointers. In this part I'll cover a more likely avenue for mocking out an interface: classes. If you're pissed by lack of more game-oriented banter, wait for Part 5. Mocking an interface w/ classes I'm not sure how many people mock out interfaces with function pointers, but I don't expect it's many. It was an interesting exercise to put the idea in perspective though. Most people mock out interfaces with classes. It's easier than micro-managing a bunch of function pointers back and forth between the mock functions and the "real" set of functions. Using classes, I can wrap all of the FMOD library calls with my own methods. Then I can call all of them as needed in a setup() method, something like so:

class Audio {
    FMOD_SYSTEM* system;
public:
    virtual FMOD_RESULT fmod_system_create() {
        return FMOD_System_Create(&system);
    }
    virtual FMOD_RESULT fmod_system_get_version(unsigned int* version) {
        return FMOD_System_GetVersion(system, version);
    }

    ...

    void setup() {
        FMOD_RESULT fmod_result = fmod_system_create();
        validate(fmod_result);

        unsigned int version;
        fmod_result = fmod_system_get_version(&version);
        validate(fmod_result);
        ...
    }
}

With this class in place, mocking out the fmod_* methods is as easy as extending the class and overriding them. For example:

class AudioMock : public Audio {
public:
    FMOD_RESULT fmod_system_create() {
        return FMOD_OK;
    }
    FMOD_RESULT fmod_system_get_version(unsigned int* version) {
        return FMOD_OK;
    }
};

With this mock available, testing is very straightforward:

TEST(setup_can_succeed) {
    AudioMock audio;
    audio.setup();
}

What if I want to exercise a failure path? I can start stuffing extra state into my mock to help influence the path of execution through setup(). To force failure, I could setup the following mechanism:

class AudioMock : public Audio {
    bool system_creation_will_fail;
public:
    void force_system_creation_failure() {
        system_creation_will_fail = true;
    }
    FMOD_RESULT fmod_system_create() {
        return system_creation_will_fail ? FMOD_ERR_BADCOMMAND : FMOD_OK;
    }
    ...
}

My test for failure might be:

TEST(setup_can_fail_on_system_creation) {
    AudioMock audio;
    audio.force_system_creation_failure();
    CHECK_THROW(audio.setup(), AudioException);
}

This assumes I've altered validate() to throw an AudioException on statuses other than FMOD_OK:

void validate(FMOD_RESULT fmod_result) {
    if (fmod_result != FMOD_OK) {
        throw AudioException(fmod_result);
    }
}

Pardon me while I patronize you for a moment: The test will pass because the setup() method from the base Audio class will still execute, but will use the fmod_system_* calls from AudioMock. AudioMock's force_system_creation_failure() will cause fmod_system_create() to return FMOD_ERR_BADCOMMAND. This will in turn cause the validate() in Audio's setup() to throw an AudioException, since FMOD_ERR_BADCOMMAND != FMOD_OK. In this way you get all of the machinery of the setup() method in the base class, but with none of the direct interaction with the audio hardware. Pretty sweet! Next time Mocking out an interface isn't always the way to go, but it's a useful skill to have when you want or need to separate your tests from the hardware or from some mystery resource X. Next time I'll be touching on the differences between a few different types of mocks. Then I'll move on to more interesting and game-oriented topics, such as an introduction to the guinea pig I selected for the process I described in Part 1's What I'm after.

Games and automated testing, Part 3

2008-02-28T01:32:00.000-08:00

In Part 2, I ended on the idea of mocking out an interface. In this part, I'll cover one possible method for doing so. Mocking an interface w/ function pointers There are different ways I could swap out the FMOD library functions with my own during testing. One possibility would be to create function pointers for all of the library functions, something like so:

FMOD_RESULT my_fmod_system_create(FMOD_SYSTEM** system) {
   return FMOD_System_Create(system);
}

FMOD_RESULT my_fmod_system_getversion(FMOD_SYSTEM* system, unsigned int* version) {
   return FMOD_System_GetVersion(system, version);
}

FMOD_RESULT (*global_fmod_system_create)(FMOD_SYSTEM**) = &my_fmod_system_create;
FMOD_RESULT (*global_fmod_system_getversion)(FMOD_SYSTEM*, unsigned int*) = &my_fmod_system_getversion;

void audio_setup() {
   fmod_result = global_fmod_system_create(&system);
   validate(fmod_result);

   fmod_result = global_fmod_system_create(system, &version);
   validate(fmod_result);
   ...
}

What this affords is the ability to swap out these global functions with whatever I choose. Let's look back at the first test:

TEST(setup_succeeds) {
   CHECK(audio_setup());
}

On a machine with a properly configured sound card, this will always succeed. The fact that this test will fail on a machine that does not have a properly configured sound card is what I would classify as a code smell. The test is brittle because it will return inconsistent results depending upon your hardware configuration. This is most especially true in terms of the physical capabilities of any given audio hardware. This is where the value of a mocked out interface comes into play (in this case, the global function pointers.) Consider the following scenario:

void validate(FMOD_RESULT fmod_result) {
   if (fmod_result != FMOD_OK) {
       // display or log an error, then abort the app
       exit(1);
   }
}

FMOD_RESULT mock_fmod_system_create(FMOD_SYSTEM** system) {
   return FMOD_OK;
}

bool audio_setup() {
   FMOD_SYSTEM* system;
   FMOD_RESULT result;
   result = global_fmod_system_create(&system);
   validate(result);
   return true;
}

...

TEST(setup_succeeds) {
   global_fmod_system_create = &mock_fmod_system_create;
   CHECK(audio_setup());
}

This test will pass, but the great thing about it is that it will do so reliably on any machine. It may appear trivial, but consider: The test is verifying that audio_setup() returns FMOD_OK. More abstractly, it is verifying that some specific function returns a success value. More specifically, that it can return a success value -- I am verifying that this code path can be successfully exercised. This is purely algorithmic in nature: Given function X() and input Y (in this case, no input,) value Z is returned. This algorithmic verification is the core value. Tests that talk directly to the hardware It's worth noting that sometimes testing with the hardware directly is acceptable, desirable, or even necessary. Thanks to a couple friends for the reminder. Generally speaking, testing the hardware directly can be... ...necessary when, well, your intent is to actually test the behavior of specific hardware. This sort of testing is on a deeper and more technical level. Your intention here would be to test specific responses from specific hardware, rather than verifying the integrity of a strategy you've conceived of at a higher level (such as a graceful path for degraded communication, as I am testing here.) ...desirable when it's less work to simply communicate with the hardware directly, and the speed of communication with the hardware isn't putting a damper on your speed of development. For many test-driven developers, this translates into whether or not the feedback loop during red-green-refactor becomes too slow to bear. I'm sure there is a parallel to the three important limits for this cycle. ...acceptable when it's less work and you don't really care whether or not the the test is fast. You may accept the fact that some tests are going to be slow (or collectively slow). It might not be worth the trouble to mock out the hardware. You might also start here and let time and experience prove to you whether or not mocking out the hardware will be worth the potential gains in speed. This is somewhat the angle I took -- I had several tests in place before I decided to explore mocking. To be continued...

Games and automated testing, Part 2

2008-02-27T20:05:00.000-08:00

In Part 1, I ended with the code for my audio subsystem and the automated tests I put in place to support it. In this part, I'll be reviewing my choices and why I made them. I'll also share some of the knowledge I've gained since. This post grew rather large, so I'm spreading it across a few separate parts. Testability concerns The audio subsystem was originally just two global functions: audio_setup() and audio_play(). How did it turn into 3 classes with a bunch of hijinx going on inside? The short answer is: testability. When I wrote my first test, it looked like this:

    TEST(setup_succeeds) {
        CHECK(audio_setup());
    }

Here, audio_setup() returns true or false to indicate success. I am verifying the simplest thing I could imagine: that the call succeeds under normal circumstances. Testing in this way is not a good idea for at least two reasons:

I'm interfacing directly with the audio hardware. If you read enough unit testing literature, you'll begin to get an inkling that unit tests should be as small and fast as possible. more The definition of a unit test will vary depending upon whom you ask, but this article parallels my stance. Interfacing with audio hardware is similar to touching a network, database, or file system.
It's difficult to ensure the initial state of that hardware when each test runs. Audio hardware is complex and the rules and intricacies will vary by system. In actuality, the audio hardware is not part of what I am interested in testing. The code I am interested in testing is the block supplied by the FMOD documentation. Essentially, that block is a solution for graceful degradation. Given the audio hardware's capabilities and user preferences, it will find the optimum way for FMOD to communicate with it. What I want to test is that this strategy for degradation is upheld.

Debugger stepping vs. automated testing In Code Complete, Steve McConnell shares the importance of stepping through all code paths in a debugger. For example, given the following code:

    if (player.get_mana() >= 8) {
        player.cast(SLEEP_SPELL);
    } else {
        messaging.tell(player, "Sorry, you don't have enough mana!");
    }

there are two primary paths. The player either has 8 or more mana points or they do not. Stepping through this code with a debugger after setting the player's mana to 8 will tell you whether or not the first block executes as expected. Doing the same at 7 will verify the other branch is executed. This example may seem trivial, but that perceived triviality is false. There are many circumstances under which it might fail or degrade. This code may reside in a function which is never called. get_mana() may be reporting mana incorrectly. cast() or tell() could throw exceptions or crash your machine. By stepping through the code with a debugger, you build familiarity with and confidence in your code. Reading Steve's words, I kept thinking to myself "This is exactly what good automated tests do." Stepping through code in a debugger is a good skill to have, but over time it is not practical to verify all code paths after each modification. Automated tests that verify these paths for you are both faster and more powerful a tool. Brainpower, manpower Veteran programmers will develop skill in realizing dependencies and gauging the impact of change. However, this skill is both localized and inconsistent (human error, etc.) When you translate your personal knowledge of dependencies and expectations into automated tests, not only do you benefit from that knowledge but others do as well. You shouldn't have to allocate so much of your own memory and time for tasks that could easily be automated. Every time you automate you are freeing some of that memory and time. These savings can be repurposed toward focusing on and analyzing newer and more interesting and important problems. Mocking an interface The question then is how am I to test audio_setup() without talking directly to the audio hardware? I've read a lot of agile literature but I've never put much of it to use. With what topical awareness I'd cultivated up to the present, mocks came to mind. The original code I transcribed from the FMOD documentation looked something like this:

    fmod_result = FMOD_System_Create(&system);
    validate(fmod_result);

    fmod_result = FMOD_System_GetVersion(system, &version);
    validate(fmod_result);

FMOD_System_Create() and FMOD_System_GetVersion() were the (first) functions I wanted to mock out. Instead of interfacing directly with the audio hardware, I wanted to replace all of the FMOD library functions with my own "stunt doubles." What possible value could doing something like this have? If my test is not using the audio hardware, then how am I actually testing anything useful to do with my audio system? Remember that I am not actually interested in verifying the behavior of the audio hardware itself. All things being equal, I'm taking for granted that the sound card or on-board audio has been configured properly by the user. I mean this in the sense that, if I were to ask the sound card whether or not it could meet some basic criteria such as "Do you exist?" and that criteria was unable to be met, my program would simply exit. In fact, that is what the first line above is doing. validate() will fail if fmod_result turns out to be anything other than FMOD_OK. As previously stated, I am interested in verifying that the strategy for graceful degradation is upheld. This means that, in a sense, I want to look at this code as if I didn't know or care it was even remotely related to audio hardware. I want to look at it as a black box: I pass values in and expect to get certain values out. To be continued...

Happy birthday, Nate!

2008-02-25T23:59:00.000-08:00

My roommate just turned 24. Happy birthday, man! This guy was a gymnast growing up, and regularly climbs and lifts at Planet Granite for 3 to 4 hours. The dude inspires me. He's a good guy, and has only influenced my life positively. I was going to call it quits early tonight -- I'm a little sick -- but when he rolled in around 11pm it was time for a shot of brand X and a few swigs of brand Y. I don't know what I was tossing back, but I was happy to celebrate. I did all the listening as we toasted and he and Ambert did a little storytelling. It was a good time. Of course, when a "scandalous scavenge" and "watermelon hats" are involved, it's bound to be. Maybe it's the booze, but I'm feeling much improved. It's amazing what a few laughs (and shots) can do for you. My advice: Take some time with your friends. It's worth it. Cheers!

Blog like you mean it

2008-02-22T22:54:00.001-08:00

With the weekend upon us, I'm plowing through a large number of game development and agile development blogs. There are some notable trends that separate the blogs I enjoy from those I do not. A few of the bad trends were really getting on my nerves. Please:

Don't give excuses or place blame. You have too many other obligations and are ever so busy? You and me both, pal. I'm either doing those things or I'm blogging about them. I don't blog about why I haven't been blogging. That is so meta. A lame excuse is worse than no post at all. You got my hopes up and then fed me some bullshit? Thanks. You are an ass.
Don't belittle yourself. Have a little self-respect. Why are you blogging? Are you actually interested in what you're doing? If you are, then don't tear yourself down. Build yourself up. Solve a problem. Celebrate your efforts. Share something and be proud. Leave the useless criticisms to YouTube comments.
Don't use a passive voice. Be assertive. Have a reason, a purpose, a goal. Again, share something and be proud. It's good to have opinions. That's how shit gets done. Lay down the law and take care of some business.

I've had many years of experience developing all of these habits. They're not flattering or interesting. I'm still overcoming a passive voice. Learn to recognize and consciously avoid them. Each is a worthy goal that will add a whole new dimension to your life. The fact that I will then find your blog worth reading is just a bonus!

Posting frequency, GDC, and upcoming topics

2008-02-22T00:43:00.000-08:00

Posting frequency Lately I've been operating very much at the edge of what are healthy levels of sleep. 4 and 5 hours of sleep surely isn't enough. It's not due to stress, quite the opposite: there are so many things Right in my life right now. One result of this is infrequent posts. I would prefer to post daily -- I certainly have blogging topics running through my head every day. So why am I not posting? Briefly, here are some of the interests that tug me in different directions:

I love this job, more than I ever thought I could love a job. I have no work-related stress, it just doesn't exist. I want to work all the time. There are a thousand things I want to code and I frequently stay late at work because of this. I want to improve the quality of our code and I want to improve our processes. Work is not work for me. I'm doing what I love and it almost always trumps everything else.
I've come to very much enjoy working out regularly. I do not drive and one of my roommates frequently visits a climbing/yoga/fitness facility relatively close to our house. As a result I often drive to the gym with him and workout for 3 or 4 hours, starting at 7 or 8pm. He goes every day, and I have to fight the urge not to go every day so that I can get devote more time to programming and blogging.
I feel compelled to "get current" on a lot of different and awesome blogs, so I am reading them constantly. Blogs about game development and agile practices are like candy as far as I'm concerned. And there are a lot of good blogs concerning both.
There are several recreational coding projects I very much enjoy working on. I use many of them as blogging fodder, but the first and foremost joy I derive from them is the education I am gaining from applying the agile practices I am learning to them. I end up spending more time pursuing that recreation and queuing blog topics than I do actually blogging. My queue is growing out of control as a result.

I'll cut myself off there. There are certainly more but those are at the front of my mind. The answer is ultimately to discipline myself to schedule time for the things I enjoy better. Solving this is a continuous background process that my brain is chewing on. I think I may need to simply bring that process more into the foreground to solve my problem and start posting as frequently and fluidly as I would like. GDC I attended the GDC today and I have to say it was very much worth it. I would have preferred to go 3 or more days, but it didn't work out that way. For a first time attendee, however, I got a good taste of what potential for learning and inspiration exists there. The lecture I most looked forward to was From DOOM to RAGE: Pushing Boundaries. The first half hour was spent discussing the business-side of id Software and how they've had to adjust as a company to changing direction and composition, such as number of employees. The last half hour began with a brief comparison of DOOM tech to RAGE tech ala running trailers for both. RAGE is very impressive. The rest of the lecture covered megatextures and how great they are. It was a very entertaining and interesting lecture throughout. I will devote an upcoming post to details and further thoughts. Upcoming topics Here are a few things I have queued up in my brain:

Game development and automated tests, Part 2
What I gleaned from attending the GDC
Personal professionalism
Things I've been learning about Python
My thoughts on Perl, since I've been learning it *gasp*
Brainstorming improvements to my company's agile processes
Why I think Google is insane and amazing

Farewell Manyu

2008-02-18T16:36:00.001-08:00

Today at an all-hands meeting it was announced that a couple of my coworkers have given their notice and will be leaving us. I don't know all of the details, but I was very saddened to hear it. One of those leaving is a fellow engineer whom I respect and admire a great deal. I'm surprised how bummed I am about this. I'm really going to miss him. He is incredibly intelligent, cranks out unbelievable amounts of code, and is very friendly. Losing him feels like we are losing a load-bearing support beam. I took his presence for granted, and now I regret not getting to know him better. I worked on the same project team with him for almost 4 months after I first took this job. He became an engineering manager and the producer of another project team which I was briefly assigned to -- I would have been working closely with him -- but I was shuffled out to another team due to limited staff and resource juggling. Fast-forward another 4 or 5 months and I haven't had the chance to work with him directly again. Sometimes things don't work out the way you want! ;-( I wish Manyu well in whatever he does next!

How to: Setup Eclipse for C++ development in Vista

2008-02-16T06:07:00.000-08:00

Below is a quick list of the steps I used to setup all of the following tools for game development: Vista, Eclipse, C++, Cygwin, Allegro, FMOD, UnitTest++.

Buy and install Vista! It's only $50 if you're a student.
Download Eclipse for C/C++. http://www.eclipse.org/ Unzip into C:\. This creates C:\eclipse.
Download and run the Cygwin setup executable. http://cygwin.com/ Choose C:\cygwin as the installation folder. You'll need the following packages to compile Allegro:
Base: bash, coreutils, cygwin, login, man, sed Devel: binutils, gcc, gdb, make, mingw-runtime Doc: texinfo Libs: w32api Utils: cygutils
Choose the binary distributions for all of the above.
Download the minimal DirectX 7 SDK for MinGW. http://alleg.sourceforge.net/wip.html Unzip into C:\temp. Copy the files inside C:\temp\lib to C:\cygwin\lib\w32api. Copy the files inside C:\temp\include to C:\cygwin\usr\include\w32api.
Edit C:\cygwin\etc\profile. Add these lines to the top:
export ALLEGRO_USE_CYGWIN=1 export MINGDIR=/usr/local export CPATH=/usr/local/include export LIBRARY_PATH=/usr/local/lib
Download Allegro. http://www.talula.demon.co.uk/allegro/ Unzip into C:\cygwin. This creates C:\cygwin\allegro.
Compile Allegro. Launch the Cygwin Bash Shell. Type:
cd /allegro ./fix.sh mingw --dtou make make install
This will attempt to copy some DLLs into a Windows folder. This fails in Vista if UAC is enabled. That's okay: copy the files yourself and click OK when Vista asks for permission. Copy C:\cygwin\allegro\lib\mingw32\liballeg.a to C:\cygwin\usr\local\lib. Copy C:\cygwin\allegro\include\allegro.h to C:\cygwin\usr\local\include. Copy the C:\cygwin\allegro\incldue\allegro folder to C:\cygwin\usr\local\include.
Download and run the FMOD installer. http://www.fmod.org/ Open the FMOD installation folder. The default is C:\Program Files\FMOD SoundSystem\FMOD Programmers API Win32\. Copy api\fmodex.dll to C:\Windows\System32. Copy the files inside api\inc to C:\cygwin\usr\local\include. Copy api\lib\libfmodex.a to C:\cygwin\usr\local\lib.
Download UnitTest++. http://unittest-cpp.sourceforge.net/ Unzip into C:\cygwin. This creates C:\cygwin\UnitTest++.
Launch the Cygwin Bash Shell. Type:
cd /UnitTest++ make all mkdir /usr/local/include/Posix mkdir /usr/local/include/Win32 cp libUnitTest++.a /usr/local/lib cp src/*.h /usr/local/include cp src/Posix/*.h /usr/local/include/Posix cp src/Win32/*.h /usr/local/include/Win32
Give everything a test-drive. under construction

Games and automated testing, Part 1

2008-02-14T00:09:00.000-08:00

I like tinkering around with game programming in my spare-spare time. Last weekend I decided to tinker around with setting up Eclipse for C++ development under Vista using Cygwin with Allegro, FMOD, and UnitTest++. I don't recommend trying this at home unless you've got a weekend to kill, own a robe and wizard hat, and are equipped with Lv. 8 Stubbornness of the Infinite. If you've got the Iteration #1 Blues, boy do I empathize. I'll post a how-to tomorrow, for those with similarly warped interests. Tool choices Why the above combination? My ultimate goal lies in exploring automated testing in relation to game development. I have a specific scenario in mind, which I'll explain momentarily, but to briefly summarize the above choices:

I've been using Eclipse since 2.0 and did a lot of Java development in it professionally back in the day. I use it professionally now for PHP and Python development. And it's free.
I've only started using Cygwin in the last year. I also use it professionally. I have general interest in *nix environments and getting more familiar with them. And it's free.
Allegro is a games programming library I played with a lot back in the day when I was around 18-22. It's easy to use and does a lot of heavy-lifting for me. And it's free.
UnitTest++ is a minimalistic unit testing framework for C++ that a coworker recently recommended to me. I've been using it for a couple months recreationally. And it's free.
FMOD is an audio library with awesome format support for tracked music. Ever since I started delving into game programming, I've been listening to tracked music. Mmm, nostalgia. And it's free (for non-commercial use.)
Vista is... not free. I like it, for mostly superficial reasons. Sexy UI (though I have themes disabled,) better fonts and font rendering, and thicker, more grabbable window borders. Some people have issues with Vista's file copy implementation, but that doesn't bother me.

My coding wanderlust The exploration I have in mind has a little background story: For many years I was involved in the development of a game creation system. I know and have met personally not one, not two, but three people who have written game creation systems. I work with two of these people currently. I also still love a couple of other more old-school game creation systems. More than anything I am interested in the development of game creation systems themselves. Even deeper than that is my growing fascination with domain-driven design and how that can help me grow a game creation system naturally. The bridge to productivity I view automated testing as a key enabling factor in my pursuit of these interests. I've had over 7 months of experience developing professionally with the support of automated testing . I have experienced its benefits first-hand and on numerous occasions. I've been saved by tests made by others and tests of my own making. I've been burned when test coverage was not in place. I've also lost productivity due to misleading, overly large, and otherwise poorly-written tests. However, the good has always far outweighed the bad. Writing automated tests is a skill, and it takes a lot of practice to do well. Realizing this, I am still actively learning. Having spent over 7 years prior as a professional web developer in an environment that did not use automated testing, I can assure you that I am sincere when I say that I attribute much of the stress-free lifestyle I now enjoy to automated testing. So, I'm cataloging my automated testing exploits. Where can I put tests to give me the most confidence? Is that all tests are good for? Do they have other useful properties? Do I really need to write a test for X, Y, or Z? I don't often know. 9 times out of 10 I'll err on the side of writing them and see how things prove out over time. There's no better way to learn than by doing! What I'm after All the above taken into consideration, the specific scenario I am interested in is to write a simple game, take stock of what was required to write it, then distill from that what would be required for a simple game creation system to do the same thing. Eventually, I write another simple game. And another. I pick different styles as I go. As I go, I am continuously extracting and integrating all of the essential ingredients that would allow me to author all of these games with my game creation system. I've been told that game creation systems are not well-suited to TDD, my favored method for creating automated tests, but I think driving toward a system "in reverse" as I've explained is a promising avenue to explore, and one that may prove that assertion to be false. I am at least eager to try -- I have a lot of faith in TDD! Worst case is, I learn a lot. Testing FMOD I've never played around much with audio. Figuring out how to get FMOD working with all my tools of choice was rather heartening and a little exciting for me. It was a lot easier than I had expected. My automated test alarm went off when I was reading through the FMOD documentation (yes, some of us actually RTFM,) and happened upon this line in the platform-specific issues section for Windows: !!! THIS CODE MUST BE USED FOR SHIPPING GAMES. DO NOT SHIP A GAME WITHOUT A STARTUP SEQUENCE BASED ON THIS CODE !!!
To follow was a ~25 line chunk of code. It seemed like something that might be useful to put under test, so I did. And of course I've never written automated tests for audio code, so that was appealing to me. I'm not sure how enduring the value of my tests will turn out to be, but I can say this: I had to put a handful of situations under test for code that I wasn't even entirely sure what it did, and now I understand it a lot better. The act of writing tests definitely forces you to become familiar with the code, as you attempt to pick out all the useful code paths and how you might test them. This process wasn't exactly test-driven, since what I wanted to test had already been conceived, but it's test writing practice all the same. This happens frequently at my job: we are using some 3rd party software that has no test coverage. Putting the entire thing under test before using it is impractical. Instead, we test bits and pieces at critical points of integration as we go and need dictates. Implementation and tests I'll cover the ideas that drove my design tomorrow, but for now I'll just paste the relevant code for my audio subsystem. Here's the integration with FMOD: audio.h

#ifndef _AUDIO_H
#define _AUDIO_H

#include <fmod.h>
#include <string>

using namespace std;

struct AudioException {
  FMOD_RESULT fmod_result;
  string message;

  AudioException(FMOD_RESULT fmod_result, string message) {
      this->fmod_result = fmod_result;
      this->message = message;
  }
};

class Audio {
  void validate(FMOD_RESULT result, string message);

protected:
  static FMOD_SYSTEM* system;

  FMOD_RESULT fmod_result;

public:
  virtual ~Audio() {}

  virtual FMOD_RESULT fmod_system_create();
  virtual FMOD_RESULT fmod_system_get_version(unsigned int* version);
  virtual FMOD_RESULT fmod_system_get_driver_capabilities(int id, FMOD_CAPS* capabilities, int* min_frequency, int* max_frequency, FMOD_SPEAKERMODE* control_panel_speaker_mode);
  virtual FMOD_RESULT fmod_system_set_speaker_mode(FMOD_SPEAKERMODE speaker_mode);
  virtual FMOD_RESULT fmod_system_set_dsp_buffer_size(unsigned int buffer_length, int num_buffers);
  virtual FMOD_RESULT fmod_system_get_dsp_buffer_size(unsigned int* buffer_length, int* num_buffers);
  virtual FMOD_RESULT fmod_system_init(int max_channels, FMOD_INITFLAGS flags, void* extra_driver_data);
  virtual FMOD_RESULT fmod_system_create_stream(const char* name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO* exinfo, FMOD_SOUND** sound);
  virtual FMOD_RESULT fmod_system_play_sound(FMOD_CHANNELINDEX channel_id, FMOD_SOUND* sound, bool paused, FMOD_CHANNEL** channel);
  virtual FMOD_RESULT fmod_system_release();

  void setup();
  void shutdown();
  void play(string filename);
};

class MockAudio : public Audio {
  int fmod_version;
  int driver_capabilities;
  int dsp_buffer_length;
  int dsp_num_buffers;
  FMOD_SPEAKERMODE speaker_mode;
  bool fail_system_creation;
  bool fail_getting_driver_capabilities;
  bool fail_setting_speaker_mode;
  bool fail_system_init_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER;

public:
  MockAudio();
  virtual ~MockAudio();

  FMOD_RESULT fmod_system_create();
  FMOD_RESULT fmod_system_get_version(unsigned int* version);
  FMOD_RESULT fmod_system_get_driver_capabilities(int id, FMOD_CAPS* capabilities, int* min_frequency, int* max_frequency, FMOD_SPEAKERMODE* control_panel_speaker_mode);
  FMOD_RESULT fmod_system_set_speaker_mode(FMOD_SPEAKERMODE speaker_mode);
  FMOD_RESULT fmod_system_get_speaker_mode(FMOD_SPEAKERMODE* speaker_mode);
  FMOD_RESULT fmod_system_set_dsp_buffer_size(unsigned int buffer_length, int num_buffers);
  FMOD_RESULT fmod_system_get_dsp_buffer_size(unsigned int* buffer_length, int* num_buffers);
  FMOD_RESULT fmod_system_init(int max_channels, FMOD_INITFLAGS flags, void* extra_driver_data);
  FMOD_RESULT fmod_system_create_stream(const char* name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO* exinfo, FMOD_SOUND** sound);
  FMOD_RESULT fmod_system_play_sound(FMOD_CHANNELINDEX channel_id, FMOD_SOUND* sound, bool paused, FMOD_CHANNEL** channel);
  FMOD_RESULT fmod_system_release();

  void force_system_creation_to_fail();
  void force_getting_driver_capabilities_to_fail();
  void force_setting_speaker_mode_to_fail();
  void force_system_init_to_fail_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER();
  void set_FMOD_version(int version);
  void set_driver_capabilities(int capabilities);
};

#endif // _AUDIO_H

audio.cpp

#include "audio.h"
#include <allegro.h>
#include <sstream>
#include <iomanip>
#include <iostream>

FMOD_SYSTEM* Audio::system = 0;

void Audio::validate(FMOD_RESULT fmod_result, string message) {
  if (FMOD_OK != fmod_result) {
      throw AudioException(fmod_result, message);
  }
}

FMOD_RESULT Audio::fmod_system_create() {
  return FMOD_System_Create(&system);
}

FMOD_RESULT Audio::fmod_system_get_version(unsigned int* version) {
  return FMOD_System_GetVersion(system, version);
}

FMOD_RESULT Audio::fmod_system_get_driver_capabilities(int id, FMOD_CAPS* capabilities, int* min_frequency, int* max_frequency, FMOD_SPEAKERMODE* control_panel_speaker_mode) {
  return FMOD_System_GetDriverCaps(system, id, capabilities, min_frequency, max_frequency, control_panel_speaker_mode);
}

FMOD_RESULT Audio::fmod_system_set_speaker_mode(FMOD_SPEAKERMODE speaker_mode) {
  return FMOD_System_SetSpeakerMode(system, speaker_mode);
}

FMOD_RESULT Audio::fmod_system_set_dsp_buffer_size(unsigned int buffer_length, int num_buffers) {
  return FMOD_System_SetDSPBufferSize(system, buffer_length, num_buffers);
}

FMOD_RESULT Audio::fmod_system_get_dsp_buffer_size(unsigned int* buffer_length, int* num_buffers) {
  return FMOD_System_GetDSPBufferSize(system, buffer_length, num_buffers);
}

FMOD_RESULT Audio::fmod_system_init(int max_channels, FMOD_INITFLAGS flags, void* extra_driver_data) {
  return FMOD_System_Init(system, max_channels, flags, extra_driver_data);
}

FMOD_RESULT Audio::fmod_system_create_stream(const char* name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO* exinfo, FMOD_SOUND** sound) {
  return FMOD_System_CreateStream(system, name_or_data, mode, exinfo, sound);
}

FMOD_RESULT Audio::fmod_system_play_sound(FMOD_CHANNELINDEX channel_id, FMOD_SOUND* sound, bool paused, FMOD_CHANNEL** channel) {
  return FMOD_System_PlaySound(system, channel_id, sound, paused, channel);
}

void Audio::setup() {
  if (system) {
      throw AudioException(FMOD_OK, "You have already called setup.");
  }

  fmod_result = fmod_system_create();
  validate(fmod_result, "fmod_system_create");

  unsigned int version;
  fmod_result = fmod_system_get_version(&version);
  validate(fmod_result, "fmod_system_get_version");
  if (version < FMOD_VERSION) {
      stringstream ss;
      ss  << "You are using an old version of FMOD " << hex << showbase << version << ". "
          << "This program requires " << hex << showbase << FMOD_VERSION << ".";
      throw AudioException(fmod_result, ss.str());
  }

  FMOD_CAPS capabilities;
  FMOD_SPEAKERMODE speaker_mode;
  fmod_result = fmod_system_get_driver_capabilities(0, &capabilities, 0, 0, &speaker_mode);
  validate(fmod_result, "fmod_system_get_driver_capabilities");

  // Set the user selected speaker mode.
  fmod_result = fmod_system_set_speaker_mode(speaker_mode);
  validate(fmod_result, "fmod_system_set_speaker_mode");

  // The user has the 'Acceleration' slider set to off!  This is really bad for latency!
  // You might want to warn the user about this.
  // At 48khz, the latency between issuing an fmod command and hearing it will now be about 213ms.
  if (capabilities & FMOD_CAPS_HARDWARE_EMULATED) {
      fmod_result = fmod_system_set_dsp_buffer_size(1024, 10);  
      validate(fmod_result, "fmod_system_set_dsp_buffer_size");
  }

  // Replace with whatever channel count and flags you use!
  fmod_result = fmod_system_init(100, FMOD_INIT_NORMAL, 0);
  if (fmod_result == FMOD_ERR_OUTPUT_CREATEBUFFER) {
      fmod_result = fmod_system_set_speaker_mode(FMOD_SPEAKERMODE_STEREO);
      validate(fmod_result, "fmod_system_set_speaker_mode");
            
      // Replace with whatever channel count and flags you use!
      fmod_result = fmod_system_init(100, FMOD_INIT_NORMAL, 0);
      validate(fmod_result, "fmod_system_init");
  }
}

FMOD_RESULT Audio::fmod_system_release() {
  return FMOD_System_Release(system);
}

void Audio::shutdown() {
  fmod_system_release();
  system = 0;
}

void Audio::play(string filename) {
  FMOD_SOUND* sound;
  // FMOD_DEFAULT uses the defaults.  These are the same as FMOD_LOOP_OFF | FMOD_2D | FMOD_HARDWARE.
  fmod_result = fmod_system_create_stream(filename.c_str(), FMOD_DEFAULT, 0, &sound);
  validate(fmod_result, "fmod_system_create_stream");

  FMOD_CHANNEL* channel;
  fmod_result = fmod_system_play_sound(FMOD_CHANNEL_FREE, sound, false, &channel);
  validate(fmod_result, "fmod_system_play_sound");
}

MockAudio::MockAudio()
: fmod_version(FMOD_VERSION)
, driver_capabilities(0)
, dsp_buffer_length(0)
, dsp_num_buffers(0)
, speaker_mode(FMOD_SPEAKERMODE_RAW)
, fail_system_creation(false)
, fail_getting_driver_capabilities(false)
, fail_setting_speaker_mode(false)
, fail_system_init_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER(false)
{
  // empty
}

MockAudio::~MockAudio() {
  // empty
}

void MockAudio::force_system_creation_to_fail() {
  fail_system_creation = true;
}

void MockAudio::force_getting_driver_capabilities_to_fail() {
  fail_getting_driver_capabilities = true;
}

void MockAudio::force_setting_speaker_mode_to_fail() {
  fail_setting_speaker_mode = true;
}

void MockAudio::force_system_init_to_fail_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER() {
  fail_system_init_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER = true;
}

FMOD_RESULT MockAudio::fmod_system_create() {
  if (fail_system_creation) {
      return FMOD_ERR_BADCOMMAND;
  }
  system = (FMOD_SYSTEM*)1;  
  return FMOD_OK;
}

void MockAudio::set_FMOD_version(int version) {
  fmod_version = version;
}

void MockAudio::set_driver_capabilities(int capabilities) {
  driver_capabilities = capabilities;
}

FMOD_RESULT MockAudio::fmod_system_get_version(unsigned int* version) {
  *version = fmod_version;
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_get_driver_capabilities(int id, FMOD_CAPS* capabilities, int* min_frequency, int* max_frequency, FMOD_SPEAKERMODE* control_panel_speaker_mode) {
  *control_panel_speaker_mode = speaker_mode;
  *capabilities = driver_capabilities;
  return fail_getting_driver_capabilities ? FMOD_ERR_BADCOMMAND : FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_set_speaker_mode(FMOD_SPEAKERMODE speaker_mode) {
  this->speaker_mode = speaker_mode;
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_get_speaker_mode(FMOD_SPEAKERMODE* speaker_mode) {
  *speaker_mode = this->speaker_mode;
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_set_dsp_buffer_size(unsigned int buffer_length, int num_buffers) {
  dsp_buffer_length = buffer_length;
  dsp_num_buffers = num_buffers;
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_get_dsp_buffer_size(unsigned int* buffer_length, int* num_buffers) {
  *buffer_length = dsp_buffer_length;
  *num_buffers = dsp_num_buffers;
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_init(int max_channels, FMOD_INITFLAGS flags, void* extra_driver_data) {
  if (fail_system_init_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER) {
      fail_system_init_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER = false;
      return FMOD_ERR_OUTPUT_CREATEBUFFER;
  }
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_create_stream(const char* name_or_data, FMOD_MODE mode, FMOD_CREATESOUNDEXINFO* exinfo, FMOD_SOUND** sound) {
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_play_sound(FMOD_CHANNELINDEX channel_id, FMOD_SOUND* sound, bool paused, FMOD_CHANNEL** channel) {
  return FMOD_OK;
}

FMOD_RESULT MockAudio::fmod_system_release() {
  return FMOD_OK;
}

And here are the tests: audio_test.cpp

#include <UnitTest++.h>
#include "audio.h"
#include <iostream>
#include <fmod.h>

using namespace std;

SUITE(audio) {

  TEST(setup_succeeds) {
      MockAudio audio;
      audio.setup();
      audio.shutdown();
  }

  TEST(calling_setup_twice_in_a_row_fails) {
      MockAudio audio;
      audio.setup();
      CHECK_THROW(audio.setup(), AudioException);
      audio.shutdown();
  }

  TEST(setup_fails_on_system_creation) {
      MockAudio audio;
      audio.force_system_creation_to_fail();
      CHECK_THROW(audio.setup(), AudioException);
  }

  TEST(setup_fails_on_old_FMOD_version) {
      MockAudio audio;
      audio.set_FMOD_version(FMOD_VERSION - 1);
      CHECK_THROW(audio.setup(), AudioException);
  }

  TEST(setup_fails_on_getting_driver_capabilities) {
      MockAudio audio;
      audio.force_getting_driver_capabilities_to_fail();
      CHECK_THROW(audio.setup(), AudioException);
  }

  TEST(setup_fails_on_setting_speaker_mode) {
      MockAudio audio;
      audio.force_setting_speaker_mode_to_fail();
      CHECK_THROW(audio.setup(), AudioException);
      audio.shutdown();
  }

  TEST(setup_forces_dsp_buffer_size_if_user_has_hardware_acceleration_turned_off) {
      MockAudio audio;
      audio.set_driver_capabilities(FMOD_CAPS_HARDWARE_EMULATED);
      audio.setup();
    
      unsigned int buffer_length = 0;
      int num_buffers = 0;
      audio.fmod_system_get_dsp_buffer_size(&buffer_length, &num_buffers);
      CHECK_EQUAL(1024U, buffer_length);
      CHECK_EQUAL(10, num_buffers);
      audio.shutdown();
  }

  TEST(setup_does_not_force_dsp_buffer_size_if_user_has_hardware_acceleration_turned_on) {
      MockAudio audio;
      audio.set_driver_capabilities(0xffff & ~FMOD_CAPS_HARDWARE_EMULATED);
      audio.fmod_system_set_dsp_buffer_size(123U, 456);
      audio.setup();
    
      unsigned int buffer_length = 0;
      int num_buffers = 0;
      audio.fmod_system_get_dsp_buffer_size(&buffer_length, &num_buffers);
      CHECK_EQUAL(123U, buffer_length);
      CHECK_EQUAL(456, num_buffers);
      audio.shutdown();
  }

  TEST(setup_forces_stereo_if_system_init_fails_with_FMOD_ERR_OUTPUT_CREATEBUFFER) {
      MockAudio audio;
      audio.fmod_system_set_speaker_mode(FMOD_SPEAKERMODE_RAW);
      audio.force_system_init_to_fail_with_result_FMOD_ERR_OUTPUT_CREATEBUFFER();
      audio.setup();
      FMOD_SPEAKERMODE speaker_mode = FMOD_SPEAKERMODE_RAW;
      audio.fmod_system_get_speaker_mode(&speaker_mode);
      CHECK_EQUAL(FMOD_SPEAKERMODE_STEREO, speaker_mode);
      audio.shutdown();
  }

  TEST(setup_does_not_force_stereo_if_system_init_succeeds) {
      MockAudio audio;
      audio.fmod_system_set_speaker_mode(FMOD_SPEAKERMODE_MONO);
      audio.setup();
      FMOD_SPEAKERMODE speaker_mode = FMOD_SPEAKERMODE_RAW;
      audio.fmod_system_get_speaker_mode(&speaker_mode);
      CHECK_EQUAL(FMOD_SPEAKERMODE_MONO, speaker_mode);
      audio.shutdown();
  }
}

Regression testing is awesome

2008-02-07T21:57:00.000-08:00

The story
I was just out to dinner with a few coworkers and was surprised to learn that our primary QA guy considers our company to be "the place where bugs don't come back." He had been with Adobe previously and observed that while Adobe was very focused on quality, recurring bugs were not uncommon.

Our QA guy has always been very clear and up front in letting management know that he does not feel we take quality seriously. He isn't derisive -- the man is impeccably professional -- but my impression up until now had been that he was largely dissatisfied with our process. His observation that we don't have recurring bugs was a happy discovery for me. We may need to focus more on quality, but we are kicking butt in the bug fixing department.

Why we kick butt at bug fixing
I'd never really thought about it and taken it mostly for granted, but we really aren't troubled by recurring bugs. And the reason why is that every time we tackle a bug, we write a regression test.

TDD is very much a part of our culture, so when it comes time to fix a bug, we first write a test which duplicates the buggy behavior. The test fails initially because we write the test to expect everything to be "OK." We are basically mimicking user actions with code to achieve this. Once the test is in place, we shift focus to the code which contains the bug. We tweak that code as necessary until the test passes.

*Poof* The bug is gone, and our automated test proves it.

If any other engineer ever happens to do something that screws with the fix, that test will fail and they simply won't be able to push their change to production. They will only be able to do so after making the changes needed for their changes to co-exist with the fix. That is, until the failing regression test passes for them as well (this is a mechanism built into our deployment process.)

Fostering our culture
Writing regression tests is a habit nurtured by the guideline that any customer-facing changes be summarized in an email sent to the company-wide mailing list. Three things are expected to be indicated in these "change mails."

Did you test your change in production?
Where did you announce the change?
Did you add or update automated tests?

#3 is the big one. If you send out a change mail without any indication of test coverage, chances are pretty good someone is going to hunt you down and demand a reason why your changes aren't covered by tests. This is especially true for bug fixes.

Stress begone
It's frustrating and morale-sapping when a feature devolves into a broken state that you were certain had already been fixed. Chances are you were right but someone accidentally trampled all over it. The fact that we really don't have to worry about that sort of thing is a tremendous win!

Designing from the middle tier

2008-02-05T09:32:00.000-08:00

Apparently I am incapable of blogging in small efficient bursts. Apologies! But I find automated testing endlessly fascinating. I was just reading Jeremy D. Miller's First Causes in Software Development: How do I decide what is good? when something gelled for me. I was reading through the list of his "choices." As soon as I saw this one I had a Keanu moment:

I design from the middle tier or user interface first, and the database last. Again, some of that is just preference, but there are some very real reasons to work first with your objects and let the database fall out secondly.

There is a swirling ether of best practices floating around in my brain and I "get" some of them more than others. Some of them seem like they have merit whereas I've directly experienced the merit of others. The above choice makes Separation of Concerns more concrete for me, and brings to mind an example. I can feed this somewhat into my previous post about "Heavy" automated tests. Imagine a function that generates the HTML for some portion of a web page. There are many such functions in the code I handle at work, and we loosely refer to them as "widgets." For example, one widget function might output a message from a discussion board. A non-TDD approach When coding in a non-TDD fashion and unaware of things like Separation of Concerns, many people tend to do something like this:

function display_message($message_id) {
    $message = query_database("SELECT * FROM messages WHERE message_id = $message_id");
    echo "<div><strong>" . $message["title"] . "</strong><br />";
    echo "by " . $message["author"] . " at " . $message["datetime"];
    echo "<p>" . $message["body"] . "<p>";
}

This approach is not friendly to changes you might make in the future such as pulling the data from a different source. Maybe you switch your database from MySQL to PostgreSQL. "No problem, that's just a simple query change." But what if you want to leverage this message displaying widget for a different type of data source entirely, such as CSV, XML, or perhaps raw data that's been decompressed from an archive? Do you write another function with almost identical code, differing only by the data source you're pulling from? I hope not, because duplication equals debt. This particular widget may be a contrived example with an unlikely need to ever switch to these sorts of data sources, but it's easily imaginable that widget X that displays data from source Y might some day want to display data from source Z instead. Pardon my alphabetic onslaught. Testability Now fast-forward to a time when you begin attempting to start automating tests for your code. Code such as the above produces the "heavy" tests I caution against in my previous post. The problem is that you cannot test display_message() without hitting the database. The function is dependent upon the database because it's running a query directly. The display of the message should be separated from the details involved in getting the data to be displayed. The TDD mindset is all about writing tests first, which is all well and good, but it's still conceivable you could arrive at the above solution while practicing TDD. You are probably practicing TDD naively, since most TDD practitioners are well-read on topics such as Separation of Concerns, but it's still a possible and perhaps even likely outcome for TDD novitiates. Jeremy's conscious choice to develop the user interface first is a result of experience with the benefits of Separation of Concerns, but his statement is much more helpful and concrete than saying "You should separate the code for data access from the code for displaying your user interface." He's providing an order here which is quite helpful. A TDD-oriented approach If you are a fan of Jeremy, like I am, even blindly following his advice without understanding Why it's a good idea leads to more testable code. You would be more likely to write the message display function more like this:

function display_message($message) {
    echo "<div><strong>" . $message["title"] . "</strong><br />";
    echo "by " . $message["author"] . " at " . $message["datetime"];
    echo "<p>" . $message["body"] . "<p>";
}

The database doesn't even enter into the picture here and you can easily write a unit test that verifies a message is displayed properly. This function accepts an array that contains the message data and does nothing more than display the message. This means you can grab your message data from wherever you want and then pass it in. For example, you might go on to write the two following functions:

function get_message_from_db($message_id) {
    $result = query_database("SELECT * FROM messages WHERE message_id = $message_id");
    $message = /* convert $result into an array somehow */;
    return $message;
}

function get_message_from_xml($message_id) {
    $handle = fopen("message-$message_id.xml");
    $message = /* read in an parse the XML, converting it into an array somehow */;
    return $message;
}

These functions grab message data in different ways. Now you display messages by querying the database in some places:

display_message(get_message_from_db($message_id));

And display message by reading an XML file in others:

display_message(get_message_from_xml($message_id));

And your unit test for displaying messages doesn't have to know about any of these data sources. Your test won't be wasting time querying the database or reading a file from disk. Make sense? I'm glossing over blurring together some things here: I'm sort of implying that if you practice TDD you also will therefore be familiar with Separation of Concerns and practice that as well. But I'm betting you can see a nugget of truth here. Making certain choices simply promote more extensible and maintainable code, even if you hate automated testing with every fiber of your being. Readings posts like Jeremy's enamors me further with TDD and helps support my feeling that TDD is a Good Idea. However, even if you're not a fan of TDD, a lot of the "choices" Jeremy has been led to through TDD make a lot of good sense. That being the case, I recommend giving TDD an earnest effort and see where it takes you!