One of the gentlemen that I was speaking with is convinced that software development is well on the road to becoming a commodity and therefore an unprofitable profession in the US. His argument was that there are tools, on the market today, that allow business people to design and generate things as technical as SOAP interfaces. And that because these tools exist today and will only become more sophisticated that our job as developers will quickly become to expensive to support stateside.

Obviously this is of great concern to me, but I’m not convinced. I countered this statement with a question about quality. How could such a process deliver a level of quality worthy of trading out a local team for an overseas consulting firm? How is acceptance handled? Who is there to judge quality?

In response, I received the reply that these new tools are starting to decouple the process of software development so that higher quality outsource-based development can occur; and that it will occur with increasing frequency as time goes on. He went on to say that he imagined that the local team would consist of sales, marketing, product managers, project managers and architects, and that the outsource team would consist of project liasons and software developers.

Indeed I had heard this time and time again, although with less frequency over the past few years. It seems so obvious doesn’t it? Software is an abstraction over hardware, and after all, hardware has become a commodity already. It’s only a matter of time, at least that’s what the CEOs of the world are hoping.

To anyone who actually practices software development day in and day out, all of this may sound terrifying. Or if you are like me, terribly naive.

I have no doubt in my mind that many attempts will be made to follow this outsourcing model of software development. Some will succeed (as they have int he past) and many will not. Part of this will be due to the general difficulty of founding a successful project, and part of it will be due to the process. It seems to me that this outsourcing thing is a sort-of sideways evolution of Big Design Up Front (BDUF), a practice which has proven time and time again to be largely ineffective at best and a complete failure at worst.

I was able to keep most of the methods very short and I didn’t write much code that isn’t being used. The greatest thing of all is how much confidence I have in this little bit of code.

There are definitely some smells around the special cases and some of the function names, but I’m ok with that for this project.

Here’s the python code:

import unittest
 
class FuzzyClock(object):
    number_map = {
            1 : "one",
            2 : "two",
            3 : "three",
            4 : "four",
            5 : "five",
            6 : "six",
            7 : "seven",
            8 : "eight",
            9 : "nine",
            10: "ten",
            11: "eleven",
            12: "twelve",
            15: "quarter",
            20: "twenty",
            25: "twentyfive",
            30: "half"
            }
 
    def _convert_hour(self, hour, minute):
 
        if minute > 30:
            hour += 1
 
        if hour > 23:
            hour = 0
 
        if hour == 12:
            return "noon"
 
        if hour == 0:
            return "midnight"
 
        hour = hour % 12
 
        return self.number_map[hour]
 
    def _get_half(self, minute):
        if minute < 31:
            return "past"
        else:
            return "to"
 
    def _round_minute(self, minute):
        remainder = minute % 5
        if remainder > 2:
            return (minute - remainder) + 5
        else:
            return minute - remainder
 
    def _shave_minute(self, minute):
        if minute <= 30:
            return minute
        else:
            return 60 - minute
 
    def _fetch_minute(self, minute):
        return self.number_map[self._round_minute(self._shave_minute(minute))]
 
    def _convert_minute(self, hour, minute):
        if hour % 12 == 0 and minute == 0:
            return ""
 
        if minute == 0:
            return "o'clock"
        if minute < 5:
            return "shortly after"
        if minute > 55:
            return "nearly"
 
        return "%s %s" % (self._fetch_minute(minute), self._get_half(minute))
 
    def _order_parts(self, hour, minute):
        if minute == "o'clock":
            return (hour,minute)
        elif minute == "":
            return (hour,)
        else:
            return (minute,hour)
 
    def convert(self, hour, minute):
        formatted_hour = self._convert_hour(hour, minute)
        formatted_minute = self._convert_minute(hour, minute)
 
        return " ".join(self._order_parts(formatted_hour,formatted_minute))
 
class TestFuzzyClock(unittest.TestCase):
 
    def setUp(self):
        self.clock = FuzzyClock()
        self.hour = 21
        self.minute = 0
 
    def _set_hour(self, hour):
        self.hour = hour
 
    def _set_minute(self, minute):
        self.minute = minute
 
    def _time_should_be(self, expected):
        self.assertEqual(self.clock.convert(self.hour, self.minute), expected)
 
    def test_whole_number(self):
        self._time_should_be("nine o'clock")
 
    def test_shortly_after(self):
        self._set_minute(1)
        self._time_should_be("shortly after nine")
 
    def test_five_past(self):
        self._set_minute(5)
        self._time_should_be("five past nine")
 
    def test_seven_past(self):
        self._set_minute(7)
        self._time_should_be("five past nine")
 
    def test_ten_past(self):
        self._set_minute(10)
        self._time_should_be("ten past nine")
 
    def test_twelve_past(self):
        self._set_minute(12)
        self._time_should_be("ten past nine")
 
    def test_fourteen_past(self):
        self._set_minute(14)
        self._time_should_be("quarter past nine")
 
    def test_quarter_past(self):
        self._set_minute(15)
        self._time_should_be("quarter past nine")
 
    def test_eighteen_past(self):
        self._set_minute(18)
        self._time_should_be("twenty past nine")
 
    def test_twenty_past(self):
        self._set_minute(20)
        self._time_should_be("twenty past nine")
 
    def test_twenty_five_past(self):
        self._set_minute(25)
        self._time_should_be("twentyfive past nine")
 
    def test_twenty_nine_past(self):
        self._set_minute(29)
        self._time_should_be("half past nine")
 
    def test_half_past(self):
        self._set_minute(30)
        self._time_should_be("half past nine")
 
    def test_twenty_five_to(self):
        self._set_minute(35)
        self._time_should_be("twentyfive to ten")
 
    def test_twenty_to(self):
        self._set_minute(40)
        self._time_should_be("twenty to ten")
 
    def test_quarter_to(self):
        self._set_minute(45)
        self._time_should_be("quarter to ten")
 
    def test_ten_to(self):
        self._set_minute(50)
        self._time_should_be("ten to ten")
 
    def test_seven_to(self):
        self._set_minute(53)
        self._time_should_be("five to ten")
 
    def test_five_to(self):
        self._set_minute(55)
        self._time_should_be("five to ten")
 
    def test_nearly(self):
        self._set_minute(58)
        self._time_should_be("nearly ten")
 
    def test_thirteen_hundred(self):
        self._set_hour(13)
        self._time_should_be("one o'clock")
 
    def test_five_to_noon(self):
        self._set_hour(11)
        self._set_minute(55)
        self._time_should_be("five to noon")
 
    def test_noon(self):
        self._set_hour(12)
        self._time_should_be("noon")
 
    def test_five_to_midnight(self):
        self._set_hour(23)
        self._set_minute(55)
        self._time_should_be("five to midnight")
 
    def test_two_to_midnight(self):
        self._set_hour(23)
        self._set_minute(58)
        self._time_should_be("nearly midnight")
 
    def test_midnight(self):
        self._set_hour(0)
        self._time_should_be("midnight")
 
    def test_two_past_midnight(self):
        self._set_hour(0)
        self._set_minute(2)
        self._time_should_be("shortly after midnight")
 
    def test_five_past_midnight(self):
        self._set_hour(0)
        self._set_minute(5)
        self._time_should_be("five past midnight")
 
if __name__ == "__main__":
    unittest.main()
 

Additionally, it was also brought to my attention that I could expect nothing less from the product owners of our project(s). The idea again being that we have a bunch of product managers who are causing havok with requirements documents that lack quality.

Now there may be some truth to both of these things, but I find it somewhat confusing that anyone could put much value on incredibly detailed technical specification documents for source code. Especially when the vast majority of the work does not pertain to any sort of public API.

Don’t get me wrong, I think we need specifications, and they should be detailed. I just disagree with the format. What’s the point in writing something in a static document that will be shelved, collect dust and never be visited by anyone other than the poor sap who is forced to update it whenever something changes? There is no point; we should be designing our implementations through unit tests, using the tests as the specification. The benefits are substantial. Here are a few of the major reasons why I think that no technical specification should be anything other than tests:

executable

The single biggest benefit of unit tests are their executability. This allows for constant monitoring of adherence to the specification (the tests are the specification) by the software. Something a document can never provide.

safe

Tests provide a safety net of sorts that allows for incremental improvements to be made to the codebase with confidence. It’s trivial to become aware of regressions and failures with enough tests wrapped around the code.

demonstrative

This point is probably the only area in which a document can compete with a test. Unit tests give programmers who are attempting to learn a system some concrete examples of how the system actually works and how it was intended to be used. A very detailed document can provide this information as well, but what do you think is used to convey examples? That’s right, code. And if you are going to be writing code why not put it in a test suite and have it constantly verify the implementation rather than rot on a shelf (or in some sort of document repository)?

This leads me to a question; what is the common attitude by upper management? Do documents make high level managers feel warm and fuzzy about the quality of work being done by the developers, quality assurance staff and product managers? If so, why? Is it actually successful, or is it just less terrible than not doing documentation and not doing tests either?


#!/bin/bash
NAME=`basename $1 | sed 's/\(.*\)\..*/\1/'`
rst2latex $1 | latex 1>/dev/null
dvipdf texput.dvi
mv texput.pdf $NAME.pdf
rm texput.*


You’ll need dvipdf and latex available on your system for the script to work.

Billfrog was using the UDP library by Bjorn Hartmann to try to fetch time from time.nist.gov but hadn’t had much luck yet. This got me interested so I set myself to the task of figuring it out.

After several hours of hacking and learning about UDP and the NTP protocol I managed to fetch time from time.nist.gov. I had to cut some corners since the timestamps come back as 64bit floating point numbers. I decided to throw away the fractional portion and just keep the whole part as an unsigned long.

Some of the fun things you can do with this is synchronize your clock using the DateTime library by Mem. So far it seems like the clock is accurate to within a few seconds. I also played with periodic resynchronizing.

I decided to package it up into an easy to use library. The code is hosted on github.

One common rule of thumb is to plan to specify about 80 percent of the requirements up front.

Now, I’ve heard this many times before I read it in this book but I have a feeling Steve McConnel’s incredibly popular book was the root of the quote (in most of the cases anyhow). I guess my question is, what in the world is 80%? It seems to me that most folks I talk to are O.K. with this notion of specifying 80% of the requirements before beginning actual work.

Bear with me a moment. It struck me the other day that if one knows what eighty percent of something is, they must know what 100% of that something is. Maybe I’m just not getting it, but I’ve never worked on anything where I felt it was absolutely complete. I’ve stopped working on things before, but usually the software isn’t completely void of requirements; they just keep showing up.

If someone asked you to draw 80% of a line, what would you do? How would you know when you reached 80% of the length of the line? Would you ask them, “What is the total length of the line?”. Do you see where I’m going?

It seems to me that the most logical approach is to try very hard to keep your project flexible in the face of changing requirements and new requirements. It’s simply impossible to know the future.

That said, I do see a lot of value in doing some design up front, and that knowing when to stop designing and start coding is more of an art than a science. It’s good to build a plan if for no other reason than to think about what problems you do know.

• The Pragmatic Programmer
• Practices of an Agile Developer
• Head First Design Patterns
• Ship It!
• Release It!
• Don’t Make Me Think
• Effective Java
• Test-Driven Development
• Refactoring
• Learning the vi and Vim editors
• Git Internals
• Twisted Network Programming Essentials
• Pragmatic Version Control Using Git
• Pragmatic Thinking and Learning
• Implementing Lean Software Development: From Concept to Cash

The following are the books I have started reading but haven’t finished yet:

• Code Complete
• Interface-Oriented Design
• Java Concurrency In Practice
• The Definitive Guide to Django
• Agile Software Development

This list if of the books I want to read but don’t have yet:

• The Enterprise and Scrum
• Lean Software Development: an Agile Toolkit
• Programming Pearls

There are some terse instructions on how to do this at the arduino site.

The remote machine I’m using already had all the avr tools installed so I figured it’d be straightforward to get a sketch compiled and running, and I was mostly right.

The idea is to simply copy the Makefile packaged in the command line hacking guide tarball into your sketch folder and customize it for the sketch. This usually amounts to configuring the name of your sketch, the serial port you want to use, the microcontroller you are using and a few other options.

So I cracked open the Makefile and made the changes I thought were appropriate to run the Blink sketch. Here are the configurable bits in the Makefile that I started with:


TARGET = Blink2
INSTALL_DIR = ~/arduino-0012
PORT = /dev/ttyS0
UPLOAD_RATE = 19200
AVRDUDE_PROGRAMMER = stk500v1
MCU = atmega168
F_CPU = 16000000


I ran `make` and received a page full of errors. Apparently my install of the arduino IDE wasn’t configured just right for using this make file so I put things where the Makefile was looking.

This amounted to adding an tools/avr/bin and tools/avr/etc directory and symlinking in the executables necessary to build. These were:


avr-ar avrdude avr-g++ avr-gcc avr-objcopy avr-size


In the tools/avr/etc directory I put the avrdude.conf file that came with the ide install.

After doing that I ran `make clean all` again and was successful.


text data bss dec hex filename
0 1118 0 1118 45e applet/Blink2.hex


Now I wanted to upload to the freeduino. So I took another peek at the Makefile and found the ‘upload’ directive. So I gave it a shot:


avrdude: stk500_recv(): programmer is not responding
avrdude: stk500_recv(): programmer is not responding
make: *** [upload] Error 1


Bummer. Apparently you have to reset the thing before you can upload a new sketch. The newest IDE does this for you by toggling the DTS bit on the serial device before uploading a new sketch so I built a small python app to do that for me.


#!/usr/bin/env python
import time
import serial

ser = serial.Serial('/dev/ttyS0', 19200, timeout=1)
ser.setDTR(0)
time.sleep(0.1)
ser.setDTR(1)
ser.close()


The serial library is here; or in Debian you can do apt-get install python-serial.

So I toggle the bit and give it another shot. No go. What gives? At this point I ask Phizone to figure out exactly how the IDE is using avrdude and compare it to the Makefile.

IDE

/arduino-0012$ hardware/tools/avrdude -Chardware/tools/avrdude.conf -v -v -v -v -pm168 -cstk500v1 -P/dev/ttyS0 -b19200 -D -Uflash:w:/home/phizone/sketchbook/Blink2/applet/Blink2.hex:i

Makefile

/arduino-0012$ hardware/tools/avrdude -V -F -C hardware/tools/avrdude.conf -p atmega168 -c stk500v1 -P /dev/ttyS0 -b 19200 -U flash:w:/home/wcoyote/sketchbook/Blink2/applet/Blink2.hex

Well it isn’t terribly different, but it is different. So I hacked the Makefile again to make the commands the same:


AVRDUDE_WRITE_FLASH = -Uflash:w:applet/$(TARGET).hex:i
AVRDUDE_FLAGS = -v -v -v -v -C$(INSTALL_DIR)/hardware/tools/avr/etc/avrdude.
-p$(MCU) -P$(AVRDUDE_PORT) -c$(AVRDUDE_PROGRAMMER) \
-b$(UPLOAD_RATE) -D


And try it again…


avrdude done. Thank you.


And, according to phizone, I had a blinking LED.

One last note, I’m not sure why the -p option used by the IDE is m168 and not atmega168 but if you don’t use atmega168 compilation seems to fail.

1. List Comprehensions
2. Generator Expressions

They are both similar to one another; the main difference is that list comprehensions return a list and generator expressions return a generator.

Here are a few examples [1]:

>>> x = [chr(i) for i in range(65,91)]
>>> x
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O',
'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z']


This is a short and elegant way to write the following:

x = []
for i in range(65,91):
x.append( chr(i) )


Here’s the same thing in generator form:

>>> y = (chr(i) for i in range(65,91))
>>> y

>>> list(y)
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O',
'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z']

Here is a more pragmatic use of a generator expression.

>>> unencoded = "foo=bar&baz=boo"
>>> "".join("%s" % hex(ord(c))[2:] for c in unencoded)
'666f6f3d6261722662617a3d626f6f'


Here’s the above example in a more verbose format:

tmp = []
for c in unencoded:
bin_hex = "%s" % hex(ord(c))[2:]
tmp.append(bin_hex)
"".join(tmp)

And here is one final example with some math geekery:
>>> [x ** 2 + x + 41 for x in range(1,42)]
[43, 47, 53, 61, 71, 83, 97, 113, 131, 151, 173, 197, 223, 251, 281, 313,
347, 383, 421, 461, 503, 547, 593, 641, 691, 743, 797, 853, 911, 971, 1033,
1097, 1163, 1231, 1301, 1373, 1447, 1523, 1601, 1681, 1763]


Footnotes

[1]

The first two examples are a little contrived. These tasks might be simpler solved by using the map builtin or the imap function in the itertools module.

>>> map(chr, range(65, 91))
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N',
'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z']


Just as Ubuntu Gutsy Gibbon was released I decided to write a script that converts vim themes to gtksourceview2 format. I opened up a google code project for it and set up the svn repo and uploaded an archive. The script is very simple to use and works well for me.

Here is the simple syntax for its use:
./vim2gtksourceview.py < input.vim > output.xml

Disclaimer: many vim themes do not implement many features provided by both vim and gtksourceview, (current line highlighting being a big one) so the converted themes may need a little tweaking afterward.