Confessions of a Wall Street Programmer

practical ideas (and perhaps some uncommon knowledge) on software architecture, design, construction and testing

Guess What Day It Is!

No, not that – it’s Perl day. (Well, actually it’s just Wednesday, but you get the idea).

Sometimes it seems that everybody likes to hate on Perl, but I think their animus is misdirected. It’s not Perl that’s the problem, it’s those \^\$(.#!)?$ regular expressions.

Or, as Jamie Zawinski once said “Some people, when confronted with a problem, think “I know, I’ll use regular expressions.” Now they have two problems.”.

Well, I’m here to tell you that it’s possible to write whole Perl programs that actually accomplish useful work, without any regular expressions at all! And, if you do that, you can actually read the code!

It turns out that Perl is a dandy scripting language, and while some may take issue with its flexibility (“There’s more than one way to do it”), others (including me) find that flexibility very useful.

Where Am I?

From Robinson Crusoe to Gilligan’s Island to Lost, tales of being stranded on a desert island seem to resonate with people in a special way. Some of that likely has to do with the exotic locales, and the practical challenges of getting water, food and shelter.

But an even more basic part is the unanswered question: “Where am I?” that makes things so – well, mysterious.

Shell scripting can be pretty mysterious too at times, but in this installment we’ll learn how to answer that basic question of “Where am I?” to make shell scripting a little less mysterious.

Visualizing Latency

I’m a visual thinker (I think I may have mentioned that before ), so when I’m analyzing performance, latency, etc. I find it really helpful to be able to visualize what is going on on the machine.

As a result, I had gotten reasonably good at using Excel to produce charts, which sometimes helped to correlate observed behaviors like latency spikes with other events on the machine.

For a bunch of reasons I wanted to move away from Excel, though, and find another tool that would give me the same or better functionality.

A Picture is Worth 1K Words

You know those mutiple-choice tests that put you in one of four quadrants based on your answers to a bunch of seemingly irrelevant questions? We’ve all taken them, and if you’re like me they’re kind of like reading your horoscope – it all seems so right and true when you’re reading it, but you wonder if it would still seem just as right and true if the horoscopes got jumbled at random?

Well, I took one of these tests a while back that actually told me something about myself – it was the “Learning-Style Inventory” test, and what it said about me is that I’m waaaayyy over at the end of the scale when it comes to visual thinking. That gave me an insight into the way my brain works that I’ve found really helpful ever since. So, this next bit was right up my alley, but I’m guessing you’ll like it too.

We read a lot lately about NUMA architecture and how it presents a fundamental change in the way we approach writing efficient code: it’s no longer about the CPU, it’s all about RAM. We all nod and say “Sure, I get that!” Well, I thought I got it too, but until I saw this web page, I really didn’t.

See the full discussion at http://overbyte.com.au/index.php/overbyte-blog/entry/optimisation-lesson-3-the-memory-bottleneck.

Using clang’s Address Sanitizer (without clang)

Valgrind has been an indispensable tool for C/C++ programmers for a long time, and I’ve used it quite happily – it’s a tremendous tool for doing dynamic analysis of program behavior at run time. valgrind1 can detect reads of uninitialized memory, heap buffer overruns, memory leaks, and other errors that can be difficult or impossible to find by eyeballing the code, or by static analysis tools. But that comes with a price, which in some cases can be quite steep, and some new tools promise to provide some or all of the functionality valgrind provides without the drawbacks.

Measuring Latency in Linux

For measuring latency in modern systems, we need to be able to measure intervals in microseconds at least, and preferably in nanoseconds or better. The good news is that with relatively modern hardware and software, it is possible to accurately measure time intervals as small as (some smallish number of) nanoseconds.

But, it’s important to understand what you’re measuring and what the different edge cases might be to ensure that your results are accurate.

You may ask yourself - “How did I get here?”

In addition to being a great line from David Byrne and Talking Heads (from “Life During Wartime”), this is also a question I often ask myself when looking at log files. Today’s tip is worth the price of the whole blog (i.e., free), but I predict that you’ll be glad you know it.

Shared Singleton’s

Singleton’s are kind of like Rodney Dangerfield – they don’t get no respect. And yet, there are scenarios where a singleton is just the ticket – for instance, when a relatively expensive and/or long-lived resource needs to be shared among a number of independent threads.

shared_ptr’s can help make this easier and less error-prone, but even so there are some edge cases that need to be considered.

FIXing less

Here’s a handy tip for those who (like me) spend a fair amount of time staring at FIX logs.