Preserving Our Digital Pre-History

I've spent a significant part of my life online. Not just on the internet, I mean, but on modems and early, primitive online communities. Today's internet is everything we couldn't have possibly dared to imagine twenty-five years ago, but there is a real risk of these early, tentative digital artifacts -- and for some, the beginnings of our Hacker Odyssey -- being lost forever in the relentless deluge of online progress. Sure, every single thing that happened in 2004 is documented exhaustively online. But 1994? 1984? Not so much.

That's where Jason Scott comes in.

You may know Jason Scott from BBS The Documentary. Or, perhaps you're familiar with textfiles.com, his massive (and growing) archive of what passed for blogs and forums in the earliest online era.

A wonderful thing happened in the 1980s: Life started to go online. And as the world continues this trend, everyone finding themselves drawn online should know what happened before, to see where it all really started to come together and to know what went on, before it's forgotten.

When a historian or reporter tries to capture the feelings and themes that proliferated through the BBS Scene of the early 1980's, the reader nearly always experiences a mere glimpse of what went on. This is probably true of most any third-party reporting, but when the culture is your own, and when the experiences were your own, the gap between story and reality is that much wider, and it's that much harder to sit back and let the cliche-filled summary become "The Way It Was." You want to do something, anything so that the people who stumble onto the part of history that was yours know what it was like to grow up through it, to meet the people you did, to do the things you enjoyed doing. Maybe, you hope, they might even see the broader picture and the conclusions that you yourself couldn't see at the time. This is history the way the chronicled want it to be.

Jason is nothing less than our generation's digital historian in residence. When GeoCities went permanently offline a week ago, he was there to help preserve it for posterity.

BBS: The Documentary was a major milestone in his ongoing effort to document our digital pre-history. But it's only the beginning; there's also a huge documentary on text adventures, Get Lamp, that's been in the works for a few years now. Unfortunately, progress has been slow. Because while being a digital historian is great, it's not exactly something you get paid to do.

But maybe we can change that. Witness Jason's kickstarter proposal:

Throughout all this, I had a day job - computer administration. It paid well, but I paid for it with my health. When my most recent employer and I parted ways, I decided I'd take this time finish some of the bigger projects I've been working on.

I suddenly thought back to Kickstarter and got this crazy idea - what if I simply asked the world and fans to contribute a bit of money towards keeping me somewhat solvent, and give me the opportunity to go full-time with computer history? If I was able to get all these things done over the years, what if I just asked people to subscribe or give me some patronage and in return I fill their free time with cool stuff to look at, learn from, and enjoy?

There are so many people whose online presences I greatly admire. But very few of them will go on to become part of the permanent written history of this era. I have no doubt whatsoever that Jason Scott is one of those people who will, thanks to his tireless efforts to preserve the flotsam and jetsam of our digital past, stuff that would otherwise be overlooked by the mainstream and lost forever.

I've pledged $100. It is an honor to support his ongoing work of preserving our shared digital pre-history. His history, is my history, is our history. A history of geeks, dorks, dweebs, nerds, and generally computer-obsessed misfits, but nonetheless -- it's something we all share.

If this is something you believe in, I urge you to pledge as well.

Stack Overflow Careers: Amplifying Your Awesome

That Stack Overflow thing we launched a year ago? It's been going pretty well so far.

Of course, everyone knows you could code Stack Overflow in a long weekend. It's trivial. Assembling a worldwide community of smart, engaged software developers? That's a whole different ball of wax. Stack Overflow is a site by programmers, for programmers; it's only as good as the programmers who choose to participate.

Stack Overflow isn't about me. Or anybody else on the Stack Overflow team for that matter.

Stack Overflow is you.

This is the scary part, the great leap of faith that Stack Overflow is predicated on: trusting your fellow programmers. The programmers who choose to participate in Stack Overflow are the "secret sauce" that makes it work. You are the reason I continue to believe in developer community as the greatest source of learning and growth. You are the reason I continue to get so many positive emails and testimonials about Stack Overflow. I can't take credit for that. But you can.

I learned the collective power of my fellow programmers long ago writing on Coding Horror. The community is far, far smarter than I will ever be. All I can ask – all any of us can ask – is to help each other along the path.

I am continually humbled by the skill and expertise of the programmers who volunteer time to Stack Overflow. These programmers graciously donate tiny slivers of their day to help us -- and themselves -- become better programmers. These 5 and 10 minute slices of effort, across hundreds of thousands of questions and answers, become a permanently archived (and creative commons wiki licensed) bread crumb content trail for future programmers to follow, edit, and contribute to themselves over time.

I'm thrilled to see Stack Overflow working so well for both askers and answerers; the "pay it forward" model of programmers helping their peers is exactly what we were shooting for. We'll never change the world, but it sure is nice to be able to improve our small corner of it just a little bit. Remember: bad code that isn't written, is bad code that another poor programmer won't have to debug. If we don't reach out to slaphelp new programmers and teach them the lessons we learned the hard way, who will? I'm only exaggerating a little when I say that the future of our entire profession depends on it.

If you're actively participating on Stack Overflow, we now have another way to convert those slices of effort into something that actively furthers your professional goals – Stack Overflow Careers.

What is careers.stackoverflow.com? It's a few things:

• a completely free, public CV hosting service for programmers, to share the cool stuff you've coded and created with the world.
• a way to explicitly link your Stack Overflow profile with your CV, to provide concrete examples of your communication skills and individual expertise to anyone who is interested.
• a better way to connect great programmers with the best programming jobs, for those who opt into the small annual listing fee.

In short, Stack Overflow Careers amplifies your awesome.

I won't lie to you. This is also a business. That's why there are nominal opt-in listing fees for those programmers interested in seeking employment, and substantial fees for hiring managers who want to tap into the smart developers who grok Stack Overflow.

update: I apologize if I wasn't clear. It is 100% free, forever, to create a public CV, put whatever HTML content you want in it, and link it to your Stack Overflow profile. Like so:

• http://careers.stackoverflow.com/klmr
• http://careers.stackoverflow.com/jongalloway
• http://careers.stackoverflow.com/brandan
• http://careers.stackoverflow.com/brentkeller
• http://careers.stackoverflow.com/mfoord

These are of course freely indexable and searchable on the web.

Beyond the free public component, there is a private (and completely optional) subscription component. For those programmers actively seeking employment, a small annual subscription fee allows inclusion in a private employer search UI. This is also explained in the faq and about.

That said, we're also trying to do something a bit different here. Something better than the endless, mind-numbing acronym sea of monster.com, dice.com, et al. Joel and I believe current hiring practices for programmers are incredibly broken. We think we can do better.

We love our work, and so should you. Our goal isn't to put warm bodies in front of interviewers. Our goal is to create love connections. Instead of avid programmers pursuing disinterested and distracted companies, it's the other way around -- savvy companies who understand the competitive advantages of having the best programmers will pursue you. We connect smart, engaged hiring managers who "get it" with top programmers who love to code.

If you love to code, too, I encourage you to create your own Stack Overflow CV. Keep it private, or make it public via the URL of your choice -- it's completely free either way. If you think you might be actively looking for a job in the next 3 years, take advantage of our outrageously low promotional pricing of $29 for a 3 year filing. That way, at any point in those 3 years, you can flip a switch and become visible to hiring managers. Or not. It's totally up to you.

(also, if you're hiring, and your company appreciates top software engineers -- and you think you can convince our tough audience of that -- email us)

Revisiting "The Fold"

After I posted my blog entry on Treating User Myopia I got a lot of advice. Some useful, some not so useful. But the one bit of advice I hadn't anticipated was that we were not making good use of the area "above the fold". This surprised me. Does the fold still matter?

The fold refers to the border at the bottom of the browser window at the user's default screen resolution. Like so:

Way back in the dark ages of 1996, it was commonly thought that users didn't know how to scroll a web page.

On the Web, the inverted pyramid becomes even more important since we know from several user studies that users don't scroll, so they will very frequently be left to read only the top part of an article.

Thus, it was critically important to cram in as much content in as possible above that fold, as anything below it was invisible to a huge number of users. They didn't know how to scroll, so they would never find it. Jacob Neilsen, renowned usability expert, is the author of the above quote. But he recanted his position in 2003:

In 1996, I said that "users don't scroll." This was true at the time: many, if not most, users only looked at the visible part of the page and rarely scrolled below the fold. The evolution of the Web has changed this conclusion. As users got more experience with scrolling pages, many of them started scrolling.

Scrolling is an example usability versus learnability. It was always my belief that users quickly learned to scroll, otherwise they were permanently crippled as web citizens. If you can't learn to scroll within an hour or so of using the web, you're going to have an awfully stunted experience -- so much so that you're probably better off not using it at all. In short, if you use the web, you know how to scroll, almost by definition. It is a fundamental skill.

Even today, people will cite the ancient, irrelevant rule of The Fold as if it's still law. In fact, I was just talking to a friend of mine who expressed his frustration at dealing with a middle manager who was using the "content must be above the fold" rule as a weapon, and demanding that all page content appear above the fold. It's terribly misguided.

Although thoroughly debunked, there are still some hidden dangers from the fold, and subtlety to how users react to it. As documented by a recent usability study on the fold, there are three specific pitfalls to watch out for:

1. Don't cram everything in above the fold. Users will explore and find your content -- as long as the page "looks" scrollable.
2. Watch out for stark, horizontal lines that happen to line up with the fold. This is the only factor that causes users to stop scrolling, because the page looks done and complete. Instead, have a small amount of content just visible, poking up above the fold. This encourages scrolling.
3. Avoid in-page scroll bars. The standard browser scrollbar is an indicator of the amount of content on the page that users learn to rely on. Placing <iframe> and other elements with scroll bars on the page can break this convention -- and may lead to users not scrolling.

These are excellent guidelines, backed by actual eye tracking and experimental results. You know, science! But how do they apply to me? First, I established where the fold actually was. Per Google Analytics, about 25% of our users are using screen resolutions where the page fold is at about 700 or 800 pixels of height. And remember, browsers have a lot of horizontal chrome that tends to squander that height -- toolbars, status bars, tabs, etcetera. The fold is probably much closer than you think it is.

Next, I looked at the advice I had been given regarding the top of the page. Sure enough, we had a bunch of irrelevant UI at the top that didn't really matter: things like redundant page titles, and two line title entry. We were wasting critical real estate at the top of the page! For the 25% of users who have a 700 or 800 pixel fold, items were pushed down far enough that they might not actually be visible. Worse still, the strong bottom border of the text entry area with the drag slider could possibly align with the page fold itself -- leading the user to believe that nothing is below there and failing to scroll.

It's not only a basic rule of writing, it's also a basic rule of the web: put the most important content at as close to the top of the page as you can. This isn't new advice, but it's so important that it never hurts to revisit it periodically in your own designs.

In treating user myopia, it's not enough to place important stuff directly in the user's eyepoint. You also need to ensure that you've placed the absolute most important stuff at the top of the page -- and haven't created any accidental barriers to scrolling, so they can find the rest of it. The fold is far less important than it used to be, but it isn't quite as mythical as Bigfoot and the Loch Ness Monster quite yet.

Treating User Myopia

I try not to talk too much about the trilogy here, because there's a whole other blog for that stuff. But some of the lessons I've learned in the last year while working on them really put into bold relief some of my earlier blog entries on usability and user behavior.

One entry in particular that I keep coming back to is Teaching Users to Read. That was specific to dialog boxes, which not only stop the proceedings with idiocy, but are their own delightful brand of user interface poison. Fortunately, you don't see dialogs in web apps much, but this sort of modal dialog lunacy is, sadly, becoming more popular in today's AJAX-y world of web 2.5. Those who can't learn from history are doomed to repeat it, I guess.

Having five more years of development experience under my belt, I no longer believe that classic Larson strip is specific to dialog boxes.

The plain fact is users will not read anything you put on the screen.

What we're doing with the trilogy is not exactly rocket surgery. At its core, we run Q&A websites. And the most basic operation of any Q&A website is … asking a question. Something any two year old child knows how to do.

When we launched superuser.com, that was our thirdfourth Q&A website. This one is for power users, and it's the broadest to date, topic-wise: anything dealing with computer software or hardware (that isn't gaming) is allowed.

We've been at this for over a year now, doing nothing but relentlessly polishing and improving our Q&A engine based on community feedback. We're not particularly good, but we do try very, very hard not to suck. I thought surely, surely we must have something as simple as the ask question form down by now.

How foolish I was.

Let's take a look at one recent superuser question. I'm presenting it here as it would have been seen by the user who asked the question, while they were entering it on the ask question form.

Immediately, there's a problem. The question formatting is completely wrong! It's one big jumble of text.

Our formatting rules aren't complicated. You can get a lot done with a bunch of simple paragraphs. We use Markdown, which offers basic formatting conventions that ape ASCII conventions. On top of that, we offer a real-time preview of how your question will look once submitted, directly under the question entry area. But none of that seemed to work for this particular asker, who, apparently, was totally satisfied with obviously broken formatting -- even though a few choice carriage returns would have worked wonders, and been immediately visible in the live preview.

Yes, yes, it inevitably gets whipped into shape through the collective efforts of our legions of community editors -- but that's not the point. It's best if the original asker gets the question formatted right to start with, and it is our job as UI designers to make that outcome as statistically likely as we can.

To that end, we've put a bunch of helpful tools on the ask question page to help users get the formatting right. As UI designers, here's how we see the ask question page:

We've provided a toolbar with a neon pink help button above the question body, and to the right of the question body, we've provided a handy formatting quick reference with a link to the full formatting reference (which opens in a tab / new window by default).

But none of that matters, because here's how the user sees the ask question page:

Or rather, here's everything the user doesn't see.

When I said users don't read anything you put on the screen, I was lying. Users do read. But users will only read the absolute minimum amount of text on the screen necessary to complete their task. I can't quite explain it, but this kind of user myopia is epidemic. It's the same problem, everywhere I turn.

How do we treat user myopia? How do we reach these users? The ask question page is already dangerously close to cluttered with helpful tips, but apparently these helpful buttons, links, and text are all but invisible to a large segment of the user population. Sure, you could argue that Super User tends to attract less sophisticated users, but I see the exact same problem with programmers on Stack Overflow. As new users, a significant percentage of them can't figure out how to format code, even though there's not only a toolbar button that does it for you, but help text on the right explicitly describing how to do it manually. (Just indent 4 spaces. Spoiler alert!)

More and more, I'm thinking we need to put the formatting help -- for new users only -- directly in their line of sight. That is, pre-populate the question entry area with some example formatting that is typical of the average question. Nothing complicated. But at least then it'd be in the one -- and apparently the only one -- place myopic users are willing to look. Right in front of their freakin' faces.

The next time you're designing a UI, consider user myopia. You might be surprised just how myopic your users can be. Think long and hard about placing things directly in front of them, where they are not just visible, but unavoidable. Otherwise they might not be seen at all.

The Interview With The Programmer

If the internet has perfected anything, it's the art of the crappy, phoned-in, half-assed email "interview". For all those who have bemoaned the often pathetic state of internet journalism, when it comes to interviews, you're largely correct. The purpose of most of these interviews is quick and dirty content filler with semi-famous folk spouting off whatever random thoughts they happen to have in their head at that exact moment. The Nixon Interviews, it ain't.

That's why I'm normally not a huge fan of interview books, because interviews take an enormous amount of time and an enormous amount of legitimate, skilled journalistic effort to get right. Almost nobody does.

Imagine my surprise when Coders at Work: Reflections on the Craft of Programming turns out to be that wonderfully rare intersection of uncommonly skilled interviewing and 15 of the most influential programmers to ever touch a keyboard.

Yes, this is the same book Joel recently recommended in his controversial Duct Tape Programmer entry, which is why I was all the more skeptical. But he's dead on. I could (and probably will, knowing me) fill a year worth of blog posts just with the thought provoking quotes and two-paragraph insights revealed in these interviews. It's astonishingly good. If, after reading what these brilliant programmers have to say, you aren't motivated to research some programming topic mentioned inside, pack it in, because you aren't even trying any more.

I also realized Coders at Work can potentially serve as a job interview filter. If the next programmer you interview can't identify at least one of the programmers interviewed in Coders at Work and tell you roughly what they're famous for …

… I'd say that's an immediate no-hire.

Incidentally, I saw the first Stack Overflow user reference on page 265, in the interview with Simon Peyton Jones, who mentions one Norman Ramsey. Hmm, I thought, that name sounds awfully familiar. And indeed it was!

I would be remiss if I did not mention that the author, Peter Seibel, was directly inspired by Susan Lammers' classic 1986 book Programmers at Work: Interviews With 19 Programmers Who Shaped the Computer Industry.

This is one of my absolute favorite musty old computer books for many of the same reasons. As sources of inspiration go, this one is particularly … er, inspired. Programmers at Work isn't just the archetypal programmer interview book -- it also holds up amazingly well for a book that is over twenty years old. It is a testament to the timelessness of not just code, but the art of coding, as exemplified by these 19 programmers. I believe Peter has legitimately crafted a modern remake that will be relevant for another twenty years. And I hope I don't have to tell you how extraordinarily rare that is among technical books.

(Some -- but not all from what I can tell -- key interviews from Programmers at Work were placed online last year by the author. So if you want to get a flavor of the book, check it out.)

Another notable recent collection of interviews is Masterminds of Programming: Conversations with the Creators of Major Programming Languages.

Although I definitely enjoyed this book, there's something about the focus on programming languages and interview style that didn't quite grab me as forcefully as Coders at Work did. Also, if we're going to do languages, I'd like to see a bit broader representation -- perhaps a Volume II with Smalltalk, Ada, Pascal and so on?

These books are a potent reminder that computers are mostly a reflection of the people using them. In the art of software development, studying code isn't enough: you have to study the people behind the software, too.