And on the seventh day He said, "Oh, one more thing..."
-Jobs 3:15
Tuesday, October 25, 2005
Wednesday, October 12, 2005
A nerdish way to spend an evening
- Get a laptop
- Install stellarium
- Go outside and bring:
- binoculars or a telescope if you have one
- a bottle of wine (or beer)
- cheese
- a blanket
Launch stellarium, see the sky. Enjoy.
I'm human, so sue me
Developers 'should be liable' for security holes - ZDNet UK News:
Making developers liable for security exploits is a stupid idea. The first problem with it is, who do we blame? Most of the security bugs nowadays are not your typical buffer overflow kind, which could have been prevented with careful programming. Most of the security bugs today have to do with interactions between system components. Let me explain.
There are two kinds of bugs, security or otherwise. Bugs that reflect the programmers lack of knowledge of a well established coding practice or API usage, and bugs that unless you have actually written the same code before in the same circumstances, you could not have predicted would occur.
If a web programmer takes a string from a user in a web form, and then passes that string unquoted to a shell, that programmer shouldn't be allowed within 15 feet of a web app. If a programmer writes a piece of code that interacts with other pieces of code, and the resulting interaction has a bug (which might be exploitable), we have an interaction bug. It's one of those "How could I have known?" moments we've all had. And if you sit and think about all the possible interactions before writing any code, you never get anything done.
What about liability? Assuming you have a sensible version control system in place, you can ask the question "Who wrote this pice of crap?" and get an answer. But who do you blame when the security bug is a result of a combination of changing APIs, implementing new features, fixing old bugs, etc. The answer to "Who wrote this piece of crap?" is "a group of programmers working at different times". For the web app example, some of the code might have been written when it wasn't a web app at all. There is human history in the code. It was written by humans.
And humans make mistakes. There is no way to avoid that. We just screw up every once in a while. Depending on how much you have to pay for your mistakes, you might be afraid of doing anything daring again. For instance, if I write a little application (single author, so all the bugs are mine), and get sued because it has a security bug, guess how many more applications I'll be writing. None. The cost of making a mistake is too expensive.
If companies are liable, you're just moving the problem to management. Companies get sued, so they are hesitant to put new applications out there. Applications that are not being used are not being debugged, and are thus less secure.
Markus Ranum wrote a great piece about this subject called Inviting Cockroaches to the Feast. It's a very good read.
Making developers liable for security exploits is a stupid idea. The first problem with it is, who do we blame? Most of the security bugs nowadays are not your typical buffer overflow kind, which could have been prevented with careful programming. Most of the security bugs today have to do with interactions between system components. Let me explain.
There are two kinds of bugs, security or otherwise. Bugs that reflect the programmers lack of knowledge of a well established coding practice or API usage, and bugs that unless you have actually written the same code before in the same circumstances, you could not have predicted would occur.
If a web programmer takes a string from a user in a web form, and then passes that string unquoted to a shell, that programmer shouldn't be allowed within 15 feet of a web app. If a programmer writes a piece of code that interacts with other pieces of code, and the resulting interaction has a bug (which might be exploitable), we have an interaction bug. It's one of those "How could I have known?" moments we've all had. And if you sit and think about all the possible interactions before writing any code, you never get anything done.
What about liability? Assuming you have a sensible version control system in place, you can ask the question "Who wrote this pice of crap?" and get an answer. But who do you blame when the security bug is a result of a combination of changing APIs, implementing new features, fixing old bugs, etc. The answer to "Who wrote this piece of crap?" is "a group of programmers working at different times". For the web app example, some of the code might have been written when it wasn't a web app at all. There is human history in the code. It was written by humans.
And humans make mistakes. There is no way to avoid that. We just screw up every once in a while. Depending on how much you have to pay for your mistakes, you might be afraid of doing anything daring again. For instance, if I write a little application (single author, so all the bugs are mine), and get sued because it has a security bug, guess how many more applications I'll be writing. None. The cost of making a mistake is too expensive.
If companies are liable, you're just moving the problem to management. Companies get sued, so they are hesitant to put new applications out there. Applications that are not being used are not being debugged, and are thus less secure.
Markus Ranum wrote a great piece about this subject called Inviting Cockroaches to the Feast. It's a very good read.
Saturday, October 08, 2005
First BitMover Regatta
Last week we had the First BitMover Regatta. On Tuesday, BitMover rented a bunch of sailboats, and a group of employees raced across the San Francisco Bay. It was my first time on a sailboat, and I had an excellent time. I had been on a Yacht before, deep sea fishing, but sailboats are an entirely different experience. Now I understand why Evi is sailing across the world. Sailing is too cool.
At around noon, we drove to Sausalito, where the sailboats were waiting for us in the dock. We divided in 3-person teams, and each team got a sailboat and a skipper. The skipper's role was giving us directions and basically taking care of us since not many
people had sailed before. At the dock, we found our sailboat, a 42-foot Bēnēteau 423. The Bēnēteau 423 was one of the top 10 sailboats of 2003 according to Sail Magazine, and it's price is around $165,000. Not bad at all! Upon boarding the sailboat, we met Stan, our skipper, who had been in the Coast Guard for 20 years before becoming a sailing instructor for the Modern Sailing Academy in Sausalito. He explained all the boat terminology (port is left, starboard is right when facing the ship's front, or rather, the bow). And taught us how to work the sails. We used the motor to get out of the docks, hung around the start line until the race started, and set sail towards San Francisco.
Stan asked for a volunteer to take the helm (steering wheel) and I immediately stepped forward (knowing that working the helm would be easier than pulling all the
ropes). Since I had never been in a sailboat, I didn't expect it to lean that much. It's a little scary at first, but I was assured by Stan that they don't tip over under normal conditions (i.e. not in a hurricane). Driving that thing was fun, but it required a lot of attention. Basically if you let your attention drift for 10 seconds, you would slightly change course and lose the lift. If you lose the lift often enough, your skipper can order a Keelhauling, which I've been told is not pleasant. I must have done allright because we were going at about 3 knots towards San Francisco.
It took us a while to find the buoy where we were supposed to turn, but after a while, we found it and headed towards the bay bridge. We were going with the wind, which is really cool because you can go really, really fast. The course took us around Alcatraz, the famous prison where Al Capone was an inmate.
to the other side. It was amazing being under the bridge. We got a little taste of the ocean too, with a slightly rougher sea than what the bay had been. After a quick turn, we got back to the dock, returned the sailboat and called it a day.
Once we were in the land again, we were tired and hungry. We headed to the Buckeye Roadhouse in Mill Valley for dinner, and what a dinner we had! If you ever go to the Buckeye, try the Ahi tuna appetizer. It's unbelievably good.
After this first taste of sailing, I'm left wanting to learn how to sail. Who knows, maybe when I retire, I'll buy a sailboat and follow Evi's course around the World.
At around noon, we drove to Sausalito, where the sailboats were waiting for us in the dock. We divided in 3-person teams, and each team got a sailboat and a skipper. The skipper's role was giving us directions and basically taking care of us since not many
people had sailed before. At the dock, we found our sailboat, a 42-foot Bēnēteau 423. The Bēnēteau 423 was one of the top 10 sailboats of 2003 according to Sail Magazine, and it's price is around $165,000. Not bad at all! Upon boarding the sailboat, we met Stan, our skipper, who had been in the Coast Guard for 20 years before becoming a sailing instructor for the Modern Sailing Academy in Sausalito. He explained all the boat terminology (port is left, starboard is right when facing the ship's front, or rather, the bow). And taught us how to work the sails. We used the motor to get out of the docks, hung around the start line until the race started, and set sail towards San Francisco.Stan asked for a volunteer to take the helm (steering wheel) and I immediately stepped forward (knowing that working the helm would be easier than pulling all the
ropes). Since I had never been in a sailboat, I didn't expect it to lean that much. It's a little scary at first, but I was assured by Stan that they don't tip over under normal conditions (i.e. not in a hurricane). Driving that thing was fun, but it required a lot of attention. Basically if you let your attention drift for 10 seconds, you would slightly change course and lose the lift. If you lose the lift often enough, your skipper can order a Keelhauling, which I've been told is not pleasant. I must have done allright because we were going at about 3 knots towards San Francisco.It took us a while to find the buoy where we were supposed to turn, but after a while, we found it and headed towards the bay bridge. We were going with the wind, which is really cool because you can go really, really fast. The course took us around Alcatraz, the famous prison where Al Capone was an inmate.
Seeing Alcatraz up close was quite a thrill. I have wanted to go to Alcatraz since 1994, when I first visited San Francisco, but because of various reasons, I have never taken a tour of "The Rock". After we went around Alcatraz, we headed back towards Sausalito, and the finish line. During that last segment, even though we were going more or less against the wind, we did a pretty good time. We won the race, crossing the finish line 7 minutes before the next sailboat.
to the other side. It was amazing being under the bridge. We got a little taste of the ocean too, with a slightly rougher sea than what the bay had been. After a quick turn, we got back to the dock, returned the sailboat and called it a day.Once we were in the land again, we were tired and hungry. We headed to the Buckeye Roadhouse in Mill Valley for dinner, and what a dinner we had! If you ever go to the Buckeye, try the Ahi tuna appetizer. It's unbelievably good.
After this first taste of sailing, I'm left wanting to learn how to sail. Who knows, maybe when I retire, I'll buy a sailboat and follow Evi's course around the World.
Technorati Tags: BitKeeper, BitMover, Sailing, San Francisco
Sunday, October 02, 2005
Yet Another Stupid Idea
Single-play DVDs will never work, at least not for the rental market. Not only because they require new players, and of course the first thing I will do is get a new DVD player so that I can watch the new rentals only once. But because they don't interact well with how people watch movies. I often fall asleep while watching a movie, how would I see the ending? How about that phone call, or little emergency right in the middle of the movie? Or when I don't have 2 hours to watch the entire movie, but I'm willing to see 30 minutes now and the rest later?
People rent because it's more flexible than going to the movies. Taking away from people the flexibility of renting without giving them any of the advantages of movie theaters is not going to fly.
People rent because it's more flexible than going to the movies. Taking away from people the flexibility of renting without giving them any of the advantages of movie theaters is not going to fly.
Unix died in 1982
If you look at this time history of Unix, you can see how after 1982 Unix became practically impossible to support. I mean, who was going to worry about compatibility between so many different flavors of Unix. I'm glad the list is short again in 2005. Now if only the GUIs had a standard ;-)
Friday, September 30, 2005
Judge this Book by its Cover
Or, how I learned the history of the computing industry and Unix by looking at the cover of a book.
Here's the deal. For reasons best left unexplained, I have every edition of the Unix System Administration Handbook by Evi Nemeth, et al. I even have the Linux System Administration Handbook. Since every book's edition is a different color, my daughter likes to play with them. The other day, while putting them back on the shelf, I noticed that the cover of the book reflects the history of the computer industry, the history of Unix, and a little history about Evi. So without further ado, here's my explanation of what it all means. You'll have to excuse the quality of the images, but I don't have a scanner so I took digital pictures.
Unix System Administration Handbook, 1st Edition
This book was published in 1989. We see that UNIX system administration is a difficult topic. There is a Unix system administrator driving his car towards a cliff. I don't know what the reference to cranberry blogs is about, but let's focus on the colorful characters.
Starting rom the left, we have a dog named biff, which is a reference to the biff(1) command in Unix that prints a message to your terminal when you have new mail. In the center is a daemon, a Unix process that runs in the background. Since the daemon is showing us his watch, maybe it's the NTP daemon? At the right of the daemon, you can see the finger(1) command, used to find information about a user and made famous by the Great Worm of '88. I don't know what the other two characters (the owl, and the cat, or is it perhaps a lady daemon?) are about. If anyone knows, or has a guess, let me know.
Unix System Administration Handbook, 2nd Edition
In the second edition, published in 1995, biff is dead (you can see the gravestone that says "R.I.P biff R.I.P"), probably replaced by POP. The cross behind reads "USL" and is a reference to the Unix System Laboratory at AT&T, the group responsible for System V, which had been sold to Novell in 1993.
In the front of biff's grave, it says "Here lies the entire Berkeley CSRG". That means the Computer Science Research Group, which was responsible for the development of the Berkeley version of Unix, called BSD, at the University of California and was disbanded in 1994.
The first edition of the book is being held by the paw of a cat. We can tell it's the first version because it's yellow, and it says "USAH", short for Unix System Administration Handbook.
The dog, Biff, is now gone from the front row. He's dead, remember? And has been substituted by a Monkey, a spider, and a Web. I don't know what the monkey has to do with anything, but the web is clearly a reference to the World Wide Web, which was invented in 1993. The spider is clearly a reference to the web spiders, or bots that search engines use to index web sites. We can still see the daemon, the finger, and a new character, Dr. SNTPd, a reference to the Simple Network Management Protocol.
Note that the car is more modern, and it has the word "Unix" written on the front. Also the System Administrator behind the wheel appears to be a woman. Could this be a reference to more women in system administration?
Unix System Administration Handbook, 3rd Edition
The third edition, published in 2001, has many of the same symbols as the second edition. In this edition, however, there's a new tomb. The Unix to Unix Copy Protocol, or UUCP, has died because most people use PPP by now. We can also see the second edition coming out of the grave being held by a tentacle.
Two more things should be noted. First, this is the first edition of the book to cover Linux, a newcomer to the Unix world, and you can see the Penguin in the co-pilot seat of the car. Also, the spider now has handcuffs, which might be a metaphor for the restrictions many sites started putting in place to prevent search engines from finding them.
Linux System Administration Handbook
This book came out in 2002, and was made specifically for Linux, which had by then surpassed in popularity all of the other Unix distributions. You can see the Linux Penguin driving now, and the sys admin as the co-pillot.
The spider's handcuffs have been replaced by a shackle, meaning the restrictions on spiders are greater now. The other books are gone, as Linux has been declared the only winner.
You can also see that the sign that used to read "Beware! Vendor Gratuitous Changes Ahead!" has been replaced by one that has Unix somewhat deviated, but still going up, Linux going straight up, and XP (a reference to Windows XP) going down. The helicopter that is crashing also has XP written on it. The authors have no love for Microsoft's Operating System.
There is also a penguin skiing on the mountain. I have no idea what that means.
The last interesting bit is that there is a sailboat in the back. Let's take a closer look at that sailboat.
See the name on the boat? It's the name of the main author of all the books, Evi Nemeth, who is now retired and sailing across the world.
Cool, isn't it? Robert Langdon isn't the only one that can find hidden symbols everywhere.
These are really good books by the way. You should probably get a copy of the Linux version.
Here's the deal. For reasons best left unexplained, I have every edition of the Unix System Administration Handbook by Evi Nemeth, et al. I even have the Linux System Administration Handbook. Since every book's edition is a different color, my daughter likes to play with them. The other day, while putting them back on the shelf, I noticed that the cover of the book reflects the history of the computer industry, the history of Unix, and a little history about Evi. So without further ado, here's my explanation of what it all means. You'll have to excuse the quality of the images, but I don't have a scanner so I took digital pictures.
Unix System Administration Handbook, 1st Edition
This book was published in 1989. We see that UNIX system administration is a difficult topic. There is a Unix system administrator driving his car towards a cliff. I don't know what the reference to cranberry blogs is about, but let's focus on the colorful characters.
Starting rom the left, we have a dog named biff, which is a reference to the biff(1) command in Unix that prints a message to your terminal when you have new mail. In the center is a daemon, a Unix process that runs in the background. Since the daemon is showing us his watch, maybe it's the NTP daemon? At the right of the daemon, you can see the finger(1) command, used to find information about a user and made famous by the Great Worm of '88. I don't know what the other two characters (the owl, and the cat, or is it perhaps a lady daemon?) are about. If anyone knows, or has a guess, let me know.
Unix System Administration Handbook, 2nd Edition
In the second edition, published in 1995, biff is dead (you can see the gravestone that says "R.I.P biff R.I.P"), probably replaced by POP. The cross behind reads "USL" and is a reference to the Unix System Laboratory at AT&T, the group responsible for System V, which had been sold to Novell in 1993.
In the front of biff's grave, it says "Here lies the entire Berkeley CSRG". That means the Computer Science Research Group, which was responsible for the development of the Berkeley version of Unix, called BSD, at the University of California and was disbanded in 1994.
The first edition of the book is being held by the paw of a cat. We can tell it's the first version because it's yellow, and it says "USAH", short for Unix System Administration Handbook.
The dog, Biff, is now gone from the front row. He's dead, remember? And has been substituted by a Monkey, a spider, and a Web. I don't know what the monkey has to do with anything, but the web is clearly a reference to the World Wide Web, which was invented in 1993. The spider is clearly a reference to the web spiders, or bots that search engines use to index web sites. We can still see the daemon, the finger, and a new character, Dr. SNTPd, a reference to the Simple Network Management Protocol.
Note that the car is more modern, and it has the word "Unix" written on the front. Also the System Administrator behind the wheel appears to be a woman. Could this be a reference to more women in system administration?
Unix System Administration Handbook, 3rd Edition
The third edition, published in 2001, has many of the same symbols as the second edition. In this edition, however, there's a new tomb. The Unix to Unix Copy Protocol, or UUCP, has died because most people use PPP by now. We can also see the second edition coming out of the grave being held by a tentacle.
Two more things should be noted. First, this is the first edition of the book to cover Linux, a newcomer to the Unix world, and you can see the Penguin in the co-pilot seat of the car. Also, the spider now has handcuffs, which might be a metaphor for the restrictions many sites started putting in place to prevent search engines from finding them.
Linux System Administration Handbook
This book came out in 2002, and was made specifically for Linux, which had by then surpassed in popularity all of the other Unix distributions. You can see the Linux Penguin driving now, and the sys admin as the co-pillot.
The spider's handcuffs have been replaced by a shackle, meaning the restrictions on spiders are greater now. The other books are gone, as Linux has been declared the only winner.
You can also see that the sign that used to read "Beware! Vendor Gratuitous Changes Ahead!" has been replaced by one that has Unix somewhat deviated, but still going up, Linux going straight up, and XP (a reference to Windows XP) going down. The helicopter that is crashing also has XP written on it. The authors have no love for Microsoft's Operating System.
There is also a penguin skiing on the mountain. I have no idea what that means.
The last interesting bit is that there is a sailboat in the back. Let's take a closer look at that sailboat.
See the name on the boat? It's the name of the main author of all the books, Evi Nemeth, who is now retired and sailing across the world.
Cool, isn't it? Robert Langdon isn't the only one that can find hidden symbols everywhere.
These are really good books by the way. You should probably get a copy of the Linux version.
Tuesday, September 27, 2005
The Lennon Syndrome
I have to confess that I'm a beatlemaniac. I have every single recording from The Beatles, I have almost all the movies, I have documentaries, I have t-shirts that draw people's attention and makes them strike conversation when I'm wearing them, I know most of the lyrics to most of their songs, I can even play some songs on the guitar.
Anyway, I've noticed an interesting phenomenon about John Lennon's life. When he was married to Cynthia, he was basically a good guy. I mean, he was a little crazy, but Cynthia, being very moderate and conservative, sort of kept him under control. On the other hand, when he was married to Yoko, he was out of control. He already had the tendency to do crazy stuff, but Yoko, instead of being the voice of sanity, encouraged him to be even crazier.
I've noticed that I have friends like this. Some of them get together with Yokos and it's all downhill from there. Some others get together with Cynthias and they do pretty well. I guess it's only human nature.
By the way, there are many other definitions of "the Lennon syndrome".
Anyway, I've noticed an interesting phenomenon about John Lennon's life. When he was married to Cynthia, he was basically a good guy. I mean, he was a little crazy, but Cynthia, being very moderate and conservative, sort of kept him under control. On the other hand, when he was married to Yoko, he was out of control. He already had the tendency to do crazy stuff, but Yoko, instead of being the voice of sanity, encouraged him to be even crazier.
I've noticed that I have friends like this. Some of them get together with Yokos and it's all downhill from there. Some others get together with Cynthias and they do pretty well. I guess it's only human nature.
By the way, there are many other definitions of "the Lennon syndrome".
Monday, September 12, 2005
Windows 2000: got pid?
If you get that joke, we want to hire you. We spent the entire last month working out various Windows 2000 problems in BitKeeper, most of them related to the agressive reuse of PIDs that Windows 2000 does.
BitKeeper uses MSYS's bash as a shell. it turns out that bash does not like PIDs to be reused. Here's the scenario. Bash launches processes A, B, and C. Due to the PID recycling of Windows 2000, both A and C get the same PID. This is possible because PIDs are guaranteed to be unique only while the process is running. If A finishes before C is started, C can, and will, get the same PID as A. The problem is that whenever bash waits for a process, it records whether it has already waited on it or not. When bash needs to wait for C, it sees in its table that it has already waited for that PID (because of A) and decides not to wait.
What are the symptoms? Well, to sum it up in one phrase: unintended parallelism. Needless to say, this wrecks havoc in every shell script ever written. It's like the system randomly adding an ampersand (&) at the end of every line of the script. Not very nice.
Microsoft seems to have fixed the problem of PID recycling in later versions of Windows. Neither XP nor 2003 reuse PIDs like Windows 2000.
I back-ported a patch to bash 2.05b into bash 2.04 that fixes this issue. I'll post it to the MSYS mailing list later this week.
By the way, I think that a cool idea for a T-Shirt would be the title of this post: "Windows 2000: got PID?"
BitKeeper uses MSYS's bash as a shell. it turns out that bash does not like PIDs to be reused. Here's the scenario. Bash launches processes A, B, and C. Due to the PID recycling of Windows 2000, both A and C get the same PID. This is possible because PIDs are guaranteed to be unique only while the process is running. If A finishes before C is started, C can, and will, get the same PID as A. The problem is that whenever bash waits for a process, it records whether it has already waited on it or not. When bash needs to wait for C, it sees in its table that it has already waited for that PID (because of A) and decides not to wait.
What are the symptoms? Well, to sum it up in one phrase: unintended parallelism. Needless to say, this wrecks havoc in every shell script ever written. It's like the system randomly adding an ampersand (&) at the end of every line of the script. Not very nice.
Microsoft seems to have fixed the problem of PID recycling in later versions of Windows. Neither XP nor 2003 reuse PIDs like Windows 2000.
I back-ported a patch to bash 2.05b into bash 2.04 that fixes this issue. I'll post it to the MSYS mailing list later this week.
By the way, I think that a cool idea for a T-Shirt would be the title of this post: "Windows 2000: got PID?"
Technorati Tags: windows
Thursday, September 08, 2005
Apple's CoreData
I'm sitting here at the Cocoa Heads meeting at Apple's headquarters in Cupertino, and Scott Stevenson is talking about CoreData.
There are a couple of Apple employees here in the room answering questions, and from what I'm hearing, here's my assessment of CoreData: stay the hell away from it. At least for a couple of years.
I'm surprised that people are still looking for the holy grail of data-driven applications: create them without programming. The problem is that this works for trivial applications. If you have any application that's big enough, or popular enough that has to be maintained, these tools just don't work.
I have yet to hear from an application built this way that's easily maintainable...
Oh, by the way, CoreData's NSPredicate class doesn't support Outer Joins.
There are a couple of Apple employees here in the room answering questions, and from what I'm hearing, here's my assessment of CoreData: stay the hell away from it. At least for a couple of years.
I'm surprised that people are still looking for the holy grail of data-driven applications: create them without programming. The problem is that this works for trivial applications. If you have any application that's big enough, or popular enough that has to be maintained, these tools just don't work.
I have yet to hear from an application built this way that's easily maintainable...
Oh, by the way, CoreData's NSPredicate class doesn't support Outer Joins.
Flame Wars Galore
Now this is just stupid... two secretaries get into a flame war over a ham sandwich and they get sacked?
Talk about extreme disciplinary action! If I were their boss I would have just bought them both ham sandwiches and let them cool off...
I don't know how people expect to have employees "not use email for personal purposes". The whole idea about employee retention is that you want them to be as personal to the company as they possibly can. If they feel like they belong to the company, they will get personal.
Talk about extreme disciplinary action! If I were their boss I would have just bought them both ham sandwiches and let them cool off...
I don't know how people expect to have employees "not use email for personal purposes". The whole idea about employee retention is that you want them to be as personal to the company as they possibly can. If they feel like they belong to the company, they will get personal.
Friday, July 01, 2005
I know how to program in...
I've been reviewing a number of resumes for programming positions because we're hiring programmers and I've started wondering what it means to know a programing language.
For some people it seems it means "I can write hello world in it", and they list dozens of programing languages. From a resume reviewer point of view, this makes it very difficult to see at what level the candidate knows a language. It'd be much better to list only a few and give examples of the programs built on those.
What I'm looking for in a programer basically boils down to this: "can they come up with general purpose abstractions that give the program a self-consistent architecture or do they copy and paste blobs of code to get the job done?".
For some people it seems it means "I can write hello world in it", and they list dozens of programing languages. From a resume reviewer point of view, this makes it very difficult to see at what level the candidate knows a language. It'd be much better to list only a few and give examples of the programs built on those.
What I'm looking for in a programer basically boils down to this: "can they come up with general purpose abstractions that give the program a self-consistent architecture or do they copy and paste blobs of code to get the job done?".
Friday, June 10, 2005
Sun NVRAM Madness
A few weeks ago I had to troubleshoot a Sun Ultra-10 system. Oh, the memories of working for a Sun Microsystems reseller and learning how to program in Forth. Anyway, as part of my debugging, I usually turn the diag-switch? to true in the OpenBoot prompt. What I'm saying is, I press
Now for the catch. When you turn the diag-switch? on the output is printed in the serial console. This means the first serial port. If you don't have a serial console, and you're using the system with a monitor and keyboard, you'll find yourself staring at a dark screen for a couple of minutes while the diagnostics run.
It's really easy to think the machine is busted if you don't know what's going on. And that is precisely what happened to me.
Now for the punchline. Apple's computers also use Open Firmware, the incantation is
Stop-a and at the OpenBoot prompt type:ok? setenv diag-switch? true
Now for the catch. When you turn the diag-switch? on the output is printed in the serial console. This means the first serial port. If you don't have a serial console, and you're using the system with a monitor and keyboard, you'll find yourself staring at a dark screen for a couple of minutes while the diagnostics run.
It's really easy to think the machine is busted if you don't know what's going on. And that is precisely what happened to me.
Now for the punchline. Apple's computers also use Open Firmware, the incantation is
Option-Command-o-f. Once you're in Apple's OpenFirmware prompt, you can also turn diag-switch? on. Guess what happens... yup, a few minutes staring at a dark screen thinking the machine had died. Only Apple computers don't really have a serial port.
Wednesday, April 13, 2005
Why a free BitKeeper clone was a bad idea (which time had come)
I've been reading all the comments about the end-of-life of BitKeeper, the source control management system used by the Linux kernel developers. After plodding through hundreds of comments I began to understand how little the general public knows about the operation of BitKeeper and why having a free (as in speech) BitKeeper clone from anyone besides BitMover was a bad idea. Let me explain.
First, lets establish some context. According to this article, Larry McVoy had two reasons for dropping the free (as in beer) BitKeeper program: (1) Corruption, and (2) IP Loss. He goes on to explain how repository corruption would have caused major problems and how his IP would have been compromised by the free clone. Not many people bought it. To be sincere, I didn't buy it at first, but after thinking about it and thinking about BitKeeper's mode of operation, I decided that Larry McVoy is completely right. Let me explain.
Corruption
Here is Larry McVoy's explanation of corruption:
Working with distributed systems is different than working in a client-server environment.
Client-server environments have a well defined protocol (or an API) used to pass data between the client and the server. There is a layer of abstraction between the data files the server uses to represent the data, and the data exchange that takes place. It should not be possible to corrupt the server's data by using (or abusing) the protocol. Most of us feel really comfortable with this model. Most of the software in the world behaves this way.
Distributed systems (and peer to peer systems) behave slightly different. There is no server, and there is no client. Instead, a collection of peers interact by passing data between themselves. There is usually a protocol for transferring the data, and that protocol should have the same properties of a client-server protocol, but there is a slight difference. Each peer has a copy of the data files that in the client-server model belong only to the server. This is an important difference.
BitKeeper belongs to the second class of systems. A BitKeeper repository consists of not only the data (source code), but also all the metadata needed for revision control. Each user has a full copy of the repository, including all the history. There are two ways in which users can exchange data in BitKeeper: (1) clones, which amount to no more than creating a tarball of the entire repository and transferring it to the requesting peer (using, say HTTP), and (2) push/pull, which have some proprietary protocol for transferring ChangeSets between the peers. Note that these repositories contain all the internal data files that BitKeeper uses to manage the changes.
Now imagine a free BitKeeper program, let's call it FreeBK (FBK). It is reasonable to assume that FBK will corrupt repositories. After all, it's dealing with proprietary data file formats and its authors are learning, by reverse-engineering the data files, what are the invariants that BitKeeper maintains.
Now imagine a developer with access to both BK and FBK. This developer can clone repositories using either tool. BK will notice corrupt repositories and warn him. FBK however, still lacks the ability to detect certain kinds of corruption. If he chooses to clone a repository with FBK, the corruption has already spread. Working on top of this bad repository is fraught with danger. Work might be lost, bad ChangeSets could be spread (using either BK or FBK, as the corruption is now hidden under new ChangeSets), and information obtained from this repository can no longer be trusted (diffs and such).
If said developer has a commercial BitKeeper license, he will certainly call BitMover support and expect them to fix his corrupt repository and help him recover his work. In order to be able to do this, BitMover would have to track FreeBK and be aware of the ways in which it can cause corruption. This is clearly a very big support load for BitMover.
Loss of intellectual Property
The other reason Larry McVoy has for not wanting a Free BitKeeper program is loss of IP. The best way I can explain how this is a real problem is by comparing it to the PalmPilot.
When USRobotics came up with the PalmPilot, they had a very simple, but very important, insight: computers sucked at recognizing handwriting, but people are very good at learning how to draw new symbols. The Apple Newton had been a big failure, mainly because it never recognized what users wrote. The PalmPilot required users to learn a new simple alphabet, and that simplified the problem of handwriting recognition the point where the PalmPilot got it correct 99.9% of the time. This idea was very hard to come up with, and very expensive to develop, but once you have seen it done, it is obvious and very easily copied.
How is this related to BitKeeper? Well, anyone who has used BitKeeper knows that there are some ideas in it that are as simple as the handwriting recognition in the PalmPilot. Once you have seen them, they become obvious. Now, BitMover has already paid a big price for coming up with this ideas, and another big sum for developing them, but they are trivial to copy. You can see how it would not be in BitMover's best interest to have developers of competing source control management systems using BitKeeper.
A BitKeeper Free Linux
Overall, I think having BitKeeper was really good for both the Linux kernel and BitMover. It gave BitMover much needed visibility when it was a small company, and it gave the Linux kernel developers a much needed tool when they needed it the most.
Like every relationship, there has to be about equal giving from both sides in order to continue. In this case, I think the breakup was inevitable as BitMover was giving more to Linux than it was receiving from it. I mostly feel sorry for all the small Open Source projects that used BitKeeper for managing their source. Also for those of us that used it for keeping personal files in single-user mode. We have lost a very powerful free (gratis) tool.
On the other hand, I feel projects like monotone, darcs, arch, etc. will get a much needed push by the community to produce better tools. It will probably take them a couple of years to be as good as the BitKeeper of today, and by then BitMover might have come up with a new and improved BitKeeper. But by then the open source community will have very good tools and people with money will be able to buy state-of-the-art tools. Isn't that the way the world works?
First, lets establish some context. According to this article, Larry McVoy had two reasons for dropping the free (as in beer) BitKeeper program: (1) Corruption, and (2) IP Loss. He goes on to explain how repository corruption would have caused major problems and how his IP would have been compromised by the free clone. Not many people bought it. To be sincere, I didn't buy it at first, but after thinking about it and thinking about BitKeeper's mode of operation, I decided that Larry McVoy is completely right. Let me explain.
Corruption
Here is Larry McVoy's explanation of corruption:
BK is a complicated system, there are >10,000 replicas of the BK database holding Linux floating around. If a problem starts moving through those there is no way to fix them all by hand. This happened once before, a user tweaked the ChangeSet file, and it costs $35,000 plus a custom release to fix it.Many people deemed this problem as a simple bug in the BitKeeper server. Others saw it as bad design. Few people understood that it was neither, and the corruption would have been inevitable.
Working with distributed systems is different than working in a client-server environment.
Client-server environments have a well defined protocol (or an API) used to pass data between the client and the server. There is a layer of abstraction between the data files the server uses to represent the data, and the data exchange that takes place. It should not be possible to corrupt the server's data by using (or abusing) the protocol. Most of us feel really comfortable with this model. Most of the software in the world behaves this way.
Distributed systems (and peer to peer systems) behave slightly different. There is no server, and there is no client. Instead, a collection of peers interact by passing data between themselves. There is usually a protocol for transferring the data, and that protocol should have the same properties of a client-server protocol, but there is a slight difference. Each peer has a copy of the data files that in the client-server model belong only to the server. This is an important difference.
BitKeeper belongs to the second class of systems. A BitKeeper repository consists of not only the data (source code), but also all the metadata needed for revision control. Each user has a full copy of the repository, including all the history. There are two ways in which users can exchange data in BitKeeper: (1) clones, which amount to no more than creating a tarball of the entire repository and transferring it to the requesting peer (using, say HTTP), and (2) push/pull, which have some proprietary protocol for transferring ChangeSets between the peers. Note that these repositories contain all the internal data files that BitKeeper uses to manage the changes.
Now imagine a free BitKeeper program, let's call it FreeBK (FBK). It is reasonable to assume that FBK will corrupt repositories. After all, it's dealing with proprietary data file formats and its authors are learning, by reverse-engineering the data files, what are the invariants that BitKeeper maintains.
Now imagine a developer with access to both BK and FBK. This developer can clone repositories using either tool. BK will notice corrupt repositories and warn him. FBK however, still lacks the ability to detect certain kinds of corruption. If he chooses to clone a repository with FBK, the corruption has already spread. Working on top of this bad repository is fraught with danger. Work might be lost, bad ChangeSets could be spread (using either BK or FBK, as the corruption is now hidden under new ChangeSets), and information obtained from this repository can no longer be trusted (diffs and such).
If said developer has a commercial BitKeeper license, he will certainly call BitMover support and expect them to fix his corrupt repository and help him recover his work. In order to be able to do this, BitMover would have to track FreeBK and be aware of the ways in which it can cause corruption. This is clearly a very big support load for BitMover.
Loss of intellectual Property
The other reason Larry McVoy has for not wanting a Free BitKeeper program is loss of IP. The best way I can explain how this is a real problem is by comparing it to the PalmPilot.
When USRobotics came up with the PalmPilot, they had a very simple, but very important, insight: computers sucked at recognizing handwriting, but people are very good at learning how to draw new symbols. The Apple Newton had been a big failure, mainly because it never recognized what users wrote. The PalmPilot required users to learn a new simple alphabet, and that simplified the problem of handwriting recognition the point where the PalmPilot got it correct 99.9% of the time. This idea was very hard to come up with, and very expensive to develop, but once you have seen it done, it is obvious and very easily copied.
How is this related to BitKeeper? Well, anyone who has used BitKeeper knows that there are some ideas in it that are as simple as the handwriting recognition in the PalmPilot. Once you have seen them, they become obvious. Now, BitMover has already paid a big price for coming up with this ideas, and another big sum for developing them, but they are trivial to copy. You can see how it would not be in BitMover's best interest to have developers of competing source control management systems using BitKeeper.
A BitKeeper Free Linux
Overall, I think having BitKeeper was really good for both the Linux kernel and BitMover. It gave BitMover much needed visibility when it was a small company, and it gave the Linux kernel developers a much needed tool when they needed it the most.
Like every relationship, there has to be about equal giving from both sides in order to continue. In this case, I think the breakup was inevitable as BitMover was giving more to Linux than it was receiving from it. I mostly feel sorry for all the small Open Source projects that used BitKeeper for managing their source. Also for those of us that used it for keeping personal files in single-user mode. We have lost a very powerful free (gratis) tool.
On the other hand, I feel projects like monotone, darcs, arch, etc. will get a much needed push by the community to produce better tools. It will probably take them a couple of years to be as good as the BitKeeper of today, and by then BitMover might have come up with a new and improved BitKeeper. But by then the open source community will have very good tools and people with money will be able to buy state-of-the-art tools. Isn't that the way the world works?
Subscribe to:
Posts (Atom)
