What a brilliant observation, and I believe that this is a truth that applies to more than just programming or business.
"The selfishness of turning the other cheek..." I never looked at it that way before, but I like the idea.
That is a pretty comprehensive list. I'm probably in the same camp as you claim to be, having broken every one.
The rules are pretty strong as well. I'm still mulling them over in my mind, but there is one that sticks out as unwise, or poorly written. It is #3, which says: "I will not write a program that fails to do tomorrow what it was able to do yesterday."
Taken purely literally, this prevents all change, all experimentation, all feature-level refactoring. I'm sure this is not what the rule was intended to say. I'm assuming it is more of an encouragement for better testing, so that a bugfix in one area doesn't break a feature in another.
Even taking this rule as a Testing Rule, I think it overreaches. There are steps that programmers can take to prevent regressions, but to assume that it is possible, or even common, to prevent them all is wishful thinking.
I think a better rule would be something like: "I will not refuse or hinder the fixing of a bug that I have caused."
It would still be nice if timegm() was in a library of its own instead of libc. But that's probably asking too much.
Good to know. Thanks!
There is also a very handy set of functions that goes in the other direction. The function gmtime() takes a time_t and converts it to a struct tm in the UTC timezone. And localtime() does the reverse of mktime(), converting a time_t to a struct tm in the local timezone.
The one function missing is the reverse of gmtime().
One way to do it, via standard library function calls, is to adjust the TZ environment variable to "UTC" and then call mktime(), which will then use it as the current timezone. But that gets messy. Not only do you have to save the old TZ setting to restore it afterward, if your program uses threads, messing about with the global TZ variable is hardly safe.
Another way to do it is to use gmtime() as a reference point, and loop while adjusting the original struct tm until mktime() + gmtime() gives you the result you started with. This is brute force and inelegant, but it manages to do the job without fiddling with the global environment.
When parsing ISO timestamp strings, this problem is hit pretty quick. An ISO timestamp can contain a time in either the local timezone or UTC, depending on the trailing 'Z' flag. In my research, it does not appear that more specific timezones can be specified in the timestamp format, so at least we only have two states to worry about.
After much searching, I've ended up with something that I find suitable enough to release. I tackled this problem back in 2007 for the OpenSync project (you can see similar code in the opensync_time.c file), but looking back on it now, the code is not clean enough for me to reuse very easily. And even though there's a vtime2unix converter function, it leaves the burden of timezones to the user.
I took a look at Boost's date_time library as well, and while huge and comprehensive, and though it breaks the problem of time, dates, and durations into nice manageable chunks, it seemed to reverse the status of the C library: it made UTC conversions easy, and local timezone conversions hard. And it reads timezone information out of CSV files... I don't want to have to maintain my own timezone database when the OS does it for me.
So, if I don't want to write my own conversion routines, and if I don't want to maintain my own timezone database, I'm stuck with the C library method, and the first two solutions. Might as well make it easy to use.
The source files for TzWrapper contain the following utilities:
Using TzWrapper, it's now possible to do things like:
struct tm pacific_tm = { ... };
time_t t = TzWrapper("Canada/Pacific").mktime(&pacific_tm);
cout << TzWrapper().Set("Australia/Brisbane").ctime(&t);
const char *iso = "20100430T231500Z";
time_t stamp = TzWrapper::iso_mktime(iso);
If you have any further tips to make this code better, or pointers to better libraries that make this code obsolete, please let me know!
I also need to tweak the CSS for this site. I like the default layout more than the rest, but there were some items that just didn't work for me.
One was that the date was on a line of its own. Since everything is HTML templates and CSS, I thought this would be easy. But obviously I'm rusty on my CSS!
So, boiled down, my problem was:
<div class="container"> <div class="element1">...<div> <div class="element2">...<div> <div>
If I make element1 to float left, and 2 to float right, then the container suddenly decides that it doesn't "contain" anything anymore, and the elements overflow the small container div, making it disappear.
There were two ways to fix this:
1) Add an empty clearing div:
<div class="container"> <div class="element1">...<div> <div class="element2">...<div> <div style="clear: both;"><div> <div>
This now expands the container to the bottom of both elements.
or,
2) Use the automatic overflow mode in the container:
# css
.container { overflow: auto; }
<div class="container">
<div class="element1">...<div>
<div class="element2">...<div>
<div>
Thanks to the kind folks on freenode's #css channel for pointing me to the following sites:
I like brittle software.
Now, what I mean by that deserves some explanation, but whenever I think of ideal software that does a job for me, I think of it as a harmonious blend of the robust and the brittle.
First, the robust. Software needs to check its buffer sizes, check OS error codes, behave defensively and not crash even if given complete garbage as input. This side of the equation is very strongly linked to security. The software should be impossible to use for malicious purposes due to a bug in the code.
Another aspect of robustness is flexibility. It should put the user in control, while defending the user from possible mistakes he might make while learning to use the program, or in just everyday use. It should be hard to use wrong.
The brittle side of software also shows up in its error detection, and is a critical part of putting the user in control. If there is an error, I want to know about it. If the program is not absolutely certain that my data is safe, I want to know. If an attacker is trying to use the program to harm me, I want it to complain loudly and often.
I would rather have the program stop running than harm my data.
Not every program can achieve these lofty goals, but this is the nebulous image my mind creates when I think of ideal software. Some of this comes from my background in writing industrial network firmware, where it was better for firmware to halt completely and go to a known safe state, than to assume it knew what you meant and just let that piston stay on your coworker's arm.
So bringing Postel's Law into the equation, there are places where I definitely do not want my software to be liberal in what it accepts. Take a git repository, for example. If there is any remote possibility of data corruption, I want to know, and I want git to refuse to proceed. Luckily, git does a fantastic job of data integrity, which is one of the reasons I like it so much.
Postel's Law was originally about TCP, and so pertains to communication. Taking TCP as a specific example, and trying to apply Postel's law, let's say that a packet arrives with the entire TCP header shifted by 1 bit. I am no TCP stack expert, but I would be surprised if any TCP stack would be able to parse such a packet correctly. The only logical way of dealing with such a packet would be to look at the existing bits with the header format in mind, try to make sense of the non-sensical data, and send back an error if possible, or drop it on the floor.
How does Postel's Law fix a situation where the very format of the data is not respected?
I can understand if there is a TCP flag that is out of order, or if some unambiguous packet abnormality exists in an incoming packet. But there are only a limited number of such possible error combinations. Things have actually progressed in the opposite direction: TCP stacks have gotten more strict to deal with various attacks, and firewalls are used to strictly guard inbound and outbound traffic.
Being too liberal has a cost. And some costs are so high, they are seen as a detriment to society. Just look at the once-popular open SMTP relay.
Even in places where it would seem obvious that being liberal in what you accept is a good thing, upon further reflection, it turns out to be not so clean cut.
Take for example, a corrupt OpenOffice document. The user definitely wants to open it. Maybe some invalid XML is getting in the way, or maybe the file is only half there. What should OpenOffice do? It should make a best effort at retrieving what data it can, and it should open the file read only. It should also warn the user loudly that the file was corrupt, invalid, and inaccurate. The user needs to know this. This may be a file from an outside source, and so the error is not important. But it may also be the first error the user receives about a dying hard disk, or a bad network, or a corrupted filesystem, and restoring from backup is the next item on the agenda.
So in my frame of reference, Postel's Law, which is also called the Robustness Principle, fits right in, but only to half of my ideal software equation. Yes, be robust in what you accept. Be able to process complete garbage input without crashing. I've heard of people who fed a program's own EXE into itself as test input. It should be safe to use input data as a baseball bat and bludgeon the program without it falling over.
Yet the Brittle half of ideal software doesn't change. It is still as loud as before, warning the user of potential pitfalls and trouble on the horizon, and will refuse to proceed if it can't guarantee his data's safety or determine his intent without ambiguity.
In the end, Postel's Law is too ambiguous to be a useful guide, let alone a law. Those that accept it are too liberal, and those that don't are too strict. :-)
I think the crux of the argument boils down to these two statements:
and
I obviously disagree with both of these statements, but I understand how you could think they were true.
On the surface, parsing doesn't seem to create a new format, but even in Avery's own example, the majority of browsers accepting an incorrectly quoted option have indeed created a new format. It isn't a documented format. It is actually an anti-documented format, because the spec says it is wrong. But anyone writing a browser today would not be able to merely follow the specs and produce a functional browser... they would have to follow the behaviour of every other browser in existence as well. Just ask the developers of the now long-defunct, but exciting, Project Mnemonic.
Now, obviously this makes it easier for Average Joe to write his own webpage, and it probably did help advance the popularity of HTML and the rise of the web. But there is a defacto HTML standard out there because parsers were not strict enough. I don't know how you can deny that. (Part of the problem was that browsers were developed alongside the spec, so that contributed a lot too. The poor spec didn't stand a chance.) :-)
And strict receiver-side validation does improve interoperability. Can you imagine if the average C++ compiler allowed a relaxed syntax? Suddenly compiling code that "works" on one platform would not work on another. I admit that this is already a problem to a smaller extent than HTML, but differences in validating the C++ language spec between compilers is seen as a bug in the compiler, and rightly so.
This raises an interesting Option 4, to add to Avery's list: let the parser be forgiving with unambiguous syntax, but warn loudly.
This would be a huge improvement to what we have today. We need more web browsers that report the number of HTML errors in a page, by default, in the status bar. And it should be hard to disable, so that a site's non-compliance is widely seen and scorned. (And yes, some of my web pages would be scorned too.)
I believe Opera has a feature like this, if memory serves.
As for the side note claiming that parsing is not the problem, but the rendering is, my argument was based on XML as a data exchange format, and less as a way to display content in a browser. For example, the opensync project uses XML for data interchange, and plugin config. These formats are defined in strict schemas, which are tested and used via libraries like libxml2, which I assume falls into Avery's Big XML Parser category. If these schemas were not correct, I would consider it a bug detrimental to future interoperatility, and something that should be fixed.
The web itself is already so goofy that trying to apply XML to it now is like nailing jello to the wall. So in that respect, I can understand Avery's pain. It just bothered me that someone was boasting about an incomplete parser and claiming they interoperate with XML better than the big libraries. It seemed to me that he was discounting the long-range goals at work in XML in order to avoid some short-term pain. Of course, in the real world of making the customer happy, such shortcuts are often needed, but it's something to hide away in that closet of programming hacks, not published as an example on the web. That sounds harsher than I mean it, but I like those long-range goals, and XML is a solid technical achievement in its own right, even if rather cumbersome. Hey, I'm a C++ programmer... I like strict syntax. :-) I believe strict syntax promotes accuracy, and that accuracy helps you down the road when projects get larger and more complex.
In some ways, you could say that the inaccuracy found all over the web results in an unstable foundation that is generally holding the web back from greater things.
I hope it is also clear that I'm not a fan of XML. (Referring to it as a monstrosity was probably a good hint.) It has its place, but I think a lot can be accomplished by drastically simpler documented formats, and I'm quite willing to hack up my own simple file format if I think it is appropriate. I just don't call it XML.
I have some thoughts percolating in my head about Postel's Law, but that will have to wait for another post.
Seriously though, I have no problem with Javascript as a language that people might want to use to get things done on the desktop. The problem is that, in almost all current implementations of Javascript, it is setup to run any random code from the internet that the user clicks on... or even code he doesn't click on, in some cases.
In order for me to consider using a web-enabled Gnome desktop, I need to be confident that I have the power to enforce this strict separation of church and state. My PC is the church, and the internet is the state. :-)
I need to be able to flip a setting that makes it impossible to run any javascript that comes from outside my machine, whether it be through email, the web, or various files left over in /tmp or .webgnome or /home/cdfrey/Desktop, and only run javascript that I've installed and authorized, such as through apt-get or /usr/local.
This is where my confidence in Gnome's security design falls apart, because history seems to show that it is always more tempting to enable the new shiny web than it is to lock it down securely.
I do believe you are missing the point of XML. Yes, it is a horrendous pile of textual complexity, piled high to the sky with syntax and nested markers. Yes, the available API's, in order to deliver the promise of XML to the end user via the application programmer, are complex enough to fall out of your head as soon as you've finished implementing the feature.
But if you're going to do XML, please do it right.
Interoperability is hard. Anyone can write their own parsers. And everyone has. That's why the monstrosity called XML was invented in the first place.
It all starts with someone writing a quick and dirty parser, thereby creating their own unique file format whether they realize it or not. And since they probably don't realize it, they don't document it. So the next person comes along, and either has to reverse engineer the parser code, or worse, guess at the format from existing examples. This is then documented, either in Engrish, or in yet more code, which is likely to have at least one bug in it that makes it incompatible with the original format. This bug means that the second programmer has, whether they realize it or not, created their own unique file format.....
Oh dear.
The DTD is the documentation. XML kinda forces you to create it. The nice thing about DTD is that the computer can check it, which means it is less ambiguous than the pages of documentation that came with the old format. And with DTD + XML data, the computer can verify both, guaranteeing that the next programmer who gets the data can parse it the way it was meant to be parsed.
If your customer had used this XML process, you could have used the Big Fancy Professional XML Library. But instead, your customer is using their own UnknownXML, and you've created ApenwarrXML. The next programmer to come along may be forced to create yet another ThirdXML version, and the pain keeps propagating.
Note that I’m not talking about a web browser. I’m talking about a full featured framework that allows every application to display (partial) web pages, execute XHRs, upload photos to flickr, run user-downloaded scripts or otherwise integrate with the web. In fact, even the browser situation is worse than ever as GNOME is currently trying to switch to a different framework.So how are we gonna marry GNOME with the web?
Maybe I'm an old foggie, yelling "get off my lawn!" here, but it is my opinion that this is the direction that caused a lot of Linux people to laugh at Microsoft.
I don't want the web everywhere on my desktop. I don't want my desktop married to the web, downloading and running scripts in the background for my convenience, or even by an accidental mouse click which is sure to happen. I don't even run javascript in my browser. Why would I want it anywhere near my web-connected desktop?
Remember back when it was impossible to get a virus via email? Then Microsoft came along, and made it default to automatically open your email in an Outlook subwindow and excute its HTML? I don't want my desktop to go in the same direction.
Perhaps you think that Gnome developers are smarter and they won't fall into the same traps that Microsoft developers did. Maybe Gnome developers are special? Perhaps. But I fear that the Linux desktop is making the same mistakes that Microsoft did, only more slowly.
It's still processing things line-by-line, which when compared to something like grep, is inefficient. Grep reads large blocks of the file, I think 16k or more at a time, and does its own very fast processing. But replacing std::string with a char buffer made the performance acceptable for me.
I believe the main cause of performance issues with std::string stem from the way iostreams insert a character at a time, and when using std::string, it is not very smart when growing its buffer.
I also find Boost regex to be rather slow at times, but perhaps I'm not using them correctly.
As for operator overloading in C++, that's a bit like saying you won't use C because it has some bloated feature like printf() instead of write(). :-) You don't have to use operator overloading, but it sure is handy when it makes sense to use, such as when creating fundamental types like money values or complex numbers.
Advogato also has XMLRPC, with which you can do programmatic access to your diary. I do this in the hope that it saves advogato bandwidth.
Below is a small python beginner script that I cooked up just for this task. It checks the dates of the posts and saves the new ones as individual files.
If someone finds it useful, please feel free to use it.
#!/usr/bin/python
import xmlrpclib import os import difflib
def Download(filename, entry): print "Downloading: " + filename out = open(filename, "w") create, update = server.diary.getDates("cdfrey", entry) out.write("%s\n" % update) out.write(server.diary.get("cdfrey", entry)) out.close()
def GetTimestamp(filename): inf = open(filename, "r") s = inf.readline() inf.close() return s[0:len(s)-1]
def Update(filename, entry): print "Updating: " + filename filetime = GetTimestamp(filename) webcreated, webupdated = server.diary.getDates("cdfrey", entry) if( filetime != webupdated ): print "Entry %d is out of date" % entry if os.access(filename + ".bak", 0): os.unlink(filename + ".bak") os.rename(filename, filename + ".bak") Download(filename, entry)
oldf = open(filename + ".bak", "r") newf = open(filename, "r") oldl = oldf.readlines() newl = newf.readlines() print ''.join(difflib.unified_diff(oldl, newl))
path = "/home/cdfrey/text/advogato/posts/" server = xmlrpclib.Server("http://www.advogato.org/XMLRPC")
entryCount = server.diary.len("cdfrey")
for entry in range(entryCount): filename = path + "advogato.%03d" % entry if os.access(filename, 0): Update(filename, entry) else: Download(filename, entry)
I often use split to break huge files into multi-CD images, such as for backups or archiving. At the same time, I often find I'm low on disk space, so I would like to process each split file chunk as they appear instead of doing it all at once at the end.
I wrote a patch that adds exec and pause functionality with the following arguments:
-e, --exec=CMD run CMD after each output file is closed
-w, --exec-wait=CMD run CMD after each output file is closed
and wait for the child to exit
-p, --pause pause for keypress after each output file is closed
I find these features very useful, so I hope they make it into the official coreutils tarball.
If anyone has any tips on how to handle recovering a tty-based STDIN (see the split source code), please let me know. I'm currently just open()ing the tty on STDOUT, but that is not ideal.
Spent most of today reading through the various websites for for C++ wrappers of SQL databases. The best ones, in my opinion, are the official ones: mysql++ and libpqxx (was libpq++, but official now (2025) is libpqxx). Unfortunately, these are both tied strictly to their specific databases and are not database independent.
Also in the mix are DBConnect and GQL. Both are rather pointer-happy, but at least DBConnect encourages use of it's own smart pointer to avoid copious use of delete. On the plus side, they are database-independent.
Only mysql++ has the SSQLS feature. What is that you ask? It is a mechanism whereby you can easily create structures that mirror your database tables, and then go around using them as members of container objects, and doing things like:
query.insert(row_data);
This makes the library code extremely hairy (so hairy that it motivated a fork of the code a few years ago), but gives the advantage of enabling the compiler to check your field names, localizing your C++-to-SQL linkage in one place, and letting the library take care of generating properly escaped SQL code.
So far, my research confirms that the free software community still lacks a proper database independent C++ SQL library that shields the programmer as much as possible from quoting issues. As always, please email me if you know better.
Cool Program of the Day
Unpaste is a program by Jeroen Vermeulen that attempts to find repeated code in your C, C++, and Java programs. It is a work in progress, but already useful. It scans all the files you give it, and detects duplication even with name changes or differences in const.
[Edited to fix and remove broken links]
boog, your Snippets project is cool, but it's not a new idea. It reminds me of a similar project with the same name which I ran across back in the days I used to be a regular FidoNet user. It was run by a fellow named Bob Stout. Here is his original Snippets archive (archive.org) (github).
The license of each of the contributions varies with the contributor, but they are all freely available for use last time I checked.
You might consider joining forces with Bob Stout and combining your efforts to produce one large repository (perhaps mirrored on both sites). Not sure if you tried this already, but it would appear that the original Snippets site could use a little help.
Fun
While perusing the site, I found a link to The Parable Of Two Programmers (alternate link). Excellent read.
C++ and the Web
After writing a dynamic website for a client in PHP, I was struck by the lack of help that the language and interpreted nature of PHP gives a programmer for dealing with large sites. Also, I'm amazed at how many XSS and SQL injection attacks I see for PHP websites posted on Bugtraq. It reminds me of the arguments against C I used hear from anti-C people, who used to blame buffer overflows on the language. At the time, I thought it wasn't right to blame the language when it was the programmer's fault, but now I'm on the other side of the fence, taking a step backward (in my opinion) from the safety of C++ to the wild west of PHP.
And the problems with C pointed out in the past by security-conscious language bigots :-) are all "fixed" in PHP. There are no buffer overflows. There are no memory leaks. There are no real number size issues.
Instead, I'm now on the bigot side, and really missing strict type checks, and really missing a compiler to check all syntax. Wondering how people manage with this.
So I got to thinking, and I'm trying to design a PHP clone in C++. I'm writing about it now so that people can head me off at the pass if something like this already exists. My goals are to make it nearly impossible, or at least harder, for a programmer to put XSS or SQL injection bugs into their websites. Make the compiler check for these things. That's what C++ is for anyway.
Perhaps C++ can give the same structure and organization to a website that it gives to a large local application.
Some of these ideas were inspired by Joel Spolsky's article referenced by ncm's recent post, which referenced titus's recent post. I agree with Nathan, but Joel's article did get me thinking. His type method should fit nicely into C++, where everything is a type, and instead of the programmer doing the checking by reading "wrong" code, the compiler will do it, and enforce it.
So that's my idea, which I plan to call CPPHP. I hope to have some code someday soon that actually works. Right now I'm just prototyping my design. If this already exists somewhere, or if you just want to send feedback, please email me.
Edited to add:
I just want to plug John Torjo's win32gui library, which has some ambitious aims for programming GUI's with C++. If that could be done for the web, that would a wonderful advance.
[Edited to fix some broken links]
fzort: Thanks for the link to Herb Sutter's concurrency article. It was quite the interesting road of travel to the end.
Here's a link link to Andrei Alexandrescu's presentation on lock-free programming, which reference's Herlihy's 1991 paper on Wait-Free Synchronization, in case anyone else is interested in blowing a few hours.
Read over last year's ACCU conference events and wished I'd been there. Would be very cool if I could make this year's.
Created by Chronicle v4.6