Something I've learned

When it comes to huge complicated things like software, it is almost always better to build on previous efforts than to try to demolish those previous efforts to start from scratch.

This is best encapsulated in the following quote, via Daring Fireball

“A complex system that works is invariably found to have evolved from a simple system that worked. The inverse proposition also appears to be true: A complex system designed from scratch never works and cannot be made to work. You have to start over, beginning with a working simple system.”

—John Gall

How to deal with the newspaper crisis

What about a journalism "auction" site? Journalists would have accounts at the site, and post briefs on a topic of investigation. Say, a 2 or 3 paragraph abstract.

Based on this, the general public is then able to contribute: in essence, throw a pledge into a hat until a given threshold of funding is reached. This money is taken from a paypal account or a credit card until that threshold is reached by a certain date. If it is not reached, the money is returned to the users account. (or perhaps the site itself could hold virtual credits). The idea here is to collectively fund the investigation, so that no single sponsor has to bear the full cost.

Pros: Places journalists in a closer relationship with their audience. Rather than a publishing company brokering the funding between audience and advertisers and journalists, it would essentially run on a social networking or web2.0 type of model funded directly by the audience. This funding is then used for investigation and reporting of that abstract. (with allowances for the investigation leading to new interesting areas at the journalist's and editor's best judgement.)

Rather than expecting users to pay directly for content which already exists, it's giving the audience personal ownership, by allowing them select and fund the creation of new content.

Cons: The tendency for the general public to vote for baser and less enlightening stories may present a problem. Perhaps this could be mitigated by some kind of "tax" on general articles which could be directed towards articles selected by a qualified editor. That way, editors could in essence overrule the public on some portion of funding for articles to get important and educational things through- while at the same time, the general public can in essence overrule an overidealogical editor for funding on other articles, thus providing some checks or balances.

Feel free to improve expand, or implement this idea in any way you choose. I, the author hereby release this blogpost, unconditionally, into the public domain.

The emotional growth of characters

Alan Moore about Watchmen:

"Watchmen" is widely regarded to have brought an aura of realism to comics.

I think that more often it's a more supposed physical realism rather than any kind of emotional realism. Yes, books like "Watchmen" did make it fashionable to show grimly the consequences of violence, which I suppose initially was a good thing because it's better that people know that violence results in terrible injury and pain and suffering than that they think that it's just something that, you know, people get a sock on the jaw and they are unconscious for a couple minutes and then they come around and they are taken off to the police station.

But I think that when you are talking realism in comics you have to realize it's an ongoing process, especially emotional realism. That when the comic book industry started you had characters who were, let us say, one-dimensional in that they only had one quality. They were good or that they were bad. By the 1960s Stan Lee with Marvel Comics had the brilliant idea of two-dimensional characterization where they are still good or bad but now they have some kind of, perhaps a medical complaint or some sort of emotional suffering. What we were trying to do with "Watchmen" was to make it at least three-dimensional. So that the characters that we were talking about were complex human beings that weren't defined by one simple set of behavior patterns. With some things like Todd McFarlane's "Spawn" or a lot of these modern comics, they will show greater violence because they know that actually that is what a lot of the audience wants, for prurient reasons, not trying to show the emotional depth and complexity of the characters.

It's occured to me that many of the characters I've seen in various fiction are incapable of growth or change. A character is concieved as a brand with a certain set of unchangable characteristics that can be exploited by the narrative again and again and again. Consider the simpsons, and their world which resets at the end of each episode to exactly the way it was at the start. None of the characters get older, nor do they actualy learn anything from their adventures that modifies their future behavior, or makes them quantifiably better people.

How would a fictional character look, if it was capable of growing and learning? What if in the process of that growth, they abandoned the main attribute that made them recognizable as that character, and that change persisted to future stories? Is emotional growth the third dimension Alan Moore speaks of?

Javascript Constructors

Javascript has a problem with constructors.

function Thing () {
   this.foo="bar";a
   this.bar="baz";
}

this is okay if you call Thing as
var thing = new Thing();

That creates a new object, whose prototype is Thing.prototype, and binds that object to "this", and then executes the Thing function.

However, if you do this:

var thing = Thing();

"this" instead gets bound to the global object, and you end up with the global variables "foo" and "bar".

Douglas Crockford concieved a function named "object", that will be baked into the next version of the ecmascript standard. (as a static method named "Object.create" )

The object function takes an object as a parameter, and creates a new object whose prototype is the given object, and simply returns it. The implementation goes like this:

function object (o) {
  var f = function () {};
  f.prototype = o;
  return new f();
}

you can use this to create safer constructor functions.

function Thing () {
   var that = object(Thing.prototype);
   that.foo="bar";
   that.bar="baz";
   return that;
}

Now, this function does exactly the same as the above Thing function when you call it like this.

var thing = new Thing();

but when you forget the "new" keyword

var thing = Thing();

it still does exactly the same, avoiding global namespace pollution.

JSON xml and the relational model part 4

An xml element is an unordered set of attribute names, and their associated values.

<div class="title" id="heading" lang="en"> </div>

a JSON object is an unordered set of attribute names, and their associated values.

{"tagName":"div", "className":"title", "id":"heading", 

"lang":"en"}

an xml element may contain an ordered tuple of "nodes", which may be plain text nodes, or could be other xml elements.

<div class="title" id="heading" lang="en">
The Grand Adventure of <i>Lucious Swan:</i> The return of elemental qualities.
</div>

a json property may contain an ordered tuple of values, which may be primatives, or could be other objects.

{"tagName":"div", "className":"title", "id":"heading",

"lang":"en",

   childNodes:[
     "The Grand Adventure of ",
     {tagName:"i", childNodes:["Lucious Swan:"]},
     " The return of elemental qualities."
 ]
}

and that's basically all there is to it. This may appear somewhat more bulky than other xml to json translations. However, the translation preserves the unique and unordered quality of xml attributes, and the ordered non-unique quality of xml node collections. As a result, it's much simpler to implement readers and writers for this format, because there's fewer exceptions or other special conditions to account for. This is more or less a direct translation of the semantics of xml into JSON.

As an additional bonus, code written against this style of structure would work exactly the same directly against a browser dom representation of an xml document, since this is essentially a stripped down subset of the browser DOM, using the same attribute names. Most server-side XML parsers produce essentially the same structure as well.

a JSON translation into xml, using the same principles however, is not so easy...

Where does the logic go?

There's a trend in data centric applications. The trend is to move more and more of the contraints and logic out of the database software, and into the application code.

The trend results from the fact that the software technology industry is populated and driven largely by humans, and thus subject to trends and irrational behavior. To understand what's going on today requires a bit of perspective in the history of databases, and their parallel development with programming languages.

SQL is the IE6 of the database languages world. It breaks many of the rules of the relational model- in other words, it's a little bit like a calculator that performs multiplication incorrectly, and doesn't have a minus operator. SQL is not complete enough to be a real solution. It was never developed beyond the prototype stage, and was never meant to be used in industrial settings. But then it was naively used by oracle, which turned out to be a "killer app", SQL became industry standard instead of its technically superior competitors, and the rest is history. SQL's syntax is based around a set of command line tabular data processing tools, and COBOL. Full of bugs, inconsistencies, and a mishmash proprietary versions and features that don't have a grounding in math or logic, results in a situation where it really is unclear what goes where.

Regarding the recent proliferation of ORMs: misguided and ill thought out attempts to patch over the obvious deficiencies of SQL. Database triggers and procedures are another misfeature trying to patch over SQL's problems.

If history had played out in a logical and orderly way, the answer to this question would be simple: Just follow the rules of the relational model and everything will work itself out. Unfortunately, the rules of the relational model don't fit cleanly into the current crop of SQL based DBMS's, so some application level fiddling, or triggers, or whatever other stupid patch is unfortunately necessary, and it ends up being a matter of subjective opinion, rather than reasoned argument, which stupid hack you use.

So the real answer is to just follow the relational model as close as you can, and then fudge it the rest of the way. Put the logic in the application if you're the only one using the db, and you need to keep all your source code in a version repository. If multiple applications are likely to use the database, make the DB as bullet proof and self sufficient as it can be- The main goal here is to ensure that the data remains consistent.

What will the future of programming look like?

it will likely be a formal specification language. Instead of indicating the "how", as we do today, we'll specify the "what". So instead of saying step by step how to implement a certain algorithm, we'll specify what the requirements for our program are, and the compiler will work out the most efficient algorithm automatically.

The spec language may or may not be text based. I do not believe we will ever over come the problem of linguistic ambiguity. Even with a computer with an equal or greater intelligence to a human. Humans still misinterpret eachother. Computers will always take what we say painfully literally. this is an anavoidable, but not necessarily intuitively obvious result of various inviolable premises we hold about the operation of a computer. But higher level, more intuitive ways to state a problem unambiguously exist.

I don't think that the star trek depiction of people programming holodecks is entirely far off. It will seem more obvious and intuitive, but computers will still make catastrophic errors based around the ambiguity of natural language.

The reason I think it seems that humans can understand eachother easily in a way that computers can't, is due to shared culture, and our shared understanding that we don't always say what we mean (sympathy), and our ability to (only occasionally) continuously clarify and disambiguate our meaning. Our ability to communicate with eachother is largely the result of the common shape of our bodies, perceptions, and our ability to imagine ourselves as other people (an ability we can see more clearly when we look at those who partially lack this ability- Autistic people and Aspergers people). This enables us to understand in an extremely intimate way, why someone else may be making a specific pattern of noises with their vocal mechanisms, and gesturing in a particular way with their faces and bodies.

We only derive meaning from these patterns of behavior by imagining what we would be thinking if we were doing those things. Computers lack human bodies, vocal mechanisms, and faces, and consequently any ability to sympathise with a human. Any attempt to make a computer truly understand us without making the computer into a human itself, will be largely fruitless.

Despite all that, human comprehension doesn't work quite as well as most of us imagine it does. Consider the challenge each of us have as programmers in determining the shape of a program that client requires. It doesn't happen instantaneously. A successful program is the result of continuous revision over the course of many weeks months and years. That revision process would be very challenging (impossible) to replace with an automatic process. We can better automate the repetitive work, but we will never be able to artificially generate a perfect sympathy for our intent, artistic vision, and personal/ethical needs.

Banking on a future AI is a bet that I wouldn't make, even if it were possible. We should think just as hard about what we would lose in such a proposition, as much as we think about what we would gain.