Taxonomy Matters

ActivatorDesk InfoGrid News Blogger (via Anil) is a combined-news-and-weblogs aggregation thing. Very, um, structured.

Taxonomy matters, though. A motley "most popular" group indeed.

Julian's doing well under the Weblogs category, too. Links to NetMarkets, Voidstar and the Rogue Moon Diaries...

So, there's a serious point here. Do you categorize weblogs, and if so, how? and why? (I did try briefly to categorize my subscriptions in SharpReader to help scan, but gave up very quickly).

Statistics and coincidence

For the first time in about a year, I fired up Analog to drill into the activity stats for cabezal.com. Analog seems to have moved along some since I last used it.

Meanwhile on plasticbag.org, this: What's the time zone?, pointing to a nice feature of the Re:invigorate open-source third-party-stats system. That looks interesting... will try it for a while. It won't give me any information about RSS usage, though.

That RSS summary for the last 2 weeks, briefly: 14MB of traffic, 6588 requests or ~400 per day (65% reading the RSS2 feed, 27% to RSS1, and a few to the RSS0.92), 209 unique readers (more than I'd expected, but that does count a large handful of robots). Roughly 25% Radio Userland, 14% SharpReader, 12% Syndirella, 11% Mozilla, 9% NIF, then a few dozen others.

Connected Specialization

When Robin Good wrote about Groove recently, I thought: hey, nice practical perspective.

Since then, I've become seriously impressed by his articles on a range of different subjects. Thoughtful reviews - with a decided focus on effectiveness - of several interesting pieces of software. Informed commentary on important topics: usability, security and marketing, intellectual property and cooperation. Today's posting includes a nice pointer to George Siemens' thoughts on the importance of networking.

When information is abundant, effectiveness is a measure of attention applied to a task (as John Maloney says in an important note, "Organized attention needs to be delivered to what matters most to improving business performance"). Robin's articles really help to direct attention.

If you're not already subscribed, you should be.

Unicode

Tim Bray has a rather wonderful exposition of Unicode and its UTF encodings: Characters vs. Bytes:

Processing UTF-8 characters sequentially is about as efficient, for practical purposes, as any other encoding.
There is one exception: you can't easily index into a buffer. If you need the 27th character, you're going to have to run through the previous twenty-six characters to figure out where it starts... UTF-8 also has the advantage that null-termination, and all the old routines like strcpy, strncpy and their friends, which in practice are insanely efficient in terms of space and time, work just fine.

Looking forward to the next instalment, wherein it sounds like he'll have some harsh words for the Unicode-is-16-bit-chars brigade.

In practice, character encoding will only give you problems if you ignore it. Don't ignore it: at each place in your application, just define which encoding you're working with (especially if you're looking at data 'on the wire' or on disk). Is this a UTF8 byte array? An ISO-8859-1 byte array? Is it a string of BMP Unicode? Or something more exotic? It doesn't matter what, as long as you know what. If there's ambiguity (about the codepoint encoding, or higher-level encodings: eg. if you don't know whether a string is XML entity-encoded or not, as I saw recently), your application is broken.

Now, someone please, give fonts the same treatment. I had such problems with Windows fonts last time I needed to deal with Asian characters. Has it got better?

Squeak

Seems there's some recent interest in Squeak and its progeny.

Dylan, being noticeably more hip than his old man, has been playing with Squeak occasionally for the last few months (and Croquet when I let him...). Here is some of his work.

RSS+presence

(After an evening chillin' with the sixteen men of Tain...)

There's now a pointer to my online presence awareness in my RSS feed now, following the spec.

You can query my presence here... PHP source code here.

Thanks in advance

The hook, when it struck out of the blue, had that hysterical acid-tinged clarity that she'd been expecting, and almost forgotten, throughout those lean years of consulting. What a plan! Rolling onto the couch, she kicked off her shoes and cried with the beauty of it. If only they'd taught this feeling at business school. Here was a certain winner, and she knew exactly where to sell it. The Man would pay top dollar for this sort of expertise.

Aaah, yes. Augmentation of human intelligence.

I mean, look at the credentials: this game comes straight from the really great greats: Engelbart, Licklider, Bush, Ted Nelson, even the unruly Parc mob. But at the same time, the mere suggestion that your customers' intelligence could use some improvement is just so ticklishly apt. Striding those long manly corridors of power.

(...to be continued...)

Daily

Don Park has been talking about a newspaper-like news-aggregator UI, and: he posted some screenshots.

Interesting. Much whitespace. Much text, few headlines. Photograph (speaks a thousand words). It loses the originals' look-and-feel, though (weblogs might be syndicated columnists anew, but don't all want to feel that way).

I'd be interested to see the widgetry around this - how you navigate through blogspace/paper. That would seem to be the tough part - could the ubernewspaper keep all the benefits of a good three-pane UI? Or could we move to a seamless read/edit environment, ever?

Social software weblog

"Many to many" is a new weblog on the subject of "social software". Joint authors: Liz Lawley, Clay Shirky, Ross Mayfield, Sébastien Paquet, and Jessica Hammer - quite an interesting crew.

Whether they can help define the "connection age" or simply comment on the mechanisms of its unfolding, this should be worth watching.

As someone here wrote some time ago,

...it appears that the transformational dynamic of this era is our newfound ability to make a connection with each other, at the same time or at different times, in the same place or in different places, and to have a meaningful and purposeful interaction as a result of that connection. In short, we find ourselves at the advent of the Connection Age.

Blogosphere

the World as a Blog.

New opportunities

Ray Ozzie, Surrounded by new opportunities:

Taken together, these changes will transform the personal computer into an interpersonal computer. This will be a rich, self-synchronized and readily interchangeable device focused specifically on people and what they do with one another online.

(via POK)

Presence, revisited

A couple of days after writing up my "presence in RSS", I still haven't written any software, although I'm sure a Groove-based backend would be quite straightforward, and that various other backends would be really easy. Still thinking.

Some interesting comments, a couple of which missed my mark. I want RSS to have "presence without identity".

On the RSS-DEV list, Roy Silvernail points to RFC2779 (IM/presence protocol requirements) and would like to see attribute-type data. Thanks - I hadn't read the RFC and it's relevant, even if not particularly specific. The XMPP <presence> element is more directly important. I don't actually like the enumeration choices in the XMPP <show> tag (schema here, page 69); they're too application-specific, and at the same time not specific enough. My "icon" element can map the "show" semantic. For other attributes, the status field is deliberately vague. Vague enough?

Mike Amundsen suggests including the vCard by reference. In a related vein, Phil Hart posted some great comments:

Ideally, an "RSS Client" (for want of a better expression) should be presence-enabled, so that all Identity/Presence/Availability information can be retrieved from the corresponding "presence server" at the very moment the document is retrieved and read.

...which is exactly what I'm hoping for. But he also notes

by the simple expedient of including only the Identity of the document creator, the rest of the elements can be obtained from the presence server.

which is very true, but I deliberately tried to avoid. My proposal has no "identity" in it, and that's intentional. For several reasons, including plain laziness: identitiy unification is a really important project, but it's a big one, and I'm not going to go there. But you can do presence without identity, by building a really simple presence-polling mechanism which is agnostic about your claims to identity or the various identity-resolution methods.

So, rather than including in my RSS feed some information which says "I am xxxx", which indirects to "you can query presence server yyy for my status using protocol zzz", it just says: "you can query this webserver for my status using HTTP GET, and expect a response in this XML format".

The RSS document could include the actual presence information, and/or (more usefully, I think) a pointer to the presence source.

Actually this proposal does ignore one of the more difficult parts of presence, which is: "who wants to know?". I show different online presence information to different people. But implicitly, the "who's asking" is "John Doe", and you'll get my public presence status.

Recycling

Chris Brumme's superb (and, ie., way-over-my-head) weblog is all about .NET/CLR technicalities. Today, talking about lifetimes and garbage collection, something familiar:

This [judicious use of GC.KeepAlive(), etc] is all far too messy. It runs counter to the goals for our new managed platform to force developers to worry about this sort of thing.

Heh, yes, indeed. Why can't software be easy and straightforward? :-)

Social software, etc

The embryonic Social Software Alliance held a "happenin'" today, where I lurked awhile, and learned, mostly: don't try to step through code in a debugger at the same time as concentrating on a conference call.

Later, Bryan Field-Elliot from PingID wrote a really interesting followup (ie. I don't fully understand it)...

to my main point: I believe the blog is the perfect foundation upon which to house a personal DigID infrastructure. That's because, unlike your email server and unlike your static web site, blog software is (A) evolving rapidly and creatively, and (B) is already well suited to produce and consume RPC events. So my participation here is aimed primarily at fishing for ideas on how to creatively fuse the worlds of DigID (and I'm thinking of Liberty specifically), and blogging.

Now, I haven't a clue about the Liberty Protocol, and I'm not at all sure how decentralised strong identity management links with blogging (which is why it's completely missing from my RSS presence strawman), but it's quite clear that weblogs and "strong social software" such as Groove will have a lot in common over time. Personally the part of that intersection I'm fascinated with is somewhere around "group usability" and "inter-domain plumbing". Bryan says (my markup):

I'll be fishing for ideas from the likes of Marc and Nikolaj, who are starting from the other end of the stack (that of interesting UI's), and work to meet them in the middle with a comprehensive, security-minded protocol infrastructure.

Presence

Everything could use presence information. (Almost everything, anyway...)

I'm starting to think about adding online presence information to RSS feeds. Would that be useful? Here's a strawman spec. Comments welcome.

Robin Good likes Groove

"Robin Good", aka Luigi Canali De Rossi, gets Groove:

While many web conferencing and live collaboration tools could be compared to convention centers, movie thetaers, snack bars, and other short-stop buildings along our main way to some other place, Groove Workspace stands out as being a wonderful virtual ranch where you may want to go to work for a while

Nice, non-technical review.

SharpReader IBlogThis

SharpReader 0.9.0.1 includes support for Simon Fell's BlogThis plugin interface. Neat.

How it feels

The art of great software design is to make things which are good to use. Particularly with social software, where properties emerge from multiple users. Some of the greatest effects are the hardest to predict.

The magic art of software product managers is to plan that in advance. How do they do it? I really don't know. I can only follow the tip of my nose.

Here a few examples I collected recently.

SharpReader, fast becoming my newsreader-of-choice, has a neat feature which groups linked posts together in a tree. I suspect Luke just had one great idea, tried it, and found that it worked. The result is emergent: the implicit weblog community becomes more visible.

Groove has a great facility which Ray calls "attention rendezvous": you can configure to be notified immediately people update content in a shared space, and this has the effect of creating a "flash crowd" when several people jump in to the same context at once, see each other present, and get to work. Was that an intended effect? If so, someone very smart designed it.

Pinboard has a nicely social feel. NewsClient gets the social juices flowing (so Jon felt, at least). On the other hand, my Start Menu groovegator isn't really social at all. It's a cute toy, but using it, there's no "there" there. (Plus, I really have way too many contacts to put on the start menu). None of these things were right or wrong by design, they just turned out that way.

Paul Graham, talking about language design ("Having strings in a language seems to be a case of premature optimization") brings up some of this, too.

An essay, in the original sense, is something you write to try to figure something out. This happens in software too. I think some of the best programs were essays, in the sense that the authors didn't know when they started exactly what they were trying to write.

In the social dimension (as in so many other areas), games are where the rubber hits the road. If a game works, it gets played and talked about. So they've been at the forefront of social software for some time, and we can learn from their experience. At the GDC, Will Wright (of The Sims fame) gave a great talk on "how we use emergence to engineer larger possibility spaces, with simpler components". His recipe: use prototypes. Start iterating that prototype "uphill" into more interesting regions of the dynamic space.

As Paul Weller says, "all the study in the world / doesn't make it science"...

Open Source Groove Newsreader

Source code for the NewsClient aggregator tool is now available under an Apache-style license.

I don't plan to continue its development, but I'll be delighted to help anyone who does.

GWS recap

So, how far have I got with Groove Web Services? Some distance. But only a part of the way.

Groove Web Services, aka GWS, is a feature of the Groove platform, new in version 2.5. GWS provides a standards-based interface to the Groove application: rich decentralised shared spaces, and their complementary infrastructure (identity, messaging, presence). That interface is SOAP over HTTP.

The name "web services" is specific and accurate, but also might be misleading. Most web services can be characterised as "network services that are very far away, and owned by strangers.". The SOAP activity in the Java world seems to reflect this: there are great toolkits for building server-based SOAP sources and sinks. Ditto, to a large extent, for the .NET environment. GWS inverts this: Groove.exe is the web server, it's very close (on the same machine as the "client" application), and owned by you.

GWS also supports remote access, but right now this is in "preview" mode, for some good reasons. Remote access to Groove presents some fascinating technical challenges: my main Groove account (which is the SOAP webserver) is spread over three machines, all behind firewalls, and usually at most two will be running at the same time. So the remote access to GWS is routed SOAP. And, unlike local GWS, there's not a secure channel (authenticated and confidential) from one end of that route to the other. All that will appear in the future, but for now: GWS == localhost access.

I started dreaming of some great remote-access applications -- picture sharing is my standard example; but I'm deliberately ignoring those use cases at the moment. For commercial users, I think the set of cases where you want to enable remote GWS access (and by doing so, compromise the security of your Groove environment) is vanishingly small. Defer.

Local SOAP
The uses for GWS, then, involve loosely-coupled applications on the same machine. Desktop software talking to Groove, using HTTP.
If you have a desktop application already, there will certainly be places it can leverage Groove and vice versa. Tim Knip's Radio Userland interop tool is a great example. There are plenty more – after all, even the humble spreadsheet was imagined as a collaborative tool, as Bob Frankston recounts:

As Dan described the product I envisioned a group of people sitting around a table with small devices pointing at a screen. Each had the ability to draw on the shared screen with graphics and formulas. The formulas would be recalculated as needed. This seemed reasonable give the technology of the day

Twenty-five years later, that sort of facility really is viable.

For my spare-time hacking, though, I was missing some direction; I don't really have a stake in any desktop applications. Much of my working life is spent using Visual Studio, Word, Notes, and so on, but I don't want to build out a production-ready tool (like, say, co-debugging) just to explore web services. But then: why integrate with an application, when the desktop itself is scriptable with my favourite tools?

Oh yes. My first programming experience was with a powerful shell language (Prime CPL). Windows 95 reinvented the shell script concept, and the script was DHTML. XMLDOM and Javascript. And that magic pair, I know well.

Data structures
I gotta admit, I think file-sharing is really boring. You call that collaboration? When documents are done, they're background – knowledge, perhaps, but not collaboration.

I'm much more interested in how the "doneness" happens. Means, not ends. Some of those means have rules, structure, process and procedure, but those structures themselves tend to be seen as a sort of "doneness", where in practice the end will never arrive. Applications evolve, and cooperative groups need environments which will carry that evolution along with the group's changing needs. Excel, Notes, SharePoint, and the Groove forms tool are all quite good examples of evolvable application environments.

But I digress. The current GWS implementation gives you quite rich access into Groove-land: you can open and interact with your account(s), identity(ies), contacts, shared spaces, and the contents of those shared spaces. But the interaction with space contents isn't very complete: GWS in Groove 2.5 talks to the Discussion, Calendar and Files tools only. All the other core tools – of which I think chat and forms are probably the most important – are still to be done. (Tim has proved that you can build your own GWS services for custom tools, but I won't be following that path for a while – too much like work!). Of the three core tools, then, file sharing is actually quite important.

So over the past few evenings and weekends, I started playing with GWS integration with the filesystem, using the lowest-possible technology: shell script (XMLDOM and JavaScript). I've got to the point where some interesting things are possible, and more possibilities become interesting.

About SendTo

Here's a brief description of what this SendTo code is and how it works.

This is string-and-glue tech. There's no compiled code here: it's all JavaScript and HTML. Which is to say: it works, and debugging is very easy, but I wouldn't necessarily recommend deploying a solution this way...!
The pieces are:

• a JavaScript library (.js) which defines "classes" (almost everything in JavaScript is a hashtable, so a "class" is an object with named properties which are its method functions) which implement the SOAP interface. The class structure is quite closely modeled on the WSDL definitions provided for GWS.
  
• a HTML Application (.HTA) in my startup. This does the heavy lifting. At startup it looks into Groove and walks through my shared spaces, looking for files tools. It also scans contacts. Then it sets up subscriptions, and a timer on which we'll ask Groove whether there are any queued events (new or modified tools, new shared spaces, and so on).
  
• For each contact it creates a shortcut, in a folder under the Start menu, The shortcut icon shows whether the contact is online or offline. The target for the shortcut is another .HTA: when you click the shortcut, it sends the contact's URL to the launcher HTA, which then tells Groove (using the "Transceiver" web service) to pop up the instant-message UI addressed to this person. If the contact subscription fires an event, we recreate the shortcut, to show the current awareness state in its icon.
  
• For each files tool it creates a shortcut, in a folder under the SendTo menu. The folder gets a desktop.ini pointing at an appropriate icon. The shortcut points at a batch file, which... (this is where things get complicated!).
  

SendTo is more complicated, unfortunately. When you click a SendTo shortcut, you're effectively initiating a drag-and-drop operation (as if you'd dropped the file onto an application or a folder). Now, HTA applications aren't usually "drop targets", but batch files are (so I don't need to write code implementing IDropTarget). So each drop target -- the target of the SendTo shortcut -- is a little batch file which launches a helper HTA with some appropriate parameters.
The parameters are: obviously the path of the file you just dropped on the shortcut; but also, the URL of the Files tool where the file should be delivered. So the batch files get that URL written into them when they're created. Then there's one batch file per tool, corresponding to the shortcut file. It's clunky.

The launcher application ("GWSSendTo.hta") is really really simple. All it needs to do is: read the commandline parameters, open the file, and stream the file into the Groove tool. The code is this.

// commandline gives us:
// sIdentityURL (the current Groove user),
// sToolDataURI (the address of the Groove files tool) and
// sFilePath (the file we had dropped)

// Initialize the GWS context for localhost SOAP
pCtx = g.CreateLocalContext( GetGrooveNonce(), GetGroovePort() );
pCtx.SetIdentity( sIdentityURL );

// Set up our reference to the tool
var pFilesData = new g.GrooveFilesData()
pFilesData.SetContext( pCtx );
pFilesData.SetURI( sToolDataURI );

// Create a descriptor for the new file in Groove
var pNewFile = new g.GrooveFileDescriptor( sFilePath );

// Create the new file in Groove
pFilesData.Create( pNewFile );

That's it. A dozen lines of code.

SendTo

Ho ho. More play. Does what it says (but not very efficiently: my current version uses Base64 encoding for the SOAP transfer, whereas DIME would be much better...). The menu is dynamic, of course.

(update: I changed the icons...)

StartMenu

Here: something nice - built with SOAP and JavaScript, of course. Awareness updates every few seconds. (It also adds files tools to the SendTo menu....)

Social software

Matt Webb has some great notes on "social software". Also see the SocialSoftware pages on MeatballWiki.

Wow!

Google buys Metafilter?!