So what exactly is a firehose.  It goes beyond being a real-time push stream of data.  If you just had my twitter stream in real-time would you call that a firehose, hardly.  Firehose is all public data, the full thing.

As the Twitter Api says:

Returns all public statuses. The Firehose is not a generally available resource. Few applications require this level of access.

Now I’m starting to see more people say “we got a firehose!” when they dont have anything close.

First, Yahoo. See the Read Write Web Article (which shows they didnt look or didnt care, just repeat the Yahoo Press release)

The first sentence makes it clear that it is not a firehose:

The Updates Firehose API gives developers access to the full, real-time Yahoo! Updates index and allows developers to search, filter and combine Updates data using YQL.

That’s right, the full firehose to query.  What?  It isnt push, it isnt a full stream of data, it isnt even close.  Instead this is just a combination of several APIs from yahoo into one endpoint.  That’s cool for what that is worth, but that isnt real-time and it isnt streaming.

Second Spinn3r.  From their site:

Spinn3r listens to a new Twitter firehose API which is a sample of the full Twitter feed.

Again, if it isnt the full stream it isnt the firehose.  That’s like firefighters showing up with a garden hose. (likely the api method they use)

I’m glad to see that we have gotten more streams of real-time data on the web.  I’m glad to see some companies providing a firehose of their content, even if they charge for it.  The ability to build  some of the next large scale ideas depends on that.  Just dont crowd the search results with overusing the term, use it correctly.

I saw a unicode symbol of a white star enclosed in a black circle the other day, in a domain name.  So last night I set out to see unicode characters in a domain name for myself.  I started at my favorite Registrar name.com.  Some of the country code domains they sell allow unicode characters.

I Bought.

So I chose ⤷.ws  the .ws was the cheapest option coming in at $14.5o, and as you no doubt know .ws is the TLD for Government of Samoa.  First thing that I notice while setting up my DNS was that domain showed up as “xn--2ri.ws”.  What is that all about?  Well, although I really kinda hoped that the internet had progressed to the point where we can all handle unicode, it hasn’t.  So in order to implement Internationalized Domain Names or IDNs, we still use Ascii.  How?

I Wikipedia’d.

Punycode is a means to represent a larger character set (unicode) with a smaller more restrictive character set (ascii).  Do to some security concerns around spoofing urls many browser will show the punycode that represents the unicode domain in the address bar, although some do this selectively such as safari.  Many countries are retrictive on the allowed characters, usually ascii + their countries language, but there are some that have just allowed them all (thanks Samoa).  So my new url shorten is in place, http://⤷.ws

Tech Community

In Boulder: At least a couple of events (great then 50 people) a week.  It isnt all the same people at all the events, although at the major ones you can expect the regulars, but in general a good diversity and networking / learning opportunity.  Some of these events are fun geek nights out (Ignite) some have a bit more structure in content (Boulder New Tech Meetup, generally has some demos / presentation about companies and products), some are just user groups exchanging knowledge.

Along with all these events comes many organizers.  These are people that want to give back to the community, want to see it grow, and want to be a part of it.  In Boulder I have seen many people reach out to be a part of this, and I’m starting to see more of that here in New Mexico (read as: a very good thing).

In The Land of Enchantment

Over the last year I think we have seen a good increase in geeky events in New Mexico.  NMTC, Barcamp, Webuquerque, IgniteNM, AIGA, SynerQue, ABQ Web Geeks, Several User Groups(ruby, python?), as well as others I’m sure.  One thing that I think we lack a bit, is cohesion.  Just last night I heard about 2 events that I would have been interested in attending, however I didnt hear about them in time to attend.  Part of this is trying to keep up with multiple groups and just getting lazy about doing so.  Some of it is not knowing about new groups that are just getting started.

The Solution?

There is a question mark, because this is still very open for discussion.

The first step that I’m going to take (and hopefully getting some others involved too) is to start a calendar of all NM Tech Events.  This will be administered by all the group organizers.  Have a group or are you planning an event? Contact me (mick@davidmichaelthompson.com) and I’ll set you up with access to the calendar.  This is based on a shared google calendar, and a website to display the events, and highlight some.

Step 2. Blog covering upcoming events, highlight recaps of past events, and discuss new or current groups.  I’m willing to do some of the writing, but I really want this to be a group effort (i.e. I’m looking for a bunch of other contributors)  This will all at nmtechal.com If you are interested in contributing in any way, let me know.  I want to run this very open to the community at large, everyone welcome!

Step 3. Talking with Chad Kieffer, he has the idea for a event where we actually all pull out our laptops and work on some code.  Very hands on.  I’m thinking about a “Web Project Night” at the Santa Fe Complex.  It would meet 1-2 times a month, we’d decide on some sort of small project, and try to use new tools to accomplish the goal.  Getting to work with others, build something, and try out some new things along the way.

That’s it from me.  Grow community, Grow!

    Each of this sites comes with, or has added a social component.  It has  made the service better, more useful, and barring a few exceptions more fun.  Here is the big exception to the better-ness/usefulness/fun-ness:  I filled in my data, I filled in my friends, I filled in my profile, my network over and over (and over again).  

    The issue is that I need a global Social network, without the copy and paste. And forget filling things in the first time…that is painful enough, but keeping it all up-to-date is all the more horrid.

    So how do you make you social network follow you, how can you centralize your info.  Well, facebook is aiming to become that central point for your social network with some of their new APIs.  But, that is not going to work for me.  No One Person / Site Should Control MY Network.  No Way.  No How.  Though a central location (Google, facebook, myspace, or joebruan2312.net) would be easy to query, easy to find, I have major trust issues, future of the system issues, and adoption issues (not that I’m looking for new parents, but if you got a lot of money drop me a line anyway, thanks.) The only way I could even dream of the centralized point for this data, is something open source, run by a non-profit foundation.  A wikipedia/mozilla/OPEN system.  But even then we have many adoption and future management  issues.

    It needs to be decentralized (like what we have, sigh), but with a common/standard (scary) way of accessing every social network(hooray!).  Intro www.dataportability.org/ stage kinda almost next to center.  They have a start (meaning the got a logo designed, which does look cool *thumbs up*)  on a draft of a draft of a proposed standard.  check them out, weigh in on the discussion, this effects you too…and maybe U2.  Here as the aspects of this that need to be in place as I see it, please keep in mind that I only got like 2hrs of sleep last night, the current time is after midnight, and I have only been thinking about this for 10-20mins.  

1. Service Definition.  A *standard* programatic way to ask a website:  what services do you provide?, where are they?, what do I need in order to use them?

2. OpenID…I think we have enough big players behind it now, but it will still be a far bit of time before any kind of wide scale implementation is seen across the web.

3.  Standard Data Requests I have been calling a lot of APIs recently, they are all different, they all have their own names for everything, and they all have different limits.  But most of them are for the same data.  Now I’m not the one to say that one approach is so much better than another, but I will say that We should see *standards*, yes standards.  Imagine if I code call information about users from Flickr, Digg, Facebook, etc.  all in the same way.  Imagine a standard that defined how I query a website about it’s data, it’s users, it’s anything.  We have made the returns of this data into standard formats, XML, RSS, JSON.  But way not the requesting of it?

 4. Syncing(No, please no).  Websites are going to store the information that they know about you, about your network on there servers.  The want to provide fast, reliable, and often profitable services.  The are going to keep a copy of the data that they need in order to make that happen.  So what is needed as a method (a standard)  for Syncing changes.  And that is a complication. Diffs, merges, revision numbers, no user will want to deal with that.  And the implementation would be left up to the service?  what if some dont care / dont have time or resources and don’t provide means for you to sync?

 5. Central, but in a good way.   We are ready agreed that centralized is bad (well at least I agreed that centralized is bad.) But much like the Open ID concept,  profiles need to be centralized to a location, tho that location is dependent on the user.  One user might choose Yahoo to keep the master of their profile.  Yahoo being a profile provider.  Maybe the user decides to keep it on their own server, whatever the case, that would be the end all be all of that users online profile.  Much like an Open Id provider.  Then whatever the service that you use on the web, for photos or news or video, or anything would and should access your master profile.(and do so on a regular basis)  The should draw information about you, from their, and anything that you added to your social network from the photo service that your are using, should be updated in your master profile.  

    Though some of your data my never leave the service (the photos for example)  there should be a clear, accessible way for them to be called from your master profile.  Your master profile not only should define you, it needs to catalog, define, and track all of your content on the web.

      This post may have gotten off track here at the end, but hopefully it was not too hard to follow.  This is the next big thing that faces the internet.  As we continue to expand our social networks, the sites that we choose to share that information with, and the burden of updating profiles / tracking web assets / and connecting with new services, this step becomes necessary, perhaps without it it becomes a roadblock to the future of the web.  

     I think I might have mentioned:  My ideas are not perfect, nor fully defined.  We need discussion, we need input , and we ideas.  Please visit www.dataportability.org/, please comment, this issue involves you too.