Archive for February, 2008

Notes: installing Django / Python / Mysql on OS X

Monday, February 18th, 2008

Trying to branch out in my scope of web development, being mostly a PHP and .NET developer, I decided to try Python. Now I still do not know much about Python development, packages, or even the syntax. Really I am just getting started with it. But I thought I would link to a couple of articles that helped me get setup to develop under OS X for anyone else starting out.

Why try Python?

I started watching some interviews on the yahoo developer network. One in that caught my attention was with Matt Mullenweg, founder of Automattic and developer of wordpress. When asked the question, if you were to start out on a project the size of wordpress today, is there any other language that you would use/consider? And sure enough he answered Python. For the pass couple of years I have thought about using python, and I even once installed it. But that has been as far as I have gotten.

I also started using the website Pownce.com recently. Pownce is a social network allowing you to send messages, files, links, images and video to your friends easily. Well, Pownce who just opened up their site to the public (they where in a private beta for about 6 months) uses the django framework for python for the whole site.

Last Friday I watched a video by Jacob Kaplan-Moss one the the lead developers of django, and found some the of the concepts really interesting. Django was and is developed out of Lawerance Kansas, you know the center of web development. It was built by the web team for the local newspaper in order to better mange the many sites they host and their constantly changing needs. Jacob explains that better than me, if you watch the video. I downloaded the Framework on Friday night (as I was driving back from Kansas City Saturday and Sunday) as something to look at on the way.

Getting Started

Well fortunately Python is included in Leopard, so I didn’t have to download that over my gprs connection in Kansas. Django is very easy to setup, basically run a script from inside the extracted django directory.

sudo python setup.py installOk with that installed where do you start? Django provides a nice walkthrough of setting up a project, what files are included and a step through of the built in features. After following about a page of the walkthrough I got to the point of being ready to sync my DB code with the real database, MySQL.

python manage.py syncdb

You then get this:

python manage.py syncdb
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/Library/Python/2.5/site-packages/django/core/management.py", line 1672, in execute_manager
execute_from_command_line(action_mapping, argv)
File "/Library/Python/2.5/site-packages/django/core/management.py", line 1571, in execute_from_command_line
action_mapping[action](int(options.verbosity), options.interactive)
File "/Library/Python/2.5/site-packages/django/core/management.py", line 486, in syncdb
from django.db import connection, transaction, models, get_creation_module
File "/Library/Python/2.5/site-packages/django/db/__init__.py", line 11, in <module>
backend = __import__('django.db.backends.%s.base' % settings.DATABASE_ENGINE, {}, {}, [''])
File "/Library/Python/2.5/site-packages/django/db/backends/mysql/base.py", line 12, in <module>
raise ImproperlyConfigured, "Error loading MySQLdb module: %s" % e
django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No module named MySQLdb

So where as Python is installed by default under OS X, the mysql package is not.

How to install MySQLdb Package for Python.

First I went to the Mysql Downloads page and got a copy of the MySQLdb Package

Download the package, extract it, and open Terminal.app and change directories to the newly extracted package.

Here is where I ran into some fun, how do you install this? Well I found a website that suggested the Easy Installer…and that sounded nice, so I downloaded it first.

Download ez_setup.py, and run it; this will download and install the appropriate setuptools egg for your Python version.

There are many ways on install a package with the Easy Installer, I choose the directly from an unzip source folder, since that is what we have. And you can find more directions about the Easy Installer here

Now that you have Easy Installer, well installed, from terminal.app in the extracted MySQLdb Package MySQLdb folder run:
easy_install .

Got some errors, yeah we all did.

First, python will not be able to find your mysql installation (you did install mysql right?) If you didn’t get the .dmg file from the Mysql website, and install it through their very easy and very friendly installer, including a preference pane. The only problem with this it it puts itself in a non-standard folder, and that will cause a couple of issues.

Python will not be able to find mysql_config, or mysql for that matter because it is not in your path. To add it to your path add this line to the .profile file in your home directory.
PATH="${PATH}:/usr/local/mysql/bin"
Save that, and close terminal.app, and reopen it, This will cause the .profile to be re-read and your new addition to the path to be evaluated. Go ahead a try, now from the command line you should be able to call mysql_config, and any other mysql executable for that matter.

Try and run that easy_install command again, and yes you will still get errors.

Now you should get something like:
Running MySQL-python-1.2.2/setup.py -q bdist_egg --dist-dir /var/folders/At/At0AMaeKEAaYVW-LYfH6e++++TI/-Tmp-/easy_install-BUSsM_/MySQL-python-1.2.2/egg-dist-tmp-qFBtdU
In file included from /usr/local/mysql/include/mysql.h:43,
from _mysql.c:40:
/usr/include/sys/types.h:92: error: duplicate ‘unsigned’
/usr/include/sys/types.h:92: error: two or more data types in declaration specifiers
error: Setup script exited with error: command 'gcc' failed with exit status 1

Now to edit a couple of the files that came with the MySQLdb package. First _mysql.c
comment out the following lines:

#ifndef uint
#define uint unsigned int
#endif

Also edit the site.cfg file and change

threadsafe = True
to
threadsafe = False

Next, just one more Mysql Path fix run the following command:

sudo ln -s /usr/local/mysql/lib/ /usr/local/mysql/lib/mysql

Python will now be able to find the lib files where they should be. Ok that is it, run the easy_install . command and everything should go smoothly. As far a django is concerned, just continue to follow the walkthrough and you should do fine. I hope this helped.

Here are the websites that helped me, So if you run into any other issues these would be a good place to start:

Social Graph & Data Portability

Wednesday, February 13th, 2008

   Wow, that is a broad subject.  It is also one that is getting a lot of attention as of late.  First, there was the Internet, Next the need for Social networks, Next the need to have your social network follow you wherever you go.  How do you do that?Currently there are many (hundreds?  thousands?) of Social networking / or (something) built around a social network websites.  There are the full blown MySpace, Facebook, Hi5, etc.  These sites are built for social interaction, it is not a by-product, it is the main purpose.  Next you have the websites devoted to Photos, Video, and News.  These websites also include social networking features in order to attract/retain users. (And to improve their service, but that is a minor detail) Flickr, a simple idea upload you photos..oh and build a social network.  Digg, vote on news…oh and build a social network.  Youtube, *I’m going to trust you to fill in the rest of this sentence, be careful.* .

    Each of this sites comes with, or has added a social component.  It has  made the service better, more useful, and barring a few exceptions more fun.  Here is the big exception to the better-ness/usefulness/fun-ness:  I filled in my data, I filled in my friends, I filled in my profile, my network over and over (and over again).  

    The issue is that I need a global Social network, without the copy and paste. And forget filling things in the first time…that is painful enough, but keeping it all up-to-date is all the more horrid.

    So how do you make you social network follow you, how can you centralize your info.  Well, facebook is aiming to become that central point for your social network with some of their new APIs.  But, that is not going to work for me.  No One Person / Site Should Control MY Network.  No Way.  No How.  Though a central location (Google, facebook, myspace, or joebruan2312.net) would be easy to query, easy to find, I have major trust issues, future of the system issues, and adoption issues (not that I’m looking for new parents, but if you got a lot of money drop me a line anyway, thanks.) The only way I could even dream of the centralized point for this data, is something open source, run by a non-profit foundation.  A wikipedia/mozilla/OPEN system.  But even then we have many adoption and future management  issues.

    It needs to be decentralized (like what we have, sigh), but with a common/standard (scary) way of accessing every social network(hooray!).  Intro www.dataportability.org/ stage kinda almost next to center.  They have a start (meaning the got a logo designed, which does look cool *thumbs up*)  on a draft of a draft of a proposed standard.  check them out, weigh in on the discussion, this effects you too…and maybe U2.  Here as the aspects of this that need to be in place as I see it, please keep in mind that I only got like 2hrs of sleep last night, the current time is after midnight, and I have only been thinking about this for 10-20mins.  

1. Service Definition.  A *standard* programatic way to ask a website:  what services do you provide?, where are they?, what do I need in order to use them?

2. OpenID…I think we have enough big players behind it now, but it will still be a far bit of time before any kind of wide scale implementation is seen across the web.

3.  Standard Data Requests I have been calling a lot of APIs recently, they are all different, they all have their own names for everything, and they all have different limits.  But most of them are for the same data.  Now I’m not the one to say that one approach is so much better than another, but I will say that We should see *standards*, yes standards.  Imagine if I code call information about users from Flickr, Digg, Facebook, etc.  all in the same way.  Imagine a standard that defined how I query a website about it’s data, it’s users, it’s anything.  We have made the returns of this data into standard formats, XML, RSS, JSON.  But way not the requesting of it?

 4. Syncing(No, please no).  Websites are going to store the information that they know about you, about your network on there servers.  The want to provide fast, reliable, and often profitable services.  The are going to keep a copy of the data that they need in order to make that happen.  So what is needed as a method (a standard)  for Syncing changes.  And that is a complication. Diffs, merges, revision numbers, no user will want to deal with that.  And the implementation would be left up to the service?  what if some dont care / dont have time or resources and don’t provide means for you to sync?

 5. Central, but in a good way.   We are ready agreed that centralized is bad (well at least I agreed that centralized is bad.) But much like the Open ID concept,  profiles need to be centralized to a location, tho that location is dependent on the user.  One user might choose Yahoo to keep the master of their profile.  Yahoo being a profile provider.  Maybe the user decides to keep it on their own server, whatever the case, that would be the end all be all of that users online profile.  Much like an Open Id provider.  Then whatever the service that you use on the web, for photos or news or video, or anything would and should access your master profile.(and do so on a regular basis)  The should draw information about you, from their, and anything that you added to your social network from the photo service that your are using, should be updated in your master profile.  

    Though some of your data my never leave the service (the photos for example)  there should be a clear, accessible way for them to be called from your master profile.  Your master profile not only should define you, it needs to catalog, define, and track all of your content on the web.

      This post may have gotten off track here at the end, but hopefully it was not too hard to follow.  This is the next big thing that faces the internet.  As we continue to expand our social networks, the sites that we choose to share that information with, and the burden of updating profiles / tracking web assets / and connecting with new services, this step becomes necessary, perhaps without it it becomes a roadblock to the future of the web.  

     I think I might have mentioned:  My ideas are not perfect, nor fully defined.  We need discussion, we need input , and we ideas.  Please visit www.dataportability.org/, please comment, this issue involves you too. 

Project 1 – Aquila Command.

Friday, February 8th, 2008

From the latin word for Eagle, comes Aquila Command. A remote alert and control platform based around Eagle-i, a video surveillance system that tracks changes between frames and then alerts the users. (www.jemeztechnology.com)Aquila Command’s function is to take in a stream of images from Eagle-i process who gets them based on shift of the users/location of the alert, and send that out to client software, currently designed for windows XP/Vista and windows mobile 6. Of course there is also the ability to lookup images and review past alerts. And with almost any project there are a 100 other features that I would like to add, but I dont have the time right now.I’m doing this project for a percentage. And I will tell you that there are some projects that I enjoy, and that I might work on without payment, but this would not have been one of them. But to make money you have to spend money they say, so here I am spending my time and some money. This project has helped keep my .NET skills sharp, and added to my knowledge in certain areas, like the pain of windows mobile development. But that is a whole other post or 5 other posts worth of links, tips, and stories of what I did wrong.Just to touch on the status, we are at a point where we are almost ready to ship, (RC2). We are still testing, fixing a few things, and giving the makers of Eagle-i a chance to make notes. We are at the point where we can do some nice demos, and impress some people. Now just to ship! hopefully.

 

 

A blog really?

Tuesday, February 5th, 2008

Yep, a Blog.  I have written for the company blog in the past(Exead), but I think it will be better if I have my own blog, so I can post about a mixture of projects.  

The first project I want to talk about is this blog itself.  I have added a couple of new features to it, specifically it automatically pulls articles that I digg and photos that I post at digg and flickr.  I have to admit though that this was not my original idea, in fact I saw this  idea on  http://www.joestump.net.  If anyone knows of an addon for wordpress that does this kinda thing, please point me in that direction, because I would like to compare it to my own.  And if anyone would like the code that I used for this please leave a comment, and I will send you a copy.