My very first Hyperforest

So today I decided to look at little deeper at WHAT I want to plot on a Hypertree anyway. It's quite a difficult question. The purpose of all this, ultimately, it to be a browsing experience (not a reporting one) so I don't care if the diagram is an accurate reflection all that matters is that if it is accurate enough to flush out some interesting connections.


The diagram should look plausible but then have oddities in. So.... today I took a simplistic view that went...

1. What web documents is someone mentioned in
2. What topics are associated with those documents
3. What terms are those documents associated with (and which topics)
4. Who else is on the documents that have those terms

I also threw in some relevancy numbers for good luck ( thanks @TomTague for the tip)

The end results are a quite good...

Gustav Delius - a mathematician who is involved in lots of web projects looks like this...

 Trevor Sheldon - the Deputy Vice Chancellor looks like this...

Roger Burrows - Sociologist who might be said to have concerns about technology and privacy (forgive me if I'm wrong Roger)... looks like this...

Sue Hodges - Educator definitely not into social media self-promotion looks like this...

It's looking promising. The pictures look great... now if I can just fathom what they mean I'll be laughing.


Tomorrow I need to research talking to the ZODB and how to make custom object types in Plone so that I can take this data and integrate it into a social networking site we're running.

My Very First Hypertree

Sometimes, in fact most of the times for me, I find it difficult to work with certain concepts until I can see them. I began today by making the templates that could just show the data I have gathered from Open Calais so that I could get a feel for what's there.

And just now I started adding a little Hypertree magic. Here's some of the data displayed quite simply.

The funny shape makes me realise that I've made a completely arbitrary three-way split of pages, people and terms and that I need to make it build its hierarchy dynamically. Ooh that one hurts.

It looks even stranger when someone happens to have been in lots of pages about meeting minutes. Here's Jo Casey as a Hypertree.

I'm pleased that I'm able to display something. I now need to work on the user interface and add a nice search interface and popup areas for "people information" and maybe to be able to load more data as you click on a node.

I also now need to think about which connections between people are most important and perhaps how to structure the industry terms.

Sprint 1 Done!

York University's web site has been crawled. The data was then manipulated (or added to) using Open Calais - although I still have to leave that cooking a while longer. And then using the Google Visualisation API I created a tag cloud of what are called the top "social tags" for the semantic information.

The entities returned look really interesting. I need to find out how they relate to each other and how I can find connections between peoples.

The next step is to clean up the code. Make it semi-functional so at very least you can explore the data and then put it into Google Code ... with some instructions! Which will look something like this...

1. Install Django and dependencies

2. crawl (http://www.york.ac.uk)

3. analyze()

4.  ./manage.py runserver

What a week...

Brief Project Description

The aims of this project are to help enable collaboration at the University of York by making people and their work more easily discoverable and more richly linked. Currently at York the people directory is very basic (shown below) revealing nothing of a person's work, interests, colleagues or personality.

This project looks to create richer connection between people using...

• mined data - such as web pages containing their email address or blog posts or Google queries
• known data - such as research repositories, such as CiteSeer, Mendeley, ePrints at White Rose Consortium or CiteULike
• known organisational data - such as what department somebody belongs to
• social media usage - such as Twitter followers, LinkedIn Connections and Delicious tags

Above is a wireframe showing a simplification of the sorts of elements we expect to be able to present, including tags, people, links and related articles.

The Caveats

Of course, it would be easy to believe that creating an Amazon-esque recommendation list (below) is easy. Creating these sorts of tools can be deceptively difficult with the development always on the brink of a breakthrough and never quite able to fully deliver.

Having experience of these pitfalls this project will initially aim to implement the simplest technological solution available. We will begin by looking at the known, later extending towards the reasoned and guessed connections. We will begin with explicit connections and later explore semantic relationships.

Rather than beginning with the technology we will simply ask people which social media tools they use with which to pre-populate our knowledge of people. This will take the form of survey. In addition to asking people for their social media usage the tool should allow people to explicitly make connections. 

So that this tool doesn't become a technological folly, presenting beautifully visualised connections between people but never actually being used, the presentation of this data and these connections needs to be embedded in an everyday online environment. By that, I mean that this tool should not be a destination in its own right but should be stumbled across naturally whilst looking someone up or engaging in discussion.