StatusNet Articles

New pump.io server, signing off of StatusNet

Over the past two years I have been a big fan of StatusNet, the social software that is open source and works similarly to twitter. On this blog I have documented my experiences running the software on my own server and it has been a fun time.

Earlier this year Evan, the founder of StatusNet, announced that he had rewritten the engine of the software with JavaScript on node.js.

I believe this was a smart choice for this software as the Apache, PHP and MySQL parts of the StatusNet stack are incredibly heavy. Evan had been working on this new JavaScript rewrite for about a year before announcing it, and I have been running it in some form since January when it was announced.

In June I finally put my instance live at comn.ca. It had the unfortunate consequence of topping out the memory on my server so I decided to shut down my StatusNet instance (see graph attached).

The largest StatusNet site, identi.ca, is due to be converted to pump.io any day now. When that happens the pump.io network will overtake StatusNet. The PHP-based StatusNet will live on under the name GNU Social (new updates to come soon).

Some of my contributions to the StatusNet community were on my SN server. If anyone would like a copy of the themes I made please reach out to me.

If you have a pump.io account please follow me at my new instance: https://comn.ca/ryanweal

... want to try pump.io? This link will give you a random server that is accepting users. Don't like the domain? Just hit that link again and you'll get a different one.

For those interested, I have recreated my Chinese language flash card bot. It posts random words every 10 minutes.

PS. The attached photo shows pump.io AND StatusNet running concurrently, until I turned off SN. You can see the result... pump.io is smooooooth running.

PPS. I hope more Drupal folks end up on pump.io than were on StatusNet. I still find it hard to believe the Drupal community uses twitter rather than open source messaging platforms...

The Kafei-Infinite StatusNet theme

Announcing our latest StatusNet theme!

This new theme goes a bit further than our last one - we worked from the default StatusNet base theme this time. The goal with this theme is to support mobile devices, though we ran into a couple limitations of the software so your mileage may vary on that front.

The theme supports the InfiniteScroll module well, and should work fine with your Realtime plugin.

Best of all, you can change all of the colors with a simple custom.css file. We even provided the example files that were used to produce the images in this posting.

How to Install

  • Download the latest files for kafei-infinite: [tar] [zip]
  • Unzip or untar the files in your StatusNet theme directory. The theme folder should now contain a folder called kafei-infinite.
  • Edit the custom.css file in the theme/kafei-infinite/css folder with your own custom settings... OR use the code in one of the files listed in theme/kafei-infinite/css/alternates to overwrite the theme/kafei-infinite/css/custom.css file.
  • Add the theme to your config.php file: $config['site']['theme'] = 'kafei-infinite';
  • Run the scripts/checkschema.php file from the root of your site on the command line.
  • Enjoy!

Learn Chinese writing with our StatusNet account

Have you ever wanted to learn Chinese? I took a course in university and I did not do all that well! Truth be told, I spent most of my time in the course researching linguistics and I still find the subject fascinating today. I really enjoy learning new languages!

Recently I was thinking about how the first 1500-3000 characters in Chinese are all you really need to get by. This number is much lower than in English and French so many published lists of all the "basic words" are out there. This weekend I searched for such a list and the author made the contents available for re-use. Excellent. Shortly thereafter I realized I could parse that file and make it into a flash card program... using StatusNet.

The cron-bot I built posts a random character from the list of ~2700 or so characters in this list every 10 minutes. Why so often? So you always see new stuff coming in. Also, because you can go through the *entire* list in less than 20 days at this rate.

Check it out: http://status.kafei.ca/dawei

If you have an account on identi.ca or another StatusNet service you can subscribe to this URL. RSS is also available.

You could also just visit this page every now and then. Note, there is a "play" button and a pop-up button beside it. Those can be handy for watching live updates if you want to enable it on your office desktop. ^_^

Eventually I might mirror it to twitter. Let me know what you think on my contact form at http://kafei.ca/contact

How it was done

I found a listing of the 3000 basic characters (well, more like 2700+) at this website: http://www.zein.se/patrick/3000char.html and they even mention that re-use of the materials for flash cards are ok! Excellent.

I took this file and pasted it into an OpenOffice Calc spreadsheet and cropped everything down to just the rows and columns I needed. I then exported the file to CSV format, and then proceeded to perform a few regex operations on it to get the data structure just right. I actually did this twice - once after launch to fix a bug. When the CSV file was successfully mutated into a file compatible with the command line tool "fortune" I was ready to go. I compiled fortune's "dat" file and then created a cron job to post random selections to my StatusNet site.

Now a cron job will run every 10 minutes, ask fortune for a random word from the dictionary of the first 3000ish words, and post it to StatusNet.

Why do it this way?

Every day you read your StatusNet feeds - sometimes many, many, many times over the course of the day. It makes sense to put repetitive information like this into a feed that you are going to read over and over. It also makes sense on the level of... knowing when you learned something. That is, in what order you learned it. So scrolling back you hit that point that you've already read.

It also isn't particularly important information. If you miss it... so what. It will eventually come around again. The cycle sound run in roughly 20 days. It is possible for repeats. It is totally random... as much as the fortune program can be random.

It is also possible to comment on the postings to practice using the text. That could be pretty cool, especially if working with a translator to make corrections as you go.

http://status.kafei.ca/dawei

Getting real time updates in StatusNet

Tonight I installed Meteor on my server to allow my StatusNet site to update pages automatically as the updates come in. This is enabled via the Realtime plugin.

To enable the realtime module you will need:

  • A firewall running on your web server (or in front of it)
  • A copy of Meteor on the server
  • Start / stop script for meteor if you are not running Fedora
  • Firewall customization - port mapping
  • A domain name for the server which is visible externally

Installing Meteor

Consider following Meteor's installation guide

  • Create a folder called /usr/local/meteor
  • Download the Meteor source code
  • Untar the file in your /usr/local/meteor directory (yes, it will drop files everywhere, sigh!)
  • Add a non-root user for it to run: useradd meteor
  • Copy over the configuration file: cp meteord.conf.dist /etc/meteord.conf
  • Run meteor: cd /usr/local/meteor && ./meteord -d did it run? (control-C to close it after testing)
  • Test the listener port: telnet localhost 4670
  • Test the command port: telnet localhost 4671
  • The installation guide recommends you create a startup script (cp daemoncontroller.dist /etc/init.d/meteord), however it only works on Fedora. Below is the "Debian version" I created.
  • Be sure to /etc/init.d/meteor stop your test run once your startup script is installed so you can /etc/init.d/meteor start it with the right permissions.
  • Repeat the telnet tests once you are live with your firewall settings.

Debian Startup Script

This is what I'm using in place of the file the meteor creators recommend for /etc/init.d/meteor. This one will run in Debian.

#!/bin/sh
# description: Runs meteord
# chkconfig: 2345 99 00

# Source function library.
#. /etc/init.d/functions

case "$1" in
'start')
echo -n "Starting Meteord: "
#echo 65535 > /proc/sys/fs/file-max
ulimit -n 65535
cd /usr/local/meteor
su meteor -c ./meteord >/var/log/meteord 2>&1 &
echo
;;
'stop')
echo -n "Stopping Meteord: "
killall meteord && success || failure
;;
'reload')
echo -n "Reloading Meteord configuration: "
killall -s SIGHUP meteord && success || failure
;;
*)
echo "Usage: $0 { start | stop | reload }"
;;
esac
exit 0

Firewall settings

  • Block all external connections on port 4671
  • Map an unused external port (such as 81) to port 4670

Update your config file

The last step is to add the configuration to your StatusNet config.php file.

addPlugin('Meteor', array(
'webserver' => 'example.com',
'webport' => '81',
'controlserver' => '127.0.0.1',
'controlport' => '4671'
));
addPlugin('Realtime');

This detail actually comes from the StatusNet site.

That's it! If realtime is enabled correctly you will see a play/pause button at the top of your feed and a link to a pop-up window you can use to monitor the internets all day long.

If your updates aren't working go back and check that the external world can access the port you have assigned for reading from meteor (in my example I had to ensure 81 was mapped to 4670 correctly). I had to try a couple times to get it right.

Enjoy getting real time updates!

Exploring StatusNet and introducing the neo-kafei theme

I have been a big fan of identi.ca for quite some time now. If I could describe it very succinctly, I would say it is a twitter clone microblogging service, but it actually does much more than that.

identi.ca is actually an implementation of OStatus by a Montréal company called StatusNet. Over the holidays this year I took some time to install my own StatusNet site.

What does it do? Well. Let me tell you... it does a lot of things.

The basics

You know how you can have your own account on twitter?

What if you could have your own twitter?
Like, the whole thing. Public or private, as many people as you want messaging each other.

At the core, this is what StatusNet does. It lets you spring up an instant community. It also doesn't strictly limit your number of characters as some other services do... .

The federation

Why should I run my own microblogging service if I only have one user? Because you can still follow other people and interact with them even if they are hosted on a different OStatus service. As they like to say over at OStatus: "people on different networks following each other." Exactly.

Remember the fail whale? You wouldn't have the fail whale if you run your own StatusNet site, because if one server goes down (assuming it's not your own!) the rest of the network keeps going. That in terms of uptime is super cool. When identi.ca goes down for maintenance I still get updates from my friends hosted on other sites.

How to install StatusNet

  • Unzip the files into your hosting space
  • Permission the root folder and three additional folders (see INSTALL file -duh)
  • Create the database in mysql (requires innodb!)
  • Run the installer script
  • Configure additional options in the aptly named config.php file (I had to do this for the theme I made below)

It is best to have the same domain name you intend to go live with. You can change it... but other sites you follow may remember your old name!

The add-ons

Out of the box StatusNet will get you and your team microblogging, either in public or in private. You can also subscribe to anyone using an OStatus feed and reply to them within your site.

Geo-location is also enabled by default, so you can get a global map of how far your correspondence is traveling. URL shortening is included too.

Special configuration steps are required for:

  • "Realtime" live updating (as seen on identi.ca: ability to play/pause your feed)
  • Queuing - if you have lots of users
  • Bots - yes you can create your own robot
  • XMPP - chat integration with your instant messaging
  • Archiving - seems like there are lots of possibilities for integrating
  • Login - you can enable Oauth and other modules (LDAP, Facebook, etc)

I have done none of these! The order above reflects how important I think they are in priority sequence.

Our "neo-kafei" StatusNet Theme

The one thing I would really change about StatusNet is the theme. In an older version of identi.ca you could change your background color, and I did change it... to an offensive green color that was so bright it burned the eyes. It had the effect of washing out the other UI elements so I could focus on reading.

The new/current identi.ca theme, well... I think it is too distracting. The interface commands a lot of attention, and I really don't need a lot of hand holding on a website I visit many times a day.

I decided complaining wasn't worth it - that I could fix this problem myself.

So I posted a "revised" version of one of the included themes with all the color taken away. The interface is zeroed out to just the basics.

Here it is: the neo-kafei theme announcement thread!

The downloads are available here if you don't want to read the whole thread:

How to install it:

  • Unzip the package in your theme folder
  • Edit your config.php file
  • Add a line like this:
  • $config['site']['theme'] = 'neo-kafei';
  • Comment out any other items which contain the theme setting, like this:
  • //$config['site']['theme'] = 'neo';

If anything looks broken double check the files are all there:

theme/
|-- base
|-- neo
|-- neo-kafei
|   |-- css
|   |   |-- display.css
|   |   `-- mp-screen.css
|   |-- default-avatar-mini.png
|   |-- default-avatar-profile.png
|   |-- default-avatar-stream.png
|   |-- logo.png
|   |-- mobilelogo.png
|   `-- theme.ini

Note that you need to have the base theme, the neo theme, and the neo-kafei theme all in the theme folder or it will not work. Be sure you have all of the files displayed under the neo-kafei folder as I have outlined above (I'm assuming you left base and neo as-is).

*some people in the thread mentioned an issue with the tar file. If you can solve it, I would be very grateful if you could explain it. I use Unicode, fr_ca-UTF8 usually, and I often do not have old style ISO-88* encodings on my systems (很好) so maybe that causes issues if you don't have Unicode?.

Finding friends

Now that you're up and running, you'll want to find some friends whose updates you can follow. If you know some people on identi.ca you can go to their page there and click the "subscribe" button. When you paste in your profile address you will be prompted (at your own site) to confirm you want to follow this person.

The same thing can be said for groups. Go find the group... probably on identi.ca and then click "subscribe" and paste in your profile URL.

There is a really great group for people running their own StatusNet site over at http://parlementum.net/group/feds

And be sure to add me, too! http://status.kafei.ca/ryanweal