State of Tweekly.fm - December 2014

It's been quite a few months for the tweekly.fm project. The original plan was to rewrite the user facing side of the service. Instead of just this, the entire platform was rewritten. The API's that power the publishing of updates and collection of user data were rewritten from the ground up.

The site and APIs have also been moved over the new servers and archiecture and now are loadbalanced which allows around 10,000 connections a second - this will mean you should no longer get timeouts on Sundays and more updates can be sent.

You will now have one tweekly.fm account that you can add connections to. This includes support for Twitter, Facebook, Tumblr, Wordpress, AppDotNet and more to come. Each one of these services can be independently configured.

This has enabled me to begin work on developing iOS and Android apps that can interact with the tweekly.fm service. This includes managing your account and posting new updates from your phone.

There will be two versions of the tweekly.fm app. The free version will allow premium users to manage their account and post updates across any of their linked accounts. There will also be a low cost ($0.99/£0.69) version which allows you to create updates on your phone - independent of the tweekly.fm service.

The people who donated earlier this year will receive a free upgrade to a premium account on the new platform and will be able to use the free version of the app. So that you're aware, enough was raised earlier this year for about two weeks work on the service and platform.

If you're interested in beta testing the new iOS app then please complete this the form at http://beta.tweekly.fm and I'll add you to the list.

The launch date for both the new apps and website is going be 11th January 2015. This is due to a few things that still need ironing out with the frontend of the website - although the apps may land in the app store before this date.

I'm still looking for sponsors/advertisers for the site as well, if you're interested please contact [email protected]. I've also been considering using crowdfunding. If you would be willing to participate in crowdfunding more development of tweekly.fm, then please post in the comments and I'll see how much reaction there is and whether to set something up.

It's been quite a few months for the tweekly.fm project. The original plan was to rewrite the user facing side of the service. Instead of just this, the entire platform was rewritten. The API's that power the publishing of updates and collection of user data were rewritten from the…

Read More

Fix Messages Stuck Offline on OSX

If you have an issue with Messages on OSX being stuck in 'Offline' mode that won't connect, open up Terminal and type the following:

ps auxwww | grep imagent

This will give you output similar to:

scott             341   0.0  0.3  2502924  12828   ??  S     1:41pm   0:04.75 /System/Library/PrivateFrameworks/IMCore.framework/imagent.app/Contents/MacOS/imagent
scott           23666   0.0  0.0  2432772    460 s003  R+    9:25am   0:00.00 grep imagent

The number you're looking for is 341, which is the process ID of imagent.

Now, type:

kill -9 341

and the process will be restarted, allowing you to change status to online.

If you have an issue with Messages on OSX being stuck in 'Offline' mode that won't connect, open up Terminal and type the following: ps auxwww | grep imagent This will give you output similar to: scott 341 0.0 0.3 2502924 12828 ?? S 1:41pm 0:04.75 /System…

Read More

SQLSplitter

If you need to split an SQL file into smaller chunks, I found this little utility called sqlsplitter that will do exactly that. You can download it from http://rodo.nl/index.php?page=mysql-splitter.

Pretty simple use, stick your SQL file into a directory along with the executable and then run:

sqlsplitter your-sql-filename.sql 5000000

Where 5000000 is the number of bytes you want to split files into.

If you need to split an SQL file into smaller chunks, I found this little utility called sqlsplitter that will do exactly that. You can download it from http://rodo.nl/index.php?page=mysql-splitter. Pretty simple use, stick your SQL file into a directory along with the executable and…

Read More

Using VMware vSphere/ESXi as a Provider for Vagrant

Please note: This post assumes you are comfortable with Vagrant, Visualisation and generic admin tasks on a server.

vSphere & ESXi

One thing I've been wanting to do for a while was control my VMWare ESXi based hosts with Vagrant, so that I could easily create and remove boxes while testing. By default, Vagrant can't do this. Similar to how the interaction with Digital Ocean works, you can install a plugin to aid in getting this to work.

The vagrant-vsphere plugin has been created by the National Snow and Ice Data Center team and open sourced for others to use. You can read more information about it on the Github page for the project at github.com/nsidc/vagrant-vsphere/. There are more configuration options that I don't mention here that you can tinker with once you are more familiar with how the setup works.

To start, grab the ISO file of your distribution of choice. I chose to go with Ubuntu 14.04 as it is an LTS version and more recently the projects I've been working on have used it. You're going to create a Virtual Machine that will serve as a base for all of your Vagrant instances that you'll create in future.

Upload the Ubuntu ISO to your datastore and remember where you placed it. Next, create a new virtual machine with a reasonable amount of RAM and CPUs for your box, select the ISO in the CDROM options and boot up your new virtual machine as normal. Set up the box as you usually would do, I left pretty much everything as default apart from the user which you'll need to setup for Vagrant to use.

During the install you will be prompted to create a user account. Use the username vagrant and the password vagrant, this is required for vagrant to run later down the line.

Complete the installation process, installing any additional software that you would like such as Nginx, Apache, MySQL etc.

After the machine has rebooted, login and run the update commands:

sudo apt-get update -y
sudo apt-get upgrade -y

Install all the updates and then reboot:

sudo shutdown -r now

Next, install the open-vm-tools package which provides VMWare integration for the guest:

sudo apt-get install open-vm-tools -y

Again, after they're installed, reboot:

sudo shutdown -r now

Next, you need to add the vagrant user to the sudoers file:

sudo su -

Enter the vagrant user password that you set to vagrant earlier and then run the following command to edit the sudoers for the machine:

visudo

At the end of the file, add the line:

vagrant ALL=(ALL) NOPASSWD:ALL

Exit the editor by pressing CTRL+X, Save changes. Next we need to add Vagrants key to the user account:

mkdir -p /home/vagrant/.ssh
wget --no-check-certificate https://raw.github.com/mitchellh/vagrant/master/keys/vagrant.pub -O /home/vagrant/.ssh/authorized_keys

After the file has downloaded, reset permissions to the correct ones:

chmod 0700 /home/vagrant/.ssh
chmod 0600 /home/vagrant/.ssh/authorized_keys
chown -R vagrant /home/vagrant/.ssh

Now we'll need to install the OpenSSH server (if you haven't already):

sudo apt-get install -y openssh-server

After installation is complete, edit the configuration file at /etc/ssh/sshd_config and make sure that following options match the below:

Port 22
PubKeyAuthentication yes
AuthorizedKeysFile %h/.ssh/authorized_keys
PermitEmptyPasswords no

Restart the SSH server:

sudo service ssh restart

Now we have our template box. In your vSphere client, right click on your newly created virtual machine, select Template, then select "Clone to Template". Follow the steps and give it a good name that you'll remember.

Creating a Template in vSphere

As you can see in the image above, I called mine ubuntu.template.dc.sw10.net which is sitting in the resource pool Linux in the host 192.168.1.2 and that host resides in the datacentre dc.sw10.net. This is just a sample configuration and you can adjust yours to suit.

Installing the Vagrant Plugin

If you've got this far, I'm going to assume that you have Vagrant installed on your on your machine. Run the following command to install the plugin that we're going to use:

vagrant plugin install vagrant-vsphere

You can learn more about the plugin and it's various options over on the Github page at github.com/nsidc/vagrant-vsphere/. There are quite a few settings that you can change which I won't detail here.

Creating a Vagrantfile

To create a Vagrantfile that we can run, we'll need to add a few configuration options in to instruct the plugin on where to locate things. I started with the following Vagrantfile:

Vagrant.configure("2") do |config|
  config.vm.box = 'vsphere'
  config.vm.box_url = 'https://vagrantcloud.com/ssx/boxes/vsphere-dummy/versions/1/providers/vsphere.box'

  config.vm.provider :vsphere do |vsphere|
    # The host we're going to connect to
    vsphere.host = '192.168.1.253'                            

   # The host for the new VM
    vsphere.compute_resource_name = '192.168.1.2'            

    # The resource pool for the new VM
    vsphere.resource_pool_name = 'Linux'                    

    # The template we're going to clone        
    vsphere.template_name = 'ubuntu.template.dc.sw10.net'    

    # The name of the new machine
    vsphere.name = 'bingo'                                     

    # vSphere login
    vsphere.user = 'root'                                    

    # vSphere password
    vsphere.password = 'password'                            

    # If you don't have SSL configured correctly, set this to 'true'
    vsphere.insecure = true                                    
  end
end

Save that file, double check your credentials and then run Vagrant:

 vagrant up --provider=vsphere

You can see progress in the task/event log in vSphere, but you should see the template being cloned and then the machine booting:

vSphere showing created VM and clone in operation

After this has completed, you'll have something similar to the following output where you ran vagrant up:

 ➜  vagrant up --provider=vsphere     
 Bringing machine 'default' up with 'vsphere' provider...
 ==> default: Calling vSphere CloneVM with the following settings:
 ==> default:  -- Template VM: dc.sw10.net/vm/ubuntu.template.dc.sw10.net
 ==> default:  -- Target VM: dc.sw10.net/vm/bingo
 ==> default: Waiting for SSH to become available...
 ==> default: New virtual machine successfully cloned and started
 ==> default: Rsyncing folder: /Users/scott/Downloads/tmp/ => /vagrant

You can now use the other Vagrant commands such as ssh:

➜  tmp  vagrant ssh     
Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-32-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

 System information disabled due to load higher than 1.0

Last login: Sat Oct 11 13:30:21 2014 from 192.168.1.3
vagrant@ubuntu:~$ hostname
ubuntu
vagrant@ubuntu:~$

and of course you can vagrant destroy too:

 ➜  vagrant destroy
==> default: Calling vSphere PowerOff    
==> default: Calling vSphere Destroy

I'll most likely update this article soon with provisioning Nginx, PHP etc. If you need a dummy box to use with vagrant-vsphere, then I've added one to Vagrant Cloud at:

https://vagrantcloud.com/ssx/boxes/vsphere-dummy/versions/1/providers/vsphere.box

Please note: This post assumes you are comfortable with Vagrant, Visualisation and generic admin tasks on a server. vSphere & ESXi One thing I've been wanting to do for a while was control my VMWare ESXi based hosts with Vagrant, so that I could easily create and remove boxes while…

Read More

OSX: Format an Encrypted Drive

If you need to format an encrypted disk on OSX, you'll need to use the command line diskutil as Disk Utility won't let you do anything with the device.

First, find the ID of the disk you wish to format, using:

diskutil list

Which will list your disks, for example I wish to format the 2TB drive, I'd use disk2

/dev/disk0
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *250.1 GB   disk0
   1:                        EFI EFI                     209.7 MB   disk0s1
   2:                  Apple_HFS Macintosh HD            249.2 GB   disk0s2
   3:                 Apple_Boot Recovery HD             650.0 MB   disk0s3
/dev/disk1
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.0 TB     disk1
   1:                        EFI EFI                     209.7 MB   disk1s1
   2:                  Apple_HFS Data                    477.0 GB   disk1s2
   3:                  Apple_HFS Scott                   522.8 GB   disk1s3
/dev/disk2
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *2.0 TB     disk2
   1:                        EFI EFI                     209.7 MB   disk2s1
   2:                  Apple_HFS Time Machine            2.0 TB     disk2s2

We can then go ahead and format the drive (remember this will format your drive and remove all data from the disk):

diskutil eraseDisk JHFS+ Test disk2

Replace the label Test with the name you wish to give the new partition. After you've done this, you can edit the disk normally with Disk Utility.

If you need to format an encrypted disk on OSX, you'll need to use the command line diskutil as Disk Utility won't let you do anything with the device. First, find the ID of the disk you wish to format, using: diskutil list Which will list your disks, for example…

Read More

Install Mailcatcher on Laravel Homestead

Update - October 18th 2014:

The wonderful Chris Fidao noted in the comments that if you're not using Laravel Homestead and are using a different box, you may need to install the build-essential package too. You can do that with:

sudo apt-get install build-essential

After that's installed the instructions below should be fine.


If you're looking to debug outbound email easily, then mailcatcher will be a life saver for you. It'll give you a nice interface to view outbound emails:

Mailcatcher

To install it, ssh to your box and follow these steps:

sudo apt-get install ruby-dev libsqlite3-dev

Once that's installed, you can install the gem with this command:

sudo gem install mailcatcher

After mailcatcher is installed, you can run the program with:

mailcatcher --http-ip 0.0.0.0

If you've using Vagrant then you'll need to forward port 1080 from the guest to the host machine as well.

Update - October 18th 2014: The wonderful Chris Fidao noted in the comments that if you're not using Laravel Homestead and are using a different box, you may need to install the build-essential package too. You can do that with: sudo apt-get install build-essential After that's installed the instructions below…

Read More

Add Logfile to Memcache/Memcached Daemon on Linux

If you need to add a log file for memcache to help debug an issue, you can either use the following syntax:

memcached -d -m 3072 -l localhost -p 11211 -u nobody -v 2>>/tmp/memcached.log

or if you're on RHEL/CentOS, you can edit the file /etc/sysconfig/memcached like so:

PORT="11211"
USER="memcached"
MAXCONN="3048"
CACHESIZE="256"
OPTIONS="-vv >> /var/log/memcached 2>&1"

Save the file, restart memcache and you'll have a log file. This was taken from this StackOverflow question.

If you need to add a log file for memcache to help debug an issue, you can either use the following syntax: memcached -d -m 3072 -l localhost -p 11211 -u nobody -v 2>>/tmp/memcached.log or if you're on RHEL/CentOS, you can edit the file…

Read More

Windows: Add user to group via Command Line

I recently had to fix a Windows machine that had a broken user profile and needed to add the new user to the admin group.

A few Google searches later and this is the command to run in the command line prompt:

net localgroup group user /add

You can find this and more syntaxes at the helpful windows-commandline.com.

I recently had to fix a Windows machine that had a broken user profile and needed to add the new user to the admin group. A few Google searches later and this is the command to run in the command line prompt: net localgroup group user /add You can find…

Read More

Laracon EU 2014

[Header Image]

Intro

On August 25th, I headed over to Amsterdam to attend Laracon EU 2014. I wasn't too sure what to expect this year, I'd taken a lot away from last years event and was hoping this would be similar.

Community Day

The biggest difference from Laracon EU 2013 was that we had a community day to attend at De Nieuwe Liefde. This turned out to be an inspired choice of venue. Both the building and staff were pleasant.

The talks through the community day were great. I attended the following talks:

  • "Handling Highly Connected Data with Neo4j Graph Databases" by Michelle Sanver
  • "Getting Started with Package Development for PHP and Laravel" by Hannes van de Vreken
  • "Filesystem Abstraction with Flysystem" by Frank de Jonge
  • "TDD: The Good Parts" by Adam Wathan
  • "Coding Like a Girl" by Gabriela D'Ávila
  • "The use of multiple Laravel applications within a huge sports platform" by Dirk Jan Menkveld
  • "Q&A Panel" with Taylor Otwell

Day 1

Day 2

Evenings & Drinks

Photos

A few people other than me had taken photos, I've listed the ones I'm aware of below. If you have taken photos, leave them in the comments and I'll add them to this list.

Flickr - Gabriela D'Ávila

Flickr - Caryn Humphreys

[Header Image] Intro On August 25th, I headed over to Amsterdam to attend Laracon EU 2014. I wasn't too sure what to expect this year, I'd taken a lot away from last years event and was hoping this would be similar. Community Day The biggest difference from Laracon EU 2013…

Read More

Twitpic is Closing Down

I read this week about Twitpic closing down and it left me feeling disappointed. It's not the first time that Twitter have pulled a move like this. For me, the difference with Twitpic is that it was one of the first services that offered something Twitter didn't do at the time.

I believe they are making a mistake with the way in which they've treated the developer community in the past couple of years. The community building applications and consuming the platform is what fuelled their early growth.

It's a shame they seem to have forgotten that.

I read this week about Twitpic closing down and it left me feeling disappointed. It's not the first time that Twitter have pulled a move like this. For me, the difference with Twitpic is that it was one of the first services that offered something Twitter didn't do at the…

Read More

Slack Beta for Mac with Multiple Team Support

If you're a Slack user (and you should be) and wish to use the Mac app, it's a bit tedious for those of us who are in multiple teams. They've recently released a beta version that has support for multiple teams in a much better interface.

You can download it from HockeyApp at this URL:

https://rink.hockeyapp.net/apps/06bd6493684f65a3b8f47aca92c9006e

If you're a Slack user (and you should be) and wish to use the Mac app, it's a bit tedious for those of us who are in multiple teams. They've recently released a beta version that has support for multiple teams in a much better interface. You can download it…

Read More

Syncable Host File to Make the Internet Suck Less

We've all been there. A friend sends you a link to an internet shock site. If you're familiar with the hosts file on your machine, then you can use this large sync-able list to compliment your own rules. If you using a Mac, you can use the nifty Gas Mask program to edit your hosts file without having to jump into a shell.

URL for the hosts file is:

http://someonewhocares.org/hosts/hosts 

As well as IPv6 files, there are a few other formats available as well.

We've all been there. A friend sends you a link to an internet shock site. If you're familiar with the hosts file on your machine, then you can use this large sync-able list to compliment your own rules. If you using a Mac, you can use the nifty Gas Mask…

Read More

Virtualmin - FTP/SFTP File Writing Errors

If you're using Virtualmin on your servers and have a few odd FTP error messages (happened for me using SFTP too), then it's one of three things.

  • Disk quota for the hosting account has exceeded
  • Server has ran out of memory
  • The configuration options below need to be set in /etc/proftpd.conf

As mentioned above, I found that these two options added into /etc/proftpd.conf fixed the file write issue in some cases:

AllowRetrieveRestart on
AllowStoreRestart on

If you're using Virtualmin on your servers and have a few odd FTP error messages (happened for me using SFTP too), then it's one of three things. Disk quota for the hosting account has exceeded Server has ran out of memory The configuration options below need to be set in…

Read More

Useful Notes for phpinfo() & Capturing Output of phpinfo()

I recently needed to capture the output of phpinfo() for a to a log upon install. I'd been able to do this in the past but couldn't quite remember how I managed it. This is where ob_start() and ob_flush() come into their own.

To get the contents of phpinfo(); into a variable, you can use the following:

ob_start();
phpinfo();
$strPhpInfo = ob_get_contents();
ob_clean();

The variable $strPhpInfo will now contain the output from phpinfo().

A side note, if you only want the variables section of phpinfo(), then use:

phpinfo(32);

and it'll give you just that in the output. You can read more about the other values you can pass to phpinfo() at php.net/manual/en/function.phpinfo.php.

I recently needed to capture the output of phpinfo() for a to a log upon install. I'd been able to do this in the past but couldn't quite remember how I managed it. This is where ob_start() and ob_flush() come into their own. To get the contents of…

Read More

PHP ActiveRecord - Get Last Query

I recently had to do some work on an older site that used the PHP implementation of ActiveRecord. One of the issues I had was that the queries being ran weren't produced the expected result. I found two commands to help with this an issue the last run query:

If you're within a model, you can do:

echo self::connection()->last_query;

and if you're outside your model, you can use:

echo model::connection()->last_query; 

I recently had to do some work on an older site that used the PHP implementation of ActiveRecord. One of the issues I had was that the queries being ran weren't produced the expected result. I found two commands to help with this an issue the last run query: If…

Read More

Install Old Version of Laravel 4.1.X for PHP 5.3 Servers using Composer

If you need to install an older version of Laravel for use with a server that has PHP 5.3.X for example, you can do so using the following command:

composer create-project laravel/laravel project-directory 4.1.*

This is useful when you can't upgrade the version of PHP but still wish to use Laravel.

If you need to install an older version of Laravel for use with a server that has PHP 5.3.X for example, you can do so using the following command: composer create-project laravel/laravel project-directory 4.1.* This is useful when you can't upgrade the version of PHP but…

Read More

Almost Time for Amsterdam!

Preparing for Laracon EU 2013

It's a couple of weeks away until I head off for Amsterdam.

I'll arrive in Amsterdam on the 25th August of August and depart on the 31st. I'll be in Amsterdam to attend the fantastic Laracon EU conference along with a work colleague.

We're currently in the process of making plans of what to do while we're there, we got a lot of the common tourist things done last year, so this year it'll be good to see some new places.

If you have experience of renting a boat of in the 'dam too, I'd like a chat.

If you've got any suggestions, leave them in the comments below. If you know anyone in Amsterdam with good local knowledge, point them in the direction of this post too!

It's a couple of weeks away until I head off for Amsterdam. I'll arrive in Amsterdam on the 25th August of August and depart on the 31st. I'll be in Amsterdam to attend the fantastic Laracon EU conference along with a work colleague. We're currently in the process of making…

Read More

Interactivity for Payment Forms

Interactivity for Payment Forms

A short while ago now I read an article by Michael Villar of Stripe on using animations when users are completing payment forms.

Villar put forward the argument that this improves the user experience.

Below is a demonstration of doing that.

Interactivity for Payment Forms A short while ago now I read an article by Michael Villar of Stripe on using animations when users are completing payment forms. Villar put forward the argument that this improves the user experience. Below is a demonstration of doing that. …

Read More

Laravel: Filter Every Array Element with a Closure

I always forget about how awesome Laravel Collection's are. I came across this gem when needing to run a filter on every element of an array. I think it may of been Dayle's book that first informed me of them. For example, if you have an array of fruits that you want to filter down, you can do the following:

// Create an array of fruits
$array = ["banana", "apple", "orange"];

// Transform the array into a Collection object
$collection = new Illuminate\Support\Collection($array);

// We don't like banana's anymore, so we're going to filter them out
$no_bananas = $collection->filter(function($element) {
    if ($element != "banana") return true;
});

// Dump out our array now, and we'll see the banana's are gone
dd($no_bananas);

I always forget about how awesome Laravel Collection's are. I came across this gem when needing to run a filter on every element of an array. I think it may of been Dayle's book that first informed me of them. For example, if you have an array of fruits that…

Read More

Laracon US 2014

If you haven't noticed yet, the Laracon US 2014 videos have been posted online for a short while now, you can grab them over on Userscape's website at http://userscape.com/laracon/2014/.

One talk of note, which I've recommended to a lot of people is by Greg Baugues, titled "Devs and Depression".

If you haven't noticed yet, the Laracon US 2014 videos have been posted online for a short while now, you can grab them over on Userscape's website at http://userscape.com/laracon/2014/. One talk of note, which I've recommended to a lot of people is by Greg Baugues, titled…

Read More