I recently got the book “MongoDB and PHP” by Steve Francia which had an interesting paragraph in the first chapter that talks about how the use of stored data has changed in recent years:
With all the problems with ORMs, you may wonder why programmers use them at all. People were willing to make the compromises to adopt ORMs for one big reason; PHP applications are by and large CRUD applications. Rarely do they use all of the rich features the relational database provides, so giving them up seemed a small price to pay for the benefit of simplified access to the data. Additionally, there weren’t really any other good options. For very simple projects, one could write SQL in one’s code, but this was hard to debug and even harder to ensure that it was done securely. PHP is famous for enabling SQL injection attacks, as inexperienced developers pass variables right into the SQL without sanitization.
Which lead me to think about how my own use of data has changed over the years.
Over the past two days I’ve been making a lot of changes to how tweekly.fm operates and functions for both users and myself. We no longer user MySQL for the majority of the site because concurrently, it just couldn’t handle what we were throwing at it very well. By moving over to a NoSQL alternative (in our case, MongoDB) we can now support a lot more users concurrently and processing over 35,000 updates on a Sunday no longer brings the service down the a slugs pace.
One of the main problems that I had was debugging when users had problems with the service. During the move over to MongoDB I’ve added a whole host of logging integration points to detect when an interaction succeeds or fails. It will now be logged to our logging store and the most recent log response will be stored in your user record and visible on your settings page. This will help identify when something does go wrong, where and why it does.
I can’t stress how important it is that you confirm your Last.fm account too. When you go through this authorisation process I get a token back to access your play data on Last.fm which I can then user to pull your data even when your account is protected. Its also a lot more failsafe than just having a legacy Last.fm username set – although for the past two months its been mandatory. There was also a slight bug that happened with a subset of protected accounts which has now been fixed too. I’m quite confident that your play data will be pulled correctly every single time. You can manually update your accounts play data by visiting the ‘Publish Now’. You don’t need to actually push an update out, just visiting the page will update your account.
The next change affects your personal tweekly.fm URL. Twitter users will now be found at http://www.tweekly.fm/twitter/dordotky and Facebook users will get their own URL at http://www.tweekly.fm/facebook/501640297 which functions in the exact same way that the original tweekly.fm user pages. Facebook users will also get a short bit.ly URL. Comments will return to these pages shortly.
The Facebook issue of cutting off updates has also been fixed with a new version of the processing engine that builds updates for users. You shouldn’t see an errors like that again. Shared play counts, artist data and comments will return within a few days. If you notice any problems or something doesn’t work as expected then please do let me know.