Rails: read_multi and dalli

Be careful when using read_multi with dalli: it may return nil-valued key instead of the correct key.

The issue is tracked here and thanks to this I dropped the read_multi usage in moebooru and used the much simpler (and most likely slower) single fetch (per entry) instead. There’s alternative way to use it – do a read_multi and refetch whatever missing/nil-keyed but apparently I’m too lazy to do it.

Moebooru 3.2

Finally reached the goal: Rails 3.2.6. The isn’t much change between 3.1 and 3.2 which is why I skipped 3.1. As there isn’t much front-end change, it means there isn’t any user-visible changes. Or at least there shouldn’t be.

Plans:

  • Move to jQuery
  • Use paper_trail for versioning (or something else)
  • Add SVG support (and maybe drop SWF support)
  • Actual news ticker
  • And more!

3.2.0 has been branched and currently running on yande.re (and konachan). The next version, 3.2.1 is on default branch and being tested on moe.myconan.net. As (finally) there are database changes, it will not be tested directly on yande.re until it is stable enough.

Changes include better support for JRuby, use mini_magick instead of custom plug-in, less monkey-patches (but more bugs), and various clean-ups.

Moebooru/Rails 3, and more

After working on it for several weeks, finally it got to the point where it’s relatively usable. No more Bundler-on-Rails2 evilry and the fact that 3.0 branch is still supported.

The best part is the one above. Yes, Moebooru now runs on Rubinius/Puma. The only foreseeable problem is it uses Process.pid on file uploads which, when several people uploading (or working with) files at same time, the temporary filename will collide.

Though due to me aiming more for workable implementation instead of correct implementation, there are quite a lot legacy stuff still in there:

  • repeated_auto_complete doesn’t work in Rails 3 without vendoring or updating the gem itself  (and I did both since Bundler’s git functionality seems broken in Rubinius).
  • prototype_legacy_helper must be used since I haven’t had time to upgrade the *_remote functions (and still thinking how to do it).
  • verification gem since I haven’t upgraded the route and its functionality, which previously built-in in Rails 2.3 has now removed.
  • Unsightly lib/core_ext. Just look at evilry I have added to make it behave like 2.3-stable.

On the brighter side, now I can start cleaning up the code. Or upgrade all the way to Rails 3.2 ( ¬‿¬)

Optimize all the queries!

While digging more into the code called “Moebooru” which was forked from “Danbooru”, I noticed this:

def self.included(m)
  m.extend(ClassMethods)
  m.after_create :increment_count
  m.after_destroy :decrement_count
end

def increment_count
  connection.execute("update table_data set row_count = row_count + 1 where name = 'users'")
end

def decrement_count
  connection.execute("update table_data set row_count = row_count - 1 where name = 'users'")
end

Counting takes ages, right. Except it is not. I’ve done this, yes, but on a table with 10+ millions of data (this one has ~400k in mainline danbooru), with multiple data inserted (this one got, uh, one every other week?) and queried every second (see below), and with the required count method not a simple select count(1) on some_table (which is what the example above used for).

The best part? It’s only used once, when user registers:

def set_role
  if User.fast_count == 0
    self.level = CONFIG["user_levels"]["Admin"]
  elsif CONFIG["enable_account_email_activation"]
    self.level = CONFIG["user_levels"]["Unactivated"]
  else
    self.level = CONFIG["starting_level"]
  end

  self.last_logged_in_at = Time.now
end

Moebooru Update

I haven’t been able to work on Moebooru for most of this month but I got to work on it this weekend. At ~30 changes, it went from -alpha to -beta.2 after receiving good amount of tests.

Summary of changes (more or less):

  • Most instances of RAILS_ENV and RAILS_ROOT have been replaced with newer Rails.env/Rails.root
  • Better Javascript handling in Development mode by moving cached Javascript from application.js to cached/all.js
  • Removed AssetCache modification library
  • Replaced HTML5Sanitizer’s html5sanitize with Rails’ built-in sanitize (because the former doesn’t work in 1.9)
  • Fixed Note Editor’s “History” button
  • Fixed Comment page’s pagination
  • Fixed Tag Relation search function
  • Updated “hack” for UTF-8 handling in Rails 2.3 with Ruby 1.9
  • Merged updates from moe branch (the one currently used in yande.re)
  • Reverted change which put class definition in <html> tag as it currently required by Javascript

Moebooru rebased

Since I forgot to branch the original source, the branching looked awesomely crappy. Therefore I decided to rebase entire thing to ease up keeping track with Moebooru “mainline”. All my commits are now in branch “default”. If you didn’t do any change, backout up to revision 9174b6b5b02d and then pull again. And then, don’t ever touch moe branch again anymore.

moebooru again

Last week I posted about my random project which involves modernizing moebooru without doing complete rewrite (see this for yet another complete rewrite attempt).

Let’s revisit the plan:

  • Upgrade to Ruby 1.9: done, need testing.
  • Update all plugins: mostly done, can use some trimming.
  • Update anything deprecated: nope
  • Migrate to Bundler: done, not sure how to test.
  • Use RMagick instead of custom ruby-gd plugin: nope
  • Use RMagick instead of calling jhead binary: nope
  • And more!: I hope you didn’t expect me to do more while there are incomplete items above.

Sure looks good. Need more testing though. There’s also one part which I totally had no idea why should be changed when upgrading to 1.9. Just grep for FIXME to see which it is and hopefully fix it up for me (or explain what it does).

As usual, having completed the work for today, live demo is up and open for everyone to break (…if there’s anyone, that is).

[ Live Demo | Repository ]

An Attempt to Update moebooru Engine

If you didn’t know, the current moebooru running on oreno.imouto is using ancient version of many things. It also uses a custom lighty module (mod_zipfile) which doesn’t seem to be available anywhere.

I’ve updated it with latest Rails 2.x and made it compatible with nginx. Mostly. You can see it running here.

The plans:

  • Upgrade to Ruby 1.9.
  • Update all plugins.
  • Update anything deprecated.
  • Migrate to Bundler.
  • Use RMagick instead of custom ruby-gd plugin.
  • Use RMagick instead of calling jhead binary.
  • And more!

We’ll see if I can actually finish this one. Grab the source here. Yeah, I’m using Mercurial for a Rails project.