Author Archives: Nishant M

About Nishant M

One day I would fly like a superman! :-)

Generate Gzipped Assets in Rails 4.2 +

Doesn’t Rails generate gzipped assets on running rake assets:precompile? It did so, until Rails 4.2. What changed?

Rails uses sprockets gem, for compiling and serving web assets. Before Sprockets 3 was introduced, rake assets:precompile generated .gz versions for all the assets.

This change, seems backward, for several reasons

  1. Gzipping should always be enabled for static assets. It’s super charges serving of static content.
  2. Assets should be gzip’ed beforehand as part of the build process, rather than letting the webserver do it on the fly, saving on CPU cycles.
  3. Also, Nginx has the ability to serve static .gz files directly, by using the gzip_static on; directive.

The reason for dropping this, was its incompatibility with Apache.

Since Sprockets 3 dropped this feature, it means Rails had to loose it too. The Rails team, plans to bring it back as an opt-in. Nevertheless, what does one do in the meantime?

Here are the options to generate Gzipped Assets with Rails

  1. depends on parallel gem to use multiple CPU cores, but can easily be simplified
  2. gzip assets with Capistrano
  3. write an assets:gzip rake task that plugs into assets:precompile (mentioned below)
namespace :assets do
  desc "Create .gz versions of assets"
  task :gzip => :environment do
    zip_types = /\.(?:css|html|js|otf|svg|txt|xml)$/

    public_assets = File.join(
      Rails.root,
      "public",
      Rails.application.config.assets.prefix)

    Dir["#{public_assets}/**/*"].each do |f|
      next unless f =~ zip_types

      mtime = File.mtime(f)
      gz_file = "#{f}.gz"
      next if File.exist?(gz_file) && File.mtime(gz_file) >= mtime

      File.open(gz_file, "wb") do |dest|
        gz = Zlib::GzipWriter.new(dest, Zlib::BEST_COMPRESSION)
        gz.mtime = mtime.to_i
        IO.copy_stream(open(f), gz)
        gz.close
      end

      File.utime(mtime, mtime, gz_file)
    end
  end

  # Hook into existing assets:precompile task
  Rake::Task["assets:precompile"].enhance do
    Rake::Task["assets:gzip"].invoke
  end
end
Minify Gzip

Minify vs gzip

While discussing some common optimisation techniques for the web, a discussion ensued on why are assets both minified and gzip’ed? Is it good for performance? Is there a benefit in minifying and then gzipping? Is it okay to gzip for HTTPS? What the heck, does HTTPS have to do with compression?

The average size of top 1,000 web pages is 1961kb today, compared to 1720kb in Aug 2014. This is a 14% growth, in just a year!  ~ HTTP Archive stats

Amount of data downloaded, by browsers to render a page has been steadily increasing. On the other hand, the performance requirements for end-users aren’t getting any better. Talking about web performance entails talking about 4 things

  1. Network transit time
  2. Response preparation (at server)
  3. Response parse (at client)
  4. Execution time (time taken to apply response on the page)

Given that, we can’t necessarily reduce data that gets generated, the only other way is to reduce its size for network transit (wire weight).

Minification is the practice of removing unnecessary characters (ex: comments, whitespace) from code

.. to reduce its size thereby improving load times. The browser can read and use it just like the original file. But since now, the wire weight is reduced, the page will load relatively faster. It also bundles various files into one, thus reducing the amount of requests to the web server.

Gzipping looks for repeating strings and replaces them with a pointer to the first occurrence

..The browser must decompress any compressed files before using them and this adds to the response parse time. We might have shaved off on some network transit time, but added to the response parse times.

Statistics to prove that gzip and minify is important

Normal Minified Gzipped Minified and gzipped
 JQUERY 2.1.3  241.59 KB 82.34 KB 71.68 KB 28.87 KB
 bootstrap-3.1.1.css  118 KB 98 KB  80 KB 17 KB

*gzip – gzip -6 file.js — the default compression level is 6 on a scale from 1 (fastest, but less compression) to 9 (slowest, but best compression).

A quick calculation gives us this.. On a 100 Mbps link,

Time to download Size
18.4319 ms 241.59 KB.
2.20261 ms 28.87 KB.

Needless to say, that this clearly tells us, to ensure static assets should always be minified and gzipped both.

Reading Code

Writing documentation, is harder than writing code.

If you thought writing code is harder, my experience tells me that writing documentation even more so!

In the calculus of communication, writing coherent paragraphs that your fellow human beings can comprehend and understand is far more difficult than tapping out a few lines of software code that the interpreter or compiler won’t barf on. ~ Coding Horror

If its not documented, it doesn’t exist.

For open source projects, ‘getting started’ documentation is usually good these days. But once you run into some obscure issue or an edge case – you would have to resort to reading the code and documentation would mostly fail.

This is the time, when people are encouraged to document there finding and help out improving documentation. But as it would, code moves at a far more pace than documentation can.

Reading Code

Reading code might be an incidental task that you do while figuring out the issue with the code. But, it should actually be done as an activity of its own too. It’s widely agreed that reading great code definitely improves one’s own programming skill.

My experience while doing these ‘code reading’ exercises have been a revelation in terms of the knowledge that one can gather and the depth of the understanding.

I have realized that reading through the issues and comments listed on rails, has made me more knowledgeable with the code base than the documentation itself could ever do. Also, reading through such a huge code base like rails – is difficult to understand just by reading it. The issues and the comments / discussion on them is the best form of documentation. More like FAQs.

Smitten by this habit of reading through issues and the comments – I have decided to start *documenting* these understandings as blogpost series where I would take up the issue listed on rails/rails and then document my understanding of the behavior and summarize the comments.

Hopefully, that’s going to be helpful to all of us

– Code Reading Groups – Code is not Literature

Where did the name Dynamic Programming come from?

… the 1950’s were not good years for mathematical research. We had a very interesting gentleman in Washington … Secretary of Defense, and  .. had a pathological fear and hatred of the word, research. not using the term lightly; … using it precisely. His face would suffuse,  he would turn red, and .. get violent if people used  the term, research, in his presence. 

What title, what name, could I choose? In the first place I was interested in planning, in decision making, in thinking. But planning, is not a good word for various reasons.

I decided therefore to use the word, ‘programming.’

I wanted to get across the idea that this was dynamic, this was multistage,…I thought, let’s … take a word that has an absolutely precise meaning, namely dynamic, in the classical physical sense. It also has a very interesting property as an adjective, and that is it’s impossible to use the word, dynamic, in a pejorative sense. Try thinking of some combination that will possibly give it a pejorative meaning.

It’s impossible.

It was something not even a Congressman could object to.

Richard Bellman, “Eye of the Hurricane: an autobiography”, 1984

Setting up Nginx with Unicorn for Rails

I have moved away from using Thin to using Unicorn in production for a number of reasons

Some articles to convince you of the same thing.

  1. Unicorn is Unix
  2. Unicorn Power
  3. Most important for me was the ability of Unicorn master and worker processes to respond to Unix signals. This means, zero downtime deployments.

You need more convincing? Let me know and we can talk!

So with that out of the way, let’s see how you can configure nginx to work with unicorn.

  1. Install nginx via the package manager. (For AMI Ec2:sudo yum install nginx, For Ubuntu : sudo apt-get install nginx)
  2. Use this template for nginx unicorn configuration and then fill in only the required details.
  3. Note that the way it differs from thin configuration is by giving the server value as a location to the socket.
worker_processes 1;
user nobody nogroup;

pid /tmp/nginx.pid;
error_log /tmp/nginx.error.log;

events {
  worker_connections 1024; # increase if you have lots of clients
  # Set this to on if you have more than 1 working processes
  # This will allow only one child to watch the pollset and accept
  # a connection to a socket
  accept_mutex off; # "on" if nginx worker_processes > 1
}

http {
  include mime.types;
  default_type application/octet-stream;
  access_log /tmp/nginx.access.log combined;

  # This tells Nginx to ignore the contents of a file it is sending
  # and uses the kernel sendfile instead
  sendfile on;

  # Set this to on if you have sendfile on
  # It will prepend the HTTP response headers before
  # calling sendfile()
  tcp_nopush on;

  # This disables the "Nagle buffering algorithm" (Nginx Docs)
  # Good for websites that send a lot of small requests that
  # don't need a response
  tcp_nodelay off;

  gzip on;
  gzip_http_version 1.0;
  gzip_proxied any;
  gzip_min_length 500;
  gzip_disable "MSIE [1-6]\.";
  gzip_types text/plain text/html text/xml text/css
             text/comma-separated-values
             text/javascript application/x-javascript
             application/atom+xml;

  upstream unicorn_server {
   # This is the socket we configured in unicorn.rb
   server unix:/var/rails/testapp/tmp/sockets/unicorn.sock »
   fail_timeout=0;
  }

  server {
    listen 80;
    client_max_body_size 4G;
    server_name _;
    keepalive_timeout 5;

    # Location of our static files
    root /home/ec2-user/testapp/public;

    location / {
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header Host $http_host;
      proxy_redirect off;

      # If you don't find the filename in the static files
      # Then request it from the unicorn server
      if (!-f $request_filename) {
        proxy_pass http://unicorn_server;
        break;
      }
    }    error_page 500 502 503 504 /500.html;
    location = /500.html {
      root /home/ec2-user/testapp/public;
    }
  }
}

If you have unicorn correctly configured then start unicorn, restart nginx and you should be set.

Image upload to the PageDown (markdown) editor via AJAX in Rails

I have been using the updated version of the WMD markdown editor called as Pagedown for a blog engine, and wanted to have an Image Upload functionality.

If you take a look at Pagedown wiki, you will realize it has a very limited set of API’s – but has the one we need , a hook for ‘insert Image button’ called as insertImageDialog.

I had the following requirements for achieving Image Upload

  1. Markdown Editor (Pagedown)
  2. Bootstrap
  3. Rails + AJAX Uploads of Images.
  4. Preview of the image + ability to select size, before inserting into post.

Markdown Hook for Image Button.

The example from the Pagedown wiki, shows how the insertImageDialog.hook can be used.

editor.hooks.set("insertImageDialog", function (callback) {
      alert("Please click okay to start scanning your brain...");
      setTimeout(function () {
          var prompt = "We have detected that you like cats. Do you want to insert an image of a cat?";
          if (confirm(prompt))
              callback("http://icanhascheezburger.files.wordpress.com/2007/06/schrodingers-lolcat1.jpg")
          else
              callback(null);
      }, 2000);
     return true; // tell the editor that we'll take care of getting the image url
    });

This tells us to register a plugin on the insertImageDialog hook that returns true (which tells the editor not to create its own dialog). so the callback method is the one which returns the actual URL of the uploaded images.

I recommend you try the above code, so that you know how the hooks are working within your editor.

Twitter Boostrap Markdown

This project is a pagedown (markdown) editor, styled for bootstrap. Take a look at the demo. I have made use of these pagedown code.

I would advice you to take a look at the source of this demo, to understand how all components tie-up.

RAILS + AJAX Uploads of Images + Image Preview and Re-sizing.

For doing AJAX uploads using a jQuery plugin and carrier-wave turned out to be easier because I went ahead with ‘jquery-fileupload-rails’ . Though there can be some challenges if you are using nested_forms. (This Rails-casts should help you figuring out the integration http://railscasts.com/episodes/381-jquery-file-upload).

With all the components laid above, I can walk you through the approach I took…

1. First Create a Modal for the dialog to appear on click of the insert Image..

<!-- Modal -->
<div id="fileModal" class="modal hide fade" tabindex="-1" role="dialog" aria-labelledby="fileModalLabel" aria-hidden="true">
 <div class="modal-header">
 <button type="button" class="close" data-dismiss="modal" aria-hidden="true">×</button>
 <h3 id="fileModalLabel">Upload Images</h3>
 </div>
 <div class="modal-form">
 <div class="tabbable tabs-left"> <!-- Only required for left/right tabs -->
 <ul class="nav nav-tabs">
 <li class="active"><a href="#tab1" data-toggle="tab">Upload</a></li>
 <li><a href="#tab2" data-toggle="tab">Enter URL</a></li>
 </ul>
 <div class="tab-content">
 <div class="tab-pane active" id="tab1">
 <%= f.file_field :photo %>
 <div class="preview" id="preview" >
 </div>
 </div>
 <div class="tab-pane" id="tab2">
 <p><input type="text" class="img-url" placeholder="Ex: http://yoursite.com/image.jpg" id="img-url" /></p>
 </div>
 </div>
 </div>
 </div> <!-- End of Modal-Body -->
 <div class="modal-footer">
 <button class="btn" data-dismiss="modal" aria-hidden="true">Close</button>
 <button class="btn" id="insert_image_post">Insert Image</button>
 </div>
</div>
<% end %>

The above modal code would render like this. Image Upload Markdown Bootstrap Rails

2. Bind the Pagedown hook to show the above modal.

This is the easy part, as you saw above how the insertImageDialog hook behaves, we would link these together as.

<script type="text/javascript">
 (function () {
 var converter = new Markdown.Converter();
 var help = function () { window.open('http://stackoverflow.com/editing-help'); }
 var editor = new Markdown.Editor(converter);
 editor.hooks.set("insertImageDialog", function (callback) {
 setTimeout(function () 
{
       $('#fileModal').modal();
        ...
        ...
       $("#insert_image_post").click(function(e) {
          e.preventDefault();
          if($(".upload-photo").length>0)
         {
            var images;
            $(".upload-photo").each(function(){
            images = "" + ($(this).filter("[src]").attr("src"));
          })
         callback(images);
       $('#upload_photo').fileupload();
        }
       else {
          var image=$("#img-url").val();
          callback(image);
        }
      $("#fileModal").modal('hide');
    });
 }, 0);
 return true; // tell the editor that we'll take care of getting the image url
 });
editor.run();
 })();
 </script>

To actually see how Ajax Jquery Uploads and carrierwave would interact you can look at the Railscasts http://railscasts.com/episodes/381-jquery-file-upload .

For the entire working code, look at the following repository.

Highly Scalable Blog

NoSQL databases are often compared by various non-functional criteria, such as scalability, performance, and consistency. This aspect of NoSQL is well-studied both in practice and theory because specific non-functional properties are often the main justification for NoSQL usage and fundamental results on distributed systems like the CAP theorem apply well to NoSQL systems.  At the same time, NoSQL data modeling is not so well studied and lacks the systematic theory found in relational databases. In this article I provide a short comparison of NoSQL system families from the data modeling point of view and digest several common modeling techniques.

I would like to thank Daniel Kirkdorffer who reviewed the article and cleaned up the grammar.

To  explore data modeling techniques, we have to start with a more or less systematic view of NoSQL data models that preferably reveals trends and interconnections. The following figure depicts imaginary “evolution” of the major NoSQL…

View original post 3,581 more words