Smart Thumbnail Cropping

Scaling an image down to a thumbnail size is a common practice when hosting it on websites to reduce page load time and save bandwidth and so on.. But very little has been done in optimization of those thumbnails from a human view-ability point of view. Human view-ability, what? Take a large image where background covers the major part of the image and shrink it down to a thumbnail size (say 192 px) and notice that the details of the image is subdued by the background.

To solve this problem of smart cropping, I am using a variation of descriptors and image processing tricks to extract only the most feature rich part of the image and preserve the aspect ratio while cropping the image. Following are the test results of algorithm used:

Sample 1
Original Thumbnail	Feature Extraction	Cropped Thumbnail
Sample 2
Original Thumbnail	Feature Extraction	Cropped Thumbnail
Sample 3
Original Thumbnail	Feature Extraction	Cropped Thumbnail

So what's in the pipeline:

• Open source the image processing code
• Build a http handler (ASP.NET http handler) for dynamic cropping

Thoughts?

Event Viewer - Image Search

"Event Viewer" is an yet another attempt to visualize images similar to my {re}Search Timeline project.
Demo at : http://karthik20522.github.io/EventViewer



The whole point of this proof-of-concept project is to visualize the images from the perspective of the events rather than just displaying an grid of images. For example, a search on GettyImages.com website basically displays a list of images in a tabular fashion which provides no sense of association between individual images being displayed. But having them grouped together as part of an event provides a sense of association and correlation between images.

Displaying images is always a tricky business. A dominant color filter technique could probably provide an alternate way of scanning thru images as user might be more interested in images of particular color than the fine details of an image.

From a technology standpoint for building this project was nothing special.

• ASP.NET MVC 4 - Razor
• Amazon SQS - for event scrapping from GettyImages
• Connect API for event and image detail lookup
• MongoDB
• Dominant Color Extraction

Source code at: https://github.com/karthik20522/EventViewer

Moq, Unity and Servicelocator

Unit testing code that utilizes Unity, ServiceLocator with Moq framework is a thing of a beauty. Well,not really but a pain if it's your first time. Why? Because if you are a heavy google first, code second type of a developer then googling for Moq + ServiceLocator implementation would only give you Moq + Unity but no ServiceLocator implementation or examples. So, after spending nearly an hr, decided to share or at least remind myself how to for the next time:

Bootstrapping your ServiceLocator:
Note: This is the initialization of Unity in your code

Then in your calling method you would do something like this:

Following is how to moq the above Unity and ServiceLocator instance in Unit test cases:

NLog - Custom Target

Nlog similar to log4net is quite a powerful logging library. Though out of the box it provides logging to text files, windows event log, database, Sentinal etc but how about about custom logging? Extending NLog to write to your own custom target is as quite simple as follows:

Create the target class:
Register the target class:
Add to nlog configuration to assign log level:
Example: Note that the example above is specific to my custom logger.

61Degree - Daily News aggregator

61degree.com (news at 61st minute) was built as an alternate approach for browsing news articles. As a loyal google news reader (every and any news) I wanted an interface for quickly viewing and skipping news article without having to scan the page left to right reading news article title's. 61degree was my solution to my problem.

61Degree has a very simple UI interface where news articles are categorized into sections (news/sports/entertainment etc) and displayed as block with distinguishable colors. On a desktop you can just scroll over the blocks and an article title and image associated with it appears but on mobile (ipad/iphone/tablets) an extra area for double tapping is provided where the user can double tap and move from one article to next. Why double tap? On mobile devices, the blocks are quite small for large fingers so double tapping effectively provides the fat finger users to scan articles.

Source code: https://github.com/karthik20522/61Degree

Screenshots:

Tablet:



IPhone:

Dominant Background color extraction in Images

Following is a continuation to my github project at https://github.com/karthik20522/DominantBackgroundColor

The problem of background color extraction hit me when I was trying to dynamically set background color based on dominant color of the image which usually is background color. Part of my solution was to average out the colors and perform a histogram equalization to extract a dominant color.

Since there are 255 x 255 x 255 possible combinations of colors it would more useful to reduce the result to more limited color space. For example if you want to provide filtering or sorting functionality of images with particular color it would be more user friendly to provide a limited/controlled set of color options instead of displaying 100's or 1000's of colors for the user to pick! In this project I used monochrome values of rainbow colors as my closest matching color subset for matching purposes.

Following is some examples of how the code performed:

Result	Result	Result	Result

There are two main functions in the code that can be utilized:

1. To Extract the most dominant color in the image:
2. Convert the dominantHTMLColor to a smaller subset result
Examples of the above Subset/Closest match code:

Actual	Actual	Actual	Actual
Subset	Subset	Subset	Subset

For the most part the results are very close to the original dominant color. The whole idea was to match the resulting dominant color to one of the following rainbow colors:

But what if you want the code to return a list of top 5 colors. Well, you can modify the code as per your requirement by making this simple change to the linq query (which is currently set to pick the top color):

Code Snippets - Nov 2012 Edition

Proxy service for downloading and serving image: (.NET 4.0) Async Http Post (.NET 4.0) Group and Count in LINQ

HttpWebRequest - 500 Internal Server Error - Faulty JSON

So while making a web request from a C# application and if you end up hitting a 500 Internal Server error its usually because of the following or atleast this is what most google would talk about:

As a note, Internal Server Error is on the server side, The server crapped out on something
Most common issues that's not related to your actual code:

• Maybe Invalid authentication or invalid credentials
• Maybe invalid contenttype being passed
• Maybe wrong request method (GET instead of POST or vice-versa)
• Maybe invalid content length being posted
• Or even maybe because of some proxy issue

But I came across something very strange recently while doing a Http POST from a c# WebPI application to a REST based WCF service. So it seems like the Newtonsoft JSON.NET that come's along with WebAPI (NuGet) is serializing the DateTime object to JSON a bit differently. The old JSON.NET library converted DateTime value into Ticks while the new JSON.NET library is coverting it to a string. Following is an example of old and new JSON.NET library:

New JSON.NET Library: Old JSON.NET Library: So it seems that WCF is unable to deserialize the new DateTime format and fails during deserialization of input as the DateTime types are different. One solution to this problem is to serialize the object to JSON using the JavascriptSerializer (System.Web.Script.Serialization namespace). Example code:

ClipTheTrip.com - How I Built This

Clipthetrip.com is a fun, free crowd sourced destination guides. From a data modeling perspective it has a pretty simple object interaction. There is Location object, there is CLIP object, there is photo object and a user object. MongoDB was used as a data store so objects were quite denormalized instead of a referential model.

Some of the technologies and API used for building the site is as follows:

• MongoDB
• .NET MVC3.0
• .NET 4.0 web optimization library for Js and CSS minimizing and combining
• jQuery Plugins
  • Wookmark
  • jQuery Lightbox
  • qTips
  • GMap3 for google maps
  • jQuery Fullscreen for homepage background
  • Imagesloaded
  • Autosuggest for location suggestion
  • Plupload for uploading images
  • Tablesorter
  • CookieControl from the pesky Europe cookie law
  • jQuery Reject
• External API's
  • Weather information using Accuweather
  • Currency using Webservicex service
  • Business search using Bing
• Image processing using SimpleDotImage library
• Generating offline PDF's using wkhtmltopdf
• HtmlAgilityPack for screen scrapping
• Amazon SES for email

As part of this article I am sharing code snippets of how I extracted information from external API's and other smaller code snippets. Since most part of the site is just CRUD operations, I feel it would be a waste to explain the code structure or architecture (considering architecture is quite simple!)

Currency data from Webservicex.net
Example page: http://www.clipthetrip.com/city/london
Business information from Bing
Weather information from Accuweather
Example page: http://www.clipthetrip.com/city/london
Image Resizing using SimpleDotImage
Javascript and CSS bundling using Web.Optimization

{Re}Search timeline - How I Built This

Demo site coming soon..

Another proof of concept and another interesting way of viewing images. Checkout my previous yearinimages.com post. The idea behind Yearinimages was to provide a very eye candy way of representing the best images of 2011. But unfortunately mosaic layout provides no sense of navigation. {Re}search project was my attempt to avoid this pitfall of navigation while still holding the visual way of presenting data with images.



The idea behind this POC is to provide the user a timeline based navigation of events. For example, when you search Japanese Earthquake or Oscar awards which when viewed as a timeline of events, would provide more visual perspective than just displaying a grid of images.

So how did I build this? Well, I used Gettyimages connect API to extract event and image information and Timeline plugin by verite for building the timeline. Addition to above api's and plugins there were quite a few hacks that was required to make this happen. Following is how I did it:

Extracting the event information:
Thou gettyimages connect API provides the search by personality , keyword and search phrases unfortunately it only provides limited refinement options on events. Basically it's not possible to get a list of events similar to the one the gettyimages website provides like following screenshot:



I do believe that the refinement can be customized per customer basis with connect API but I am not sure. Anyways, for extracting the list of events for a particular search I screen scrapped from gettyimages.com website. Following is code snippet of screen scraping using WATIN ui automation framework:



Now that we have list of event ids screen scrapped we can use gettyimages connect to get event details for displaying on UI. Following is code snippet to call connect API using my connect API library (its nothing but a helper library to call the connect API rest service)



Setting up timeline ui:
Now that we have list of events we need to convert it to the format the timeline plugin expects. Timeline plugin takes json as its data source. I used json2csharp.com site to convert the sample json format that timeline github page provided to c# data model that can be used code behind. I used json.net to convert the timeline c# object to json string. Code snippet for converting event list to timeline object.



Note that the hard-coded html in timeline.text property is to provide a autosuggest capability for user. The code above is just for demo purpose's to show that timeline plugin is capable of taking in Html as part of it's json data. Html rendering of the above code:



At my 1st iteration, I hooked up the events to default timeline functionality. But what I would like is to allow users to click on an event and display all images associated to that event. Since this is a custom functionality some hacks to timeline code had to be done. Like the following:
- Associating eventID to the event timeline div so when timeline is clicked the code can determine which eventID it's referencing too
- Adding a Ajax loading gif to every event
- Upon sliding to an event, do Ajax call to server to get images associated to that event
- Display the images when Ajax call is completed
- Show image information when clicked

Framework, plugins, API used:
- jQuery 1.7.1
- jQuery light box
- wookmark jQuery plugin
- Timeline by Verite.Co
- .net framework 4.0/mvc3
- Newtonsoft json.net
- WATIN ui framework
- json2csharp.com for converting the template json to c# object
- gettyimages connect API

Unhandled GZip Encoding

What happens when you have GZip turned on and there is an unhandled exception is thrown, look no further.. a completely cryptic garbage page is returned



So whats the problem, well when an unhandled exception is thrown ASP.NET/MVC removes the GZip header or any other custom header that was set and returns a non-zipped content while your IIS decides to send to browser that the content is GZipped when it's not really. So your browser is receiving a non-zipped content with GZIP content type, thus the garbage content.

To fix this in the global.ascx add the following to the Application_PreSendRequestHeaders function:

Handling Unhandled Exception Asp.NET MVC

Well, having unhandled exception is bad in the first place but displaying user a default IIS broken exception page or the yellow broken code page is even worse. Not a good user experience. But as all code/software there is always bugs and all ways a edge case that breaks unexpectedly. These unhandled expections can be handled in controller level in Asp.NET MVC. Following is a code snipped of handling errors and routing the user to a more user friendly error page.



In the above code, once the error is handled it's redirected to an Error Page. A user friendly error page such as this from clipthetrip.com

Amazon SES (Simple Email Service) - Code

SES (simple email service) provided by Amazon is sleek easy to use email service. Unfortunately as always, anything new has limited documentation or overly complicated documentation. Following is a ready to use simple code snippet for using Amazon SES written in CSharp

Note: Amazon SDK needs to be installed or AWSSDK.dll need to be referenced. Download AWS SDK here So where can I find the “awsAccessKey” (Access key id) and “awsSecretKey” (Secret Key). Well, first login into AWS portal and then head over to Security Credentials page.



In the Security Credentials page look for Access Credentials section and you should have your Access Key Id and your Secret key.

YearInImages.com - How I built this?

Image mosaic source at https://github.com/karthik20522/ImageMosaic



Following post is a very high level technical overview of how the images were constructed and stored and how Getty Images Connect API was used to make this site possible. Site link: yearinimages.com

Connect API story:
All editorial images in Getty images are associated to an event. An event could be anything, Oscar awards, Olympics etc with a bunch of images associated to it. Every event has a numerical id that makes the event unique in our system. The years in focus 2011 images are categorized into 4 main categories (News, Sports, Entertainment, Archive). Each of these categories has many Events and with some simple scripting the EventId’s of each Event was scraped from the site and the image associated with these events was queried using Connect API. Following is a code snippet of querying Getty Connect API:
More technical information of GettyImages Connect API can be found at https://api.gettyimages.com/apis

C# wrapper to call GettyImages connect API: https://github.com/karthik20522/GettyConnect

Note that Connect API only returns back maximum of 75 images per search, so you would have to loop if an event has more than 75 images. The image metadata that Connect API provides has enough information to describe an image but you could also query GetImageDetail endpoint to get complete metadata details of an image. Once the image information was queried and saved (yearinimage uses mongodb as datastore), the image thumbnails were then downloaded for building image mosaic using following code snippet:

Image mosaic story:
The idea behind building image mosaic is to take the original image and split it to a grid say 20x20 pixels (depending on the size of the image) and compute the average RGB within each of the blocks.



Now unfortunately the thumbnail images that were downloaded from GettyImages are not uniform in size (landscape vs portrait). Since we are splitting the image into square Grid, the thumbnails had to be resized to be uniform. Once the thumbnails were resized, their average RGB values were calculated. Average color code snippet:

The next step was to iterate through each image and find the closest match to each block on the image. Since the image grid is basically a matrix of n x m, this can be serialized and stored in file/db. YearinImage used the following class model to store the matrix:


Upon page load, the browser builds a table/grid and places the image in the appropriate location (x,y) to make it look like a single mosaic. With client side scripting (JavaScript), user interaction with each of the mosaic image was possible like displaying image information when a user hovers over the image. This image information (caption) is fetched from the server using ajax calls. The ajax calls by themselves are relatively fast (less than 100ms) but when a user hovers over many images really fast these ajax calls get’s queued up and the user experiences slowness as the browser is waiting for queued ajax calls to complete. This delay can be avoided by aborting the previous ajax call to the server when a user hovers away from the image. Example of aborting an ajax call is like follows:


As a side note, since the browser displays 100’s of images for the mosaic, this potentially could lead to memory build up on the browser. This can be handled by using revokeObjectURL method. Code snippet:

Random Bookmarks

[[updates]]

DevRates - Open source reviews

http://devrates.com/project/search

Simple.Web

https://github.com/markrendle/simple.web

Git Branching Strategy

http://drewfradette.ca/a-simpler-successful-git-branching-model

Interactive GIT cheatsheet

http://ndpsoftware.com/git-cheatsheet.html

Take Home Interview Question (s)

Question 1:

Overview:
We are having web server uptime problems and we observe that the web servers running .NET website crash on a daily basis. More specifically, a web server "crash" results in users receiving a “server unavailable” result or very long page load times exceeding 30 seconds. We observe that during this "crash" period the overall CPU utilization hits 100%, and specifically the .NET worker process uses 100% CPU utilization for an extended period of time (several hours) without recovering. Under normal conditions, CPU utilization on the web server is less than 5%. We notice that the first step is an increase in the CPU utilization to 20-25%. Then finally we see a step to nearly 90-100% CPU utilization. It is at this point that the website starts to perform very slowly. Our website runs .NET pages and receives about 20,000 visitors per day and 80,000-90,000 page views per day. We would like your help in recommending how to prevent this type of system crash.

System Details:
Windows 2003
IIS 6
Dual processor 3.0 Ghz processors
4 GB Ram

Observations:
1)We observe that web server processor utilization increases in a “step manner” over time across all of our webservers. The site performs very slowly when processor utilization reaches 50% and more. Y Axis on this graph and all the others is the % processor utilization.

Data source: IIS performance monitor


2) We observe that memory utilization increases marginally over a 16 hour period. See graph below.


3.We have also graphed Webserver utilization vs. DB utilization to see if there is some relationship between DB processor and Webserver processor utilization. While there is not a direct relationship, it may still be true that deadlocking is causing problems in our webserver.

Code Snippets - June 2012 Edition

I keep coming across scenarios where I need to check if an Object is truly empty and not just NULL. Part of the solution was to do perform conditional checks (if-then-else) on every property but it turned out to be very manual, high maintenance and time consuming work. Following is a quick and dirty way to check if a class is truly empty:


One of the most annoying part of statically typed programming language is the problem of NULL exceptions. Again "if-the-else" was the initial solution and following is a dynamic way of handling NULL values of any type. The following code takes in any property of an object (the object can be NULL by itself) and if the property is NULL it would spit out the default value that was provided.

2011 Year In Images - Image Mosaic

Checkout a Mosaic of 2011 Best images (images from GettyImages) at http://www.yearinimages.com

The images information is fetched using GettyImages Connect API using the following C# library
https://github.com/karthik20522/GettyConnect

I have also opened sourced the the Image Mosaic code used for building the mosaic at
https://github.com/karthik20522/ImageMosaic

GettyImages Connect API

Read my blog post on GettyImages Tech blog about how to leverage GettyImages Connect API:

http://blog.gettyimages.com/2012/05/17/leveraging-our-connect-api-oh-the-things-you-can-do

Programming Images: Using WIC to extract metadata

Read my blog post on GettyImages Tech blog about how to use WIC to extract Image Metadata (XMP/IPTC):

http://blog.gettyimages.com/2012/03/27/programming-images-using-wic-to-extract-metadata