WYSIWYG

http://kufli.blogspot.com
http://github.com/karthik20522

Thursday, November 29, 2012

Code Snippets - Nov 2012 Edition

Proxy service for downloading and serving image: (.NET 4.0)
?
1
2
3
4
5
6
7
8
9
[HttpGet]
public async Task<FileResult> GetImage(string url)
{
 var _client = new HttpClient();           
  
 var imageData = await _client.GetStreamAsync(url);
 
 return new FileStreamResult(imageData, "image/jpeg");
}
Async Http Post (.NET 4.0)
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
public Task<string> PostData(string uri, string postData)
{
 byte[] postBytes = Encoding.ASCII.GetBytes(postData);
 
 HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
 request.ContentType = "application/json";
 request.Method = WebRequestMethods.Http.Post;
 request.ContentLength = postBytes.Length;
 request.Timeout = 20000;
 request.KeepAlive = false;
  
 var requestStream = request.GetRequestStream();
 requestStream.Write(postBytes, 0, postBytes.Length);
 requestStream.Close();
 
 Task<WebResponse> task = Task.Factory.FromAsync(
  request.BeginGetResponse,
  asyncResult => request.EndGetResponse(asyncResult),
  (object)null);
 
 return task.ContinueWith(t => ReadStreamFromResponse(t.Result));
}
 
private static string ReadStreamFromResponse(WebResponse response)
{
 using (var responseStream = response.GetResponseStream())
 {
  using (var sr = new StreamReader(responseStream))
  {
   return sr.ReadToEnd();
  }
 }
}
Group and Count in LINQ
?
1
2
3
4
5
6
7
8
9
10
11
public class Asset
{
 public string KnownColorName { get; set; }       
}
 
var assets = GetListOfAssetsFromDatasource();
 
var colorGroup = assets.
                GroupBy(n => n.KnownColorName)
                .Select(n => new { Count = n.Count(), Color = n.Key })
                .OrderBy(n => n.Color);

Labels: ,

posted by ..:: karthik ::.. @ 5:43 PM 

HttpWebRequest - 500 Internal Server Error - Faulty JSON



So while making a web request from a C# application and if you end up hitting a 500 Internal Server error its usually because of the following or atleast this is what most google would talk about:

As a note, Internal Server Error is on the server side, The server crapped out on something
Most common issues that's not related to your actual code:
  • Maybe Invalid authentication or invalid credentials
  • Maybe invalid contenttype being passed
  • Maybe wrong request method (GET instead of POST or vice-versa)
  • Maybe invalid content length being posted
  • Or even maybe because of some proxy issue
But I came across something very strange recently while doing a Http POST from a c# WebPI application to a REST based WCF service. So it seems like the Newtonsoft JSON.NET that come's along with WebAPI (NuGet) is serializing the DateTime object to JSON a bit differently. The old JSON.NET library converted DateTime value into Ticks while the new JSON.NET library is coverting it to a string. Following is an example of old and new JSON.NET library:

New JSON.NET Library: Old JSON.NET Library: So it seems that WCF is unable to deserialize the new DateTime format and fails during deserialization of input as the DateTime types are different. One solution to this problem is to serialize the object to JSON using the JavascriptSerializer (System.Web.Script.Serialization namespace). Example code:
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
{
 var requestBody = new {
   RequestHeader = requestHeader,
   SearchForImages2RequestBody = searchBody
 };
 
 //Using JSON.NET
 //var postData = JsonConvert.SerializeObject(requestBody);
 
 //Using Web.Script
 var postData = new JavaScriptSerializer().Serialize(requestBody);
 
 return PostData(uri, postData);
}

Labels: , , , ,

posted by ..:: karthik ::.. @ 3:03 PM 

Saturday, November 10, 2012

ClipTheTrip.com - How I Built This

Clipthetrip.com is a fun, free crowd sourced destination guides. From a data modeling perspective it has a pretty simple object interaction. There is Location object, there is CLIP object, there is photo object and a user object. MongoDB was used as a data store so objects were quite denormalized instead of a referential model.

Some of the technologies and API used for building the site is as follows:
As part of this article I am sharing code snippets of how I extracted information from external API's and other smaller code snippets. Since most part of the site is just CRUD operations, I feel it would be a waste to explain the code structure or architecture (considering architecture is quite simple!)

Currency data from Webservicex.net
Example page: http://www.clipthetrip.com/city/london
?
1
2
3
4
5
6
7
8
  var xElement = System.Xml.Linq.XElement.Load("http://www.webservicex.net/CurrencyConvertor.asmx/ConversionRate?FromCurrency=USD&ToCurrency=" + code);
//code being 3 letter code like USD, INR etc.
  var cur = decimal.Parse(xElement.Value).ToString("#.##");  
 
  jsonResult.Data = new { c = cur, code = code };
   
  return jsonResult;

Business information from Bing
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
//example: http://www.bing.com/local/default.aspx?what=Saravana&where=London
var html = new WebClient().DownloadString("http://www.bing.com/local/default.aspx?what=" + name + "&where=" + location);
 
var htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.OptionFixNestedTags = true;
htmlDoc.LoadHtml(html);
 
var nodes = htmlDoc.DocumentNode.SelectNodes("//div[@class='ecEnclosure']");
if (nodes != null)
{
 foreach (HtmlNode node in nodes)
 {
  var bName = node.Descendants("a")
    .Where(s => s.GetAttributeValue("class", "").Contains("ecHeaderLink"))
    .Select(s => s.InnerText).FirstOrDefault();
 
  var address = node.Descendants("span")
    .Where(s => s.GetAttributeValue("id", "").Contains("AddressLine"))
    .Select(s => s.InnerText).FirstOrDefault();
 
  var phone = node.Descendants("span")
    .Where(s => s.GetAttributeValue("class", "").Contains("ecAddress"))
    .Select(s => s.InnerText).FirstOrDefault();
 
  var website = string.Empty;
 
  var links = node.Descendants("a")
    .Where(s => s.GetAttributeValue("id", "").Contains("ohlWeb"))
    .Select(s => s).FirstOrDefault();
 
  if (links != null)
   website = links.GetAttributeValue("href", "");
 
  if (!string.IsNullOrEmpty(bName))
  {
   var bCard = new Business()
   {
    Address = address ?? string.Empty,
    Name = bName ?? string.Empty,
    PhoneNumber = phone ?? string.Empty,
    Website = website ?? string.Empty
   };   
                        //do something with bCard
  }
 }
}

Weather information from Accuweather
Example page: http://www.clipthetrip.com/city/london
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
   public class Weather
{
 public string city;
 public int curr_temp;
 public string curr_text;
 public int curr_icon;
 
 public List<Forecast> forecast = new List<Forecast>();
}
public class Forecast
{
 public string day_date;
 public string day_text;
 public int day_icon;
 public int day_htemp;
 public int day_ltemp;
}
 
//code
Weather _weather;
 
if (_weather == null)
{
 _weather = new Weather();
 
 string _location = code;
 string _metric = "0";
 
 string _url = string.Format("http://wwwa.accuweather.com/adcbin/forecastfox/weather_data.asp?location={0}&metric={1}", _location, _metric);
 
 string _xml = Utils.DownloadWebPage(_url);
 
 var _xmlDocument = new XmlDocument();
 _xmlDocument.LoadXml(_xml);
 
 var _mgr = new XmlNamespaceManager(_xmlDocument.NameTable);
 _mgr.AddNamespace("pf", _xmlDocument.DocumentElement.NamespaceURI);
 
 _weather.city =_xmlDocument.SelectSingleNode("/pf:adc_database/pf:local/pf:city", _mgr).InnerText;
 _weather.curr_temp = Convert.ToInt32(_xmlDocument.SelectSingleNode("/pf:adc_database/pf:currentconditions/pf:temperature", _mgr).InnerText);
 _weather.curr_text = _xmlDocument.SelectSingleNode("/pf:adc_database/pf:currentconditions/pf:weathertext", _mgr).InnerText;
 _weather.curr_icon = Convert.ToInt32(_xmlDocument.SelectSingleNode("/pf:adc_database/pf:currentconditions/pf:weathericon", _mgr).InnerText);
 
 var _xmlNodeList = _xmlDocument.SelectNodes("/pf:adc_database/pf:forecast/pf:day", _mgr);
 int _day = _xmlNodeList.Count;
 int i = 0;
 foreach (var _dayItem in _xmlNodeList)
 {
  var _forecast = new Forecast();
 
  _forecast.day_date = _dayItem["obsdate"].InnerXml;
  _forecast.day_text = _dayItem.SelectSingleNode("pf:daytime", _mgr)["txtshort"].InnerXml;
  _forecast.day_icon = Convert.ToInt32(_dayItem.SelectSingleNode("pf:daytime", _mgr)["weathericon"].InnerXml);
  _forecast.day_htemp = Convert.ToInt32(_dayItem.SelectSingleNode("pf:daytime", _mgr)["hightemperature"].InnerXml);
  _forecast.day_ltemp = Convert.ToInt32(_dayItem.SelectSingleNode("pf:daytime", _mgr)["lowtemperature"].InnerXml);
 
  _weather.forecast.Add(_forecast);
 
  i++;
  // 5 day forecast
  if (i == 5) break;
 }
}

Image Resizing using SimpleDotImage
?
1
2
3
4
5
6
7
8
9
10
using (var _imgProcessing = new ImageProcessing())
{
 using (var _waterMarkedImage = _imgProcessing.Process(
         imagePath: file,
         resize: 164
       ))
 {
  ImageHelper.SaveStream(_waterMarkedImage, thumbFile);
 }
}

Javascript and CSS bundling using Web.Optimization
?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
using System.Web.Optimization;
 
namespace ClipTheTripWeb
{
 public class BundleConfig
 {
  public static void RegisterBundles(BundleCollection bundles)
  {
   var homeBundleJs = new Bundle("~/content/js/home", new JsMinify());
   homeBundleJs.Include (
      "~/Content/Js/jquery.fullscreenr.js",      
      "~/Content/Js/jquery.autoSuggest.js",
      "~/Content/Js/home.js"
   );
 
   var homeBundleCss = new Bundle("~/content/css/home", new CssMinify());
   homeBundleCss.Include (
    "~/Content/Css/Site.css",
    "~/Content/Css/autoSuggest.css",
    "~/Content/Css/tiles.css"
   );
    
   .
   .
   .
   bundles.Add(homeBundleCss);
   bundles.Add(homeBundleJs);
  }
 }
}
 
 
//Global.ascx
protected void Application_Start()
{
 AreaRegistration.RegisterAllAreas();
 BundleConfig.RegisterBundles(BundleTable.Bundles);
 .
 .
}

Labels: , , ,

posted by ..:: karthik ::.. @ 1:44 PM 

Sunday, November 4, 2012

{Re}Search timeline - How I Built This

Demo site coming soon..

Another proof of concept and another interesting way of viewing images. Checkout my previous yearinimages.com post. The idea behind Yearinimages was to provide a very eye candy way of representing the best images of 2011. But unfortunately mosaic layout provides no sense of navigation. {Re}search project was my attempt to avoid this pitfall of navigation while still holding the visual way of presenting data with images.



The idea behind this POC is to provide the user a timeline based navigation of events. For example, when you search Japanese Earthquake or Oscar awards which when viewed as a timeline of events, would provide more visual perspective than just displaying a grid of images.

So how did I build this? Well, I used Gettyimages connect API to extract event and image information and Timeline plugin by verite for building the timeline. Addition to above api's and plugins there were quite a few hacks that was required to make this happen. Following is how I did it:

Extracting the event information:
Thou gettyimages connect API provides the search by personality , keyword and search phrases unfortunately it only provides limited refinement options on events. Basically it's not possible to get a list of events similar to the one the gettyimages website provides like following screenshot:



I do believe that the refinement can be customized per customer basis with connect API but I am not sure. Anyways, for extracting the list of events for a particular search I screen scrapped from gettyimages.com website. Following is code snippet of screen scraping using WATIN ui automation framework:

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
using (var _browser = new IE("http://www.gettyimages.com"))
{
 topCelebs.ForEach(s =>
 {
  var keywordEvent = new Keyword();
  keywordEvent.Term = s.Trim().ToLower();
 
  _browser.TextField(Find.ById("txtPhrase")).Clear();
  _browser.TextField(Find.ById("txtPhrase")).TypeText(keywordEvent.Term);
  var editorialChkfield = _browser.CheckBox(Find.ById("cbxEditorial"));
 
  if (!editorialChkfield.Checked)
   editorialChkfield.Click();
 
  _browser.Button(Find.ById("btnSearch")).Click();
 
  if (_browser.Link(Find.ById("ctl00_ctl00_ctl11_gcc_mc_re_flEvent_lnkSeeMore")).Exists)
  {
   _browser.Link(Find.ById("ctl00_ctl00_ctl11_gcc_mc_re_flEvent_lnkSeeMore")).Click();
   _browser.Div(Find.ById("ctl00_ctl00_ctl11_gcc_mc_re_flEvent_refinementContent")).WaitUntilExists();
 
   var filterContentDiv = _browser.Div(Find.ById("ctl00_ctl00_ctl11_gcc_mc_re_flEvent_refinementContent"));
   var eventIds = new List<int>();
 
   foreach (var link in filterContentDiv.Links.Filter(Find.ByClass("refineItem")))
   {
    var splitList = link.OuterHtml.Split('\'');
 
    if (splitList.Length > 2)
     eventIds.Add(int.Parse(splitList[1]));
   }
 
   //do something with captured eventId's like save to DB
  }
 });
}


Now that we have list of event ids screen scrapped we can use gettyimages connect to get event details for displaying on UI. Following is code snippet to call connect API using my connect API library (its nothing but a helper library to call the connect API rest service)

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//get 50 events at a time instead of making individual calls to Connect API
for (int i = 0; i < eventIdList.Count; i = i + 50)
{
 var geteventRequest = new connectSv.GetEventsRequestBody();
 geteventRequest.EventIds = eventIdList.Skip(i).Take(50).ToList();
 
 var eventResponse = operations.GetEventInfo(geteventRequest, session.TokenResponse.Token);
 
 if (eventResponse != null && eventResponse.ResponseHeader.Status.Equals("success"))
 {
  eventResponse.GetEventsResult.Events.ForEach(ev =>
  {
   // save event information to DB
  });
 }
}


Setting up timeline ui:
Now that we have list of events we need to convert it to the format the timeline plugin expects. Timeline plugin takes json as its data source. I used json2csharp.com site to convert the sample json format that timeline github page provided to c# data model that can be used code behind. I used json.net to convert the timeline c# object to json string. Code snippet for converting event list to timeline object.

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
var timeLine = new Timeline();
timeLine.headline = "{Re}Search - Images from GettyImages";
timeLine.type = "default";
timeLine.text = "<form class='form-wrapper cf' style='top:35%;position:absolute;'>" +
    "<input type='text' id='autoCompleteInput' placeholder='Search here...'>" +
    "<button type='submit'>Search</button>" +
    "</form>";
 
//provide a default start date
timeLine.startDate = "2010,12,29";
timeLine.date = new List<Date>();
 
var eInfos = GetEvents();
 
eInfos.ForEach(e =>
{
 if (e.StartDate > DateTime.MinValue && e.EventId > 0)
 {
  timeLine.date.Add(new Date()
  {
   startDate = e.StartDate.ToString("yyyy,MM,dd"),
   headline = e.EventName,
   eventId = e.EventId,
   text = string.IsNullOrEmpty(e.Description) ? e.EventName : e.Description,
   asset = new Asset()
   {
    //media = e.RepresentativeImageId,
    thumbnail = e.RepresentativeImageId,
    caption = e.EventName,
    credit = "GettyImages"                          
   }
  });
 }
});


Note that the hard-coded html in timeline.text property is to provide a autosuggest capability for user. The code above is just for demo purpose's to show that timeline plugin is capable of taking in Html as part of it's json data. Html rendering of the above code:



At my 1st iteration, I hooked up the events to default timeline functionality. But what I would like is to allow users to click on an event and display all images associated to that event. Since this is a custom functionality some hacks to timeline code had to be done. Like the following:
- Associating eventID to the event timeline div so when timeline is clicked the code can determine which eventID it's referencing too
- Adding a Ajax loading gif to every event
- Upon sliding to an event, do Ajax call to server to get images associated to that event
- Display the images when Ajax call is completed
- Show image information when clicked

Framework, plugins, API used:
- jQuery 1.7.1
- jQuery light box
- wookmark jQuery plugin
- Timeline by Verite.Co
- .net framework 4.0/mvc3
- Newtonsoft json.net
- WATIN ui framework
- json2csharp.com for converting the template json to c# object
- gettyimages connect API

Labels: , , , ,

posted by ..:: karthik ::.. @ 2:28 PM 

New Jersey Path Alerts - Uh-Oh

My first reaction was as follows.. WWWWWWTTTTTTFFFFFFFF..... I couldn't believe what I was seeing. I knew it was ASP.NET website but site running on .NET 2.0 and written in VB.NET :-/ WTF.. On top of that a dead lock exception, that's plain insult to injury! And no exception handling for unhandled exceptions and no custom error page/message instead of showing in all glory the yellow screen of death with .net stack trace!! Niceeee.. Well atleast no inner details is in the stack trace like db name and so forth.. phew.

Labels: ,

posted by ..:: karthik ::.. @ 12:38 PM 

Thursday, November 1, 2012

Beers in my belly - XI


Oskar blues ten fidy

Sofie Belgium Ale

Labels:

posted by ..:: karthik ::.. @ 2:30 PM