Trivial bash Script to restart services in AWS

Too many aws servers? Been there and I hate it. Following is a simple script that I use to restart services running on EC2 instances. I am using AWS Cli to get the ip address based on tag names and then ssh into the box and to the command. Note that the tag names are the same as service names in my case.

RabbitMQ Upgrade - Bash Script

An easy to read, trivial bash script for upgrading RabbitMQ service.

ReIndexing Elasticsearch in Scala

The following scala script reads from one index and writes to another script using Scan and scroll method. The script also takes in a partial function where the values from one index can be manipulated before saving into another index. This script assumes you have a field called "id" and an field called "submitDate" so it can continually perform scan and scroll once the preliminary index copy is done, so keep the index's in sync. Notes:

• The ESClient is an extension of on wabisabi Library for elasticsearch
• The Actor initially performs a scan-scroll with submit date gte 1900
• Once the initial scan-scroll is done, it pauses for a minute and performs a scan-scroll again with the submitDate of previous endTime (dateTime.now minus 1 minute)
• This way every minute after the previous run it will continually keep the index in sync
• The partial function "processData" provides a way to manipulate the original data, manipulate it and save it to the new index
• Bulk-indexing is used for saving to the new index, hence a the "id" field is required to determine the "id" of the new document

Usage:

Calling SOAP Service in Scala

Scala out of the box has limited capability to call SOAP service neither does libraries such as http-dispatch or spray client. SOAP service is reality is just a xml request/response service and lo and behold, XML is a first class citizen in Scala.

One of the widely used library is ScalaXB which helps in generating case classes given a xsd or wsdl file. Scalaxb is an XML data-binding tool for Scala that supports W3C XML Schema (xsd) and Web Services Description Language (wsdl) as the input file. This is great but it's quite hard to maintain and the code readability goes down the drain as the code is dynamically generated. For example, the following screen shot is what scalaxb generates when either a wsdl or xsd is provided


But what we really need is a trivial way to call a webservice using our existing http clients. Following is one way of doing so.
In this below example, i am calling a service that returns back a list of keywords given a list of keywordid's.
In the above script, all that i am doing is constructing the soap request headers manually and performing a POST operation. There are couple of things to be noted:

• "SOAPAction" header is manually added to let the service know which service operation it is intended for
• Setting the charset to UTF-8 and removing unicode characters "[^\\x20-\\x7e]"
• Removing the unicode characters are necessary as scala fails to parse the response. This mostly seems to happens when calling .NET WCF services

GetKeywordDetailsRequest is a class that has the input parameters and has a function that generates the formatted xml for the soap request

Scala Parser Combinators - SQL Parser Example

Scala Parser Combinators: https://github.com/scala/scala-parser-combinators

Scala Parser Combinators is basically a parsing framework for extracting data when there is a pattern in the given input. This framework provides a more statically typed, functional way of extracting instead of using regex expression which can get hard to read.

In this post, lets build a SQL parser where given a valid sql statement we can identify the "table" name, "column" names and other sql properties. Following are some fundamental functions, operations that Scala combinator provides which would help in parsing:

• " | ": says “succeed if either the left or right operand parse successfully”
• " ~ ": says “succeed if the left operand parses successfully, and then the right parses successfully on the remaining input”
• " ~> ": says “succeed if the left operand parses successfully followed by the right, but do not include the left content in the result”
• " <~ ": is the reverse, “succeed if the left operand is parsed successfully followed by the right, but do not include the right content in the result”
• " ^^ ": says “if the left operand parses successfully, transform the result using the function on the right”
• " ^^^ ": says “if the left operand parses successfully, ignore the result and use the value from the right”
• " rep(fn) ": says "parse the given input using the parser function fn"
• " repsep(ident, char) ": says "parse the given input and split the input using the given 'char'"

Lets start out with a set of SQL statements and it's associated Parser code

select * from users

select name,age from users

select count(name) from users

select * from users order by age desc

select * from users order by name, age desc

select age from users where age>30