Using Amazon SES to send email

If you have a legitimate need to send lot of emails from your software or application and you are already Amazon AWS user, you should definitely check Amazon Simple Email Serivce (SES). The total setup took me less than 30 minutes before I was able to send email using their transport.
There is good documentation on Amazon SES that you want to read to decide if it’s the right email service to use for your scenario. I have yet to evaluate the turn around time for AWS SES in sending email. The reviews that I came across have been mixed.
Here is my logic for trying this service

• GoDaddy, where my domain is registered, provides email service but it’s seriously limiting. They only allow 250 emails per day and paying more still does not let you scale easily. GMail also has limitations. Being able to talk to customers in a scalable way is very important to Mofinto. SES offers that.
• Since Mofinto already uses AWS, it’s no brainer for me to try to use this.

Using SendMail or someother email server may come free but it is involved and would take up a lot of time. Even then configuring for secure email with protocols like DKIM, SSL or TSL etc. is going to involve more work.

• SES provides TSL, DKIM authentication out of the box. More on DKIM later.
• SES also lets you test out sending the email, deal with bounced email, complaints etc.
• SES comes with monitoring and other conveniences and is a “Pay As You Go” service.

Here are the steps to enable it:

• If you have AWS account, login. If you do not, sign up for one.
• In the AWS management console, select SES.
• Click on SMTP settings and then click on mySMTP credentials. Details are given in AWS documentation. You can use console or API to do this. It’s very important that you save the credentials locally on your machine somewhere in a safe place. You need this later and this is the only time you get to download and save it. You can also copy and past it into a text document. It’s a good idea to also save the IAM User name, that AWS console generates, somewhere safe.
• In the SES management console, click on “Email Addresses” under “Verify Senders”
• Click on Verify a New EMail Address
• Type in the email you want to use for sending.
• AWS SES will send email to this account with a link that you have to click to verify.
• Repeat the above two steps for each of the email accounts you want to send mail to while testing the service.
• Note that each of the emails is added as a line to the “Email Addresse” under “Verify Senders”. It also indicates status of the verification process for each of the emails. Once you click on the link in the respective emails, the status for that particular email will change to verified.
• When all the emails are verified, you are ready. SES will place you in a sandbox environment. The only thing you have left to do is to test it, make sure everything is working and when you are all satisfied, request for production access.
• Since you are in sandbox, you can’t send to any email addresses outside those you verified.

The very basic test involves being able to send mail using SES. This is done by sending test mail through management console itself. To do this, go to “Email Addresses” under “Verify Senders” and select the email you want to use as sender. This will enable “Send a Test EMail” button next to “Verify a New EMail Address” button. Click on this, fill the form and vefify that you received the email.
There are multiple ways to send email using SES. You can use the console, existing software packages, integrate with your email server, or your application might send mail programatically either using SMTP or using the AWS SES API. Here is a document that describes how to do it using Php and mysql.
In case you are using PhpMailer (an open source php helper that lets you send SMTP mail), here is what you need to do make it work.
$mail = new PHPMailer();
$mail->IsSMTP(); // send via SMTP
$mail->SMTPAuth = true; // turn on SMTP authentication
$mail->Port = 465;// SSL port
$mail->Host = “ssl://email-smtp.us-east-1.amazonaws.com”; // This may be different depending on your region.
/*You can use tsl: or ssl:. I spent several hours struggling with why the mail was not going through. Until I realized that Apache is refusing TSL protocol, because I haven’t enabled it. I was able to find it when I turned on $mail->SMTPDebug. However, the debug outputs were ‘php echo’ statements in PhpMailer. To enable debug output to showup, I buffered the output and redirected it to a string that could be logged.*/
$mail->Username = “user name part of credentials you saved before”; // SMTP username
$mail->Password = “password part of credentials”; // SMTP password
$mail->Sender = “sender@email.com”; // verfied sender’s email
$mail->From = “sender@email.com;
$mail->FromName = “Name of the sender”;
$mail->AddAddress(“to@email.com”,”receiver’s name”); // verified receiver’s email and name.
$mail->AddReplyTo(“sender@email.com”, “Webmaster or Sender’s name”);
$mail->WordWrap = 50; // set word wrap
$mail->IsHTML(true); // send as HTML
$mail->Subject = “subject”;
$mail->Body = “body of text”;
$mail->AltBody = “Alt body”;
$mail->SMTPDebug = true;
/*As mentioned earlier, Php_mailer calls echo for all debug information. If SMTPDebug is turned on, buffer the echos and then output them to the log file.
Also be sure to end bufferring in case exception happens or otherwise.*/
if($mail->SMTPDebug) ob_start();
if(!$mail->Send())
{
if($mail->SMTPDebug){$strE = ob_get_contents();ob_end_clean();}

/* RaiseException simply logs and throws exception. You can append $strE saved above if you want to log. PhpMailer error messages are not descriptive enough in most cases and it’s a good idea to log the debug info collected through $strE */
$this->RaiseException(“Error in sending mail=”.$mail->ErrorInfo);
}
ob_end_clean();
// Successfully sent mail.

Ensuring that you are good citizen and behave with etiquette:

When email is sent because of the nature of email, delivery can not be guaranteed. However, you want your email to be not viewed as spam or annoying. As such there are certain steps you need to take to ensure your email is not misconstrued as spam. This url gives guidelines on how you can prevent your email is not viewed by receiving parties, Internet providers and Amazon SES as SPAM.
All the documents in this url should be read thoroughly.

It’s also important to have a process for handling bounced emails, complaints, Out of Office, and “suppressionlist” notifications. You should take action to remove emails that are notified as no longer in service, wrong, or against which you received complaints from SES. If your bounce or complaint statistics goes up, SES may deny your request to send email.
Amazon SES provide sandbox email to verify the process you have set up to deal with them. It’s outlined here. Note you should only send mail to the listed addresses @simulator.amazonses.com. Sending to other addresses that are not in this list even @simulator.amazonses.com, may result in being marked as spammer.

To increase authenticity, it’s advisable you sign all your communication with DKIM (Domain Key Identified Mail). DKIM attaches a new domain name identifier to a message and uses cryptographic techniques to validate authorization for its presence. The identifier is independent of any other identifier in the message, such in the author’s From: field. Amazon does the DKIM signing for you. All you have to do is, enable your domain to be verified by Amazon so that it can authenticate and sign your email before it sends it out.
The process is really simple. Follow the instructions in this link.
If you are using GoDaddy as the registrar for the domain, you need to add the CNAME records in GoDaddy service.
I looked for how to do this and there is very little help on this. So I will give you instructions on how to do this.

• Login to your Go Daddy account.
• Domain Manager
• Select
• Click on Edit
• Under the CName record table, click on quick Add button
• Now copy each of the three CName records from DKIM CName records generated using EasyDKIM as described in SES Easy DKIM document
• Note the Name should go in “Host” field and Value should go in “Points to field”.
• While Amazon says it will take 72 hours to verify, the DKIM status is updated in a few minutes. You go to the “Email Address”, click on the search icon against the particular email for which you are setting up DKIM. Click on DKIM, it will show the status as “pending” and once verified, it will change to “verified”.

The last step in DKIM setup is to enable it. The link to do this is in the same place as above.
After it’s enabled, send a test mail again. If you can view the header on your email client, you should be able to see the header contain “dkim=pass (ok)”. This means you are all setup to send DKIM signed emails and it’s successfully verified by the ISP and email client.
After you complete whatever other tests you want to run, the last thing to do is to request for production access. Click the button that says “Production Access” that’s in SES console dashboard. A support request form shows up, fill the fields appropriately and submit. You wait for Amazon to get back to you and in a day or two you should hear from them.

While this is long blog and seems like there are a lot of steps involved, the whole process took a day or so but with quick series of steps and then wait for the verification process. The actual time I spent doing the work was only an hour or so. Most of this hour was also consumed by trying to debug the problem with PhpMailer.

Hope this is helpful. As always, love to hear comments. While you are at it, check out Mofinto. It’s totally free.

Everyone needs financial planning. Mofinto is a simple tool that puts you in charge of your financial future, educates you and helps you plan your finances. Visit Mofinto.

Node.js module.exports vs exports

I am beginning to explore node as an alternative or to augment Apache and PHP for server-side programming. There is a lot to like there. It’s fast, simple, and asynchronous. There are also a lot of things that make it hard. For example the fact that there is no real process isolation in node means any exceptions, if not, properly handled, will bring down the web site. Debugging is not the easiest. I will probably write comparing Apache and PHP with Node at a future time and if possible, how they can be combined. Not today.
This assumes you already have node installed and you are familiar with node. If not, read the manual, and download instructions etc. at Node.js.

I created a simple hello world service. Where there is a file hw.js which has the following:

http.createServer(

function(request, response)

{

ProcessRequest(request, response);

}

).listen(8888);

The function ProcessRequest is given below:

function ProcessRequest(request, response)

{

response.writeHead(200, {“Content-Type”: “text/html”});

var h = require(“./hello.js”);

h.Hello1(response);

var o = h.c();

response.write(“A=”+o.a+” B=”+o.b);

o.h2(response);

response.end();

}

Note the require keyword here which instantiates the model hello.js. In Node a module is an object that refers the script object defined in this case by hello.js. ‘module’ object has one specific object hanging off it called ‘exports’.
Now let us look at hello.js. It’s contents are fairly simple. There are two functions “Hello1” which writes out “Hello World1” and “Hello2” which writes out “Hello World2”.

function Hello1(response)

{

response.write(“Hello World1”);

};

function Hello2(response)

{

response.write(“Hello world2”);

}

exports.Hello1 = Hello1;

function CreateHello()

{

var o = {};

o.a = 1;

o.b = 4;

o.h2 = Hello2;

return o;

}

module.exports.c = CreateHello;

It’s the line after that makes it a little more interesting. “exports.Hello1 = Hello1;” tells node that the module.exports object contains a function Hello1.
In hw.js the two lines “var h = require(“./hello.js”);h.Hello1(request, response);” essentially indicate that hello.js should be instantiated and the value module.exports returned from hello.js should be stored in h. h.Hello1 will simply call the function Hello1 in hello.js.
The function “CreateHello” is a constructor example. However, the important point that should be observed is that “CreateHello” is assigned to module.exports.c. This is essentially the same as assigning it to exports.c. The rest of the code is self-explanatory.
The question that I could not understand is the difference. Hope the following explanation makes the distinction clear.
“require” always exposes the object “module”. “exports” is a short cut to the module.exports object. However, if you assign another object or value to module.exports, that’s the point where module.exports starts referring to a different object than exports.
Say, you do the following in hello.js

module.exports = CreateHello;
instead of

module.exports.c = CreateHello;
then in hw.js the line

h.Hello1(request, response);
will fail.
This is because Hello1 is hanging off of exports where as module.exports now contains reference to function CreateHello only.
If you want to create modules, generally it’s a good practice to use exports.
However, there are a lot of times when you want to return an object that’s created in the module rather than returning the system created module object.
In this case, at the top of the hello.js add the two lines

var ho = {};

module.exports = ho;

Now instead of assigning Hello1 to exports object

exports.Hello1 = Hello1;

simply assign it to the object you created.

ho.Hello1 = Hello1;

Similarly replace

module.exports.c = CreateHello;
with

ho.c = CreateHello;

The distinction is subtle but important to understand especially if you want to create code that’s object oriented, create objects with properties and methods that you want to expose from each of the modules.
Happy scripting and debugging!

Everyone needs financial planning. Mofinto is a simple tool that puts you in charge of your financial future, educates you and helps you plan your finances. Visit Mofinto.

Debugging web pages and javascript

The first thing I look for before writing a line of code is a way to debug the code I produce, for it’s impossible to get any real development without introducing bugs. This is true with JavaScript too.
Today, I thought I would document simple way of enabling debugging in each of the popular browsers:

• Chrome
• Firefox
• Internet Explorer
• Safari

All of these come with html/css inspectors, profiler, network monitoring, script debugger, and last but not least a console for seeing errors, warnings etc. I am not going into details of how to use them.

Chrome

Select the Chrome menu (the one with 3 horizontal bars) at the top-right of your browser window, then select Tools > Developer tools.
There are several useful shortcuts for opening the DevTools:
Use Ctrl+Shift+I (or Cmd+Opt+I on Mac) to open the DevTools.
Use Ctrl+Shift+J (or Cmd+Opt+J on Mac) to open the DevTools and bring focus to the Console.
Use Ctrl+Shift+C (or Cmd+Shift+C on Mac) to open the DevTools in Inspect Element mode, or toggle Inspect Element mode if the DevTools are already open.
You can also use F12 (function key) to open the dev tools on windows.
For information on how to use the debugger I refer you to Chrome Dev tools

Firefox

The instructions for Firefox are simple at the same time the usage is complex. You need to install a tool called Firebug
Once installed there will be a Firebug menu (that looks like a “Firebug”) installed on the top-right of your browser window. Simple click on this to see a drop down. The usage is pretty similar to the debuggers in other browser.

Internet Explorer

This is similar to Firebug. You need to download and install IE Debugger.
After installing you can either type F12 or use Tools>F12 Developer Tools to launch the debugger. Note IE, by default, opens a separate window for debugging.

Safari

In Safari browser, go Preferences menu. Select Advanced. At the bottom of the Advanced dialog, you will see a check box that says “Show Developer menu in menu bar”.
If you already don’t have the menu bar showing, you need to go to the spoked wheel menu at the top right of the browser window and click on Show Menu Bar. This will display the menu bar. You will find the Developer menu on the menu bar. It has various options. You can launch console, start debugging etc. to do various things you want to do while debugging.

Note internet explorer debugger has option to mimic various versions of IE, which is really useful to make sure your website or script is compatible with different versions of IE.
The UserAgent option on Safari developer menu emulates not only Safari versions, but other browser versions too. While this is useful to test browser compatibility, my recommendation is to use the different browsers.
Happy scripting and debugging!

Everyone needs financial planning. Mofinto is a simple tool that puts you in charge of your financial future, educates you and helps you plan your finances. Visit Mofinto.

MySQL cursors

I know, cursors are considered as bad. Why is this? Cursors take up memory and as long as you have cursor open, they lock the records, and sometimes entire table. If you are inserting or creating other tables while looping through the cursor performance can take a serious hit.
Sometimes, looping through the result of a select seems far easier to do and probably the efficient thing to do rather than complex select statements that are too specific to the situation. At least, I find in Mofinto one place I had to use cursor since I wanted to keep the stored procedure general and not be too specific to the situation. Hopefully, you don’t need to use cursors but if you do you may find this useful.
Let us say you have a table

MyTable

ID	column1	column2
1	value11	value12
2	value21	value22

Say, your goal is to iterate over each of the rows in “MyTable” and call stored procedure “MyProc” which takes two scalar values “val1” and “val2”.
To use a cursor you declare a cursor on a select statement. Then you declare at least one handler for catching the end of the cursor. You write your loop for cursor where it calls the stored procedure. At the end you call close curl.
It’s very important that you make sure close curl is reached.
The following code should do what we want – namely call MyProc for each row in MyTable.

declare val1 int;
declare val2 int;
declare done int default 0;
— Declare the cursor on the simple select statement.
declare cur1 cursor for
select MyTable.Column1, MyTable.Column2 from MyTable;
— Declare continue handler. Not Found essentially catches end of cursor, which sets done to 1
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
— open the cursor
open cur1;
— Start the cursor loop which ends in end loop; as below.
read_loop:loop
   fetch cur1 into val1, val2;
   IF done = 1 THEN
          LEAVE read_loop;
   END IF;
   call MyProc(val1, val2);
end loop;
— Very important to close curl after the end loop;
close cur1;

It’s that simple. Of course, the select statement can be complex with multiple joins etc.

Declare handler can be fancier. I refer you to the MySQL documentation for details of Declare handler usage.

MySQL – select into

I only cover simple select. But the other day I ran into an issue with a simple joined select statement that gave a cryptic error ERROR 1172 (42000): Result consisted of more than one row.

The stored procedure was complex one. So it took me a while to notice that I had at some point in the stored procedure, I had a “select into” statement.Say you have a table as following

MyTable

ID	column1	column2
1	value11	value12
2	value21	value22

select ID into @id from MyTable;
will produce the Error 1172.
This is because select into expects the result of the select to be single value and not a row set.
This is simple enough to spot. The issue I was tracking down is in a slightly complex and big stored procedure. Also it was a complex join where three tables were involved. I accidentally left out a join condition when I extended the select by introducing a join with fourth table.

Setting up LAMP (Linux,Apache,MySql,Php) stack on Mac

I just got MacBookAir. I was very excited because it’s my first MacBookAir. I also thought I can easily get Mofinto dev environment setup on that since it’s Linux, correct?
Boy, was this wishful thinking on my part. My existing Ubuntu shell scripts did not work. I found that LAMP is not properly setup.

It took me close to 3 days of intense Googling, debugging, fixing etc. Finally, I have it all working.

I created the steps that it would take to set up LAMP on Mac. In this blog all the lines marked by italics and red color represent the steps you need to take in order for LAMP to be setup. (All these steps assume you are in the terminal window). The rest are commentary or explanation of the steps. If you are interested, it’s not difficult to turn all these steps into a shell script. But I am not attempting to do that here.

Note my assumption was that Mac OS/X is linux underneath. That’s correct. Only thing is it’s modified. It also came with Apache and PHP5.3 installed. There was no mysql on the machine.
I could have installed MAMP instead of enabling what’s already there. That may have avoided one or two issues like the one where MySQL setup and PHP connection were looking in different directories. However, a lot of the complications arose because of the differences between Ubuntu-Linux and Mac-Linux. Don’t get me wrong. I have no idea if MAMP is a lot smoother route to get LAMP on Mac.

With that introduction, let us jump into what needs to be done to get LAMP setup properly.

The first step is to get Apache and PHP setup properly.

I found that Apache is installed in /etc/apache2. Error logging goes to /var/log as pointed to by http.conf located in /etc/apache2. Note that the DocumentRoot is pointing to /Library/WebServer/Documents. However, there is no index.html file there. Instead there is an index.html.en. Copy this to index.html and you will notice that http://localhost shows the typical ‘ItWorks!’ in the browser.

Note – In mine both /var and /etc are symlinked to /private/etc and /private/var. So when I refer to /etc, it’s the same as /private/etc.

I found there is no php.ini in /etc where it’s expected to be. Not having php.ini is resulting in Apache not logging any php errors.

– cd to /etc
– sudo cp php.ini.default php.ini
– Edit httpd.conf under /etc/apache2

The following lines will make Apache see and process PHP files

– uncomment the line that says LoadModule php5_module/usr/local/opt/php53…
– uncomment line that specifies ServerName

The following lines will set the DocumentRoot to /var/www. You can skip this if you don’t want to set it or change it to whatever you want your document root to be.

– Change the DocumentRoot from /Library/WebServer/Documents to /var/www
– Change also reference to /Library/WebServer/Documents under /var/www

If you need to enable SSL, you have to change the DocumentRoot (if you choose to) in httpd-ssl.conf under /etc/apache2/extra.

After the above changes are done, restart apache service.

– sudo apachectl restart

The following steps are for setting up MySql:

– download MySQL
– Download the MySQL community server(GPL) version
– Follow the instructions given here to install http://dev.mysql.com/doc/refman/5.0/en/macosx-installation.html
– For this follow the instructions from MySQL documentation

I found this blog which explains things a little more clearly if you are confused by MySQL documentation
After everthing is installed you will find MySQL in /usr/local/mysql. Do the following to set up the aliases and paths so that you can type mysql, mysqlstart, mysqlstop etc. from anywhere.

– nano .bash_profile in home directory
– Add the following lines
PATH=${PATH}:/usr/local/mysql/bin
alias mysqlstart=”sudo /Library/StartupItems/MySQLCOM/MySQLCOM start”
alias mysqlstop=”sudo /Library/StartupItems/MySQLCOM/MySQLCOM stop”
alias mysqlstatus=”ps aux | grep mysql | grep -v grep”
– Save and close nano

Typing mysql into the terminal window should work.
If you want to a specific domain name for your development, you have to update hosts file. The hosts file is located in /private/etc/hosts.

– sudo nano /private/etc/hosts
– add “127.0.0.1 devdomainname” under the line which says “127.0.0.1 localhost”
– Save and close

By default, MySQL is setup without any password. It’s advised that you set a root password. The following lines describe how to do that. Here the ‘rootpass’ should be replaced by the password you choose for your MySQL root account. ‘MacBookAir.local’ is the local machine account. If you do “select host,user from mysql.user” you will find the proper name for your machine account.

– mysql -uroot
– use mysql;
– SET PASSWORD FOR ‘root’@’localhost’ = password(‘rootpass’);
– update user set password=password(‘rootpass’) where user=’root’ and host=’MacBookAir.local’;
– SET PASSWORD FOR ‘root’@’127.0.0.1’ = password(‘rootpass’);
– SET PASSWORD FOR ‘root’@’::1′ = password(‘rootpass’); 

Note that this can only be run once. Next time, you need to supply the password with -p option to get root access to your installation of MySQL.

Now there is just one more thing to be done. I found MySQL is setup properly and Apache is processing PHP files. But PHP was not able to connect to MySQL. After feverish searching on internet for the cause, I found that in php5 which moved the mysql.sock location to /var/mysql instead of tmp. MySQL still is putting mysql.sock in /tmp. To get around this, do the following fix namely symlink to tmp so that php is happy.

– mkdir /var/mysql
– ln -s /tmp/mysql.sock /var/mysql/mysql.sock

This got me up and running the full LAMP stack on Mac. With LAMP setup properly, my shell scripts for Ubuntu worked. Now my MacBookAir is proving itself useful.

Hope you find my experience useful. Please leave a comment if you feel like it.

PHP to JSON- How to serialize PHP class to JSON format?

If you are using PHP to implement REST API, you will come across this question. The answer to this, in general, is really simple.
PHP comes with two functions json_encode and json_decode, for encoding a PHP object into JSON and JSON string into PHP object respectively.
Check out the manual at Php.Net for the full details of this function.
The call is simple $j = json_encode($v);
$j contains the JSON for $v which can be an array, object or scalar variable.
However, here are a few limitations to json_encode.

1. If you have strings stored in the object your encoding, they have to be UTF-8 strings or you need to convert them to UTF-8 using utf8_encode before calling json_encode
2. If you have member variables that are marked as private, json_encode can’t see them.
3. Associative arrays will be represented as objects. Sequential arrays will be represented as arrays in JSON. You can force them to be represented as objects by either casting the variable to object or pass in the option JSON_FORCEOBJECT

Being able to expose private and protected variables is very important. This can be done in various ways. You can use an Iterator like an ArrayIterator and call json_encode on that. The solution I am going to outline involves writing your own custom encode function that uses json_encode.
As an example you take the following classes in PHP. JsonTest contains JSon_Test1 which in turn contains JSonTest_2.

class Marshall
{

public function propvalues()
{

return get_object_vars($this);

}

}

class JSonTest_2 extends Marshall
{

public $arr2;
public function __construct()
{

$this->arr2 = array(1 => “stest”, 4 =>”stest2″, 3=>5.1);

}

}

class JSonTest_1 extends Marshall
{

public $arr1;
public $j2;
private $jp2;
public function __construct()
{

$this->arr1 = array(“test”, “test2”);
$this->j2 = new JSonTest_2();
$this->jp2 = “jp2”;

}
public function propvalues()
{

return array(“arr1” =>$this->arr1, “j2″=>$this->j2->propvalues(), “jp2″=>$this->jp2);

}

}

class JsonTest extends Marshall
{

private $arrActuary = array();
public $jt1;
private $jt11;
private $arrTestM = array();
public function __construct()
{

$this->jt1 = new JSonTest_1();
$this->jt11 = new JSonTest_1();
$this->arrTestM = array(1 =>”value”, “test” =>100.21);

}
public function propvalues()
{

return array(“jt1″=>$this->jt1->propvalues(), “jt11″=>$this->jt11->propvalues(), “arrTestM”=>$this->arrTestM);

}

}

class JSon
{

static public function encode($o)
{

if(is_a($o, “Marshall”))
return json_encode($o->propvalues());
return json_encode($o);

}

}

The following code will call normal json_encode passing in $j and then call the special encode on $j.
$j = new JsonTest();
echo “normal encode”;
echo json_encode($j);
echo “special encode”;
echo JSon::encode($j);
This will echo the following two lines: (You can pretty print these to make them more readable etc.)
normal encode
{“jt1”:{“arr1”:[“test”,”test2″],”j2″:{“arr2”:{“1″:”stest”,”4″:”stest2″,”3″:5.1}}}}
special encode
{“jt1”:{“arr1”:[“test”,”test2″],”j2″:{“arr2”:{“1″:”stest”,”4″:”stest2″,”3″:5.1}},”jp2″:”jp2″},”jt11″:{“arr1”:[“test”,”test2″],”j2″:{“arr2”:{“1″:”stest”,”4″:”stest2″,”3″:5.1}},”jp2″:”jp2″},”arrTestM”:{“1″:”value”,”test”:100.21}}

I define a class called Marshall which when inherited automatically exposes all the public variables as an array. If you override the function propvalues in the inherited class as is done in the class JSonTest_1, you will be able to iterate over the private and protected variables too.
I also defined a helper class JSon with static function encode which calls json_encode on the object if it does not inherit from Marshall otherwise it will call propvalues function and json_encode the return value.

Here we make use of two properties of json_encode to achieve this. First one is the fact that json_encode recursively encodes the objects. Second one is that associative arrays are encoded as objects. The second property is the one that’s used in propvalues where we are constructing the associative array with the private, protected and public variables.

Note that propvalues itself can check if the contained class in this case JSonTest_2 is a Marshall class and call propvalues on it.

The code above has an advantage over normal json_encode function in that you can transform the class to be different from what it is in PHP when you serialize in JSON.. can be selective about which member variables in the class you want to add to JSON. You can also decide what to name them.

(Note – json_encode is available in 5.2 version or later and get_object_vars in PHP version 4.0 or later)

A really good way to compress javascript before it’s deployed on your server

If you have JavaScript served up as part of your web service at least for the UI portion, you may want to make it as compact as possible. But for development time, you want the code to be readable and debugged. There are various tools available on the web to make the JavaScript compact and also to obfuscate so that it’s a lot harder for others to decipher the logic in your script.

The tool I like is google’s closure compiler.
The compiler operates at different levels. At Basic level it simply removes white spaces, comments, and also renames local functions. If you have one file, you can use the advanced option which will obfuscate all the function names also. However, it’s more complex to use the advanced option, if you have multiple JavaScript files that refer to external functions. For this closure compiler does provide a way of providing mark up for functions that are exported. However, we only use the basic option. For processing function names, I have a php file that takes input file of functions that should be excluded from obfuscation. The php file process javascript files to obfuscate the function names.

The first step is to download the closure compiler. The details of download and getting started are given here.

The compiler requires java. Put the following lines in a shell script and run it.
#sudo apt-get install gcj-4.4-jre-headless
#sudo apt-get install gcj-4.6-jre-headless
#sudo apt-get install openjdk-6-jre-headless
#sudo apt-get install gcj-4.5-jre-headless
#sudo apt-get install openjdk-7-jre-headless
sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install oracle-java7-installer
sudo java -version

Once java is installed, you run the following command.
java -jar compiler.jar –js=dirname/javascriptfile.js –js_output_file=dirname/outjsfile.js

That’s all there is to it. Your outjsfile.js will be reduced by about 30% compared to the javascriptfile.js. The more comments and white spaces you have the bigger the reduction in size.
Happy compression and obfuscation.

Starting on Ubuntu but don’t have awk?

GNU version of the awk is called gawk. Getting “gawk” installed is simple.

Simply type the following:

sudo apt-get install gawk

Here is where you can find the manual for gawk. gawk script processes the input line by line.

For example the following searches for java in the file process.sh shell script and prints out the lines that have some occurrence of the word java in them.
gawk ‘/java/ { print $l }’ < process.sh

The following will print the number of characters in the lines that contain the word java in the file process.sh
gawk ‘/java/ { c=length($l} END { print c }’ <process.sh

There are some built-in variables that are maintained by gawk and can be used.

For example NR counts the lines and can be used to number the lines. The following will print all the lines in process.sh with line number in front of each line.
gawk ‘ {print NR $l}’ <process.sh

The following will print the total number of lines in the file process.sh
gawk ‘END { print NR }’ <process.sh

Note I these are the simplest possible gawk scripts. Gawk is very powerful langugage and whenever you need to process line by line, you can do it using gawk. The resulting script is most likely much smaller than anything you can do in shell script.

Getting started on Linux

To develop or test on Linux first thing to do is to decide which Linux version you want to get. There are a lot of different Linux options to choose from. RedHat, Susie, Ubuntu to name a few. I chose Ubuntu because of its long-standing reputation and community support it provides.
Next step is to decide if you want server or desktop. Get desktop if you want GUI. Server exists in both. You can install Apache, MySQL, php (LAMP stack) on it easily.
There are various ways of installing Ubuntu on windows machine. I chose the windows installer as it’s probably the easiest. Because it’s a windows installer and you can install it to run inside windows, you have the option of being able to uninstall Ubuntu from the control panel, if you ever choose to. See the instructions in this article on how to download and use the windows installer.
I did this on Windows 7 and chose the 64 bit version. Of course, I never tried to uninstall.
I would recommend 12.04LTE version of Ubuntu as it’s fully supported.

Once your Ubuntu installation is all complete and you reboot your machine, you should see option to boot into Windows 7 or Ubuntu. You should be able to login to your ubuntu partition and start using.
If you are new to Linux or have not used it in a long time, the following would make it a little easier to start.
Ubuntu desktop is easy to use and should be somewhat familiar to windows users. However, the power of Linux is in the command line.
Pressing Ctrl+F1 through F6 would give you command line modules to operate in. Ctrl+F7 should take you to the desktop.
In desktop mode, you can use the graphical text editor gedit. But I prefer to use the command line mode. The editor that I use is nano.
First thing we want to do after installing ubuntu is to get LAMP stack. We got Linux. Now we need to get Apache, MySQL, and Php.
I created a shell script that will prepare the Ubuntu machine for being web server. This shell script call it preplamp.sh can be used to start any Linux installation you want subsequently.
Put the following lines in preplamp.sh:

• sudo apt-get install tasksel
• sudo apt-get update
• sudo tasksel install lamp-server
• sudo /etc/init.d/apache2 restart

If you want you can enable various apache modules using a2enmod. Add the following lines

• sudo a2enmod ssl
• sudo a2enmod rewrite
• sudo a2enmod headers
• sudo a2enmod expires

depending on what you want enable.
That’s it. Now type the following at the command line.
bash preplamp.sh
This will setup Apache, MySQL, and php automatically but the only point that requires your intervention is when you need to type in the root password for MySQL installation.
To test your installation is complete, simply go to the browser and type http://localhost. You should see the standard It Works page from apache.
Now that you have LAMP installed, the fun begins. More on this in the days to come.