Wednesday, September 29, 2010

Set Default Priority for all Processes for a Specific User

A few years ago, my server has been DDoS. There were more than 7000 connections. I couldn't ssh into my servers to monitor what was going on live. After that, I found out a trick that you could set the priority for any running processes.

But it's very inconvenience as most server services are auto start. Anyway, there is an easier way to get this problem resolve. Linux allows us to set the default priority for all processes for a specific user. By giving www-data user a lower priority, I can ssh into the server even if it's under attack. Assuming that we have sufficient memory. 
sudo vi /etc/security/limits.conf
The configuration syntax is as follow:  
[username] [hard|soft] priority [nice value]
Insert the following line:
www-data hard priority 10
Now, reboot the server.

To verify that you've done it correctly, run top:
Expected results should be similar to this:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  GROUP    COMMAND
 1295 www-data  20  10 33688 6388 2544 S    2  0.2   0:00.32 www-data apache2
 1296 www-data  20  10 33064 6100 2492 S    2  0.2   0:00.35 www-data apache2

Monday, September 27, 2010

Experian Hitwise - Company Profile

Hitwise Home 
Custom Data & Analytics
  • Has the largest online customer behavior datasets
  • Solve marketing objectives 
Each day, Hitwise analyzes more than 3 million Australian Internet users interacting with more than 1 millions websites across 165+ industries.


Products & Services

Competitive Intelligence
By analysing user browsing pattern, Hitwise identifies online marketing opportunities for clients. This information can be used to compare with competitive websites.

Features
Hitwise Rankings - provides an instant overview of key players and fast-moving websites across 160+ industries.

Hitwise Search Intelligene - provides extensive insights on how people have searched for products and services across all major search engines.

Hitwise Clickstream - provides extensive reports on upstream traffic and downstream traffic from a specific website, industry or category. 

Hitwise Demograhics - visistors' profile including age, gender and geographic location.

Hitwise Lifestyle - identifies the top performing sites that attract your most desirable customers. 

Hitwise Conversions - how customers interact with specific sections on competitive websites. Also show which customer acquisition tactics have proven to be the most successful.

Hitwise Charting - create customized charts how websites, industry categories or search terms against each other.

Hitwise Customization and Alerts - email notification when there is a change in competitive activity within your industry.

Hitwise Dashboards - view all your favorite report from one single page.

Hitwise Custom Data and Analytics - Hitwise has 25 millions internet users world wide and enormous proprietary datasets to deliver information and analysis to specific business objectives.

Hitwise University - training and eduction resource for all Hitwise clients.


Search Marketing
Hitwise specialized in  Search Engine Optimization, Pay Per Click, Conversion Optimization and Analytics since 1997.

Pay Per Click - provides a complete end-to-end solution for small and enterprise businesses that will increase return on investment, drive sales and leads at a lower cost. It's 100% transparent and measurable.

Search Engine Optimization - focus on strategies that maximizes search engine visibility and apply best practices to deliver return on investment for clients.

Conversion Optimization - making it easier for customers to take action such as buy a product on your website.

Analytics - comprehensive Analytics services that help clients better understand their online visitors and to reach them effectively and efficiently. 


Search Engine Optimization
The art and science of increasing your website visibility to search engines for those important keywords and phrases that are relevant to your business. 

How we do it
Understand the behavior of clients' customers, how they search and ultimately drive them to the most relevant content of your website.

Model
Business Objective -> Measuring Progress Against Objectives -> Unique Technical Recommendations - >Market Research -> LOOP
  • Business Objectives - achieving clients' objectives; increase sales, lead, market share or brand awareness.
  • Market Research - Hitwise have access to the habits of over 3 million Australian internet users. We know what they are searching for and how to reach them.
  • Recommendation and Implementation - provides clients with the most effective recommendations to help achieve their objectives.
    • Keyword Research
    • Competitor Research
    • On Page Optimization
    • Site Wide Recommendations
    • Link Discovery
    • Link Building
    • Actionable Recommendations So You Can See Results
  • Measuring Progress Against Objectives - Ongoing review and analysis of progress against objectives helps to ensures clients key performance indicators are achieved and maintained.
    Parameters measured:
    • Rankings
    • Traffic
    • Sales
    • Leads
    • Conversions
    • Channel performance

About Us

  • Provide data and analytical tools to clients in more than 65 countries. 
  • Over 100,000 clients and annual revenue of nearly $4.0 billion. 
  • Over 15,000 employees worldwide.
  • Help clients protect and grow their market share.
  • Has the largest sample of internet users of 25 million worldwide, including 3 million in Australia.
Company Milestones
1997
  • Founded by Adrian Giles and Andrew Barlow as Australasia's first search marketing and consulting company (Sinewave Interactive).
  • R&D begins into popularity based search engine (Top100.com.au) and Hitwise Rankings service.
1999
  • Top100 Search Engine launched as the world's first popularity based search engine using ISP data.
2000
  • Hitwise Competitive Intelligence Service is launched with ranking, profile and charting features only. Subscription to the service sold directly over the internet.
2001
  • Launched Clickstream counting technology
  • Launched Hitwise New Zealand
  • Launched Hitwise Hong Kong
  • Launched Hitwise UK
  • Launched Hitwise Singapore
2002
  • Awarded 5th fastest growing technology company in Australia by Deloittes
  • Awarded 10th fastest growing technology company in Asia by Deloittes
  • Acquired IP assets of Plurimus Corporation in USA
  • Launched Hitwise Demographics, Benchmark Reports and Media Alerts
  • Formed alliance with Media Monitors to license media alerts technology
  • Grew sample size to 13M Internet users and 200,000 websites
2003
  • Launched Hitwise US
  • Awarded Telstra Small Business of the Year Finalist
  • Adrian Giles named Ernst & Young, Entrepreneur of the Year Finalist
  • Grew Sample size to 25M Internet users and 500,000 websites measured
  • Launched Hitwise Search Intelligence
  • Launched Hitwise Lifestyle
  • Established strategic partnerships with Claritas and Experian
2004
  • Telstra Small Business of the Year Finalists
  • Awarded 6 fastest growing technology company by Deloittes
  • American Business Award Finalist
2005
  • Named Victorian Export Awards Finalists
  • Launched Keyword Intelligence service
2006
  • Awarded Deloitte & Touche Technology Fast 50 - New York, 2006
  • Launched Hitwise Conversions Technology
  • Grew database to 800,000 websites measured
  • Purchased HitDynamics marketing management platform
2007
  • Grew database to 1 million websites measured
  • Announcement of Experian's acquisition of Hitwise
  • Launched Hitwise Dashboards interface
2008
  • 1500 client worldwide
  • Completed acquisition and integration of Experian
  • Publication of Click
  • Launched new Hitwise Search Intelligence
  • Ranked 37th on the Honomichi list of the top 50 US Market Research Firms
2009
  • Launched Hitwise Canada
  • Launched Hitwise Brazil


    Friday, September 24, 2010

    Interviewing techniques

    My wife is pregnant, she is expected to deliver the baby sometimes on February next year. 

    I believe it's best for my wife to deliver the baby in Australia. I had to sponsor my partner to Australia. The application is done. I think she will be here late next month.  

    I came back to Melbourne early this month. Now, I have to look for a new job to support my new family. All my families and friends warned me that getting a job these days is very tough. Recruitment agencies and companies are getting hundred if not thousand of emails everyday. Well, I have to work harder if I want to land a good job. 

    My quest start from here!

    In order to land a job, I need to learn 3 skills. 
    1. Writing the resume
    2. Writing the cover letter
    3. Answer interview questions
    Hence, I have to read 3 books during the last 2 weeks:
    1. The Quick Resume & Cover Letter Book
    2. 201 Killer Cover Letters
    3. Interview Skills That Win The Job
    I also read other books roughly, but all of them teaching similar concepts. 

    On this blog entry, I want to focus on interview techniques as I will get an interview very soon. I want to remind myself that cover letter and resume only lead to the interview. In order to win the job. I have to do well in the interview. I might be the best technical person, but I might not be able to secure the job.

    Interview is one of the hardest stage for me as I am one of a few people who suffer from impostor syndrome. This is why I am writing this entry to help me overcome it.

    Here are some of the tips I got from the books:

    Myths
    1. Prepare for the interview regardless of your experience. You may not be able to convoy this message at an interview.
    2. There is no right or wrong answer. Justify your statement is important.
    3. Experienced interviewers let you do most of the talk whilst inexperienced interviewers usually unprepared and keeps their phone ringing during the interview.
    4. If you have to say I don't know, say it! I think honesty is the best policy.
    5. Dress neatly and be friendly.
    6. Good answer is important, but it might not give you the job. Interviews are about rapport and trust. You should be a nice person to work with.
    7. There is no perfect answer because we don't know what the interviewer wants to hear. Keep it simple! Don't go around in circle trying to cover all possibilities.
    8. If you have no relevant question at the end of the interview, don't ask! 
    9. Just relax and be yourself. That doesn't mean you can lean back or making jokes.
    10. Interviewers are not looking for flaws. If you think that way, you cannot establish rapport and trust. Treat each question as an opportunity to excel yourself.
    Convincing them you're right for the job
    1. Common mistakes
      • Failing to express oneself clearly - simple language is the most effective. Avoid jargons or complex sentences.
      • Not being aware of one's body language - eye contact, sitting position and facial expression are important. 
      • Failing to control those nerves - feeling anxious before and during the interview is common. However, try to avoid it.
      • Trying too hard to please the interviewer - building rapport and trust during the interview is critical. However, very few interviewers appreciate interviewee going overboard.
    2. There is nothing wrong with you, everyone should commit some of the mistakes above in the interview. Try not to repeat your mistakes.
    3. Correct preparation, even if you are the only candidate preparing before hand would likely to give you a higher salary.
    4. The case for preparation
      • Interviews are rare events
      • Many people find it hard to sell themselves. I am included.
      • In most interviews, coming second isn't good enough. You need to beat other people.
    5. What is incorrect preparation
      • Rote-learning generic questions.
      • Learn other people responses by heart.
    6. Advantages for preparation
      • Improve confidence
      • Answer question succinctly 
      • Help you know what to say and how to say it
      • Avoid making negative impression
      • Import rapport-building skills
    7. Things that are important to interviewers - narrow down to 3 generic questions:
      • Can you do the job? They need to know your knowledge, experience, skills.
        What you have done?
        How you did it?
        Results...
      • Are you the sort of person they can work with? 
        Will you fit in the organization?
        Can you work with someone you don't like?
      • How motivated are you?
        You may not be asked this question, but failing to address it will risk you the job
      • Practice - success is one part talent and nine parts perseverance
      • Practicing your answers aloud. I got this from the book, I find it very interesting:
        It is important to practice your answers aloud, rather than just mentally rehearsing them. That’s because the human brain distinguishes between talking and thinking and you need to stimulate the talking part of your brain. Thinking your answers at an interview will get you nowhere, unless the interviewer is a mind reader.
      • Get some feedback - if you are not the winning candidate ask for feedback. Most interviewers will ignore you, but if you get the feedback, you can improve your future interview performance.
      • Mock interviews - setup mock interview with your friend will help measure your performance. It's a good idea if your friends had their own questions that would surprise you. However, you should prepare a list of questions to make it easier for them.
    8. Perseverance - Quitters usually get nowhere. Believe in yourself!
    9. Suggested activity: Neurolinguistic programming - This program help you perform better in the interview:
      • Close your eyes and imagine yourself performing extremely well in an interview. Capture the moment,  sound, smell, temperature, and so on. Don't hold back! Just relax and let it flow. 
      • Repeat the exercise until you capture the feeling of excitement. Picture yourself on the job after the interview. 
      • Every time you feel like quiting or thinking it too hard. Replay the excitement moment you've captured before. 
    To be continue!

    Wednesday, September 22, 2010

    Las Vegas - NABSHOW 2010

    Early April 2010, I flew all the way from Vietnam with my younger brother to America for the NAB show in Las Vegas. I must admit it was the biggest show I ever seen. In fact, it's the largest media conference in the world.

    There were 3 gigantic buildings. Each building took us one whole day to wondering around. All the big players were there:
    • Google
    • Yahoo
    • Microsoft
    • Adobe
    • Alkamine
    • Sony
    • Panasonic
    • Canon
    • Apple
    • ...
    From the NAB show, I learned that current market is about:
    • 3D TV
    • HD TV
    • Broadcasting/Content Delivery
    • Encoding
    • Workflow
    • Handheld device
    However, my unforgettable moment is not the NAB show. It was the 7.2 Magnitude Earthquake while I was in California. Initially, I was on the 4th floor at Huntington Beach Hotel. I want to get a better view so I moved to level 6th. At about 3:40PM. The hotel started to shake. I almost fell off my bed. That was the biggest earthquake I ever encountered.  

    That's my brother Cong and the VNN press. We were on the top level of the RIO building.
    We met a number of executives here including
    Tony, the former PlayBoy CTO,
    Khuong Nguyen, Deputy president of VTC, one of the largest multimedia corporation in Vietnam,
    and a number of other executes which I cannot recall their names.
    Some of them flew to Las Vegas using their private JET.
    I couldn't believe that they paid $US16,000 per night for this wing.
    That's me at the NAB. We were looking at the IO Drive. A SSD hard drive that can do up to 238,000 IOPS. This is very good for database application.
    This is our data center in the heart of Los Angeles. We should have ran the cable properly. Too bad, I usually thousands of miles from data center. I can't do anything about it. 

    Saturday, September 18, 2010

    Fundamental Concepts in Object Oriented Programming

    I never thought of this before reading Andrew Stuart blog. However, it's good for me to rehearse it before my university knowledge getting rusty. I work with PHP most of the time, so OO is not being utilized much. Let's see how I go.
    1. class, object (and the difference between the two)
      Class is a definition or blueprint of the object type.
      Object is created from a class. Object's state and behavior usually undergo significant change during the execution of the program.
    2. instantiation
      The process of creating the object.
    3. method (as opposed to, say, a C function)
      Method must be a member of a class whilst function can have independent existence. I think this only apply to Java. I never heard of a method in C/C++ or PHP.
    4. virtual method, pure virtual method
      Wow, I must admit that I could not remember this one. I must have been working with PHP and servers for so long. Anyway, a virtual method is a method whose behavior can be overridden within an inheriting class by a method with the same name and signature. If I remember correctly, this process is called method overriding.
      REF: http://en.wikipedia.org/wiki/Virtual_function
    5. class/static method
      A static method can be called without creating the object. This is very useful in math calculation. 
    6. static/class initializer
      I don't know. Will find out!
    7. constructor
      A special method that's being called at the creation of the object.
    8. destructor/finalizer
      Similar to constructor, destructor is called just before the object is destroyed.
    9. superclass or base class
      As the name implies, base class is the highest class in the hierarchy and does not inherit from any other class. 
    10. subclass or derived class
      Subclass is a sometimes called a derived class. It inherits some properties from its base class
    11. inheritance
      I think this is the most important feature of Object Oriented. It's the process in which a class inherits all the state and behavior from it parent class. Its parent class could further inherits from another class until the superclass is reached.
    12. encapsulation
      My lecturer Rob Allen usually said this whilst handling out the software project:
      If I see public variable, you fail. You have been warned! 
      This is exactly what encapsulation is all about. There should be no direct access to the data member.  
    13. multiple inheritance (and give an example)
      Most programming languages only support single inheritance. However, some programming languages like C++, Perl and Python do support multiple inheritance. An example would be:
       a mermaid could inherit from fish and human.
    14. delegation/forwarding
    15. composition/aggregation
      A way to combine simple objects or data types into a more complex objects. An example would be: A bicycle has wheels, gears, chains...
    16. abstract class
      Class that cannot be instantiated. A superclass is usually an abstract class.
    17. interface/protocol (and different from abstract class)
      Similar to class, but it only has a list of method names and method signatures. When a class implements an interface, it must define all methods listed in the interface.
    18. method overriding
      When subclass overrides a method defined in the base class. Remember that the method name and method signature must be the same.
    19. method overloading (and difference from overriding)
      More than one method with the same method name can be created within a class. However, the method signatures must be different. This is called method overloading.
    20. polymorphism (without resorting to examples)
      I've been struggling with this concept since University. I still don't understand much about polymorphism. Anyway, this is what I think: there are 2 basic type of polymorphism; static and dynamic.
      Static polymorphism allows programmer to create method with the same name but different method signature. This is method overloading.
      Dynamic polymorphism is where the child class overrides a superclass method.   
    21. is-a versus has-a relationships (with examples)
      Is a
      can be seen from inheritance while has a can seen from composition. An example would be:
      Man is a Human
      Man has a head
    22. method signatures (what's included in one)
      Method signature consisted of method name and parameters. E.g.
      withdraw(int amount)
      Note: return type are not considered to be as part of the method signature.
    23. method visibility (e.g. public/private/other)
      public - can be accessed by foreign class/object
      protected - can be accessed by inherited class/object
      private - can only be accessed locally within the class/object

    Thursday, August 5, 2010

    My Love Story

    Linh & Danh

    Love at first sight

    We found each other about 4~5 years ago through the introduction of our friend Duc. When we first met, I thought she is older than me due to the thick make up on her face. But she is actually 7 years younger. Well, she thought that I was Duc's boyfriend. Duc is gay!

    The next day, we went to Vung Tau beach with my staffs. We stayed there for 3 days. We had so much fun. I still remembered the way she looks in the white dress. On special occasions, I asked her to put on that dress. The dress is still around but its hasn't been worn for years.

    We started living together after about 1 month. Linh was 41 kg. After 3 months, she gained about 6 kg. That's why she could not wear that memorial white dress. Well, she wore that dress when she dropped a few kilos. However, I might not be able to see that innocent look again.

    Happiness

    During the first few months, we enjoyed every moment together. We went shopping, dinning on daily basis. We went to Vung Tau again with a gay couple. They made us laugh every second. We never will forget them!

    We traveled to many tourist attraction places like Da Lat, Nha Trang, Vung Tau, Phan Thiet, Mui Ne. Occasionally, I took Linh to Dong Nai, my home town. We had a lot of fun with my little cousins. There were so many of them.

    Unfortunately, we do not have pictures of the gay couple. These pictures illustrated some of our happy moments:

    Thung Lung Tinh Yeu (Love Valley), Da Lat.
    Linh is leading. My brothers Hoa and Cong at the back.
    Me on horseback.
    I fell off the horse a few times in my life because I always going fast.
    Vinpearl, Nha Trang. Left to right Me, Linh, Cong, Hoa.
    Hoa is my youngest brother, his first time in Vietnam.
    Vinpearl, Nha Trang. Linh putting sun screen on for me.
    My uncle Samson from Australia next to Linh on the left.
    We went to our home town to visit our relatives. 


    Thailand 

    On May 2008, my younger sister (Rachel) went to Vietnam. We went to Thailand altogether for 6 days tour. The happiest moments together were the Tiffany show and other gay shows. Those shows were unbelievable. Just amazing!

    On the last day of the tour, we went shopping for at least 10 continuous hours. Actually, there were clear tasks for each of us. My partner and sister responsible for shopping. I do the paying and carrying. I could see people on the street sympathized for me. I must be carried approximately 10 kg at the end of the day.

    And this is all I got from the trip, a couple t-shirt:

    This photo was taken on the Boat Restaurant, Bach Dang, Vietnam.
    That magnet couple t-shirt was the only thing I got from Thailand tour.
    Safari, Thailand. That's little tiger was cute.
    My sister Giang (Rachel).

    Accident

    One mid-night, while I was working. My mobile phone rang. A complete stranger informed me that my partner was in Saigon hospital. I rushed to the hospital, seeing my love crying, lying on the bed with blood all over her face and body. She was half dead. Doctors and nurses were at the desk chatting and doing nothing! Absolutely nothing! I paid for the services, then they started cutting her cloths. They could not take it off naturally.

    I wish we both in Australia. We would have much better services. Anyway, we moved to Cho Ray hospital the next day because they could not do anything here.

    Cho Ray Hospital

    I could not believe that they arranged my partner to share a standard hospital bed with another unnown patient. I told them that I want a better paid service. They told me that all better services were booked out.

    Now, I better take it or leave it! At that time, I didn't know of any other hospitals. I had to accept the offer before they are putting us on the floor.

    While they were moving us to the room, I tried to locate the person in charge of the hospital beds. I found her and explain our situation, I was almost cry. I think she could felt what I had been through. We finally get our own bed in a better room.

    For one whole day, I didn't see any doctor doing any analysis. This is wrong! My partner was in pain, constantly shaking and I do not see any body here doing anything. That night, I called my best friend's Tai in Australia to get his brother Dr. Hop number. Hop is a practitioner in Vietnam, I went to his house at about 7~8PM with the X-Ray of my partner's skull. I didn't know why the hospital gave me the X-Ray! This is insane, I am not a doctor. I could remember clearly because of the heavy rain that night. I was all wet like a dog!

    Unfortunately, Dr. Hop did not specialize in face and road accident. Nonetheless, he introduced me to doctor Cuong. He analyzed the X-Ray and told me that there were 2 broken bones on her face:
    1. A broken jaw.
    2. A tiny bone near her nose. Nothing can be done as it was located underneath the face bone.
    I informed Dr. Cuong that I already contacted the Australian embassy to arrange medical treatment in Australia for my partner. He insisted there shouldn't be any problem with the operation here. Well, I trusted his judgement. In fact, taking my partner to Australia would required a lot of paperworks. It would takes months. Perhaps, it could not be done because of the way the Vietnamese government system works. 

    The next morning, a doctor came to the room. He asked: 
    Who's the family member of Dr. Cuong here?
    Almost everything flowed quite nicely thereafter. Blankets and bedsheets were changed quite often. 

    Change another hospital

    Later that day, we were informed that this Hospital cannot do anything as the injuries were face related. They asked us to move to another hospital. I called Dr. Cuong, he told me that he already arranged us to Rang Ham Mat (Teeth Jaw Face) hospital tomorrow. I didn't know what to do but wait for further advices. 

    At 10 AM on the third day, Linh was asked to take blood sample. They took 10 cc of her blood. She was in serious pain, every movement would caused tear drop on her cheeks and shattered my heart. The procedures were as follow:
    1. Nurse informed Linh to take blood sample.
    2. I went from level 7 to ground floor to do the payment for taking blood sample. I used the staircases as there were too much people waiting for lift. It would be much faster.
    3. A nurse and I help Linh sit on the wheel chair. We moved to ground zero, wait in the long queue for about 30~45 minutes. 
    4. Thanks god! Tran Ai Linh was called. They took 10 cc of her blood. In case you don't know 1 cc = 1 ml. Therefore, they took about 10 ml. 
    5. I asked the nurse why taking so much, she told me it's for sample test. But Linh would be transferred to another hospital at 2PM today. She said: I don't know
    6. We moved all the way back to level 7 again, on lift!
    Later that day, I found out that they took the patients' blood and sell to other patients. I could never ever thought of this in my entire life. Well, it's happening right here in Vietnam. I strongly believed they were taking patients' blood to sell because no results were sent to any other hospital and they knew that we would be leaving later that day. 
    We hope that no internal organs are missing.
    My first time on Ambulance 

    Again, we had to do the payment before booking the ambulance. I had to hold on to the receipt. Otherwise, we have to pay again. 

    I think the driver was insane! We were not in a hurry. But he turned on the siren and drove like F1 drivers. He went real fast through a lot of bumps and holes on the road. 

    Anyway, nobody was hurt except my partner constantly crying as she swung left, right, up, down along with the motion of the ambulance. 

    Rang Ham Mat Hospital

    This hospital is a little bit more civilized. The procedures were as follow:
    1. Take a ticket
    2. Wait for your number
    3. Pay, keep receipt
    4. Move to your room
    At the other hospital, you wait in the line. No ticket queueing system. There would be a lot of people trying to push in. 

    As soon as we visit the hospital, people knew that we were family member of Dr. Hien Duc. Thanks God! The relationships were arranged by Dr. Cuong. The flowed was quite smooth. 

    I believed it was Friday, Linh met a few room mates of similar situation. It turned out, all of them were here as a result of motorbike accident. They could hardly talk, but all seemed to understand each other.

    Here are some of the pictures. I took these photos because I need to send these pics to my mum for doctor advices in Australia.

    One broken jaw and another broken bone near the nose.
    This look much better than the first day.
    Imagine a bed like that with 2 people.
    Scars are still on both of her arms.
    Her legs took the longest time to heal.
    There are still big noticeable scars her foot.
    After 2 years, all the wounds are healed.
    Linh's legs still hurt occasionally. I don't know why!
    I still could not understand why she got the injury on the inner thigh.
    Operation

    The following Monday, Linh had an operation by Dr. Hien Duc. I heard that she is the best surgery doctor in the hospital.  She was very nice. She advised us what to prepare before and after the operation. My main tasks were:
    1. Pay for the operation.
    2. Buy the titanium plates for connecting the broken jaw.
    3. Buy milk for Linh to drink after the operation.
    4. Buy some other medicines I couldn't recalled.
    The operation went well. Linh was transferred to the resting room. She slept most of the time there. If I remember correctly, I had to constantly applied ice pad to the operated area for two days. I think this would minimize the bleeding and pain.

    Blended congee for 2 months

    Linh's mouth was sealed after the operation. Her teeth were tied together to prevent movement of the jaws. Everyday, Linh's could only eat blended food. She didn't even have the energy to suck it. I constantly whispered on her ears:
    If you love me, you must eat. You must eat a lot as well. You need food and minerals to repair your body. Otherwise, we would have to be in the hospital for a very long time.
    The congee at the hospital wasn't soft enough. It could not get through her teeth. I had to bring our own fruit blender from home. I bought it a while ago for Linh on our anniversary. Anyway, her main course meal was blended congee. To be exact, these were the daily diet:
    • 3 x Ensure Milk Can
    • 6 x Bowl of congee 
    • 1 x Glass of orange juice
    • 1 x Glass of grape juice or apple juice
    Because Linh's couldn't feed herself, I feed her using the following procedures:
    1. Linh laid on the bed.
    2. I raised her back about 45 degrees angle.
    3. I used the big syringe: 
    4. Suck the blended congee.
    5. Pump it into her mouth.  
    6. Naturally, the blended congee would flow to the mouth, down the throat through the gaps on her teeth.
    My partner consumed the food this way for the entire month in the hospital. During this time, I contacted my mum from Australia to buy us medicines for scar treatment. Those medicines were amazing, Linh still using it everyday since the last 2 years.

    Family and Friends

    Linh has a lot of visitors during the first week. I saw many of her friends cried. These visitors were very important for her. They were great supporters. Nonetheless, only a few came back on the second weeks. I, Linh's mother and her step father were around most of the time. I visited Linh everyday. Occasionally, I slept most of the time on my visit. I knew that I couldn't do much, but my presence would encourage Linh to recover much faster.

    Home sweet home

    Linh was really strong. I wonder how she could consume a lot of the same food every single day. She left the hospital earlier than her room mates even though their injuries were not as serious.

    Anyway, we were informed by Dr. Hien Duc that her mouth would be sealed for at least another month. That's mean another month of blended congee. At least, it would be much better than staying at the hospital.

    Seal removed

    A month later, the seal was removed. He mouth was full of blood after removing the seal. We were advice that she could only consume soft food. Well, anything should taste good now. So it doesn't really matter.

    Lost sense

    Linh's told me that she lost the sense on lips and chin a while ago after the accident. Occasionally, Linh unintentionally bit her lips whilst eating because she could not feel it. We contacted Dr. Hien Duc, she advised:
    This is normal. Some nerves were cut during the operation. But they will reconnect later.
    Thanks God! After about a year, her sense fully recovered. That's better. I didn't feel like kissing the living dead lips.

    Pregnancy 

    Last week, I was informed by Linh that I will be dad. I was so happy. The sort feeling that could not be expressed. We lost a pregnancy once. I fear that it could happen again.

    I didn't want Linh to get pregnant for years. The reason was we both not ready yet. I was a youth social worker in Australia whilst doing University degree. I think I knew what children really need. This is what ready really mean from my point of view:
    • Must be able to spend at least 30 minutes a day with my child for at least the first 7 years. This is the most important period of the development of the brain of children.
    • Take a walk with my wife 30 minutes daily after dinner during the pregnancy period.
    • Must do more research about the psychology of kids. 
    We both trying to be ready now. I already quited my current job in Vietnam. I'll get another as soon as I go back to Australia. The environment in Australia is much better than Vietnam. It should be number one in the world. I made a commitment to myself that I will not work as hard as I did during the past years. My family is much more important. I would do the best I could for my wife and my child.

    Our story is endless. Yet, I have a feeling it has just begun.

    Preparing to go to Australia

    In order to provide our kid the best possible life, we have to go to Australia. The following picture story will illustrate what we have done so far: 

    Requirement Elicitations
    Cong is doing the statutory declaration.
    Linh writing our story history from her aspect.   
    I went to  my friend office in the Suh Wah Tower to get him to do the
    statutory declaration. Giang is the Director of Vina Security.
    He got his own lawyer which can be the prescribed person for
    the statutory declaration. Very convenience for us.
    Cong help us finding the old images on his laptop.
    OMG, there are over 5,500 of them in an unordered list.
    I have to preview every single one of them.
    Linh continue writing her story. She is very slow.
    Well, she is pregnant. I don't want to put to much pressure on her.
    I was playing with the remote control of the camera.
    Finally, I got it working.
    Now, I can take pictures of us without the help from other people.
    Getting the document verified, certified in Australia is easy. In contrast, it's the pain in the ass to get any document certified in Vietnam. Here is one of our experience. Criminal record for Linh is required by the department of immigration of Australia, we went to the local police station where Linh lived many years ago. We must do the paperwork there even if it's thousands of miles away!  A cop informed us:
    The person in charged is on vacation. He will be back next week.
    I wondered if I am in the 21st century. We asked for the form, they didn't even have it. There is no such form. We went back home to ask the local cop I know. He showed us how to write it. We got it done. we went back to the other police station to hand in the application. One of the cop questioned:
    Is this how the application must be done?
    Every form or application must goes through the cops. If cops doesn't know how to do it, what do they expect from people like us.

    For every form, we must go back and forth at least 3 times. This is how you do it in Vietnam:
    1. Buy the form from the local police station, if the have it. Otherwise, you must write your own in the correct way. 
    2. Complete the form. Hand it in. They will tell you when you can pick it up. They will not inform you if they have not complete it. You have to go there to find out!
    3. Pick up your form.
    Well, we got most if not all of the required documents done. We are ready to go now!

    Saturday, July 17, 2010

    InnoDB crash recovery after power failure

    On the Friday, 16th July 2010. There was a power outage at our data center in LA. After power is restored, MySQL service refused to start. I checked the error log and realized InnoDB has been crashed.
     
    1. The first thing I do is copy the raw database file as backup before doing anything:
      sudo cp -R /var/lib/mysql /var/lib/mysql-crash
       
    2. Edit the MySQL configuration file:
      sudo vi /etc/mysql/my.cnf
    3. Insert the following line in [mysqld] section to force InnoDB recovery mode:
      innodb_force_recovery = 1

      From MySQL manual:
    If you are able to dump your tables with an option value of at most 4, then you are relatively safe that only some data on corrupt individual pages is lost.
    I tried from mode 1 to 4, but MySQL service refused to start. That's mean I have to accept some losses. I tried mode 5 (SRV_FORCE_NO_UNDO_LOG_SCAN), still no luck. This make me really worry as mode 6 is the last option. If MySQL service doesn't start my last option would be restored yesterday's backup.

    Thanks God! Mode 6 (SRV_FORCE_NO_LOG_REDO) works! MySQL service started! Nonetheless, I couldn't query any InnoDb table. Every time  I do, I got this error:
    Lost connection to MySQL server during query (XXXX)
    When I tried to use mysqldump, I got the following error:
    Couldn't execute 'show table status like '__session'': Lost connection to MySQL server during query (XXXX)
    I tried to ignore the __session table, but the next table came up with the same problem and so on.

    It took me hours to look for solutions on the Internet. Again, the best solution was restore from backup. Another solutions is restore InnoDB from only an .ibd file. Solution from www.chriscalender.com would work well if I enable innodb_file_per_table option. This allowed each InnoDB table to have its own tablespace like MyISAM storage engine. There would be an .ibd file created for every InnoDB table. But it's too late now. I can still extract the data from the .ibd raw file, but it would cost me a lot of time and effort.

    I almost gave up!

    Until now, I still don't know what made me change the recovery mode to 1 and restart MySQL service. Eureka, MySQL server restarted without any error.

    Now, I backup all databases using the following command:
    mysqldump --routines --triggers --single-transaction --quick -uroot -p --all-databases > all-dbs-100716.sql

    In order to restore, I shutdown MySQL service and renamed the crashed database datadir. I installed the new database structure and restored everything back to just before the crashed using the following commands:
    sudo mysql_install_db
    mysql -uroot mysql < all-dbs-100716.sql

    Thanks God! After the investigation, the corrupted table was __session. Our PHP session data are stored in the database to enable session between sub-domain. Some data from this table is lost, but it doesn't matter at all.

    Here are some lessons learned:
    1. Enable innodb_file_per_table. In the worst scenario, we can restore InnoDB from only an .ibd file.
    2. It's good that we backup our database daily. Here is a simple command which can be executed from cron job:
      mysqldump --routines --triggers --single-transaction --quick -uroot -pXXX database-name | gzip -9 > database-name-$(date +%F).sql.gz
    3. Change the InnoDB recovery mode from 1 to 6. If you cannot dump the databases, change back to mode 1. It might work! At least in our case. We already replicate the test with another server with the same crashed data.
    4. When doing the recovery, change MySQL port to any other port instead of 3306. This make sure that all other services cannot access MySQL. No DELETE/UPDATE/INSERT statements can be performed in recovery mode. But it doesn't hurt to take extra precautions.
    5. innodb_log_file_size is recommended to be about 25% of system RAM. Personally, I think this only apply to smaller/older server with less than 4GB RAM. In my case, I have innodb_log_file_size = 1GB even though my system have 16GB RAM. I don't think log file bigger than 1GB will do any benefit. However, setting the right figure would greatly reduce the insert and recovery time by at least 10 folds. Our database is about 2GB with more than 10 millions records. The default innodb_log_file_size is 5MB, which would takes forever to restore. I will do a test on this for the exact figures.
    I am glad that I went through all these troubles. My old man told me over and over again that no pain, no gain!

    Tuesday, July 13, 2010

    Installing Zend Debugger for Ubuntu

    I am still seeing so many people using var_dump(). I think most of us spend more than 50% of our time debugging. There is no excuse for investing some time for a better debugging tool. This guide will show you how to configure Zend Debugger for Zend Studio under Ubuntu.

    Requirements:
    • Zend Studio 7.x.x
    • Firefox with Zend Studio toolbar
    Assumptions:
    • You are using Linux Ubuntu distribution. Other Linux distribution should be similar.
    • You must already have Zend Studio installed.
    You need the ZendDebugger.so extension loaded in order to enable Zend Debugger. This PHP extension comes with Zend Studio.

    Finding ZendDebugger.so
    If Zend Studio is installed, you can locate ZendDebugger.so as follow:
    1. sudo updatedb
      Update the database of all the files and directories in our system. This command should be run by the cron job everyday. You may omit this step unless Zend Studio is newly installed.
    2. locate ZendDebugger.so
      Find the exact path to ZendDebugger.so
    Expected results:
    /home/user/Zend/ZendStudio-7.2.1/plugins/org.zend.php.debug.debugger.linux.x86_5.3.7.v20100625/resources/php5/ZendDebugger.so
    /home/user/Zend/ZendStudio-7.2.1/plugins/org.zend.php.debug.debugger.linux.x86_5.3.7.v20100625/resources/php53/ZendDebugger.so
    Load ZendDebugger.so extension
    I do not recommend editing the php.ini file as we can easily create a mess in there. Instead, we should create a new configuration file under /etc/php5/conf.d directory. All configuration files end with ini under that directory will be loaded every time PHP execution is called. We can easily maintain our configuration that way:
    sudo cat > /etc/php5/conf.d/zend_debugger.ini
    Paste from command line (ctrl+shift+v):
    zend_extension=/full/path/to/ZendDebugger.so
    zend_debugger.allow_hosts=127.0.0.1, 192.168.0.1/16
    zend_debugger.expose_remotely=always
    done with ctrl+c.
    zend_debugger.allow_hosts - which hosts allowed to debug, separated by comma.
    zend_debugger.expose_remotely - telling the web browser that Zend Debugger is enable on this server.

    If you are using PHP 5.3, you must use .../php53/ZendDebugger.so. Otherwise, use .../php5/ZendDebugger.so.


    Restart apache server:
    sudo service apaches restart
    Hopefully, there is no error. Otherwise, check the path to your ZendDebugger.so or the version of ZendDebugger.so you are loading. Most of the problem occur there.

    Confirm Zend Debugger is installed
    Create a test page as follow:
    cat > test.php
    Paste from command line (ctrl+shift+v):
    <?php
    echo phpinfo();
    done with ctrl+c.
    If you see the following image, your Zend Debugger is successfully configured on your server. In my case, it's just my personal laptop.


    Let's debug...
    Open Firefox with Zend Studio toolbar, go to your website click on the Debug button as show below:



    Enjoy! But please remember, It is twice as difficult to debug a program as to write it. Therefore, if you put all of your creativity and effort into writing the program, you are not smart enough to debug it.
    Anonymous
    Enhanced by Zemanta

    Saturday, July 10, 2010

    Vietnam - 4. Simplifying

    How do I simplify the following problems?
    Super long functions
    Make staffs understand that ONE function should do ONE and only ONE specific task! Normally, the a function should contain no more than 20 LOC for our PHP web projects, excluding comments and line separators. Unless, it's a controller function.

    Big fat controllers
    To simplify the project, we should keep the controller skin, model fat!
    Why? All the logics are in the controller, we shouldn't make it more complicated than it should be!

    Logics found in the views
    All the logics should be in the controller. The purpose of the view is displaying the data. The decoration should be in the CSS. Sound simple, but making all staffs understand this concept is not as simple as it should be. I need to give them concrete examples.

    Zend_Form not being used
    Most of our staffs do not understand Zend_Form. They think it's too complicated. I have to show them a concrete example of Zend_Form. The code from controller has been reduce from about 100 LOC to about 20 LOC.
    Here are some features from Zend_Form:
    • Has a lot of built in validators. Can validate number, string, phone number, credit card...
    • Separate the code from HTML
    • Can be decorated with Zend_Decorator
    • Reusable code
    • Easier to debug
    Zend_Form is quite complex, example and documentation can be found here:
    http://akrabat.com/php/simple-zend_form-example/
    http://framework.zend.com/manual/en/zend.form.html

    Variable name has no meaning
    This is very hard to simplify! One simple rule is no abbreviation.
    As most of the data type we are dealing with are string, number, array, Zend_Db_Table_Row, Zend_Db_Table_Rowset, therefore, I defined the following rules for our developers:
    • Zend_Db_Table_Row - variable must end with Row. E.g.: $userRow
    • Zend_Db_Table_Rowset - variable must end with Rowset. E.g.: $userRowset
    • Array - we use plural to indicate array or self-describing name. E.g.: $userRows, $usernames, $userProfile
    • String - normal variable name. E.g.: $username, $firstName,...
    • Number - normal variable name. E.g.: $userAge, $earning,...
    The most important concept I've taught our staffs about naming convention are:
    • Variable/Class names should be noun unless you have a good reason for it not to be noun.
    • Function names must be verb.
    • Always use self-describing names.
    • Never use shortcut.
    English is a second language for our developers, therefore, we have to limit the vocabularies being used.
    • All functions return a boolean must start with is, has, can.
    • Function start with set accepts only ONE input parameter. E.g.: setEmail($email), setAge($age), setPassword($password)...
    • Function start with update accepts an array as input parameter. E.g.: updateUserProfile($userProfile), updatePostalAddress($postalAddress), updateCreditCardInfos($creditCardInfos)...
    • We have a lot of problems with naming the action controller. Here are some examples of inconsistency: getUsersAction, listUsersAction, showUsersAction,... I forced all developers to name it show. In fact, any action name from controller that involve displaying data must be started with show. Similar for edit/update/change & delete/remove/del & add/insert/insert-new/new...
    Magic numbers
    Magic numbers can be eliminated by moving to a constant file.

    Global variables
    Global variables can be used but it need to be well explained & self-describing. However, we should try to minimize the use of global variables.

    It cost me a lot of time to put our staffs on track. Here are some fundamental rules for my staffs:
    • All functions must have comments and @author.
    • Global variables must have comments.
    • Write short simple functions.
    • No magic numbers.
    • Variable and function names must be self-described.
    • No abbreviation.
    • If you think that section of code you've wrote is complex, then it is! Find a way to simplify it.
    Everything should be made as simple as possible, but not simpler. Albert Einstein.

      Wednesday, July 7, 2010

      Vietnam - 3. Welcome to the real world

      Please check my previous post if you haven't read it Standardizing.

      After a few lectures about standardizing, simplifying, planning... Our staffs start working on the new project. I am quite busy designing other system whilst taking care of the servers. I let the staff work alone for one whole week!

      Week 1:
      Objective: Get the job done using the standards.

      Results:
      I expect the front end of the site to be completed. Yes, it is! I tested the site and found a few bugs. I dug into the code to fix it. Omg, I cannot believe that I could not understand what's going on! Here are some of the problems found:
      1. Big fat controllers
      2. Logic found in views
      3. Models not being used much
      4. Zend_Form is not used at all
      5. Only a handful number of functions is found
      6. Variable name has no meaning
      7. LOOP in LOOP in LOOP
      8. Hardly see any comment
      9. Global variables are widely used
      10. Magic numbers everywhere
      I decided to inspect the CSS code, I found the following problems:
      1. Inline style everywhere
      2. ID selectors are widely used instead of class selectors
      3. Many ID selectors having the same name
      4. Normalization not being used
      5. Short hand not being use much
      6. Magic numbers everywhere, they are not even nice number. Imo, nice number are number that easy to remember and can divide many other numbers. Odd number should be avoided. 
      The biggest problem is none of the standards specified is followed.

      Well, one thing I cannot deny is they work hard and willing to learn. Remember, professionals built the Titanic, amateurs built Noah's Ark.

      Monday, July 5, 2010

      Vietnam - 2. Standardizing

      Existing Office
      There isn't any problem with setting up the office Vietnam as we already have an existing office with about 20 CSR. The CSR are working around the clock. We moved in, share the existing facilities.

      SVN & Linux
      Initially, I setup the subversion and local development environment. As a Linux geek, I forced everyone to use Linux, Ubuntu distribution. I don't see the need for Windows at all. Here are some reasons I think Linux is better. At least for our PHP developers.
      • Free, easy to install
      • No drivers needed
      • Haven't got any complaint from our staffs about virus
      • Represent our server environment
      • Staffs cannot do much beside working
      • The list is endless...
      There are many cases you need Windows. Please do not argue here as I don't want to end up fighting the never ending war between Windows and Linux.

      Coding Standard
      I divided the coding standard to 3 sections:
      1. PHP - Most of our web projects are based on the Zend Framework. Hence, we use the Zend Framework coding standard.
      2. Database - Coding standard for SQL is a little bit tricky. I couldn't find any standard that suit our needs. I have to invent my own standard. Here are some guidelines:


        • Database name: lowercase, usually project name
        • Table name: plural, uppercase first letter, then CamelCase. Only the last word is plural. E.g.: UserProfiles
        • Reference table name: we use the word Maps to indicate. E.g.: UserGroupMaps
        • Field name: singular, uppercase first letter, then CamelCase. E.g.: FirstName, UserId
        • Primary key: should be the singular table name follow by Id. E.g.: UserId
        • Avoid using `-` and spaces
        • All table, field, functions, triggers, stored procedure names should be as meaningful as possible.
        • Never use abbreviation. Some people understand others don't, so avoid using it.
        • SQL keywords must be CAPITALIZED
      3. CSS - All our web projects are using jQuery extensively and we are planning to keep it that way. jQuery/UI using hyphen `-` as word separator for CSS, we will stick with it. The key is making the class name as meaningful as possible.
      The key here is to make everyone using the same simple standard. Talking the same language. Thinking of the same concept.

      I long know simplicity is the key! Can I simplify it?